mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-09 16:35:27 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Managed worktree services run isolated Paperclip instances with cloned databases. > - A reachable service was reported as ready even when its database, runtime identity, or login path was not usable. > - The first candidate added verified database seeding and managed repair in #11665. > - This pull request consolidates that candidate with signed login handoff and a complete readiness contract. > - Post-QA fixes close five defects in repair identity, repair responses, UI retry, seed journal handling, and seed-source trust. > - The benefit is a workspace that either opens safely or reports one accurate recovery action. ## Linked Issues or Issue Description No public GitHub issue exists for this work, so the problem is described here. **What happened** Managed workspace URLs could return HTTP 200 and report ready while login failed. QA also found cases where repair used the wrong instance identity, returned a generic error, left the UI stuck, rejected a safe journal lag, or trusted a mutable workspace manifest. **Expected behavior** Opening a ready workspace signs the board user in to the correct isolated instance. Provisioning and repair use a registered source and report a structured recovery state. **Actual behavior** Entry depended on a password copied into the clone. Several failure paths could publish stale readiness, hide the repair precondition, or trust state that the workspace could modify. **Additional context** This pull request includes the commits first published in #11665. That pull request keeps the original base head for review history. This consolidated pull request is the merge candidate. Related open readiness work includes #11575 and #11621. ## What Changed - Adds a short-lived, signed, single-use login ticket. It binds the user, workspace, instance, and runtime origin. - Exchanges the ticket through Better Auth. It creates the session and cookie through the supported adapter path. - Adds protected workspace readiness fields for the database, clone data, login handoff, seed phase, and runtime identity. - Fails readiness closed when the guest has no company or execution-workspace binding. - Binds ticket issuance to the exact cloned user and active company membership selected for the handoff. - Verifies every current active board identity through the exact-user handoff before publication or reuse. - Gates managed runtime publication on the readiness contract and the recorded worktree instance identity. - Refreshes runtime work products from the live runtime row after a port change. - Adds one workspace access card with ready, degraded, repairing, and failed states. - Uses the runtime response identity for repair. It returns structured repair precondition errors. - Lets a valid source journal lag converge during provisioning. - Binds seed and repair manifests to a source registered outside the agent-writable worktree. - Clears recovered UI errors so a successful retry can open the workspace. - Makes runtime tests register canonical sources and avoid ports owned by live host listeners. - Keeps Vitest on source suites when compiled `dist` trees exist. - Isolates CLI and adapter tests from ambient AWS and runtime API environment variables. - Preserves a 404 response for cross-company workspace ID lookups before runtime authorization. - Makes concurrent single-flight coverage independent of path-canonicalization scheduling order. ## Verification The following checks passed on the integrated head: ```sh pnpm -r typecheck pnpm build pnpm check:token-gates pnpm --filter @paperclipai/db check:migrations ``` - The server source lane passed 420 files and 4,953 tests. Five tests were skipped. - The CLI lane passed 57 files and 385 tests. - The database lane passed 26 files and 97 tests. - The shared package passed 58 files and 506 tests. - The adapter utility lane passed 640 tests. Four tests were skipped. - The Claude adapter passed 220 tests. One test was skipped. - The Codex adapter passed 323 tests. - The OpenClaw adapter passed 13 tests. - The OpenCode adapter passed 42 tests. - The plugin SDK passed 45 tests. - The workspace runtime suite passed 124 tests. - The caller-scoped readiness and handoff suite passed 52 tests. - The workspace provisioning shell suite passed 7 tests. - The runtime exposure suite passed 17 tests while live host mappings occupied fixed test ports. - `git diff --check` passed and the worktree is clean. The serialized route lane will run in GitHub CI with its normal shards. No deployment or active-workspace migration was performed. ## Risks - This is a medium-risk authentication and runtime-readiness change. - The login ticket uses exact origin, workspace, instance, and user binding. It has a short expiry and a one-time nonce. - Runtime publication is stricter. A real readiness, identity, per-user handoff, or control-plane database disagreement now blocks publication. - This pull request supersedes #11665 as the merge candidate. Close #11665 after this pull request merges. - No new database migration is included. The lockfile and workflow files are unchanged. - Deployment and active-workspace migration are intentionally outside this pull request. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used Claude Opus 5 (`claude-opus-5[1m]`), 1M context, extended thinking, tool use, and code execution produced the main candidate. OpenAI GPT-5 (`gpt-5`) through Codex, with agentic reasoning, tool use, and code execution, integrated the post-QA fixes and hardened the test gates. The Codex context-window size was not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
337 lines
9.7 KiB
TypeScript
337 lines
9.7 KiB
TypeScript
import { randomInt } from "node:crypto";
|
|
import path from "node:path";
|
|
import type { PaperclipConfig } from "../config/schema.js";
|
|
import { expandHomePrefix } from "../config/home.js";
|
|
|
|
export const DEFAULT_WORKTREE_HOME = "~/.paperclip-worktrees";
|
|
export const WORKTREE_SEED_MODES = ["minimal", "full"] as const;
|
|
export const WORKTREE_SEED_MANIFEST = "seed-manifest.json";
|
|
export const WORKTREE_SEED_PENDING_MARKER = "seed-pending";
|
|
export const WORKTREE_SEED_COMPLETE_MARKER = "seed-complete";
|
|
export const WORKTREE_SEED_LOCK_MARKER = "seed.lock";
|
|
|
|
export type WorktreeSeedMode = (typeof WORKTREE_SEED_MODES)[number];
|
|
|
|
export const WORKTREE_SEED_PHASES = [
|
|
"pending",
|
|
"source_validation",
|
|
"snapshot",
|
|
"restore",
|
|
"migrations",
|
|
"execution_quarantine",
|
|
"routine_pause",
|
|
"workspace_rebind",
|
|
"post_restore_validation",
|
|
"complete",
|
|
] as const;
|
|
|
|
export type WorktreeSeedPhase = (typeof WORKTREE_SEED_PHASES)[number];
|
|
export type WorktreeSeedState = "pending" | "running" | "verified" | "failed";
|
|
|
|
export type WorktreeSeedManifest = {
|
|
version: 2;
|
|
source: {
|
|
instanceId: string;
|
|
configPath: string;
|
|
};
|
|
snapshotAt: string | null;
|
|
seedMode: WorktreeSeedMode;
|
|
migrationRevision: string | null;
|
|
targetInstanceId: string;
|
|
phase: WorktreeSeedPhase;
|
|
state: WorktreeSeedState;
|
|
attemptId: string;
|
|
startedAt: string | null;
|
|
finishedAt: string | null;
|
|
diagnostics: Array<{
|
|
phase: WorktreeSeedPhase;
|
|
status: "started" | "succeeded" | "failed";
|
|
at: string;
|
|
message?: string;
|
|
}>;
|
|
};
|
|
|
|
export type WorktreeSeedPlan = {
|
|
mode: WorktreeSeedMode;
|
|
excludedTables: string[];
|
|
nullifyColumns: Record<string, string[]>;
|
|
};
|
|
|
|
const MINIMAL_WORKTREE_EXCLUDED_TABLES = [
|
|
"activity_log",
|
|
"agent_runtime_state",
|
|
"agent_task_sessions",
|
|
"agent_wakeup_requests",
|
|
"cost_events",
|
|
"heartbeat_run_events",
|
|
"heartbeat_runs",
|
|
"workspace_runtime_services",
|
|
];
|
|
|
|
const MINIMAL_WORKTREE_NULLIFIED_COLUMNS: Record<string, string[]> = {
|
|
issues: ["checkout_run_id", "execution_run_id"],
|
|
};
|
|
|
|
export type WorktreeLocalPaths = {
|
|
cwd: string;
|
|
repoConfigDir: string;
|
|
configPath: string;
|
|
envPath: string;
|
|
homeDir: string;
|
|
instanceId: string;
|
|
instanceRoot: string;
|
|
contextPath: string;
|
|
embeddedPostgresDataDir: string;
|
|
backupDir: string;
|
|
logDir: string;
|
|
secretsKeyFilePath: string;
|
|
storageDir: string;
|
|
};
|
|
|
|
export type WorktreeUiBranding = {
|
|
name: string;
|
|
color: string;
|
|
};
|
|
|
|
export type WorktreeSeedMarkerPaths = {
|
|
manifest: string;
|
|
pending: string;
|
|
complete: string;
|
|
lock: string;
|
|
};
|
|
|
|
export function resolveWorktreeSeedMarkerPaths(configPath: string): WorktreeSeedMarkerPaths {
|
|
const configDir = path.dirname(path.resolve(configPath));
|
|
return {
|
|
manifest: path.resolve(configDir, WORKTREE_SEED_MANIFEST),
|
|
pending: path.resolve(configDir, WORKTREE_SEED_PENDING_MARKER),
|
|
complete: path.resolve(configDir, WORKTREE_SEED_COMPLETE_MARKER),
|
|
lock: path.resolve(configDir, WORKTREE_SEED_LOCK_MARKER),
|
|
};
|
|
}
|
|
|
|
export function isWorktreeSeedMode(value: string): value is WorktreeSeedMode {
|
|
return (WORKTREE_SEED_MODES as readonly string[]).includes(value);
|
|
}
|
|
|
|
export function resolveWorktreeSeedPlan(mode: WorktreeSeedMode): WorktreeSeedPlan {
|
|
if (mode === "full") {
|
|
return {
|
|
mode,
|
|
excludedTables: [],
|
|
nullifyColumns: {},
|
|
};
|
|
}
|
|
return {
|
|
mode,
|
|
excludedTables: [...MINIMAL_WORKTREE_EXCLUDED_TABLES],
|
|
nullifyColumns: {
|
|
...MINIMAL_WORKTREE_NULLIFIED_COLUMNS,
|
|
},
|
|
};
|
|
}
|
|
|
|
function nonEmpty(value: string | null | undefined): string | null {
|
|
return typeof value === "string" && value.trim().length > 0 ? value.trim() : null;
|
|
}
|
|
|
|
export function sanitizeWorktreeInstanceId(rawValue: string): string {
|
|
const trimmed = rawValue.trim().toLowerCase();
|
|
const normalized = trimmed
|
|
.replace(/[^a-z0-9_-]+/g, "-")
|
|
.replace(/-+/g, "-")
|
|
.replace(/^[-_]+|[-_]+$/g, "");
|
|
return normalized || "worktree";
|
|
}
|
|
|
|
export function resolveSuggestedWorktreeName(cwd: string, explicitName?: string): string {
|
|
return nonEmpty(explicitName) ?? path.basename(path.resolve(cwd));
|
|
}
|
|
|
|
function hslComponentToHex(n: number): string {
|
|
return Math.round(Math.max(0, Math.min(255, n)))
|
|
.toString(16)
|
|
.padStart(2, "0");
|
|
}
|
|
|
|
function hslToHex(hue: number, saturation: number, lightness: number): string {
|
|
const s = Math.max(0, Math.min(100, saturation)) / 100;
|
|
const l = Math.max(0, Math.min(100, lightness)) / 100;
|
|
const c = (1 - Math.abs((2 * l) - 1)) * s;
|
|
const h = ((hue % 360) + 360) % 360;
|
|
const x = c * (1 - Math.abs(((h / 60) % 2) - 1));
|
|
const m = l - (c / 2);
|
|
|
|
let r = 0;
|
|
let g = 0;
|
|
let b = 0;
|
|
|
|
if (h < 60) {
|
|
r = c;
|
|
g = x;
|
|
} else if (h < 120) {
|
|
r = x;
|
|
g = c;
|
|
} else if (h < 180) {
|
|
g = c;
|
|
b = x;
|
|
} else if (h < 240) {
|
|
g = x;
|
|
b = c;
|
|
} else if (h < 300) {
|
|
r = x;
|
|
b = c;
|
|
} else {
|
|
r = c;
|
|
b = x;
|
|
}
|
|
|
|
return `#${hslComponentToHex((r + m) * 255)}${hslComponentToHex((g + m) * 255)}${hslComponentToHex((b + m) * 255)}`;
|
|
}
|
|
|
|
export function generateWorktreeColor(): string {
|
|
return hslToHex(randomInt(0, 360), 68, 56);
|
|
}
|
|
|
|
export function resolveWorktreeLocalPaths(opts: {
|
|
cwd: string;
|
|
homeDir?: string;
|
|
instanceId: string;
|
|
}): WorktreeLocalPaths {
|
|
const cwd = path.resolve(opts.cwd);
|
|
const homeDir = path.resolve(expandHomePrefix(opts.homeDir ?? DEFAULT_WORKTREE_HOME));
|
|
const instanceRoot = path.resolve(homeDir, "instances", opts.instanceId);
|
|
const repoConfigDir = path.resolve(cwd, ".paperclip");
|
|
return {
|
|
cwd,
|
|
repoConfigDir,
|
|
configPath: path.resolve(repoConfigDir, "config.json"),
|
|
envPath: path.resolve(repoConfigDir, ".env"),
|
|
homeDir,
|
|
instanceId: opts.instanceId,
|
|
instanceRoot,
|
|
contextPath: path.resolve(homeDir, "context.json"),
|
|
embeddedPostgresDataDir: path.resolve(instanceRoot, "db"),
|
|
backupDir: path.resolve(instanceRoot, "data", "backups"),
|
|
logDir: path.resolve(instanceRoot, "logs"),
|
|
secretsKeyFilePath: path.resolve(instanceRoot, "secrets", "master.key"),
|
|
storageDir: path.resolve(instanceRoot, "data", "storage"),
|
|
};
|
|
}
|
|
|
|
export function rewriteLocalUrlPort(rawUrl: string | undefined, port: number): string | undefined {
|
|
if (!rawUrl) return undefined;
|
|
try {
|
|
const parsed = new URL(rawUrl);
|
|
// The URL API normalizes default ports like :80/:443 to "", so treat them as stable URLs.
|
|
if (!parsed.port) return rawUrl;
|
|
parsed.port = String(port);
|
|
return parsed.toString();
|
|
} catch {
|
|
return rawUrl;
|
|
}
|
|
}
|
|
|
|
export function buildWorktreeConfig(input: {
|
|
sourceConfig: PaperclipConfig | null;
|
|
paths: WorktreeLocalPaths;
|
|
serverPort: number;
|
|
databasePort: number;
|
|
now?: Date;
|
|
}): PaperclipConfig {
|
|
const { sourceConfig, paths, serverPort, databasePort } = input;
|
|
const nowIso = (input.now ?? new Date()).toISOString();
|
|
|
|
const source = sourceConfig;
|
|
const authPublicBaseUrl = rewriteLocalUrlPort(source?.auth.publicBaseUrl, serverPort);
|
|
|
|
return {
|
|
$meta: {
|
|
version: 1,
|
|
updatedAt: nowIso,
|
|
source: "configure",
|
|
},
|
|
...(source?.llm ? { llm: source.llm } : {}),
|
|
database: {
|
|
mode: "embedded-postgres",
|
|
embeddedPostgresDataDir: paths.embeddedPostgresDataDir,
|
|
embeddedPostgresPort: databasePort,
|
|
backup: {
|
|
enabled: false,
|
|
intervalMinutes: source?.database.backup.intervalMinutes ?? 60,
|
|
retentionDays: source?.database.backup.retentionDays ?? 30,
|
|
dir: paths.backupDir,
|
|
},
|
|
},
|
|
logging: {
|
|
mode: source?.logging.mode ?? "file",
|
|
logDir: paths.logDir,
|
|
},
|
|
server: {
|
|
deploymentMode: source?.server.deploymentMode ?? "local_trusted",
|
|
exposure: source?.server.exposure ?? "private",
|
|
...(source?.server.bind ? { bind: source.server.bind } : {}),
|
|
...(source?.server.customBindHost ? { customBindHost: source.server.customBindHost } : {}),
|
|
host: source?.server.host ?? "127.0.0.1",
|
|
port: serverPort,
|
|
allowedHostnames: source?.server.allowedHostnames ?? [],
|
|
serveUi: source?.server.serveUi ?? true,
|
|
},
|
|
auth: {
|
|
baseUrlMode: source?.auth.baseUrlMode ?? "auto",
|
|
...(authPublicBaseUrl ? { publicBaseUrl: authPublicBaseUrl } : {}),
|
|
disableSignUp: source?.auth.disableSignUp ?? false,
|
|
},
|
|
telemetry: {
|
|
enabled: source?.telemetry?.enabled ?? true,
|
|
},
|
|
storage: {
|
|
provider: source?.storage.provider ?? "local_disk",
|
|
localDisk: {
|
|
baseDir: paths.storageDir,
|
|
},
|
|
s3: {
|
|
bucket: source?.storage.s3.bucket ?? "paperclip",
|
|
region: source?.storage.s3.region ?? "us-east-1",
|
|
endpoint: source?.storage.s3.endpoint,
|
|
prefix: source?.storage.s3.prefix ?? "",
|
|
forcePathStyle: source?.storage.s3.forcePathStyle ?? false,
|
|
},
|
|
},
|
|
secrets: {
|
|
provider: source?.secrets.provider ?? "local_encrypted",
|
|
strictMode: source?.secrets.strictMode ?? false,
|
|
localEncrypted: {
|
|
keyFilePath: paths.secretsKeyFilePath,
|
|
},
|
|
},
|
|
};
|
|
}
|
|
|
|
export function buildWorktreeEnvEntries(
|
|
paths: WorktreeLocalPaths,
|
|
branding?: WorktreeUiBranding,
|
|
): Record<string, string> {
|
|
return {
|
|
PAPERCLIP_HOME: paths.homeDir,
|
|
PAPERCLIP_INSTANCE_ID: paths.instanceId,
|
|
PAPERCLIP_CONFIG: paths.configPath,
|
|
PAPERCLIP_CONTEXT: paths.contextPath,
|
|
PAPERCLIP_IN_WORKTREE: "true",
|
|
PAPERCLIP_DB_BACKUP_ENABLED: "false",
|
|
...(branding?.name ? { PAPERCLIP_WORKTREE_NAME: branding.name } : {}),
|
|
...(branding?.color ? { PAPERCLIP_WORKTREE_COLOR: branding.color } : {}),
|
|
};
|
|
}
|
|
|
|
function shellEscape(value: string): string {
|
|
return `'${value.replaceAll("'", `'\"'\"'`)}'`;
|
|
}
|
|
|
|
export function formatShellExports(entries: Record<string, string>): string {
|
|
return Object.entries(entries)
|
|
.filter(([, value]) => typeof value === "string" && value.trim().length > 0)
|
|
.map(([key, value]) => `export ${key}=${shellEscape(value)}`)
|
|
.join("\n");
|
|
}
|