feat(workspaces): sign the workspace login handoff and gate readiness (#11671)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Managed worktree services run isolated Paperclip instances with
cloned databases.
> - A reachable service was reported as ready even when its database,
runtime identity, or login path was not usable.
> - The first candidate added verified database seeding and managed
repair in #11665.
> - This pull request consolidates that candidate with signed login
handoff and a complete readiness contract.
> - Post-QA fixes close five defects in repair identity, repair
responses, UI retry, seed journal handling, and seed-source trust.
> - The benefit is a workspace that either opens safely or reports one
accurate recovery action.

## Linked Issues or Issue Description

No public GitHub issue exists for this work, so the problem is described
here.

**What happened**

Managed workspace URLs could return HTTP 200 and report ready while
login failed. QA also found cases where repair used the wrong instance
identity, returned a generic error, left the UI stuck, rejected a safe
journal lag, or trusted a mutable workspace manifest.

**Expected behavior**

Opening a ready workspace signs the board user in to the correct
isolated instance. Provisioning and repair use a registered source and
report a structured recovery state.

**Actual behavior**

Entry depended on a password copied into the clone. Several failure
paths could publish stale readiness, hide the repair precondition, or
trust state that the workspace could modify.

**Additional context**

This pull request includes the commits first published in #11665. That
pull request keeps the original base head for review history. This
consolidated pull request is the merge candidate. Related open readiness
work includes #11575 and #11621.

## What Changed

- Adds a short-lived, signed, single-use login ticket. It binds the
user, workspace, instance, and runtime origin.
- Exchanges the ticket through Better Auth. It creates the session and
cookie through the supported adapter path.
- Adds protected workspace readiness fields for the database, clone
data, login handoff, seed phase, and runtime identity.
- Fails readiness closed when the guest has no company or
execution-workspace binding.
- Binds ticket issuance to the exact cloned user and active company
membership selected for the handoff.
- Verifies every current active board identity through the exact-user
handoff before publication or reuse.
- Gates managed runtime publication on the readiness contract and the
recorded worktree instance identity.
- Refreshes runtime work products from the live runtime row after a port
change.
- Adds one workspace access card with ready, degraded, repairing, and
failed states.
- Uses the runtime response identity for repair. It returns structured
repair precondition errors.
- Lets a valid source journal lag converge during provisioning.
- Binds seed and repair manifests to a source registered outside the
agent-writable worktree.
- Clears recovered UI errors so a successful retry can open the
workspace.
- Makes runtime tests register canonical sources and avoid ports owned
by live host listeners.
- Keeps Vitest on source suites when compiled `dist` trees exist.
- Isolates CLI and adapter tests from ambient AWS and runtime API
environment variables.
- Preserves a 404 response for cross-company workspace ID lookups before
runtime authorization.
- Makes concurrent single-flight coverage independent of
path-canonicalization scheduling order.

## Verification

The following checks passed on the integrated head:

```sh
pnpm -r typecheck
pnpm build
pnpm check:token-gates
pnpm --filter @paperclipai/db check:migrations
```

- The server source lane passed 420 files and 4,953 tests. Five tests
were skipped.
- The CLI lane passed 57 files and 385 tests.
- The database lane passed 26 files and 97 tests.
- The shared package passed 58 files and 506 tests.
- The adapter utility lane passed 640 tests. Four tests were skipped.
- The Claude adapter passed 220 tests. One test was skipped.
- The Codex adapter passed 323 tests.
- The OpenClaw adapter passed 13 tests.
- The OpenCode adapter passed 42 tests.
- The plugin SDK passed 45 tests.
- The workspace runtime suite passed 124 tests.
- The caller-scoped readiness and handoff suite passed 52 tests.
- The workspace provisioning shell suite passed 7 tests.
- The runtime exposure suite passed 17 tests while live host mappings
occupied fixed test ports.
- `git diff --check` passed and the worktree is clean.

The serialized route lane will run in GitHub CI with its normal shards.
No deployment or active-workspace migration was performed.

## Risks

- This is a medium-risk authentication and runtime-readiness change.
- The login ticket uses exact origin, workspace, instance, and user
binding. It has a short expiry and a one-time nonce.
- Runtime publication is stricter. A real readiness, identity, per-user
handoff, or control-plane database disagreement now blocks publication.
- This pull request supersedes #11665 as the merge candidate. Close
#11665 after this pull request merges.
- No new database migration is included. The lockfile and workflow files
are unchanged.
- Deployment and active-workspace migration are intentionally outside
this pull request.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

Claude Opus 5 (`claude-opus-5[1m]`), 1M context, extended thinking, tool
use, and code execution produced the main candidate. OpenAI GPT-5
(`gpt-5`) through Codex, with agentic reasoning, tool use, and code
execution, integrated the post-QA fixes and hardened the test gates. The
Codex context-window size was not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-08-19 02:37:02 -05:00
1 parent 6aaef2998f
commit a2bf936f9a
59 files changed
+7754 -243

No files matched your search

+3
View File
@@ -139,6 +139,9 @@ describe("secrets CLI helpers", () => {
delete process.env.AWS_DEFAULT_REGION;
delete process.env.PAPERCLIP_SECRETS_AWS_DEPLOYMENT_ID;
delete process.env.PAPERCLIP_SECRETS_AWS_KMS_KEY_ID;
delete process.env.AWS_ACCESS_KEY_ID;
delete process.env.AWS_SECRET_ACCESS_KEY;
delete process.env.AWS_SESSION_TOKEN;
});
afterEach(() => {
+357 -45
View File
@@ -8,13 +8,17 @@ import { eq } from "drizzle-orm";
import { afterEach, describe, expect, it, vi } from "vitest";
import {
agents,
authAccounts,
authUsers,
companies,
companyMemberships,
createDb,
executionWorkspaces,
inspectMigrations,
issueComments,
issues,
projectWorkspaces,
instanceUserRoles,
projects,
routines,
routineTriggers,
@@ -27,6 +31,7 @@ import {
markWorktreeSeedPending,
pauseSeededScheduledRoutines,
quarantineSeededWorktreeExecutionState,
readWorktreeSeedManifest,
readSourceAttachmentBody,
rebindWorkspaceCwd,
resolveSourceConfigPath,
@@ -35,6 +40,7 @@ import {
resolveGitWorktreeAddArgs,
resolvePnpmInstallInvocation,
resolveCurrentWorktreeEndpoint,
resolveWorktreeSeedMigrationRevision,
resolveWorktreeSeedBackupEngine,
resolveWorktreeMakeTargetPath,
worktreeRepairCommand,
@@ -64,6 +70,88 @@ const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
const itEmbeddedPostgres = embeddedPostgresSupport.supported ? it : it.skip;
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
function mockVerifiedSeedResult() {
return {
backupSummary: "snapshot.sql",
snapshotAt: "2026-08-18T00:00:00.000Z",
migrationRevision: "0142_test.sql",
pausedScheduledRoutines: 0,
executionQuarantine: {
disabledTimerHeartbeats: 0,
resetRunningAgents: 0,
quarantinedInProgressIssues: 0,
unassignedTodoIssues: 0,
unassignedReviewIssues: 0,
stoppedProjectWorkspaceRuntimes: 0,
stoppedExecutionWorkspaceRuntimes: 0,
stoppedRuntimeServices: 0,
},
reboundWorkspaces: [],
validation: {
authUserCount: 1,
credentialAccountCount: 1,
instanceAdminCount: 1,
activeMembershipCount: 1,
companyCount: 1,
issueCount: 1,
representativeCompanyId: "00000000-0000-4000-8000-000000000001",
representativeIssueId: "00000000-0000-4000-8000-000000000002",
migrationRevision: "0142_test.sql",
},
};
}
async function seedValidWorktreeSource(connectionString: string) {
const db = createDb(connectionString);
const companyId = randomUUID();
const issueId = randomUUID();
const now = new Date();
await db.insert(authUsers).values({
id: "user-existing",
email: "existing@paperclip.ing",
name: "Existing User",
emailVerified: true,
createdAt: now,
updatedAt: now,
});
await db.insert(authAccounts).values({
id: "credential-existing",
accountId: "existing@paperclip.ing",
providerId: "credential",
userId: "user-existing",
password: "fixture-password-hash",
createdAt: now,
updatedAt: now,
});
await db.insert(instanceUserRoles).values({
userId: "user-existing",
role: "instance_admin",
});
await db.insert(companies).values({
id: companyId,
name: "Seed Source",
issuePrefix: "SEED",
requireBoardApprovalForNewAgents: false,
});
await db.insert(companyMemberships).values({
companyId,
principalType: "user",
principalId: "user-existing",
status: "active",
});
await db.insert(issues).values({
id: issueId,
companyId,
title: "Representative seed issue",
status: "backlog",
priority: "medium",
issueNumber: 1,
identifier: "SEED-1",
});
await db.$client.end({ timeout: 5 });
return { companyId, issueId };
}
if (!embeddedPostgresSupport.supported) {
console.warn(
`Skipping embedded Postgres worktree CLI tests on this host: ${embeddedPostgresSupport.reason ?? "unsupported environment"}`,
@@ -400,7 +488,34 @@ describe("worktree helpers", () => {
expect(full.nullifyColumns).toEqual({});
});
it("ensure-seeded seeds once and fast-exits on the seed-complete marker", async () => {
it("rejects a source migration journal that is ahead of the code journal", () => {
expect(() => resolveWorktreeSeedMigrationRevision({
status: "upToDate",
tableCount: 1,
availableMigrations: ["0001_initial.sql", "0002_current.sql"],
appliedMigrations: ["0001_initial.sql", "0002_current.sql"],
journalEntryCount: 3,
}, "sourcePrefix")).toThrow("Migration journal is ahead of this Paperclip checkout");
});
it("accepts a source migration journal that is multiple revisions behind", () => {
expect(resolveWorktreeSeedMigrationRevision({
status: "needsMigrations",
tableCount: 1,
availableMigrations: [
"0001_initial.sql",
"0002_applied.sql",
"0003_pending.sql",
"0004_pending.sql",
],
appliedMigrations: ["0001_initial.sql", "0002_applied.sql"],
pendingMigrations: ["0003_pending.sql", "0004_pending.sql"],
journalEntryCount: 2,
reason: "pending-migrations",
}, "sourcePrefix")).toBe("0002_applied.sql");
});
it("ensure-seeded seeds once and fast-exits on the verified manifest", async () => {
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-worktree-ensure-seeded-"));
try {
const sourceConfigPath = path.join(tempRoot, "source", "config.json");
@@ -421,6 +536,7 @@ describe("worktree helpers", () => {
fs.mkdirSync(path.dirname(sourceConfigPath), { recursive: true });
fs.mkdirSync(path.dirname(targetConfigPath), { recursive: true });
fs.writeFileSync(sourceConfigPath, `${JSON.stringify(sourceConfig)}\n`);
fs.writeFileSync(path.join(path.dirname(sourceConfigPath), ".env"), "PAPERCLIP_INSTANCE_ID=source\n");
fs.writeFileSync(targetConfigPath, `${JSON.stringify(targetConfig)}\n`);
fs.writeFileSync(
path.join(targetRoot, ".paperclip", ".env"),
@@ -429,7 +545,7 @@ describe("worktree helpers", () => {
markWorktreeSeedPending({ configPath: targetConfigPath, sourceConfigPath });
const seedDatabase = vi.fn().mockResolvedValue({
backupSummary: "snapshot.sql",
...mockVerifiedSeedResult(),
pausedScheduledRoutines: 2,
executionQuarantine: {
disabledTimerHeartbeats: 1,
@@ -441,15 +557,14 @@ describe("worktree helpers", () => {
stoppedExecutionWorkspaceRuntimes: 0,
stoppedRuntimeServices: 0,
},
reboundWorkspaces: [],
});
await expect(
ensureWorktreeSeeded({ config: targetConfigPath }, { seedDatabase }),
ensureWorktreeSeeded({ config: targetConfigPath, fromConfig: sourceConfigPath }, { seedDatabase }),
).resolves.toMatchObject({ seeded: true, reason: "seeded" });
await expect(
ensureWorktreeSeeded({ config: targetConfigPath }, { seedDatabase }),
).resolves.toEqual({ seeded: false, reason: "complete_marker" });
).resolves.toEqual({ seeded: false, reason: "verified_manifest" });
expect(seedDatabase).toHaveBeenCalledTimes(1);
expect(seedDatabase).toHaveBeenCalledWith(expect.objectContaining({
@@ -458,12 +573,127 @@ describe("worktree helpers", () => {
instanceId: "ensure-seeded-test",
}));
expect(fs.existsSync(path.join(targetRoot, ".paperclip", "seed-pending"))).toBe(false);
expect(fs.existsSync(path.join(targetRoot, ".paperclip", "seed-complete"))).toBe(true);
expect(fs.existsSync(path.join(targetRoot, ".paperclip", "seed-complete"))).toBe(false);
expect(readWorktreeSeedManifest(targetConfigPath)).toMatchObject({
version: 2,
state: "verified",
phase: "complete",
migrationRevision: "0142_test.sql",
targetInstanceId: "ensure-seeded-test",
});
} finally {
fs.rmSync(tempRoot, { recursive: true, force: true });
}
});
it("managed ensure-seeded derives a valid source from the registered base workspace", async () => {
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-worktree-managed-seed-"));
try {
const baseRoot = path.join(tempRoot, "base");
const sourceConfigPath = path.join(baseRoot, ".paperclip", "config.json");
const targetRoot = path.join(tempRoot, "worktree");
const targetConfigPath = path.join(targetRoot, ".paperclip", "config.json");
const targetPaths = resolveWorktreeLocalPaths({
cwd: targetRoot,
homeDir: path.join(tempRoot, "worktree-home"),
instanceId: "managed-target",
});
const sourceConfig = buildSourceConfig();
const targetConfig = buildWorktreeConfig({
sourceConfig,
paths: targetPaths,
serverPort: 3195,
databasePort: 54995,
});
fs.mkdirSync(path.dirname(sourceConfigPath), { recursive: true });
fs.mkdirSync(path.dirname(targetConfigPath), { recursive: true });
fs.writeFileSync(sourceConfigPath, `${JSON.stringify(sourceConfig)}\n`);
fs.writeFileSync(path.join(path.dirname(sourceConfigPath), ".env"), "PAPERCLIP_INSTANCE_ID=managed-source\n");
fs.writeFileSync(targetConfigPath, `${JSON.stringify(targetConfig)}\n`);
fs.writeFileSync(
path.join(path.dirname(targetConfigPath), ".env"),
`PAPERCLIP_HOME=${targetPaths.homeDir}\nPAPERCLIP_INSTANCE_ID=managed-target\n`,
);
markWorktreeSeedPending({ configPath: targetConfigPath, sourceConfigPath });
const seedDatabase = vi.fn().mockResolvedValue(mockVerifiedSeedResult());
await expect(ensureWorktreeSeeded({
config: targetConfigPath,
registeredBaseWorkspaceCwd: baseRoot,
registeredProjectWorkspaceId: "project-workspace-1",
expectedCompanyId: "company-1",
}, { seedDatabase })).resolves.toMatchObject({ seeded: true, reason: "seeded" });
expect(seedDatabase).toHaveBeenCalledWith(expect.objectContaining({
sourceConfigPath,
expectedCompanyId: "company-1",
}));
} finally {
fs.rmSync(tempRoot, { recursive: true, force: true });
}
});
it.each(["sibling", "foreign_instance", "symlink", "instance_mismatch"] as const)(
"managed ensure-seeded rejects a %s manifest source before lock or seed mutation",
async (variant) => {
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), `paperclip-worktree-managed-${variant}-`));
try {
const baseRoot = path.join(tempRoot, "base");
const canonicalSource = path.join(baseRoot, ".paperclip", "config.json");
const targetRoot = path.join(tempRoot, "worktree");
const targetConfigPath = path.join(targetRoot, ".paperclip", "config.json");
const attackerRoot = path.join(tempRoot, variant);
const attackerConfig = path.join(attackerRoot, "config.json");
fs.mkdirSync(path.dirname(canonicalSource), { recursive: true });
fs.mkdirSync(path.dirname(targetConfigPath), { recursive: true });
fs.mkdirSync(attackerRoot, { recursive: true });
fs.writeFileSync(canonicalSource, `${JSON.stringify(buildSourceConfig())}\n`);
fs.writeFileSync(path.join(path.dirname(canonicalSource), ".env"), "PAPERCLIP_INSTANCE_ID=registered-source\n");
fs.writeFileSync(targetConfigPath, `${JSON.stringify(buildSourceConfig())}\n`);
fs.writeFileSync(
path.join(path.dirname(targetConfigPath), ".env"),
`PAPERCLIP_HOME=${path.join(tempRoot, "worktree-home")}\nPAPERCLIP_INSTANCE_ID=managed-target\n`,
);
fs.writeFileSync(attackerConfig, `${JSON.stringify(buildSourceConfig())}\n`);
fs.writeFileSync(
path.join(attackerRoot, ".env"),
`PAPERCLIP_INSTANCE_ID=${variant === "foreign_instance" ? "foreign" : "registered-source"}\n`,
);
const diagnosticPath = variant === "instance_mismatch"
? canonicalSource
: variant === "symlink"
? path.join(attackerRoot, "source-link.json")
: attackerConfig;
if (variant === "symlink") fs.symlinkSync(canonicalSource, diagnosticPath);
markWorktreeSeedPending({
configPath: targetConfigPath,
sourceConfigPath: diagnosticPath,
targetInstanceId: "managed-target",
});
if (variant === "instance_mismatch") {
const manifest = readWorktreeSeedManifest(targetConfigPath)!;
fs.writeFileSync(
path.join(path.dirname(targetConfigPath), "seed-manifest.json"),
JSON.stringify({ ...manifest, source: { ...manifest.source, instanceId: "foreign" } }),
);
}
const seedDatabase = vi.fn();
await expect(ensureWorktreeSeeded({
config: targetConfigPath,
registeredBaseWorkspaceCwd: baseRoot,
registeredProjectWorkspaceId: "project-workspace-1",
expectedCompanyId: "company-1",
}, { seedDatabase })).rejects.toThrow();
expect(seedDatabase).not.toHaveBeenCalled();
expect(fs.existsSync(path.join(targetRoot, ".paperclip", "seed.lock"))).toBe(false);
} finally {
fs.rmSync(tempRoot, { recursive: true, force: true });
}
},
);
it("ensure-seeded keeps the pending marker when seeding fails", async () => {
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-worktree-ensure-seeded-failure-"));
try {
@@ -485,6 +715,7 @@ describe("worktree helpers", () => {
fs.mkdirSync(path.dirname(sourceConfigPath), { recursive: true });
fs.mkdirSync(path.dirname(targetConfigPath), { recursive: true });
fs.writeFileSync(sourceConfigPath, `${JSON.stringify(sourceConfig)}\n`);
fs.writeFileSync(path.join(path.dirname(sourceConfigPath), ".env"), "PAPERCLIP_INSTANCE_ID=source\n");
fs.writeFileSync(targetConfigPath, `${JSON.stringify(targetConfig)}\n`);
fs.writeFileSync(
path.join(targetRoot, ".paperclip", ".env"),
@@ -494,12 +725,16 @@ describe("worktree helpers", () => {
await expect(
ensureWorktreeSeeded(
{ config: targetConfigPath },
{ config: targetConfigPath, fromConfig: sourceConfigPath },
{ seedDatabase: vi.fn().mockRejectedValue(new Error("seed failed")) },
),
).rejects.toThrow("seed failed");
expect(fs.existsSync(path.join(targetRoot, ".paperclip", "seed-pending"))).toBe(true);
expect(readWorktreeSeedManifest(targetConfigPath)).toMatchObject({
state: "failed",
phase: "pending",
});
expect(fs.existsSync(path.join(targetRoot, ".paperclip", "seed-pending"))).toBe(false);
expect(fs.existsSync(path.join(targetRoot, ".paperclip", "seed-complete"))).toBe(false);
expect(fs.existsSync(path.join(targetRoot, ".paperclip", "seed.lock"))).toBe(false);
} finally {
@@ -528,6 +763,7 @@ describe("worktree helpers", () => {
fs.mkdirSync(path.dirname(sourceConfigPath), { recursive: true });
fs.mkdirSync(path.dirname(targetConfigPath), { recursive: true });
fs.writeFileSync(sourceConfigPath, `${JSON.stringify(sourceConfig)}\n`);
fs.writeFileSync(path.join(path.dirname(sourceConfigPath), ".env"), "PAPERCLIP_INSTANCE_ID=source\n");
fs.writeFileSync(targetConfigPath, `${JSON.stringify(targetConfig)}\n`);
fs.writeFileSync(
path.join(targetRoot, ".paperclip", ".env"),
@@ -537,31 +773,17 @@ describe("worktree helpers", () => {
const seedDatabase = vi.fn(async () => {
await new Promise((resolve) => setTimeout(resolve, 100));
return {
backupSummary: "snapshot.sql",
pausedScheduledRoutines: 0,
executionQuarantine: {
disabledTimerHeartbeats: 0,
resetRunningAgents: 0,
quarantinedInProgressIssues: 0,
unassignedTodoIssues: 0,
unassignedReviewIssues: 0,
stoppedProjectWorkspaceRuntimes: 0,
stoppedExecutionWorkspaceRuntimes: 0,
stoppedRuntimeServices: 0,
},
reboundWorkspaces: [],
};
return mockVerifiedSeedResult();
});
const results = await Promise.all([
ensureWorktreeSeeded({ config: targetConfigPath }, { seedDatabase }),
ensureWorktreeSeeded({ config: targetConfigPath }, { seedDatabase }),
ensureWorktreeSeeded({ config: targetConfigPath, fromConfig: sourceConfigPath }, { seedDatabase }),
ensureWorktreeSeeded({ config: targetConfigPath, fromConfig: sourceConfigPath }, { seedDatabase }),
]);
expect(results).toEqual(expect.arrayContaining([
expect.objectContaining({ seeded: true, reason: "seeded" }),
{ seeded: false, reason: "complete_marker" },
{ seeded: false, reason: "verified_manifest" },
]));
expect(seedDatabase).toHaveBeenCalledTimes(1);
expect(fs.existsSync(path.join(targetRoot, ".paperclip", "seed.lock"))).toBe(false);
@@ -570,6 +792,59 @@ describe("worktree helpers", () => {
}
});
it("records an interrupted phase before retrying to a verified terminal state", async () => {
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-worktree-interrupted-seed-"));
try {
const sourceConfigPath = path.join(tempRoot, "source", "config.json");
const targetRoot = path.join(tempRoot, "worktree");
const targetConfigPath = path.join(targetRoot, ".paperclip", "config.json");
const targetPaths = resolveWorktreeLocalPaths({
cwd: targetRoot,
homeDir: path.join(tempRoot, "worktree-home"),
instanceId: "interrupted-seed",
});
const sourceConfig = buildSourceConfig();
const targetConfig = buildWorktreeConfig({
sourceConfig,
paths: targetPaths,
serverPort: 3196,
databasePort: 54996,
});
fs.mkdirSync(path.dirname(sourceConfigPath), { recursive: true });
fs.mkdirSync(path.dirname(targetConfigPath), { recursive: true });
fs.writeFileSync(sourceConfigPath, `${JSON.stringify(sourceConfig)}\n`);
fs.writeFileSync(path.join(path.dirname(sourceConfigPath), ".env"), "PAPERCLIP_INSTANCE_ID=source\n");
fs.writeFileSync(targetConfigPath, `${JSON.stringify(targetConfig)}\n`);
fs.writeFileSync(
path.join(targetRoot, ".paperclip", ".env"),
`PAPERCLIP_HOME=${targetPaths.homeDir}\nPAPERCLIP_INSTANCE_ID=${targetPaths.instanceId}\n`,
);
markWorktreeSeedPending({ configPath: targetConfigPath, sourceConfigPath });
const interrupted = readWorktreeSeedManifest(targetConfigPath)!;
fs.writeFileSync(
path.join(targetRoot, ".paperclip", "seed-manifest.json"),
`${JSON.stringify({ ...interrupted, state: "running", phase: "restore" }, null, 2)}\n`,
);
await expect(ensureWorktreeSeeded(
{ config: targetConfigPath, fromConfig: sourceConfigPath },
{ seedDatabase: vi.fn().mockResolvedValue(mockVerifiedSeedResult()) },
)).resolves.toMatchObject({ seeded: true, reason: "seeded" });
const verified = readWorktreeSeedManifest(targetConfigPath)!;
expect(verified.state).toBe("verified");
expect(verified.diagnostics).toEqual(expect.arrayContaining([
expect.objectContaining({
phase: "restore",
status: "failed",
message: "The previous seed attempt ended without a terminal result.",
}),
]));
} finally {
fs.rmSync(tempRoot, { recursive: true, force: true });
}
});
it("fails closed instead of racing to reclaim a stale seed lock", async () => {
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-worktree-ensure-seeded-stale-lock-"));
try {
@@ -969,7 +1244,7 @@ describe("worktree helpers", () => {
});
itEmbeddedPostgres(
"seeds authenticated users into minimally cloned worktree instances",
"seeds a source whose migration journal is behind the code journal",
async () => {
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-worktree-auth-seed-"));
const worktreeRoot = path.join(tempRoot, "PAP-999-auth-seed");
@@ -983,15 +1258,23 @@ describe("worktree helpers", () => {
const sourceDb = await startEmbeddedPostgresTestDatabase("paperclip-worktree-auth-source-");
try {
await seedValidWorktreeSource(sourceDb.connectionString);
const sourceDbClient = createDb(sourceDb.connectionString);
await sourceDbClient.insert(authUsers).values({
id: "user-existing",
email: "existing@paperclip.ing",
name: "Existing User",
emailVerified: true,
createdAt: new Date(),
updatedAt: new Date(),
});
await sourceDbClient.$client.unsafe(`
DELETE FROM "drizzle"."__drizzle_migrations"
WHERE "id" = (
SELECT max("id") FROM "drizzle"."__drizzle_migrations"
)
`);
await sourceDbClient.$client.end({ timeout: 5 });
const laggingMigrationState = await inspectMigrations(sourceDb.connectionString);
expect(laggingMigrationState.status).toBe("needsMigrations");
if (laggingMigrationState.status !== "needsMigrations") {
throw new Error("Expected the source migration journal to lag the code journal");
}
expect(laggingMigrationState.pendingMigrations).toHaveLength(1);
const sourceMigrationRevision = laggingMigrationState.appliedMigrations.at(-1);
expect(sourceMigrationRevision).toBeTruthy();
fs.mkdirSync(path.dirname(sourceKeyPath), { recursive: true });
fs.mkdirSync(worktreeRoot, { recursive: true });
@@ -1028,6 +1311,19 @@ describe("worktree helpers", () => {
const targetConfig = JSON.parse(
fs.readFileSync(path.join(worktreeRoot, ".paperclip", "config.json"), "utf8"),
) as PaperclipConfig;
const manifestText = fs.readFileSync(
path.join(worktreeRoot, ".paperclip", "seed-manifest.json"),
"utf8",
);
expect(JSON.parse(manifestText)).toMatchObject({
version: 2,
seedMode: "minimal",
state: "verified",
phase: "complete",
});
expect(manifestText).toContain(`Validated migration ${sourceMigrationRevision}`);
expect(manifestText).not.toContain("fixture-password-hash");
expect(manifestText).not.toContain("source-master-key");
const { default: EmbeddedPostgres } = await import("embedded-postgres");
const targetPg = new EmbeddedPostgres({
databaseDir: targetConfig.database.embeddedPostgresDataDir,
@@ -1284,9 +1580,8 @@ describe("worktree helpers", () => {
const originalCwd = process.cwd();
const originalPaperclipConfig = process.env.PAPERCLIP_CONFIG;
const currentDatabaseReservation = await reserveTestPort();
const sourceDatabaseReservation = await reserveTestPort();
const currentDatabasePort = currentDatabaseReservation.port;
const sourceDatabasePort = sourceDatabaseReservation.port;
const sourceDb = await startEmbeddedPostgresTestDatabase("paperclip-worktree-reseed-source-");
try {
fs.mkdirSync(path.dirname(currentPaths.configPath), { recursive: true });
@@ -1301,15 +1596,28 @@ describe("worktree helpers", () => {
serverPort: 3114,
databasePort: currentDatabasePort,
});
const sourceConfig = buildWorktreeConfig({
sourceConfig: buildSourceConfig(),
paths: sourcePaths,
serverPort: 3200,
databasePort: sourceDatabasePort,
});
const sourceConfig = buildSourceConfig();
sourceConfig.database = {
mode: "postgres",
embeddedPostgresDataDir: sourcePaths.embeddedPostgresDataDir,
embeddedPostgresPort: 54329,
backup: {
enabled: true,
intervalMinutes: 60,
retentionDays: 30,
dir: sourcePaths.backupDir,
},
connectionString: sourceDb.connectionString,
};
sourceConfig.logging.logDir = sourcePaths.logDir;
sourceConfig.storage.localDisk.baseDir = sourcePaths.storageDir;
sourceConfig.secrets.localEncrypted.keyFilePath = sourcePaths.secretsKeyFilePath;
await seedValidWorktreeSource(sourceDb.connectionString);
fs.writeFileSync(currentPaths.configPath, JSON.stringify(currentConfig, null, 2), "utf8");
fs.writeFileSync(sourcePaths.configPath, JSON.stringify(sourceConfig, null, 2), "utf8");
fs.writeFileSync(sourcePaths.secretsKeyFilePath, "source-secret", "utf8");
const worktreeSentinelPath = path.join(repoRoot, "user-worktree-file.txt");
fs.writeFileSync(worktreeSentinelPath, "preserve me", "utf8");
fs.writeFileSync(
currentPaths.envPath,
[
@@ -1325,11 +1633,11 @@ describe("worktree helpers", () => {
process.chdir(repoRoot);
await currentDatabaseReservation.release();
await sourceDatabaseReservation.release();
await worktreeReseedCommand({
fromConfig: sourcePaths.configPath,
yes: true,
backupTarget: true,
});
const rewrittenConfig = JSON.parse(fs.readFileSync(currentPaths.configPath, "utf8"));
@@ -1341,9 +1649,13 @@ describe("worktree helpers", () => {
expect(rewrittenEnv).toContain(`PAPERCLIP_INSTANCE_ID=${currentInstanceId}`);
expect(rewrittenEnv).toContain("PAPERCLIP_WORKTREE_NAME=existing-name");
expect(rewrittenEnv).toContain("PAPERCLIP_WORKTREE_COLOR=\"#112233\"");
expect(fs.readFileSync(worktreeSentinelPath, "utf8")).toBe("preserve me");
expect(
fs.readdirSync(path.join(currentPaths.backupDir, "repair")).some((name) => name.endsWith(".sql.gz")),
).toBe(true);
} finally {
await currentDatabaseReservation.release();
await sourceDatabaseReservation.release();
await sourceDb.cleanup();
process.chdir(originalCwd);
if (originalPaperclipConfig === undefined) {
delete process.env.PAPERCLIP_CONFIG;
+42
View File
@@ -5,12 +5,52 @@ import { expandHomePrefix } from "../config/home.js";
export const DEFAULT_WORKTREE_HOME = "~/.paperclip-worktrees";
export const WORKTREE_SEED_MODES = ["minimal", "full"] as const;
export const WORKTREE_SEED_MANIFEST = "seed-manifest.json";
export const WORKTREE_SEED_PENDING_MARKER = "seed-pending";
export const WORKTREE_SEED_COMPLETE_MARKER = "seed-complete";
export const WORKTREE_SEED_LOCK_MARKER = "seed.lock";
export type WorktreeSeedMode = (typeof WORKTREE_SEED_MODES)[number];
export const WORKTREE_SEED_PHASES = [
"pending",
"source_validation",
"snapshot",
"restore",
"migrations",
"execution_quarantine",
"routine_pause",
"workspace_rebind",
"post_restore_validation",
"complete",
] as const;
export type WorktreeSeedPhase = (typeof WORKTREE_SEED_PHASES)[number];
export type WorktreeSeedState = "pending" | "running" | "verified" | "failed";
export type WorktreeSeedManifest = {
version: 2;
source: {
instanceId: string;
configPath: string;
};
snapshotAt: string | null;
seedMode: WorktreeSeedMode;
migrationRevision: string | null;
targetInstanceId: string;
phase: WorktreeSeedPhase;
state: WorktreeSeedState;
attemptId: string;
startedAt: string | null;
finishedAt: string | null;
diagnostics: Array<{
phase: WorktreeSeedPhase;
status: "started" | "succeeded" | "failed";
at: string;
message?: string;
}>;
};
export type WorktreeSeedPlan = {
mode: WorktreeSeedMode;
excludedTables: string[];
@@ -54,6 +94,7 @@ export type WorktreeUiBranding = {
};
export type WorktreeSeedMarkerPaths = {
manifest: string;
pending: string;
complete: string;
lock: string;
@@ -62,6 +103,7 @@ export type WorktreeSeedMarkerPaths = {
export function resolveWorktreeSeedMarkerPaths(configPath: string): WorktreeSeedMarkerPaths {
const configDir = path.dirname(path.resolve(configPath));
return {
manifest: path.resolve(configDir, WORKTREE_SEED_MANIFEST),
pending: path.resolve(configDir, WORKTREE_SEED_PENDING_MARKER),
complete: path.resolve(configDir, WORKTREE_SEED_COMPLETE_MARKER),
lock: path.resolve(configDir, WORKTREE_SEED_LOCK_MARKER),
+674 -73
View File
@@ -7,6 +7,7 @@ import {
readdirSync,
readFileSync,
readlinkSync,
renameSync,
rmSync,
statSync,
symlinkSync,
@@ -21,11 +22,15 @@ import { Readable } from "node:stream";
import * as p from "@clack/prompts";
import pc from "picocolors";
import { and, eq, inArray, sql } from "drizzle-orm";
import { resolveCanonicalWorktreeSeedSource } from "@paperclipai/shared/worktree-seed-source";
import {
applyPendingMigrations,
agents,
authAccounts,
authUsers,
assets,
companies,
companyMemberships,
createDb,
documentRevisions,
documents,
@@ -39,6 +44,7 @@ import {
issueComments,
issueDocuments,
issues,
instanceUserRoles,
projectWorkspaces,
projects,
routines,
@@ -65,13 +71,16 @@ import {
formatShellExports,
generateWorktreeColor,
isWorktreeSeedMode,
WORKTREE_SEED_PHASES,
resolveSuggestedWorktreeName,
resolveWorktreeSeedPlan,
resolveWorktreeSeedMarkerPaths,
resolveWorktreeLocalPaths,
sanitizeWorktreeInstanceId,
type WorktreeSeedPlan,
type WorktreeSeedManifest,
type WorktreeSeedMode,
type WorktreeSeedPhase,
type WorktreeLocalPaths,
} from "./worktree-lib.js";
import {
@@ -137,6 +146,7 @@ type WorktreeReseedOptions = {
preserveLiveWork?: boolean;
yes?: boolean;
allowLiveTarget?: boolean;
backupTarget?: boolean;
};
type WorktreeRepairOptions = {
@@ -157,6 +167,9 @@ type WorktreeEnsureSeededOptions = {
fromDataDir?: string;
fromInstance?: string;
preserveLiveWork?: boolean;
registeredBaseWorkspaceCwd?: string;
registeredProjectWorkspaceId?: string;
expectedCompanyId?: string;
};
type EmbeddedPostgresInstance = {
@@ -197,6 +210,8 @@ type CopiedGitHooksResult = {
type SeedWorktreeDatabaseResult = {
backupSummary: string;
snapshotAt: string;
migrationRevision: string;
pausedScheduledRoutines: number;
executionQuarantine: SeededWorktreeExecutionQuarantineSummary;
reboundWorkspaces: Array<{
@@ -204,28 +219,26 @@ type SeedWorktreeDatabaseResult = {
fromCwd: string;
toCwd: string;
}>;
validation: WorktreeSeedValidationSummary;
};
type WorktreeSeedPendingMarker = {
version: 1;
state: "pending";
sourceConfigPath: string;
seedMode: "minimal";
createdAt: string;
};
type WorktreeSeedCompleteMarker = {
version: 1;
state: "complete";
seedMode: WorktreeSeedMode;
completedAt: string;
export type WorktreeSeedValidationSummary = {
authUserCount: number;
credentialAccountCount: number;
instanceAdminCount: number;
activeMembershipCount: number;
companyCount: number;
issueCount: number;
representativeCompanyId: string;
representativeIssueId: string;
migrationRevision: string;
};
type SeedWorktreeDatabase = typeof seedWorktreeDatabase;
export type EnsureWorktreeSeededResult = {
seeded: boolean;
reason: "seeded" | "complete_marker" | "legacy_unmarked";
reason: "seeded" | "verified_manifest" | "complete_marker" | "legacy_unmarked";
details?: SeedWorktreeDatabaseResult;
};
@@ -1393,6 +1406,174 @@ export async function quarantineSeededWorktreeExecutionState(
}
}
type WorktreeSeedValidationExpectation = {
adminUserId: string;
representativeCompanyId: string;
representativeIssueId: string;
};
export function resolveWorktreeSeedMigrationRevision(
migrationState: Awaited<ReturnType<typeof inspectMigrations>>,
requirement: "sourcePrefix" | "upToDate",
): string {
if (migrationState.journalEntryCount > migrationState.availableMigrations.length) {
throw new Error(
`Migration journal is ahead of this Paperclip checkout (${migrationState.journalEntryCount} applied migration(s), ${migrationState.availableMigrations.length} available).`,
);
}
const expectedAppliedPrefix = migrationState.availableMigrations.slice(
0,
migrationState.appliedMigrations.length,
);
if (
migrationState.appliedMigrations.some(
(migration, index) => migration !== expectedAppliedPrefix[index],
)
) {
throw new Error("Migration journal is not a prefix of this Paperclip checkout's migration journal.");
}
if (requirement === "upToDate" && migrationState.status !== "upToDate") {
throw new Error(
`Migration journal is not current (${migrationState.pendingMigrations.length} pending migration(s)).`,
);
}
const migrationRevision = migrationState.appliedMigrations.at(-1);
if (!migrationRevision) {
throw new Error("Migration journal has no applied revision.");
}
return migrationRevision;
}
async function inspectVerifiedSeedDatabase(
connectionString: string,
expected?: WorktreeSeedValidationExpectation,
migrationRequirement: "sourcePrefix" | "upToDate" = "upToDate",
requiredCompanyId?: string,
): Promise<{ summary: WorktreeSeedValidationSummary; expectation: WorktreeSeedValidationExpectation }> {
const migrationState = await inspectMigrations(connectionString);
const migrationRevision = resolveWorktreeSeedMigrationRevision(
migrationState,
migrationRequirement,
);
const db = createDb(connectionString);
try {
const [counts] = await db
.select({
authUserCount: sql<number>`count(distinct ${authUsers.id})::int`,
credentialAccountCount: sql<number>`count(distinct ${authAccounts.id})::int`,
instanceAdminCount: sql<number>`count(distinct ${instanceUserRoles.userId})::int`,
activeMembershipCount: sql<number>`count(distinct ${companyMemberships.id})::int`,
companyCount: sql<number>`count(distinct ${companies.id})::int`,
issueCount: sql<number>`count(distinct ${issues.id})::int`,
})
.from(authUsers)
.leftJoin(authAccounts, eq(authAccounts.userId, authUsers.id))
.leftJoin(
instanceUserRoles,
and(eq(instanceUserRoles.userId, authUsers.id), eq(instanceUserRoles.role, "instance_admin")),
)
.leftJoin(
companyMemberships,
and(
eq(companyMemberships.principalType, "user"),
eq(companyMemberships.principalId, authUsers.id),
eq(companyMemberships.status, "active"),
),
)
.leftJoin(companies, eq(companies.id, companyMemberships.companyId))
.leftJoin(issues, eq(issues.companyId, companies.id));
const admin = await db
.select({ userId: authUsers.id })
.from(authUsers)
.innerJoin(
instanceUserRoles,
and(eq(instanceUserRoles.userId, authUsers.id), eq(instanceUserRoles.role, "instance_admin")),
)
.innerJoin(
authAccounts,
and(
eq(authAccounts.userId, authUsers.id),
sql`length(trim(${authAccounts.providerId})) > 0`,
sql`length(trim(${authAccounts.accountId})) > 0`,
),
)
.innerJoin(
companyMemberships,
and(
eq(companyMemberships.principalType, "user"),
eq(companyMemberships.principalId, authUsers.id),
eq(companyMemberships.status, "active"),
),
)
.where(and(
expected ? eq(authUsers.id, expected.adminUserId) : undefined,
requiredCompanyId ? eq(companyMemberships.companyId, requiredCompanyId) : undefined,
))
.limit(1)
.then((rows) => rows[0] ?? null);
if (!admin) {
throw new Error(
"No auth user has a non-empty credential account, instance-admin role, and active company membership.",
);
}
const representative = await db
.select({ companyId: companies.id, issueId: issues.id })
.from(companies)
.innerJoin(issues, eq(issues.companyId, companies.id))
.where(
and(
expected ? eq(companies.id, expected.representativeCompanyId) : undefined,
expected ? eq(issues.id, expected.representativeIssueId) : undefined,
requiredCompanyId ? eq(companies.id, requiredCompanyId) : undefined,
),
)
.limit(1)
.then((rows) => rows[0] ?? null);
if (!representative) {
throw new Error("No representative cloned company and issue pair is readable.");
}
const summary: WorktreeSeedValidationSummary = {
authUserCount: counts?.authUserCount ?? 0,
credentialAccountCount: counts?.credentialAccountCount ?? 0,
instanceAdminCount: counts?.instanceAdminCount ?? 0,
activeMembershipCount: counts?.activeMembershipCount ?? 0,
companyCount: counts?.companyCount ?? 0,
issueCount: counts?.issueCount ?? 0,
representativeCompanyId: representative.companyId,
representativeIssueId: representative.issueId,
migrationRevision,
};
if (
summary.authUserCount < 1
|| summary.credentialAccountCount < 1
|| summary.instanceAdminCount < 1
|| summary.activeMembershipCount < 1
|| summary.companyCount < 1
|| summary.issueCount < 1
) {
throw new Error("Seed validation found an incomplete auth, membership, company, or issue shape.");
}
return {
summary,
expectation: {
adminUserId: admin.userId,
representativeCompanyId: representative.companyId,
representativeIssueId: representative.issueId,
},
};
} finally {
await db.$client?.end?.({ timeout: 5 }).catch(() => undefined);
}
}
async function seedWorktreeDatabase(input: {
sourceConfigPath: string;
sourceConfig: PaperclipConfig;
@@ -1401,16 +1582,12 @@ async function seedWorktreeDatabase(input: {
instanceId: string;
seedMode: WorktreeSeedMode;
preserveLiveWork?: boolean;
expectedCompanyId?: string;
onPhase?: (phase: WorktreeSeedPhase, status: "started" | "succeeded", message?: string) => void;
}): Promise<SeedWorktreeDatabaseResult> {
const seedPlan = resolveWorktreeSeedPlan(input.seedMode);
const sourceEnvFile = resolvePaperclipEnvFile(input.sourceConfigPath);
const sourceEnvEntries = readPaperclipEnvEntries(sourceEnvFile);
copySeededSecretsKey({
sourceConfigPath: input.sourceConfigPath,
sourceConfig: input.sourceConfig,
sourceEnvEntries,
targetKeyFilePath: input.targetPaths.secretsKeyFilePath,
});
let sourceHandle: EmbeddedPostgresHandle | null = null;
let targetHandle: EmbeddedPostgresHandle | null = null;
@@ -1428,6 +1605,27 @@ async function seedWorktreeDatabase(input: {
sourceEnvEntries,
sourceHandle?.port,
);
input.onPhase?.("source_validation", "started");
const sourceValidation = await inspectVerifiedSeedDatabase(
sourceConnectionString,
undefined,
"sourcePrefix",
input.expectedCompanyId,
);
input.onPhase?.(
"source_validation",
"succeeded",
`Validated migration ${sourceValidation.summary.migrationRevision}, ${sourceValidation.summary.companyCount} company record(s), and ${sourceValidation.summary.issueCount} issue record(s).`,
);
copySeededSecretsKey({
sourceConfigPath: input.sourceConfigPath,
sourceConfig: input.sourceConfig,
sourceEnvEntries,
targetKeyFilePath: input.targetPaths.secretsKeyFilePath,
});
const snapshotAt = new Date().toISOString();
input.onPhase?.("snapshot", "started");
const backup = await runDatabaseBackup({
connectionString: sourceConnectionString,
backupDir: path.resolve(input.targetPaths.backupDir, "seed"),
@@ -1438,6 +1636,7 @@ async function seedWorktreeDatabase(input: {
excludeTables: seedPlan.excludedTables,
nullifyColumns: seedPlan.nullifyColumns,
});
input.onPhase?.("snapshot", "succeeded", `Created ${path.basename(backup.backupFile)}.`);
targetHandle = await ensureEmbeddedPostgres(
input.targetConfig.database.embeddedPostgresDataDir,
@@ -1445,27 +1644,56 @@ async function seedWorktreeDatabase(input: {
);
const adminConnectionString = `postgres://paperclip:paperclip@127.0.0.1:${targetHandle.port}/postgres`;
input.onPhase?.("restore", "started");
await resetPostgresDatabase(adminConnectionString, "paperclip");
const targetConnectionString = `postgres://paperclip:paperclip@127.0.0.1:${targetHandle.port}/paperclip`;
await runDatabaseRestore({
connectionString: targetConnectionString,
backupFile: backup.backupFile,
});
input.onPhase?.("restore", "succeeded");
input.onPhase?.("migrations", "started");
await applyPendingMigrations(targetConnectionString);
input.onPhase?.("migrations", "succeeded");
input.onPhase?.("execution_quarantine", "started");
const executionQuarantine = input.preserveLiveWork
? { ...EMPTY_SEEDED_WORKTREE_EXECUTION_QUARANTINE_SUMMARY }
: await quarantineSeededWorktreeExecutionState(targetConnectionString);
input.onPhase?.(
"execution_quarantine",
"succeeded",
input.preserveLiveWork
? "Preserved copied live work by explicit request."
: formatSeededWorktreeExecutionQuarantineSummary(executionQuarantine),
);
input.onPhase?.("routine_pause", "started");
const pausedScheduledRoutines = await pauseSeededScheduledRoutines(targetConnectionString);
input.onPhase?.("routine_pause", "succeeded", `Paused ${pausedScheduledRoutines} scheduled routine(s).`);
input.onPhase?.("workspace_rebind", "started");
const reboundWorkspaces = await rebindSeededProjectWorkspaces({
targetConnectionString,
currentCwd: input.targetPaths.cwd,
});
input.onPhase?.("workspace_rebind", "succeeded", `Rebound ${reboundWorkspaces.length} workspace path(s).`);
input.onPhase?.("post_restore_validation", "started");
const targetValidation = await inspectVerifiedSeedDatabase(
targetConnectionString,
sourceValidation.expectation,
);
input.onPhase?.(
"post_restore_validation",
"succeeded",
`Validated migration ${targetValidation.summary.migrationRevision}.`,
);
return {
backupSummary: formatDatabaseBackupResult(backup),
snapshotAt,
migrationRevision: targetValidation.summary.migrationRevision,
pausedScheduledRoutines,
executionQuarantine,
reboundWorkspaces,
validation: targetValidation.summary,
};
} finally {
if (targetHandle?.startedByThisProcess) {
@@ -1477,46 +1705,184 @@ async function seedWorktreeDatabase(input: {
}
}
function writeWorktreeSeedMarker(
filePath: string,
marker: WorktreeSeedPendingMarker | WorktreeSeedCompleteMarker,
): void {
const WORKTREE_SEED_DIAGNOSTIC_LIMIT = 32;
const WORKTREE_SEED_DIAGNOSTIC_MESSAGE_LIMIT = 512;
const activeSeedInterruptHandlers = new Map<string, (signal: NodeJS.Signals) => void>();
function dispatchSeedInterruption(signal: NodeJS.Signals): void {
for (const handler of activeSeedInterruptHandlers.values()) {
try {
handler(signal);
} catch {
// Continue terminalizing the other active manifests before exiting.
}
}
process.exit(signal === "SIGINT" ? 130 : 143);
}
const dispatchSeedSigint = () => dispatchSeedInterruption("SIGINT");
const dispatchSeedSigterm = () => dispatchSeedInterruption("SIGTERM");
function registerSeedInterruptHandler(handler: (signal: NodeJS.Signals) => void): () => void {
const id = randomUUID();
if (activeSeedInterruptHandlers.size === 0) {
process.once("SIGINT", dispatchSeedSigint);
process.once("SIGTERM", dispatchSeedSigterm);
}
activeSeedInterruptHandlers.set(id, handler);
return () => {
activeSeedInterruptHandlers.delete(id);
if (activeSeedInterruptHandlers.size === 0) {
process.off("SIGINT", dispatchSeedSigint);
process.off("SIGTERM", dispatchSeedSigterm);
}
};
}
type LegacyWorktreeSeedPendingMarker = {
version: 1;
state: "pending";
sourceConfigPath: string;
};
function resolveSeedInstanceId(configPath: string): string {
const envEntries = readPaperclipEnvEntries(resolvePaperclipEnvFile(configPath));
return nonEmpty(envEntries.PAPERCLIP_INSTANCE_ID)
?? sanitizeWorktreeInstanceId(path.basename(path.dirname(path.resolve(configPath))));
}
function writeWorktreeSeedManifest(filePath: string, manifest: WorktreeSeedManifest): void {
mkdirSync(path.dirname(filePath), { recursive: true });
writeFileSync(filePath, `${JSON.stringify(marker, null, 2)}\n`, { mode: 0o600 });
const temporaryPath = `${filePath}.${process.pid}.${randomUUID()}.tmp`;
writeFileSync(temporaryPath, `${JSON.stringify(manifest, null, 2)}\n`, { mode: 0o600 });
renameSync(temporaryPath, filePath);
}
export function readWorktreeSeedManifest(configPath: string): WorktreeSeedManifest | null {
const manifestPath = resolveWorktreeSeedMarkerPaths(configPath).manifest;
if (!existsSync(manifestPath)) return null;
let parsed: unknown;
try {
parsed = JSON.parse(readFileSync(manifestPath, "utf8"));
} catch (error) {
throw new Error(
`Invalid worktree seed manifest at ${manifestPath}: ${error instanceof Error ? error.message : String(error)}`,
);
}
const value = parsed as Partial<WorktreeSeedManifest>;
const diagnosticsValid = Array.isArray(value.diagnostics) && value.diagnostics.every((diagnostic) => (
diagnostic
&& typeof diagnostic === "object"
&& WORKTREE_SEED_PHASES.includes(diagnostic.phase)
&& ["started", "succeeded", "failed"].includes(diagnostic.status)
&& typeof diagnostic.at === "string"
&& (diagnostic.message === undefined || typeof diagnostic.message === "string")
));
const verifiedTerminalValid = value.state !== "verified" || (
value.phase === "complete"
&& typeof value.snapshotAt === "string"
&& value.snapshotAt.length > 0
&& typeof value.migrationRevision === "string"
&& value.migrationRevision.length > 0
&& typeof value.startedAt === "string"
&& typeof value.finishedAt === "string"
&& value.diagnostics?.some((diagnostic) => (
diagnostic.phase === "complete" && diagnostic.status === "succeeded"
)) === true
);
if (
!value
|| typeof value !== "object"
|| value.version !== 2
|| !value.source
|| typeof value.source.instanceId !== "string"
|| typeof value.source.configPath !== "string"
|| typeof value.targetInstanceId !== "string"
|| value.targetInstanceId.length === 0
|| !isWorktreeSeedMode(String(value.seedMode ?? ""))
|| !WORKTREE_SEED_PHASES.includes(value.phase as WorktreeSeedPhase)
|| !["pending", "running", "verified", "failed"].includes(String(value.state ?? ""))
|| typeof value.attemptId !== "string"
|| value.attemptId.length === 0
|| !diagnosticsValid
|| !verifiedTerminalValid
) {
throw new Error(`Invalid worktree seed manifest at ${manifestPath}.`);
}
return value as WorktreeSeedManifest;
}
export function markWorktreeSeedPending(input: {
configPath: string;
sourceConfigPath: string;
now?: Date;
}): void {
const markers = resolveWorktreeSeedMarkerPaths(input.configPath);
rmSync(markers.complete, { force: true });
writeWorktreeSeedMarker(markers.pending, {
version: 1,
state: "pending",
sourceConfigPath: path.resolve(input.sourceConfigPath),
seedMode: "minimal",
createdAt: (input.now ?? new Date()).toISOString(),
});
}
export function markWorktreeSeedComplete(input: {
configPath: string;
targetInstanceId?: string;
seedMode?: WorktreeSeedMode;
now?: Date;
}): void {
const markers = resolveWorktreeSeedMarkerPaths(input.configPath);
writeWorktreeSeedMarker(markers.complete, {
version: 1,
state: "complete",
const at = (input.now ?? new Date()).toISOString();
writeWorktreeSeedManifest(markers.manifest, {
version: 2,
source: {
instanceId: resolveSeedInstanceId(input.sourceConfigPath),
configPath: path.resolve(input.sourceConfigPath),
},
snapshotAt: null,
seedMode: input.seedMode ?? "minimal",
completedAt: (input.now ?? new Date()).toISOString(),
migrationRevision: null,
targetInstanceId: input.targetInstanceId ?? resolveSeedInstanceId(input.configPath),
phase: "pending",
state: "pending",
attemptId: randomUUID(),
startedAt: null,
finishedAt: null,
diagnostics: [{ phase: "pending", status: "succeeded", at }],
});
// New manifests are authoritative. Legacy files are removed so no caller can
// mistake a stale binary marker for current verified seed state.
rmSync(markers.complete, { force: true });
rmSync(markers.pending, { force: true });
}
function readWorktreeSeedPendingMarker(filePath: string): WorktreeSeedPendingMarker {
function updateWorktreeSeedManifest(input: {
configPath: string;
phase: WorktreeSeedPhase;
status: "started" | "succeeded" | "failed";
state?: WorktreeSeedManifest["state"];
message?: string;
snapshotAt?: string | null;
migrationRevision?: string | null;
now?: Date;
}): WorktreeSeedManifest {
const markers = resolveWorktreeSeedMarkerPaths(input.configPath);
const current = readWorktreeSeedManifest(input.configPath);
if (!current) throw new Error(`Worktree seed manifest does not exist at ${markers.manifest}.`);
const at = (input.now ?? new Date()).toISOString();
const nextState = input.state ?? current.state;
const diagnostic = {
phase: input.phase,
status: input.status,
at,
...(input.message
? { message: input.message.slice(0, WORKTREE_SEED_DIAGNOSTIC_MESSAGE_LIMIT) }
: {}),
};
const next: WorktreeSeedManifest = {
...current,
phase: input.phase,
state: nextState,
snapshotAt: input.snapshotAt === undefined ? current.snapshotAt : input.snapshotAt,
migrationRevision:
input.migrationRevision === undefined ? current.migrationRevision : input.migrationRevision,
startedAt: current.startedAt ?? (input.status === "started" ? at : null),
finishedAt: nextState === "verified" || nextState === "failed" ? at : null,
diagnostics: [...current.diagnostics, diagnostic].slice(-WORKTREE_SEED_DIAGNOSTIC_LIMIT),
};
writeWorktreeSeedManifest(markers.manifest, next);
return next;
}
function readLegacyWorktreeSeedPendingMarker(filePath: string): LegacyWorktreeSeedPendingMarker {
let parsed: unknown;
try {
parsed = JSON.parse(readFileSync(filePath, "utf8"));
@@ -1537,7 +1903,7 @@ function readWorktreeSeedPendingMarker(filePath: string): WorktreeSeedPendingMar
throw new Error(`Invalid worktree seed-pending marker at ${filePath}.`);
}
return parsed as WorktreeSeedPendingMarker;
return parsed as LegacyWorktreeSeedPendingMarker;
}
const WORKTREE_SEED_LOCK_POLL_MS = 50;
@@ -1631,41 +1997,222 @@ async function acquireWorktreeSeedLock(lockPath: string): Promise<() => Promise<
}
}
function startWorktreeSeedAttempt(configPath: string, now = new Date()): WorktreeSeedManifest {
const markers = resolveWorktreeSeedMarkerPaths(configPath);
const current = readWorktreeSeedManifest(configPath);
if (!current) throw new Error(`Worktree seed manifest does not exist at ${markers.manifest}.`);
const at = now.toISOString();
const next: WorktreeSeedManifest = {
...current,
state: "running",
phase: "pending",
attemptId: randomUUID(),
snapshotAt: null,
migrationRevision: null,
startedAt: at,
finishedAt: null,
diagnostics: [
...current.diagnostics,
{ phase: "pending" as const, status: "started" as const, at },
].slice(-WORKTREE_SEED_DIAGNOSTIC_LIMIT),
};
writeWorktreeSeedManifest(markers.manifest, next);
return next;
}
async function runVerifiedWorktreeSeed(input: {
configPath: string;
sourceConfigPath: string;
sourceConfig: PaperclipConfig;
targetConfig: PaperclipConfig;
targetPaths: WorktreeLocalPaths;
instanceId: string;
seedMode: WorktreeSeedMode;
preserveLiveWork?: boolean;
expectedCompanyId?: string;
seedDatabase: SeedWorktreeDatabase;
}): Promise<SeedWorktreeDatabaseResult> {
let activePhase: WorktreeSeedPhase = "pending";
const previous = readWorktreeSeedManifest(input.configPath);
if (previous?.state === "running") {
updateWorktreeSeedManifest({
configPath: input.configPath,
phase: previous.phase,
status: "failed",
state: "failed",
message: "The previous seed attempt ended without a terminal result.",
});
}
startWorktreeSeedAttempt(input.configPath);
const unregisterInterruption = registerSeedInterruptHandler((signal) => {
updateWorktreeSeedManifest({
configPath: input.configPath,
phase: activePhase,
status: "failed",
state: "failed",
message: `Seed interrupted by ${signal} during ${activePhase}.`,
});
});
try {
const details = await input.seedDatabase({
sourceConfigPath: input.sourceConfigPath,
sourceConfig: input.sourceConfig,
targetConfig: input.targetConfig,
targetPaths: input.targetPaths,
instanceId: input.instanceId,
seedMode: input.seedMode,
preserveLiveWork: input.preserveLiveWork,
expectedCompanyId: input.expectedCompanyId,
onPhase: (phase, status, message) => {
activePhase = phase;
updateWorktreeSeedManifest({
configPath: input.configPath,
phase,
status,
state: "running",
message,
...(phase === "snapshot" && status === "started"
? { snapshotAt: new Date().toISOString() }
: {}),
});
},
});
if (!details.snapshotAt || !details.migrationRevision || !details.validation) {
throw new Error("Seed implementation returned without required validation evidence.");
}
updateWorktreeSeedManifest({
configPath: input.configPath,
phase: "complete",
status: "succeeded",
state: "verified",
snapshotAt: details.snapshotAt,
migrationRevision: details.migrationRevision,
message:
`Verified ${details.validation.companyCount} company record(s), `
+ `${details.validation.issueCount} issue record(s), auth, admin, membership, and migration state.`,
});
return details;
} catch (error) {
updateWorktreeSeedManifest({
configPath: input.configPath,
phase: activePhase,
status: "failed",
state: "failed",
// Do not persist the underlying error: database/driver errors may contain
// connection credentials. The CLI still returns the exact error to its caller.
message: `Seed failed during ${activePhase}.`,
});
throw error;
} finally {
unregisterInterruption();
}
}
export async function ensureWorktreeSeeded(
opts: WorktreeEnsureSeededOptions = {},
dependencies: { seedDatabase?: SeedWorktreeDatabase } = {},
): Promise<EnsureWorktreeSeededResult> {
const configPath = resolveConfigPath(opts.config);
const markers = resolveWorktreeSeedMarkerPaths(configPath);
let initialManifest = readWorktreeSeedManifest(configPath);
if (initialManifest?.state === "verified") {
return { seeded: false, reason: "verified_manifest" };
}
if (!initialManifest && existsSync(markers.complete)) {
return { seeded: false, reason: "complete_marker" };
}
const legacyPending = !initialManifest && existsSync(markers.pending)
? readLegacyWorktreeSeedPendingMarker(markers.pending)
: null;
if (!initialManifest && !legacyPending) {
if (existsSync(markers.lock)) {
const releaseExistingLock = await acquireWorktreeSeedLock(markers.lock);
await releaseExistingLock();
}
return { seeded: false, reason: "legacy_unmarked" };
}
const hasExplicitSource = Boolean(opts.fromConfig || opts.fromDataDir || opts.fromInstance);
const explicitSourceConfigPath = hasExplicitSource
? resolveSourceConfigPath({
fromConfig: opts.fromConfig,
fromDataDir: opts.fromDataDir,
fromInstance: opts.fromInstance,
})
: null;
const registeredBaseWorkspaceCwd = opts.registeredBaseWorkspaceCwd
?? nonEmpty(process.env.PAPERCLIP_WORKSPACE_BASE_CWD)
?? null;
const registeredProjectWorkspaceId = opts.registeredProjectWorkspaceId
?? nonEmpty(process.env.PAPERCLIP_PROJECT_WORKSPACE_ID)
?? null;
const expectedCompanyId = opts.expectedCompanyId
?? nonEmpty(process.env.PAPERCLIP_SEED_EXPECTED_COMPANY_ID)
?? nonEmpty(process.env.PAPERCLIP_COMPANY_ID)
?? undefined;
if (!explicitSourceConfigPath && registeredBaseWorkspaceCwd && (!registeredProjectWorkspaceId || !expectedCompanyId)) {
throw new Error(
"Managed worktree seed registration is incomplete; project workspace and company bindings are required.",
);
}
const targetRoot = path.dirname(path.dirname(configPath));
const targetPaths = resolveWorktreeReseedTargetPaths({ configPath, rootPath: targetRoot });
const resolveSeedSource = (manifest: WorktreeSeedManifest | null) => {
const diagnosticSource = manifest?.source ?? (legacyPending
? {
configPath: legacyPending.sourceConfigPath,
instanceId: resolveSeedInstanceId(legacyPending.sourceConfigPath),
}
: null);
return resolveCanonicalWorktreeSeedSource({
registeredBaseWorkspaceCwd,
explicitSourceConfigPath,
targetConfigPath: configPath,
expectedTargetInstanceId: targetPaths.instanceId,
manifestSource: diagnosticSource,
manifestTargetInstanceId: manifest?.targetInstanceId ?? targetPaths.instanceId,
});
};
// Fail before creating the seed lock or rewriting a legacy marker. The manifest
// is agent-writable diagnostic evidence and can never select this source.
let canonicalSource = resolveSeedSource(initialManifest);
mkdirSync(path.dirname(markers.lock), { recursive: true });
const releaseLock = await acquireWorktreeSeedLock(markers.lock);
try {
// These checks deliberately happen under the cross-process lock. A second
// service process waits for the first seed transaction, then observes the
// complete marker instead of cloning the same database concurrently.
if (existsSync(markers.complete)) {
// verified manifest instead of cloning the same database concurrently.
let manifest = readWorktreeSeedManifest(configPath);
if (manifest?.state === "verified") {
return { seeded: false, reason: "verified_manifest" };
}
if (!manifest && existsSync(markers.complete)) {
return { seeded: false, reason: "complete_marker" };
}
if (!existsSync(markers.pending)) {
if (!manifest && existsSync(markers.pending)) {
const currentLegacyPending = readLegacyWorktreeSeedPendingMarker(markers.pending);
if (currentLegacyPending.sourceConfigPath !== legacyPending?.sourceConfigPath) {
throw new Error("Worktree seed source diagnostics changed while waiting for the seed lock.");
}
markWorktreeSeedPending({
configPath,
sourceConfigPath: canonicalSource.configPath,
targetInstanceId: targetPaths.instanceId,
seedMode: "minimal",
});
manifest = readWorktreeSeedManifest(configPath);
}
if (!manifest) {
// Worktrees created before lazy seeding shipped were seeded eagerly and
// have neither marker. Preserve that compatibility without re-cloning.
return { seeded: false, reason: "legacy_unmarked" };
}
const pending = readWorktreeSeedPendingMarker(markers.pending);
const sourceConfigPath = opts.fromConfig || opts.fromDataDir || opts.fromInstance
? resolveSourceConfigPath({
fromConfig: opts.fromConfig,
fromDataDir: opts.fromDataDir,
fromInstance: opts.fromInstance,
})
: path.resolve(pending.sourceConfigPath);
if (path.resolve(sourceConfigPath) === path.resolve(configPath)) {
throw new Error(
"Source and target Paperclip configs are the same. Pass --from-config for the source instance.",
);
}
canonicalSource = resolveSeedSource(manifest);
const sourceConfigPath = canonicalSource.configPath;
const sourceConfig = readConfig(sourceConfigPath);
if (!sourceConfig) {
@@ -1676,19 +2223,19 @@ export async function ensureWorktreeSeeded(
throw new Error(`Target config not found at ${configPath}.`);
}
const targetRoot = path.dirname(path.dirname(configPath));
const targetPaths = resolveWorktreeReseedTargetPaths({ configPath, rootPath: targetRoot });
const seedDatabase = dependencies.seedDatabase ?? seedWorktreeDatabase;
const details = await seedDatabase({
const details = await runVerifiedWorktreeSeed({
configPath,
sourceConfigPath,
sourceConfig,
targetConfig,
targetPaths,
instanceId: targetPaths.instanceId,
seedMode: "minimal",
seedMode: manifest.seedMode,
preserveLiveWork: opts.preserveLiveWork,
expectedCompanyId,
seedDatabase,
});
markWorktreeSeedComplete({ configPath });
return { seeded: true, reason: "seeded", details };
} finally {
await releaseLock();
@@ -1762,6 +2309,8 @@ async function runWorktreeInit(opts: WorktreeInitOptions): Promise<void> {
markWorktreeSeedPending({
configPath: paths.configPath,
sourceConfigPath,
targetInstanceId: instanceId,
seedMode,
});
const sourceEnvEntries = readPaperclipEnvEntries(resolvePaperclipEnvFile(sourceConfigPath));
const existingAgentJwtSecret =
@@ -1790,8 +2339,11 @@ async function runWorktreeInit(opts: WorktreeInitOptions): Promise<void> {
}
const spinner = p.spinner();
spinner.start(`Seeding isolated worktree database from source instance (${seedMode})...`);
const markers = resolveWorktreeSeedMarkerPaths(paths.configPath);
const releaseSeedLock = await acquireWorktreeSeedLock(markers.lock);
try {
const seeded = await seedWorktreeDatabase({
const seeded = await runVerifiedWorktreeSeed({
configPath: paths.configPath,
sourceConfigPath,
sourceConfig,
targetConfig,
@@ -1799,16 +2351,18 @@ async function runWorktreeInit(opts: WorktreeInitOptions): Promise<void> {
instanceId,
seedMode,
preserveLiveWork: opts.preserveLiveWork,
seedDatabase: seedWorktreeDatabase,
});
seedSummary = seeded.backupSummary;
seedExecutionQuarantineSummary = seeded.executionQuarantine;
pausedScheduledRoutineCount = seeded.pausedScheduledRoutines;
reboundWorkspaceSummary = seeded.reboundWorkspaces;
markWorktreeSeedComplete({ configPath: paths.configPath, seedMode });
spinner.stop(`Seeded isolated worktree database (${seedMode}).`);
} catch (error) {
spinner.stop(pc.red("Failed to seed worktree database."));
throw error;
} finally {
await releaseSeedLock();
}
}
@@ -3480,6 +4034,34 @@ export async function worktreeMergeHistoryCommand(sourceArg: string | undefined,
}
}
async function backupWorktreeReseedTarget(input: {
targetConfig: PaperclipConfig;
targetPaths: WorktreeLocalPaths;
}): Promise<string> {
if (input.targetConfig.database.mode !== "embedded-postgres") {
throw new Error("Managed worktree repair requires an embedded PostgreSQL target.");
}
const targetHandle = await ensureEmbeddedPostgres(
input.targetConfig.database.embeddedPostgresDataDir,
input.targetConfig.database.embeddedPostgresPort,
);
try {
const adminConnectionString = `postgres://paperclip:paperclip@127.0.0.1:${targetHandle.port}/postgres`;
await ensurePostgresDatabase(adminConnectionString, "paperclip");
const result = await runDatabaseBackup({
connectionString: `postgres://paperclip:paperclip@127.0.0.1:${targetHandle.port}/paperclip`,
backupDir: path.resolve(input.targetPaths.backupDir, "repair"),
retention: { dailyDays: 30, weeklyWeeks: 12, monthlyMonths: 12 },
filenamePrefix: `${input.targetPaths.instanceId}-pre-repair`,
backupEngine: "auto",
includeMigrationJournal: true,
});
return formatDatabaseBackupResult(result);
} finally {
if (targetHandle.startedByThisProcess) await targetHandle.stop();
}
}
async function runWorktreeReseed(opts: WorktreeReseedOptions): Promise<void> {
const seedMode = opts.seedMode ?? "full";
if (!isWorktreeSeedMode(seedMode)) {
@@ -3535,8 +4117,23 @@ async function runWorktreeReseed(opts: WorktreeReseedOptions): Promise<void> {
const spinner = p.spinner();
spinner.start(`Reseeding ${targetEndpoint.label} from ${source.label} (${seedMode})...`);
const markers = resolveWorktreeSeedMarkerPaths(targetEndpoint.configPath);
mkdirSync(path.dirname(markers.lock), { recursive: true });
const releaseSeedLock = await acquireWorktreeSeedLock(markers.lock);
try {
const seeded = await seedWorktreeDatabase({
let targetBackupSummary: string | null = null;
if (opts.backupTarget) {
targetBackupSummary = await backupWorktreeReseedTarget({ targetConfig, targetPaths });
p.log.message(pc.dim(`Recoverable pre-repair backup: ${targetBackupSummary}`));
}
markWorktreeSeedPending({
configPath: targetEndpoint.configPath,
sourceConfigPath: source.configPath,
targetInstanceId: targetPaths.instanceId,
seedMode,
});
const seeded = await runVerifiedWorktreeSeed({
configPath: targetEndpoint.configPath,
sourceConfigPath: source.configPath,
sourceConfig,
targetConfig,
@@ -3544,8 +4141,9 @@ async function runWorktreeReseed(opts: WorktreeReseedOptions): Promise<void> {
instanceId: targetPaths.instanceId,
seedMode,
preserveLiveWork: opts.preserveLiveWork,
expectedCompanyId: nonEmpty(process.env.PAPERCLIP_SEED_EXPECTED_COMPANY_ID) ?? undefined,
seedDatabase: seedWorktreeDatabase,
});
markWorktreeSeedComplete({ configPath: targetEndpoint.configPath, seedMode });
spinner.stop(`Reseeded ${targetEndpoint.label} (${seedMode}).`);
p.log.message(pc.dim(`Source: ${source.configPath}`));
p.log.message(pc.dim(`Target: ${targetEndpoint.configPath}`));
@@ -3567,6 +4165,8 @@ async function runWorktreeReseed(opts: WorktreeReseedOptions): Promise<void> {
} catch (error) {
spinner.stop(pc.red("Failed to reseed worktree database."));
throw error;
} finally {
await releaseSeedLock();
}
}
@@ -3748,6 +4348,7 @@ export function registerWorktreeCommands(program: Command): void {
.option("--preserve-live-work", "Do not quarantine copied agent work or workspace runtime services in the seeded worktree", false)
.option("--yes", "Skip the destructive confirmation prompt", false)
.option("--allow-live-target", "Override the guard that requires the target worktree DB to be stopped first", false)
.option("--backup-target", "Retain a recoverable full backup of the isolated target DB before reseeding", false)
.action(worktreeReseedCommand);
worktree