mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-12 18:06:11 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Execution workspaces need isolated databases, ports, and runtime services > - Concurrent workspaces could reuse ports or lose service ownership after a restart > - A markerless worktree also needed seed recovery, but normal markerless instances still needed to boot > - This pull request makes seed, port, and service ownership state explicit and recoverable > - It also checks live process and listener identity before it reclaims shared resources > - The benefit is reliable workspace startup, restart, adoption, and concurrent provisioning ## Linked Issues or Issue Description **What happened?** Managed workspaces could lose runtime service ownership after a control-plane restart. Concurrent worktrees could also reuse a port when their parent paths differed. A seed recovery change made every markerless instance resolve a worktree seed source, so normal instances without a source could not start. **Expected behavior** Paperclip must preserve healthy managed services across restarts. It must reserve unique ports across worktree parents. It must provision a registered markerless worktree, but it must skip seed work for a normal markerless instance. **Steps to reproduce** 1. Start two managed worktrees under different parent paths at the same time. 2. Restart the control plane while a managed service stays alive. 3. Start Paperclip with a config that has no seed markers and no registered worktree source. 4. Observe duplicate port selection, lost service adoption, or a seed-source startup error. **Paperclip version or commit** Current `master` plus the workspace runtime reliability changes in this pull request. **Deployment mode** Local development with managed execution workspaces and embedded Postgres. ## What Changed - Added a shared port registry with lease heartbeats, process identity checks, and live listener probes. - Reserved worktree ports across custom parent paths and repaired duplicate legacy assignments. - Preserved and adopted healthy managed services across control-plane restarts. - Reconciled guest bind modes and verified listener ownership before termination or reuse. - Provisioned registered markerless worktree databases and kept normal markerless instance startup as a no-op. - Added CLI, shared, server, and shell regression tests for seed, port, listener, restart, and adoption behavior. - Updated the worktree development documentation. ## Verification - `pnpm exec vitest run cli/src/__tests__/worktree.test.ts --reporter=verbose` — 63 tests passed. - `pnpm exec vitest run packages/shared/src/worktree-port-registry.test.ts --reporter=verbose` — 5 tests passed. - Focused runtime Vitest set — 199 tests passed across 37 suites. - `node --test scripts/__tests__/provision-worktree-self-heal.test.mjs` — 10 tests passed. - `git diff --check` passed. ## Risks - Port reservation now depends on lease and process identity data. The fallback listener probe prevents early reclamation when process metadata is incomplete. - Runtime adoption is stricter about bind and owner identity. The tests cover healthy adoption, stale records, PID reuse, and unrelated listeners. - Markerless seed detection now separates registered worktrees from normal instances. The tests cover both paths. - There are no database schema migrations. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with the `gpt-5` model family. The serving snapshot and context-window size are not exposed. The agent used reasoning, repository tools, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Dev Agent <dev@paperclip.ing>
230 lines
8.9 KiB
TypeScript
230 lines
8.9 KiB
TypeScript
import { execFile } from "node:child_process";
|
|
import fs from "node:fs/promises";
|
|
import os from "node:os";
|
|
import path from "node:path";
|
|
import { fileURLToPath } from "node:url";
|
|
import { promisify } from "node:util";
|
|
import { afterEach, describe, expect, it } from "vitest";
|
|
import {
|
|
ensureWorktreeSeeded,
|
|
readWorktreeSeedManifest,
|
|
} from "../../../cli/src/commands/worktree.ts";
|
|
import { realizeExecutionWorkspace } from "../services/workspace-runtime.ts";
|
|
|
|
const execFileAsync = promisify(execFile);
|
|
const cleanup: string[] = [];
|
|
const originalConfig = process.env.PAPERCLIP_CONFIG;
|
|
const originalWorktreesDir = process.env.PAPERCLIP_WORKTREES_DIR;
|
|
|
|
async function runGit(cwd: string, args: string[]) {
|
|
await execFileAsync("git", args, { cwd });
|
|
}
|
|
|
|
async function writeConfig(configPath: string, instanceId: string) {
|
|
const instanceRoot = path.dirname(configPath);
|
|
await fs.mkdir(path.dirname(configPath), { recursive: true });
|
|
await fs.writeFile(configPath, `${JSON.stringify({
|
|
$meta: { version: 1, updatedAt: "2026-08-19T00:00:00.000Z", source: "configure" },
|
|
database: {
|
|
mode: "embedded-postgres",
|
|
embeddedPostgresDataDir: path.join(instanceRoot, "db"),
|
|
embeddedPostgresPort: 54329,
|
|
backup: { enabled: false, intervalMinutes: 60, retentionDays: 30, dir: path.join(instanceRoot, "backups") },
|
|
},
|
|
logging: { mode: "file", logDir: path.join(instanceRoot, "logs") },
|
|
server: {
|
|
deploymentMode: "local_trusted",
|
|
exposure: "private",
|
|
host: "127.0.0.1",
|
|
port: 3100,
|
|
allowedHostnames: [],
|
|
serveUi: true,
|
|
},
|
|
}, null, 2)}\n`, "utf8");
|
|
await fs.writeFile(
|
|
path.join(path.dirname(configPath), ".env"),
|
|
`PAPERCLIP_INSTANCE_ID=${instanceId}\n`,
|
|
"utf8",
|
|
);
|
|
}
|
|
|
|
function verifiedSeedResult() {
|
|
return {
|
|
backupSummary: "snapshot.sql",
|
|
snapshotAt: "2026-08-19T00:00:00.000Z",
|
|
migrationRevision: "0223_test.sql",
|
|
pausedScheduledRoutines: 0,
|
|
executionQuarantine: {
|
|
disabledTimerHeartbeats: 0,
|
|
resetRunningAgents: 0,
|
|
quarantinedInProgressIssues: 0,
|
|
unassignedTodoIssues: 0,
|
|
unassignedReviewIssues: 0,
|
|
stoppedProjectWorkspaceRuntimes: 0,
|
|
stoppedExecutionWorkspaceRuntimes: 0,
|
|
stoppedRuntimeServices: 0,
|
|
},
|
|
reboundWorkspaces: [],
|
|
validation: {
|
|
authUserCount: 1,
|
|
credentialAccountCount: 1,
|
|
instanceAdminCount: 1,
|
|
activeMembershipCount: 1,
|
|
companyCount: 1,
|
|
issueCount: 1,
|
|
representativeCompanyId: "00000000-0000-4000-8000-000000000001",
|
|
representativeIssueId: "00000000-0000-4000-8000-000000000002",
|
|
migrationRevision: "0223_test.sql",
|
|
},
|
|
};
|
|
}
|
|
|
|
afterEach(async () => {
|
|
if (originalConfig === undefined) delete process.env.PAPERCLIP_CONFIG;
|
|
else process.env.PAPERCLIP_CONFIG = originalConfig;
|
|
if (originalWorktreesDir === undefined) delete process.env.PAPERCLIP_WORKTREES_DIR;
|
|
else process.env.PAPERCLIP_WORKTREES_DIR = originalWorktreesDir;
|
|
for (const dir of cleanup.splice(0)) {
|
|
await fs.rm(dir, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
describe("managed worktree seed source through the server spawn path", () => {
|
|
it("re-derives an ambient-instance manifest written before provisioning", async () => {
|
|
const tempRoot = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-server-seed-source-"));
|
|
cleanup.push(tempRoot);
|
|
const repoRoot = path.join(tempRoot, "repo");
|
|
const hooksDir = path.join(tempRoot, "hooks");
|
|
const ambientConfigPath = path.join(tempRoot, "ambient", "config.json");
|
|
const registeredConfigPath = path.join(repoRoot, ".paperclip", "config.json");
|
|
const worktreeHome = path.join(tempRoot, "worktree-home");
|
|
|
|
await fs.mkdir(repoRoot, { recursive: true });
|
|
await runGit(repoRoot, ["init", "-q"]);
|
|
await runGit(repoRoot, ["config", "user.email", "paperclip@example.com"]);
|
|
await runGit(repoRoot, ["config", "user.name", "Paperclip Test"]);
|
|
await fs.mkdir(path.join(repoRoot, "scripts"), { recursive: true });
|
|
await fs.writeFile(path.join(repoRoot, "README.md"), "server spawn regression\n", "utf8");
|
|
await fs.copyFile(
|
|
fileURLToPath(new URL("../../../scripts/provision-worktree.sh", import.meta.url)),
|
|
path.join(repoRoot, "scripts", "provision-worktree.sh"),
|
|
);
|
|
await fs.chmod(path.join(repoRoot, "scripts", "provision-worktree.sh"), 0o755);
|
|
await runGit(repoRoot, ["add", "README.md", "scripts/provision-worktree.sh"]);
|
|
await runGit(repoRoot, ["commit", "-qm", "Add managed provision script"]);
|
|
await writeConfig(registeredConfigPath, "registered-source");
|
|
await writeConfig(ambientConfigPath, "ambient-instance");
|
|
await fs.mkdir(worktreeHome, { recursive: true });
|
|
|
|
// Reproduce the integrated lane's ordering with a checkout hook: an earlier
|
|
// worktree-init-style writer inherits the server environment and leaves a valid
|
|
// target config whose manifest diagnostic points at the ambient instance.
|
|
await fs.mkdir(hooksDir, { recursive: true });
|
|
const hookPath = path.join(hooksDir, "post-checkout");
|
|
await fs.writeFile(
|
|
hookPath,
|
|
`#!/usr/bin/env node
|
|
const crypto = require("node:crypto");
|
|
const fs = require("node:fs");
|
|
const path = require("node:path");
|
|
const cwd = process.cwd();
|
|
if (cwd === ${JSON.stringify(repoRoot)}) process.exit(0);
|
|
const stateDir = path.join(cwd, ".paperclip");
|
|
const normalized = path.basename(cwd).trim().toLowerCase().replace(/[^a-z0-9_-]+/g, "-").replace(/-+/g, "-").replace(/^[-_]+|[-_]+$/g, "");
|
|
const instanceId = \`${"${(normalized || \"worktree\").slice(0, 48)}"}-\${crypto.createHash("sha256").update(path.resolve(cwd)).digest("hex").slice(0, 12)}\`;
|
|
const targetConfigPath = path.join(stateDir, "config.json");
|
|
const instanceRoot = path.join(process.env.PAPERCLIP_WORKTREES_DIR, "instances", instanceId);
|
|
fs.mkdirSync(stateDir, { recursive: true });
|
|
fs.mkdirSync(instanceRoot, { recursive: true });
|
|
fs.writeFileSync(targetConfigPath, JSON.stringify({
|
|
$meta: { version: 1, updatedAt: "2026-08-19T00:00:00.000Z", source: "configure" },
|
|
database: {
|
|
mode: "embedded-postgres",
|
|
embeddedPostgresDataDir: path.join(instanceRoot, "db"),
|
|
embeddedPostgresPort: 54330,
|
|
backup: { enabled: false, intervalMinutes: 60, retentionDays: 30, dir: path.join(instanceRoot, "backups") },
|
|
},
|
|
logging: { mode: "file", logDir: path.join(instanceRoot, "logs") },
|
|
server: {
|
|
deploymentMode: "local_trusted",
|
|
exposure: "private",
|
|
host: "127.0.0.1",
|
|
port: 3101,
|
|
allowedHostnames: [],
|
|
serveUi: true,
|
|
},
|
|
}, null, 2) + "\\n");
|
|
fs.writeFileSync(path.join(stateDir, ".env"), [
|
|
"PAPERCLIP_HOME=" + JSON.stringify(process.env.PAPERCLIP_WORKTREES_DIR),
|
|
"PAPERCLIP_INSTANCE_ID=" + JSON.stringify(instanceId),
|
|
"PAPERCLIP_CONFIG=" + JSON.stringify(targetConfigPath),
|
|
"",
|
|
].join("\\n"));
|
|
fs.writeFileSync(path.join(stateDir, "seed-manifest.json"), JSON.stringify({
|
|
version: 2,
|
|
source: { instanceId: "ambient-instance", configPath: process.env.PAPERCLIP_CONFIG },
|
|
snapshotAt: null,
|
|
seedMode: "minimal",
|
|
migrationRevision: null,
|
|
targetInstanceId: instanceId,
|
|
phase: "pending",
|
|
state: "pending",
|
|
attemptId: "ambient-writer",
|
|
startedAt: null,
|
|
finishedAt: null,
|
|
diagnostics: [{ phase: "pending", status: "succeeded", at: new Date().toISOString() }],
|
|
}, null, 2) + "\\n");
|
|
`,
|
|
"utf8",
|
|
);
|
|
await fs.chmod(hookPath, 0o755);
|
|
await runGit(repoRoot, ["config", "core.hooksPath", hooksDir]);
|
|
|
|
process.env.PAPERCLIP_CONFIG = ambientConfigPath;
|
|
process.env.PAPERCLIP_WORKTREES_DIR = worktreeHome;
|
|
const workspace = await realizeExecutionWorkspace({
|
|
base: {
|
|
baseCwd: repoRoot,
|
|
source: "project_primary",
|
|
projectId: "project-1",
|
|
workspaceId: "project-workspace-1",
|
|
repoUrl: null,
|
|
repoRef: "HEAD",
|
|
},
|
|
config: {
|
|
workspaceStrategy: {
|
|
type: "git_worktree",
|
|
branchTemplate: "{{issue.identifier}}-{{slug}}",
|
|
provisionCommand: "bash ./scripts/provision-worktree.sh",
|
|
},
|
|
},
|
|
issue: {
|
|
id: "issue-1",
|
|
identifier: "PAP-17681",
|
|
title: "Recover ambient seed source",
|
|
},
|
|
agent: { id: "agent-1", name: "Coder", companyId: "company-1" },
|
|
});
|
|
|
|
expect(readWorktreeSeedManifest(path.join(workspace.cwd, ".paperclip", "config.json"))).toMatchObject({
|
|
source: { instanceId: "ambient-instance", configPath: ambientConfigPath },
|
|
state: "pending",
|
|
});
|
|
|
|
await expect(ensureWorktreeSeeded({
|
|
config: path.join(workspace.cwd, ".paperclip", "config.json"),
|
|
registeredBaseWorkspaceCwd: repoRoot,
|
|
registeredProjectWorkspaceId: "project-workspace-1",
|
|
expectedCompanyId: "company-1",
|
|
}, {
|
|
seedDatabase: async () => verifiedSeedResult(),
|
|
})).resolves.toMatchObject({ seeded: true, reason: "seeded" });
|
|
|
|
expect(readWorktreeSeedManifest(path.join(workspace.cwd, ".paperclip", "config.json"))).toMatchObject({
|
|
source: { instanceId: "registered-source", configPath: registeredConfigPath },
|
|
state: "verified",
|
|
phase: "complete",
|
|
});
|
|
}, 20_000);
|
|
});
|