mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-30 13:20:54 +01:00
fix(workspaces): make managed runtimes reliable across restarts (#11740)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Execution workspaces need isolated databases, ports, and runtime services > - Concurrent workspaces could reuse ports or lose service ownership after a restart > - A markerless worktree also needed seed recovery, but normal markerless instances still needed to boot > - This pull request makes seed, port, and service ownership state explicit and recoverable > - It also checks live process and listener identity before it reclaims shared resources > - The benefit is reliable workspace startup, restart, adoption, and concurrent provisioning ## Linked Issues or Issue Description **What happened?** Managed workspaces could lose runtime service ownership after a control-plane restart. Concurrent worktrees could also reuse a port when their parent paths differed. A seed recovery change made every markerless instance resolve a worktree seed source, so normal instances without a source could not start. **Expected behavior** Paperclip must preserve healthy managed services across restarts. It must reserve unique ports across worktree parents. It must provision a registered markerless worktree, but it must skip seed work for a normal markerless instance. **Steps to reproduce** 1. Start two managed worktrees under different parent paths at the same time. 2. Restart the control plane while a managed service stays alive. 3. Start Paperclip with a config that has no seed markers and no registered worktree source. 4. Observe duplicate port selection, lost service adoption, or a seed-source startup error. **Paperclip version or commit** Current `master` plus the workspace runtime reliability changes in this pull request. **Deployment mode** Local development with managed execution workspaces and embedded Postgres. ## What Changed - Added a shared port registry with lease heartbeats, process identity checks, and live listener probes. - Reserved worktree ports across custom parent paths and repaired duplicate legacy assignments. - Preserved and adopted healthy managed services across control-plane restarts. - Reconciled guest bind modes and verified listener ownership before termination or reuse. - Provisioned registered markerless worktree databases and kept normal markerless instance startup as a no-op. - Added CLI, shared, server, and shell regression tests for seed, port, listener, restart, and adoption behavior. - Updated the worktree development documentation. ## Verification - `pnpm exec vitest run cli/src/__tests__/worktree.test.ts --reporter=verbose` — 63 tests passed. - `pnpm exec vitest run packages/shared/src/worktree-port-registry.test.ts --reporter=verbose` — 5 tests passed. - Focused runtime Vitest set — 199 tests passed across 37 suites. - `node --test scripts/__tests__/provision-worktree-self-heal.test.mjs` — 10 tests passed. - `git diff --check` passed. ## Risks - Port reservation now depends on lease and process identity data. The fallback listener probe prevents early reclamation when process metadata is incomplete. - Runtime adoption is stricter about bind and owner identity. The tests cover healthy adoption, stale records, PID reuse, and unrelated listeners. - Markerless seed detection now separates registered worktrees from normal instances. The tests cover both paths. - There are no database schema migrations. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with the `gpt-5` model family. The serving snapshot and context-window size are not exposed. The agent used reasoning, repository tools, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Dev Agent <dev@paperclip.ing>
This commit is contained in:
21 files changed
+2406
-345
No files matched your search
@@ -0,0 +1,155 @@
|
||||
import { spawn } from "node:child_process";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import fs from "node:fs/promises";
|
||||
import net from "node:net";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import {
|
||||
resetRuntimeServicesForTests,
|
||||
startRuntimeServicesForWorkspaceControl,
|
||||
} from "../services/workspace-runtime.js";
|
||||
import {
|
||||
doesLocalServiceCommandLineMatch,
|
||||
listLocalServiceRegistryRecords,
|
||||
readLocalServicePortOwner,
|
||||
resolveLocalServiceLogPath,
|
||||
terminateLocalService,
|
||||
} from "../services/local-service-supervisor.js";
|
||||
|
||||
describe("local service supervision", () => {
|
||||
afterEach(async () => {
|
||||
await resetRuntimeServicesForTests({ terminateProcesses: true });
|
||||
});
|
||||
|
||||
it("keeps request-logging runtime stdio usable after the supervisor side closes", async () => {
|
||||
const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-service-stdio-"));
|
||||
const paperclipHome = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-service-home-"));
|
||||
const previousPaperclipHome = process.env.PAPERCLIP_HOME;
|
||||
const previousInstanceId = process.env.PAPERCLIP_INSTANCE_ID;
|
||||
process.env.PAPERCLIP_HOME = paperclipHome;
|
||||
process.env.PAPERCLIP_INSTANCE_ID = `service-stdio-${randomUUID()}`;
|
||||
|
||||
let registryRecord: Awaited<ReturnType<typeof listLocalServiceRegistryRecords>>[number] | null = null;
|
||||
try {
|
||||
const [service] = await startRuntimeServicesForWorkspaceControl({
|
||||
actor: { id: null, name: "Board", companyId: randomUUID() },
|
||||
issue: null,
|
||||
workspace: {
|
||||
baseCwd: workspaceRoot,
|
||||
source: "agent_home",
|
||||
projectId: null,
|
||||
workspaceId: null,
|
||||
repoUrl: null,
|
||||
repoRef: null,
|
||||
strategy: "project_primary",
|
||||
cwd: workspaceRoot,
|
||||
branchName: null,
|
||||
worktreePath: null,
|
||||
warnings: [],
|
||||
created: false,
|
||||
},
|
||||
config: {
|
||||
workspaceRuntime: {
|
||||
services: [{
|
||||
name: "web",
|
||||
command: "node -e \"const http=require('node:http'); process.on('SIGTERM',()=>{}); http.createServer((req,res)=>{ process.stdout.write('request '+req.url+'\\\\n',(error)=>{ if (!error) res.end('ok'); }); }).listen(Number(process.env.PORT), '127.0.0.1')\"",
|
||||
port: { type: "auto" },
|
||||
readiness: {
|
||||
type: "http",
|
||||
urlTemplate: "http://127.0.0.1:{{port}}",
|
||||
timeoutSec: 10,
|
||||
intervalMs: 100,
|
||||
},
|
||||
lifecycle: "shared",
|
||||
reuseScope: "agent",
|
||||
stopPolicy: { type: "manual" },
|
||||
}],
|
||||
},
|
||||
},
|
||||
adapterEnv: {},
|
||||
});
|
||||
|
||||
expect(service?.url).toMatch(/^http:\/\/127\.0\.0\.1:\d+$/);
|
||||
expect(service?.port).toBeTypeOf("number");
|
||||
registryRecord = (await listLocalServiceRegistryRecords({ profileKind: "workspace-runtime" }))
|
||||
.find((record) => record.runtimeServiceId === service!.id) ?? null;
|
||||
expect(registryRecord).not.toBeNull();
|
||||
|
||||
await resetRuntimeServicesForTests({ simulateSupervisorExit: true });
|
||||
|
||||
await expect(fetch(`${service!.url}/after-restart`)).resolves.toMatchObject({ ok: true });
|
||||
const log = await fs.readFile(resolveLocalServiceLogPath(registryRecord!.serviceKey), "utf8");
|
||||
expect(log).toContain("request /after-restart");
|
||||
|
||||
await terminateLocalService(registryRecord!);
|
||||
expect(await readLocalServicePortOwner(service!.port!)).toBeNull();
|
||||
await expect(fetch(service!.url!)).rejects.toThrow();
|
||||
registryRecord = null;
|
||||
} finally {
|
||||
if (registryRecord) await terminateLocalService(registryRecord).catch(() => undefined);
|
||||
if (previousPaperclipHome === undefined) delete process.env.PAPERCLIP_HOME;
|
||||
else process.env.PAPERCLIP_HOME = previousPaperclipHome;
|
||||
if (previousInstanceId === undefined) delete process.env.PAPERCLIP_INSTANCE_ID;
|
||||
else process.env.PAPERCLIP_INSTANCE_ID = previousInstanceId;
|
||||
await fs.rm(paperclipHome, { recursive: true, force: true });
|
||||
await fs.rm(workspaceRoot, { recursive: true, force: true });
|
||||
}
|
||||
}, 15_000);
|
||||
|
||||
it("does not wait for an unrelated process that takes over the service port", async () => {
|
||||
if (process.platform === "win32") return;
|
||||
const unrelatedListener = net.createServer();
|
||||
await new Promise<void>((resolve) => unrelatedListener.listen(0, "127.0.0.1", resolve));
|
||||
const address = unrelatedListener.address();
|
||||
const port = typeof address === "object" && address ? address.port : null;
|
||||
if (!port) throw new Error("Failed to allocate unrelated listener port");
|
||||
const child = spawn(process.execPath, ["-e", "setInterval(()=>{},1000)"], {
|
||||
detached: true,
|
||||
stdio: "ignore",
|
||||
});
|
||||
child.unref();
|
||||
|
||||
try {
|
||||
await terminateLocalService({
|
||||
pid: child.pid!,
|
||||
processGroupId: child.pid!,
|
||||
port,
|
||||
});
|
||||
expect(await readLocalServicePortOwner(port)).toBe(process.pid);
|
||||
} finally {
|
||||
try {
|
||||
process.kill(-child.pid!, "SIGKILL");
|
||||
} catch {
|
||||
// The target process group was already terminated.
|
||||
}
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
unrelatedListener.close((error) => error ? reject(error) : resolve());
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it("recognizes a pnpm command after the launcher becomes pnpm.cjs", () => {
|
||||
expect(doesLocalServiceCommandLineMatch({
|
||||
commandLine: "/usr/bin/node /opt/pnpm/pnpm.cjs dev -- --bind custom --bind-host 127.0.0.1",
|
||||
recordedCommand: "pnpm dev -- --bind custom --bind-host 127.0.0.1",
|
||||
serviceName: "paperclip-dev",
|
||||
})).toBe(true);
|
||||
});
|
||||
|
||||
it("does not accept a different command merely because it uses node", () => {
|
||||
expect(doesLocalServiceCommandLineMatch({
|
||||
commandLine: "/usr/bin/node /workspace/server/dist/index.js",
|
||||
recordedCommand: "pnpm dev -- --bind custom --bind-host 127.0.0.1",
|
||||
serviceName: "paperclip-dev",
|
||||
})).toBe(false);
|
||||
});
|
||||
|
||||
it("does not collapse different script paths to the same basename", () => {
|
||||
expect(doesLocalServiceCommandLineMatch({
|
||||
commandLine: "/usr/bin/node /other-workspace/server.js",
|
||||
recordedCommand: "node ./server.js",
|
||||
serviceName: "web",
|
||||
})).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -448,8 +448,6 @@ describe("resolveRuntimeProvisionCommand", () => {
|
||||
path.join(baseCwd, "scripts", "provision-worktree-runtime.sh"),
|
||||
"#!/usr/bin/env bash\n",
|
||||
);
|
||||
await fs.mkdir(path.join(cwd, ".paperclip"), { recursive: true });
|
||||
await fs.writeFile(path.join(cwd, ".paperclip", "seed-pending"), "{}\n");
|
||||
const workspace = {
|
||||
...buildWorkspace(cwd),
|
||||
baseCwd,
|
||||
@@ -457,6 +455,17 @@ describe("resolveRuntimeProvisionCommand", () => {
|
||||
worktreePath: cwd,
|
||||
};
|
||||
|
||||
expect(resolveRuntimeProvisionCommand({ config: {}, workspace })).toBe(
|
||||
"bash ./scripts/provision-worktree-runtime.sh",
|
||||
);
|
||||
|
||||
await fs.mkdir(path.join(cwd, ".paperclip"), { recursive: true });
|
||||
await fs.writeFile(path.join(cwd, ".paperclip", "config.json"), "{}\n");
|
||||
expect(resolveRuntimeProvisionCommand({ config: {}, workspace })).toBe(
|
||||
"bash ./scripts/provision-worktree-runtime.sh",
|
||||
);
|
||||
|
||||
await fs.writeFile(path.join(cwd, ".paperclip", "seed-pending"), "{}\n");
|
||||
expect(resolveRuntimeProvisionCommand({ config: {}, workspace })).toBe(
|
||||
"bash ./scripts/provision-worktree-runtime.sh",
|
||||
);
|
||||
@@ -466,7 +475,9 @@ describe("resolveRuntimeProvisionCommand", () => {
|
||||
})).toBe("./custom-provision.sh");
|
||||
|
||||
await fs.writeFile(path.join(cwd, ".paperclip", "seed-complete"), "{}\n");
|
||||
expect(resolveRuntimeProvisionCommand({ config: {}, workspace })).toBe("");
|
||||
expect(resolveRuntimeProvisionCommand({ config: {}, workspace })).toBe(
|
||||
"bash ./scripts/provision-worktree-runtime.sh",
|
||||
);
|
||||
|
||||
await fs.writeFile(
|
||||
path.join(cwd, ".paperclip", "seed-manifest.json"),
|
||||
@@ -772,6 +783,55 @@ describe("realizeExecutionWorkspace", () => {
|
||||
expect(second.branchName).toBe(first.branchName);
|
||||
});
|
||||
|
||||
it("defaults the repo-provided worktree provisioner for git worktree strategies", async () => {
|
||||
const repoRoot = await createTempRepo();
|
||||
await fs.mkdir(path.join(repoRoot, "scripts"), { recursive: true });
|
||||
await fs.writeFile(
|
||||
path.join(repoRoot, "scripts", "provision-worktree.sh"),
|
||||
[
|
||||
"#!/usr/bin/env bash",
|
||||
"set -euo pipefail",
|
||||
"mkdir -p .paperclip",
|
||||
"printf 'provisioned\\n' > .paperclip/default-provision-ran",
|
||||
"",
|
||||
].join("\n"),
|
||||
"utf8",
|
||||
);
|
||||
await runGit(repoRoot, ["add", "scripts/provision-worktree.sh"]);
|
||||
await runGit(repoRoot, ["commit", "-m", "Add repository worktree provisioner"]);
|
||||
|
||||
const workspace = await realizeExecutionWorkspace({
|
||||
base: {
|
||||
baseCwd: repoRoot,
|
||||
source: "project_primary",
|
||||
projectId: "project-1",
|
||||
workspaceId: "workspace-1",
|
||||
repoUrl: null,
|
||||
repoRef: "HEAD",
|
||||
},
|
||||
config: {
|
||||
workspaceStrategy: {
|
||||
type: "git_worktree",
|
||||
branchTemplate: "{{issue.identifier}}-{{slug}}",
|
||||
},
|
||||
},
|
||||
issue: {
|
||||
id: "issue-default-provision",
|
||||
identifier: "PAP-17684",
|
||||
title: "Default worktree provisioning",
|
||||
},
|
||||
agent: {
|
||||
id: "agent-1",
|
||||
name: "Codex Coder",
|
||||
companyId: "company-1",
|
||||
},
|
||||
});
|
||||
|
||||
await expect(
|
||||
fs.readFile(path.join(workspace.cwd, ".paperclip", "default-provision-ran"), "utf8"),
|
||||
).resolves.toBe("provisioned\n");
|
||||
});
|
||||
|
||||
it("warns when reusing a git worktree whose base ref has advanced", async () => {
|
||||
const repoRoot = await createTempRepo();
|
||||
|
||||
@@ -7230,7 +7290,7 @@ describeEmbeddedPostgres("workspace runtime startup reconciliation", () => {
|
||||
}
|
||||
}, 40_000);
|
||||
|
||||
it("adopts a live auto-port shared service after runtime state is reset", async () => {
|
||||
it("re-adopts a request-logging service on the same auto port after supervisor stdio closes", async () => {
|
||||
const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-runtime-reconcile-"));
|
||||
const paperclipHome = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-runtime-home-"));
|
||||
process.env.PAPERCLIP_HOME = paperclipHome;
|
||||
@@ -7239,7 +7299,6 @@ describeEmbeddedPostgres("workspace runtime startup reconciliation", () => {
|
||||
const companyId = randomUUID();
|
||||
const agentId = randomUUID();
|
||||
const runId = randomUUID();
|
||||
const executionWorkspaceId = randomUUID();
|
||||
|
||||
await db.insert(companies).values({
|
||||
id: companyId,
|
||||
@@ -7273,6 +7332,16 @@ describeEmbeddedPostgres("workspace runtime startup reconciliation", () => {
|
||||
projectId: null,
|
||||
workspaceId: null,
|
||||
};
|
||||
await fs.writeFile(
|
||||
path.join(workspaceRoot, "request-logger.cjs"),
|
||||
[
|
||||
"const http = require('node:http');",
|
||||
"http.createServer((req, res) => {",
|
||||
" process.stdout.write(`request ${req.url}\\n`, (error) => { if (!error) res.end('ok'); });",
|
||||
"}).listen(Number(process.env.PORT), '127.0.0.1');",
|
||||
].join("\n"),
|
||||
"utf8",
|
||||
);
|
||||
leasedRunIds.add(runId);
|
||||
|
||||
const services = await ensureRuntimeServicesForRun({
|
||||
@@ -7290,8 +7359,7 @@ describeEmbeddedPostgres("workspace runtime startup reconciliation", () => {
|
||||
services: [
|
||||
{
|
||||
name: "web",
|
||||
command:
|
||||
"node -e \"require('node:http').createServer((req,res)=>res.end('ok')).listen(Number(process.env.PORT), '127.0.0.1')\"",
|
||||
command: "node request-logger.cjs",
|
||||
port: { type: "auto" },
|
||||
readiness: {
|
||||
type: "http",
|
||||
@@ -7315,9 +7383,13 @@ describeEmbeddedPostgres("workspace runtime startup reconciliation", () => {
|
||||
const service = services[0];
|
||||
expect(service?.url).toMatch(/^http:\/\/127\.0\.0\.1:\d+$/);
|
||||
await expect(fetch(service!.url!)).resolves.toMatchObject({ ok: true });
|
||||
const originalPort = service!.port;
|
||||
const originalProviderRef = service!.providerRef;
|
||||
|
||||
await fs.rm(paperclipHome, { recursive: true, force: true });
|
||||
await resetRuntimeServicesForTests();
|
||||
// Closing the parent's pipe endpoints reproduces a real control-plane exit.
|
||||
// A service whose per-request logger still targets those pipes wedges (or
|
||||
// exits) on the reconciliation health probe instead of answering it.
|
||||
await resetRuntimeServicesForTests({ simulateSupervisorExit: true });
|
||||
|
||||
const result = await reconcilePersistedRuntimeServicesOnStartup(db);
|
||||
expect(result).toMatchObject({ reconciled: 1, adopted: 1, stopped: 0 });
|
||||
@@ -7328,17 +7400,221 @@ describeEmbeddedPostgres("workspace runtime startup reconciliation", () => {
|
||||
.where(eq(workspaceRuntimeServices.id, service!.id))
|
||||
.then((rows) => rows[0] ?? null);
|
||||
expect(persisted?.status).toBe("running");
|
||||
expect(persisted?.providerRef).toMatch(/^\d+$/);
|
||||
expect(persisted?.port).toBe(originalPort);
|
||||
expect(persisted?.providerRef).toBe(originalProviderRef);
|
||||
await expect(fetch(service!.url!)).resolves.toMatchObject({ ok: true });
|
||||
|
||||
await stopRuntimeServicesForExecutionWorkspace({
|
||||
db,
|
||||
executionWorkspaceId,
|
||||
workspaceCwd: workspace.cwd,
|
||||
});
|
||||
await resetRuntimeServicesForTests({ terminateProcesses: true });
|
||||
leasedRunIds.delete(runId);
|
||||
await fs.rm(paperclipHome, { recursive: true, force: true });
|
||||
await fs.rm(workspaceRoot, { recursive: true, force: true });
|
||||
|
||||
await expect(fetch(service!.url!)).rejects.toThrow();
|
||||
});
|
||||
|
||||
it("re-adopts a desired service when pnpm is represented as the pnpm.cjs launcher", async () => {
|
||||
const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-runtime-pnpm-reconcile-"));
|
||||
const paperclipHome = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-runtime-home-"));
|
||||
const previousPaperclipHome = process.env.PAPERCLIP_HOME;
|
||||
const previousInstanceId = process.env.PAPERCLIP_INSTANCE_ID;
|
||||
process.env.PAPERCLIP_HOME = paperclipHome;
|
||||
process.env.PAPERCLIP_INSTANCE_ID = `runtime-pnpm-reconcile-${randomUUID()}`;
|
||||
|
||||
const portProbe = net.createServer();
|
||||
await new Promise<void>((resolve) => portProbe.listen(0, "127.0.0.1", resolve));
|
||||
const address = portProbe.address();
|
||||
const port = typeof address === "object" && address ? address.port : null;
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
portProbe.close((error) => error ? reject(error) : resolve());
|
||||
});
|
||||
if (!port) throw new Error("Failed to reserve pnpm reconciliation test port");
|
||||
|
||||
const companyId = randomUUID();
|
||||
const projectId = randomUUID();
|
||||
const executionWorkspaceId = randomUUID();
|
||||
const runtimeServiceId = randomUUID();
|
||||
const command = "pnpm dev";
|
||||
const service = {
|
||||
name: "web",
|
||||
command,
|
||||
port,
|
||||
readiness: {
|
||||
type: "http",
|
||||
urlTemplate: "http://127.0.0.1:{{port}}",
|
||||
timeoutSec: 10,
|
||||
intervalMs: 50,
|
||||
},
|
||||
expose: null,
|
||||
lifecycle: "shared",
|
||||
reuseScope: "execution_workspace",
|
||||
stopPolicy: { type: "manual" },
|
||||
};
|
||||
const reuseKey = createHash("sha256")
|
||||
.update(stableStringifyForTest({
|
||||
scopeType: "execution_workspace",
|
||||
scopeId: executionWorkspaceId,
|
||||
serviceName: service.name,
|
||||
command,
|
||||
cwd: workspaceRoot,
|
||||
port,
|
||||
env: {},
|
||||
expose: null,
|
||||
}))
|
||||
.digest("hex");
|
||||
const wrapperPath = path.join(workspaceRoot, "pnpm.cjs");
|
||||
await fs.writeFile(
|
||||
wrapperPath,
|
||||
[
|
||||
"const http = require('node:http');",
|
||||
"const port = Number(process.argv[3]);",
|
||||
"http.createServer((_req, res) => res.end('ok')).listen(port, '127.0.0.1');",
|
||||
].join("\n"),
|
||||
"utf8",
|
||||
);
|
||||
const child = spawn(process.execPath, [wrapperPath, "dev", String(port)], {
|
||||
cwd: workspaceRoot,
|
||||
detached: process.platform !== "win32",
|
||||
stdio: "ignore",
|
||||
});
|
||||
child.unref();
|
||||
|
||||
try {
|
||||
for (let attempt = 0; attempt < 50; attempt += 1) {
|
||||
try {
|
||||
if ((await fetch(`http://127.0.0.1:${port}`)).ok) break;
|
||||
} catch {
|
||||
// Wait for the simulated package-manager launcher to bind.
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
}
|
||||
await expect(fetch(`http://127.0.0.1:${port}`)).resolves.toMatchObject({ ok: true });
|
||||
|
||||
await db.insert(companies).values({
|
||||
id: companyId,
|
||||
name: "Paperclip",
|
||||
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
|
||||
requireBoardApprovalForNewAgents: false,
|
||||
});
|
||||
await db.insert(projects).values({
|
||||
id: projectId,
|
||||
companyId,
|
||||
name: "Runtime pnpm reconciliation",
|
||||
status: "in_progress",
|
||||
});
|
||||
await db.insert(executionWorkspaces).values({
|
||||
id: executionWorkspaceId,
|
||||
companyId,
|
||||
projectId,
|
||||
projectWorkspaceId: null,
|
||||
mode: "isolated_workspace",
|
||||
strategyType: "git_worktree",
|
||||
name: "Runtime pnpm reconciliation workspace",
|
||||
status: "active",
|
||||
cwd: workspaceRoot,
|
||||
providerType: "local_fs",
|
||||
providerRef: workspaceRoot,
|
||||
metadata: {
|
||||
config: {
|
||||
workspaceRuntime: { services: [service] },
|
||||
desiredState: "running",
|
||||
serviceStates: { "0": "running" },
|
||||
},
|
||||
},
|
||||
});
|
||||
await db.insert(workspaceRuntimeServices).values({
|
||||
id: runtimeServiceId,
|
||||
companyId,
|
||||
projectId,
|
||||
projectWorkspaceId: null,
|
||||
executionWorkspaceId,
|
||||
issueId: null,
|
||||
scopeType: "execution_workspace",
|
||||
scopeId: executionWorkspaceId,
|
||||
serviceName: service.name,
|
||||
status: "running",
|
||||
lifecycle: "shared",
|
||||
reuseKey,
|
||||
command,
|
||||
cwd: workspaceRoot,
|
||||
port,
|
||||
url: `http://127.0.0.1:${port}`,
|
||||
provider: "local_process",
|
||||
providerRef: String(child.pid),
|
||||
ownerAgentId: null,
|
||||
startedByRunId: null,
|
||||
lastUsedAt: new Date(),
|
||||
startedAt: new Date(),
|
||||
stoppedAt: null,
|
||||
stopPolicy: { type: "manual" },
|
||||
healthStatus: "healthy",
|
||||
});
|
||||
await writeLocalServiceRegistryRecord({
|
||||
version: 1,
|
||||
serviceKey: `workspace-runtime-web-${randomUUID()}`,
|
||||
profileKind: "workspace-runtime",
|
||||
serviceName: service.name,
|
||||
command,
|
||||
cwd: workspaceRoot,
|
||||
envFingerprint: reuseKey,
|
||||
port,
|
||||
url: `http://127.0.0.1:${port}`,
|
||||
pid: child.pid!,
|
||||
processGroupId: child.pid ?? null,
|
||||
provider: "local_process",
|
||||
runtimeServiceId,
|
||||
reuseKey,
|
||||
startedAt: new Date().toISOString(),
|
||||
lastSeenAt: new Date().toISOString(),
|
||||
metadata: { executionWorkspaceId },
|
||||
});
|
||||
|
||||
const result = await reconcilePersistedRuntimeServicesOnStartup(db);
|
||||
|
||||
expect(result).toMatchObject({
|
||||
reconciled: 1,
|
||||
adopted: 1,
|
||||
stopped: 0,
|
||||
restarted: 0,
|
||||
restartFailed: 0,
|
||||
});
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(workspaceRuntimeServices)
|
||||
.where(eq(workspaceRuntimeServices.executionWorkspaceId, executionWorkspaceId));
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0]).toMatchObject({
|
||||
id: runtimeServiceId,
|
||||
status: "running",
|
||||
healthStatus: "healthy",
|
||||
port,
|
||||
providerRef: String(child.pid),
|
||||
});
|
||||
expect(await readLocalServicePortOwner(port)).toBe(child.pid);
|
||||
await expect(fetch(`http://127.0.0.1:${port}`)).resolves.toMatchObject({ ok: true });
|
||||
} finally {
|
||||
await stopRuntimeServicesForExecutionWorkspace({
|
||||
db,
|
||||
executionWorkspaceId,
|
||||
workspaceCwd: workspaceRoot,
|
||||
}).catch(() => undefined);
|
||||
if (child.pid) {
|
||||
try {
|
||||
if (process.platform === "win32") process.kill(child.pid, "SIGKILL");
|
||||
else process.kill(-child.pid, "SIGKILL");
|
||||
} catch {
|
||||
// The adopted runtime was already stopped through the managed path.
|
||||
}
|
||||
}
|
||||
await resetRuntimeServicesForTests();
|
||||
if (previousPaperclipHome === undefined) delete process.env.PAPERCLIP_HOME;
|
||||
else process.env.PAPERCLIP_HOME = previousPaperclipHome;
|
||||
if (previousInstanceId === undefined) delete process.env.PAPERCLIP_INSTANCE_ID;
|
||||
else process.env.PAPERCLIP_INSTANCE_ID = previousInstanceId;
|
||||
await fs.rm(paperclipHome, { recursive: true, force: true });
|
||||
await fs.rm(workspaceRoot, { recursive: true, force: true });
|
||||
}
|
||||
}, 20_000);
|
||||
|
||||
it("does not reuse a stopped auto-port service port while another process owns it", async () => {
|
||||
const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-runtime-unhealthy-adopt-"));
|
||||
const paperclipHome = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-runtime-home-"));
|
||||
|
||||
@@ -444,13 +444,27 @@ describe("worktree config repair", () => {
|
||||
const firstConfigPath = path.join(firstWorktreeRoot, ".paperclip", "config.json");
|
||||
const secondConfigPath = path.join(secondWorktreeRoot, ".paperclip", "config.json");
|
||||
|
||||
const writeWorktree = async (worktreeRoot: string, name: string) => {
|
||||
const writeWorktree = async (
|
||||
worktreeRoot: string,
|
||||
name: string,
|
||||
databaseMode: "embedded-postgres" | "postgres" = "embedded-postgres",
|
||||
) => {
|
||||
const paperclipDir = path.join(worktreeRoot, ".paperclip");
|
||||
const instanceRoot = path.join(isolatedHome, "instances", name.toLowerCase());
|
||||
const config = buildIsolatedConfig(instanceRoot, 45439, 55439);
|
||||
await fs.mkdir(paperclipDir, { recursive: true });
|
||||
await fs.writeFile(
|
||||
path.join(paperclipDir, "config.json"),
|
||||
`${JSON.stringify(buildIsolatedConfig(instanceRoot, 45439, 55439), null, 2)}\n`,
|
||||
`${JSON.stringify({
|
||||
...config,
|
||||
database: {
|
||||
...config.database,
|
||||
mode: databaseMode,
|
||||
...(databaseMode === "postgres"
|
||||
? { connectionString: "postgres://paperclip:paperclip@127.0.0.1:55439/paperclip" }
|
||||
: {}),
|
||||
},
|
||||
}, null, 2)}\n`,
|
||||
"utf8",
|
||||
);
|
||||
await fs.writeFile(
|
||||
@@ -480,7 +494,7 @@ describe("worktree config repair", () => {
|
||||
delete process.env.DATABASE_URL;
|
||||
};
|
||||
|
||||
await writeWorktree(firstWorktreeRoot, "PAP-14013-import-bulk-skills");
|
||||
await writeWorktree(firstWorktreeRoot, "PAP-14013-import-bulk-skills", "postgres");
|
||||
await writeWorktree(secondWorktreeRoot, "PAP-14069-port-conflicts");
|
||||
const staleLockPath = path.join(isolatedHome, ".worktree-port-reservations.lock");
|
||||
await fs.mkdir(staleLockPath, { recursive: true });
|
||||
|
||||
@@ -0,0 +1,229 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { promisify } from "node:util";
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import {
|
||||
ensureWorktreeSeeded,
|
||||
readWorktreeSeedManifest,
|
||||
} from "../../../cli/src/commands/worktree.ts";
|
||||
import { realizeExecutionWorkspace } from "../services/workspace-runtime.ts";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
const cleanup: string[] = [];
|
||||
const originalConfig = process.env.PAPERCLIP_CONFIG;
|
||||
const originalWorktreesDir = process.env.PAPERCLIP_WORKTREES_DIR;
|
||||
|
||||
async function runGit(cwd: string, args: string[]) {
|
||||
await execFileAsync("git", args, { cwd });
|
||||
}
|
||||
|
||||
async function writeConfig(configPath: string, instanceId: string) {
|
||||
const instanceRoot = path.dirname(configPath);
|
||||
await fs.mkdir(path.dirname(configPath), { recursive: true });
|
||||
await fs.writeFile(configPath, `${JSON.stringify({
|
||||
$meta: { version: 1, updatedAt: "2026-08-19T00:00:00.000Z", source: "configure" },
|
||||
database: {
|
||||
mode: "embedded-postgres",
|
||||
embeddedPostgresDataDir: path.join(instanceRoot, "db"),
|
||||
embeddedPostgresPort: 54329,
|
||||
backup: { enabled: false, intervalMinutes: 60, retentionDays: 30, dir: path.join(instanceRoot, "backups") },
|
||||
},
|
||||
logging: { mode: "file", logDir: path.join(instanceRoot, "logs") },
|
||||
server: {
|
||||
deploymentMode: "local_trusted",
|
||||
exposure: "private",
|
||||
host: "127.0.0.1",
|
||||
port: 3100,
|
||||
allowedHostnames: [],
|
||||
serveUi: true,
|
||||
},
|
||||
}, null, 2)}\n`, "utf8");
|
||||
await fs.writeFile(
|
||||
path.join(path.dirname(configPath), ".env"),
|
||||
`PAPERCLIP_INSTANCE_ID=${instanceId}\n`,
|
||||
"utf8",
|
||||
);
|
||||
}
|
||||
|
||||
function verifiedSeedResult() {
|
||||
return {
|
||||
backupSummary: "snapshot.sql",
|
||||
snapshotAt: "2026-08-19T00:00:00.000Z",
|
||||
migrationRevision: "0223_test.sql",
|
||||
pausedScheduledRoutines: 0,
|
||||
executionQuarantine: {
|
||||
disabledTimerHeartbeats: 0,
|
||||
resetRunningAgents: 0,
|
||||
quarantinedInProgressIssues: 0,
|
||||
unassignedTodoIssues: 0,
|
||||
unassignedReviewIssues: 0,
|
||||
stoppedProjectWorkspaceRuntimes: 0,
|
||||
stoppedExecutionWorkspaceRuntimes: 0,
|
||||
stoppedRuntimeServices: 0,
|
||||
},
|
||||
reboundWorkspaces: [],
|
||||
validation: {
|
||||
authUserCount: 1,
|
||||
credentialAccountCount: 1,
|
||||
instanceAdminCount: 1,
|
||||
activeMembershipCount: 1,
|
||||
companyCount: 1,
|
||||
issueCount: 1,
|
||||
representativeCompanyId: "00000000-0000-4000-8000-000000000001",
|
||||
representativeIssueId: "00000000-0000-4000-8000-000000000002",
|
||||
migrationRevision: "0223_test.sql",
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
if (originalConfig === undefined) delete process.env.PAPERCLIP_CONFIG;
|
||||
else process.env.PAPERCLIP_CONFIG = originalConfig;
|
||||
if (originalWorktreesDir === undefined) delete process.env.PAPERCLIP_WORKTREES_DIR;
|
||||
else process.env.PAPERCLIP_WORKTREES_DIR = originalWorktreesDir;
|
||||
for (const dir of cleanup.splice(0)) {
|
||||
await fs.rm(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
describe("managed worktree seed source through the server spawn path", () => {
|
||||
it("re-derives an ambient-instance manifest written before provisioning", async () => {
|
||||
const tempRoot = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-server-seed-source-"));
|
||||
cleanup.push(tempRoot);
|
||||
const repoRoot = path.join(tempRoot, "repo");
|
||||
const hooksDir = path.join(tempRoot, "hooks");
|
||||
const ambientConfigPath = path.join(tempRoot, "ambient", "config.json");
|
||||
const registeredConfigPath = path.join(repoRoot, ".paperclip", "config.json");
|
||||
const worktreeHome = path.join(tempRoot, "worktree-home");
|
||||
|
||||
await fs.mkdir(repoRoot, { recursive: true });
|
||||
await runGit(repoRoot, ["init", "-q"]);
|
||||
await runGit(repoRoot, ["config", "user.email", "paperclip@example.com"]);
|
||||
await runGit(repoRoot, ["config", "user.name", "Paperclip Test"]);
|
||||
await fs.mkdir(path.join(repoRoot, "scripts"), { recursive: true });
|
||||
await fs.writeFile(path.join(repoRoot, "README.md"), "server spawn regression\n", "utf8");
|
||||
await fs.copyFile(
|
||||
fileURLToPath(new URL("../../../scripts/provision-worktree.sh", import.meta.url)),
|
||||
path.join(repoRoot, "scripts", "provision-worktree.sh"),
|
||||
);
|
||||
await fs.chmod(path.join(repoRoot, "scripts", "provision-worktree.sh"), 0o755);
|
||||
await runGit(repoRoot, ["add", "README.md", "scripts/provision-worktree.sh"]);
|
||||
await runGit(repoRoot, ["commit", "-qm", "Add managed provision script"]);
|
||||
await writeConfig(registeredConfigPath, "registered-source");
|
||||
await writeConfig(ambientConfigPath, "ambient-instance");
|
||||
await fs.mkdir(worktreeHome, { recursive: true });
|
||||
|
||||
// Reproduce the integrated lane's ordering with a checkout hook: an earlier
|
||||
// worktree-init-style writer inherits the server environment and leaves a valid
|
||||
// target config whose manifest diagnostic points at the ambient instance.
|
||||
await fs.mkdir(hooksDir, { recursive: true });
|
||||
const hookPath = path.join(hooksDir, "post-checkout");
|
||||
await fs.writeFile(
|
||||
hookPath,
|
||||
`#!/usr/bin/env node
|
||||
const crypto = require("node:crypto");
|
||||
const fs = require("node:fs");
|
||||
const path = require("node:path");
|
||||
const cwd = process.cwd();
|
||||
if (cwd === ${JSON.stringify(repoRoot)}) process.exit(0);
|
||||
const stateDir = path.join(cwd, ".paperclip");
|
||||
const normalized = path.basename(cwd).trim().toLowerCase().replace(/[^a-z0-9_-]+/g, "-").replace(/-+/g, "-").replace(/^[-_]+|[-_]+$/g, "");
|
||||
const instanceId = \`${"${(normalized || \"worktree\").slice(0, 48)}"}-\${crypto.createHash("sha256").update(path.resolve(cwd)).digest("hex").slice(0, 12)}\`;
|
||||
const targetConfigPath = path.join(stateDir, "config.json");
|
||||
const instanceRoot = path.join(process.env.PAPERCLIP_WORKTREES_DIR, "instances", instanceId);
|
||||
fs.mkdirSync(stateDir, { recursive: true });
|
||||
fs.mkdirSync(instanceRoot, { recursive: true });
|
||||
fs.writeFileSync(targetConfigPath, JSON.stringify({
|
||||
$meta: { version: 1, updatedAt: "2026-08-19T00:00:00.000Z", source: "configure" },
|
||||
database: {
|
||||
mode: "embedded-postgres",
|
||||
embeddedPostgresDataDir: path.join(instanceRoot, "db"),
|
||||
embeddedPostgresPort: 54330,
|
||||
backup: { enabled: false, intervalMinutes: 60, retentionDays: 30, dir: path.join(instanceRoot, "backups") },
|
||||
},
|
||||
logging: { mode: "file", logDir: path.join(instanceRoot, "logs") },
|
||||
server: {
|
||||
deploymentMode: "local_trusted",
|
||||
exposure: "private",
|
||||
host: "127.0.0.1",
|
||||
port: 3101,
|
||||
allowedHostnames: [],
|
||||
serveUi: true,
|
||||
},
|
||||
}, null, 2) + "\\n");
|
||||
fs.writeFileSync(path.join(stateDir, ".env"), [
|
||||
"PAPERCLIP_HOME=" + JSON.stringify(process.env.PAPERCLIP_WORKTREES_DIR),
|
||||
"PAPERCLIP_INSTANCE_ID=" + JSON.stringify(instanceId),
|
||||
"PAPERCLIP_CONFIG=" + JSON.stringify(targetConfigPath),
|
||||
"",
|
||||
].join("\\n"));
|
||||
fs.writeFileSync(path.join(stateDir, "seed-manifest.json"), JSON.stringify({
|
||||
version: 2,
|
||||
source: { instanceId: "ambient-instance", configPath: process.env.PAPERCLIP_CONFIG },
|
||||
snapshotAt: null,
|
||||
seedMode: "minimal",
|
||||
migrationRevision: null,
|
||||
targetInstanceId: instanceId,
|
||||
phase: "pending",
|
||||
state: "pending",
|
||||
attemptId: "ambient-writer",
|
||||
startedAt: null,
|
||||
finishedAt: null,
|
||||
diagnostics: [{ phase: "pending", status: "succeeded", at: new Date().toISOString() }],
|
||||
}, null, 2) + "\\n");
|
||||
`,
|
||||
"utf8",
|
||||
);
|
||||
await fs.chmod(hookPath, 0o755);
|
||||
await runGit(repoRoot, ["config", "core.hooksPath", hooksDir]);
|
||||
|
||||
process.env.PAPERCLIP_CONFIG = ambientConfigPath;
|
||||
process.env.PAPERCLIP_WORKTREES_DIR = worktreeHome;
|
||||
const workspace = await realizeExecutionWorkspace({
|
||||
base: {
|
||||
baseCwd: repoRoot,
|
||||
source: "project_primary",
|
||||
projectId: "project-1",
|
||||
workspaceId: "project-workspace-1",
|
||||
repoUrl: null,
|
||||
repoRef: "HEAD",
|
||||
},
|
||||
config: {
|
||||
workspaceStrategy: {
|
||||
type: "git_worktree",
|
||||
branchTemplate: "{{issue.identifier}}-{{slug}}",
|
||||
provisionCommand: "bash ./scripts/provision-worktree.sh",
|
||||
},
|
||||
},
|
||||
issue: {
|
||||
id: "issue-1",
|
||||
identifier: "PAP-17681",
|
||||
title: "Recover ambient seed source",
|
||||
},
|
||||
agent: { id: "agent-1", name: "Coder", companyId: "company-1" },
|
||||
});
|
||||
|
||||
expect(readWorktreeSeedManifest(path.join(workspace.cwd, ".paperclip", "config.json"))).toMatchObject({
|
||||
source: { instanceId: "ambient-instance", configPath: ambientConfigPath },
|
||||
state: "pending",
|
||||
});
|
||||
|
||||
await expect(ensureWorktreeSeeded({
|
||||
config: path.join(workspace.cwd, ".paperclip", "config.json"),
|
||||
registeredBaseWorkspaceCwd: repoRoot,
|
||||
registeredProjectWorkspaceId: "project-workspace-1",
|
||||
expectedCompanyId: "company-1",
|
||||
}, {
|
||||
seedDatabase: async () => verifiedSeedResult(),
|
||||
})).resolves.toMatchObject({ seeded: true, reason: "seeded" });
|
||||
|
||||
expect(readWorktreeSeedManifest(path.join(workspace.cwd, ".paperclip", "config.json"))).toMatchObject({
|
||||
source: { instanceId: "registered-source", configPath: registeredConfigPath },
|
||||
state: "verified",
|
||||
phase: "complete",
|
||||
});
|
||||
}, 20_000);
|
||||
});
|
||||
Reference in new issue
Block a user