mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-25 09:41:04 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Execution workspaces need isolated databases, ports, and runtime services > - Concurrent workspaces could reuse ports or lose service ownership after a restart > - A markerless worktree also needed seed recovery, but normal markerless instances still needed to boot > - This pull request makes seed, port, and service ownership state explicit and recoverable > - It also checks live process and listener identity before it reclaims shared resources > - The benefit is reliable workspace startup, restart, adoption, and concurrent provisioning ## Linked Issues or Issue Description **What happened?** Managed workspaces could lose runtime service ownership after a control-plane restart. Concurrent worktrees could also reuse a port when their parent paths differed. A seed recovery change made every markerless instance resolve a worktree seed source, so normal instances without a source could not start. **Expected behavior** Paperclip must preserve healthy managed services across restarts. It must reserve unique ports across worktree parents. It must provision a registered markerless worktree, but it must skip seed work for a normal markerless instance. **Steps to reproduce** 1. Start two managed worktrees under different parent paths at the same time. 2. Restart the control plane while a managed service stays alive. 3. Start Paperclip with a config that has no seed markers and no registered worktree source. 4. Observe duplicate port selection, lost service adoption, or a seed-source startup error. **Paperclip version or commit** Current `master` plus the workspace runtime reliability changes in this pull request. **Deployment mode** Local development with managed execution workspaces and embedded Postgres. ## What Changed - Added a shared port registry with lease heartbeats, process identity checks, and live listener probes. - Reserved worktree ports across custom parent paths and repaired duplicate legacy assignments. - Preserved and adopted healthy managed services across control-plane restarts. - Reconciled guest bind modes and verified listener ownership before termination or reuse. - Provisioned registered markerless worktree databases and kept normal markerless instance startup as a no-op. - Added CLI, shared, server, and shell regression tests for seed, port, listener, restart, and adoption behavior. - Updated the worktree development documentation. ## Verification - `pnpm exec vitest run cli/src/__tests__/worktree.test.ts --reporter=verbose` — 63 tests passed. - `pnpm exec vitest run packages/shared/src/worktree-port-registry.test.ts --reporter=verbose` — 5 tests passed. - Focused runtime Vitest set — 199 tests passed across 37 suites. - `node --test scripts/__tests__/provision-worktree-self-heal.test.mjs` — 10 tests passed. - `git diff --check` passed. ## Risks - Port reservation now depends on lease and process identity data. The fallback listener probe prevents early reclamation when process metadata is incomplete. - Runtime adoption is stricter about bind and owner identity. The tests cover healthy adoption, stale records, PID reuse, and unrelated listeners. - Markerless seed detection now separates registered worktrees from normal instances. The tests cover both paths. - There are no database schema migrations. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with the `gpt-5` model family. The serving snapshot and context-window size are not exposed. The agent used reasoning, repository tools, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Dev Agent <dev@paperclip.ing>
148 lines
4.9 KiB
TypeScript
148 lines
4.9 KiB
TypeScript
import fs from "node:fs";
|
|
import os from "node:os";
|
|
import path from "node:path";
|
|
import { setTimeout as delay } from "node:timers/promises";
|
|
import { afterEach, describe, expect, it } from "vitest";
|
|
import {
|
|
withWorktreePortRegistryLock,
|
|
withWorktreePortRegistryLockSync,
|
|
} from "./worktree-port-registry.js";
|
|
|
|
const temporaryRoots: string[] = [];
|
|
|
|
function makeTemporaryRoot(): string {
|
|
const root = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-port-registry-lock-"));
|
|
temporaryRoots.push(root);
|
|
return root;
|
|
}
|
|
|
|
function deferred(): { promise: Promise<void>; resolve: () => void } {
|
|
let resolve!: () => void;
|
|
const promise = new Promise<void>((done) => {
|
|
resolve = done;
|
|
});
|
|
return { promise, resolve };
|
|
}
|
|
|
|
afterEach(() => {
|
|
for (const root of temporaryRoots.splice(0)) {
|
|
fs.rmSync(root, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
describe("worktree port registry lock", () => {
|
|
it("does not reclaim a stale lock while its fallback ownership probe responds", async () => {
|
|
const homeDir = makeTemporaryRoot();
|
|
const lockPath = path.join(homeDir, ".worktree-port-reservations.lock");
|
|
const firstEntered = deferred();
|
|
const releaseFirst = deferred();
|
|
let secondEntered = false;
|
|
|
|
const first = withWorktreePortRegistryLock(homeDir, async () => {
|
|
fs.renameSync(path.join(lockPath, "owner.json"), path.join(lockPath, "owner.unavailable.json"));
|
|
const backupOwnerPath = path.join(lockPath, "owner.backup.json");
|
|
const owner = JSON.parse(fs.readFileSync(backupOwnerPath, "utf8"));
|
|
fs.writeFileSync(backupOwnerPath, `${JSON.stringify({
|
|
...owner,
|
|
processIdentity: "unavailable-process-identity",
|
|
})}\n`);
|
|
const oldTimestamp = new Date(Date.now() - 10_000);
|
|
fs.utimesSync(lockPath, oldTimestamp, oldTimestamp);
|
|
firstEntered.resolve();
|
|
await releaseFirst.promise;
|
|
});
|
|
await firstEntered.promise;
|
|
|
|
expect(Date.now() - fs.statSync(lockPath).mtimeMs).toBeGreaterThan(5_000);
|
|
|
|
const second = withWorktreePortRegistryLock(homeDir, async () => {
|
|
secondEntered = true;
|
|
});
|
|
await delay(100);
|
|
|
|
expect(secondEntered).toBe(false);
|
|
releaseFirst.resolve();
|
|
await Promise.all([first, second]);
|
|
expect(secondEntered).toBe(true);
|
|
}, 10_000);
|
|
|
|
it("refreshes the lease throughout an async critical section", async () => {
|
|
const homeDir = makeTemporaryRoot();
|
|
const lockPath = path.join(homeDir, ".worktree-port-reservations.lock");
|
|
|
|
await withWorktreePortRegistryLock(homeDir, async () => {
|
|
await delay(5_250);
|
|
expect(Date.now() - fs.statSync(lockPath).mtimeMs).toBeLessThan(2_000);
|
|
});
|
|
|
|
expect(fs.existsSync(lockPath)).toBe(false);
|
|
}, 10_000);
|
|
|
|
it("reclaims an old lock after its owner process exits", async () => {
|
|
const homeDir = makeTemporaryRoot();
|
|
const lockPath = path.join(homeDir, ".worktree-port-reservations.lock");
|
|
fs.mkdirSync(lockPath);
|
|
fs.writeFileSync(
|
|
path.join(lockPath, "owner.json"),
|
|
`${JSON.stringify({
|
|
version: 1,
|
|
pid: 2_147_483_647,
|
|
processIdentity: "dead-process",
|
|
probePort: 1,
|
|
token: "dead-owner",
|
|
})}\n`,
|
|
);
|
|
const oldTimestamp = new Date(Date.now() - 10_000);
|
|
fs.utimesSync(lockPath, oldTimestamp, oldTimestamp);
|
|
|
|
let entered = false;
|
|
await withWorktreePortRegistryLock(homeDir, async () => {
|
|
entered = true;
|
|
});
|
|
|
|
expect(entered).toBe(true);
|
|
expect(fs.existsSync(lockPath)).toBe(false);
|
|
});
|
|
|
|
it("reclaims an old lock when its pid belongs to a different process", async () => {
|
|
const homeDir = makeTemporaryRoot();
|
|
const lockPath = path.join(homeDir, ".worktree-port-reservations.lock");
|
|
fs.mkdirSync(lockPath);
|
|
fs.writeFileSync(
|
|
path.join(lockPath, "owner.json"),
|
|
`${JSON.stringify({
|
|
version: 1,
|
|
pid: process.pid,
|
|
processIdentity: "reused-pid-owner",
|
|
probePort: 1,
|
|
token: "abandoned-owner",
|
|
})}\n`,
|
|
);
|
|
const oldTimestamp = new Date(Date.now() - 10_000);
|
|
fs.utimesSync(lockPath, oldTimestamp, oldTimestamp);
|
|
|
|
let entered = false;
|
|
await withWorktreePortRegistryLock(homeDir, async () => {
|
|
entered = true;
|
|
});
|
|
|
|
expect(entered).toBe(true);
|
|
expect(fs.existsSync(lockPath)).toBe(false);
|
|
});
|
|
|
|
it("refreshes the lease while a synchronous critical section blocks the main thread", () => {
|
|
const homeDir = makeTemporaryRoot();
|
|
const lockPath = path.join(homeDir, ".worktree-port-reservations.lock");
|
|
const blocker = new Int32Array(new SharedArrayBuffer(4));
|
|
|
|
withWorktreePortRegistryLockSync(homeDir, () => {
|
|
const oldTimestamp = new Date(Date.now() - 10_000);
|
|
fs.utimesSync(lockPath, oldTimestamp, oldTimestamp);
|
|
Atomics.wait(blocker, 0, 0, 1_500);
|
|
expect(Date.now() - fs.statSync(lockPath).mtimeMs).toBeLessThan(1_250);
|
|
});
|
|
|
|
expect(fs.existsSync(lockPath)).toBe(false);
|
|
});
|
|
});
|