Files
PaperClipAI/server/src/services/local-service-supervisor.ts
DottaandPaperclip e4e12bfb89 fix(workspaces): persist readiness state and validate ports (#9408)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Execution workspaces can run managed services that must report
reliable lifecycle and readiness state
> - Service startup previously waited for readiness before committing
the starting row, making concurrent control actions see stale state
> - Fixed service ports also needed clearer configuration and ownership
diagnostics to avoid cross-workspace collisions
> - This pull request persists startup state before readiness, validates
port ownership, and exposes configurable service ports in the workspace
UI
> - The benefit is dependable service controls and actionable
diagnostics when workspace runtimes start slowly or compete for ports

## Linked Issues or Issue Description

### What happened?

Slow-starting workspace services could remain invisible to concurrent
stop/restart controls until readiness completed, and fixed-port
conflicts lacked enough ownership context for safe repair.

### Expected behavior

A starting service is persisted immediately, control operations can
observe it, configured ports are editable, and conflicts identify the
owning process/workspace.

### Steps to reproduce

1. Configure a workspace service that delays binding its HTTP port.
2. Start the service and immediately request another control action.
3. Observe stale persisted state before this change.
4. Configure two workspaces for the same fixed port and observe limited
conflict diagnostics.

### Paperclip version or commit

`origin/master` at `02e2dd271`

### Deployment mode

Local dev; built from source; not adapter-specific; database-backed
workspace runtime state.

## What Changed

- Commit the `starting` runtime-service row before waiting for readiness
and transition it after the probe completes.
- Add port-owner inspection and cross-workspace conflict details to
local service supervision.
- Preserve configurable runtime service ports through workspace
configuration updates.
- Surface service-port editing and validation in the execution workspace
details UI.
- Add server and UI regression coverage for slow readiness, concurrent
controls, port persistence, and conflict diagnostics.

## Verification

- `vitest --project @paperclipai/server
src/__tests__/workspace-runtime.test.ts
src/__tests__/execution-workspaces-service.test.ts` — 118 tests passed.
- `vitest --project @paperclipai/ui
src/pages/ExecutionWorkspaceDetail.service-ports.test.ts` — 4 tests
passed.
- `node scripts/check-token-gates.mjs` — all token gates clean.

## Risks

- Moderate risk: changes touch workspace service lifecycle persistence
and local process/port inspection.
- No schema migration is required; tests exercise slow readiness,
concurrent control, persisted ports, and cross-workspace conflicts.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, GPT-5.3 Codex, reasoning with repository tool use and
code execution; context-window size was not exposed by the runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-11 03:02:21 -05:00

455 lines
15 KiB
TypeScript

import { execFile } from "node:child_process";
import { createHash } from "node:crypto";
import fs from "node:fs/promises";
import path from "node:path";
import { setTimeout as delay } from "node:timers/promises";
import { promisify } from "node:util";
import { resolvePaperclipInstanceRoot } from "../home-paths.js";
const execFileAsync = promisify(execFile);
export interface LocalServiceRegistryRecord {
version: 1;
serviceKey: string;
profileKind: string;
serviceName: string;
command: string;
cwd: string;
envFingerprint: string;
port: number | null;
url: string | null;
pid: number;
processGroupId: number | null;
provider: "local_process";
runtimeServiceId: string | null;
reuseKey: string | null;
startedAt: string;
lastSeenAt: string;
metadata: Record<string, unknown> | null;
}
export interface LocalServiceIdentityInput {
profileKind: string;
serviceName: string;
cwd: string;
command: string;
envFingerprint: string;
port: number | null;
scope: Record<string, unknown> | null;
}
function stableStringify(value: unknown): string {
if (Array.isArray(value)) {
return `[${value.map((entry) => stableStringify(entry)).join(",")}]`;
}
if (value && typeof value === "object") {
const rec = value as Record<string, unknown>;
return `{${Object.keys(rec).sort().map((key) => `${JSON.stringify(key)}:${stableStringify(rec[key])}`).join(",")}}`;
}
return JSON.stringify(value);
}
function sanitizeServiceKeySegment(value: string, fallback: string): string {
const normalized = value
.trim()
.toLowerCase()
.replace(/[^a-z0-9._-]+/g, "-")
.replace(/-+/g, "-")
.replace(/^-+|-+$/g, "");
return normalized || fallback;
}
function getRuntimeServicesDir() {
return path.resolve(resolvePaperclipInstanceRoot(), "runtime-services");
}
function getRuntimeServiceRegistryPath(serviceKey: string) {
return path.resolve(getRuntimeServicesDir(), `${serviceKey}.json`);
}
function normalizeRegistryRecord(raw: unknown): LocalServiceRegistryRecord | null {
if (!raw || typeof raw !== "object") return null;
const rec = raw as Record<string, unknown>;
if (
rec.version !== 1 ||
typeof rec.serviceKey !== "string" ||
typeof rec.profileKind !== "string" ||
typeof rec.serviceName !== "string" ||
typeof rec.command !== "string" ||
typeof rec.cwd !== "string" ||
typeof rec.envFingerprint !== "string" ||
typeof rec.pid !== "number"
) {
return null;
}
return {
version: 1,
serviceKey: rec.serviceKey,
profileKind: rec.profileKind,
serviceName: rec.serviceName,
command: rec.command,
cwd: rec.cwd,
envFingerprint: rec.envFingerprint,
port: typeof rec.port === "number" ? rec.port : null,
url: typeof rec.url === "string" ? rec.url : null,
pid: rec.pid,
processGroupId: typeof rec.processGroupId === "number" ? rec.processGroupId : null,
provider: "local_process",
runtimeServiceId: typeof rec.runtimeServiceId === "string" ? rec.runtimeServiceId : null,
reuseKey: typeof rec.reuseKey === "string" ? rec.reuseKey : null,
startedAt: typeof rec.startedAt === "string" ? rec.startedAt : new Date().toISOString(),
lastSeenAt: typeof rec.lastSeenAt === "string" ? rec.lastSeenAt : new Date().toISOString(),
metadata:
rec.metadata && typeof rec.metadata === "object" && !Array.isArray(rec.metadata)
? (rec.metadata as Record<string, unknown>)
: null,
};
}
async function safeReadRegistryRecord(filePath: string) {
try {
const raw = JSON.parse(await fs.readFile(filePath, "utf8")) as unknown;
return normalizeRegistryRecord(raw);
} catch {
return null;
}
}
export function createLocalServiceKey(input: LocalServiceIdentityInput) {
const digest = createHash("sha256")
.update(
stableStringify({
profileKind: input.profileKind,
serviceName: input.serviceName,
cwd: path.resolve(input.cwd),
command: input.command,
envFingerprint: input.envFingerprint,
port: input.port,
scope: input.scope ?? null,
}),
)
.digest("hex")
.slice(0, 24);
return `${sanitizeServiceKeySegment(input.profileKind, "service")}-${sanitizeServiceKeySegment(input.serviceName, "service")}-${digest}`;
}
export async function writeLocalServiceRegistryRecord(record: LocalServiceRegistryRecord) {
await fs.mkdir(getRuntimeServicesDir(), { recursive: true });
await fs.writeFile(
getRuntimeServiceRegistryPath(record.serviceKey),
`${JSON.stringify(record, null, 2)}\n`,
"utf8",
);
}
export async function removeLocalServiceRegistryRecord(serviceKey: string) {
await fs.rm(getRuntimeServiceRegistryPath(serviceKey), { force: true });
}
export async function readLocalServiceRegistryRecord(serviceKey: string) {
return await safeReadRegistryRecord(getRuntimeServiceRegistryPath(serviceKey));
}
export async function listLocalServiceRegistryRecords(filter?: {
profileKind?: string;
metadata?: Record<string, unknown>;
}) {
try {
const entries = await fs.readdir(getRuntimeServicesDir(), { withFileTypes: true });
const records = await Promise.all(
entries
.filter((entry) => entry.isFile() && entry.name.endsWith(".json"))
.map((entry) => safeReadRegistryRecord(path.resolve(getRuntimeServicesDir(), entry.name))),
);
return records
.filter((record): record is LocalServiceRegistryRecord => record !== null)
.filter((record) => {
if (filter?.profileKind && record.profileKind !== filter.profileKind) return false;
if (!filter?.metadata) return true;
return Object.entries(filter.metadata).every(([key, value]) => record.metadata?.[key] === value);
})
.sort((left, right) => left.serviceKey.localeCompare(right.serviceKey));
} catch {
return [];
}
}
export async function findLocalServiceRegistryRecordByRuntimeServiceId(input: {
runtimeServiceId: string;
profileKind?: string;
}) {
const records = await listLocalServiceRegistryRecords(
input.profileKind ? { profileKind: input.profileKind } : undefined,
);
const record = records.find((entry) => entry.runtimeServiceId === input.runtimeServiceId) ?? null;
if (!record) return null;
let candidate = record;
if (!isPidAlive(candidate.pid)) {
const ownerPid = candidate.port ? await readLocalServicePortOwner(candidate.port) : null;
if (!ownerPid) {
await removeLocalServiceRegistryRecord(candidate.serviceKey);
return null;
}
candidate = {
...candidate,
pid: ownerPid,
processGroupId: candidate.processGroupId && isPidAlive(candidate.processGroupId) ? candidate.processGroupId : ownerPid,
lastSeenAt: new Date().toISOString(),
};
await writeLocalServiceRegistryRecord(candidate);
}
if (!(await isLikelyMatchingCommand(candidate))) {
await removeLocalServiceRegistryRecord(record.serviceKey);
return null;
}
if (!(await doesLocalServiceRecordMatchCwd(candidate))) {
await removeLocalServiceRegistryRecord(record.serviceKey);
return null;
}
return candidate;
}
export function isPidAlive(pid: number) {
if (!Number.isInteger(pid) || pid <= 0) return false;
try {
process.kill(pid, 0);
return true;
} catch {
return false;
}
}
export function isProcessGroupAlive(processGroupId: number | null | undefined) {
if (process.platform === "win32") return false;
if (typeof processGroupId !== "number" || !Number.isInteger(processGroupId) || processGroupId <= 0) return false;
try {
process.kill(-processGroupId, 0);
return true;
} catch {
return false;
}
}
async function isLikelyMatchingCommand(record: LocalServiceRegistryRecord) {
if (process.platform === "win32") return true;
try {
const { stdout } = await execFileAsync("ps", ["-o", "command=", "-p", String(record.pid)]);
const commandLine = stdout.trim();
if (!commandLine) return false;
const normalize = (value: string) => value.replace(/["']/g, "").replace(/\s+/g, " ").trim();
const normalizedCommandLine = normalize(commandLine);
const normalizedRecordedCommand = normalize(record.command);
return normalizedCommandLine.includes(normalizedRecordedCommand) || normalizedCommandLine.includes(record.serviceName);
} catch {
return true;
}
}
export async function findAdoptableLocalService(input: {
serviceKey: string;
profileKind?: string | null;
serviceName?: string | null;
command?: string | null;
cwd?: string | null;
envFingerprint?: string | null;
port?: number | null;
url?: string | null;
}) {
const record =
await readLocalServiceRegistryRecord(input.serviceKey)
?? await adoptLocalServiceFromPortOwner(input);
if (!record) return null;
if (!isPidAlive(record.pid)) {
await removeLocalServiceRegistryRecord(input.serviceKey);
return null;
}
if (!(await isLikelyMatchingCommand(record))) {
await removeLocalServiceRegistryRecord(input.serviceKey);
return null;
}
if (!(await doesLocalServiceRecordMatchCwd(record))) {
await removeLocalServiceRegistryRecord(input.serviceKey);
return null;
}
if (input.command && record.command !== input.command) return null;
if (input.cwd && path.resolve(record.cwd) !== path.resolve(input.cwd)) return null;
if (input.envFingerprint && record.envFingerprint !== input.envFingerprint) return null;
if (input.port !== undefined && input.port !== null && record.port !== input.port) return null;
return record;
}
async function readProcessGroupId(pid: number) {
if (process.platform === "win32") return null;
try {
const { stdout } = await execFileAsync("ps", ["-o", "pgid=", "-p", String(pid)]);
const parsed = Number.parseInt(stdout.trim(), 10);
return Number.isInteger(parsed) && parsed > 0 ? parsed : null;
} catch {
return null;
}
}
async function adoptLocalServiceFromPortOwner(input: {
serviceKey: string;
profileKind?: string | null;
serviceName?: string | null;
command?: string | null;
cwd?: string | null;
envFingerprint?: string | null;
port?: number | null;
url?: string | null;
}) {
if (!input.port) return null;
const ownerPid = await readLocalServicePortOwner(input.port);
if (!ownerPid) return null;
if (input.cwd) {
const ownerCwd = await readLocalServiceProcessCwd(ownerPid);
if (!ownerCwd || !(await isLocalServiceProcessInWorkspace(ownerCwd, input.cwd))) {
return null;
}
}
const processGroupId = await readProcessGroupId(ownerPid);
const pid = processGroupId && isPidAlive(processGroupId) ? processGroupId : ownerPid;
const now = new Date().toISOString();
const record: LocalServiceRegistryRecord = {
version: 1,
serviceKey: input.serviceKey,
profileKind: input.profileKind ?? "workspace-runtime",
serviceName: input.serviceName ?? "service",
command: input.command ?? input.serviceName ?? "service",
cwd: input.cwd ?? process.cwd(),
envFingerprint: input.envFingerprint ?? "",
port: input.port,
url: input.url ?? null,
pid,
processGroupId: processGroupId ?? pid,
provider: "local_process",
runtimeServiceId: null,
reuseKey: input.envFingerprint ?? null,
startedAt: now,
lastSeenAt: now,
metadata: null,
};
if (!(await isLikelyMatchingCommand(record))) return null;
await writeLocalServiceRegistryRecord(record);
return record;
}
export async function touchLocalServiceRegistryRecord(
serviceKey: string,
patch?: Partial<Omit<LocalServiceRegistryRecord, "serviceKey" | "version">>,
) {
const existing = await readLocalServiceRegistryRecord(serviceKey);
if (!existing) return null;
const next: LocalServiceRegistryRecord = {
...existing,
...patch,
version: 1,
serviceKey,
lastSeenAt: patch?.lastSeenAt ?? new Date().toISOString(),
};
await writeLocalServiceRegistryRecord(next);
return next;
}
export async function terminateLocalService(
record: Pick<LocalServiceRegistryRecord, "pid" | "processGroupId">,
opts?: { signal?: NodeJS.Signals; forceAfterMs?: number },
) {
const signal = opts?.signal ?? "SIGTERM";
const targetProcessGroup = process.platform !== "win32" && record.processGroupId && record.processGroupId > 0;
try {
if (targetProcessGroup) {
process.kill(-record.processGroupId!, signal);
} else {
process.kill(record.pid, signal);
}
} catch {
return;
}
const deadline = Date.now() + (opts?.forceAfterMs ?? 2_000);
while (Date.now() < deadline) {
const targetAlive = targetProcessGroup
? isProcessGroupAlive(record.processGroupId)
: isPidAlive(record.pid);
if (!targetAlive) {
return;
}
await delay(100);
}
const stillAlive = targetProcessGroup
? isProcessGroupAlive(record.processGroupId)
: isPidAlive(record.pid);
if (!stillAlive) return;
try {
if (targetProcessGroup) {
process.kill(-record.processGroupId!, "SIGKILL");
} else {
process.kill(record.pid, "SIGKILL");
}
} catch {
// Ignore cleanup races.
}
}
export async function readLocalServicePortOwner(port: number) {
if (!Number.isInteger(port) || port <= 0 || process.platform === "win32") return null;
try {
const { stdout } = await execFileAsync("lsof", ["-nP", `-iTCP:${port}`, "-sTCP:LISTEN", "-t"]);
const firstPid = stdout
.split("\n")
.map((line) => Number.parseInt(line.trim(), 10))
.find((value) => Number.isInteger(value) && value > 0);
return firstPid ?? null;
} catch {
return null;
}
}
export async function readLocalServiceProcessCwd(pid: number) {
if (!Number.isInteger(pid) || pid <= 0 || process.platform !== "linux") return null;
try {
return await fs.readlink(`/proc/${pid}/cwd`);
} catch {
return null;
}
}
export async function isLocalServiceProcessInWorkspace(processCwd: string, workspaceCwd: string) {
try {
const [resolvedProcessCwd, resolvedWorkspaceCwd] = await Promise.all([
fs.realpath(processCwd),
fs.realpath(workspaceCwd),
]);
const relativePath = path.relative(resolvedWorkspaceCwd, resolvedProcessCwd);
return relativePath === "" || (!relativePath.startsWith(`..${path.sep}`) && relativePath !== "..");
} catch {
return false;
}
}
export async function isLocalServiceRegistryCwdCompatible(processCwd: string | null, workspaceCwd: string) {
if (!processCwd) return process.platform !== "linux";
return isLocalServiceProcessInWorkspace(processCwd, workspaceCwd);
}
async function doesLocalServiceRecordMatchCwd(record: LocalServiceRegistryRecord) {
if (!record.port) return true;
const ownerPid = await readLocalServicePortOwner(record.port);
if (!ownerPid) return false;
const ownerCwd = await readLocalServiceProcessCwd(ownerPid);
return isLocalServiceRegistryCwdCompatible(ownerCwd, record.cwd);
}