Files
PaperClipAI/server/src/services/workspace-instance-cleanup.ts
T
DottaandPaperclip a2bf936f9a feat(workspaces): sign the workspace login handoff and gate readiness (#11671)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Managed worktree services run isolated Paperclip instances with
cloned databases.
> - A reachable service was reported as ready even when its database,
runtime identity, or login path was not usable.
> - The first candidate added verified database seeding and managed
repair in #11665.
> - This pull request consolidates that candidate with signed login
handoff and a complete readiness contract.
> - Post-QA fixes close five defects in repair identity, repair
responses, UI retry, seed journal handling, and seed-source trust.
> - The benefit is a workspace that either opens safely or reports one
accurate recovery action.

## Linked Issues or Issue Description

No public GitHub issue exists for this work, so the problem is described
here.

**What happened**

Managed workspace URLs could return HTTP 200 and report ready while
login failed. QA also found cases where repair used the wrong instance
identity, returned a generic error, left the UI stuck, rejected a safe
journal lag, or trusted a mutable workspace manifest.

**Expected behavior**

Opening a ready workspace signs the board user in to the correct
isolated instance. Provisioning and repair use a registered source and
report a structured recovery state.

**Actual behavior**

Entry depended on a password copied into the clone. Several failure
paths could publish stale readiness, hide the repair precondition, or
trust state that the workspace could modify.

**Additional context**

This pull request includes the commits first published in #11665. That
pull request keeps the original base head for review history. This
consolidated pull request is the merge candidate. Related open readiness
work includes #11575 and #11621.

## What Changed

- Adds a short-lived, signed, single-use login ticket. It binds the
user, workspace, instance, and runtime origin.
- Exchanges the ticket through Better Auth. It creates the session and
cookie through the supported adapter path.
- Adds protected workspace readiness fields for the database, clone
data, login handoff, seed phase, and runtime identity.
- Fails readiness closed when the guest has no company or
execution-workspace binding.
- Binds ticket issuance to the exact cloned user and active company
membership selected for the handoff.
- Verifies every current active board identity through the exact-user
handoff before publication or reuse.
- Gates managed runtime publication on the readiness contract and the
recorded worktree instance identity.
- Refreshes runtime work products from the live runtime row after a port
change.
- Adds one workspace access card with ready, degraded, repairing, and
failed states.
- Uses the runtime response identity for repair. It returns structured
repair precondition errors.
- Lets a valid source journal lag converge during provisioning.
- Binds seed and repair manifests to a source registered outside the
agent-writable worktree.
- Clears recovered UI errors so a successful retry can open the
workspace.
- Makes runtime tests register canonical sources and avoid ports owned
by live host listeners.
- Keeps Vitest on source suites when compiled `dist` trees exist.
- Isolates CLI and adapter tests from ambient AWS and runtime API
environment variables.
- Preserves a 404 response for cross-company workspace ID lookups before
runtime authorization.
- Makes concurrent single-flight coverage independent of
path-canonicalization scheduling order.

## Verification

The following checks passed on the integrated head:

```sh
pnpm -r typecheck
pnpm build
pnpm check:token-gates
pnpm --filter @paperclipai/db check:migrations
```

- The server source lane passed 420 files and 4,953 tests. Five tests
were skipped.
- The CLI lane passed 57 files and 385 tests.
- The database lane passed 26 files and 97 tests.
- The shared package passed 58 files and 506 tests.
- The adapter utility lane passed 640 tests. Four tests were skipped.
- The Claude adapter passed 220 tests. One test was skipped.
- The Codex adapter passed 323 tests.
- The OpenClaw adapter passed 13 tests.
- The OpenCode adapter passed 42 tests.
- The plugin SDK passed 45 tests.
- The workspace runtime suite passed 124 tests.
- The caller-scoped readiness and handoff suite passed 52 tests.
- The workspace provisioning shell suite passed 7 tests.
- The runtime exposure suite passed 17 tests while live host mappings
occupied fixed test ports.
- `git diff --check` passed and the worktree is clean.

The serialized route lane will run in GitHub CI with its normal shards.
No deployment or active-workspace migration was performed.

## Risks

- This is a medium-risk authentication and runtime-readiness change.
- The login ticket uses exact origin, workspace, instance, and user
binding. It has a short expiry and a one-time nonce.
- Runtime publication is stricter. A real readiness, identity, per-user
handoff, or control-plane database disagreement now blocks publication.
- This pull request supersedes #11665 as the merge candidate. Close
#11665 after this pull request merges.
- No new database migration is included. The lockfile and workflow files
are unchanged.
- Deployment and active-workspace migration are intentionally outside
this pull request.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

Claude Opus 5 (`claude-opus-5[1m]`), 1M context, extended thinking, tool
use, and code execution produced the main candidate. OpenAI GPT-5
(`gpt-5`) through Codex, with agentic reasoning, tool use, and code
execution, integrated the post-QA fixes and hardened the test gates. The
Codex context-window size was not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-19 02:37:02 -05:00

412 lines
17 KiB
TypeScript

import { execFile } from "node:child_process";
import { createHash } from "node:crypto";
import { readFileSync } from "node:fs";
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { setTimeout as delay } from "node:timers/promises";
import { promisify } from "node:util";
import { parse as parseEnvContents } from "dotenv";
import { expandHomePrefix } from "../home-paths.js";
import type { WorkspaceOperationRecorder } from "./workspace-operations.js";
const execFileAsync = promisify(execFile);
const INSTANCE_ID_RE = /^[A-Za-z0-9_-]+$/;
const POSTGRES_STOP_TIMEOUT_MS = 10_000;
export const WORKTREE_INSTANCE_ROOT_METADATA_KEY = "worktreeInstanceRoot";
export function deriveWorktreeInstanceId(workspacePath: string): string {
const resolvedWorkspacePath = path.resolve(workspacePath);
const normalized = path.basename(resolvedWorkspacePath)
.trim()
.toLowerCase()
.replace(/[^a-z0-9_-]+/g, "-")
.replace(/-+/g, "-")
.replace(/^[-_]+|[-_]+$/g, "");
const prefix = (normalized || "worktree").slice(0, 48);
const pathHash = createHash("sha256").update(resolvedWorkspacePath).digest("hex").slice(0, 12);
return `${prefix}-${pathHash}`;
}
/**
* The instance id a seeded worktree actually runs as, read from the pointer the
* guest process itself loads.
*
* Synchronous and null-on-anything-unexpected by design: callers use it to
* decide whether they can prove workspace identity at all, and an unsafe or
* unreadable pointer must fail closed rather than fall back to a guess.
*/
export function readWorktreeInstanceId(workspacePath: string): string | null {
const envPath = path.join(path.resolve(workspacePath), ".paperclip", ".env");
let contents: string;
try {
contents = readFileSync(envPath, "utf8");
} catch {
return null;
}
const instanceId = parseEnvContents(contents).PAPERCLIP_INSTANCE_ID?.trim();
if (!instanceId || !INSTANCE_ID_RE.test(instanceId)) return null;
return instanceId;
}
export type WorktreeInstancePointer = {
envPath: string;
envContents: string;
};
export type WorktreeInstanceCleanupResult =
| { status: "not_configured" }
| { status: "already_absent"; instanceRoot: string }
| { status: "refused"; instanceRoot: string | null; warning: string }
| { status: "removed"; instanceRoot: string; postgresStopped: boolean };
export type WorktreeInstanceCleanupDependencies = {
stopEmbeddedPostgres: (dataDir: string) => Promise<boolean>;
removeInstanceRoot: (instanceRoot: string) => Promise<void>;
};
export type EmbeddedPostgresStopDependencies = {
processIsAlive: (pid: number) => boolean;
readVerifiedPostgresCommand: (pid: number, dataDir: string) => Promise<string | null>;
signalProcess: (pid: number, signal: NodeJS.Signals) => void;
wait: (milliseconds: number) => Promise<unknown>;
};
const defaultCleanupDependencies: WorktreeInstanceCleanupDependencies = {
stopEmbeddedPostgres: stopEmbeddedPostgresIfRunning,
removeInstanceRoot: async (instanceRoot) => {
await fs.rm(instanceRoot, { recursive: true, force: true });
},
};
const defaultPostgresStopDependencies: EmbeddedPostgresStopDependencies = {
processIsAlive,
readVerifiedPostgresCommand,
signalProcess: (pid, signal) => process.kill(pid, signal),
wait: delay,
};
function isStrictChildPath(candidatePath: string, rootPath: string): boolean {
const relative = path.relative(rootPath, candidatePath);
return relative.length > 0 && relative !== ".." && !relative.startsWith(`..${path.sep}`) && !path.isAbsolute(relative);
}
async function pathExists(value: string): Promise<boolean> {
return fs.lstat(value).then(() => true).catch(() => false);
}
function processIsAlive(pid: number): boolean {
try {
process.kill(pid, 0);
return true;
} catch (error) {
return (error as NodeJS.ErrnoException).code === "EPERM";
}
}
async function readVerifiedPostgresCommand(pid: number, dataDir: string): Promise<string | null> {
if (process.platform === "linux") {
const commandLinePath = `/proc/${pid}/cmdline`;
let commandLine: string;
try {
commandLine = await fs.readFile(commandLinePath, "utf8");
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") return null;
throw error;
}
const args = commandLine.split("\0").filter(Boolean);
const executable = path.basename(args[0] ?? "");
const dataDirFlagIndex = args.indexOf("-D");
const configuredDataDir = dataDirFlagIndex >= 0 ? args[dataDirFlagIndex + 1] : null;
if (!executable.includes("postgres") || !configuredDataDir) {
throw new Error(`Refusing to signal process ${pid}: it is not the expected embedded PostgreSQL process.`);
}
const canonicalConfiguredDataDir = await fs.realpath(configuredDataDir).catch(() => path.resolve(configuredDataDir));
if (canonicalConfiguredDataDir !== dataDir) {
throw new Error(`Refusing to signal process ${pid}: its PostgreSQL data directory does not match ${dataDir}.`);
}
return args.join(" ");
}
if (process.platform === "win32") {
throw new Error(`Refusing to signal process ${pid}: safe PostgreSQL process verification is unavailable on Windows.`);
}
try {
const { stdout } = await execFileAsync("ps", ["-p", String(pid), "-o", "command="], { encoding: "utf8" });
const command = stdout.trim();
if (!command) return null;
if (!/(?:^|\/)postgres(?:\s|$)/.test(command) || !command.includes(dataDir)) {
throw new Error(`Refusing to signal process ${pid}: it is not the expected embedded PostgreSQL process.`);
}
return command;
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ESRCH") return null;
throw error;
}
}
export async function stopEmbeddedPostgresIfRunning(
dataDir: string,
dependencies: EmbeddedPostgresStopDependencies = defaultPostgresStopDependencies,
): Promise<boolean> {
const postmasterPidPath = path.join(dataDir, "postmaster.pid");
if (!await pathExists(postmasterPidPath)) return false;
const canonicalDataDir = await fs.realpath(dataDir);
const pidContents = await fs.readFile(postmasterPidPath, "utf8");
const pidLines = pidContents.split(/\r?\n/);
const pid = Number(pidLines[0]?.trim());
const recordedDataDir = pidLines[1]?.trim();
if (!Number.isInteger(pid) || pid <= 0 || !recordedDataDir) {
throw new Error(`Refusing to remove ${dataDir}: its postmaster.pid is malformed.`);
}
const canonicalRecordedDataDir = await fs.realpath(recordedDataDir).catch(() => path.resolve(recordedDataDir));
if (canonicalRecordedDataDir !== canonicalDataDir) {
throw new Error(`Refusing to remove ${dataDir}: postmaster.pid points at a different PostgreSQL data directory.`);
}
if (!dependencies.processIsAlive(pid)) return false;
if (await dependencies.readVerifiedPostgresCommand(pid, canonicalDataDir) === null) return false;
try {
dependencies.signalProcess(pid, "SIGINT");
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ESRCH") return false;
throw error;
}
const deadline = Date.now() + POSTGRES_STOP_TIMEOUT_MS;
while (Date.now() < deadline) {
if (!dependencies.processIsAlive(pid)) return true;
await dependencies.wait(100);
}
throw new Error(`Embedded PostgreSQL process ${pid} did not stop within ${POSTGRES_STOP_TIMEOUT_MS}ms.`);
}
export async function readWorktreeInstancePointer(workspacePath: string): Promise<WorktreeInstancePointer | null> {
const envPath = path.join(workspacePath, ".paperclip", ".env");
try {
return {
envPath,
envContents: await fs.readFile(envPath, "utf8"),
};
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") return null;
throw error;
}
}
function resolveConfiguredInstanceRoot(pointer: WorktreeInstancePointer, expectedInstanceId?: string):
| { instanceRoot: string; instanceId: string }
| { warning: string; instanceRoot: string | null; refusalReason: string | null } {
const env = parseEnvContents(pointer.envContents);
const configuredHome = env.PAPERCLIP_HOME?.trim();
const instanceId = env.PAPERCLIP_INSTANCE_ID?.trim();
if (!configuredHome || !instanceId) {
return { warning: "", instanceRoot: null, refusalReason: null };
}
if (!INSTANCE_ID_RE.test(instanceId)) {
return {
instanceRoot: null,
warning: `Refusing worktree instance cleanup from ${pointer.envPath}: PAPERCLIP_INSTANCE_ID is not a safe path segment.`,
refusalReason: "unsafe_instance_id",
};
}
const expandedHome = expandHomePrefix(configuredHome);
if (!path.isAbsolute(expandedHome)) {
return {
instanceRoot: null,
warning: `Refusing worktree instance cleanup from ${pointer.envPath}: PAPERCLIP_HOME is not absolute.`,
refusalReason: "non_absolute_home",
};
}
const instanceRoot = path.resolve(expandedHome, "instances", instanceId);
if (expectedInstanceId && instanceId !== expectedInstanceId) {
return {
instanceRoot,
warning: `Refusing worktree instance cleanup from ${pointer.envPath}: PAPERCLIP_INSTANCE_ID "${instanceId}" does not match the expected workspace instance "${expectedInstanceId}".`,
refusalReason: "instance_id_mismatch",
};
}
return { instanceRoot, instanceId };
}
function resolveManagedInstancesDir(worktreesDir?: string): string {
const managedWorktreesDir = path.resolve(
expandHomePrefix(
worktreesDir?.trim()
|| process.env.PAPERCLIP_WORKTREES_DIR?.trim()
|| path.join(os.homedir(), ".paperclip-worktrees"),
),
);
return path.join(managedWorktreesDir, "instances");
}
export async function readManagedWorktreeInstanceOwnership(
workspacePath: string,
worktreesDir?: string,
): Promise<{ instanceRoot: string; instanceId: string } | null> {
const pointer = await readWorktreeInstancePointer(workspacePath);
if (!pointer) return null;
const configured = resolveConfiguredInstanceRoot(pointer);
if ("warning" in configured) {
if (!configured.warning) return null;
throw new Error(configured.warning);
}
const managedInstancesDir = resolveManagedInstancesDir(worktreesDir);
if (!isStrictChildPath(configured.instanceRoot, managedInstancesDir)) {
throw new Error(
`Refusing to record worktree instance ownership for "${configured.instanceRoot}" because it is outside "${managedInstancesDir}".`,
);
}
return configured;
}
export async function cleanupWorktreeInstanceArtifacts(input: {
pointer: WorktreeInstancePointer;
workspaceId: string;
workspacePath: string;
expectedInstanceId: string;
expectedInstanceRoot: string | null;
recorder?: WorkspaceOperationRecorder | null;
worktreesDir?: string;
dependencies?: WorktreeInstanceCleanupDependencies;
}): Promise<WorktreeInstanceCleanupResult> {
const configured = resolveConfiguredInstanceRoot(input.pointer, input.expectedInstanceId);
if ("warning" in configured && !configured.warning) return { status: "not_configured" };
const managedInstancesDir = resolveManagedInstancesDir(input.worktreesDir);
const managedWorktreesDir = path.dirname(managedInstancesDir);
const configuredInstanceRoot = configured.instanceRoot;
let warning = "warning" in configured ? configured.warning : "";
const recordRefusal = async (
metadata: Record<string, unknown>,
refusalWarning = warning,
) => {
if (!input.recorder) return;
await input.recorder.recordOperation({
phase: "workspace_teardown",
cwd: input.workspacePath,
metadata: {
workspaceId: input.workspaceId,
workspacePath: input.workspacePath,
instanceRoot: configuredInstanceRoot,
managedInstancesDir,
cleanupAction: "remove_worktree_instance",
...metadata,
},
run: async () => ({ status: "skipped", system: `${refusalWarning}\n` }),
});
};
if ("warning" in configured) {
await recordRefusal({ refusalReason: configured.refusalReason }, configured.warning);
return { status: "refused", instanceRoot: configured.instanceRoot, warning: configured.warning };
}
if (!configuredInstanceRoot || !isStrictChildPath(configuredInstanceRoot, managedInstancesDir)) {
warning ||= `Refusing to remove instance directory "${configuredInstanceRoot ?? "unknown"}" because it is outside "${managedInstancesDir}".`;
await recordRefusal({ refusalReason: "outside_managed_instances_dir" });
return { status: "refused", instanceRoot: configuredInstanceRoot, warning };
}
const expectedInstanceRoot = input.expectedInstanceRoot ? path.resolve(input.expectedInstanceRoot) : null;
if (!expectedInstanceRoot) {
warning = `Refusing to remove instance directory "${configuredInstanceRoot}" because execution workspace ${input.workspaceId} has no persisted instance root.`;
await recordRefusal({ refusalReason: "persisted_instance_root_missing" });
return { status: "refused", instanceRoot: configuredInstanceRoot, warning };
}
if (expectedInstanceRoot && configuredInstanceRoot !== expectedInstanceRoot) {
warning = `Refusing to remove instance directory "${configuredInstanceRoot}" because it does not match execution workspace ${input.workspaceId}'s persisted instance root "${expectedInstanceRoot}".`;
await recordRefusal({
expectedInstanceRoot,
refusalReason: "instance_root_workspace_mismatch",
});
return { status: "refused", instanceRoot: configuredInstanceRoot, warning };
}
if (!await pathExists(configuredInstanceRoot)) {
return { status: "already_absent", instanceRoot: configuredInstanceRoot };
}
let canonicalManagedWorktreesDir: string;
let canonicalManagedInstancesDir: string;
let canonicalInstanceRoot: string;
try {
[canonicalManagedWorktreesDir, canonicalManagedInstancesDir, canonicalInstanceRoot] = await Promise.all([
fs.realpath(managedWorktreesDir),
fs.realpath(managedInstancesDir),
fs.realpath(configuredInstanceRoot),
]);
} catch (error) {
warning = `Refusing to remove instance directory "${configuredInstanceRoot}" because its canonical path could not be verified: ${error instanceof Error ? error.message : String(error)}`;
await recordRefusal({ refusalReason: "canonical_path_unavailable" });
return { status: "refused", instanceRoot: configuredInstanceRoot, warning };
}
if (canonicalManagedInstancesDir !== path.join(canonicalManagedWorktreesDir, "instances")) {
warning = `Refusing to remove instance directory "${configuredInstanceRoot}" because the managed instances directory resolves outside "${canonicalManagedWorktreesDir}".`;
await recordRefusal({
canonicalInstanceRoot,
canonicalManagedInstancesDir,
refusalReason: "managed_instances_dir_symlink",
});
return { status: "refused", instanceRoot: configuredInstanceRoot, warning };
}
if (!isStrictChildPath(canonicalInstanceRoot, canonicalManagedInstancesDir)) {
warning = `Refusing to remove instance directory "${configuredInstanceRoot}" because its canonical path "${canonicalInstanceRoot}" is outside "${canonicalManagedInstancesDir}".`;
await recordRefusal({
canonicalInstanceRoot,
canonicalManagedInstancesDir,
refusalReason: "canonical_path_outside_managed_instances_dir",
});
return { status: "refused", instanceRoot: configuredInstanceRoot, warning };
}
const dependencies = input.dependencies ?? defaultCleanupDependencies;
let postgresStopped = false;
const cleanup = async () => {
postgresStopped = await dependencies.stopEmbeddedPostgres(path.join(canonicalInstanceRoot, "db"));
const [currentManagedInstancesDir, currentInstanceRoot] = await Promise.all([
fs.realpath(managedInstancesDir),
fs.realpath(configuredInstanceRoot),
]);
if (
currentManagedInstancesDir !== canonicalManagedInstancesDir
|| currentInstanceRoot !== canonicalInstanceRoot
|| !isStrictChildPath(currentInstanceRoot, currentManagedInstancesDir)
) {
throw new Error(`Refusing to remove instance directory "${configuredInstanceRoot}" because its canonical path changed during cleanup.`);
}
await dependencies.removeInstanceRoot(currentInstanceRoot);
};
if (input.recorder) {
await input.recorder.recordOperation({
phase: "workspace_teardown",
cwd: input.workspacePath,
metadata: {
workspaceId: input.workspaceId,
workspacePath: input.workspacePath,
instanceRoot: canonicalInstanceRoot,
managedInstancesDir: canonicalManagedInstancesDir,
cleanupAction: "remove_worktree_instance",
},
run: async () => {
await cleanup();
return {
status: "succeeded",
system: `Removed worktree instance directory ${canonicalInstanceRoot}\n`,
metadata: { postgresStopped },
};
},
});
} else {
await cleanup();
}
return { status: "removed", instanceRoot: canonicalInstanceRoot, postgresStopped };
}