mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-06 21:05:21 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip creates isolated instances for server-managed git worktrees. > - The worktree teardown path removes the git worktree but leaves its isolated instance directory behind. > - The leaked directory can retain an embedded PostgreSQL process and database files. > - Teardown must remove only the collision-resistant instance assigned to that exact worktree path. > - This pull request stops the verified embedded PostgreSQL process and removes the guarded instance directory. > - The benefit is complete worktree cleanup without risk to another, default, or live Paperclip instance. ## Linked Issues or Issue Description **What happened?** Closing a server-managed git worktree removed the git worktree and branch, but it left the isolated Paperclip instance directory behind. A live embedded PostgreSQL process could also keep running against that directory. **Expected behavior** Worktree teardown must stop the isolated embedded PostgreSQL process and remove only the instance assigned to that exact worktree. It must refuse mismatched instance IDs and all paths outside `PAPERCLIP_WORKTREES_DIR/instances/`. **Steps to reproduce** 1. Create a server-managed git worktree with a repo-local `.paperclip/.env` file. 2. Start its isolated embedded PostgreSQL instance. 3. Close the execution workspace. 4. Observe that the git worktree is removed but the isolated instance directory remains. **Paperclip version or commit** The bug reproduces on `master` before this change. **Deployment mode** Local development with a server-managed git worktree and embedded PostgreSQL. ## What Changed - Give server-managed worktrees collision-resistant instance IDs derived from their resolved absolute paths. - Capture the repo-local instance pointer before custom teardown commands can remove it. - Require the pointer's instance ID to match the exact worktree-derived ID. - Resolve and validate the instance path against the canonical managed worktree instance root. - Verify and stop the matching embedded PostgreSQL process before directory removal, including process-exit races. - Record successful and refused cleanup operations in the workspace operation log. - Add focused ownership, process-race, path-safety, and runtime integration tests. - Document automatic isolated-instance cleanup for server-managed worktrees. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/workspace-instance-cleanup.test.ts` — 9 tests passed. - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/workspace-runtime.test.ts -t "records teardown and cleanup operations when a recorder is provided"` — 1 test passed and 99 tests skipped. - `node scripts/__tests__/provision-worktree-self-heal.test.mjs` — 4 tests passed. - `bash -n scripts/provision-worktree.sh` — passed. - `pnpm --filter @paperclipai/server build` — passed. - `git diff --check` — passed. ## Risks The main risk is removal of the wrong instance directory. Provisioning assigns a path-derived ID with a SHA-256 suffix, and cleanup requires that exact ID in addition to a safe instance identifier, an absolute configured home, a strict child path, canonical path checks, and a second canonical path check immediately before removal. It refuses legacy or mismatched IDs, symlink escapes, and all paths outside the managed worktree instance root. Cleanup failures become visible warnings and do not delete an unverified path. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex with GPT-5. The runtime does not expose the exact model snapshot or context-window size. The agent used reasoning, repository tools, GitHub tools, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing>
706 lines
24 KiB
Bash
706 lines
24 KiB
Bash
#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
|
|
base_cwd="${PAPERCLIP_WORKSPACE_BASE_CWD:?PAPERCLIP_WORKSPACE_BASE_CWD is required}"
|
|
worktree_cwd="${PAPERCLIP_WORKSPACE_CWD:?PAPERCLIP_WORKSPACE_CWD is required}"
|
|
paperclip_home="${PAPERCLIP_HOME:-$HOME/.paperclip}"
|
|
paperclip_instance_id="${PAPERCLIP_INSTANCE_ID:-default}"
|
|
paperclip_dir="$worktree_cwd/.paperclip"
|
|
worktree_config_path="$paperclip_dir/config.json"
|
|
worktree_env_path="$paperclip_dir/.env"
|
|
worktree_name="${PAPERCLIP_WORKSPACE_BRANCH:-$(basename "$worktree_cwd")}"
|
|
worktree_instance_id="$(WORKTREE_CWD="$worktree_cwd" node <<'EOF'
|
|
const crypto = require("node:crypto");
|
|
const path = require("node:path");
|
|
|
|
const resolvedWorkspacePath = path.resolve(process.env.WORKTREE_CWD);
|
|
const normalized = path.basename(resolvedWorkspacePath)
|
|
.trim()
|
|
.toLowerCase()
|
|
.replace(/[^a-z0-9_-]+/g, "-")
|
|
.replace(/-+/g, "-")
|
|
.replace(/^[-_]+|[-_]+$/g, "");
|
|
const prefix = (normalized || "worktree").slice(0, 48);
|
|
const pathHash = crypto.createHash("sha256").update(resolvedWorkspacePath).digest("hex").slice(0, 12);
|
|
process.stdout.write(`${prefix}-${pathHash}`);
|
|
EOF
|
|
)"
|
|
|
|
if [[ ! -d "$base_cwd" ]]; then
|
|
echo "Base workspace does not exist: $base_cwd" >&2
|
|
exit 1
|
|
fi
|
|
|
|
if [[ ! -d "$worktree_cwd" ]]; then
|
|
echo "Derived worktree does not exist: $worktree_cwd" >&2
|
|
exit 1
|
|
fi
|
|
|
|
source_config_path="${PAPERCLIP_CONFIG:-}"
|
|
if [[ -z "$source_config_path" && ( -e "$base_cwd/.paperclip/config.json" || -L "$base_cwd/.paperclip/config.json" ) ]]; then
|
|
source_config_path="$base_cwd/.paperclip/config.json"
|
|
fi
|
|
if [[ -z "$source_config_path" ]]; then
|
|
source_config_path="$paperclip_home/instances/$paperclip_instance_id/config.json"
|
|
fi
|
|
source_env_path="$(dirname "$source_config_path")/.env"
|
|
|
|
mkdir -p "$paperclip_dir"
|
|
|
|
base_cli_runner_path="$base_cwd/cli/node_modules/tsx/dist/cli.mjs"
|
|
base_cli_entry_path="$base_cwd/cli/src/index.ts"
|
|
|
|
base_cli_files_present() {
|
|
[[ -f "$base_cli_runner_path" && -f "$base_cli_entry_path" ]]
|
|
}
|
|
|
|
# File existence is not enough: pnpm links package node_modules into the
|
|
# versioned virtual store, so a lockfile change plus a partial/filtered install
|
|
# in the base workspace leaves dangling symlinks that fail ESM resolution at
|
|
# runtime. Actually boot the CLI to prove its import graph resolves.
|
|
base_cli_healthy() {
|
|
base_cli_files_present || return 1
|
|
(cd "$base_cwd" && node "$base_cli_runner_path" "$base_cli_entry_path" --help >/dev/null 2>&1)
|
|
}
|
|
|
|
repair_base_workspace_install() {
|
|
command -v pnpm >/dev/null 2>&1 || return 1
|
|
[[ -f "$base_cwd/package.json" && -f "$base_cwd/pnpm-lock.yaml" ]] || return 1
|
|
echo "Base workspace CLI at $base_cli_entry_path failed its health check (typically dangling pnpm symlinks after a partial install); repairing with pnpm install in $base_cwd." >&2
|
|
# --force guarantees relinking even when pnpm's up-to-date heuristics would
|
|
# otherwise skip the dangling symlinks; --frozen-lockfile keeps the repair
|
|
# from mutating the shared base workspace's lockfile.
|
|
local repair_cmd=(pnpm install --prod=false --force --frozen-lockfile --config.confirmModulesPurge=false)
|
|
# Resolve the real git dir so locking also covers base workspaces that are
|
|
# linked worktrees, where "$base_cwd/.git" is a file rather than a directory.
|
|
local repair_lock_dir=""
|
|
if command -v git >/dev/null 2>&1; then
|
|
repair_lock_dir="$(git -C "$base_cwd" rev-parse --absolute-git-dir 2>/dev/null || true)"
|
|
fi
|
|
if [[ ! -d "$repair_lock_dir" && -d "$base_cwd/.git" ]]; then
|
|
repair_lock_dir="$base_cwd/.git"
|
|
fi
|
|
if command -v flock >/dev/null 2>&1 && [[ -d "$repair_lock_dir" ]]; then
|
|
# The post-repair verification must run under the same lock: a concurrent
|
|
# provision's forced install could be mid-relink during an unlocked check
|
|
# and fail a repair that actually succeeded. Holding the lock also means a
|
|
# process that queued behind a peer's repair can skip its own reinstall.
|
|
(
|
|
cd "$base_cwd" || exit 1
|
|
exec 9>"$repair_lock_dir/paperclip-provision-repair.lock"
|
|
flock 9
|
|
if base_cli_healthy; then
|
|
echo "Base workspace CLI became healthy while waiting for the repair lock; skipping reinstall." >&2
|
|
exit 0
|
|
fi
|
|
env -u NODE_ENV CI=true "${repair_cmd[@]}" >&2 || exit 1
|
|
base_cli_healthy
|
|
)
|
|
else
|
|
(cd "$base_cwd" && env -u NODE_ENV CI=true "${repair_cmd[@]}" >&2 && base_cli_healthy)
|
|
fi
|
|
}
|
|
|
|
ensure_base_cli_healthy() {
|
|
base_cli_files_present || return 1
|
|
base_cli_healthy && return 0
|
|
repair_base_workspace_install
|
|
}
|
|
|
|
run_isolated_worktree_init() {
|
|
if ensure_base_cli_healthy; then
|
|
(
|
|
cd "$worktree_cwd" &&
|
|
node "$base_cli_runner_path" "$base_cli_entry_path" worktree init --force --seed-mode minimal --name "$worktree_name" --instance "$worktree_instance_id" --from-config "$source_config_path"
|
|
)
|
|
return
|
|
fi
|
|
|
|
if command -v pnpm >/dev/null 2>&1 && pnpm paperclipai --help >/dev/null 2>&1; then
|
|
(
|
|
cd "$worktree_cwd" &&
|
|
pnpm paperclipai worktree init --force --seed-mode minimal --name "$worktree_name" --instance "$worktree_instance_id" --from-config "$source_config_path"
|
|
)
|
|
return
|
|
fi
|
|
|
|
if command -v paperclipai >/dev/null 2>&1; then
|
|
(
|
|
cd "$worktree_cwd" &&
|
|
paperclipai worktree init --force --seed-mode minimal --name "$worktree_name" --instance "$worktree_instance_id" --from-config "$source_config_path"
|
|
)
|
|
return
|
|
fi
|
|
|
|
return 127
|
|
}
|
|
|
|
paperclipai_command_available() {
|
|
if command -v pnpm >/dev/null 2>&1 && pnpm paperclipai --help >/dev/null 2>&1; then
|
|
return 0
|
|
fi
|
|
|
|
if command -v node >/dev/null 2>&1 && base_cli_files_present; then
|
|
return 0
|
|
fi
|
|
|
|
if command -v paperclipai >/dev/null 2>&1; then
|
|
return 0
|
|
fi
|
|
|
|
return 1
|
|
}
|
|
|
|
existing_worktree_config_is_usable() {
|
|
WORKTREE_CONFIG_PATH="$worktree_config_path" \
|
|
WORKTREE_ENV_PATH="$worktree_env_path" \
|
|
WORKTREE_INSTANCE_ID="$worktree_instance_id" \
|
|
node <<'EOF'
|
|
const fs = require("node:fs");
|
|
const os = require("node:os");
|
|
const path = require("node:path");
|
|
|
|
function expandHomePrefix(value) {
|
|
if (!value) return value;
|
|
if (value === "~") return os.homedir();
|
|
if (value.startsWith("~/")) return path.resolve(os.homedir(), value.slice(2));
|
|
return value;
|
|
}
|
|
|
|
function parseEnvFile(contents) {
|
|
const entries = {};
|
|
for (const rawLine of contents.split(/\r?\n/)) {
|
|
const line = rawLine.trim();
|
|
if (!line || line.startsWith("#")) continue;
|
|
const match = rawLine.match(/^\s*(?:export\s+)?([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*)\s*$/);
|
|
if (!match) continue;
|
|
const [, key, rawValue] = match;
|
|
const value = rawValue.trim();
|
|
if (
|
|
(value.startsWith("\"") && value.endsWith("\"")) ||
|
|
(value.startsWith("'") && value.endsWith("'"))
|
|
) {
|
|
entries[key] = value.slice(1, -1);
|
|
continue;
|
|
}
|
|
entries[key] = value.replace(/\s+#.*$/, "").trim();
|
|
}
|
|
return entries;
|
|
}
|
|
|
|
function fail(reason) {
|
|
console.error(reason);
|
|
process.exit(1);
|
|
}
|
|
|
|
const configPath = path.resolve(process.env.WORKTREE_CONFIG_PATH);
|
|
const envPath = path.resolve(process.env.WORKTREE_ENV_PATH);
|
|
const config = JSON.parse(fs.readFileSync(configPath, "utf8"));
|
|
const env = parseEnvFile(fs.readFileSync(envPath, "utf8"));
|
|
const envConfigPath = expandHomePrefix(env.PAPERCLIP_CONFIG);
|
|
if (envConfigPath && path.resolve(envConfigPath) !== configPath) {
|
|
fail(`existing worktree env points at ${envConfigPath}, not ${configPath}`);
|
|
}
|
|
|
|
const homeDir = expandHomePrefix(env.PAPERCLIP_HOME);
|
|
const instanceId = env.PAPERCLIP_INSTANCE_ID;
|
|
const expectedInstanceId = process.env.WORKTREE_INSTANCE_ID;
|
|
if (!homeDir || !instanceId) {
|
|
fail("existing worktree env is missing PAPERCLIP_HOME or PAPERCLIP_INSTANCE_ID");
|
|
}
|
|
if (instanceId !== expectedInstanceId) {
|
|
fail(`existing worktree env names legacy or mismatched instance ${instanceId}, expected ${expectedInstanceId}`);
|
|
}
|
|
if (!fs.existsSync(homeDir)) {
|
|
fail(`existing worktree home does not exist on this host: ${homeDir}`);
|
|
}
|
|
|
|
const instanceRoot = path.resolve(homeDir, "instances", instanceId);
|
|
const runtimePaths = [
|
|
config.database?.embeddedPostgresDataDir,
|
|
config.database?.backup?.dir,
|
|
config.logging?.logDir,
|
|
config.storage?.localDisk?.baseDir,
|
|
config.secrets?.localEncrypted?.keyFilePath,
|
|
].filter((value) => typeof value === "string" && value.length > 0);
|
|
|
|
for (const rawValue of runtimePaths) {
|
|
const resolved = path.resolve(expandHomePrefix(rawValue));
|
|
const relative = path.relative(instanceRoot, resolved);
|
|
if (relative.startsWith("..") || path.isAbsolute(relative)) {
|
|
fail(`existing worktree config path is outside ${instanceRoot}: ${resolved}`);
|
|
}
|
|
}
|
|
EOF
|
|
}
|
|
|
|
write_fallback_worktree_config() {
|
|
WORKTREE_NAME="$worktree_name" \
|
|
BASE_CWD="$base_cwd" \
|
|
WORKTREE_CWD="$worktree_cwd" \
|
|
PAPERCLIP_DIR="$paperclip_dir" \
|
|
SOURCE_CONFIG_PATH="$source_config_path" \
|
|
SOURCE_ENV_PATH="$source_env_path" \
|
|
WORKTREE_INSTANCE_ID="$worktree_instance_id" \
|
|
PAPERCLIP_WORKTREES_DIR="${PAPERCLIP_WORKTREES_DIR:-}" \
|
|
node <<'EOF'
|
|
const fs = require("node:fs");
|
|
const os = require("node:os");
|
|
const path = require("node:path");
|
|
const net = require("node:net");
|
|
|
|
function expandHomePrefix(value) {
|
|
if (!value) return value;
|
|
if (value === "~") return os.homedir();
|
|
if (value.startsWith("~/")) return path.resolve(os.homedir(), value.slice(2));
|
|
return value;
|
|
}
|
|
|
|
function nonEmpty(value) {
|
|
return typeof value === "string" && value.trim().length > 0 ? value.trim() : null;
|
|
}
|
|
|
|
function parseEnvFile(contents) {
|
|
const entries = {};
|
|
for (const rawLine of contents.split(/\r?\n/)) {
|
|
const line = rawLine.trim();
|
|
if (!line || line.startsWith("#")) continue;
|
|
const match = rawLine.match(/^\s*(?:export\s+)?([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*)\s*$/);
|
|
if (!match) continue;
|
|
const [, key, rawValue] = match;
|
|
const value = rawValue.trim();
|
|
if (!value) {
|
|
entries[key] = "";
|
|
continue;
|
|
}
|
|
if (
|
|
(value.startsWith("\"") && value.endsWith("\"")) ||
|
|
(value.startsWith("'") && value.endsWith("'"))
|
|
) {
|
|
entries[key] = value.slice(1, -1);
|
|
continue;
|
|
}
|
|
entries[key] = value.replace(/\s+#.*$/, "").trim();
|
|
}
|
|
return entries;
|
|
}
|
|
|
|
async function findAvailablePort(preferredPort, reserved = new Set()) {
|
|
const startPort = Number.isFinite(preferredPort) && preferredPort > 0 ? Math.trunc(preferredPort) : 0;
|
|
if (startPort > 0) {
|
|
for (let port = startPort; port < startPort + 100; port += 1) {
|
|
if (reserved.has(port)) continue;
|
|
const available = await new Promise((resolve) => {
|
|
const server = net.createServer();
|
|
server.unref();
|
|
server.once("error", () => resolve(false));
|
|
server.listen(port, "127.0.0.1", () => {
|
|
server.close(() => resolve(true));
|
|
});
|
|
});
|
|
if (available) return port;
|
|
}
|
|
}
|
|
|
|
return await new Promise((resolve, reject) => {
|
|
const server = net.createServer();
|
|
server.unref();
|
|
server.once("error", reject);
|
|
server.listen(0, "127.0.0.1", () => {
|
|
const address = server.address();
|
|
if (!address || typeof address === "string") {
|
|
server.close(() => reject(new Error("Failed to allocate a port.")));
|
|
return;
|
|
}
|
|
const port = address.port;
|
|
server.close(() => resolve(port));
|
|
});
|
|
});
|
|
}
|
|
|
|
function isLoopbackHost(hostname) {
|
|
const value = hostname.trim().toLowerCase();
|
|
return value === "127.0.0.1" || value === "localhost" || value === "::1";
|
|
}
|
|
|
|
function rewriteLocalUrlPort(rawUrl, port) {
|
|
if (!rawUrl) return undefined;
|
|
try {
|
|
const parsed = new URL(rawUrl);
|
|
if (!isLoopbackHost(parsed.hostname)) return rawUrl;
|
|
parsed.port = String(port);
|
|
return parsed.toString();
|
|
} catch {
|
|
return rawUrl;
|
|
}
|
|
}
|
|
|
|
function resolveRuntimeLikePath(value, configPath) {
|
|
const expanded = expandHomePrefix(value);
|
|
if (path.isAbsolute(expanded)) return expanded;
|
|
return path.resolve(path.dirname(configPath), expanded);
|
|
}
|
|
|
|
async function main() {
|
|
const worktreeName = process.env.WORKTREE_NAME;
|
|
const paperclipDir = process.env.PAPERCLIP_DIR;
|
|
const sourceConfigPath = process.env.SOURCE_CONFIG_PATH;
|
|
const sourceEnvPath = process.env.SOURCE_ENV_PATH;
|
|
const worktreeHome = path.resolve(expandHomePrefix(nonEmpty(process.env.PAPERCLIP_WORKTREES_DIR) ?? "~/.paperclip-worktrees"));
|
|
const instanceId = process.env.WORKTREE_INSTANCE_ID;
|
|
if (!/^[A-Za-z0-9_-]+$/.test(instanceId ?? "")) {
|
|
throw new Error("WORKTREE_INSTANCE_ID is missing or unsafe");
|
|
}
|
|
const instanceRoot = path.resolve(worktreeHome, "instances", instanceId);
|
|
const configPath = path.resolve(paperclipDir, "config.json");
|
|
const envPath = path.resolve(paperclipDir, ".env");
|
|
|
|
let sourceConfig = null;
|
|
if (sourceConfigPath && fs.existsSync(sourceConfigPath)) {
|
|
sourceConfig = JSON.parse(fs.readFileSync(sourceConfigPath, "utf8"));
|
|
}
|
|
|
|
const sourceEnvEntries =
|
|
sourceEnvPath && fs.existsSync(sourceEnvPath)
|
|
? parseEnvFile(fs.readFileSync(sourceEnvPath, "utf8"))
|
|
: {};
|
|
|
|
const preferredServerPort = Number(sourceConfig?.server?.port ?? 3101) + 1;
|
|
const serverPort = await findAvailablePort(preferredServerPort);
|
|
const preferredDbPort = Number(sourceConfig?.database?.embeddedPostgresPort ?? 54329) + 1;
|
|
const databasePort = await findAvailablePort(preferredDbPort, new Set([serverPort]));
|
|
|
|
fs.rmSync(configPath, { force: true });
|
|
fs.mkdirSync(path.dirname(configPath), { recursive: true });
|
|
fs.mkdirSync(instanceRoot, { recursive: true });
|
|
|
|
const authPublicBaseUrl = rewriteLocalUrlPort(sourceConfig?.auth?.publicBaseUrl, serverPort);
|
|
const targetConfig = {
|
|
$meta: {
|
|
version: 1,
|
|
updatedAt: new Date().toISOString(),
|
|
source: "configure",
|
|
},
|
|
...(sourceConfig?.llm ? { llm: sourceConfig.llm } : {}),
|
|
database: {
|
|
mode: "embedded-postgres",
|
|
embeddedPostgresDataDir: path.resolve(instanceRoot, "db"),
|
|
embeddedPostgresPort: databasePort,
|
|
backup: {
|
|
enabled: sourceConfig?.database?.backup?.enabled ?? true,
|
|
intervalMinutes: sourceConfig?.database?.backup?.intervalMinutes ?? 60,
|
|
retentionDays: sourceConfig?.database?.backup?.retentionDays ?? 30,
|
|
dir: path.resolve(instanceRoot, "data", "backups"),
|
|
},
|
|
},
|
|
logging: {
|
|
mode: sourceConfig?.logging?.mode ?? "file",
|
|
logDir: path.resolve(instanceRoot, "logs"),
|
|
},
|
|
server: {
|
|
deploymentMode: sourceConfig?.server?.deploymentMode ?? "local_trusted",
|
|
exposure: sourceConfig?.server?.exposure ?? "private",
|
|
...(sourceConfig?.server?.bind ? { bind: sourceConfig.server.bind } : {}),
|
|
...(sourceConfig?.server?.customBindHost ? { customBindHost: sourceConfig.server.customBindHost } : {}),
|
|
host: sourceConfig?.server?.host ?? "127.0.0.1",
|
|
port: serverPort,
|
|
allowedHostnames: sourceConfig?.server?.allowedHostnames ?? [],
|
|
serveUi: sourceConfig?.server?.serveUi ?? true,
|
|
},
|
|
auth: {
|
|
baseUrlMode: sourceConfig?.auth?.baseUrlMode ?? "auto",
|
|
...(authPublicBaseUrl ? { publicBaseUrl: authPublicBaseUrl } : {}),
|
|
disableSignUp: sourceConfig?.auth?.disableSignUp ?? false,
|
|
},
|
|
storage: {
|
|
provider: sourceConfig?.storage?.provider ?? "local_disk",
|
|
localDisk: {
|
|
baseDir: path.resolve(instanceRoot, "data", "storage"),
|
|
},
|
|
s3: {
|
|
bucket: sourceConfig?.storage?.s3?.bucket ?? "paperclip",
|
|
region: sourceConfig?.storage?.s3?.region ?? "us-east-1",
|
|
endpoint: sourceConfig?.storage?.s3?.endpoint,
|
|
prefix: sourceConfig?.storage?.s3?.prefix ?? "",
|
|
forcePathStyle: sourceConfig?.storage?.s3?.forcePathStyle ?? false,
|
|
},
|
|
},
|
|
secrets: {
|
|
provider: sourceConfig?.secrets?.provider ?? "local_encrypted",
|
|
strictMode: sourceConfig?.secrets?.strictMode ?? false,
|
|
localEncrypted: {
|
|
keyFilePath: path.resolve(instanceRoot, "secrets", "master.key"),
|
|
},
|
|
},
|
|
};
|
|
|
|
fs.writeFileSync(configPath, `${JSON.stringify(targetConfig, null, 2)}\n`, { mode: 0o600 });
|
|
|
|
const inlineMasterKey = nonEmpty(sourceEnvEntries.PAPERCLIP_SECRETS_MASTER_KEY);
|
|
if (inlineMasterKey) {
|
|
fs.mkdirSync(path.resolve(instanceRoot, "secrets"), { recursive: true });
|
|
fs.writeFileSync(targetConfig.secrets.localEncrypted.keyFilePath, inlineMasterKey, {
|
|
encoding: "utf8",
|
|
mode: 0o600,
|
|
});
|
|
} else {
|
|
const sourceKeyFilePath = nonEmpty(sourceEnvEntries.PAPERCLIP_SECRETS_MASTER_KEY_FILE)
|
|
? resolveRuntimeLikePath(sourceEnvEntries.PAPERCLIP_SECRETS_MASTER_KEY_FILE, sourceConfigPath)
|
|
: nonEmpty(sourceConfig?.secrets?.localEncrypted?.keyFilePath)
|
|
? resolveRuntimeLikePath(sourceConfig.secrets.localEncrypted.keyFilePath, sourceConfigPath)
|
|
: null;
|
|
|
|
if (sourceKeyFilePath && fs.existsSync(sourceKeyFilePath)) {
|
|
fs.mkdirSync(path.resolve(instanceRoot, "secrets"), { recursive: true });
|
|
fs.copyFileSync(sourceKeyFilePath, targetConfig.secrets.localEncrypted.keyFilePath);
|
|
fs.chmodSync(targetConfig.secrets.localEncrypted.keyFilePath, 0o600);
|
|
}
|
|
}
|
|
|
|
const envLines = [
|
|
"PAPERCLIP_HOME=" + JSON.stringify(worktreeHome),
|
|
"PAPERCLIP_INSTANCE_ID=" + JSON.stringify(instanceId),
|
|
"PAPERCLIP_CONFIG=" + JSON.stringify(configPath),
|
|
"PAPERCLIP_CONTEXT=" + JSON.stringify(path.resolve(worktreeHome, "context.json")),
|
|
"PAPERCLIP_IN_WORKTREE=true",
|
|
"PAPERCLIP_WORKTREE_NAME=" + JSON.stringify(worktreeName),
|
|
];
|
|
|
|
// Secrets that must be carried over from the source instance so the worktree's
|
|
// dev server behaves like the real one. PAPERCLIP_TOOL_ACTION_SIGNING_SECRET is
|
|
// required for signed tool-gateway approvals (ask-first MCP policies); without
|
|
// it the first gated POST /tool-gateway/tools/call returns Internal server error.
|
|
// BETTER_AUTH_SECRET keeps auth tokens compatible across the source/worktree pair.
|
|
const propagatedSecretKeys = [
|
|
"PAPERCLIP_AGENT_JWT_SECRET",
|
|
"PAPERCLIP_TOOL_ACTION_SIGNING_SECRET",
|
|
"BETTER_AUTH_SECRET",
|
|
];
|
|
for (const key of propagatedSecretKeys) {
|
|
const value = nonEmpty(sourceEnvEntries[key]);
|
|
if (value) {
|
|
envLines.push(key + "=" + JSON.stringify(value));
|
|
}
|
|
}
|
|
|
|
fs.writeFileSync(envPath, `${envLines.join("\n")}\n`, { mode: 0o600 });
|
|
}
|
|
|
|
main().catch((error) => {
|
|
console.error(error instanceof Error ? error.message : String(error));
|
|
process.exit(1);
|
|
});
|
|
EOF
|
|
}
|
|
|
|
if [[ -e "$worktree_config_path" && -e "$worktree_env_path" ]] && existing_worktree_config_is_usable; then
|
|
echo "Reusing existing isolated Paperclip worktree config at $worktree_config_path" >&2
|
|
else
|
|
if [[ -e "$worktree_config_path" || -e "$worktree_env_path" ]]; then
|
|
echo "Existing isolated Paperclip worktree config is stale for this host; regenerating." >&2
|
|
fi
|
|
if paperclipai_command_available; then
|
|
if run_isolated_worktree_init; then
|
|
:
|
|
else
|
|
init_exit_code=$?
|
|
if [[ "$init_exit_code" -eq 127 ]]; then
|
|
# Every CLI candidate was unusable (e.g. an unhealthy base install that
|
|
# the repair could not fix); degrade instead of stranding the run.
|
|
echo "No usable paperclipai CLI found; writing isolated fallback config without DB seeding." >&2
|
|
write_fallback_worktree_config
|
|
else
|
|
# A CLI that ran and failed signals a real problem; do not paper over
|
|
# it with an unseeded fallback config.
|
|
echo "paperclipai worktree init failed (exit $init_exit_code); failing provisioning instead of writing an unseeded fallback config." >&2
|
|
exit "$init_exit_code"
|
|
fi
|
|
fi
|
|
else
|
|
echo "paperclipai worktree init unavailable; writing isolated fallback config without DB seeding." >&2
|
|
write_fallback_worktree_config
|
|
fi
|
|
fi
|
|
|
|
list_base_node_modules_paths() {
|
|
cd "$base_cwd" &&
|
|
find . \
|
|
-mindepth 1 \
|
|
-maxdepth 4 \
|
|
-type d \
|
|
-name node_modules \
|
|
! -path './.git/*' \
|
|
! -path './.paperclip/*' \
|
|
| sed 's#^\./##'
|
|
}
|
|
|
|
compute_pnpm_install_fingerprint() {
|
|
WORKTREE_CWD="$worktree_cwd" node <<'EOF'
|
|
const crypto = require("node:crypto");
|
|
const fs = require("node:fs");
|
|
const path = require("node:path");
|
|
|
|
const root = process.env.WORKTREE_CWD;
|
|
const ignoredDirs = new Set([".git", ".paperclip", "node_modules", "dist", "storybook-static"]);
|
|
const files = [];
|
|
|
|
function walk(dir) {
|
|
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
|
|
if (ignoredDirs.has(entry.name)) continue;
|
|
|
|
const absolutePath = path.join(dir, entry.name);
|
|
if (entry.isDirectory()) {
|
|
walk(absolutePath);
|
|
continue;
|
|
}
|
|
|
|
if (
|
|
entry.isFile()
|
|
&& (entry.name === "package.json" || entry.name === "pnpm-lock.yaml" || entry.name === "pnpm-workspace.yaml")
|
|
) {
|
|
files.push(absolutePath);
|
|
}
|
|
}
|
|
}
|
|
|
|
walk(root);
|
|
files.sort((left, right) => path.relative(root, left).localeCompare(path.relative(root, right)));
|
|
|
|
const hash = crypto.createHash("sha256");
|
|
for (const file of files) {
|
|
const relativePath = path.relative(root, file).replaceAll(path.sep, "/");
|
|
hash.update(relativePath);
|
|
hash.update("\0");
|
|
hash.update(fs.readFileSync(file));
|
|
hash.update("\0");
|
|
}
|
|
|
|
process.stdout.write(hash.digest("hex"));
|
|
EOF
|
|
}
|
|
|
|
if [[ -f "$worktree_cwd/package.json" && -f "$worktree_cwd/pnpm-lock.yaml" ]]; then
|
|
needs_install=0
|
|
install_fingerprint_path="$paperclip_dir/pnpm-install-fingerprint"
|
|
current_install_fingerprint="$(compute_pnpm_install_fingerprint)"
|
|
previous_install_fingerprint=""
|
|
if [[ -f "$install_fingerprint_path" ]]; then
|
|
previous_install_fingerprint="$(cat "$install_fingerprint_path")"
|
|
fi
|
|
|
|
while IFS= read -r relative_path; do
|
|
[[ -n "$relative_path" ]] || continue
|
|
target_path="$worktree_cwd/$relative_path"
|
|
|
|
if [[ -L "$target_path" || ! -e "$target_path" ]]; then
|
|
needs_install=1
|
|
break
|
|
fi
|
|
done < <(list_base_node_modules_paths)
|
|
|
|
if [[ "$needs_install" -eq 0 && "$current_install_fingerprint" != "$previous_install_fingerprint" ]]; then
|
|
needs_install=1
|
|
fi
|
|
|
|
if [[ "$needs_install" -eq 1 ]]; then
|
|
backup_suffix=".paperclip-backup-${BASHPID:-$$}"
|
|
moved_symlink_paths=()
|
|
|
|
while IFS= read -r relative_path; do
|
|
[[ -n "$relative_path" ]] || continue
|
|
target_path="$worktree_cwd/$relative_path"
|
|
if [[ -L "$target_path" ]]; then
|
|
backup_path="${target_path}${backup_suffix}"
|
|
rm -rf "$backup_path"
|
|
mv "$target_path" "$backup_path"
|
|
moved_symlink_paths+=("$relative_path")
|
|
fi
|
|
done < <(list_base_node_modules_paths)
|
|
|
|
restore_moved_symlinks() {
|
|
local relative_path target_path backup_path
|
|
[[ ${#moved_symlink_paths[@]} -gt 0 ]] || return 0
|
|
for relative_path in "${moved_symlink_paths[@]}"; do
|
|
target_path="$worktree_cwd/$relative_path"
|
|
backup_path="${target_path}${backup_suffix}"
|
|
[[ -L "$backup_path" ]] || continue
|
|
rm -rf "$target_path"
|
|
mv "$backup_path" "$target_path"
|
|
done
|
|
}
|
|
|
|
cleanup_moved_symlinks() {
|
|
local relative_path target_path backup_path
|
|
[[ ${#moved_symlink_paths[@]} -gt 0 ]] || return 0
|
|
for relative_path in "${moved_symlink_paths[@]}"; do
|
|
target_path="$worktree_cwd/$relative_path"
|
|
backup_path="${target_path}${backup_suffix}"
|
|
[[ -L "$backup_path" ]] && rm "$backup_path"
|
|
done
|
|
}
|
|
|
|
run_pnpm_install() {
|
|
local stdout_path stderr_path
|
|
stdout_path="$(mktemp)"
|
|
stderr_path="$(mktemp)"
|
|
|
|
if (
|
|
cd "$worktree_cwd"
|
|
pnpm install --prod=false "$@"
|
|
) >"$stdout_path" 2>"$stderr_path"; then
|
|
cat "$stdout_path"
|
|
cat "$stderr_path" >&2
|
|
rm -f "$stdout_path" "$stderr_path"
|
|
return 0
|
|
fi
|
|
|
|
local exit_code=$?
|
|
cat "$stdout_path"
|
|
cat "$stderr_path" >&2
|
|
if grep -q "ERR_PNPM_OUTDATED_LOCKFILE" "$stdout_path" "$stderr_path"; then
|
|
rm -f "$stdout_path" "$stderr_path"
|
|
return 90
|
|
fi
|
|
|
|
rm -f "$stdout_path" "$stderr_path"
|
|
return "$exit_code"
|
|
}
|
|
|
|
if run_pnpm_install --frozen-lockfile; then
|
|
:
|
|
else
|
|
install_exit_code=$?
|
|
if [[ "$install_exit_code" -eq 90 ]]; then
|
|
echo "pnpm-lock.yaml is out of date in this execution workspace; retrying install without --frozen-lockfile." >&2
|
|
run_pnpm_install --no-frozen-lockfile || {
|
|
restore_moved_symlinks
|
|
exit 1
|
|
}
|
|
else
|
|
restore_moved_symlinks
|
|
exit "$install_exit_code"
|
|
fi
|
|
fi
|
|
|
|
cleanup_moved_symlinks
|
|
current_install_fingerprint="$(compute_pnpm_install_fingerprint)"
|
|
printf '%s\n' "$current_install_fingerprint" >"$install_fingerprint_path"
|
|
fi
|
|
|
|
exit 0
|
|
fi
|
|
|
|
while IFS= read -r relative_path; do
|
|
[[ -n "$relative_path" ]] || continue
|
|
source_path="$base_cwd/$relative_path"
|
|
target_path="$worktree_cwd/$relative_path"
|
|
|
|
[[ -d "$source_path" ]] || continue
|
|
[[ -e "$target_path" || -L "$target_path" ]] && continue
|
|
|
|
mkdir -p "$(dirname "$target_path")"
|
|
ln -s "$source_path" "$target_path"
|
|
done < <(
|
|
list_base_node_modules_paths
|
|
)
|