Files
PaperClipAI/scripts/provision-worktree-runtime.sh
T
bd059a073d fix(workspaces): make managed runtimes reliable across restarts (#11740)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Execution workspaces need isolated databases, ports, and runtime
services
> - Concurrent workspaces could reuse ports or lose service ownership
after a restart
> - A markerless worktree also needed seed recovery, but normal
markerless instances still needed to boot
> - This pull request makes seed, port, and service ownership state
explicit and recoverable
> - It also checks live process and listener identity before it reclaims
shared resources
> - The benefit is reliable workspace startup, restart, adoption, and
concurrent provisioning

## Linked Issues or Issue Description

**What happened?**

Managed workspaces could lose runtime service ownership after a
control-plane restart. Concurrent worktrees could also reuse a port when
their parent paths differed. A seed recovery change made every
markerless instance resolve a worktree seed source, so normal instances
without a source could not start.

**Expected behavior**

Paperclip must preserve healthy managed services across restarts. It
must reserve unique ports across worktree parents. It must provision a
registered markerless worktree, but it must skip seed work for a normal
markerless instance.

**Steps to reproduce**

1. Start two managed worktrees under different parent paths at the same
time.
2. Restart the control plane while a managed service stays alive.
3. Start Paperclip with a config that has no seed markers and no
registered worktree source.
4. Observe duplicate port selection, lost service adoption, or a
seed-source startup error.

**Paperclip version or commit**

Current `master` plus the workspace runtime reliability changes in this
pull request.

**Deployment mode**

Local development with managed execution workspaces and embedded
Postgres.

## What Changed

- Added a shared port registry with lease heartbeats, process identity
checks, and live listener probes.
- Reserved worktree ports across custom parent paths and repaired
duplicate legacy assignments.
- Preserved and adopted healthy managed services across control-plane
restarts.
- Reconciled guest bind modes and verified listener ownership before
termination or reuse.
- Provisioned registered markerless worktree databases and kept normal
markerless instance startup as a no-op.
- Added CLI, shared, server, and shell regression tests for seed, port,
listener, restart, and adoption behavior.
- Updated the worktree development documentation.

## Verification

- `pnpm exec vitest run cli/src/__tests__/worktree.test.ts
--reporter=verbose` — 63 tests passed.
- `pnpm exec vitest run
packages/shared/src/worktree-port-registry.test.ts --reporter=verbose` —
5 tests passed.
- Focused runtime Vitest set — 199 tests passed across 37 suites.
- `node --test scripts/__tests__/provision-worktree-self-heal.test.mjs`
— 10 tests passed.
- `git diff --check` passed.

## Risks

- Port reservation now depends on lease and process identity data. The
fallback listener probe prevents early reclamation when process metadata
is incomplete.
- Runtime adoption is stricter about bind and owner identity. The tests
cover healthy adoption, stale records, PID reuse, and unrelated
listeners.
- Markerless seed detection now separates registered worktrees from
normal instances. The tests cover both paths.
- There are no database schema migrations.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with the `gpt-5` model family. The serving snapshot and
context-window size are not exposed. The agent used reasoning,
repository tools, code execution, and test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Dev Agent <dev@paperclip.ing>
2026-08-19 14:55:16 -05:00

158 lines
5.5 KiB
Bash
Executable File

#!/usr/bin/env bash
set -euo pipefail
base_cwd="${PAPERCLIP_WORKSPACE_BASE_CWD:?PAPERCLIP_WORKSPACE_BASE_CWD is required}"
worktree_cwd="${PAPERCLIP_WORKSPACE_CWD:?PAPERCLIP_WORKSPACE_CWD is required}"
paperclip_dir="$worktree_cwd/.paperclip"
worktree_config_path="$paperclip_dir/config.json"
seed_manifest_path="$paperclip_dir/seed-manifest.json"
if [[ ! -d "$base_cwd" ]]; then
echo "Base workspace does not exist: $base_cwd" >&2
exit 1
fi
if [[ ! -d "$worktree_cwd" ]]; then
echo "Derived worktree does not exist: $worktree_cwd" >&2
exit 1
fi
if [[ -e "$seed_manifest_path" ]]; then
seed_manifest_state="$(SEED_MANIFEST_PATH="$seed_manifest_path" node <<'EOF'
const fs = require("node:fs");
try {
const value = JSON.parse(fs.readFileSync(process.env.SEED_MANIFEST_PATH, "utf8"));
const complete = value?.version === 2
&& value?.state === "verified"
&& value?.phase === "complete"
&& typeof value?.source?.instanceId === "string" && value.source.instanceId.length > 0
&& typeof value?.source?.configPath === "string" && value.source.configPath.length > 0
&& (value?.seedMode === "minimal" || value?.seedMode === "full")
&& typeof value?.snapshotAt === "string" && value.snapshotAt.length > 0
&& typeof value?.migrationRevision === "string" && value.migrationRevision.length > 0
&& typeof value?.targetInstanceId === "string" && value.targetInstanceId.length > 0
&& typeof value?.attemptId === "string" && value.attemptId.length > 0
&& typeof value?.startedAt === "string"
&& typeof value?.finishedAt === "string"
&& Array.isArray(value?.diagnostics)
&& value.diagnostics.some((entry) => entry?.phase === "complete" && entry?.status === "succeeded" && typeof entry?.at === "string");
process.stdout.write(complete ? "verified" : "incomplete");
} catch {
process.stdout.write("invalid");
}
EOF
)"
if [[ "$seed_manifest_state" == "verified" ]]; then
echo "Worktree database has a verified seed manifest; skipping runtime provisioning." >&2
exit 0
fi
fi
if [[ ! -f "$worktree_config_path" ]]; then
initial_provision_script="$base_cwd/scripts/provision-worktree.sh"
if [[ ! -f "$initial_provision_script" ]]; then
echo "Worktree config does not exist and the built-in provision script is unavailable: $worktree_config_path" >&2
exit 1
fi
echo "Worktree config is missing; running the built-in worktree provisioner before database seeding." >&2
(
cd "$worktree_cwd" &&
bash "$initial_provision_script"
)
fi
if [[ ! -f "$worktree_config_path" ]]; then
echo "Worktree config still does not exist after built-in provisioning: $worktree_config_path" >&2
exit 1
fi
# The CLI derives the source from PAPERCLIP_WORKSPACE_BASE_CWD, which the
# control plane injects from the registered project-workspace row. The seed
# manifest is diagnostic evidence only and must never choose the clone source.
source_config_args=()
base_cli_runner_path="$base_cwd/cli/node_modules/tsx/dist/cli.mjs"
base_cli_entry_path="$base_cwd/cli/src/index.ts"
base_cli_files_present() {
[[ -f "$base_cli_runner_path" && -f "$base_cli_entry_path" ]]
}
base_cli_healthy() {
base_cli_files_present || return 1
(cd "$base_cwd" && node "$base_cli_runner_path" "$base_cli_entry_path" --help >/dev/null 2>&1)
}
repair_base_workspace_install() {
command -v pnpm >/dev/null 2>&1 || return 1
[[ -f "$base_cwd/package.json" && -f "$base_cwd/pnpm-lock.yaml" ]] || return 1
echo "Base workspace CLI at $base_cli_entry_path failed its health check (typically dangling pnpm symlinks after a partial install); repairing with pnpm install in $base_cwd." >&2
local repair_cmd=(pnpm install --prod=false --force --frozen-lockfile --config.confirmModulesPurge=false)
local repair_lock_dir=""
if command -v git >/dev/null 2>&1; then
repair_lock_dir="$(git -C "$base_cwd" rev-parse --absolute-git-dir 2>/dev/null || true)"
fi
if [[ ! -d "$repair_lock_dir" && -d "$base_cwd/.git" ]]; then
repair_lock_dir="$base_cwd/.git"
fi
if command -v flock >/dev/null 2>&1 && [[ -d "$repair_lock_dir" ]]; then
(
cd "$base_cwd" || exit 1
exec 9>"$repair_lock_dir/paperclip-provision-repair.lock"
flock 9
if base_cli_healthy; then
echo "Base workspace CLI became healthy while waiting for the repair lock; skipping reinstall." >&2
exit 0
fi
env -u NODE_ENV CI=true "${repair_cmd[@]}" >&2 || exit 1
base_cli_healthy
)
else
(cd "$base_cwd" && env -u NODE_ENV CI=true "${repair_cmd[@]}" >&2 && base_cli_healthy)
fi
}
ensure_base_cli_healthy() {
base_cli_files_present || return 1
base_cli_healthy && return 0
repair_base_workspace_install
}
run_ensure_seeded() {
if ensure_base_cli_healthy; then
(
cd "$worktree_cwd" &&
node "$base_cli_runner_path" "$base_cli_entry_path" worktree ensure-seeded --config "$worktree_config_path" "${source_config_args[@]}"
)
return
fi
if command -v pnpm >/dev/null 2>&1 && pnpm paperclipai --help >/dev/null 2>&1; then
(
cd "$worktree_cwd" &&
pnpm paperclipai worktree ensure-seeded --config "$worktree_config_path" "${source_config_args[@]}"
)
return
fi
if command -v paperclipai >/dev/null 2>&1; then
(
cd "$worktree_cwd" &&
paperclipai worktree ensure-seeded --config "$worktree_config_path" "${source_config_args[@]}"
)
return
fi
return 127
}
if run_ensure_seeded; then
exit 0
else
exit_code=$?
if [[ "$exit_code" -eq 127 ]]; then
echo "No usable paperclipai CLI found; cannot seed the worktree database." >&2
fi
exit "$exit_code"
fi