mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-06 21:05:21 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Isolated workspaces give each task a safe and reproducible checkout. > - The existing setup cloned the development database before an agent needed to run the app. > - This made worktree creation slower and heavier for tasks that never start a service. > - Runtime services already use one server start path for heartbeat, operator, and startup recovery flows. > - This pull request moves heavy setup to that start path and keeps worktree creation lean. > - The benefit is faster isolated workspace creation with the same reliable runtime setup when a service starts. ## Linked Issues or Issue Description Related pull request: #10652 covers the initial deferred database-seeding slice. This pull request supersedes it with end-to-end runtime provisioning and safe cleanup. **What existing behavior does this improve?** This improves isolated worktree creation, runtime service startup, and isolated instance cleanup. **Subsystem affected** Cross-cutting: CLI worktree setup, server runtime orchestration, shared workspace contracts, and development scripts. **Current behavior** Paperclip seeds an isolated development database during worktree creation. It can also leave an isolated instance directory after workspace teardown. This work happens even when no runtime service starts. **Proposed behavior** Paperclip creates the worktree with a lean eager setup. It runs an idempotent runtime provision command before the first managed service spawn. Concurrent starts share one provision attempt. Teardown removes the isolated instance safely. **Reason and benefit** Many agent tasks only edit and test code. They do not need a running Paperclip instance. Deferring the database seed reduces workspace startup cost while preserving automatic setup for tasks that start the app. **Breaking changes** None. The new runtime provision command is optional. Existing workspace behavior is unchanged when it is absent. ## What Changed - Split Paperclip worktree setup into a lean eager script and an idempotent runtime provision script. - Added `runtimeProvisionCommand` to project, issue, realized workspace, and persisted workspace contracts. - Added a per-workspace provision mutex before local service spawn for heartbeat, operator, and startup recovery flows. - Added a persisted `provisioning` service state and the `workspace_runtime_provision` operation phase. - Kept provision time outside the service readiness timeout and made failed attempts visible and retryable. - Reclaimed isolated instance data during safe workspace teardown. - Serialized deferred database seeding across processes and bound teardown to the instance root captured in persisted workspace metadata. - Added tests for config flow, concurrency, retry, no-op behavior, readiness timing, scripts, CLI commands, and cleanup. - Documented the eager and runtime provisioning contracts. ## Verification - `pnpm -r typecheck` - `pnpm build` - `pnpm test:run` (server: 3,201 passed; UI: 3,345 passed; the CLI phase exposed one environment-sensitive AWS doctor assertion because the agent runtime injects static AWS credentials) - `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY pnpm exec vitest run cli/src/__tests__/secrets.test.ts -t 'passes AWS doctor checks when non-secret provider config is present'` - Focused runtime tests cover serialized provisioning, retry after stderr failure, absent-command no-op behavior, operation logging, persisted state order, and readiness timeout exclusion. - Focused CLI and cleanup tests cover concurrent seed serialization, stale-lock fail-closed behavior, persisted instance ownership, and rewritten sibling pointers. ## Risks - A faulty runtime provision script blocks service startup. Paperclip records stderr, marks the service failed, and retries on the next start. - Concurrent service requests share an in-process provision attempt, while the seed command uses an atomic filesystem lock across processes. A stale lock fails closed and requires an operator to verify no seed is running before removing it. - Isolated instance cleanup is destructive. The cleanup service validates ownership and path containment before removal. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, `gpt-5.6-sol`, with agentic reasoning, tool use, and code execution. The service does not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
132 lines
4.3 KiB
Bash
Executable File
132 lines
4.3 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
|
|
base_cwd="${PAPERCLIP_WORKSPACE_BASE_CWD:?PAPERCLIP_WORKSPACE_BASE_CWD is required}"
|
|
worktree_cwd="${PAPERCLIP_WORKSPACE_CWD:?PAPERCLIP_WORKSPACE_CWD is required}"
|
|
paperclip_home="${PAPERCLIP_HOME:-$HOME/.paperclip}"
|
|
paperclip_instance_id="${PAPERCLIP_INSTANCE_ID:-default}"
|
|
paperclip_dir="$worktree_cwd/.paperclip"
|
|
worktree_config_path="$paperclip_dir/config.json"
|
|
seed_pending_marker_path="$paperclip_dir/seed-pending"
|
|
seed_complete_marker_path="$paperclip_dir/seed-complete"
|
|
|
|
if [[ ! -d "$base_cwd" ]]; then
|
|
echo "Base workspace does not exist: $base_cwd" >&2
|
|
exit 1
|
|
fi
|
|
|
|
if [[ ! -d "$worktree_cwd" ]]; then
|
|
echo "Derived worktree does not exist: $worktree_cwd" >&2
|
|
exit 1
|
|
fi
|
|
|
|
if [[ -e "$seed_complete_marker_path" || ! -e "$seed_pending_marker_path" ]]; then
|
|
echo "Worktree database is already seeded; skipping runtime provisioning." >&2
|
|
exit 0
|
|
fi
|
|
|
|
if [[ ! -f "$worktree_config_path" ]]; then
|
|
echo "Worktree config does not exist: $worktree_config_path" >&2
|
|
exit 1
|
|
fi
|
|
|
|
source_config_path="${PAPERCLIP_CONFIG:-}"
|
|
if [[ -z "$source_config_path" && ( -e "$base_cwd/.paperclip/config.json" || -L "$base_cwd/.paperclip/config.json" ) ]]; then
|
|
source_config_path="$base_cwd/.paperclip/config.json"
|
|
fi
|
|
if [[ -z "$source_config_path" ]]; then
|
|
source_config_path="$paperclip_home/instances/$paperclip_instance_id/config.json"
|
|
fi
|
|
source_config_args=(--from-config "$source_config_path")
|
|
if [[ "$source_config_path" == "$worktree_config_path" ]]; then
|
|
# A human may invoke this after sourcing `worktree env`, which points
|
|
# PAPERCLIP_CONFIG at the target. In that case the CLI reads the original
|
|
# source config from the seed-pending marker instead.
|
|
source_config_args=()
|
|
fi
|
|
|
|
base_cli_runner_path="$base_cwd/cli/node_modules/tsx/dist/cli.mjs"
|
|
base_cli_entry_path="$base_cwd/cli/src/index.ts"
|
|
|
|
base_cli_files_present() {
|
|
[[ -f "$base_cli_runner_path" && -f "$base_cli_entry_path" ]]
|
|
}
|
|
|
|
base_cli_healthy() {
|
|
base_cli_files_present || return 1
|
|
(cd "$base_cwd" && node "$base_cli_runner_path" "$base_cli_entry_path" --help >/dev/null 2>&1)
|
|
}
|
|
|
|
repair_base_workspace_install() {
|
|
command -v pnpm >/dev/null 2>&1 || return 1
|
|
[[ -f "$base_cwd/package.json" && -f "$base_cwd/pnpm-lock.yaml" ]] || return 1
|
|
echo "Base workspace CLI at $base_cli_entry_path failed its health check (typically dangling pnpm symlinks after a partial install); repairing with pnpm install in $base_cwd." >&2
|
|
local repair_cmd=(pnpm install --prod=false --force --frozen-lockfile --config.confirmModulesPurge=false)
|
|
local repair_lock_dir=""
|
|
if command -v git >/dev/null 2>&1; then
|
|
repair_lock_dir="$(git -C "$base_cwd" rev-parse --absolute-git-dir 2>/dev/null || true)"
|
|
fi
|
|
if [[ ! -d "$repair_lock_dir" && -d "$base_cwd/.git" ]]; then
|
|
repair_lock_dir="$base_cwd/.git"
|
|
fi
|
|
if command -v flock >/dev/null 2>&1 && [[ -d "$repair_lock_dir" ]]; then
|
|
(
|
|
cd "$base_cwd" || exit 1
|
|
exec 9>"$repair_lock_dir/paperclip-provision-repair.lock"
|
|
flock 9
|
|
if base_cli_healthy; then
|
|
echo "Base workspace CLI became healthy while waiting for the repair lock; skipping reinstall." >&2
|
|
exit 0
|
|
fi
|
|
env -u NODE_ENV CI=true "${repair_cmd[@]}" >&2 || exit 1
|
|
base_cli_healthy
|
|
)
|
|
else
|
|
(cd "$base_cwd" && env -u NODE_ENV CI=true "${repair_cmd[@]}" >&2 && base_cli_healthy)
|
|
fi
|
|
}
|
|
|
|
ensure_base_cli_healthy() {
|
|
base_cli_files_present || return 1
|
|
base_cli_healthy && return 0
|
|
repair_base_workspace_install
|
|
}
|
|
|
|
run_ensure_seeded() {
|
|
if ensure_base_cli_healthy; then
|
|
(
|
|
cd "$worktree_cwd" &&
|
|
node "$base_cli_runner_path" "$base_cli_entry_path" worktree ensure-seeded --config "$worktree_config_path" "${source_config_args[@]}"
|
|
)
|
|
return
|
|
fi
|
|
|
|
if command -v pnpm >/dev/null 2>&1 && pnpm paperclipai --help >/dev/null 2>&1; then
|
|
(
|
|
cd "$worktree_cwd" &&
|
|
pnpm paperclipai worktree ensure-seeded --config "$worktree_config_path" "${source_config_args[@]}"
|
|
)
|
|
return
|
|
fi
|
|
|
|
if command -v paperclipai >/dev/null 2>&1; then
|
|
(
|
|
cd "$worktree_cwd" &&
|
|
paperclipai worktree ensure-seeded --config "$worktree_config_path" "${source_config_args[@]}"
|
|
)
|
|
return
|
|
fi
|
|
|
|
return 127
|
|
}
|
|
|
|
if run_ensure_seeded; then
|
|
exit 0
|
|
else
|
|
exit_code=$?
|
|
if [[ "$exit_code" -eq 127 ]]; then
|
|
echo "No usable paperclipai CLI found; cannot seed the worktree database." >&2
|
|
fi
|
|
exit "$exit_code"
|
|
fi
|