mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 22:39:44 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Before an agent turn starts, the harness prepares the task's git worktree > - The harness runs a repository's `scripts/provision-worktree.sh` when the project sets no `provisionCommand` and the file exists > - The harness runs it on every turn, both when it creates the worktree and when it reuses it > - In the Paperclip repository, that script writes the worktree's isolated instance config and runs `pnpm install` > - Its "install needed" check compares against every `node_modules` directory in the base checkout, including other worktrees, so it reinstalls on every turn > - This pull request renames the Paperclip script so that the harness default does not find it in this repository > - The benefit is that Paperclip agent turns no longer wait on a 6–9 second install. Other repositories keep the harness feature. ## Linked Issues or Issue Description **What existing behavior does this improve?** Time to first token for agent turns in managed worktrees of the Paperclip repository. **Subsystem affected** Execution workspace provisioning (`server/src/services/workspace-runtime.ts`) and the repository script `scripts/provision-worktree.sh`. **Current behavior** Every agent turn in a Paperclip worktree first runs `bash ./scripts/provision-worktree.sh`. In a local test, a `pnpm install --frozen-lockfile` with nothing to change took 6.1 s and 6.3 s, and a full install from a warm store took 9.0 s. The script's fingerprint skip does not prevent this, because the script also requires every base-checkout `node_modules` path (up to depth 4) to exist in the worktree. On a developer machine that list had 1,735 entries, including other worktrees under `.claude/worktrees/` and `.worktrees/` and pnpm's internal `node_modules/.pnpm/*/node_modules`. A managed worktree never contains the other worktrees' paths, so the script always decides to reinstall. **Proposed behavior** The Paperclip repository does not ship a file at `scripts/provision-worktree.sh`. The setup script becomes `scripts/setup-paperclip-worktree.sh`. People can run it by hand or set it as the project `provisionCommand` to opt in. The harness default stays for other repositories. **Reason and benefit** Agents install dependencies when they need them (`node scripts/agent-preflight.mjs`, as `AGENTS.md` says). Many turns never run code. Instance setup stays available on demand: `scripts/provision-worktree-runtime.sh` runs the setup script when a dev server starts in a worktree that has no config. Database seeding was already deferred. **Breaking changes** A project that explicitly sets `provisionCommand` to `bash ./scripts/provision-worktree.sh` for this repository will fail with "file not found". Change it to `bash ./scripts/setup-paperclip-worktree.sh` or clear it. ## What Changed - Rename `scripts/provision-worktree.sh` to `scripts/setup-paperclip-worktree.sh`. The content is unchanged. - `scripts/provision-worktree-runtime.sh` calls the new name when the worktree has no config. - Tests read the real script from its new path (`provision-worktree-self-heal`, `workspace-runtime`, `worktree-seed-server-spawn`). Server fixtures still copy it into their test repositories as `scripts/provision-worktree.sh`, so the harness default stays covered. - `doc/DEVELOPING.md` says that the Paperclip repository deliberately has no `provision-worktree.sh`, and tells people how to opt in. ## Verification - `node --test scripts/__tests__/provision-worktree-self-heal.test.mjs`: 33 passed, 1 skipped (the skip already existed). - `cd server && npx vitest run src/__tests__/workspace-runtime.test.ts src/__tests__/heartbeat-workspace-source-trust.test.ts src/__tests__/execution-workspace-reopen.test.ts`: 242 passed on the rebased head. - `cd server && npx vitest run src/__tests__/worktree-seed-server-spawn.test.ts`: 2 passed. - Manual: in a Paperclip worktree, start an agent turn. The run log does not show a `workspace_provision` operation for the script. ## Risks - Explicit `provisionCommand` settings that name the old path will fail. See "Breaking changes". - Open pull requests that edit `scripts/provision-worktree.sh` (for example #16213, #11335, #11733, #19764) will conflict on the rename. Git rename detection should carry most edits over. - An agent that runs tests in a new worktree without first running preflight will see a missing-module error. `AGENTS.md` already tells agents to run `node scripts/agent-preflight.mjs`. - The script's install check still has the bug described above. It now runs only on demand. A separate fix can address it. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (shell, file edits). Standard context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
186 lines
7.2 KiB
Bash
Executable File
186 lines
7.2 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Keep this script compatible with macOS's system Bash 3.2.
|
|
set -euo pipefail
|
|
|
|
base_cwd="${PAPERCLIP_WORKSPACE_BASE_CWD:?PAPERCLIP_WORKSPACE_BASE_CWD is required}"
|
|
worktree_cwd="${PAPERCLIP_WORKSPACE_CWD:?PAPERCLIP_WORKSPACE_CWD is required}"
|
|
paperclip_home="${PAPERCLIP_HOME:-$HOME/.paperclip}"
|
|
paperclip_instance_id="${PAPERCLIP_INSTANCE_ID:-default}"
|
|
paperclip_dir="$worktree_cwd/.paperclip"
|
|
worktree_config_path="$paperclip_dir/config.json"
|
|
seed_manifest_path="$paperclip_dir/seed-manifest.json"
|
|
seed_empty_path="$paperclip_dir/seed-empty"
|
|
|
|
if [[ ! -d "$base_cwd" ]]; then
|
|
echo "Base workspace does not exist: $base_cwd" >&2
|
|
exit 1
|
|
fi
|
|
|
|
if [[ ! -d "$worktree_cwd" ]]; then
|
|
echo "Derived worktree does not exist: $worktree_cwd" >&2
|
|
exit 1
|
|
fi
|
|
|
|
if [[ -e "$seed_empty_path" ]]; then
|
|
echo "Worktree instance is explicitly empty; automatic database copying is disabled." >&2
|
|
exit 0
|
|
fi
|
|
|
|
if [[ -e "$seed_manifest_path" ]]; then
|
|
seed_manifest_state="$(SEED_MANIFEST_PATH="$seed_manifest_path" node <<'EOF'
|
|
const fs = require("node:fs");
|
|
try {
|
|
const value = JSON.parse(fs.readFileSync(process.env.SEED_MANIFEST_PATH, "utf8"));
|
|
const complete = value?.version === 2
|
|
&& value?.state === "verified"
|
|
&& value?.phase === "complete"
|
|
&& typeof value?.source?.instanceId === "string" && value.source.instanceId.length > 0
|
|
&& typeof value?.source?.configPath === "string" && value.source.configPath.length > 0
|
|
&& (value?.seedMode === "minimal" || value?.seedMode === "full")
|
|
&& typeof value?.snapshotAt === "string" && value.snapshotAt.length > 0
|
|
&& typeof value?.migrationRevision === "string" && value.migrationRevision.length > 0
|
|
&& typeof value?.targetInstanceId === "string" && value.targetInstanceId.length > 0
|
|
&& typeof value?.attemptId === "string" && value.attemptId.length > 0
|
|
&& typeof value?.startedAt === "string"
|
|
&& typeof value?.finishedAt === "string"
|
|
&& Array.isArray(value?.diagnostics)
|
|
&& value.diagnostics.some((entry) => entry?.phase === "complete" && entry?.status === "succeeded" && typeof entry?.at === "string");
|
|
process.stdout.write(complete ? "verified" : "incomplete");
|
|
} catch {
|
|
process.stdout.write("invalid");
|
|
}
|
|
EOF
|
|
)"
|
|
if [[ "$seed_manifest_state" == "verified" ]]; then
|
|
echo "Worktree database has a verified seed manifest; skipping runtime provisioning." >&2
|
|
exit 0
|
|
fi
|
|
fi
|
|
|
|
if [[ ! -f "$worktree_config_path" ]]; then
|
|
initial_provision_script="$base_cwd/scripts/setup-paperclip-worktree.sh"
|
|
if [[ ! -f "$initial_provision_script" ]]; then
|
|
echo "Worktree config does not exist and the built-in provision script is unavailable: $worktree_config_path" >&2
|
|
exit 1
|
|
fi
|
|
echo "Worktree config is missing; running the built-in worktree provisioner before database seeding." >&2
|
|
(
|
|
cd "$worktree_cwd" &&
|
|
bash "$initial_provision_script"
|
|
)
|
|
fi
|
|
|
|
if [[ ! -f "$worktree_config_path" ]]; then
|
|
echo "Worktree config still does not exist after built-in provisioning: $worktree_config_path" >&2
|
|
exit 1
|
|
fi
|
|
|
|
# The CLI derives the source from PAPERCLIP_WORKSPACE_BASE_CWD, which the control
|
|
# plane injects from the registered project-workspace row. A base workspace that is
|
|
# a plain checkout carries no instance config of its own, so name the control plane's
|
|
# own registered instance config explicitly. The seed manifest stays diagnostic
|
|
# evidence only and must never choose the clone source.
|
|
if [[ -L "$base_cwd/.paperclip" && ! -d "$base_cwd/.paperclip" ]]; then
|
|
echo "Registered base project workspace .paperclip is a broken symlink: $base_cwd/.paperclip" >&2
|
|
exit 1
|
|
fi
|
|
source_config_args=()
|
|
if [[ ! -e "$base_cwd/.paperclip/config.json" && ! -L "$base_cwd/.paperclip/config.json" ]]; then
|
|
source_config_path="${PAPERCLIP_CONFIG:-$paperclip_home/instances/$paperclip_instance_id/config.json}"
|
|
# A human may invoke this after sourcing `worktree env`, which points
|
|
# PAPERCLIP_CONFIG at the target. Naming the target as its own source is never
|
|
# right, so leave the source to the CLI in that case.
|
|
if [[ "$source_config_path" != "$worktree_config_path" ]]; then
|
|
source_config_args=(--from-config "$source_config_path")
|
|
fi
|
|
fi
|
|
|
|
base_cli_runner_path="$base_cwd/cli/node_modules/tsx/dist/cli.mjs"
|
|
base_cli_entry_path="$base_cwd/cli/src/index.ts"
|
|
|
|
base_cli_files_present() {
|
|
[[ -f "$base_cli_runner_path" && -f "$base_cli_entry_path" ]]
|
|
}
|
|
|
|
base_cli_healthy() {
|
|
base_cli_files_present || return 1
|
|
(cd "$base_cwd" && node "$base_cli_runner_path" "$base_cli_entry_path" --help >/dev/null 2>&1)
|
|
}
|
|
|
|
repair_base_workspace_install() {
|
|
command -v pnpm >/dev/null 2>&1 || return 1
|
|
[[ -f "$base_cwd/package.json" && -f "$base_cwd/pnpm-lock.yaml" ]] || return 1
|
|
echo "Base workspace CLI at $base_cli_entry_path failed its health check (typically dangling pnpm symlinks after a partial install); repairing with pnpm install in $base_cwd." >&2
|
|
local repair_cmd=(pnpm install --prod=false --force --frozen-lockfile --config.confirmModulesPurge=false)
|
|
# pnpm 9.15.4 calls the deprecated url.parse() in toNerfDart on every
|
|
# install. Node 24 reports that call as DEP0169. Remove this flag when the
|
|
# pinned pnpm no longer calls url.parse() in that path.
|
|
local repair_node_options="${NODE_OPTIONS:-} --disable-warning=DEP0169"
|
|
local repair_lock_dir=""
|
|
if command -v git >/dev/null 2>&1; then
|
|
repair_lock_dir="$(git -C "$base_cwd" rev-parse --absolute-git-dir 2>/dev/null || true)"
|
|
fi
|
|
if [[ ! -d "$repair_lock_dir" && -d "$base_cwd/.git" ]]; then
|
|
repair_lock_dir="$base_cwd/.git"
|
|
fi
|
|
if command -v flock >/dev/null 2>&1 && [[ -d "$repair_lock_dir" ]]; then
|
|
(
|
|
cd "$base_cwd" || exit 1
|
|
exec 9>"$repair_lock_dir/paperclip-provision-repair.lock"
|
|
flock 9
|
|
if base_cli_healthy; then
|
|
echo "Base workspace CLI became healthy while waiting for the repair lock; skipping reinstall." >&2
|
|
exit 0
|
|
fi
|
|
env -u NODE_ENV CI=true NODE_OPTIONS="$repair_node_options" "${repair_cmd[@]}" >&2 || exit 1
|
|
base_cli_healthy
|
|
)
|
|
else
|
|
(cd "$base_cwd" && env -u NODE_ENV CI=true NODE_OPTIONS="$repair_node_options" "${repair_cmd[@]}" >&2 && base_cli_healthy)
|
|
fi
|
|
}
|
|
|
|
ensure_base_cli_healthy() {
|
|
base_cli_files_present || return 1
|
|
base_cli_healthy && return 0
|
|
repair_base_workspace_install
|
|
}
|
|
|
|
run_ensure_seeded() {
|
|
if ensure_base_cli_healthy; then
|
|
(
|
|
cd "$worktree_cwd" &&
|
|
node "$base_cli_runner_path" "$base_cli_entry_path" worktree ensure-seeded --config "$worktree_config_path" ${source_config_args[@]+"${source_config_args[@]}"}
|
|
)
|
|
return
|
|
fi
|
|
|
|
if command -v pnpm >/dev/null 2>&1 && pnpm paperclipai --help >/dev/null 2>&1; then
|
|
(
|
|
cd "$worktree_cwd" &&
|
|
pnpm paperclipai worktree ensure-seeded --config "$worktree_config_path" ${source_config_args[@]+"${source_config_args[@]}"}
|
|
)
|
|
return
|
|
fi
|
|
|
|
if command -v paperclipai >/dev/null 2>&1; then
|
|
(
|
|
cd "$worktree_cwd" &&
|
|
paperclipai worktree ensure-seeded --config "$worktree_config_path" ${source_config_args[@]+"${source_config_args[@]}"}
|
|
)
|
|
return
|
|
fi
|
|
|
|
return 127
|
|
}
|
|
|
|
if run_ensure_seeded; then
|
|
exit 0
|
|
else
|
|
exit_code=$?
|
|
if [[ "$exit_code" -eq 127 ]]; then
|
|
echo "No usable paperclipai CLI found; cannot seed the worktree database." >&2
|
|
fi
|
|
exit "$exit_code"
|
|
fi
|