From a3a25ce6c947f0666eaa1233fa373e6f9024aeaf Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Mon, 9 Nov 2026 08:03:24 -0600 Subject: [PATCH] fix(workspaces): stop running Paperclip's worktree setup before every agent turn (#20269) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Before an agent turn starts, the harness prepares the task's git worktree > - The harness runs a repository's `scripts/provision-worktree.sh` when the project sets no `provisionCommand` and the file exists > - The harness runs it on every turn, both when it creates the worktree and when it reuses it > - In the Paperclip repository, that script writes the worktree's isolated instance config and runs `pnpm install` > - Its "install needed" check compares against every `node_modules` directory in the base checkout, including other worktrees, so it reinstalls on every turn > - This pull request renames the Paperclip script so that the harness default does not find it in this repository > - The benefit is that Paperclip agent turns no longer wait on a 6–9 second install. Other repositories keep the harness feature. ## Linked Issues or Issue Description **What existing behavior does this improve?** Time to first token for agent turns in managed worktrees of the Paperclip repository. **Subsystem affected** Execution workspace provisioning (`server/src/services/workspace-runtime.ts`) and the repository script `scripts/provision-worktree.sh`. **Current behavior** Every agent turn in a Paperclip worktree first runs `bash ./scripts/provision-worktree.sh`. In a local test, a `pnpm install --frozen-lockfile` with nothing to change took 6.1 s and 6.3 s, and a full install from a warm store took 9.0 s. The script's fingerprint skip does not prevent this, because the script also requires every base-checkout `node_modules` path (up to depth 4) to exist in the worktree. On a developer machine that list had 1,735 entries, including other worktrees under `.claude/worktrees/` and `.worktrees/` and pnpm's internal `node_modules/.pnpm/*/node_modules`. A managed worktree never contains the other worktrees' paths, so the script always decides to reinstall. **Proposed behavior** The Paperclip repository does not ship a file at `scripts/provision-worktree.sh`. The setup script becomes `scripts/setup-paperclip-worktree.sh`. People can run it by hand or set it as the project `provisionCommand` to opt in. The harness default stays for other repositories. **Reason and benefit** Agents install dependencies when they need them (`node scripts/agent-preflight.mjs`, as `AGENTS.md` says). Many turns never run code. Instance setup stays available on demand: `scripts/provision-worktree-runtime.sh` runs the setup script when a dev server starts in a worktree that has no config. Database seeding was already deferred. **Breaking changes** A project that explicitly sets `provisionCommand` to `bash ./scripts/provision-worktree.sh` for this repository will fail with "file not found". Change it to `bash ./scripts/setup-paperclip-worktree.sh` or clear it. ## What Changed - Rename `scripts/provision-worktree.sh` to `scripts/setup-paperclip-worktree.sh`. The content is unchanged. - `scripts/provision-worktree-runtime.sh` calls the new name when the worktree has no config. - Tests read the real script from its new path (`provision-worktree-self-heal`, `workspace-runtime`, `worktree-seed-server-spawn`). Server fixtures still copy it into their test repositories as `scripts/provision-worktree.sh`, so the harness default stays covered. - `doc/DEVELOPING.md` says that the Paperclip repository deliberately has no `provision-worktree.sh`, and tells people how to opt in. ## Verification - `node --test scripts/__tests__/provision-worktree-self-heal.test.mjs`: 33 passed, 1 skipped (the skip already existed). - `cd server && npx vitest run src/__tests__/workspace-runtime.test.ts src/__tests__/heartbeat-workspace-source-trust.test.ts src/__tests__/execution-workspace-reopen.test.ts`: 242 passed on the rebased head. - `cd server && npx vitest run src/__tests__/worktree-seed-server-spawn.test.ts`: 2 passed. - Manual: in a Paperclip worktree, start an agent turn. The run log does not show a `workspace_provision` operation for the script. ## Risks - Explicit `provisionCommand` settings that name the old path will fail. See "Breaking changes". - Open pull requests that edit `scripts/provision-worktree.sh` (for example #16213, #11335, #11733, #19764) will conflict on the rename. Git rename detection should carry most edits over. - An agent that runs tests in a new worktree without first running preflight will see a missing-module error. `AGENTS.md` already tells agents to run `node scripts/agent-preflight.mjs`. - The script's install check still has the bug described above. It now runs only on demand. A separate fix can address it. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (shell, file edits). Standard context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Paperclip --- doc/DEVELOPING.md | 7 ++++--- scripts/__tests__/provision-worktree-self-heal.test.mjs | 6 +++--- scripts/provision-worktree-runtime.sh | 2 +- .../{provision-worktree.sh => setup-paperclip-worktree.sh} | 0 server/src/__tests__/workspace-runtime.test.ts | 4 ++-- server/src/__tests__/worktree-seed-server-spawn.test.ts | 2 +- 6 files changed, 11 insertions(+), 10 deletions(-) rename scripts/{provision-worktree.sh => setup-paperclip-worktree.sh} (100%) diff --git a/doc/DEVELOPING.md b/doc/DEVELOPING.md index 5dd82a8c312..8318e22e2c4 100644 --- a/doc/DEVELOPING.md +++ b/doc/DEVELOPING.md @@ -2597,7 +2597,7 @@ The default `worktree init` still seeds eagerly. A lean worktree (created withou - `pnpm paperclipai worktree ensure-seeded` performs the deferred seed **exactly once**. It is lock-guarded and idempotent: only a complete `verified` manifest short-circuits it, so it is safe to call repeatedly and from concurrent processes. Managed workspaces derive the source from the control-plane-provided base project workspace when it carries its own `.paperclip/config.json`, and otherwise from the control plane's own registered instance config; either way the workspace's manifest never selects it. Manual worktrees must pass `--from-config`. - `paperclipai run` calls `ensureWorktreeSeeded` automatically before doctor/boot. Managed runs transparently seed a lean worktree from their registered base workspace; an unmanaged lean worktree must first run `worktree ensure-seeded --from-config `. -- Managed Paperclip git worktrees default to the repository's `scripts/provision-worktree.sh` when the strategy omits `provisionCommand`. When a registered source config exists, setup creates the isolated config and pending manifest. Runtime startup also runs `scripts/provision-worktree-runtime.sh` automatically when no explicit runtime provision command is configured and the manifest is not verified. Explicitly configured provision commands still take precedence. +- Managed git worktrees run a repository's `scripts/provision-worktree.sh` when the strategy omits `provisionCommand` and that file exists. The Paperclip repository deliberately does not ship a file at that path, so agent turns in Paperclip worktrees do not wait on dependency installs or instance setup. Its setup script is `scripts/setup-paperclip-worktree.sh`: run it by hand, or set it as the project's `provisionCommand` to opt in. When a registered source config exists, it creates the isolated config and pending manifest, then installs dependencies. Runtime startup still runs `scripts/provision-worktree-runtime.sh` automatically when no explicit runtime provision command is configured and the manifest is not verified; that script runs `scripts/setup-paperclip-worktree.sh` first when the worktree has no config. Explicitly configured provision commands still take precedence. - An environment-configured server may have no local seed config. If neither the base checkout nor the default control-plane instance has one, a fresh worktree prepares its dependencies without creating a development instance. This also applies when `PAPERCLIP_CONFIG` names the default `$PAPERCLIP_HOME/instances/$PAPERCLIP_INSTANCE_ID/config.json` path, as the Docker image does. Setup creates no config, environment file, or seed manifest and does not claim runtime or seed readiness. A later request for a seeded development runtime still needs a canonical registered source config. Once that source exists, provisioning the checkout again creates the development instance normally. - A missing custom `PAPERCLIP_CONFIG`, rejected symlink, or non-regular source file still fails setup. An existing worktree with a config, environment file, or seed state also fails if its source disappears; setup never downgrades that instance to a plain checkout. Repair the source before retrying. - The built-in deferred seed is recorded as its own terminal `workspace_seed` operation. A zero exit code is not enough for success: the operation succeeds only when `.paperclip/seed-manifest.json` contains complete verified evidence; failed, missing, or malformed manifests produce a failed operation with the seed phase in metadata. @@ -4690,8 +4690,9 @@ whether or not anything is left to compact. A compacted worktree is in the same state as a freshly created one, so when the task resumes, whatever produced those files the first time produces them again: -reuse runs the workspace provision command (or `scripts/provision-worktree.sh`), -which reinstalls dependencies when `node_modules` is missing. Repositories without +reuse runs the workspace provision command (or a repository's +`scripts/provision-worktree.sh`), which can reinstall dependencies when +`node_modules` is missing. Repositories without a provision command rely on the agent's own install step, as they do for a new worktree. diff --git a/scripts/__tests__/provision-worktree-self-heal.test.mjs b/scripts/__tests__/provision-worktree-self-heal.test.mjs index 0219bf0e820..201a5daf1d0 100644 --- a/scripts/__tests__/provision-worktree-self-heal.test.mjs +++ b/scripts/__tests__/provision-worktree-self-heal.test.mjs @@ -5,7 +5,7 @@ import fs from "node:fs"; import os from "node:os"; import path from "node:path"; -const script = new URL("../provision-worktree.sh", import.meta.url).pathname; +const script = new URL("../setup-paperclip-worktree.sh", import.meta.url).pathname; const runtimeScript = new URL("../provision-worktree-runtime.sh", import.meta.url).pathname; // Keep the PATH minimal so the fallback ladder is deterministic: node must be @@ -233,7 +233,7 @@ test("initializes a previously plain checkout once a registered seed source is a test("a plain checkout without a source is not ready for a seeded runtime", () => { const baseCwd = makeBaseWorkspace({ helpExit: 0, initExit: 0 }); fs.mkdirSync(path.join(baseCwd, "scripts")); - fs.copyFileSync(script, path.join(baseCwd, "scripts", "provision-worktree.sh")); + fs.copyFileSync(script, path.join(baseCwd, "scripts", "setup-paperclip-worktree.sh")); const setupInstance = (home) => fs.rmSync(path.join(home, "instances"), { recursive: true }); const { result, worktreeCwd } = runProvision(baseCwd, { setupInstance }); assert.equal(result.status, 0, result.stderr); @@ -568,7 +568,7 @@ test("runtime provisioning seeds a worktree config that has no seed markers", () test("runtime provisioning bootstraps and seeds an empty .paperclip directory", () => { const baseCwd = makeBaseWorkspace({ helpExit: 0, initExit: 0 }); fs.mkdirSync(path.join(baseCwd, "scripts"), { recursive: true }); - fs.copyFileSync(script, path.join(baseCwd, "scripts", "provision-worktree.sh")); + fs.copyFileSync(script, path.join(baseCwd, "scripts", "setup-paperclip-worktree.sh")); const worktreeCwd = makeTempDir("paperclip-provision-runtime-empty-state-"); fs.mkdirSync(path.join(worktreeCwd, ".paperclip"), { recursive: true }); diff --git a/scripts/provision-worktree-runtime.sh b/scripts/provision-worktree-runtime.sh index 534f537862c..f1cb2cb5edd 100755 --- a/scripts/provision-worktree-runtime.sh +++ b/scripts/provision-worktree-runtime.sh @@ -58,7 +58,7 @@ EOF fi if [[ ! -f "$worktree_config_path" ]]; then - initial_provision_script="$base_cwd/scripts/provision-worktree.sh" + initial_provision_script="$base_cwd/scripts/setup-paperclip-worktree.sh" if [[ ! -f "$initial_provision_script" ]]; then echo "Worktree config does not exist and the built-in provision script is unavailable: $worktree_config_path" >&2 exit 1 diff --git a/scripts/provision-worktree.sh b/scripts/setup-paperclip-worktree.sh similarity index 100% rename from scripts/provision-worktree.sh rename to scripts/setup-paperclip-worktree.sh diff --git a/server/src/__tests__/workspace-runtime.test.ts b/server/src/__tests__/workspace-runtime.test.ts index 4e4ca127631..00a438cc03e 100644 --- a/server/src/__tests__/workspace-runtime.test.ts +++ b/server/src/__tests__/workspace-runtime.test.ts @@ -148,7 +148,7 @@ if (!embeddedPostgresSupport.supported) { `Skipping embedded Postgres workspace-runtime tests on this host: ${embeddedPostgresSupport.reason ?? "unsupported environment"}`, ); } -const provisionWorktreeScriptPath = new URL("../../../scripts/provision-worktree.sh", import.meta.url); +const provisionWorktreeScriptPath = new URL("../../../scripts/setup-paperclip-worktree.sh", import.meta.url); async function runGit(cwd: string, args: string[]) { await execFileAsync("git", args, { cwd }); @@ -2045,7 +2045,7 @@ process.stderr.write(${JSON.stringify(stderr)}, () => { process.exitCode = ${cod await fs.mkdir(path.join(repoRoot, "scripts"), { recursive: true }); await fs.copyFile( - fileURLToPath(new URL("../../../scripts/provision-worktree.sh", import.meta.url)), + fileURLToPath(new URL("../../../scripts/setup-paperclip-worktree.sh", import.meta.url)), path.join(repoRoot, "scripts", "provision-worktree.sh"), ); await runGit(repoRoot, ["add", "scripts/provision-worktree.sh"]); diff --git a/server/src/__tests__/worktree-seed-server-spawn.test.ts b/server/src/__tests__/worktree-seed-server-spawn.test.ts index 3d697c046e2..e5607ba28b3 100644 --- a/server/src/__tests__/worktree-seed-server-spawn.test.ts +++ b/server/src/__tests__/worktree-seed-server-spawn.test.ts @@ -174,7 +174,7 @@ describe("managed worktree seed source through the server spawn path", () => { await fs.mkdir(path.join(repoRoot, "scripts"), { recursive: true }); await fs.writeFile(path.join(repoRoot, "README.md"), "server spawn regression\n", "utf8"); await fs.copyFile( - fileURLToPath(new URL("../../../scripts/provision-worktree.sh", import.meta.url)), + fileURLToPath(new URL("../../../scripts/setup-paperclip-worktree.sh", import.meta.url)), path.join(repoRoot, "scripts", "provision-worktree.sh"), ); await fs.chmod(path.join(repoRoot, "scripts", "provision-worktree.sh"), 0o755);