3 Commits

Include renames

Author	SHA1	Message	Date
DottaandPaperclip	2aca41464b	fix(workspaces): preserve compaction and expire standard Git bases (#16476) ... ## Thinking Path > - Paperclip manages AI agents and their task workspaces. > - Finished tasks can release reproducible Git files and restore them later. > - Unique work must stay on disk, while regenerable dependencies can be removed. > - Linux Git templates include an empty legacy directory that blocked base cleanup. > - The new expiration branch also skipped the existing guarded dependency cleanup. > - This change accepts the standard empty directory and retains that cleanup order. > - Private bases can expire, and protected workspaces still shed regenerable files. ## Linked Issues or Issue Description Refs: #16457 Live acceptance of #16457 removed a worktree but retained its private base because Linux Git created an empty `.git/branches` directory. A pending-base dry run then tried to resolve the already removed worktree and hid the preservation reason. A local integration comparison also showed that enabling Git expiration skipped existing dependency compaction. It deferred pending startup compaction by an hour. Both failures were reproduced before the fix. ## What Changed - Accept `.git/branches` only when it is an empty real directory. Preserve populated entries, files, and symlinks. - Inspect pending base cleanup without resolving an absent worktree. - Run existing guarded compaction before Git expiration. Exclude already closed rows from compaction and retain its pending deadline. - Keep failed expiration checks from reaching legacy whole-workspace deletion. Existing compaction remains independent of Git expiration dry-run mode. - Extend real Git/PostgreSQL tests and retain the acceptance failures in the delivery state note. ## Verification - Linux-template baseline: private-base expiration failed before the fix. The first sandboxed attempt could not bind PostgreSQL; the authorized retry reproduced the behavior. - Compaction baseline: two regression cases failed in 7.67 seconds. They reproduced skipped dependency cleanup and the lost startup deadline. - After the fixes, `pnpm exec vitest run server/src/__tests__/execution-workspace-compaction.test.ts server/src/__tests__/git-workspace-expiration.test.ts server/src/__tests__/terminal-workspace-work.test.ts` passed all 49 cases in 43.79 seconds. - Controls cover unique work, active runs, unfinished descendants, previews, concurrent reopening, interrupted removal, exact-commit restoration, and retryable upstream failures. - Full `pnpm -r typecheck` and `pnpm build` passed for the final commit. All exact-head CI and security checks passed. Greptile is 5/5 with all findings resolved. - Final staging acceptance will let the restarted worker resume the retained base cleanup, then reopen normally at the recorded commit and persist subsequent work. The base has not been manually removed. The superseded preview build was cancelled before dispatching a replacement. ## Risks - Legacy branch definitions can hold unique data. Only an empty real directory qualifies for deletion. - Compaction preserves its existing lifecycle guard and regenerable-directory allowlist. It cannot fall through to legacy deletion after an expiration failure. - Already closed workspaces must only resume their recorded expiration; they do not enter compaction. ## Model Used OpenAI Codex, GPT-6.1-sol, with reasoning, code execution, Git, database, and browser tools. The exact context-window size is not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-15 14:36:00 -05:00
Devin FoleyandPaperclip	aec80cbc8c	fix(server): keep workspace compaction out of the startup sweep (#16328) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - When the server starts, `/api/health` answers "starting" until startup recovery finishes, and deploy tooling waits for "ok" > - Startup recovery waits for the first sweep of the terminal-workspace worker > - #16304 added workspace compaction to that sweep. Compaction walks worktrees, runs Git, and deletes dependency trees, and it re-checks every eligible workspace after a restart > - Compaction is never needed to serve requests, so it should not delay readiness > - This pull request skips compaction in the startup sweep only; later sweeps compact as before > - The benefit is that a compaction backlog cannot make a restart slower ## Linked Issues or Issue Description Follow-up to #16304. **What existing behavior does this improve?** Server startup time after #16304. **Current behavior** `await terminalWorkspaceWorker.ready` blocks the readiness flip until the first terminal-workspace sweep ends. That sweep compacts every eligible workspace in its page. The per-cycle compaction memo is in memory, so after each restart every eligible workspace is walked and checked with Git again, and any directories found are deleted before the sweep ends. **Proposed behavior** The first sweep skips compaction. Pending compactions keep their deadlines, so the worker wakes and a later sweep compacts them. **Reason and benefit** Readiness depends only on work that is needed to serve requests. On one measured instance the first sweep took 5–8 s with compaction; with a larger backlog, compaction could add seconds per workspace. **Breaking changes** None. Compaction happens one sweep later after a restart. ## What Changed - `sweepTerminalWorkspaces(limit, signal, { compact })`: `compact: false` skips compaction for that sweep and keeps each pending compaction deadline. - `registerTerminalWorkspaceWork` passes `compact: false` to its first sweep (the one readiness waits for) and `compact: true` after that. - `server/src/index.ts` forwards the option through the logging wrapper. - `doc/DEVELOPING.md`: the compaction section says the startup sweep never compacts. ## Verification - `pnpm exec vitest run src/__tests__/terminal-workspace-work.test.ts src/__tests__/execution-workspace-compaction.test.ts src/__tests__/workspace-compaction.test.ts` (in `server/`): 35 passed. - New: the worker passes `compact: false` only to the startup sweep and `compact: true` to the next one. - New: a sweep with `compact: false` leaves the files in place, the compaction deadline stays due, and the next sweep compacts. - `pnpm typecheck` in `server/`: passed. ## Risks - Low risk. After a restart, compaction runs a few seconds later than before. Archive behavior does not change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 15:27:28 -07:00
Devin FoleyandPaperclip	5b29b9062e	fix(server): compact finished task worktrees and shorten the reaper cooldown (#16304) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Each task can run in its own isolated Git worktree, and each worktree installs its own dependencies > - The terminal-workspace reaper removes a worktree only when the task tree is finished, the delivery is merged, the files are clean, and a 7-day cooldown has passed > - A recurring task that runs every hour keeps a week of worktrees, and each worktree holds gigabytes of `node_modules`. A finished task with no merged pull request, or with leftover agent files, keeps its worktree forever > - On a busy instance this fills the disk in days, and a larger volume only delays the next failure > - This pull request adds compaction: one hour after a task tree finishes, the reaper removes regenerable directories from the worktree and keeps the workspace, the branch, and all source changes > - The benefit is that finished tasks stop holding gigabytes of disk, and no work is lost because a resumed task regenerates the files when its workspace is provisioned again ## Linked Issues or Issue Description Refs #2022 (retention for workspace cleanup). Related: #11936 and #14608 add external reclaim scripts (a scheduled tool and a disk-guard step). This pull request does the reclaim inside the server's terminal-workspace reaper instead. That way it uses the reaper's lifecycle lock, issue-tree checks, and live-run checks, and needs no extra scheduler. **What existing behavior does this improve?** Disk retention of execution workspaces after their tasks finish. **Current behavior** A finished task keeps its full worktree, including installed dependencies and build output, for at least 7 days. If the delivery is not merged, or the worktree has uncommitted or untracked files, the reaper never removes it. On one instance, an hourly recurring task created about 22 worktrees per day at about 3 GB each (about 2.8 GB of that was `node_modules`). Together with finished tasks that had no merged pull request, they filled a 100 GB volume in about two days. **Proposed behavior** One hour after a task tree becomes terminal, the reaper removes regenerable directories (installed dependencies, build output, tool caches) from the worktree. It keeps the worktree, the branch, the commits, and every tracked, untracked, or other ignored file. When the task resumes, the existing reuse path runs the provision step, which reinstalls the dependencies. Clean, merged worktrees are archived after 1 day instead of 7. **Reason and benefit** Most of a finished worktree's size is regenerable. Compaction removes that part without the risk of deleting work, so it can apply to workspaces that the reaper must keep (no merged pull request, leftover files). The shorter cooldown limits the cost of the source checkouts that stay. **Breaking changes** None. The default archive cooldown changes from 7 days to 1 day. `PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS` still overrides it. `PAPERCLIP_WORKSPACE_COMPACTION_ENABLED=false` turns compaction off. ## What Changed - New `server/src/services/workspace-compaction.ts`: - Finds directories with a regenerable name (`node_modules`, `dist`, `build`, `coverage`, `storybook-static`, `target`, `.cache`, `.next`, `.nuxt`, `.parcel-cache`, `.svelte-kit`, `.turbo`, `.vite`). - Keeps only the directories that the repository's own `.gitignore` files ignore and that hold no tracked file. A negated pattern (`!build/`), a global excludes file, and `.git/info/exclude` do not count. Each candidate is checked per path, so dependencies below an ignored parent (`vendor/tool/node_modules`) are found. - Searches inside a regenerable-named directory that does not qualify (for example a tracked `build/`), so its ignored dependencies still go. - Never enters `.git` or `.paperclip` (the worktree instance data), and never follows a symlink. - Each Git read has a 60-second deadline and stops when the sweep is aborted. - Moves the directories into `.paperclip-compaction-trash` beside the worktree with a same-filesystem rename, and can restore them. - `sweepTerminalWorkspaces` compacts each terminal workspace once per terminal cycle, after the compaction delay and before the archive cooldown check: - Only isolated linked Git worktrees qualify. A shared workspace, a project's main checkout, or a row whose `.git` is a directory is never compacted. - The move runs inside a transaction that holds the workspace lifecycle lock. The guard checks a fresh row and the same lifecycle generation. It also checks that the issue tree is terminal, that no queued or running run works on the tree or the workspace (by checkout or execution run, or by run context), and that no runtime service runs from the workspace. - The guard runs before the move and again after it. If the second check fails (for example, the task reopened or a run started), the directories move back. Otherwise the trash is deleted. - The worker wakes at the compaction deadline, not only at the archive deadline. - Stale trash from an interrupted compaction or a failed delete is purged by later sweeps. The sweep checks each worktree parent at most every 30 minutes, whether or not anything is left to compact. - New counters `compacted` and `compactionFailed` in the sweep result. The activity log gets an `execution_workspace.regenerable_files_removed` entry with the removed paths. - Config: new `PAPERCLIP_WORKSPACE_COMPACTION_DELAY_HOURS` (default `1`) and `PAPERCLIP_WORKSPACE_COMPACTION_ENABLED`. The default for `PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS` changes from `7` to `1`. Both new keys are added to the run-failure diagnostics allowlists. - Docs: `doc/DEVELOPING.md` documents the new variables and has a new "Terminal workspace compaction" section. `doc/SPEC-implementation.md` describes compaction. - `execution-workspaces.ts` was already above 1,000 lines. The file-system logic is in the new module. Only the guarded orchestration, which uses the existing lifecycle lock and run predicates, is in the service. ## Verification - `pnpm exec vitest run src/__tests__/workspace-compaction.test.ts src/__tests__/execution-workspace-compaction.test.ts` (in `server/`): 27 passed. `workspace-reaper-cooldown-config.test.ts`: passed. - Unit tests: directory selection (ignored, untracked, regenerable name only, `.env` and local data kept, tracked `coverage` kept, `.paperclip` skipped, symlinks not followed), nested candidates under a source `build/` and under an ignored `vendor/`, a negated `!build/` kept, global and `info/exclude` rules ignored, glob characters in names, abort, main checkout vs linked worktree, move and restore, restore that keeps a recreated directory, and stale-trash purge. - Embedded Postgres tests: - A no-PR, unmerged, dirty workspace is compacted. HEAD, the branch ref, `git status --porcelain --untracked-files=all`, untracked notes, `.env`, local data, and `.paperclip` data stay the same, and the row stays active. - A delivered workspace inside the cooldown is compacted. - The delay is respected, and the worker wakes at the delay. - An active checkout run, a queued run found through its context, an unfinished child issue, and a running runtime service all block compaction. - The task reopens during the move: the files are restored and unchanged. - A run starts during the move: the files are restored. - A shared workspace and a main checkout are never compacted. - Compaction runs once per terminal cycle and again after a later cycle. - Disabled compaction does nothing. - End to end: `realizeExecutionWorkspace` creates a worktree, the sweep compacts it, and a second `realizeExecutionWorkspace` reuses the same worktree. The provision command regenerates `node_modules`, and the untracked notes and the unmerged commit are still present. - Leftover trash is purged when the workspace has nothing left to compact. - Mutation checks: when the post-move guard is disabled, both race tests fail. Without the negation fix or the repository-only source fix, their tests fail. - `pnpm exec vitest run` for `execution-workspaces-service.test.ts`, `terminal-workspace-work.test.ts`, `execution-workspace-reopen.test.ts`, `server-startup-feedback-export.test.ts`, and `workspace-runtime.test.ts`: 335 passed. Two `workspace-runtime.test.ts` cases fail locally with and without this change (`does not diagnose a missing remote branch as a rejected credential`, `rejects explicit main on a master-only remote ...`). They also fail on unmodified `master` on the same machine, so the cause is the local Git environment. - `pnpm typecheck` in `server/`: passed. `pnpm check:module-boundaries`: passed. ## Risks - A repository can keep something that is not regenerable in an ignored `dist`, `build`, `target`, or `.cache` directory. Compaction removes it. The fixed name list, the ignore check, and the tracked-file check limit this. `PAPERCLIP_WORKSPACE_COMPACTION_ENABLED=false` turns compaction off. - A repository without a provision command does not get dependencies back automatically. This is the same state as a new worktree for that repository, where the agent already installs them. - A resumed task pays for a dependency install again. This happens only for tasks that finished more than an hour earlier. - The shorter default cooldown removes clean, merged worktrees after 1 day instead of 7. An operator who wants the old hold can set `PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS=7`. - Self-hosted instances get the same behavior. There is no schema change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, test runs). A separate Claude agent read the reaper code during the diagnosis. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 09:49:04 -07:00