DottaandPaperclip
59a70dc9e2
## Thinking Path > - Paperclip manages AI agents and their task workspaces. > - Isolated Git worktrees and private repository copies remain on the host after tasks finish. > - These files can fill the disk even when every commit is available upstream. > - Cleanup must retain unique files, local Git state, live work, and recovery data. > - This pull request records and verifies an exact-commit restoration recipe before expiration. > - Normal reopening restores that commit even after upstream advances. > - The benefit is lower host disk use while finished tasks remain available for later work. ## Linked Issues or Issue Description Refs #10285 and #2022. Follows #16433 for skill-cache expiration. Related #11936 provides separate host-local cleanup tooling; this change extends the existing company-scoped lifecycle worker and normal workspace reopening. ROADMAP.md does not specify a competing implementation. ## What Changed - Add opt-in expiration for verified Paperclip-owned Git worktrees after their whole task tree has been done or cancelled for 24 hours. - Preserve active and scheduled runs, previews, pending reopening, unresolved recovery, dirty files, unknown ignored files, and unsupported or unique Git state. - Verify an authenticated fetch in a fresh object store. Save repository identity, exact commit, branch, and ownership before any deletion. - Rename eligible files into recorded managed trash. Resume interrupted cleanup under source and lifecycle locks. - Expire private base clones only after consumer, file, reference, reflog, and object checks pass. Preserve shared project base clones. - Rebuild missing bases and worktrees through normal reopening at the recorded commit. Return an explicit retryable error when repository access fails. - Add independent workspace/base switches, dry-run reports, reclaimed-byte counts, and operational documentation. No table, public API, UI, or recovery-retention change. ## Verification - Real local bare remote plus Postgres: create at A, advance upstream to B, expire the worktree and private base, reopen at A, then make and retain a subsequent commit. - Final lifecycle suite: all 20 tests passed in 46.41 seconds at `d72947a2e45be5d0c912debb03c46b15863b4798`. Review fixes cover second-cycle expiration, newer-branch recovery, normal preparation after restoration, branch adoption, and Git administration path controls. Eleven existing branch reconciliation controls also passed. Covers unique files and Git state, live descendants, scheduled runs, recovery, previews, unauthorized access, upstream failure, concurrent reopening, interrupted deletion, shared bases, and nested private worktrees. - Final guard regressions: 3 affected tests passed. Existing workspace/runtime/preparation/source controls: 328 tests passed in 238.69 seconds. - Full repository typecheck and build passed locally. The full local test attempt did not pass: initial native-session cases hit macOS Postgres shared-memory exhaustion, source changes invalidated two discovery cases, and the checkout disappeared near the end. The final-code retry of the remaining skill and unrelated controls passed all 103 tests. All checks passed on this exact head, including typecheck, general and serialized tests, Runner checks, build, canary dry-run, browser E2E, and security scans. Greptile reviewed the head at 5/5 with all findings resolved. - Fixed a blocking CI policy false positive for `docker-compose-secrets.test.mjs`. The test still rejects GitHub secret expressions, including dot, bracket, and multiline forms. Its full 15-test suite passes. - Map validation and `git diff --check` passed. - Staging dry-run inspection, measured reclamation, and ordinary product acceptance remain pending. No live acceptance is claimed. ## Risks - Git restoration depends on continued upstream availability and credentials. Cleanup skips unverified commits; restoration never substitutes the default branch. - File deletion requires multiple conservative proofs. Unknown or unsupported state is retained and can limit reclamation. - Source and lifecycle locks serialize preparation, reopening, and expiration. Remote verification has a deadline but can delay work on the same source. - Start with both expiration switches disabled. When enabled, dry-run remains the default. Rollback disables further deletion while retaining restoration support. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, tool use, and code execution. The exact backend model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused tests; full local limitation documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
10 KiB
navigation
| navigation | ||||||
|---|---|---|---|---|---|---|
|
Workspaces, services, and files
A project or task can use a managed workspace with inspectable files, Git state, services, and runtime controls. Operators can follow its lifecycle through provisioning, use, and closure.
Implementation: workspace list, execution workspace, project workspace.
Agent access: companion.
Sub-features
binding: identify the workspace actually bound to a task/run.provisioning: observe readiness and failures instead of inferring readiness from a created record.services: start/stop supported services and inspect logs and exposed endpoints.files-git: inspect workspace files and repository state used for the deliverable.closure: close/reopen or reconcile through permitted controls without losing work.
How to get to it (user POV)
task-workspace
Use the task’s workspace card or Files panel and follow its workspace link.
workspace-management
Open /workspaces, a project workspace, or /execution-workspaces/:workspaceId and its Services/Configuration/Runtime logs views.
Driving it with Playwright and the Paperclip CLI
Preconditions: follow the baseline. Enable isolated-workspace UI where needed. Use an owned disposable repository and environment; record workspace ID and branch.
task-workspace
Automated: workspace card and task binding cover host/binding contracts.
Manual: Run a task that writes a small file, follow the bound workspace, and inspect the file and revision. Reload and verify links still refer to the same workspace, not another checkout or a stale run.
For fresh standard-trust tasks using an implicit repository default, create a disposable project checkout whose shared Git storage overlaps a private task workspace. Verify that source admission creates a separate checkout from the configured remote, with no private local branches, commits or shared Git storage. Reusing, closing and reopening that task must retain the company/task-bound source; a failed workspace save must clean up only its owned checkout. Explicit workspace, parent/channel selections, literal or task-independent branch templates, low-trust tasks, and local or credential-bearing remote URLs must not use this fallback. A selected checkout requires recorded default provenance. Create tasks through the task service with an explicit checkout ID or an inherited parent source but no typed selection; verify denial, no clone or environment acquisition, and preservation of unpublished branch commits. Older ambiguous bindings and source choices changed during preparation must retain the denial. The remote-source tests and preparation tests exercise the real Git isolation and lifecycle guards. They do not qualify live remote credentials or a provider deployment.
For reconstructable Git expiration, finish a disposable task tree, run the internal lifecycle sweep after its 24-hour grace, and inspect candidate/reclaimed bytes. Advance its upstream first; use the ordinary task/workspace reopen control and verify the original commit, file contents, and subsequent persisted work. Current runs, previews, recovery, unique files and shared base clones must stay. The expiration tests exercise real local Git and database paths, interruption and concurrent reopening; cloud credentials and the deployed user journey require separate live evidence.
For task workspace selection, use the authorized task workspace API to choose a task directory, project source, or existing workspace. Verify that a change made during a run stays pending until the next normal admission. Submit another choice using the returned revision, then retry the superseded request; it must fail without replacing the newer pending choice. A projectless task must retain its files across resume and reassignment. Use the repository preparation API to add an authorized repository beneath that workspace; repeat the request and verify that it reuses the checkout and pinned revision. See task binding tests and repository access tests. An existing projectless folder alias of a sandbox-restricted project must be rejected for a standard local task before its adapter executes. Repeat after relocating the configured source; retained ownership must still enforce the boundary. This is separate from ordinary authorized file preview/download. For a registered worktree selected for a sandbox or SSH environment, configured and built-in provision scripts must not run on the controller during fresh creation, reuse, or rebuild, including host runtime provisioning. Repeat with standard trust and a protected read-only source, then with source write grants; permissions must not change execution placement. Verify isolated source copying and selected-environment dispatch still work. Repeat an explicit local isolated source choice: copying remains available but controller scripts must stay absent, including after binding or metadata changes and closed-worktree reopening. An implicit trusted local project default remains a positive provisioning control. Repeat with a newer configured default overriding an older primary source, then with a newer primary overriding an older nonprimary source. Both configured and built-in hooks must run for the implicit local default. Task creation and controller classification share configured-default, primary, creation-time and ID precedence. Explicit choices and saved copy-only markers still suppress hooks; changing the default must not rotate an existing binding or clear its marker. SSH keeps its explicit remote provision-command path; these checks do not claim a new sandbox provision transport. Low-trust closed-worktree reopening, including a board-comment reopen of a projectless task, must also suppress both kinds of host provision script. Returned checkpoints must reject file, symlink or missing parents before host writes, even if a nested file is marked unchanged. The retained host file and recovery evidence must survive rejection. These are automated safety contracts, not new live runtime acceptance.
An unchanged returned snapshot skips the host scan after acquiring the writer lock and checking authority. Concurrent host edits remain untouched; changed files, modes, symlinks and deletions still take the conflict path. The unchanged restore regression checks both hooks under the lock. This is source-level coverage, not a cloud latency result.
For a child created by an admitted agent with an explicit existing workspace, verify that a matching sandbox boundary permits creation before the child has an ID. A changed child policy, unrelated parent, or forged creator run must fail. See the child workspace trust regression. This automated contract does not add live acceptance evidence.
workspace-management
Automated: workspace detail and runtime service cover UI and runtime contracts separately.
Manual: Provision a disposable workspace, wait for readiness, start an available service, open its endpoint, and inspect a real response and logs. Stop it and verify it actually stops. Exercise close/reopen only after inspecting pending changes and active runs.
Gotchas
-
UI visibility, default isolation policy, the local environment row, a workspace record, and a running service are distinct prerequisites. A fresh instance still needs an owned cwd/repository and runtime commands;
/workspaceslists existing work. Reopening depends on lifecycle, not a generic detail action. -
A link or running badge does not prove the service is reachable from the user’s browser.
-
Workspace IDs, runtime leases, and repository paths are not interchangeable.
-
Terminal workspace cleanup wakes after committed task/workspace/PR changes and restores cooldown deadlines at startup. Empty lanes stop scanning. Display Git results can lag by thirty seconds. Compatible records share one advisory Git inspection per bounded page, while each record keeps its own delivery check. Destructive cleanup starts a separate Git scan even if an older scan is running. The worker tests cover quiet lanes and deadlines. The service tests cover real PostgreSQL signals and retained worktrees. These tests do not qualify an external wake alarm or a live provider deployment.
-
Organizational project, source project, task files, personal
AGENT_HOME, and provider home have separate meanings. Private source ownership remains attached to copied files after source configuration changes or workspace closure. -
Use recovery for divergence/export repair and CLI operations for local worktree tooling.
Storybook verification
closure
Scope: ui Storybook: Close-readiness checks, pending changes/active runs, blocked/pending/error close, retry close and eligible reopening. Production: Confirm owned workspace shutdown, preserved files/Git state and permitted lifecycle transition/reopening; Browser Use session closure is unrelated. Sources: Source: ExecutionWorkspaceDetail.tsx, ExecutionWorkspaceCloseDialog.tsx, execution-workspaces.ts. Tests: execution-workspaces-routes.test.ts, execution-workspaces-service.test.ts, execution-workspace-reopen.test.ts