mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 16:27:18 +02:00
DottaandPaperclip
7c7d2d8d2c
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The runtime must preserve saved work and prevent duplicate execution. > - Older cleanup could destroy a sandbox before its workspace export finished. > - The existing recovery controls require either the original sandbox or recovered original files. > - Surviving attachments can support a rebuild, but they do not prove that missing edits were recovered. > - This pull request adds an explicit reconstruction decision and one fresh continuation through the normal execution gates. > - Operators can continue useful work while the original failed run remains truthful. ## Linked Issues or Issue Description **What happened?** A historical failed export can have an accepted result, a prepared workspace descriptor, and a confirmed destroyed sandbox. Its notice can also say `new_source_execution_path` and disappear. The task then has neither a truthful original-file repair path nor a supported reconstruction path. **Expected behavior** Keep the missing-file obligation visible. Let an authorized operator choose surviving task attachments and record lost work and external action outcomes. Start one fresh continuation after current execution checks. Keep the old result and failed export intact. **Steps to reproduce** 1. Seed a failed native export with a prepared descriptor and an exact destroyed-lease receipt. 2. Resolve its notice with the historical `new_source_execution_path` bookkeeping value. 3. Read the task. The baseline hides the hold even though no source reconciliation exists. 4. With this change, choose **Continue from saved sources** and submit a reconstruction decision. Related public work: #16451 preserves healthy warm sessions. #16583 prevents destructive cleanup of unexported work. This change handles historical source loss that those fixes cannot undo. Searches of open and closed work found no equivalent reconstruction endpoint. ## What Changed - Add a board-only reconstruction endpoint with `runtime:manage` and task visibility checks. - Validate the exact run, result, descriptor and destroyed lease. Reject active copyback, competing owners, newer work, stale task status, and foreign attachments. - Record attachment hashes, missing work, and external action outcomes. Preserve the original failed run, accepted result, prepared descriptor and terminal failure. - Hold other continuation paths until the durable reconstruction intent admits its successor. Record that successor in the same transaction and preserve the retry count. - Add an operator control that distinguishes reconstruction from original-file recovery. Update the recovery runbook and capability maps. ## Verification - Exact baseline source returns no blocker for the destroyed-source fixture. The same fixture returns a required recovery hold with this change. - New real PostgreSQL and assembled API/heartbeat suite: 25/25 pass. It covers duplicate submissions, lost acknowledgement, context reset, source identity, task ownership, concurrent execution attachment scope, later human blocks, deleted sources and agent pause. - Existing export retry, explicit continuation and recovery-action suites: 386 cases pass. One new-fixture constraint error in the combined run was corrected; the full new suite then passed. - UI recovery suites: 46/46 pass across three files. They cover disabled actions, loss acknowledgement, failure retention and runtime authority. - `pnpm -r typecheck` and `pnpm build` pass. The first sandboxed typecheck attempt could not open the TypeScript IPC socket; the authorized rerun passed. - Module boundaries, map checks, token gates and diff whitespace checks pass. - OpenAPI route coverage: 21/21 pass. The first CI head exposed a missing endpoint registration; this head adds the registration and the focused suite passes. - Review identified a missing dispatch-time check and an isolated attachment cache. Both are fixed. Claim and final handoff recheck the exact successor, sources and current execution gates. A controlled post-claim operator block prevents the assembled heartbeat from entering the provider. The shared attachment cache refreshes uploads and removes deleted selections. - Final focused run: 180/180 pass across six suites, including reconstruction, existing dispatch/stale-queue contracts and UI recovery. - A second review reproduced cancellation by an already satisfied historical connection card while the task was `todo`. Read-only connection availability now works before checkout; active-task mutation checks remain strict. The baseline failed two cases; after the fix all 74 connection and reconstruction tests pass, including revoked access, pending questions and pending approvals. - All current-head CI gates pass on `9d197236da11fa24f1ceadf5589cb2b81179d824`, including general/serialized tests, Runner, build and browser gates. Greptile is 5/5 for that exact head with no unresolved threads. The full local `pnpm test:run` is still running its serial server group; no full local pass is claimed. Live staging recovery is pending deployment. ## Risks - Reconstruction cannot restore missing bytes. The operator must identify surviving sources and actions that must not be repeated. The agent must verify those attachments before rebuilding. - This path is limited to exact, confirmed destroyed remote sources with an accepted result. Retained or uncertain sources keep their existing repair path. - No schema migration, credential change, budget increase or warm-session policy change is included. Normal pause, ownership, budget, approval and dispatch gates still apply. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use and code execution. The session does not expose an exact backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; full local run limitation recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
156 lines
11 KiB
Markdown
156 lines
11 KiB
Markdown
---
|
||
navigation: {"section":"Projects and execution","sectionOrder":40,"order":30}
|
||
---
|
||
|
||
# Workspaces, services, and files
|
||
|
||
A project or task can use a managed workspace with inspectable files, Git state, services, and runtime controls. Operators can follow its lifecycle through provisioning, use, and closure.
|
||
|
||
Implementation: [workspace list](../../ui/src/pages/Workspaces.tsx), [execution workspace](../../ui/src/pages/ExecutionWorkspaceDetail.tsx), [project workspace](../../ui/src/pages/ProjectWorkspaceDetail.tsx).
|
||
|
||
Agent access: [companion](../agent-access-map/workspaces.md).
|
||
|
||
## Sub-features
|
||
|
||
- `binding`: identify the workspace actually bound to a task/run.
|
||
- `provisioning`: observe readiness and failures instead of inferring readiness from a created record.
|
||
- `services`: start/stop supported services and inspect logs and exposed endpoints.
|
||
- `files-git`: inspect workspace files and repository state used for the deliverable.
|
||
- `closure`: close/reopen or reconcile through permitted controls without losing work.
|
||
|
||
For a historical failed export whose exact sandbox was destroyed, the task recovery
|
||
notice offers **Continue from saved sources** to an authorized board operator.
|
||
Select surviving task attachments, record lost work and external action outcomes,
|
||
then rebuild through fresh-session admission. The old failed run remains failed.
|
||
Retained sandboxes keep their existing export recovery path. Automated coverage:
|
||
[reconstruction](../../server/src/services/native-runtime/native-workspace-reconstruction.test.ts).
|
||
Live staging acceptance remains separate from these regression tests.
|
||
|
||
## How to get to it (user POV)
|
||
|
||
### `task-workspace`
|
||
|
||
Use the task’s workspace card or Files panel and follow its workspace link.
|
||
|
||
### `workspace-management`
|
||
|
||
Open `/workspaces`, a project workspace, or `/execution-workspaces/:workspaceId` and its Services/Configuration/Runtime logs views.
|
||
|
||
## Driving it with Playwright and the Paperclip CLI
|
||
|
||
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enable isolated-workspace UI where needed. Use an owned disposable repository and environment; record workspace ID and branch.
|
||
|
||
### `task-workspace`
|
||
|
||
Automated: [workspace card](../../ui/src/components/IssueWorkspaceCard.test.tsx) and [task binding](../../server/src/__tests__/issue-runtime-workspace-binding.test.ts) cover host/binding contracts.
|
||
|
||
Manual: Run a task that writes a small file, follow the bound workspace, and inspect the file and revision. Reload and verify links still refer to the same workspace, not another checkout or a stale run.
|
||
|
||
For fresh standard-trust tasks using an implicit repository default, create a
|
||
disposable project checkout whose shared Git storage overlaps a private task
|
||
workspace. Verify that source admission creates a separate checkout from the
|
||
configured remote, with no private local branches, commits or shared Git storage.
|
||
Reusing, closing and reopening that task must retain the company/task-bound source;
|
||
a failed workspace save must clean up only its owned checkout. Explicit workspace,
|
||
parent/channel selections, literal or task-independent branch templates, low-trust
|
||
tasks, and local or credential-bearing remote
|
||
URLs must not use this fallback. A selected checkout requires recorded default
|
||
provenance. Create tasks
|
||
through the task service with an explicit checkout ID or an inherited parent
|
||
source but no typed selection; verify denial, no clone or environment acquisition,
|
||
and preservation of unpublished branch commits. Older ambiguous bindings and
|
||
source choices changed during preparation must retain the denial. The
|
||
[remote-source tests](../../server/src/services/isolated-remote-workspace-source.test.ts)
|
||
and [preparation tests](../../server/src/services/heartbeat/workspace-preparation.test.ts)
|
||
exercise the real Git isolation and lifecycle guards. They do not qualify live
|
||
remote credentials or a provider deployment.
|
||
|
||
For reconstructable Git expiration, finish a disposable task tree, run the
|
||
internal lifecycle sweep after its 24-hour grace, and inspect candidate/reclaimed
|
||
bytes. Advance its upstream first; use the ordinary task/workspace reopen control
|
||
and verify the original commit, file contents, and subsequent persisted work.
|
||
Current runs, previews, recovery, unique files and shared base clones must stay.
|
||
The [expiration tests](../../server/src/__tests__/git-workspace-expiration.test.ts)
|
||
exercise real local Git and database paths, interruption and concurrent reopening;
|
||
cloud credentials and the deployed user journey require separate live evidence.
|
||
|
||
For task workspace selection, use the authorized task workspace API to choose a
|
||
task directory, project source, or existing workspace. Verify that a change
|
||
made during a run stays pending until the next normal admission. Submit another
|
||
choice using the returned revision, then retry the superseded request; it must
|
||
fail without replacing the newer pending choice. A projectless
|
||
task must retain its files across resume and reassignment. Use the repository
|
||
preparation API to add an authorized repository beneath that workspace;
|
||
repeat the request and verify that it reuses the checkout and pinned revision.
|
||
See [task binding tests](../../server/src/__tests__/task-workspace-binding.test.ts)
|
||
and [repository access tests](../../server/src/__tests__/task-workspace-repositories-trust.test.ts).
|
||
An existing projectless folder alias of a sandbox-restricted project must be
|
||
rejected for a standard local task before its adapter executes. Repeat after
|
||
relocating the configured source; retained ownership must still enforce the
|
||
boundary. This is separate from ordinary authorized file preview/download.
|
||
For a registered worktree selected for a sandbox or SSH environment, configured
|
||
and built-in provision scripts must not run on the controller during fresh
|
||
creation, reuse, or rebuild, including host runtime provisioning. Repeat with
|
||
standard trust and a protected read-only source, then with source write grants;
|
||
permissions must not change execution placement. Verify isolated source copying
|
||
and selected-environment dispatch still work. Repeat an explicit local isolated
|
||
source choice: copying remains available but controller scripts must stay absent,
|
||
including after binding or metadata changes and closed-worktree reopening. An
|
||
implicit trusted local project default remains a positive provisioning control.
|
||
Repeat with a newer configured default overriding an older primary source, then
|
||
with a newer primary overriding an older nonprimary source. Both configured and
|
||
built-in hooks must run for the implicit local default. Task creation and controller
|
||
classification share configured-default, primary, creation-time and ID precedence.
|
||
Explicit choices and saved copy-only markers still suppress hooks; changing the
|
||
default must not rotate an existing binding or clear its marker.
|
||
SSH keeps its explicit remote provision-command path; these checks
|
||
do not claim a new sandbox provision transport.
|
||
Low-trust closed-worktree reopening, including a board-comment reopen of a projectless task, must also suppress both kinds of host provision script. Returned checkpoints must reject file, symlink or missing parents before host writes, even if a nested file is marked unchanged. The retained host file and recovery evidence must survive rejection. These are automated safety contracts, not new live runtime acceptance.
|
||
|
||
An unchanged returned snapshot skips the host scan after acquiring the writer lock
|
||
and checking authority. Concurrent host edits remain untouched; changed files,
|
||
modes, symlinks and deletions still take the conflict path. The
|
||
[unchanged restore regression](../../packages/adapter-utils/src/workspace-restore-unchanged.test.ts)
|
||
checks both hooks under the lock. This is source-level coverage, not a cloud latency result.
|
||
|
||
For a child created by an admitted agent with an explicit existing workspace, verify that a matching sandbox boundary permits creation before the child has an ID. A changed child policy, unrelated parent, or forged creator run must fail. See the [child workspace trust regression](../../server/src/__tests__/task-workspace-child-trust.test.ts). This automated contract does not add live acceptance evidence.
|
||
|
||
### `workspace-management`
|
||
|
||
Automated: [workspace detail](../../ui/src/pages/ExecutionWorkspaceDetail.test.tsx) and [runtime service](../../server/src/__tests__/workspace-runtime.test.ts) cover UI and runtime contracts separately.
|
||
|
||
Manual: Provision a disposable workspace, wait for readiness, start an available service, open its endpoint, and inspect a real response and logs. Stop it and verify it actually stops. Exercise close/reopen only after inspecting pending changes and active runs.
|
||
|
||
## Gotchas
|
||
|
||
- UI visibility, default isolation policy, the local environment row, a workspace
|
||
record, and a running service are distinct prerequisites. A fresh instance
|
||
still needs an owned cwd/repository and runtime commands; `/workspaces` lists
|
||
existing work. Reopening depends on lifecycle, not a generic detail action.
|
||
|
||
- A link or running badge does not prove the service is reachable from the user’s browser.
|
||
- Workspace IDs, runtime leases, and repository paths are not interchangeable.
|
||
- Terminal workspace cleanup wakes after committed task/workspace/PR changes and
|
||
restores cooldown deadlines at startup. Empty lanes stop scanning. Display Git
|
||
results can lag by thirty seconds. Compatible records share one advisory Git
|
||
inspection per bounded page, while each record keeps its own delivery check.
|
||
Destructive cleanup starts a separate Git scan even if an older scan is running.
|
||
The [worker tests](../../server/src/__tests__/terminal-workspace-work.test.ts)
|
||
cover quiet lanes and deadlines. The
|
||
[service tests](../../server/src/__tests__/execution-workspaces-service.test.ts)
|
||
cover real PostgreSQL signals and retained worktrees. These tests do not
|
||
qualify an external wake alarm or a live provider deployment.
|
||
- Organizational project, source project, task files, personal `AGENT_HOME`, and
|
||
provider home have separate meanings. Private source ownership remains attached
|
||
to copied files after source configuration changes or workspace closure.
|
||
- Use [recovery](./recovery.md) for divergence/export repair and [CLI operations](./cli-operations.md) for local worktree tooling.
|
||
|
||
### Storybook verification
|
||
|
||
#### `closure`
|
||
|
||
Scope: ui
|
||
Storybook: Close-readiness checks, pending changes/active runs, blocked/pending/error close, retry close and eligible reopening.
|
||
Production: Confirm owned workspace shutdown, preserved files/Git state and permitted lifecycle transition/reopening; Browser Use session closure is unrelated.
|
||
Sources: Source: [ExecutionWorkspaceDetail.tsx](../../ui/src/pages/ExecutionWorkspaceDetail.tsx), [ExecutionWorkspaceCloseDialog.tsx](../../ui/src/components/ExecutionWorkspaceCloseDialog.tsx), [execution-workspaces.ts](../../server/src/routes/execution-workspaces.ts). Tests: [execution-workspaces-routes.test.ts](../../server/src/__tests__/execution-workspaces-routes.test.ts), [execution-workspaces-service.test.ts](../../server/src/__tests__/execution-workspaces-service.test.ts), [execution-workspace-reopen.test.ts](../../server/src/__tests__/execution-workspace-reopen.test.ts)
|