Files
PaperClipAI/maps/agent-access-map/workspaces.md
T
DottaandPaperclip 5a07c30c1a fix(workspaces): reduce temporary disk use during remote capture (#18644)
## Thinking Path

> - Paperclip manages AI agents and preserves their completed work.
> - Native remote finalization must save workspace files before it
releases the writer.
> - Portable capture copied the working tree and Git history, repacked
the history, and then wrote an archive.
> - These temporary copies can exhaust a small sandbox after normal
development work.
> - A completed provider turn then fails finalization and file
preservation repeats the same failure.
> - This change streams checked working files and keeps only an
independent Git copy in scratch.
> - The save still rejects changed sources and incomplete output.

## Linked Issues or Issue Description

Related: #18403 added portable remote workspace storage. #18172 handles
disk pressure in Daytona directory transfer. This failure occurs
earlier, while the portable archive helper creates its source archive.
That earlier repair does not cover this path.

**What happened?**
A completed native run failed workspace finalization three times. Its
file-preservation retries also failed. In a retained 10 GB Linux sandbox
with about 1.5 GB free, the deployed helper failed during `git repack -a
-d` with `Out of diskspace`. The working files used about 603 MB, and
Git used about 444 MiB.

**Expected behavior**
Capture a coherent workspace when free space can hold one full archive
plus an independent Git copy. Preserve working files, staged changes,
unpushed commits, and existing recovery gates.

**Steps to reproduce**
1. Use a disposable Linux sandbox with a Git repository and generated
working files.
2. Leave enough free space for an archive and a Git copy, but not a
second working tree and another repack.
3. Run the portable capture helper. The old helper fails while creating
temporary data.
4. Run the fixed helper against the same source and free-space limit.
Verify each archived file hash.

The small regression uses real file serialization and a controlled
filesystem capacity failure at archive creation. It failed with `ENOSPC`
before the production edit and passes afterward.

**Paperclip version or commit**
The observed staging server served
`42c64d4ec4b9b0898556fcdf6ba304c4285a7819`. The repair is based on
`a92f616cd0`.

**Deployment mode**
Hosted staging. Linux x64, Node 25.9.0, Daytona sandbox. Local tests use
a macOS source checkout.

## What Changed

- Stream working files from checked file handles. Keep only normalized
Git metadata in scratch.
- Remove the second repack after a mirror clone that already uses
`--dissociate` and `--no-hardlinks`.
- Check each streamed file's hash, size, mode, and filesystem identity.
Recheck the source inventory and Git state before returning a receipt.
- Remove incomplete archives after failure or cancellation. Keep
pre-existing archive files intact.
- Add space-limit, concurrent-change, cancellation, and existing-file
safety cases. Update execution semantics and workspace verification
recipes.

## Verification

- Red before the production change: the capacity regression failed with
`ENOSPC`. Green afterward: 54 archive and transport tests pass.
- The existing tests restore ordinary, linked, and borrowed Git
repositories after their sources disappear. They retain staged,
unstaged, and unpushed work. Credential and path guards also pass.
- Live comparison on the same retained sandbox: the deployed helper
failed after 19.4 seconds with Git disk exhaustion. The fixed helper
completed in 35.3 seconds. It produced a 1,081,872,803-byte archive with
24,111 entries. An independent reader checked every archived file hash
and the manifest digest. The source commit stayed unchanged. Diagnostic
scratch was removed.
- This live probe exercised the changed capture boundary. It did not
deploy a server, upload a production snapshot, finalize the failed run,
or replay its provider turn. Full task recovery still needs rollout and
the normal export recovery path.
- `node scripts/check-maps.mjs` and `git diff --check` pass.
- Current-head CI passes: [PR
workflow](https://github.com/paperclipai/paperclip/actions/runs/41499985180),
commit `d42700f7f4b0bef14a5969a44c2da0b2513e0ed2`. Greptile rates the
same head 5/5 with no actionable findings or unresolved threads.
- The independent Ponytail advisory also rates this exact head 5/5 with
no findings. Its structured check remains queued because the automatic
request was bound to the earlier head. A [fresh current-head review
request](https://github.com/paperclipai/paperclip/pull/18644#issuecomment-6585296011)
was accepted and queued. Its predecessor has a successful workspace
export but remains active in finalization, so the new review cannot
start yet. This review-service publication gate must complete before
claiming every check is green.
- `pnpm -r typecheck` and `pnpm build` pass. The final server-only
typecheck also passes after the test additions.
- Supported-runtime verification on Node 24.11.0: all 54
archive/transport tests and all 46 existing workspace lifecycle tests
pass. The lifecycle suite uses a disposable real Postgres database and
local filesystem transport with encrypted snapshot storage. It checks
save/restore after losing the original allocation, ownership, retention,
and file continuity. This is separate from the live Daytona capture
probe.
- The first extra lifecycle attempt could not start its local Postgres
binary because the installed Darwin package lacked library aliases.
Local dependency setup was repaired before the passing run. No
production or repository code changed for that setup.
- The full local `pnpm test:run` on the host default Node 26.4.0 stops
in chat-test collection with `No test found ... in lines ...`; it has no
suite verdict. A clean retry repeats the failure. The unchanged
`a92f616cd0` test wrapper reproduces it with the same installed
dependencies, after correcting an initial baseline setup that lacked
built package entries. This is a local test-runner limitation, not a
full local pass. Current-head CI remains the full-suite gate.
- One focused run timed out while deeply comparing 4 MB buffers under
concurrent build load. Complete SHA-256 comparisons retain the content
check without that assertion cost. The capacity case then passes in
under one second, and the final 54-test run passes. The disk allowance
and production code did not change.

## Risks

- Streaming reads the live source instead of a copied working tree. File
identity, content hashes, mode checks, the final inventory read, and Git
signatures must reject concurrent changes. Regression tests exercise
these checks.
- Capture still needs space for the full archive and normalized Git
metadata together. It cannot export from a completely full disk.
- Archive framing and restore validation stay compatible. The change has
no database migration or new execution authority.
- The incident remains unrecovered until the fixed server performs
normal workspace finalization. No deployment or merge is included in
this PR.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository inspection, code
editing, shell tools, browser diagnostics, and automated tests. The
exact serving variant and context-window size are not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-02 13:22:23 -06:00

355 lines
20 KiB
Markdown

# Execution workspaces, services, and files — agent access
Product map: [Execution workspaces, services, and files](../feature-map/workspaces.md).
Feature ID: `workspaces`.
Lost-source reconstruction adds the board-only
`POST /api/issues/{id}/recovery-actions/reconstruct-workspace` action. It requires
`runtime:manage` and ordinary task visibility. Agents cannot authorize it. The
result is a durable fresh-session admission request, subject to current execution
gates. It grants no additional tool access, budget, work window or permission to
repeat completed actions. Attachments must belong to the same task and company.
The original failed run and lost-export evidence remain unchanged.
Later native run-only cancellations can coexist only with confirmed lease stops,
board file-only repair and an external-action outcome of `not_performed`.
Their immutable evidence is checked again at dispatch. Historical cancellation
retention runs internally and grants no new agent operation or authority.
Review date: 2026-10-12. Source commit: `58e16dbddfc9c8d0cf60867f9d9847f0feda055b`.
Native REST transport reconciled in this change; see the
[shared transport contract](README.md#native-rest-transport).
This is a source-reviewed baseline of representative operations and known gaps.
Each capability needs operation-level review before its entire scope can be
qualified. No live agent execution is claimed. Follow the [map contract](../README.md)
for format, permission checks, evidence and maintenance.
## `binding`
Capability ID: `workspaces/binding`.
Review: `baseline`.
API: `present` — representative operation IDs:
- `GET /api/issues/{id}/heartbeat-context`.
- `GET /api/execution-workspaces/{id}`.
- `GET /api/issues/{id}/workspace` inspects the durable task binding.
- `PUT /api/issues/{id}/workspace` requests the next binding.
- `GET /api/companies/{companyId}/execution-workspaces?summary=true&selectableForTask=true`
lists authorized task-selectable workspaces.
Agent access: `available` — Native get_task_context / REST / CLI / MCP workspace read.
The task-bound native tools `get_workspace`, `list_workspaces`, and
`select_workspace` use the ordinary authenticated APIs. Legacy agents can call
those APIs with their admitted run token. These paths do not require a project.
Policy: Company/project/workspace visibility and runtime authority; active run, trust
preset and task ownership where required.
Ordinary input may reuse the exact currently bound private task directory after
an explicitly fresh retry. Preparation rechecks company, source task, current
binding and canonical path; missing directories, symlinks and foreign roots are
rejected. This preserves files without granting new access or replaying the
previous turn. An explicitly fresh retry still creates a clean sibling.
Task workspace writes require `issue:mutate`, an active owned task/run for agents,
and current source visibility. Shared physical roots also require source-project
workspace write authority. Accepting a new pending choice advances its binding
revision; superseded retries fail while current-choice retries remain idempotent.
Binding revisions and request keys fence retries;
selection applies at the next normal admission without interrupting a live run.
The managed `remoteWorkspaceStorageEnabled` setting defaults on for Cloud and
self-hosted instances. New native Paperclip Runner sandbox workspaces with
supported fresh sources reserve company/task-bound remote authority through
existing admission only when durable Postgres, S3, and a stable key are configured.
Existing host workspaces and configured local folders stay on the legacy path
until an explicit import. Stored `false` disables new enrollment unless managed
configuration overrides it. Enrolled workspaces keep remote authority when new
enrollment is disabled, and missing persistence prerequisites fail admission
without a controller checkout fallback. Current authorization and responsible-user
checks still apply;
read-only cross-project referenced folders and explicitly concurrent shared
writers are rejected. Managed mutable same-project repositories are supported.
Portable capture streams validated working files while staging self-contained
Git metadata. The smaller temporary-space requirement grants no new agent access
or execution authority. Changed sources still fail before a save receipt.
An inherited isolated workspace or operator branch can be reserved through the
task's current persisted binding. The original source task remains provenance.
Reservation checks task/run, binding, trust, and retained source visibility before
claiming the single writer; an unbound task cannot claim that workspace.
The [reservation tests](../../server/src/services/remote-workspace-reservation.test.ts)
exercise that service boundary with real PostgreSQL, including competing children,
private sources, foreign companies, and closed records. This is automated evidence,
not live agent qualification.
Importing an existing local task folder is an operator source-checkout command,
not an agent API or tool: see the
[dry-run/apply runbook](../../doc/DEVELOPING.md#experimental-native-remote-workspace-storage).
It requires deployment storage/key access and blocks unfinished writers/recovery.
Source: [remote source admission](../../server/src/services/heartbeat/remote-workspace-source.ts)
and [migration](../../server/src/services/remote-workspace-migration.ts).
Evidence: `source-only` for these additions; live agent qualification is separate.
Admission checks current task/run authority and the persisted selected configuration.
It does not inventory physical aliases or authorize historical sources/readers.
Explicit workspace-record selection and full workspace metadata disclosure retain
app-record privacy. Existing isolated remote-source receipts remain restorable;
new preparation does not create a remote fallback due to historical file ownership.
Historical `workspace_source_restricted` failures retain existing recovery support.
An authorized board recovery through `POST /api/issues/{id}/recovery-actions/resolve`
can schedule a server-owned `recovery` selection for the latest restricted-source
failure only when its saved evidence proves provider work never started. The
selection requires task mutation and configured-source access and applies at normal
admission. It keeps task environment settings and never reads or deletes the denied
checkout. Pause, budget, approval, interaction, dependency, and ownership gates
remain effective. Repeated resolution of the same action does not enqueue another
run. Agents cannot mint recovery provenance through the workspace selection API;
they can observe the existing recovery action and request board recovery.
Local paths, unsupported transports and credential-bearing URLs are not fallback
sources. Company/task/remote bindings, real paths, Git storage and the exact origin
are revalidated for reuse, cleanup and reopening. A failed save removes only the
owned task worktree. The [isolated-source service](../../server/src/services/isolated-remote-workspace-source.ts)
and its local regression tests establish source and lifecycle boundaries; live
agent/provider qualification remains separate.
Reconstructable Git expiration adds no agent endpoint or permission. It records
company/workspace/repository identity and an exact commit before removing owned
cache files. Existing task/workspace reopening retains current company, source,
and trust checks. Missing upstream access returns a retryable error; it does not
choose a newer commit or grant access to another company's source. Shared project
bases and unresolved recovery remain protected. Local lifecycle evidence is in
[expiration tests](../../server/src/__tests__/git-workspace-expiration.test.ts).
Only directly selected configuration policies participate in current execution trust.
Physical aliases and historical source projects do not contribute filesystem policy.
The
authoritative sandbox or SSH selection suppresses configured and built-in
controller worktree and runtime provisioning regardless of task trust, app configuration
write grants or board sponsorship. Genuine isolated worktrees still permit
authorized read-only source copying. Configuration write permission does not authorize
controller execution. Explicit, channel, inherited, foreign or nondefault isolated
source choices remain copy-only for controller provisioning even on local runs.
Their deny-only workspace marker survives metadata and binding changes; a board
reopen does not turn it into approval. Implicit trusted local defaults and explicit
SSH remote provisioning retain their existing placement; no new sandbox provision transport
is claimed. The implicit default follows task creation: configured policy pointer,
then primary source, creation time and ID. A missing or foreign configured source
does not authorize controller hooks. This precedence neither changes existing
creation validation nor rotates bindings or clears retained copy-only markers.
Closed-worktree reopening checks current task, agent, organizational and source policies without assuming a future driver. Effective low-trust reopens suppress configured and built-in host scripts. Returned checkpoints must have directory parents for every nested entry before merging, including unchanged entries; invalid output leaves retained host files intact and keeps recovery evidence.
Skipping the host scan for a fully unchanged returned snapshot grants no new
access. The merge still holds its canonical writer lock and runs both authority
and publication hooks. Denials still propagate, and checkpoint/archive validation
is unchanged. [Regression coverage](../../packages/adapter-utils/src/workspace-restore-unchanged.test.ts)
is source-level evidence only.
Child creation by an admitted agent checks an explicit prospective-task context before the child is saved. The creator run must have an authoritative saved policy and a verified company/agent/parent lineage. Current parent, prospective child and selected source configuration policies must fit that boundary. This creation path does not relax exact task/run checks for existing-task writes. The [child trust regression](../../server/src/__tests__/task-workspace-child-trust.test.ts) is automated source-level evidence, not live agent qualification.
Source review for these additions: PR #15749 and
[task workspace routes](../../server/src/routes/task-workspaces.ts),
[native authority](../../server/src/services/native-runtime/paperclip-runner-tool-authority.ts),
and [semantic transport](../../server/src/services/task-workspace-tools.ts).
Evidence: `source-only` —
[implementation](../../server/src/routes/workspace-runtime-service-authz.ts#L76) at the
reviewed commit above.
Verification: `not-run` — exercise a listed permitted operation with actual agent
credentials/run context; inspect its result and applicable persistence/attribution.
Assert the relevant denied actor, owner, company or mode from Policy. Review the
remaining operations before qualifying the whole capability.
Gaps: Run/task context identifies its actual realized workspace; record existence is not
a running service.
## `provisioning`
Capability ID: `workspaces/provisioning`.
Source preparation keeps its dedicated lock connection active during work on
other connections. It preserves company/source exclusion and existing admission
authority; it adds no agent API or permission. The
[source-lock regression](../../server/src/services/isolated-remote-workspace-source-lock.test.ts)
covers PostgreSQL idle timeout, competing preparation, ordinary pool availability,
and failure cleanup. Deployed agent recovery requires separate live evidence.
Review: `baseline`.
API: `present` — representative operation IDs:
- `GET /api/execution-workspaces/{id}`.
- `GET /api/execution-workspaces/{id}/workspace-operations`.
- `POST /api/issues/{id}/workspace/repositories` requests repository preparation.
Agent access: `partial` — REST / native call_api reads; task/project assignment triggers
server provisioning.
Native `prepare_repository` and admitted legacy REST callers can request a
catalog repository or validated repository URL for their current task.
Preparation occurs at normal admission; the agent cannot supply a host path or
controller identity. Current selected-configuration permissions, responsible-user authority, trust
restrictions, immutable receipts, and concrete binding/path integrity are checked
before acquisition and publication. Low-trust callers cannot trigger a
new controller-side Git acquisition. Evidence remains source-only here; focused
[repository tests](../../server/src/__tests__/task-workspace-repositories-trust.test.ts)
are separate from live agent qualification.
An admitted native standard task can read a same-agent completion continuation
for its own pending repository in its current workspace. An explicit yield may
use that saved continuation; explicit completion remains completion. Existing
status decisions and wake admission retain Stop and authorization gates. The
[workspace API regression](../../server/src/__tests__/task-workspace-api.test.ts)
and [driver regression](../../packages/paperclip-runner/src/drivers/completion-continuation.test.ts)
cover this contract; live qualification remains separate.
Policy: Company/project/workspace visibility and runtime authority; active run, trust
preset and task ownership where required.
Evidence: `source-only` —
[implementation](../../server/src/routes/workspace-runtime-service-authz.ts#L76) at the
reviewed commit above.
Verification: `not-run` — exercise a listed permitted operation with actual agent
credentials/run context; inspect its result and applicable persistence/attribution.
Assert the relevant denied actor, owner, company or mode from Policy. Review the
remaining operations before qualifying the whole capability.
Gaps: Observing readiness exists. Creating host commands or arbitrary provision targets
is a separate protected operation.
## `services`
Capability ID: `workspaces/services`.
Review: `baseline`.
API: `present` — representative operation IDs:
- `POST /api/execution-workspaces/{id}/runtime-services/{action}`.
Agent access: `available` — Authorized agent REST / native `call_api` / CLI runtime
service controls and existing stdio MCP controls.
Policy: Company/project/workspace visibility and runtime authority; active run, trust
preset and task ownership where required.
Evidence: `source-only` —
[implementation](../../server/src/routes/workspace-runtime-service-authz.ts#L76) at the
reviewed commit above.
Verification: `not-run` — exercise a listed permitted operation with actual agent
credentials/run context; inspect its result and applicable persistence/attribution.
Assert the relevant denied actor, owner, company or mode from Policy. Review the
remaining operations before qualifying the whole capability.
Gaps: No dedicated production control_workspace_service binding. Generic native
calls retain runtime trust and workspace/task scope checks. Optional MCP availability
must be recorded.
## `files-git`
Capability ID: `workspaces/files-git`.
Review: `baseline`.
API: `present` — representative operation IDs:
- `GET /api/issues/{issueId}/file-resources/content`.
- `GET /api/issues/{issueId}/file-resources/list`.
Agent access: `partial` — Authorized workspace filesystem tools; Dot workspace bridge
when enabled; board file-resource HTTP UI. Native standard-mode `download_artifact`
installs an authorized saved asset (up to 10 MiB) into the admitted workspace,
requiring exact saved size/hash. Linux local and lease-owned remote command
runners are supported; plugin task providers without an installer and Dot/Muse/Grok
profiles do not expose it. The tool follows API rollout policy, current task/run,
issue mutation and asset-content ACLs, including responsible-user capture ownership.
Publication is create-only after staging/hash checks; conflicting files and symlink
paths fail. Retry the identical identity after an unknown outcome. Audit records
contain identity/outcome only; contents remain untrusted and screening still applies.
The [authority regression](../../server/src/services/native-runtime/artifact-transfer.integration.test.ts)
and [filesystem faults](../../server/src/services/native-runtime/artifact-transfer.test.ts)
use disposable data and real files; provider transport qualification is separate.
Policy: Company/project/workspace visibility and runtime authority; active run, trust
preset and task ownership where required.
Evidence: `source-only` —
[implementation](../../server/src/routes/workspace-runtime-service-authz.ts#L76) at the
reviewed commit above.
Verification: `not-run` — exercise a listed permitted operation with actual agent
credentials/run context; inspect its result and applicable persistence/attribution.
Assert the relevant denied actor, owner, company or mode from Policy. Review the
remaining operations before qualifying the whole capability.
Gaps: Files panel HTTP requires board. Agents can use their admitted workspace file
tools; remote/Dot scope and Git command availability differ.
Task-bound file reads authorize the current task, company, and concrete binding;
selected project roots require selected configuration access. Historical task,
reader, physical-root, and directory acquisition classifications have no effect.
Root containment and symlink escape checks remain, and attachments, task records,
workspace metadata, and operation logs retain their app-data privacy.
For enrolled remote workspaces the same board file-resource endpoints read the
authorized live allocation or a verified committed saved version. Returned
`source.kind` is `live` or `saved`, with `savedAt` for a saved copy. An unavailable
dirty allocation cannot silently fall back to stale saved bytes. Remote paths
never become controller filesystem authority. Confined path/symlink checks,
bounded search/preview, download streaming, and cancellation remain enforced by
[remote file access](../../server/src/services/remote-workspace-files.ts).
Evidence: `source-only` here; regression suites are not a live agent read proof.
Verification: `not-run` — exercise allowed live/saved reads and deny another
company, changed binding, traversal, and unsaved-allocation fallback.
## `closure`
Capability ID: `workspaces/closure`.
Review: `baseline`.
API: `present` — representative operation IDs:
- `PATCH /api/execution-workspaces/{id}`.
- `GET /api/execution-workspaces/{id}/close-readiness`.
- `POST /api/execution-workspaces/{id}/reconcile-branch`.
Agent access: `partial` — REST / native fallback for permitted update; CLI workspace;
reconcile-branch requires board.
Policy: Company/project/workspace visibility and runtime authority; active run, trust
preset and task ownership where required.
Workspace metadata reads do not perform storage inspection or certify delivery.
The explicit close-readiness API and destructive cleanup retain live safety checks.
Automatic GitHub delivery checks reuse explicit source-project GitHub consent,
rechecking the stored repository, selected grant, project access and trust on each
proof. Missing or revoked consent never falls back to another identity. The resolved destination and
provider recovery hint cannot be supplied as an authorization override through
workspace metadata. Ambiguous or inaccessible proof retains storage.
Evidence: `source-only` —
[implementation](../../server/src/routes/workspace-runtime-service-authz.ts#L76) at the
reviewed commit above.
Verification: `not-run` — exercise a listed permitted operation with actual agent
credentials/run context; inspect its result and applicable persistence/attribution.
Assert the relevant denied actor, owner, company or mode from Policy. Review the
remaining operations before qualifying the whole capability.
Gaps: Closure/readiness exists; not every workspace can be reopened, and branch
reconciliation has a board boundary.
Remote closure additionally requires a clean committed snapshot and no active
writer. It releases confirmed remote compute while retaining the saved authority;
reopen restores from that saved version when the allocation is gone. Custom
cleanup/teardown commands block remote close. Manual runtime commands, jobs,
repairs, and services reject remote workspaces before local filesystem access.
Agent tasks remain the supported execution path. No new agent authority is granted.
Source: [remote lifecycle](../../server/src/services/remote-workspace-lifecycle.ts)
and [close readiness](../../server/src/services/execution-workspaces.ts).
Evidence: `source-only` for the remote extension. Verification: `not-run` —
reject dirty/unfinished/custom-command close, confirm clean release retains the
snapshot, then reopen under current task/company authorization.