Files
PaperClipAI/doc
DottaandClaude 6e524d8a69 fix(workspaces): stop retrying a copy-back that hit a full sandbox disk (#20678)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - After a native run, Paperclip copies the agent's sandbox workspace
back to durable storage. This is finalization.
> - The copy-back builds an archive inside the sandbox. When the sandbox
disk is full, the copy-back fails with
`remote_workspace_sandbox_disk_full`.
> - Finalization treated that failure as transient. It retried three
times. Each attempt took about 15 minutes for a large workspace, and
nothing freed space between attempts.
> - Saved-work recovery already reclaims the retained sandbox's inactive
provider cache when a save hits a full disk. It runs only after
finalization ends.
> - This pull request ends finalization on the first full-disk copy-back
failure, with the accepted result kept, so saved-work recovery starts at
once.
> - The benefit is that the task gets its files saved sooner, and the
user sees what happened instead of "failed after 3 attempts".

## Linked Issues or Issue Description

Refs #19619 (related: records why the read-only capture fallback failed;
this PR does not change that path)

**What happened?**
On staging, a native run finished its turn and its result was accepted.
The workspace copy-back then failed three times with
`remote_workspace_sandbox_disk_full` (about 7.5 GB of workspace, about 6
GB free). The three attempts spanned about one hour across deploys.
Finalization then ended as `native_workspace_sync_out_retry_exhausted`.
Saved-work recovery reclaimed the inactive provider cache (about 3 GB
freed) and saved the files.

**Expected behavior**
A full sandbox disk is recognised on the first failure. Paperclip does
not spend the finalization budget on attempts that cannot succeed. The
agent's result is kept. The user sees that the disk is full and that
Paperclip saves the files again without another provider turn.

**Steps to reproduce**
1. Run a native task on a remote sandbox whose workspace is larger than
the free sandbox space.
2. Let the provider turn finish.
3. Observe three copy-back attempts that each fail with
`remote_workspace_sandbox_disk_full`.

## What Changed

- `recordRetryableFailure` treats a workspace failure whose cause starts
with `remote_workspace_sandbox_disk_full:` as an exhausted budget on the
first attempt. The code stays
`native_workspace_sync_out_retry_exhausted`, so the existing export
hold, saved-work recovery, and export Retry apply unchanged.
- The run error for that case says that the sandbox disk is full and
that Paperclip saves the files again without repeating the agent's work.
It no longer says "after 3 attempts".
- Other copy-back failures keep their bounded retry.
- `captureRemoteWorkspaceArchive` rethrows the read-only fallback's own
failure when that fallback only lost its connection (`transport_closed`,
`provider_unavailable`, `provider_rate_limited`,
`sandbox_state_transition`). That failure is not disk-full, so
finalization keeps its bounded retry for it. Other fallback failures
keep the disk-full code.
- `remoteWorkspaceFailureCause` records the server-built read-only
capture failure, so a failed save names where the fallback stopped.
- The transport exports `REMOTE_WORKSPACE_SANDBOX_DISK_FULL_CODE`. The
finalizer and tests use it.
- `doc/execution-semantics.md` describes the new rule.

## Verification

- `cd server && npx vitest run
src/services/native-runtime/native-run-finalizer-telemetry.test.ts
src/services/remote-workspace-transport.test.ts` (195 pass).
- New test "hands a full sandbox disk to saved-work recovery on the
first copy-back failure" fails on `origin/master` (the failure stays
`retryable_failure`) and passes with this change. It completes an
accepted result first and checks that the result stays linked and that
no agent wakeup is created.
- New transport test "keeps a full disk only when the read-only fallback
did not just lose its connection": the two dropped-connection cases fail
on `origin/master` and pass with this change.
- New test "keeps other copy-back failures on their bounded retry"
passes before and after.
- `cd server && npx tsc --noEmit` passes. `pnpm verify` selected 2504
test files because the finalizer is widely imported; its first selected
batch passed, and required CI supplies the full run.

## Risks

- Low risk. The terminal state is the same state that three failed
attempts produce today. Only the timing changes.
- A sandbox whose disk frees space by itself between attempts now waits
for saved-work recovery instead of the next finalizer attempt.
Saved-work recovery repeats the save with its own bounded attempts, so
this does not lose work.
- Upgrade safety: a run saved by an older release in `retryable_failure`
with this cause ends on its next full-disk failure. No migration is
needed.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use (shell,
file edits) in Claude Code. Extended reasoning was not configured.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-11-10 19:31:33 -06:00
..
…
…
…
…