Files
PaperClipAI/server/src/services
DottaandClaude Opus 5.5 e8163fc860 fix(remote-workspaces): restore newer saved work over a retained sandbox's own retired root (#19231)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Remote workspaces save a task's files as snapshots and restore them
on the sandbox that runs the next turn
> - A parent task can delegate subtasks. The subtasks inherit the
parent's workspace, run on their own sandboxes, and save newer snapshots
> - When the parent wakes on its retained (warm, reused) sandbox, its
earlier workspace root is still on that sandbox at the same published
path
> - The restore refused that root with `reason=occupied_root`, so the
parent run failed at setup and the task became blocked with a board
recovery action
> - This pull request lets the restore replace that root only when the
server proves it is the workspace's own earlier, fully saved allocation
on this sandbox
> - The benefit is that a delegating parent on a retained sandbox
continues with the children's saved work, and unknown occupants are
still refused

## Linked Issues or Issue Description

**What happened?**

A parent task on an agent with a retained sandbox delegates two subtasks
to other agents. Both subtasks finish. The parent's wake run fails
before the provider starts with `remote_workspace_restore_failed:
step=sandbox_restore; ... stage=inspect_root reason=occupied_root`. The
task becomes `blocked` with a stranded-work recovery action. Parents on
agents with ephemeral sandboxes complete in the same wave.

Root cause: the subtasks inherit the parent's execution workspace. Each
subtask becomes the writer at a new epoch on its own sandbox, restores
at the same published directory name, and saves a newer snapshot. The
workspace location then points at the last subtask's sandbox. The
parent's wake run gets a lease on its retained sandbox, so preparation
is not "same allocation" and must restore the newer snapshot. The target
path on the retained sandbox still holds the parent's epoch-1 root,
which has a preparation receipt (`workspace-ready.json`) and no restore
receipt. The restore script refuses any occupied root that is not the
same restore, so it fails with `occupied_root`. A warm reuse cannot just
keep the old root, because the snapshot is newer.

**Expected behavior**

The parent restores the newest saved work (its own files plus the
children's results) into its published root and continues. A foreign or
unknown occupant is still refused.

**Steps to reproduce**

1. Prepare a remote workspace on sandbox A, write files, and save (final
boundary).
2. Make a second run the writer on sandbox B. It restores, writes new
files, and saves.
3. Make a third run the writer on sandbox A again (same provider lease
id, new lease row) and prepare.
4. Before this change, step 3 fails with `stage=inspect_root
reason=occupied_root`.

**Paperclip version or commit**

`master` at `47027e7ba5`.

## What Changed

- `server/src/services/remote-workspaces.ts`: add
`isRetiredOwnAllocation`. It reads this workspace's snapshot and restore
operations recorded at the same environment, provider lease id, and root
path. It returns true only when every such use is from an earlier epoch
than the current writer, the last use ended in a succeeded `final` save
with quiesced save work (`unknown=false`, no active invocations), no
other save in that epoch is newer or unsettled, and the saved snapshot
is committed or later. Preparation passes the result to the restore as
`replaceRetiredRoot`, and only when there is no migration and the
authority is clean.
- `server/src/services/remote-workspace-transport.ts`: the restore
script accepts `replaceRetiredRoot`. In that mode it replaces an
occupied root only when the root carries a well-formed Paperclip receipt
(`workspace-ready.json` or `workspace-restored.json` with a 64-hex
identity). It parks the old root at `<root>.retired-<identity>`,
publishes the restored folder, rolls the old root back if the publish
rename fails, and then removes the parked copy. A repeated restore with
the same identity adopts the published folder and removes any parked
copy left by an interrupted attempt. Without a receipt, the script still
fails with `reason=occupied_root`. `replaceRetiredRoot` and
`allowEmptyRoot` cannot be combined.
- `doc/execution-semantics.md`: document the rule.
- `server/src/services/remote-workspaces.integration.test.ts`: add a
red/green regression and the safety cases.

## Verification

- Red before the fix: `npx vitest run
src/services/remote-workspaces.integration.test.ts -t "retired root"`
failed in the `own retired root` case with
`remote_workspace_restore_failed: step=sandbox_restore;
remote_workspace_operation_failed: operation=restore exit=1;
remote_workspace_restore_failed: stage=inspect_root
reason=occupied_root`. This is the same cause text as the staging
failure.
- Green after the fix: all six cases pass:
- `own retired root`: the parent restores on its retained sandbox. The
root has the parent's file and the child's file, the parked and staging
folders are removed, and the authority records the parent's lease. A
repeated preparation (lost reply) keeps the parent's new unsaved edits.
- `own restored root`: an occupant with a restore receipt from a
different preparation is also replaced.
- `receipt missing` and `forged receipt`: the occupant is refused with
`occupied_root`, and its files stay.
- `later allocation`: a later use of that root on that sandbox without a
final save is refused.
- `never saved here`: a different sandbox with an unknown folder at that
path is refused.
- `npx vitest run src/services/remote-workspaces.integration.test.ts
src/services/remote-workspace-transport.test.ts
src/services/heartbeat/run-completion.test.ts`: 247 passed. This
includes the existing `occupied root` cold restore case, which is still
refused.
- `tsc --noEmit` shows no errors in the changed files. The local tree
has unrelated errors from unbuilt workspace packages.
- I did not run a live staging run. Staging is read-only for this work,
and this PR does not deploy.

## Risks

- The replaced root is removed after the publish. The server proof
states that its content is in the saved lineage. If a process wrote to
that root after its final save and outside a Paperclip run, those writes
are lost. The same assumption applies when authority moves to another
allocation today.
- The proof uses operation records that have `exportCleanup` location
metadata. Older records without that metadata do not qualify, and those
cases still fail closed with `occupied_root`.
- A warm runner process that kept an open working directory in the old
root sees a removed folder. This is not verified live. The new turn is
given the published path, which holds the restored files.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic. It used tool use for
code search, shell, test runs, and read-only staging inspection in the
browser.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 07:07:13 -06:00
..
…
…
…
…
…