fix(native): let a Stop before the provider session opens continue once (#19206)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A remote task restores its saved workspace into a sandbox before the
agent starts.
> - When that restore fails, the run records only
`remote_workspace_operation_failed: operation=restore exit=1`.
> - The run then fails as `setup_failed`. No retry starts, and a
`blocked` task gets no recovery action.
> - An operator cannot tell which step failed, and nobody owns the next
step.
> - This pull request makes the sandbox scripts name their failed step
in a closed grammar, and it gives a restore failure a bounded retry or a
named board recovery action.
> - The benefit is a visible cause and a clear owner. Saved work stays
protected.

## Linked Issues or Issue Description

Refs #19152 and #19187. Those changes name copy-back (save) failures.
Refs #18852 and #19195. Those changes added the bounded bootstrap retry
that this change reuses.

**What happened?**

A board reply woke an agent on a `blocked` parent task. The run failed
during setup 13 seconds later. The only run event was
`remote_workspace_operation_failed: operation=restore exit=1`. The run
had no workspace operation row. The task stayed `blocked`, and its
recovery actions showed `active: null`. The same opaque family also
appeared for workspace preparation (`exit=1` only).

**Expected behavior**

The run names which restore step failed, without command output or
paths. A failure before provider work uses a bounded retry when a retry
can help. Otherwise, and when the retry budget is spent, the board gets
a named recovery action. The saved version is never changed or
overwritten.

**Steps to reproduce**

1. Save a remote workspace, then start a new run on a different
allocation.
2. Put a folder at the published workspace path in the sandbox before
restore.
3. Observe that the base code records only `operation=restore exit=1`,
then `setup_failed`, with no recovery action for a `blocked` task.

**Paperclip version or commit**

Base: `87dfba6818`.

**Deployment mode**

Legacy runtime with sandbox (Daytona) remote workspace storage.

## What Changed

- The prepare, repository, and restore sandbox scripts track their
current stage. On failure they print `remote_workspace_<op>_failed:
stage=<stage>` plus closed fields: `reason=` (the script's own
snake_case refusal), `archive=` (the archive helper's fixed reason),
`code=` (errno), `disk=full`, `git=` (a fixed label chosen from Git
output), and the existing `exit`/`signal`/`program` fields.
- The server accepts that text only when it matches a strict grammar for
the same operation. Other output is dropped, as before.
- Provider refusals during the restore upload and restore command are
recorded by closed labels, like copy-back.
- A failed saved-workspace restore raises `RemoteWorkspaceRestoreError`
with the server step (`snapshot_lookup`, `snapshot_download`,
`sandbox_restore`). It is retryable only when the owner re-reads the
same writer, epoch, and saved version with no unsaved work, and the
sandbox did not name a refusal reason.
- Setup failure handling classifies a pre-dispatch legacy restore
failure as `remote_workspace_restore_failed`. A retryable failure uses
the existing bounded bootstrap retry. A non-retryable or exhausted
failure records a board-owned recovery action with the redacted cause,
when stranded-work recovery does not scan the task status (for example
`blocked`).
- Stranded-work recovery gets a named notice for this code. Generic
continuation recovery does not grant it a second budget, and legacy
reconciliation does not invent provider actions for it.
- `doc/execution-semantics.md` documents the contract.

## Verification

- Red: the new transport tests failed on the base code with
`remote_workspace_operation_failed: operation=restore exit=1` (and the
same for prepare and repository). The new completion tests failed
because the run became `setup_failed` with no retry and no recovery
action.
- Green: `npx vitest run src/services/remote-workspace-transport.test.ts
src/services/heartbeat/run-completion.test.ts
src/services/remote-workspaces.integration.test.ts
src/__tests__/heartbeat-retry-scheduling.test.ts
src/services/legacy-execution-recovery.test.ts src/services/recovery
src/services/remote-workspace-save-concurrency.test.ts
src/services/native-runtime/native-workspace-finalizer-failure.test.ts
src/__tests__/native-workspace-finalization-work.test.ts` passes (563
tests).
- The transport tests use real sandbox commands. They prove an occupied
folder reports `stage=inspect_root reason=occupied_root` and keeps its
files. They prove a wrong manifest reports `stage=extract
archive=manifest_mismatch`. They prove a missing branch reports
`stage=checkout_branch`, and a changed origin reports
`stage=verify_origin`. Forged output with a path is dropped.
- The real-PostgreSQL cold-restore test proves an occupied root gives a
non-retryable `sandbox_restore` failure and keeps the unknown files.
- The completion tests cover retried, exhausted, refused, post-dispatch,
and `in_progress` cases.
- `tsc --noEmit` for the server reports no errors in the changed files.

## Risks

- A retry keeps the same restore path. If the cause is deterministic but
unnamed, the run uses its two retries before the board action. Named
refusals skip the retries.
- The board action is recorded only for statuses that stranded-work
recovery does not scan, and only when no other action is active. This
prevents a second recovery identity for one failure.
- Script stderr grammar is new. Old sandboxes run the script that the
server sends, so no runner update is necessary.
- No migration, deployment, or credential change.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with
tool use, code execution, and browser tools for read-only staging
evidence. Context window and reasoning mode are host-managed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed 2026-11-05 03:13:44 -06:00
1 parent 833d068741
commit beb73381ca
6 files changed
+322 -13

No files matched your search

+17
View File
@@ -2154,6 +2154,23 @@ is the current review participant. Before the receipt, the hold prevents a secon
executor on the workspace. A snapshot-only save does not release the hold,
because the task workspace does not contain the saved files.
A Stop or reassignment can reach a native run while its provider session is
still opening. The startup owner then fences the late handle before any prompt,
and the Stop acknowledgement records `nativeCancellation.beforeProviderTurn`.
A reattached provider from restart recovery never gets this proof. When the run
also has no runner event outside the startup lifecycle (harness, session, workspace,
runtime phase, credential refresh), no turn or provider event, and
no native result, no provider outcome
is unknown. The completed copyback receipt then releases the hold for the stopped
run's own agent as well, so the user's redirect continues once. The hold row and
its evidence are unchanged.
A queued turn for the same task that never started is not a workspace writer for
automatic recovery. The hold gates its dispatch until the recovery receipt is
complete, so recovery and that turn cannot both wait for each other. Running
turns, and queued turns for other tasks that share the workspace, still block
recovery.
The deadline is a backstop, not the normal failure path. When native task
preparation fails after a fresh controller claim and before any provider turn,
the controller stops renewal and releases only that exact claim. Ordinary