fix(native): let a Stop before the provider session opens continue once (#19206)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A remote task restores its saved workspace into a sandbox before the
agent starts.
> - When that restore fails, the run records only
`remote_workspace_operation_failed: operation=restore exit=1`.
> - The run then fails as `setup_failed`. No retry starts, and a
`blocked` task gets no recovery action.
> - An operator cannot tell which step failed, and nobody owns the next
step.
> - This pull request makes the sandbox scripts name their failed step
in a closed grammar, and it gives a restore failure a bounded retry or a
named board recovery action.
> - The benefit is a visible cause and a clear owner. Saved work stays
protected.

## Linked Issues or Issue Description

Refs #19152 and #19187. Those changes name copy-back (save) failures.
Refs #18852 and #19195. Those changes added the bounded bootstrap retry
that this change reuses.

**What happened?**

A board reply woke an agent on a `blocked` parent task. The run failed
during setup 13 seconds later. The only run event was
`remote_workspace_operation_failed: operation=restore exit=1`. The run
had no workspace operation row. The task stayed `blocked`, and its
recovery actions showed `active: null`. The same opaque family also
appeared for workspace preparation (`exit=1` only).

**Expected behavior**

The run names which restore step failed, without command output or
paths. A failure before provider work uses a bounded retry when a retry
can help. Otherwise, and when the retry budget is spent, the board gets
a named recovery action. The saved version is never changed or
overwritten.

**Steps to reproduce**

1. Save a remote workspace, then start a new run on a different
allocation.
2. Put a folder at the published workspace path in the sandbox before
restore.
3. Observe that the base code records only `operation=restore exit=1`,
then `setup_failed`, with no recovery action for a `blocked` task.

**Paperclip version or commit**

Base: `87dfba6818`.

**Deployment mode**

Legacy runtime with sandbox (Daytona) remote workspace storage.

## What Changed

- The prepare, repository, and restore sandbox scripts track their
current stage. On failure they print `remote_workspace_<op>_failed:
stage=<stage>` plus closed fields: `reason=` (the script's own
snake_case refusal), `archive=` (the archive helper's fixed reason),
`code=` (errno), `disk=full`, `git=` (a fixed label chosen from Git
output), and the existing `exit`/`signal`/`program` fields.
- The server accepts that text only when it matches a strict grammar for
the same operation. Other output is dropped, as before.
- Provider refusals during the restore upload and restore command are
recorded by closed labels, like copy-back.
- A failed saved-workspace restore raises `RemoteWorkspaceRestoreError`
with the server step (`snapshot_lookup`, `snapshot_download`,
`sandbox_restore`). It is retryable only when the owner re-reads the
same writer, epoch, and saved version with no unsaved work, and the
sandbox did not name a refusal reason.
- Setup failure handling classifies a pre-dispatch legacy restore
failure as `remote_workspace_restore_failed`. A retryable failure uses
the existing bounded bootstrap retry. A non-retryable or exhausted
failure records a board-owned recovery action with the redacted cause,
when stranded-work recovery does not scan the task status (for example
`blocked`).
- Stranded-work recovery gets a named notice for this code. Generic
continuation recovery does not grant it a second budget, and legacy
reconciliation does not invent provider actions for it.
- `doc/execution-semantics.md` documents the contract.

## Verification

- Red: the new transport tests failed on the base code with
`remote_workspace_operation_failed: operation=restore exit=1` (and the
same for prepare and repository). The new completion tests failed
because the run became `setup_failed` with no retry and no recovery
action.
- Green: `npx vitest run src/services/remote-workspace-transport.test.ts
src/services/heartbeat/run-completion.test.ts
src/services/remote-workspaces.integration.test.ts
src/__tests__/heartbeat-retry-scheduling.test.ts
src/services/legacy-execution-recovery.test.ts src/services/recovery
src/services/remote-workspace-save-concurrency.test.ts
src/services/native-runtime/native-workspace-finalizer-failure.test.ts
src/__tests__/native-workspace-finalization-work.test.ts` passes (563
tests).
- The transport tests use real sandbox commands. They prove an occupied
folder reports `stage=inspect_root reason=occupied_root` and keeps its
files. They prove a wrong manifest reports `stage=extract
archive=manifest_mismatch`. They prove a missing branch reports
`stage=checkout_branch`, and a changed origin reports
`stage=verify_origin`. Forged output with a path is dropped.
- The real-PostgreSQL cold-restore test proves an occupied root gives a
non-retryable `sandbox_restore` failure and keeps the unknown files.
- The completion tests cover retried, exhausted, refused, post-dispatch,
and `in_progress` cases.
- `tsc --noEmit` for the server reports no errors in the changed files.

## Risks

- A retry keeps the same restore path. If the cause is deterministic but
unnamed, the run uses its two retries before the board action. Named
refusals skip the retries.
- The board action is recorded only for statuses that stranded-work
recovery does not scan, and only when no other action is active. This
prevents a second recovery identity for one failure.
- Script stderr grammar is new. Old sandboxes run the script that the
server sends, so no runner update is necessary.
- No migration, deployment, or credential change.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with
tool use, code execution, and browser tools for read-only staging
evidence. Context window and reasoning mode are host-managed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

server/src/services/native-runtime/native-session-executor.test.ts

@@ -8421,6 +8421,107 @@ describe("native session cancellation", () => {
     }
   });
 
+  it.each(["in-time", "late"] as const)("records that a Stop during startup (%s) preceded any provider turn", async timing => {
+    const root = await mkdtemp(join(tmpdir(), "native-startup-stop-proof-"));
+    const previous = process.env.PAPERCLIP_RUNNER_STATE_DIR;
+    process.env.PAPERCLIP_RUNNER_STATE_DIR = root;
+    let open!: () => void, started!: () => void;
+    const opening = new Promise<void>(resolve => { open = resolve; });
+    const admitted = new Promise<void>(resolve => { started = resolve; });
+    const submitTurn = vi.fn();
+    state.cancel.mockClear();
+    state.execute.mockReset().mockImplementationOnce(async (options) => {
+      started(); await opening;
+      await options.onSession({ cancel: state.cancel });
+      submitTurn();
+      throw new Error("a stopped startup must not reach prompt submission");
+    });
+    const runResultJson: Record<string, unknown> = {};
+    const updates: Array<{ table: unknown; values: Record<string, unknown> }> = [];
+    const outcome = executePaperclipNativeSession({ db: leaseDb(execution, {}, runResultJson, updates), execution, runnerInstanceId: "runner" })
+      .catch(error => error);
+    const persistence = cancellationDb();
+    try {
+      await admitted;
+      if (timing === "in-time") {
+        const stopping = cancelNativeSession(execution.binding.runId, "operator Stop", { db: persistence.db, scope: "run" });
+        await new Promise(resolve => setImmediate(resolve));
+        open();
+        expect(await stopping).toMatchObject({ dispatched: true });
+        await outcome;
+        expect(persistence.getResultJson().nativeCancellation).toMatchObject({ dispatchState: "acknowledged", beforeProviderTurn: true });
+      } else {
+        vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] });
+        const stopping = cancelNativeSession(execution.binding.runId, "operator Stop", { db: persistence.db, scope: "run" }).catch(error => error);
+        await new Promise(resolve => setImmediate(resolve));
+        await vi.advanceTimersByTimeAsync(30_000);
+        expect(await stopping).toMatchObject({ name: "NativeCancellationPendingRecoveryError" });
+        vi.useRealTimers();
+        Object.assign(runResultJson, persistence.getResultJson());
+        open(); await outcome;
+        const acknowledgement = updates.find(update => update.table === heartbeatRuns &&
+          (update.values.resultJson as Record<string, any> | undefined)?.nativeCancellation?.dispatchState === "acknowledged");
+        expect(acknowledgement?.values.resultJson).toMatchObject({ nativeCancellation: { beforeProviderTurn: true } });
+      }
+      expect(state.cancel).toHaveBeenCalledOnce();
+      expect(submitTurn).not.toHaveBeenCalled();
+    } finally {
+      vi.useRealTimers(); open(); await outcome;
+      if (previous === undefined) delete process.env.PAPERCLIP_RUNNER_STATE_DIR;
+      else process.env.PAPERCLIP_RUNNER_STATE_DIR = previous;
+      await rm(root, { recursive: true, force: true });
+    }
+  });
+
+  it("records no pre-turn proof for a Stop after the provider handle was published", async () => {
+    const running = executePaperclipNativeSession({ db: leaseDb(), execution, runnerInstanceId: "runner" });
+    await vi.waitFor(() => expect(state.release).toBeTypeOf("function"));
+    const persistence = cancellationDb();
+    try {
+      expect(await cancelNativeSession(execution.binding.runId, "operator Stop", { db: persistence.db, scope: "run" }))
+        .toMatchObject({ dispatched: true });
+      expect(persistence.getResultJson().nativeCancellation).toMatchObject({ dispatchState: "acknowledged" });
+      expect(persistence.getResultJson().nativeCancellation).not.toHaveProperty("beforeProviderTurn");
+    } finally { state.release?.(); await running; }
+  });
+
+  it("records no pre-turn proof for a startup Stop under restart recovery", async () => {
+    // A recovered controller may reattach a provider that already worked.
+    const current = structuredClone(execution);
+    current.binding.runId = "restart-recovery-startup-stop";
+    const { currentNativeControllerIdentity } = await import("./native-restart-recovery.js");
+    const controller = await currentNativeControllerIdentity();
+    const db = leaseDb(current, { leaseOwner: "recovered-controller", controllerBootId: controller.bootId,
+      controllerPid: controller.pid, controllerGeneration: 2, attempt: 1 });
+    let open!: () => void, started!: () => void;
+    const opening = new Promise<void>(resolve => { open = resolve; });
+    const admitted = new Promise<void>(resolve => { started = resolve; });
+    const submitTurn = vi.fn();
+    state.cancel.mockClear();
+    state.execute.mockReset().mockImplementationOnce(async (options) => {
+      started(); await opening;
+      await options.onSession({ cancel: state.cancel });
+      submitTurn();
+      throw new Error("a stopped startup must not reach prompt submission");
+    });
+    const outcome = executePaperclipNativeSession({ db, execution: current, runnerInstanceId: "runner",
+      leaseOwner: "recovered-controller", restartRecovery: { kind: "bootstrap_incomplete" as const, runId: current.binding.runId,
+        leaseOwner: "recovered-controller", controllerGeneration: 2, providerAttempt: 1, restartKind: "hard" as const,
+        recoveryRequestId: null } }).catch(error => error);
+    const persistence = cancellationDb({ runId: current.binding.runId });
+    try {
+      await admitted;
+      const stopping = cancelNativeSession(current.binding.runId, "operator Stop", { db: persistence.db, scope: "run" });
+      await new Promise(resolve => setImmediate(resolve));
+      open();
+      expect(await stopping).toMatchObject({ dispatched: true });
+      await outcome;
+      expect(persistence.getResultJson().nativeCancellation).toMatchObject({ dispatchState: "acknowledged" });
+      expect(persistence.getResultJson().nativeCancellation).not.toHaveProperty("beforeProviderTurn");
+      expect(submitTurn).not.toHaveBeenCalled();
+    } finally { open(); await outcome; }
+  });
+
   it("routes control-plane cancellation to the active normalized session and removes the handle", async () => {
     const running = executePaperclipNativeSession({
       db: leaseDb(),