Files
PaperClipAI/server
DottaandPaperclip ce7766dc11 fix(recovery): resume retries with remote workspace storage (#18910)
## Thinking Path

> - Paperclip manages work performed by AI agents.
> - Native recovery must resume an interrupted run under its saved
ownership.
> - Remote workspace storage records that ownership in
`runnerProfileJson.remoteWorkspace`.
> - Recovery recognized only the older workspace record and plugin task
leases.
> - A retained remote PID could therefore leave a due retry without a
dispatch.
> - This PR routes the new record through the existing remote recovery
verifier.
> - Checkpoint, lease, process and single-executor checks remain
authoritative.

## Linked Issues or Issue Description

**What happened?**

After a native sandbox turn returned without a semantic result, same-run
recovery was saved with a retry deadline. The run retained its sandbox
PID and the new remote workspace reference. The recovery selector
skipped it. Later task attempts could then fail on the unfinished
workspace writer.

**What did you expect to happen?**

The due retry must enter existing remote checkpoint and ownership
verification. Recovery must resume the original run once or expose the
real verification blocker.

**Steps to reproduce**

Persist a retryable native failure with a remote PID, a prepared
`paperclip.remote-workspace-run.v1` reference, an existing lease and
checkpoint, then run heartbeat recovery. The added PostgreSQL cases
reproduce this shape.

**Deployment mode**

Native Paperclip Runner in a sandbox with remote workspace storage.

Related: #18841 handles legacy workspace recovery during storage
migration. #18908 handles failed native preparation. This repair covers
a later, provider-started retry.

## What Changed

- Recognize the current remote workspace record when classifying process
location and selecting remote retries.
- Extend the existing database and real-process fixtures to cover the
new record, including a retained PID without a process group.
- Reuse existing restart recovery. Add no new retry loop, ownership
bypass, migration or workflow change.

## Verification

- Before the repair: three new behavioral cases failed. Two retries
never dispatched; a missing checkpoint never reached its visible
recovery hold.
- After the repair: all 82 heartbeat recovery tests pass.
- All 57 neighboring resumption-binding, restart-recovery and
remote-probe tests pass.
- Safety cases retain retry deadlines, live controller ownership, active
execution, missing-checkpoint rejection, single dispatch and zero host
signaling of the sandbox PID.
- Local platform: macOS arm64, Node 24.22.2, PostgreSQL 18.1. The tests
use disposable databases and an owned live process as the remote PID
collision control. Provider execution is outside this routing
regression.
- Full `pnpm -r typecheck` and `pnpm build` pass at
`34aa2d08d38e8afec7f8d98ce984a74551caeda6`.
- Current-head GitHub checks: 53 successful; `ci / verify` passed.
Paperclip Review completed neutral with a 5/5 assessment and no
findings. Two Storybook jobs were skipped by scope; the external
security scan was neutral. No failures, pending checks, or merge
conflicts remain.
- Greptile: 5/5 at the same commit, with no review threads.
- The duplicate full local `pnpm test:run` was interrupted after
current-head CI passed. Its first two chat shards passed (574 cases); it
has no complete local full-suite verdict. This is separate from the
passing targeted recovery suites and required CI test lanes.
- Staging diagnosis used read-only run, finalization, workspace and
lease evidence. The fixed revision has not been deployed. The original
task's useful continuation remains unverified.
- Initial local attempts stopped before test execution because generated
SDK output and PostgreSQL library aliases were missing. Those setup
failures are retained separately from the behavioral red/green evidence.

## Risks

A wrong remote classification could send a run to the wrong ownership
path. The existing verifier still checks the saved checkpoint,
controller lease and target. Tests require one dispatch, preserve
process identifiers, and prove that recovery never inspects or signals a
coincident host PID. Legacy workspace and plugin task coverage remains
green. This repairs missed continuation; it does not establish the cause
of the earlier missing semantic result.

## Model Used

OpenAI GPT-6 through Codex, with reasoning and tool execution. The exact
model snapshot and context window were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-03 21:06:50 -06:00
..