Files
PaperClipAI/server
DottaandPaperclip f513f0efd9 fix(recovery): admit certified process-loss preparation retries (#19552)
## Thinking Path

> - Paperclip keeps task work durable across server restarts.
> - Restart recovery records a certificate when a Runner preparation
ended before provider work started.
> - Automatic recovery already verifies that certificate together with
saved files and stopped leases.
> - Explicit Retry called a preparation helper that only reached this
proof for status `interrupted`.
> - A certified `failed/process_lost` source therefore stayed blocked
after its files were recovered.
> - This pull request makes the helper reuse the same existing proof.
> - A queued user Retry can then admit one new turn while all current
safety and policy gates remain effective.

## Linked Issues or Issue Description

**What happened?**

A task had a certified process-lost preparation, verified
saved-workspace receipts, a destroyed source sandbox, and one or two
later attempts refused before dispatch. Its accepted Retry request
stayed deferred with `execution_recovery` and created no successor.

The explicit admission helper excluded this source before it could use
the preparation proof added in #19464. This also prevented the
refused-attempt continuation path from #19466 from advancing.

**Expected behavior**

Explicit Retry must use the same positive pre-provider proof as restart
recovery. It must preserve source history, refuse contradictory
execution evidence, honor current policy gates, and admit at most one
successor for the existing request.

**Steps to reproduce**

Persist a legacy preparing run as failed with `process_lost` and the
host bootstrap certificate that provider work did not start. Record its
canonical committed snapshot, recovery/finalization operations, clean
workspace authority, and exact destroyed-lease receipt. Add one or two
unstarted cancelled descendants with the dispatcher's execution-wait
receipt. Accept one Retry request and resume it concurrently twice.
Before this fix, no successor exists and the same request remains
deferred.

**Paperclip version or commit**

Reproduced on master `a6c22806308e8a3c7b21145f80fb471756bb443d`. Related
repairs: #19464 and #19466.

## What Changed

- Reuse `isInterruptedNativePreparation` before the existing
failed-preparation checks. The existing no-coordinator and
historical-ancestor restrictions remain.
- Add real-database regressions for one and two refused descendants,
duplicate clicks, concurrent resume, continuation dispatch, preserved
history, and unchanged recovery attempt counts.
- Add late-gate checks for provider work, provider events, a newer
native checkpoint, missing bootstrap proof, missing stop proof, a
changed snapshot, formal approval, user Stop, agent/company pause, and a
spent budget.
- Clarify that explicit admission uses the existing certified
preparation proof.

## Verification

- Red: both real-database cases passed the saved-file,
interrupted-preparation, and refused-dispatch proof checks, but the
original successor-count assertion found zero queued runs instead of
one. The accepted requests retained `execution_recovery`.
- Green: the same cases admit one successor after two concurrent resume
calls. Continuation dispatch resolves to the latest refused attempt.
Source and cancelled-attempt history stay intact.
- Complete relevant local suites: **456/456 passed** before rebase.
Command from `server`: `../node_modules/.bin/vitest run --config
vitest.config.ts --fsModuleCache src/services/task-retry.test.ts
src/services/explicit-native-continuation.test.ts`.
- Final head `9788382c019df57899af06e81211e7c5165e7f0d`: the thirteen
new cases passed **13/13** again after rebase. Required CI and review
remain pending.
- Environment: macOS ARM64, Node 26, Vitest 5.0.3, disposable real
PostgreSQL. Tests exercise production request, admission, and
continuation services. No provider executes and no hosted task is
changed.
- Hosted recovery still requires rollout and verification through the
original queued requests.

Final reviewed head: `9788382c019df57899af06e81211e7c5165e7f0d`.
[Required
CI](https://github.com/paperclipai/paperclip/actions/runs/42273663624)
passed (56 successful checks, four expected skips, none pending or
failed). Greptile and Paperclip Review both rated this head 5/5 with no
findings; all review threads are resolved.

## Risks

The change accepts only the existing verified preparation contract. An
error label alone is insufficient. Provider-started or contradictory
state remains blocked. Saved-file, lease, approval, Stop, pause, and
budget checks remain effective. No schema, retry limit, provider
setting, account selection, or production scheduler changes.

## Model Used

OpenAI GPT-6 family via Codex, with coding and tool use. The exact model
ID and context window are unavailable in this session's metadata.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-06 13:03:21 -06:00
..