Files
PaperClipAI/server
DottaandClaude Opus 5.5 20cc6faca3 feat(recovery): settle legacy shutdown fences of proven-gone servers on the shared path (#20017)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - When a server shuts down gracefully during a sandbox run, it commits
a fence on the run's lease so no cleanup worker destroys the sandbox
before its files are exported
> - The fence names its server only by boot id. It is released when its
owner settles, or when its 30 s deadline (renewed every 10 s) passes.
After a restart, the sweep still waits for that deadline even though the
old server is gone
> - The durable ownership plan proves owners by identity through one
shared prover and settles them from one reconciler
> - This pull request stamps the committing process on the fence and
settles it when the shared prover shows that process gone. The outcome
is the same safe one an expired deadline gives: the source is preserved
and the restore failure recorded
> - The benefit is that the fence releases on proof, not age, and a
fence whose owner may still be alive (another host, or an older marker)
keeps its deadline

## Linked Issues or Issue Description

Refs #19831 (plan, Phase 2 sections A and C), #19838 (inventory: "Legacy
graceful-shutdown workspace settlement fence", and "A boot-id mismatch
alone does not release").

**What happened?**
`legacy_shutdown_settlement_fence` was report-only, and its owner stamp
had only a boot id. A fence left by a crashed or replaced server could
be released only by its deadline.

**Expected behavior**
A fence whose committing process is proven gone (a dead or reused PID in
the same PID space) settles from the recovery sweep and from the shared
reconciler. An unprovable owner keeps the renewed deadline as its fenced
takeover.

## What Changed

- `legacy-shutdown-workspace-settlement.ts`:
- `beginLegacyShutdownWorkspaceSettlement` stamps `controller`
(`currentControllerProcessStamp`, no I/O).
  - New `legacyShutdownSettlementOwnerStamp`.
- `allowLegacyShutdownWorkspaceCleanup` takes `ownerProvenGone`, which
replaces only the deadline check, and only while the marker still names
that process. Every other guard is unchanged: owner stop, competing
allocations, in-flight cleanup.
- The sweep also proves the owners of fences from other boots that are
still before their deadline.
  - New `settleLegacyShutdownSettlementFence` for the reconciler.
- `recovery/ownership-registry.ts`: the kind uses the full stamp when
present and registers the settle.
- The settling-kinds list in `ownership-reconciliation-report.test.ts`
is updated.

## Verification

- `heartbeat/recovery.test.ts`: 100 pass. Three new tests fail on
`master` and pass here:
  - the fence stamps the committing process;
- a previous server proven gone settles before the deadline (restore
failure recorded);
- the reconciler settles only the proven-gone owner (a read-only pass
changes nothing). Another host and a boot-id-only marker stay pending
until their deadline, then expire as before.
- `ownership-reconciliation-report.test.ts` (25) and
`environment-retained-orphan-sandbox.test.ts` (19) pass.
- The server package typecheck passes.

## Risks

- A proven-gone owner now expires the fence up to 30 s earlier than
before. Expiry preserves the source (stop-and-retain or a
`retain_on_failure` hold) and never destroys provider files.
- Markers written before this change have no `controller` and behave
exactly as before.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 06:15:36 -06:00
..