Files
PaperClipAI/server
DottaandClaude Opus 5.5 b6f58dc6e2 feat(recovery): settle runtime-control operations of proven-gone owners on the shared path (#20016)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A managed runtime control (start, stop, restart, repair, run) on an
execution workspace writes a `running` operation row that blocks every
other control on that workspace
> - If the server that ran it crashes, the row stays `running` until the
next control request sweeps it. The sweep had its own `kill(pid, 0)`
check, with no process start time, so a reused PID looked alive until 60
s of heartbeat silence passed
> - The durable ownership plan proves owners by identity through one
shared prover and settles them from one reconciler
> - This pull request stamps the owner's boot id and start time, judges
it with `proveOwnerGone`, and registers a settle for the kind
> - The benefit is that a workspace stranded by a crashed server frees
itself in the next reconciler pass, and a reused PID no longer holds it

## Linked Issues or Issue Description

Refs #19831 (plan, Phase 2 sections A and C), #19838 (inventory:
"Managed runtime-control operation", and the "One fact, many proofs"
table row `ownerProcessIsAlive` workspace-operations.ts).

**What happened?**
`workspace_runtime_control_operation` was a report-only kind with a
bespoke liveness check: `ownerProcessProvenGone` probed the PID with
signal 0 and had no start time. A server restart that reused the PID
kept the operation "alive" until its heartbeat aged out, and nothing
settled the row until another control request arrived.

**Expected behavior**
The owner is proven by the shared prover: a dead or reused PID in the
stamp's PID space is gone. Another host, a stamp without a PID space, or
a live PID without a recorded start time stays unprovable and keeps the
heartbeat lease as its fenced takeover at the next control.

## What Changed

- `workspace-operations.ts`:
- The owner stamp adds the server's `bootId` and `processStartedAt`
(`currentRuntimeControlOwner`, no I/O).
- The new `runtimeControlOperationOwnerStamp` is shared with the
registry.
- The sweep proves the owner with `proveOwnerGone`.
`ownerProcessProvenGone` is deleted.
- The terminal CAS and log append are factored out
(`claimRuntimeControlTermination`,
`appendRuntimeControlReconciliationLog`).
- New `settleRuntimeControlOperation` for the reconciler: it skips
operations this process is driving, requires the same owner as the
proof, CAS-fails the row and writes a `workspace_operation.reconciled`
activity entry.
- `recovery/ownership-registry.ts`: the kind uses the shared stamp and
registers the settle.
- The settling-kinds list in `ownership-reconciliation-report.test.ts`
is updated.

## Verification

- `workspace-runtime-control-recovery.test.ts`: 21 pass. The 6 new tests
are in "shared ownership path". 5 fail on `master` and pass here:
  - a reused PID is recovered at once;
  - the reconciler settles a dead owner with activity and frees control;
- an unprovable owner is never settled, and a read-only pass changes
nothing;
  - a re-stamped owner is kept;
  - new operations carry the boot id and start time.

  The live-owner guard passes on both.
- `ownership-reconciliation-report.test.ts` (25),
`workspace-operations-reconciliation.test.ts` (9) and
`execution-workspace-runtime-control-conflict.test.ts` (24) pass.
- The server package typecheck passes.

## Risks

- Rows written before this change have no start time. A live PID on such
a row stays unprovable, and the 60 s heartbeat lease still applies at
the next control. A dead PID is still proof.
- The reconciler now terminalizes these rows. It does so only on `gone`,
and only with the same owner and an `updated_at` CAS.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 05:07:58 -06:00
..