Files
PaperClipAI/packages
DottaandClaude Opus 5.5 68db7ed9e3 fix(recovery): never accept a dead local pid as proof for an owner on another host (#19868)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - After a restart, recovery decides that an in-progress record's owner
is gone and takes it over
> - Two of those proofs use `kill(pid, 0)` on the local host. A pid that
is dead here says nothing about a process on another host or in another
PID namespace
> - So a controller or runtime-control owner that is still alive
elsewhere can be taken over while it still runs
> - This pull request records the owner's PID space (the existing
`controller-process-identity` host hash) and judges a pid only in that
same space
> - The benefit is that a different host is never proof. Those records
follow their existing lease or TTL instead

## Linked Issues or Issue Description

**Bug description**
The Phase 0 ownership inventory (finding 2) found three proofs that
accept "the pid is dead here" as proof for an owner on another host:
`evaluateNativeControllerTakeover` (`native_run_finalizations` has no
host column), the runtime-control sweep (`workspace-operations.ts`), and
the orphan reaper `terminalizeOrphanedRunningRun`. Only
`controllerProcessExited` checked the host.

**Steps to reproduce**
1. Run two control-plane processes on different hosts, or in different
PID namespaces, against one database.
2. Let host A own a native finalization lease or a runtime-control
operation.
3. On host B, run restart recovery. If A's pid number is not alive on B,
B takes over the live owner.

**Expected behavior**
An owner whose PID space differs from this process's, or cannot be read,
is unprovable. Its record waits for its lease, fence, or TTL.

Refs #19846

## What Changed

- Migration `0441`: add nullable
`native_run_finalizations.controller_host_hash`.
- `controller-process-identity.ts`: export `currentProcessSpaceHash()`
and `sameProcessSpace()`. An unreadable space is stored as `"unknown"`,
which never matches.
- `native-restart-recovery.ts`: `currentNativeControllerIdentity`
carries `hostHash`, which both coordinator claim writes
(`native-restart-recovery.ts`, `native-session-executor.ts`) store.
`evaluateNativeControllerTakeover` returns `controller_host_unprovable`
before any pid probe when the recorded space is not this one.
`native-dead-controller-result-lease.ts` selects the column.
- `workspace-operations.ts`: runtime-control owner stamps record
`hostHash`. `ownerProcessProvenGone` requires the same PID space (EPERM
= alive), otherwise the existing staleness TTL applies.
- Not in this PR: the orphan reaper `terminalizeOrphanedRunningRun`.
`heartbeat_runs` has no host for `process_pid`, and its writers are
spread across legacy and native paths. It follows as a separate PR with
its own column.

## Verification

- `npx vitest run
src/services/native-runtime/native-restart-recovery.test.ts
src/__tests__/workspace-runtime-control-recovery.test.ts`: 55 passed.
New cases cover another host and an unreadable space (dead and recycled
pid) for takeover, plus another host, unreadable space, and legacy stamp
for the sweep. The sweep cases also confirm that the TTL still reclaims
after the heartbeat lapses.
- Red before the fix: the 5 new cases fail on the old code. Green after.
- `native-controller-recovery.integration.test.ts`,
`native-failed-run-owner-release.test.ts`,
`native-startup-cleanup-recovery.test.ts`: 71 passed.
- `pnpm db:generate` produced `0441_lucky_chameleon.sql` (one `ADD
COLUMN`) and pruned to five snapshots. `pnpm --filter @paperclipai/db
check:migrations` passed. Server `tsc --noEmit` shows no errors in the
changed files.

## Risks

- Previous-release story: `native_run_finalizations` rows saved before
this column have `controller_host_hash = null`. They keep the previous
local dead-pid proof for this release. Making them unprovable would
leave every native run in flight at the upgrade in `awaiting_evidence`
with no TTL, because local takeovers have no lease fallback. Rows
written by this release onward are strict. Runtime-control stamps
without `hostHash` are strict now, because that path has a staleness
TTL.
- Owners built from other receipts (workspace finalization owner,
workspace stop proof) pass no host and keep the previous behavior. Track
2's shared `OwnerStamp` should absorb them.
- A new row on a host that really moved (redeploy onto a new machine)
waits for its lease or an operator instead of being taken over by pid.
That is the intended "different host is never proof" rule.
- No new I/O before `turn.start`. The hash is computed once at module
load.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used with tool use and
code execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 11:14:48 -06:00
..