Files
PaperClipAI/server/src/__tests__
DottaandClaude Opus 5.5 5a20fca9f7 fix(recovery): judge an orphaned run's pid only in the PID space that recorded it (#19871)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The orphan reaper (`terminalizeOrphanedRunningRun`) ends a `running`
run whose recorded process is gone
> - It judged "gone" with `kill(pid, 0)` on the local host, wherever the
process actually ran: another host, another PID namespace, or a sandbox
> - So a run whose process is alive elsewhere could be terminalized
while it still works
> - #19868 fixed the same gap for native coordinator takeover and the
runtime-control sweep. This pull request finishes finding 2 for the
reaper
> - The benefit is that a dead local pid never proves an owner on
another host is gone

## Linked Issues or Issue Description

**Bug description**
The Phase 0 ownership inventory (finding 2) names the orphan reaper
`terminalizeOrphanedRunningRun` (`recovery/service.ts`) as a proof that
accepts "the pid is dead here" for a process anywhere. `heartbeat_runs`
recorded `process_pid` with no PID space, and a sandbox pid could
collide with a local one.

**Steps to reproduce**
1. A run records `process_pid` on host A, or in a sandbox.
2. On host B, that pid number is not alive.
3. B's stale-lock sweep terminalizes the run as `orphaned_running_run`.

**Expected behavior**
The process-death authority applies only in the PID space that recorded
the pid. Issue-terminal authority is unchanged.

Refs #19846, follows #19868

## What Changed

- Migration `0442`: add nullable `heartbeat_runs.process_host_hash`.
- `heartbeat/run-lifecycle.ts` `persistHeartbeatRunProcessMetadata`, the
one writer of `process_pid`, stores `currentProcessSpaceHash()` for a
local pid and `"remote"` for a sandbox pid.
- `recovery/service.ts`: the process-death authority runs only when
`process_host_hash` is null (rows saved before this column, for one
release) or `sameProcessSpace` matches.
- Tests: the reaper leaves a dead pid alone for another host, a sandbox,
and an unreadable space, and terminalizes it in this space. The writer
stamps local and remote pids.

## Verification

- `npx vitest run
src/__tests__/recovery-stale-issue-lock-sweep.test.ts`: 46 passed.
- Red before the fix: with the old `recovery/service.ts`, the three
cross-space cases fail (the run is terminalized). Green after.
- `pnpm db:generate` produced `0442_low_salo.sql` (one `ADD COLUMN`),
and the snapshot set was pruned to five. `pnpm --filter @paperclipai/db
check:migrations` passed. Server `tsc --noEmit` shows no errors in the
changed files.

## Risks

- Previous-release story: runs saved with `process_host_hash = null`
keep the previous local proof for this release. That matches #19868,
because stranding in-flight runs at upgrade would be a restart failure.
New rows are strict.
- A run whose process really lived on another host now stays `running`
until issue-terminal authority, its controller lease, or another owner
settles it.
- No new I/O before `turn.start`. The hash is computed at module load,
and the stamp rides the existing process-metadata update.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used with tool use and
code execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 12:22:15 -06:00
..
…

Server Tests

Server tests that need a real PostgreSQL process must use ./helpers/embedded-postgres.ts instead of constructing embedded-postgres directly.

The shared helper creates a throwaway data directory and a reserved-safe loopback port for each test database. This protects the live Paperclip control-plane Postgres from server vitest runs; see PAP-2033 for the incident that introduced this guard.