Files
PaperClipAI/tests
DottaandClaude Opus 5.5 3fedfe8763 fix(runner-e2e): wait out a briefly uncertain owned group instead of failing the stop (#19692)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The restart chaos harness and the runner e2e suites stop and restart
the control plane through one process-tree owner
(`tests/runner-e2e/process-tree-owner.ts`)
> - The owner signals only process groups whose identity it has proven,
and refuses everything when a group's identity becomes uncertain
> - Under load a group can turn over between two `ps` snapshots: the
observed members exit, and a straggler that nobody saw start is left in
the group, reparented away from every owned anchor
> - The stopper treated that as fatal on the first inspection, so a K2
graceful stop at load 28 failed in under a second, and a chaos storm
wave was marked as a harness error although the server came back
> - This pull request makes the stopper re-inspect within its existing
deadline until the uncertainty settles. It still never signals the
uncertain group, and it fails with the original error if the uncertainty
persists
> - The benefit is that chaos waves under load stop failing on a harness
race, without weakening the rule against signaling unproven processes

## Linked Issues or Issue Description

Refs #19470

**What is the problem?**
`stopOwnedProcessTree` fails at once with "Owned process group identity
became uncertain" when, under load, a group's observed members exit and
an unobserved straggler remains for a moment. In a chaos storm this
failed a K2 stop 0.9 s after SIGTERM (load 28), and the wave exited as a
harness error.

**What is the expected behavior?**
The stopper waits (within the same grace or forced deadline) for a later
inspection that settles the group, and it never signals the uncertain
group. If the straggler outlives the deadline, the stop still fails with
the same error.

**Why does it matter?**
A harness-only race must not invalidate a chaos wave, and the fix must
not let the harness signal a process whose ownership is not proven.

## What Changed

- `tests/runner-e2e/server-stop.ts`: the chaos and e2e server stopper
(the actual K2 path) uses the same bounded settle-wait for its graceful
observe/signal and its SIGKILL escalation. Live evidence: a K2 at load
13.9 failed in 0.5 s on an ACPX Claude provider group (launcher, reaper,
module-hook loader, `claude` runtime) that tore itself down during the
drain and was gone 0.4 s later.
- `tests/runner-e2e/process-tree-owner.ts`: the uncertainty error now
names the unowned members (pid, ppid, pgid, kind, start).
`settleUncertainIdentity` is exported and shared. Export the
`UNCERTAIN_GROUP_IDENTITY` message prefix. `stopOwnedProcessTree`
retries the grace signal, the SIGKILL escalation and the stop check
every 100 ms while that error persists, bounded by the existing grace
deadline and one forced window (set before escalation and reused by the
escalation retry, the stop check and the fallback), so a stop never
exceeds gracefulMs + forcedMs. Other errors, and an uncertainty that
outlives the deadline, take the existing fallback path unchanged. The
owner's identity checks do not change, and neither does `signal()`.
- `tests/runner-e2e/process-tree-owner.test.ts`: three tests. An
uncertainty that starts only at SIGKILL escalation fails within one
forced window and never sends SIGKILL. A brief turnover settles and the
stop succeeds without signaling the uncertain group or killing the
direct child. A persistent turnover still rejects with "identity became
uncertain" and never signals.

## Verification

- `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts
tests/runner-e2e/process-tree-owner.test.ts
tests/runner-e2e/server-stop.test.ts`: 35 pass across
`process-tree-owner.test.ts` and `server-stop.test.ts`, including 5 new
tests; the new brief-turnover tests fail on the old stoppers. With the
old stopper, the first new test fails: it rejects on the first
inspection and then kills the direct child.
- `node scripts/typecheck-test-projects.mjs tests/runner-e2e`: passes.

## Risks

- Low risk, test infrastructure only.
- A stop that hits a persistent uncertainty now fails at the deadline
(up to the grace time) instead of at once. It sends no extra signals.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 20:55:26 -06:00
..