mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 07:10:34 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart chaos harness and the runner e2e suites stop and restart the control plane through one process-tree owner (`tests/runner-e2e/process-tree-owner.ts`) > - The owner signals only process groups whose identity it has proven, and refuses everything when a group's identity becomes uncertain > - Under load a group can turn over between two `ps` snapshots: the observed members exit, and a straggler that nobody saw start is left in the group, reparented away from every owned anchor > - The stopper treated that as fatal on the first inspection, so a K2 graceful stop at load 28 failed in under a second, and a chaos storm wave was marked as a harness error although the server came back > - This pull request makes the stopper re-inspect within its existing deadline until the uncertainty settles. It still never signals the uncertain group, and it fails with the original error if the uncertainty persists > - The benefit is that chaos waves under load stop failing on a harness race, without weakening the rule against signaling unproven processes ## Linked Issues or Issue Description Refs #19470 **What is the problem?** `stopOwnedProcessTree` fails at once with "Owned process group identity became uncertain" when, under load, a group's observed members exit and an unobserved straggler remains for a moment. In a chaos storm this failed a K2 stop 0.9 s after SIGTERM (load 28), and the wave exited as a harness error. **What is the expected behavior?** The stopper waits (within the same grace or forced deadline) for a later inspection that settles the group, and it never signals the uncertain group. If the straggler outlives the deadline, the stop still fails with the same error. **Why does it matter?** A harness-only race must not invalidate a chaos wave, and the fix must not let the harness signal a process whose ownership is not proven. ## What Changed - `tests/runner-e2e/server-stop.ts`: the chaos and e2e server stopper (the actual K2 path) uses the same bounded settle-wait for its graceful observe/signal and its SIGKILL escalation. Live evidence: a K2 at load 13.9 failed in 0.5 s on an ACPX Claude provider group (launcher, reaper, module-hook loader, `claude` runtime) that tore itself down during the drain and was gone 0.4 s later. - `tests/runner-e2e/process-tree-owner.ts`: the uncertainty error now names the unowned members (pid, ppid, pgid, kind, start). `settleUncertainIdentity` is exported and shared. Export the `UNCERTAIN_GROUP_IDENTITY` message prefix. `stopOwnedProcessTree` retries the grace signal, the SIGKILL escalation and the stop check every 100 ms while that error persists, bounded by the existing grace deadline and one forced window (set before escalation and reused by the escalation retry, the stop check and the fallback), so a stop never exceeds gracefulMs + forcedMs. Other errors, and an uncertainty that outlives the deadline, take the existing fallback path unchanged. The owner's identity checks do not change, and neither does `signal()`. - `tests/runner-e2e/process-tree-owner.test.ts`: three tests. An uncertainty that starts only at SIGKILL escalation fails within one forced window and never sends SIGKILL. A brief turnover settles and the stop succeeds without signaling the uncertain group or killing the direct child. A persistent turnover still rejects with "identity became uncertain" and never signals. ## Verification - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/process-tree-owner.test.ts tests/runner-e2e/server-stop.test.ts`: 35 pass across `process-tree-owner.test.ts` and `server-stop.test.ts`, including 5 new tests; the new brief-turnover tests fail on the old stoppers. With the old stopper, the first new test fails: it rejects on the first inspection and then kills the direct child. - `node scripts/typecheck-test-projects.mjs tests/runner-e2e`: passes. ## Risks - Low risk, test infrastructure only. - A stop that hits a persistent uncertainty now fails at the deadline (up to the grace time) instead of at once. It sends no extra signals. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>