mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 06:10:44 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart chaos harness crashes the local Paperclip server with SIGKILL (K1, K13 storms) and grades how tasks survive > - `crashForRestart` sends SIGKILL to every process group the server owns > - On macOS, `killpg` can report EPERM for a group whose members are still exiting > - When that happens the fault is recorded as failed, and the wave exits 2, even though the server process itself is dead > - This pull request tolerates EPERM in the crash path once the server pid is confirmed dead, and logs it as a warning > - The benefit is that a wave grades the product, not a cleanup race in the harness ## Linked Issues or Issue Description **What happened?** A local Codex K13 storm at concurrency 6 (wave w-20261109T084023-s9403, 112 tasks) exited 2 because faults f04 and f09 failed with `kill EPERM`. The server pids of both faults had died, and the tasks recovered normally. **Steps to reproduce** 1. Run a long K13 storm on macOS with high concurrency. 2. Occasionally the SIGKILL of the server's process groups races with members that are exiting. **Expected behavior** If the server process is dead, the crash completes and the remaining cleanup continues; the EPERM is recorded as a warning. **Actual behavior** The fault fails with `kill EPERM` and the wave reports an error. ## What Changed - `crashForRestart` catches EPERM from the group SIGKILL. It then sends SIGKILL to the server pid directly if needed and waits for it to exit within the forced timeout. - If the server exited, it re-observes the tree and sends SIGKILL to each remaining group on its own, so one group's EPERM does not leave the others to drain gracefully. It logs a warning naming the server pid and any group that still refused SIGKILL, then continues with the normal post-kill cleanup. - If the server is still alive, or the error is not EPERM, the error is rethrown as before. - Four tests: EPERM with a dead server finishes with a warning; a detached descendant is still killed without a graceful drain when the first group kill reports EPERM; EPERM with a surviving server fails; a non-EPERM error fails. The failure tests stop their owners in `finally`. ## Verification - `npx vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/server-stop.test.ts`: the dead-server test fails before the change (`kill EPERM`), the descendant test fails against the first revision (the descendant drains on SIGTERM), and all 22 pass after it. - `node scripts/typecheck-test-projects.mjs tests/runner-e2e`: clean. ## Risks - A group that is still running and returns EPERM is no longer reported in the crash step itself. The cleanup that follows still checks every observed group, and it fails if any group survives. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>