Files
PaperClipAI/tests
DottaandClaude Opus 5.5 0d80b74c21 test(chaos): treat a crash-restart EPERM as a warning once the server pid is dead (#20232)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The restart chaos harness crashes the local Paperclip server with
SIGKILL (K1, K13 storms) and grades how tasks survive
> - `crashForRestart` sends SIGKILL to every process group the server
owns
> - On macOS, `killpg` can report EPERM for a group whose members are
still exiting
> - When that happens the fault is recorded as failed, and the wave
exits 2, even though the server process itself is dead
> - This pull request tolerates EPERM in the crash path once the server
pid is confirmed dead, and logs it as a warning
> - The benefit is that a wave grades the product, not a cleanup race in
the harness

## Linked Issues or Issue Description

**What happened?**
A local Codex K13 storm at concurrency 6 (wave w-20261109T084023-s9403,
112 tasks) exited 2 because faults f04 and f09 failed with `kill EPERM`.
The server pids of both faults had died, and the tasks recovered
normally.

**Steps to reproduce**
1. Run a long K13 storm on macOS with high concurrency.
2. Occasionally the SIGKILL of the server's process groups races with
members that are exiting.

**Expected behavior**
If the server process is dead, the crash completes and the remaining
cleanup continues; the EPERM is recorded as a warning.

**Actual behavior**
The fault fails with `kill EPERM` and the wave reports an error.

## What Changed

- `crashForRestart` catches EPERM from the group SIGKILL. It then sends
SIGKILL to the server pid directly if needed and waits for it to exit
within the forced timeout.
- If the server exited, it re-observes the tree and sends SIGKILL to
each remaining group on its own, so one group's EPERM does not leave the
others to drain gracefully. It logs a warning naming the server pid and
any group that still refused SIGKILL, then continues with the normal
post-kill cleanup.
- If the server is still alive, or the error is not EPERM, the error is
rethrown as before.
- Four tests: EPERM with a dead server finishes with a warning; a
detached descendant is still killed without a graceful drain when the
first group kill reports EPERM; EPERM with a surviving server fails; a
non-EPERM error fails. The failure tests stop their owners in `finally`.

## Verification

- `npx vitest run --config tests/runner-e2e/vitest.config.ts
tests/runner-e2e/server-stop.test.ts`: the dead-server test fails before
the change (`kill EPERM`), the descendant test fails against the first
revision (the descendant drains on SIGTERM), and all 22 pass after it.
- `node scripts/typecheck-test-projects.mjs tests/runner-e2e`: clean.

## Risks

- A group that is still running and returns EPERM is no longer reported
in the crash step itself. The cleanup that follows still checks every
observed group, and it fails if any group survives.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 05:39:40 -06:00
..