Files
PaperClipAI/tests
DottaandClaude Fable 5.2 bc71c23e0a fix(test): remove the Pi fault calibration file race and halve its runtime (#21286)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Runner support suite checks the Pi provider fault helper. The
helper kills the exact Pi child process on Linux with a pidfd.
> - `tests/runner-e2e/pi-provider-fault.test.ts` failed in 3 unrelated
PRs on 2026-11-12. One failure passed on a rerun of the same commit.
> - The failures had two causes. A fixture file race gave a JSON parse
error. A CPU-heavy calibration ran past the 25 s outer timeout on the
slower shared fleet.
> - This pull request removes the file race and removes about half of
the calibration work. It also states the outer timeout margin.
> - The benefit is a deterministic test that runs in about 3.4 s instead
of about 5.9 s on Linux.

## Linked Issues or Issue Description

**What happened?**
The test "calibrates the exact Pi fault helper with metadata negatives
and native Linux pidfd" failed intermittently in the "Runner support
suite" lane. There were two errors:
- Job 131195539083: `test_real_descriptor_launch_then_exact_child_pidfd
(descriptor=7)` failed at `json.loads(ready.read_text())` with
`JSONDecodeError: Expecting value: line 1 column 1 (char 0)`.
- Jobs 131420813114 and 131420614853: `spawnSync /usr/bin/python3
ETIMEDOUT` after 25 s. Both jobs ran on the `paperclip-public-pr-x64`
fleet (m5n.xlarge, 4 cores, load average about 4). In those jobs, every
file in the lane ran about 1.8x slower than on the hosted `ubuntu-24.04`
image.

**Expected behavior**
The test passes on every run of the same commit.

**Steps to reproduce**
1. Error A: add a delay between file creation and write in the fixture
child (for example `openSync`, then `Atomics.wait` for 500 ms, then
`writeSync`). Run `/usr/bin/python3 -B
tests/runner-e2e/pi-provider-fault.test.py` on Linux. All four real
calibrations fail with the same `JSONDecodeError`.
2. Error B: run the suite on a loaded 4-core runner. The Python suite
needs more than 25 s.

**Paperclip version or commit**
`master` at 9b613562b7.

## What Changed

- Root cause A: the fixture child wrote `child.json` and `exit.json`
with `writeFileSync`. That call creates the file and then writes it. The
test polls only `exists()` and then parses. A reader could see the
created but empty file. Now the fixture writes `<name>.tmp` and renames
it into place, so the file appears only with complete content. This also
fixes the same race on `exit.json`.
- Root cause B: a profile of one calibration showed that about 1.0 s of
1.28 s was spent in `inspect()`. Each `inspect()` call reads and SHA-256
hashes the runner executable and the distribution Node executable. Both
were copies of the 123 MB Node binary. Each calibration calls
`inspect()` five times, and the suite runs four calibrations, so the
suite hashed about 5 GB. Polling sleeps were only 0.09 s. The synthetic
runner root now is a copy of `/bin/sh` that runs a short script. It is
still a distinct, pinned ELF, and `/proc/<pid>/exe` still resolves to
it. The script keeps the same launch shape (direct path, or
`/proc/self/fd/3` or `/proc/self/fd/7` with the executable descriptor
opened only for the wrapper child). It keeps the same SIGTERM cascade.
The distribution Node, the wrapper and the Pi child are unchanged.
- The outer `spawnSync` timeout is now a named constant (45 s) with its
margin in a comment. The Vitest test timeout derives from it (+5 s), so
a slow Python run reports its own output instead of a Vitest timeout.
- No assertion is removed, skipped or weakened. The fault helper
(`pi-provider-fault.py`) is unchanged.

## Verification

All Linux runs use Docker `node:24-bookworm` (Python 3.11, kernel 6.12)
on an Apple Silicon host.

- Red (master + 500 ms create-to-write delay): `Ran 21 tests`, `FAILED
(errors=4)`. Each of the four real calibrations failed with
`json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char
0)`.
- Green (this branch + the same 500 ms delay between creating and
writing the temporary file): `Ran 21 tests in 7.788s`, `OK`.
- Timing, `python3 -B pi-provider-fault.test.py`, 3 runs each:
- Unlimited CPU: master 6.77 / 5.88 / 5.75 s; this branch 3.45 / 3.34 /
3.43 s.
- `--cpus=1`: master 6.16 / 6.08 / 5.53 s; this branch 3.37 / 3.34 /
3.04 s.
- macOS: `pnpm exec vitest run --config
tests/runner-e2e/vitest.config.ts pi-provider-fault` passes. macOS skips
the 3 Linux-only calibrations, as before.
- `pnpm verify` exits 2 because a `.py` file is a global CI input.
Required CI supplies the broad coverage.
- Not run: the test on an x64 GitHub-hosted or fleet runner before this
PR's CI. I could not reproduce the fleet's load locally.

## Risks

- Low risk. The change is test-only.
- The runner root fixture now is a shell instead of Node. The fault
helper checks only the root's executable identity, hash, arguments,
process group and working directory. A copied shell satisfies these
checks in the same way.
- If `/bin/sh` is bash instead of dash, the script still works. It uses
only POSIX `trap`, `kill`, `wait` and `$!`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, Docker, file edits).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes (not
applicable: test-only change)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 23:19:29 -06:00
..