mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 07:10:34 +01:00
DottaandClaude Opus 5.5
644ee66f5d
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart chaos harness (`tests/restart-chaos/`) grades every task on R1–R6; R5 catches silent stalls, including a run that stays `running` with no live evidence > - A B5 task runs `bench.sh` for about four minutes with no output, so a healthy Retry run was graded `R5:silent_dead_run` while its runner process was alive and its tool was still running > - Separately, a runner `dist` rebuilt in the harness worktree during a wave made later restarts run different code and produced a false signature (RC-32's investigation) > - This pull request makes R5 treat a run as live while a tool it started has not ended or while its recorded local runner process still runs, and fails a wave whose worktree changes mid-wave > - The benefit is fewer false R5 failures and no silently mixed-code waves ## Linked Issues or Issue Description Refs #19629 Refs #19641 **What is the problem?** R5 flags a quiet but live run (a long tool call, a live runner process) as `silent_dead_run`. A build, install or branch switch in the harness worktree during a wave goes unnoticed, though every restart re-runs the control plane from that worktree. **What is the expected behavior?** A run with a tool in flight, or with its recorded local runner process still alive (same PID and start time), is live for R5. A wave whose checkout or runner build output changes during the wave fails as a harness error. **Why does it matter?** False R5 failures send fixers after non-bugs, and mixed-code waves produce signatures that are not product behavior. ## What Changed - `observer.ts`: the run-event reader counts tool starts minus tool ends (PRP item ids are not unique per tool call), and remembers the run's latest local process identity until recovery reports it stopped. Samples carry `toolInFlight` and `processAlive`. `localProcessAlive` checks `ps` for the same PID with the same start time, so a recycled PID never counts. Daytona runs are not checked (their process is remote). - `grader.ts`: `deadRunningRun` skips a run with a tool in flight or a live process. - `types.ts`: `RunSample.toolInFlight` and `RunSample.processAlive`. - `wave-guard.ts` (new) and `main.ts`: fingerprint `HEAD`, the tracked diff contents (`git diff --binary HEAD`), untracked file contents, and size/mtime of `packages/paperclip-runner/dist` and the runnerd binary. The baseline is taken after the harness's own setup (preflight rebuilds provider entrypoints). Re-check after each fault and at the end; fail the wave (`worktree_changed_mid_wave`) if it changed. - `main.ts`, `scheduler.ts`: `faults.json` and a new `task-faults.json` (each task's launch time and crossing faults) are written after every fault. `--replay` falls back to that log when a task has no grade file (an interrupted or wedged wave), so a storm task replays the faults that crossed it at the same offsets. - `README.md`: both grading notes. ## Verification - `pnpm exec vitest run --config tests/restart-chaos/vitest.config.ts`: 111 tests pass. New tests cover: - a B5 run quiet for 4 minutes passes with a tool in flight or a live process, and fails otherwise - tool and process liveness come from run events and clear on tool end and process stop - `localProcessAlive` accepts this process's real PID and start time and rejects a different start time - the guard catches rebuilt dist, a re-edited dirty file, an edited untracked file and a rebuilt runner binary, in a real temporary git repo - a storm task with no grade file replays the faults that crossed it at their offsets from its launch - `node scripts/typecheck-test-projects.mjs tests/restart-chaos`: passes. ## Risks - Low risk, test harness only. - The R5 exemption narrows R5. A run that is truly stuck while its process lives (a hung provider) now passes R5 and falls to R1's time budget instead. - One `ps` call per running run per poll on local waves. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>