mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 22:56:58 +01:00
DottaandClaude Opus 5.5
610b54f551
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart chaos harness grades how real tasks survive control-plane restarts, and its grades decide which product fixes come next > - Reviewers left 46 unresolved threads on ten merged harness PRs (#20071, #20094, #20106, #20111, #20119, #20147, #20158, #20161, #20178, #20194) > - Several are real harness defects: false speed breaches from one-sample rows, a codex-only wave that cannot start, event paging that can loop forever, an S1 task failed for a Stop the harness never sent, and a readiness wait that aborts on a transient lifecycle error > - Wave S1 t05 (w-20261109T091247-s9806) hit the S1 defect: the run finished before the stop window, so neither the Stop nor the continue comment was sent, and the task was graded `done_too_late Infinity` > - This pull request fixes the valid findings and adds the missing tests > - The benefit is that a failed task points at the product, not at the harness ## Linked Issues or Issue Description Refs #20071, #20094, #20111, #20147, #20158, #20161, #20178, #20194 (review threads). The security finding from #20111 was fixed separately in #20228. **What happened?** Harness defects found in review and in waves: - Aggregate p50/p95 rows breached at n = 1–3, which caused most marginal speed failures. - `--adapters codex` with the default workloads failed at parse because M1 needs two adapters. - `runEvents` paged forever if a full page had no `seq`. - A timed-out task whose last sample was `done` was graded `done_too_late Infinity`, and could pass when a fault had no ready time. - S1 waited forever for a running run to stop; it also clicked Stop while another task's fault had the server down or recovering. - The readiness wait aborted on any `lifecycleError`, including the transient one lifecycle `defer` sets before a 60 s retry, and hid read errors as "not ready". Daytona used the 240 s local timeout. - One failed parent read could hide an M1 stall for the whole 10 s check window. ## What Changed - Speed: aggregate rows breach only from 5 samples (`insufficient sample (n < 5)` otherwise). Restart recoveries over the control-plane p95 budget now count in the per-run check, so a single slow recovery still fails. - Config: the default workloads skip M1 when fewer than two adapters are given; an explicit `--workloads M1` with one adapter still fails. - `runEvents` throws when a full page has no later `seq`; the run's speed read fails instead of hanging. - `gradeR1` fails a timed-out task as `no_rest_within_budget` before checking its status. - S1: the Stop waits for a ready server with no fault in progress. If the run already finished when the window opens, the harness records a missed Stop, skips the stop-triggered fault, still posts the continue comment, and the oracle checks only the continue. - Readiness: only a terminal lifecycle state (rejected, terminating, cleaning up, terminated) aborts at once. Any other lifecycle error is logged when it changes (`seed_agent_lifecycle_error`) and the wait continues until ready or the timeout, which names each agent's last lifecycle or read error. The agent API exposes neither `lifecycleOperation.retryRequired` nor a retry-claim timestamp, so a deferred retry cannot be told apart from a parked failure earlier. Daytona waits up to 10 min. - M1: a failed parent read retries on the next poll. - Child speed collection moved into `childSpeedTasksInTurn` so its one-at-a-time order is tested. - Tests for each change, plus the gaps reviewers named: the stop trigger's arming, stall-wake parsing, and `seedChaosLab`'s readiness wiring. - README: S1 missed Stop, R1 timeout rule, speed sample minimum. ## Verification - `npx vitest run --config tests/restart-chaos/vitest.config.ts`: 230 passed. Each new test failed before its fix, such as the paging test running out of memory in the loop and the stop-trigger test firing a fault after a missed Stop. - `node scripts/typecheck-test-projects.mjs tests/restart-chaos`: clean. - Wave evidence for the defects: S1 t05 in w-20261109T091247-s9806 (missed Stop); codex warm agent_boot p50 breach at n = 3 in w-20261109T104527-s9901. ## Risks - Aggregate rows with fewer than 5 samples no longer fail a wave. Per-run p95 checks still catch individual slow runs and recoveries, but a consistently slightly slow p50 in a small wave is now only reported. - A lifecycle error that persists without `retryRequired` now waits for the readiness timeout (4 min local, 10 min Daytona) instead of failing at once. The timeout message names the error. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>