mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-10 00:56:52 +01:00
DottaandClaude Opus 5.5
da850990f8
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart chaos harness grades every task on rules R1 to R6. R4 means "no duplicate execution" > - R4 counted every queued, running or scheduled-retry run in one poll as an active run, and failed the task when there were two > - A follow-up comment while a run is in progress normally queues a wake behind that run. The product cancels the wake when the running run finishes, so the wake never executes > - A Daytona wave failed an opencode task for exactly this case, so R4 reported a duplicate-execution failure that did not happen > - This pull request grades R4 on actual overlapping execution > - The benefit is that R4 failures again mean that two runs really executed together, or a comment was consumed twice ## Linked Issues or Issue Description **Bug description** `gradeR4` failed a task whenever one poll showed two runs in `ACTIVE_RUN_STATUSES` (queued, running, scheduled_retry). This includes a wake queued behind a running run that never starts. **Steps to reproduce** Daytona wave `w-20261107T013205-s11`, task t06 (opencode C1 K2): - Run b4b16a4d is running. - The C1 follow-up comment queues run c090799a. - c090799a is cancelled 1 s after b4b16a4d succeeds and never starts. - Before this change, R4 reported `concurrent_runs: c090799a(queued), b4b16a4d(running)`. **Expected behavior** R4 passes. It fails only when two runs execute at the same time, or when a side effect (a consumed comment) repeats. Refs #19544 ## What Changed - `tests/restart-chaos/grader.ts`: `gradeR4` fails when two runs are `running` in the same sample, or when the started→finished windows of two started runs overlap. A still-active run's window lasts until the last sample. Queued and scheduled-retry runs that never started do not count. The comment-consumed-twice check is unchanged. - `tests/restart-chaos/grader.test.ts`: a test with the t06 shape (a queued wake cancelled behind a running run) that passes; queued plus scheduled_retry without a start passes; overlapping windows fail even when no single poll saw both running. The existing two-running and comment tests are kept. - `tests/restart-chaos/README.md`: the R4 grading note. - R6 settle for Daytona (`tests/restart-chaos/daytona-sweep.ts`, `main.ts`). The product retries a `pending_cleanup` lease after `ENVIRONMENT_CLEANUP_RETRY_MS` (5 min), so a 2-min settle reported normal cleanup as a leak. R6 now waits `PENDING_CLEANUP_RETRY_MS + 90 s`, capped at 8 min; a test keeps the constant equal to the server's. An environment DELETE that returns 409 while a sandbox cleanup is pending is correct product behaviour. It is retried for the same window. If it is still blocked after that, it is reported as `environmentDeleteBlocked`, and it fails the wave and keeps the instance (whose database owns the cleanup work). It is not counted as a harness error. Tests cover the 409 retry and a real (500) delete failure. ## Verification - `pnpm exec vitest run --config tests/restart-chaos/vitest.config.ts`: 109/109 pass. `node scripts/typecheck-test-projects.mjs tests/restart-chaos`: clean. - R6 evidence: in wave `w-20261107T015455-s12`, three leases were still `pending_cleanup` at the 2-min mark, and the environment DELETE returned 409 "Cannot delete this environment while a sandbox cleanup is pending". - The t06 sample sequence from `w-20261107T013205-s11` is the new test case. ## Risks - This change touches the test harness only. - R4 is now less strict about queued rows. If a queued duplicate starts executing, its start time makes its window overlap, so R4 still fails. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use (shell, file edits, GitHub CLI). Default context window, extended thinking. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>