Commit Graph
1 Commits
Author SHA1 Message Date
DottaandClaude Opus 5.5 bf67e2fbd9 test(chaos): honor 429 retry delays on harness reads (#20194)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The restart chaos harness collects each wave's speed from the board
API after the tasks rest, including the M1 subtasks' runs
> - The server rate-limits concurrent issue-list requests per client and
answers 429 with `retryAfterSeconds`
> - After a 95-task storm, the harness requested every M1 parent's
subtask list at once. The server returned 429, speed collection failed,
and the wave exited 1 with no speed grade
> - This pull request makes harness GET requests honor a 429's retry
delay with a bounded number of attempts, and collects subtask speed one
parent at a time
> - The benefit is that a busy wave keeps its speed grade, and the
harness behaves like a polite API client

## Linked Issues or Issue Description

Refs #20071, #20147

**What happened?**
Wave `w-20261109T055902-s9402` (K13 storm, A1/C1/M1, 95 tasks) finished
its tasks. Speed collection then failed with: `GET
/api/companies/…/issues?view=index&parentIds=…&limit=20 returned 429:
{"error":"Too many concurrent issue-list requests for this
actor/client","retryAfterSeconds":1}`. `main.ts` ran `childSpeedTasks()`
for every observer in parallel.

**Expected behavior**
- The harness waits and retries a rate-limited read.
- It does not flood the issue-list endpoint while collecting speed.
- A busy wave still gets graded.

## What Changed

- `api.ts`: `ChaosApi` retries a GET that returns 429.
- It waits for the delay from `Retry-After` or the body's
`retryAfterSeconds` (default 1 s, capped at 10 s), scaled by the attempt
number.
  - It makes at most 5 attempts by default.
- Writes (POST/PUT/PATCH/DELETE) are never retried, so a user action is
never repeated.
- `rateLimitDelayMs` is exported for tests.
- `main.ts`: subtask speed is collected one observer at a time instead
of with `Promise.all`.
- New `api.test.ts` against a real local HTTP server:
  - a GET that gets 429 twice, then 200, succeeds;
  - a POST that gets 429 is sent once;
  - retries stop at the bound;
  - delay parsing (header, body, cap, default).

## Verification

- `npx vitest run` in `tests/restart-chaos`: 216 passed.
- `node scripts/typecheck-test-projects.mjs tests/restart-chaos`: clean.

## Risks

- A persistently rate-limited read now takes up to about 15 s before it
fails, instead of failing at once.
- Collecting subtask speed one parent at a time adds a few seconds to
the end of large M1 waves.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 01:06:41 -06:00