mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 06:10:44 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart chaos harness collects each wave's speed from the board API after the tasks rest, including the M1 subtasks' runs > - The server rate-limits concurrent issue-list requests per client and answers 429 with `retryAfterSeconds` > - After a 95-task storm, the harness requested every M1 parent's subtask list at once. The server returned 429, speed collection failed, and the wave exited 1 with no speed grade > - This pull request makes harness GET requests honor a 429's retry delay with a bounded number of attempts, and collects subtask speed one parent at a time > - The benefit is that a busy wave keeps its speed grade, and the harness behaves like a polite API client ## Linked Issues or Issue Description Refs #20071, #20147 **What happened?** Wave `w-20261109T055902-s9402` (K13 storm, A1/C1/M1, 95 tasks) finished its tasks. Speed collection then failed with: `GET /api/companies/…/issues?view=index&parentIds=…&limit=20 returned 429: {"error":"Too many concurrent issue-list requests for this actor/client","retryAfterSeconds":1}`. `main.ts` ran `childSpeedTasks()` for every observer in parallel. **Expected behavior** - The harness waits and retries a rate-limited read. - It does not flood the issue-list endpoint while collecting speed. - A busy wave still gets graded. ## What Changed - `api.ts`: `ChaosApi` retries a GET that returns 429. - It waits for the delay from `Retry-After` or the body's `retryAfterSeconds` (default 1 s, capped at 10 s), scaled by the attempt number. - It makes at most 5 attempts by default. - Writes (POST/PUT/PATCH/DELETE) are never retried, so a user action is never repeated. - `rateLimitDelayMs` is exported for tests. - `main.ts`: subtask speed is collected one observer at a time instead of with `Promise.all`. - New `api.test.ts` against a real local HTTP server: - a GET that gets 429 twice, then 200, succeeds; - a POST that gets 429 is sent once; - retries stop at the bound; - delay parsing (header, body, cap, default). ## Verification - `npx vitest run` in `tests/restart-chaos`: 216 passed. - `node scripts/typecheck-test-projects.mjs tests/restart-chaos`: clean. ## Risks - A persistently rate-limited read now takes up to about 15 s before it fails, instead of failing at once. - Collecting subtask speed one parent at a time adds a few seconds to the end of large M1 waves. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
65 lines
3.4 KiB
TypeScript
65 lines
3.4 KiB
TypeScript
/** Minimal board API client for an isolated local_trusted instance. */
|
|
export function isSocketTypeOfServiceRace(error: unknown): boolean {
|
|
const candidate = error as { code?: unknown; syscall?: unknown } | null;
|
|
return candidate?.code === "EINVAL" && candidate.syscall === "setTypeOfService";
|
|
}
|
|
|
|
export class ChaosApiError extends Error {
|
|
constructor(readonly status: number | null, message: string) { super(message); this.name = "ChaosApiError"; }
|
|
}
|
|
|
|
export function rateLimitDelayMs(response: { status: number; headers: { get(name: string): string | null } }, text: string): number | null {
|
|
if (response.status !== 429) return null;
|
|
let seconds = Number(response.headers.get("retry-after"));
|
|
if (!Number.isFinite(seconds) || seconds <= 0) {
|
|
try { seconds = Number((JSON.parse(text) as { retryAfterSeconds?: unknown }).retryAfterSeconds); } catch { seconds = NaN; }
|
|
}
|
|
return Number.isFinite(seconds) && seconds > 0 ? Math.min(10_000, seconds * 1_000) : 1_000;
|
|
}
|
|
|
|
export class ChaosApi {
|
|
constructor(readonly baseUrl: string, private readonly timeoutMs = 10_000, private readonly rateLimitAttempts = 5) {}
|
|
|
|
private async request<T>(method: string, route: string, body?: unknown, timeoutMs = this.timeoutMs): Promise<T> {
|
|
for (let attempt = 1; ; attempt += 1) {
|
|
const outcome = await this.attempt<T>(method, route, body, timeoutMs);
|
|
if (!("retryAfterMs" in outcome)) return outcome.value;
|
|
if (method !== "GET" || attempt >= this.rateLimitAttempts) throw outcome.error;
|
|
await new Promise(resolve => setTimeout(resolve, Math.min(10_000, outcome.retryAfterMs * attempt)));
|
|
}
|
|
}
|
|
|
|
private async attempt<T>(method: string, route: string, body: unknown, timeoutMs: number):
|
|
Promise<{ value: T } | { retryAfterMs: number; error: ChaosApiError }> {
|
|
let response: Response;
|
|
try {
|
|
response = await fetch(`${this.baseUrl}${route}`, {
|
|
method,
|
|
headers: { origin: this.baseUrl, ...(body === undefined ? {} : { "content-type": "application/json" }) },
|
|
body: body === undefined ? undefined : JSON.stringify(body),
|
|
signal: AbortSignal.timeout(timeoutMs),
|
|
cache: "no-store",
|
|
});
|
|
} catch (error) {
|
|
throw new ChaosApiError(null, `${method} ${route} unreachable: ${error instanceof Error ? error.message : String(error)}`);
|
|
}
|
|
const text = await response.text();
|
|
if (!response.ok) {
|
|
const error = new ChaosApiError(response.status, `${method} ${route} returned ${response.status}: ${text.slice(0, 500)}`);
|
|
const retryAfterMs = rateLimitDelayMs(response, text);
|
|
if (retryAfterMs !== null) return { retryAfterMs, error };
|
|
throw error;
|
|
}
|
|
return { value: (text ? JSON.parse(text) : null) as T };
|
|
}
|
|
|
|
get<T>(route: string, timeoutMs?: number) { return this.request<T>("GET", route, undefined, timeoutMs); }
|
|
post<T>(route: string, body?: unknown, timeoutMs?: number) { return this.request<T>("POST", route, body ?? {}, timeoutMs); }
|
|
put<T>(route: string, body?: unknown) { return this.request<T>("PUT", route, body ?? {}); }
|
|
patch<T>(route: string, body?: unknown) { return this.request<T>("PATCH", route, body ?? {}); }
|
|
async delete(route: string, options?: { allowNotFound?: boolean; timeout?: number }): Promise<void> {
|
|
try { await this.request<unknown>("DELETE", route, undefined, options?.timeout); }
|
|
catch (error) { if (!(options?.allowNotFound && error instanceof ChaosApiError && error.status === 404)) throw error; }
|
|
}
|
|
}
|