Files
PaperClipAI/ui/src/api/client.test.ts
T
DottaandPaperclip 176645187c Fix request storm polling and issue-list coalescing (#9190)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The board UI keeps issue, agent, activity, and run state fresh
through polling across several pages and sidebar surfaces
> - When multiple components or browser tabs poll the same company data
at the same time, the API can receive bursts of duplicate issue-list
requests
> - Those duplicate requests increase database and server load without
returning meaningfully different data
> - This pull request adds server-side compression/coalescing plus
client-side visibility-aware and cross-tab shared polling
> - The benefit is lower request volume during normal board usage while
preserving fresh UI data for active users

## Linked Issues or Issue Description

No exact public GitHub issue was found for this request.

Problem:
- The board can issue redundant polling requests for the same issue-list
data from multiple UI surfaces and tabs.
- In busy operator sessions, those bursts can trigger request-storm
behavior and unnecessary issue-list load.
- Expected behavior is to reuse identical in-flight work server-side and
reduce hidden-tab or duplicate-tab polling client-side while preserving
normal refresh behavior.

Related public context found during duplicate search:
- #8206 covers a different board UI 404-storm scope.
- #5165 covers separate issue-list behavior around page-size truncation.

## What Changed

- Added API compression middleware foundation and a company/created-at
index for heartbeat run access.
- Added server-side issue-list request storm detection and identical
in-flight request coalescing.
- Added UI fetch metadata, visibility-aware polling, and request
deduplication for issue/activity/client calls.
- Added cross-tab shared polling primitives and wired them into the
sidebar, inbox, dashboard, issue, project, routine, and agent surfaces.
- Resolved the latest `master` migration collision by keeping upstream
`0140_built_in_managed_resources.sql` and renumbering this branch's
heartbeat-run index migration to
`0141_heartbeat_runs_company_created_at_index.sql`; the SQL uses `CREATE
INDEX IF NOT EXISTS` for idempotency.
- Stabilized server heartbeat cleanup tests exposed by the PR check
matrix.
- Fixed the Greptile compression follow-up by weakening strong ETags on
encoded JSON responses and bypassing compression for streamed/download
responses.

## Verification

- `pnpm exec vitest run ui/src/components/IssuesList.test.tsx
ui/src/pages/Inbox.test.tsx` — passed after resolving the latest
`master` conflict in `IssuesList.tsx` and updating the 200-result cap
expectations.
- `pnpm check:token-gates` — passed after the UI conflict resolution.
- `jq -e '.entries | length as $n | (map(.idx) | unique | length == $n)
and (map(.tag) | unique | length == $n)'
packages/db/src/migrations/meta/_journal.json` — passed after
renumbering the migration to `0141`.
- `pnpm exec vitest run server/src/__tests__/api-compression.test.ts` -
passed after the compression follow-up.
- `pnpm exec vitest run
server/src/__tests__/issue-list-assignee-filter-routes.test.ts` - passed
after the compression follow-up.
- `pnpm --filter @paperclipai/server typecheck` - passed after the
compression follow-up.
- Greptile Review for head `8dbddac41ec273fda404100b4981ddb912fad57b` -
passed after the latest conflict/migration fix; all Greptile review
threads are resolved.
- `pnpm exec vitest run server/src/__tests__/api-compression.test.ts
server/src/__tests__/issue-list-assignee-filter-routes.test.ts
ui/src/api/client.test.ts ui/src/api/issues.test.ts
ui/src/lib/polling.test.ts ui/src/lib/cross-tab-poll.test.ts
ui/src/pages/Inbox.test.tsx ui/src/components/SidebarProjects.test.tsx
ui/src/components/SidebarAccountMenu.test.tsx
ui/src/lib/issueDetailCache.test.ts` — passed.
- `pnpm exec vitest run ui/src/api/client.test.ts` — passed.
- `pnpm exec vitest run ui/src/pages/Inbox.test.tsx
ui/src/components/SidebarProjects.test.tsx
ui/src/components/SidebarAccountMenu.test.tsx
ui/src/lib/issueDetailCache.test.ts` — passed.
- `pnpm exec vitest run
server/src/__tests__/heartbeat-worktree-suppression.test.ts` — passed.
- `pnpm exec vitest run
server/src/__tests__/low-trust-red-team-routes.test.ts` — passed.
- `pnpm exec vitest run
server/src/__tests__/heartbeat-workspace-finalize-branch.test.ts` —
passed.
- `pnpm --filter @paperclipai/ui typecheck` — passed.
- `pnpm --filter @paperclipai/server typecheck` — passed.
- GitHub PR checks for head `8dbddac41ec273fda404100b4981ddb912fad57b`:
all GitHub Actions/status checks passed; Greptile, Superagent, Socket,
Snyk, build, typecheck/release registry, general tests, serialized
server suites, e2e, canary dry run, policy, and commitperclip review are
green; Storybook visual regression and security-review were
skipped/neutral by policy.
- Confirmed this branch does not include `pnpm-lock.yaml` or
`.github/workflows` changes.

## Risks

- Medium risk: issue-list coalescing changes request timing and cache
semantics for a hot API path.
- Medium risk: cross-tab polling uses browser coordination primitives,
so older or unusual browser environments need fallback behavior to stay
correct.
- Low migration risk: the new index migration is ordered after current
`master` and uses `CREATE INDEX IF NOT EXISTS`.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex coding agent, GPT-5-based model, tool-enabled with
shell/git execution. Exact hosted deployment identifier and
context-window size were not surfaced in the agent runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-09 17:38:34 -05:00

133 lines
4.4 KiB
TypeScript

import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { __inflightGetCount, api } from "./client";
interface Deferred<T> {
promise: Promise<T>;
resolve: (value: T) => void;
reject: (reason?: unknown) => void;
}
function deferred<T>(): Deferred<T> {
let resolve!: (value: T) => void;
let reject!: (reason?: unknown) => void;
const promise = new Promise<T>((res, rej) => {
resolve = res;
reject = rej;
});
return { promise, resolve, reject };
}
function jsonResponse(body: unknown) {
return { ok: true, status: 200, json: async () => body } as unknown as Response;
}
const fetchMock = vi.fn();
beforeEach(() => {
fetchMock.mockReset();
vi.stubGlobal("fetch", fetchMock);
});
afterEach(() => {
vi.unstubAllGlobals();
});
describe("in-tab GET coalescing", () => {
it("shares one underlying fetch for identical in-flight GETs", async () => {
const d = deferred<Response>();
fetchMock.mockReturnValue(d.promise);
const p1 = api.get("/coalesce-a");
const p2 = api.get("/coalesce-a");
expect(fetchMock).toHaveBeenCalledTimes(1);
expect(__inflightGetCount()).toBe(1);
d.resolve(jsonResponse({ value: 1 }));
expect(await p1).toEqual({ value: 1 });
expect(await p2).toEqual({ value: 1 });
// Entry cleared once settled.
expect(__inflightGetCount()).toBe(0);
});
it("issues a fresh fetch after the previous one settles", async () => {
fetchMock.mockResolvedValue(jsonResponse({ value: "x" }));
await api.get("/coalesce-b");
await api.get("/coalesce-b");
expect(fetchMock).toHaveBeenCalledTimes(2);
});
it("does not coalesce different paths", async () => {
fetchMock.mockReturnValue(deferred<Response>().promise);
const c1 = new AbortController();
const c2 = new AbortController();
const p1 = api.get("/path-1", { signal: c1.signal });
const p2 = api.get("/path-2", { signal: c2.signal });
expect(fetchMock).toHaveBeenCalledTimes(2);
// Clean up the never-settling shared entries so the module map does not leak.
c1.abort();
c2.abort();
await expect(p1).rejects.toMatchObject({ name: "AbortError" });
await expect(p2).rejects.toMatchObject({ name: "AbortError" });
});
it("never coalesces mutations", async () => {
fetchMock.mockResolvedValue(jsonResponse({ ok: true }));
await Promise.all([api.post("/mutate", { a: 1 }), api.post("/mutate", { a: 1 })]);
expect(fetchMock).toHaveBeenCalledTimes(2);
});
});
describe("per-caller abort semantics", () => {
it("aborting one caller rejects only that caller, not the shared fetch", async () => {
const d = deferred<Response>();
let sharedSignal: AbortSignal | undefined;
fetchMock.mockImplementation((_url: string, init: RequestInit) => {
sharedSignal = init.signal ?? undefined;
return d.promise;
});
const c1 = new AbortController();
const c2 = new AbortController();
const p1 = api.get("/abort-a", { signal: c1.signal });
const p2 = api.get("/abort-a", { signal: c2.signal });
c1.abort();
await expect(p1).rejects.toMatchObject({ name: "AbortError" });
// The shared fetch is still alive because caller 2 has not aborted.
expect(sharedSignal?.aborted).toBe(false);
d.resolve(jsonResponse({ value: 2 }));
expect(await p2).toEqual({ value: 2 });
});
it("aborts the shared fetch once every caller has aborted", async () => {
const d = deferred<Response>();
let sharedSignal: AbortSignal | undefined;
fetchMock.mockImplementation((_url: string, init: RequestInit) => {
sharedSignal = init.signal ?? undefined;
return d.promise;
});
const c1 = new AbortController();
const c2 = new AbortController();
const p1 = api.get("/abort-b", { signal: c1.signal });
const p2 = api.get("/abort-b", { signal: c2.signal });
c1.abort();
c2.abort();
await expect(p1).rejects.toMatchObject({ name: "AbortError" });
await expect(p2).rejects.toMatchObject({ name: "AbortError" });
expect(sharedSignal?.aborted).toBe(true);
expect(__inflightGetCount()).toBe(0);
});
it("rejects immediately if the caller signal is already aborted", async () => {
const controller = new AbortController();
controller.abort();
await expect(api.get("/already-aborted", { signal: controller.signal })).rejects.toMatchObject({
name: "AbortError",
});
expect(fetchMock).not.toHaveBeenCalled();
});
});