mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-17 20:09:05 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The board UI keeps issue, agent, activity, and run state fresh through polling across several pages and sidebar surfaces > - When multiple components or browser tabs poll the same company data at the same time, the API can receive bursts of duplicate issue-list requests > - Those duplicate requests increase database and server load without returning meaningfully different data > - This pull request adds server-side compression/coalescing plus client-side visibility-aware and cross-tab shared polling > - The benefit is lower request volume during normal board usage while preserving fresh UI data for active users ## Linked Issues or Issue Description No exact public GitHub issue was found for this request. Problem: - The board can issue redundant polling requests for the same issue-list data from multiple UI surfaces and tabs. - In busy operator sessions, those bursts can trigger request-storm behavior and unnecessary issue-list load. - Expected behavior is to reuse identical in-flight work server-side and reduce hidden-tab or duplicate-tab polling client-side while preserving normal refresh behavior. Related public context found during duplicate search: - #8206 covers a different board UI 404-storm scope. - #5165 covers separate issue-list behavior around page-size truncation. ## What Changed - Added API compression middleware foundation and a company/created-at index for heartbeat run access. - Added server-side issue-list request storm detection and identical in-flight request coalescing. - Added UI fetch metadata, visibility-aware polling, and request deduplication for issue/activity/client calls. - Added cross-tab shared polling primitives and wired them into the sidebar, inbox, dashboard, issue, project, routine, and agent surfaces. - Resolved the latest `master` migration collision by keeping upstream `0140_built_in_managed_resources.sql` and renumbering this branch's heartbeat-run index migration to `0141_heartbeat_runs_company_created_at_index.sql`; the SQL uses `CREATE INDEX IF NOT EXISTS` for idempotency. - Stabilized server heartbeat cleanup tests exposed by the PR check matrix. - Fixed the Greptile compression follow-up by weakening strong ETags on encoded JSON responses and bypassing compression for streamed/download responses. ## Verification - `pnpm exec vitest run ui/src/components/IssuesList.test.tsx ui/src/pages/Inbox.test.tsx` — passed after resolving the latest `master` conflict in `IssuesList.tsx` and updating the 200-result cap expectations. - `pnpm check:token-gates` — passed after the UI conflict resolution. - `jq -e '.entries | length as $n | (map(.idx) | unique | length == $n) and (map(.tag) | unique | length == $n)' packages/db/src/migrations/meta/_journal.json` — passed after renumbering the migration to `0141`. - `pnpm exec vitest run server/src/__tests__/api-compression.test.ts` - passed after the compression follow-up. - `pnpm exec vitest run server/src/__tests__/issue-list-assignee-filter-routes.test.ts` - passed after the compression follow-up. - `pnpm --filter @paperclipai/server typecheck` - passed after the compression follow-up. - Greptile Review for head `8dbddac41ec273fda404100b4981ddb912fad57b` - passed after the latest conflict/migration fix; all Greptile review threads are resolved. - `pnpm exec vitest run server/src/__tests__/api-compression.test.ts server/src/__tests__/issue-list-assignee-filter-routes.test.ts ui/src/api/client.test.ts ui/src/api/issues.test.ts ui/src/lib/polling.test.ts ui/src/lib/cross-tab-poll.test.ts ui/src/pages/Inbox.test.tsx ui/src/components/SidebarProjects.test.tsx ui/src/components/SidebarAccountMenu.test.tsx ui/src/lib/issueDetailCache.test.ts` — passed. - `pnpm exec vitest run ui/src/api/client.test.ts` — passed. - `pnpm exec vitest run ui/src/pages/Inbox.test.tsx ui/src/components/SidebarProjects.test.tsx ui/src/components/SidebarAccountMenu.test.tsx ui/src/lib/issueDetailCache.test.ts` — passed. - `pnpm exec vitest run server/src/__tests__/heartbeat-worktree-suppression.test.ts` — passed. - `pnpm exec vitest run server/src/__tests__/low-trust-red-team-routes.test.ts` — passed. - `pnpm exec vitest run server/src/__tests__/heartbeat-workspace-finalize-branch.test.ts` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - GitHub PR checks for head `8dbddac41ec273fda404100b4981ddb912fad57b`: all GitHub Actions/status checks passed; Greptile, Superagent, Socket, Snyk, build, typecheck/release registry, general tests, serialized server suites, e2e, canary dry run, policy, and commitperclip review are green; Storybook visual regression and security-review were skipped/neutral by policy. - Confirmed this branch does not include `pnpm-lock.yaml` or `.github/workflows` changes. ## Risks - Medium risk: issue-list coalescing changes request timing and cache semantics for a hot API path. - Medium risk: cross-tab polling uses browser coordination primitives, so older or unusual browser environments need fallback behavior to stay correct. - Low migration risk: the new index migration is ordered after current `master` and uses `CREATE INDEX IF NOT EXISTS`. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex coding agent, GPT-5-based model, tool-enabled with shell/git execution. Exact hosted deployment identifier and context-window size were not surfaced in the agent runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
136 lines
5.3 KiB
TypeScript
136 lines
5.3 KiB
TypeScript
import { useMemo, useRef } from "react";
|
|
import { getPageVisibility, usePageVisibility, type PageVisibility } from "./page-visibility";
|
|
|
|
/**
|
|
* Polling-cadence and startup-jitter helpers for restore-storm mitigation
|
|
* (PAP-12556 / Phase 1). See PAP-12552 UX guidance §2/§6.
|
|
*
|
|
* Goals:
|
|
* - Visible+focused tabs poll at their normal cadence (active-tab exemption).
|
|
* - Merely-visible-but-unfocused tabs may poll a little slower.
|
|
* - Hidden tabs poll slowly or not at all.
|
|
* - Restored tabs add a small, bounded, randomized jitter so a whole restored
|
|
* window doesn't fire every poller on the same millisecond.
|
|
*
|
|
* Jitter is a herd-breaking tool, not a user-facing delay: content is already on
|
|
* screen from cache (stale-while-revalidate), so jitter never gates first paint.
|
|
*/
|
|
|
|
/** Default bound for jitter added to a focused tab's first poll (ms). */
|
|
export const FOCUSED_JITTER_MAX_MS = 250;
|
|
/** Default bound for jitter added on background / restored tabs (ms). */
|
|
export const BACKGROUND_JITTER_MAX_MS = 3_000;
|
|
|
|
export interface PollingIntervalOptions {
|
|
/** Interval when the tab is visible AND focused. Required. */
|
|
visibleMs: number;
|
|
/**
|
|
* Interval when visible but not focused. Defaults to `visibleMs`
|
|
* (keep near-normal cadence for on-screen tabs).
|
|
*/
|
|
unfocusedMs?: number;
|
|
/**
|
|
* Interval when hidden. `false` (default) stops polling entirely while hidden;
|
|
* react-query also pauses interval timers for hidden tabs, so this is defence in depth.
|
|
*/
|
|
hiddenMs?: number | false;
|
|
}
|
|
|
|
/**
|
|
* Pure resolver: given a visibility state and cadence options, return the react-query
|
|
* `refetchInterval` value (`number` to poll, `false` to pause).
|
|
*/
|
|
export function resolvePollingInterval(
|
|
state: PageVisibility,
|
|
options: PollingIntervalOptions,
|
|
): number | false {
|
|
if (!state.visible) return options.hiddenMs ?? false;
|
|
if (state.focused) return options.visibleMs;
|
|
return options.unfocusedMs ?? options.visibleMs;
|
|
}
|
|
|
|
/**
|
|
* Pure jitter calculator. `rng` returns a float in [0, 1) (injectable for tests).
|
|
* Result is clamped to `[0, maxMs]` and rounded to an integer number of ms.
|
|
*/
|
|
export function computeStartupJitterMs(maxMs: number, rng: () => number): number {
|
|
if (maxMs <= 0) return 0;
|
|
const clamped = Math.min(Math.max(rng(), 0), 0.999999);
|
|
return Math.round(clamped * maxMs);
|
|
}
|
|
|
|
/**
|
|
* Pick the appropriate jitter bound for the current visibility: small for a focused
|
|
* tab (don't make active use feel laggy), wider for background/restored tabs.
|
|
*/
|
|
export function jitterBoundForVisibility(
|
|
state: PageVisibility,
|
|
opts?: { focusedMaxMs?: number; backgroundMaxMs?: number },
|
|
): number {
|
|
const focusedMax = opts?.focusedMaxMs ?? FOCUSED_JITTER_MAX_MS;
|
|
const backgroundMax = opts?.backgroundMaxMs ?? BACKGROUND_JITTER_MAX_MS;
|
|
return state.focused ? focusedMax : backgroundMax;
|
|
}
|
|
|
|
export interface VisibilityRefetchOptions extends PollingIntervalOptions {
|
|
/**
|
|
* Add a small, stable, per-mount jitter to the active interval so a window full of
|
|
* restored tabs doesn't fire the same poller on the same millisecond. Defaults to
|
|
* true. The jitter is computed once at mount (so react-query is not rescheduled on
|
|
* every render) and is bounded to `min(jitterMaxMs, visibleMs * 0.2)`.
|
|
*/
|
|
jitter?: boolean;
|
|
/** Upper bound for the polling jitter (ms). Default 1000. */
|
|
jitterMaxMs?: number;
|
|
}
|
|
|
|
/**
|
|
* React hook: a visibility-aware `refetchInterval` value for react-query.
|
|
* Re-renders (and thus reschedules the query) when visibility/focus changes, but the
|
|
* de-phasing jitter is fixed per mount so the scheduler is not thrashed.
|
|
*
|
|
* Usage:
|
|
* const refetchInterval = useVisibilityRefetchInterval({ visibleMs: 5000 });
|
|
* useQuery({ ..., refetchInterval });
|
|
*/
|
|
export function useVisibilityRefetchInterval(options: VisibilityRefetchOptions): number | false {
|
|
const state = usePageVisibility();
|
|
const jitterRef = useRef<number | null>(null);
|
|
if (jitterRef.current === null) {
|
|
if (options.jitter === false) {
|
|
jitterRef.current = 0;
|
|
} else {
|
|
const bound = Math.min(options.jitterMaxMs ?? 1_000, Math.round(options.visibleMs * 0.2));
|
|
jitterRef.current = computeStartupJitterMs(bound, Math.random);
|
|
}
|
|
}
|
|
const jitter = jitterRef.current;
|
|
return useMemo(() => {
|
|
const base = resolvePollingInterval(state, options);
|
|
return base === false ? false : base + jitter;
|
|
// Options are typically inline literals; depend on their primitive fields.
|
|
// eslint-disable-next-line react-hooks/exhaustive-deps
|
|
}, [state, options.visibleMs, options.unfocusedMs, options.hiddenMs, jitter]);
|
|
}
|
|
|
|
/**
|
|
* React hook returning a stable, bounded startup-jitter delay (ms), computed once
|
|
* per mount from the visibility state at mount time. Use to stagger a non-critical
|
|
* refresh after restore, e.g. gate an `enabled` flag or delay an initial refetch.
|
|
*
|
|
* The value is memoised so it does not change on every render (which would defeat
|
|
* the purpose and thrash timers).
|
|
*/
|
|
export function useStartupJitterMs(opts?: {
|
|
focusedMaxMs?: number;
|
|
backgroundMaxMs?: number;
|
|
}): number {
|
|
const ref = useRef<number | null>(null);
|
|
if (ref.current === null) {
|
|
const state = getPageVisibility();
|
|
const bound = jitterBoundForVisibility(state, opts);
|
|
ref.current = computeStartupJitterMs(bound, Math.random);
|
|
}
|
|
return ref.current;
|
|
}
|