mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-27 11:20:42 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The board UI keeps issue, agent, activity, and run state fresh through polling across several pages and sidebar surfaces > - When multiple components or browser tabs poll the same company data at the same time, the API can receive bursts of duplicate issue-list requests > - Those duplicate requests increase database and server load without returning meaningfully different data > - This pull request adds server-side compression/coalescing plus client-side visibility-aware and cross-tab shared polling > - The benefit is lower request volume during normal board usage while preserving fresh UI data for active users ## Linked Issues or Issue Description No exact public GitHub issue was found for this request. Problem: - The board can issue redundant polling requests for the same issue-list data from multiple UI surfaces and tabs. - In busy operator sessions, those bursts can trigger request-storm behavior and unnecessary issue-list load. - Expected behavior is to reuse identical in-flight work server-side and reduce hidden-tab or duplicate-tab polling client-side while preserving normal refresh behavior. Related public context found during duplicate search: - #8206 covers a different board UI 404-storm scope. - #5165 covers separate issue-list behavior around page-size truncation. ## What Changed - Added API compression middleware foundation and a company/created-at index for heartbeat run access. - Added server-side issue-list request storm detection and identical in-flight request coalescing. - Added UI fetch metadata, visibility-aware polling, and request deduplication for issue/activity/client calls. - Added cross-tab shared polling primitives and wired them into the sidebar, inbox, dashboard, issue, project, routine, and agent surfaces. - Resolved the latest `master` migration collision by keeping upstream `0140_built_in_managed_resources.sql` and renumbering this branch's heartbeat-run index migration to `0141_heartbeat_runs_company_created_at_index.sql`; the SQL uses `CREATE INDEX IF NOT EXISTS` for idempotency. - Stabilized server heartbeat cleanup tests exposed by the PR check matrix. - Fixed the Greptile compression follow-up by weakening strong ETags on encoded JSON responses and bypassing compression for streamed/download responses. ## Verification - `pnpm exec vitest run ui/src/components/IssuesList.test.tsx ui/src/pages/Inbox.test.tsx` — passed after resolving the latest `master` conflict in `IssuesList.tsx` and updating the 200-result cap expectations. - `pnpm check:token-gates` — passed after the UI conflict resolution. - `jq -e '.entries | length as $n | (map(.idx) | unique | length == $n) and (map(.tag) | unique | length == $n)' packages/db/src/migrations/meta/_journal.json` — passed after renumbering the migration to `0141`. - `pnpm exec vitest run server/src/__tests__/api-compression.test.ts` - passed after the compression follow-up. - `pnpm exec vitest run server/src/__tests__/issue-list-assignee-filter-routes.test.ts` - passed after the compression follow-up. - `pnpm --filter @paperclipai/server typecheck` - passed after the compression follow-up. - Greptile Review for head `8dbddac41ec273fda404100b4981ddb912fad57b` - passed after the latest conflict/migration fix; all Greptile review threads are resolved. - `pnpm exec vitest run server/src/__tests__/api-compression.test.ts server/src/__tests__/issue-list-assignee-filter-routes.test.ts ui/src/api/client.test.ts ui/src/api/issues.test.ts ui/src/lib/polling.test.ts ui/src/lib/cross-tab-poll.test.ts ui/src/pages/Inbox.test.tsx ui/src/components/SidebarProjects.test.tsx ui/src/components/SidebarAccountMenu.test.tsx ui/src/lib/issueDetailCache.test.ts` — passed. - `pnpm exec vitest run ui/src/api/client.test.ts` — passed. - `pnpm exec vitest run ui/src/pages/Inbox.test.tsx ui/src/components/SidebarProjects.test.tsx ui/src/components/SidebarAccountMenu.test.tsx ui/src/lib/issueDetailCache.test.ts` — passed. - `pnpm exec vitest run server/src/__tests__/heartbeat-worktree-suppression.test.ts` — passed. - `pnpm exec vitest run server/src/__tests__/low-trust-red-team-routes.test.ts` — passed. - `pnpm exec vitest run server/src/__tests__/heartbeat-workspace-finalize-branch.test.ts` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - GitHub PR checks for head `8dbddac41ec273fda404100b4981ddb912fad57b`: all GitHub Actions/status checks passed; Greptile, Superagent, Socket, Snyk, build, typecheck/release registry, general tests, serialized server suites, e2e, canary dry run, policy, and commitperclip review are green; Storybook visual regression and security-review were skipped/neutral by policy. - Confirmed this branch does not include `pnpm-lock.yaml` or `.github/workflows` changes. ## Risks - Medium risk: issue-list coalescing changes request timing and cache semantics for a hot API path. - Medium risk: cross-tab polling uses browser coordination primitives, so older or unusual browser environments need fallback behavior to stay correct. - Low migration risk: the new index migration is ordered after current `master` and uses `CREATE INDEX IF NOT EXISTS`. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex coding agent, GPT-5-based model, tool-enabled with shell/git execution. Exact hosted deployment identifier and context-window size were not surfaced in the agent runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
251 lines
8.8 KiB
TypeScript
251 lines
8.8 KiB
TypeScript
import { memo, useMemo } from "react";
|
|
import { Link } from "@/lib/router";
|
|
import { useQueries, useQuery } from "@tanstack/react-query";
|
|
import type { Issue, IssueRecoveryAction } from "@paperclipai/shared";
|
|
import { heartbeatsApi, type LiveRunForIssue } from "../api/heartbeats";
|
|
import type { TranscriptEntry } from "../adapters";
|
|
import { issuesApi } from "../api/issues";
|
|
import { queryKeys } from "../lib/queryKeys";
|
|
import { cn, relativeTime } from "../lib/utils";
|
|
import {
|
|
deriveActiveRecoveryDisplayState,
|
|
RECOVERY_CHIP_DEFAULT_TONE,
|
|
} from "../lib/recovery-display";
|
|
import { ExternalLink } from "lucide-react";
|
|
import { Identity } from "./Identity";
|
|
import { RunChatSurface } from "./RunChatSurface";
|
|
import { useLiveRunTranscripts } from "./transcript/useLiveRunTranscripts";
|
|
import { usePublishSharedQueryData, useSharedPollingQuery } from "../hooks/useSharedPolling";
|
|
import { Badge } from "@/components/ui/badge";
|
|
|
|
function RunCardRecoveryChip({ action }: { action: IssueRecoveryAction }) {
|
|
const state = deriveActiveRecoveryDisplayState(action);
|
|
if (!state) return null;
|
|
const tone = RECOVERY_CHIP_DEFAULT_TONE[state];
|
|
const Icon = tone.icon;
|
|
return (
|
|
<Badge variant="outline"
|
|
data-testid="active-agent-run-recovery-indicator"
|
|
data-recovery-state={state}
|
|
role="status"
|
|
aria-label={tone.label}
|
|
title={`${tone.label} — open the source task to act.`}
|
|
className={cn(
|
|
"gap-0.5 px-1.5 text-(length:--text-nano)",
|
|
tone.className,
|
|
)}
|
|
>
|
|
<Icon className="h-2.5 w-2.5" aria-hidden />
|
|
{tone.label}
|
|
</Badge>
|
|
);
|
|
}
|
|
|
|
const MIN_DASHBOARD_RUNS = 4;
|
|
const DASHBOARD_RUN_CARD_LIMIT = 4;
|
|
const DASHBOARD_LOG_POLL_INTERVAL_MS = 15_000;
|
|
const DASHBOARD_LOG_READ_LIMIT_BYTES = 64_000;
|
|
const DASHBOARD_MAX_CHUNKS_PER_RUN = 40;
|
|
const EMPTY_TRANSCRIPT: TranscriptEntry[] = [];
|
|
|
|
function isRunActive(run: LiveRunForIssue): boolean {
|
|
return run.status === "queued" || run.status === "running";
|
|
}
|
|
|
|
interface ActiveAgentsPanelProps {
|
|
companyId: string;
|
|
title?: string;
|
|
minRunCount?: number;
|
|
fetchLimit?: number;
|
|
cardLimit?: number;
|
|
gridClassName?: string;
|
|
cardClassName?: string;
|
|
emptyMessage?: string;
|
|
queryScope?: string;
|
|
showMoreLink?: boolean;
|
|
}
|
|
|
|
export function ActiveAgentsPanel({
|
|
companyId,
|
|
title = "Agents",
|
|
minRunCount = MIN_DASHBOARD_RUNS,
|
|
fetchLimit,
|
|
cardLimit = DASHBOARD_RUN_CARD_LIMIT,
|
|
gridClassName,
|
|
cardClassName,
|
|
emptyMessage = "No recent agent runs.",
|
|
queryScope = "dashboard",
|
|
showMoreLink = true,
|
|
}: ActiveAgentsPanelProps) {
|
|
const liveRunsQueryKey = [...queryKeys.liveRuns(companyId), queryScope, { minRunCount, fetchLimit }] as const;
|
|
const sharedLiveRuns = useSharedPollingQuery({
|
|
companyId,
|
|
resourceKey: `live-runs:${queryScope}:${minRunCount}:${fetchLimit ?? "default"}`,
|
|
queryKey: liveRunsQueryKey,
|
|
enabled: !!companyId,
|
|
leaderOnly: true,
|
|
});
|
|
const { data: liveRuns, dataUpdatedAt: liveRunsUpdatedAt } = useQuery({
|
|
queryKey: liveRunsQueryKey,
|
|
queryFn: () => heartbeatsApi.liveRunsForCompany(companyId, { minCount: minRunCount, limit: fetchLimit }),
|
|
enabled: sharedLiveRuns.enabled,
|
|
});
|
|
usePublishSharedQueryData(sharedLiveRuns, liveRuns, liveRunsUpdatedAt);
|
|
|
|
const runs = liveRuns ?? [];
|
|
const visibleRuns = useMemo(() => runs.slice(0, cardLimit), [cardLimit, runs]);
|
|
const hiddenRunCount = Math.max(0, runs.length - visibleRuns.length);
|
|
const visibleIssueIds = useMemo(
|
|
() => [...new Set(visibleRuns.map((run) => run.issueId).filter((issueId): issueId is string => Boolean(issueId)))],
|
|
[visibleRuns],
|
|
);
|
|
|
|
const issueQueries = useQueries({
|
|
queries: visibleIssueIds.map((issueId) => ({
|
|
queryKey: queryKeys.issues.detail(issueId),
|
|
queryFn: () => issuesApi.get(issueId),
|
|
staleTime: 30_000,
|
|
retry: false,
|
|
})),
|
|
});
|
|
|
|
const issueById = useMemo(() => {
|
|
const map = new Map<string, Issue>();
|
|
for (const query of issueQueries) {
|
|
const issue = query.data;
|
|
if (issue) map.set(issue.id, issue);
|
|
}
|
|
return map;
|
|
}, [issueQueries]);
|
|
|
|
const { transcriptByRun, hasOutputForRun } = useLiveRunTranscripts({
|
|
runs: visibleRuns,
|
|
companyId,
|
|
maxChunksPerRun: DASHBOARD_MAX_CHUNKS_PER_RUN,
|
|
logPollIntervalMs: DASHBOARD_LOG_POLL_INTERVAL_MS,
|
|
logReadLimitBytes: DASHBOARD_LOG_READ_LIMIT_BYTES,
|
|
enableRealtimeUpdates: false,
|
|
});
|
|
|
|
return (
|
|
<div>
|
|
<h3 className="mb-3 text-sm font-semibold uppercase tracking-wide text-muted-foreground">
|
|
{title}
|
|
</h3>
|
|
{runs.length === 0 ? (
|
|
<div className="rounded-xl border border-border p-4">
|
|
<p className="text-sm text-muted-foreground">{emptyMessage}</p>
|
|
</div>
|
|
) : (
|
|
<div className={cn("grid grid-cols-1 gap-2 sm:grid-cols-2 sm:gap-4 xl:grid-cols-4", gridClassName)}>
|
|
{visibleRuns.map((run) => (
|
|
<AgentRunCard
|
|
key={run.id}
|
|
companyId={companyId}
|
|
run={run}
|
|
issue={run.issueId ? issueById.get(run.issueId) : undefined}
|
|
transcript={transcriptByRun.get(run.id) ?? EMPTY_TRANSCRIPT}
|
|
hasOutput={hasOutputForRun(run.id)}
|
|
isActive={isRunActive(run)}
|
|
className={cardClassName}
|
|
/>
|
|
))}
|
|
</div>
|
|
)}
|
|
{showMoreLink && hiddenRunCount > 0 && (
|
|
<div className="mt-3 flex justify-end text-xs text-muted-foreground">
|
|
<Link to="/dashboard/live" className="hover:text-foreground hover:underline">
|
|
{hiddenRunCount} more active/recent run{hiddenRunCount === 1 ? "" : "s"}
|
|
</Link>
|
|
</div>
|
|
)}
|
|
</div>
|
|
);
|
|
}
|
|
|
|
const AgentRunCard = memo(function AgentRunCard({
|
|
companyId,
|
|
run,
|
|
issue,
|
|
transcript,
|
|
hasOutput,
|
|
isActive,
|
|
className,
|
|
}: {
|
|
companyId: string;
|
|
run: LiveRunForIssue;
|
|
issue?: Issue;
|
|
transcript: TranscriptEntry[];
|
|
hasOutput: boolean;
|
|
isActive: boolean;
|
|
className?: string;
|
|
}) {
|
|
return (
|
|
<div className={cn(
|
|
"flex h-(--sz-320px) flex-col overflow-hidden rounded-xl border shadow-sm",
|
|
isActive
|
|
? "border-blue-500/25 bg-blue-500/[0.04] shadow-(--shadow-extract-1)"
|
|
: "border-border bg-background/70",
|
|
className,
|
|
)}>
|
|
<div className="border-b border-border/60 px-3 py-3">
|
|
<div className="flex items-start justify-between gap-2">
|
|
<div className="min-w-0">
|
|
<div className="flex items-center gap-2">
|
|
{isActive ? (
|
|
<span className="relative flex h-2.5 w-2.5 shrink-0">
|
|
<span className="absolute inline-flex h-full w-full animate-ping rounded-full bg-blue-400 opacity-70" />
|
|
<span className="relative inline-flex h-2.5 w-2.5 rounded-full bg-blue-500" />
|
|
</span>
|
|
) : (
|
|
<span className="inline-flex h-2.5 w-2.5 rounded-full bg-muted-foreground/35" />
|
|
)}
|
|
<Identity name={run.agentName} size="sm" className="[&>span:last-child]:!text-(length:--text-micro)" />
|
|
</div>
|
|
<div className="mt-2 flex items-center gap-2 text-(length:--text-micro) text-muted-foreground">
|
|
<span>{isActive ? "Live now" : run.finishedAt ? `Finished ${relativeTime(run.finishedAt)}` : `Started ${relativeTime(run.createdAt)}`}</span>
|
|
</div>
|
|
</div>
|
|
|
|
<Link
|
|
to={`/agents/${run.agentId}/runs/${run.id}`}
|
|
className="inline-flex items-center gap-1 rounded-full border border-border/70 bg-background/70 px-2 py-1 text-(length:--text-nano) text-muted-foreground transition-colors hover:text-foreground"
|
|
>
|
|
<ExternalLink className="h-2.5 w-2.5" />
|
|
</Link>
|
|
</div>
|
|
|
|
{run.issueId && (
|
|
<div className="mt-3 rounded-lg border border-border/60 bg-background/60 px-2.5 py-2 text-xs">
|
|
<Link
|
|
to={`/issues/${issue?.identifier ?? run.issueId}`}
|
|
className={cn(
|
|
"line-clamp-2 hover:underline",
|
|
isActive ? "text-blue-700 dark:text-blue-300" : "text-muted-foreground hover:text-foreground",
|
|
)}
|
|
title={issue?.title ? `${issue?.identifier ?? run.issueId.slice(0, 8)} - ${issue.title}` : issue?.identifier ?? run.issueId.slice(0, 8)}
|
|
>
|
|
{issue?.identifier ?? run.issueId.slice(0, 8)}
|
|
{issue?.title ? ` - ${issue.title}` : ""}
|
|
</Link>
|
|
{issue?.activeRecoveryAction ? (
|
|
<div className="mt-1.5">
|
|
<RunCardRecoveryChip action={issue.activeRecoveryAction} />
|
|
</div>
|
|
) : null}
|
|
</div>
|
|
)}
|
|
</div>
|
|
|
|
<div className="min-h-0 flex-1 overflow-y-auto p-3">
|
|
<RunChatSurface
|
|
run={run}
|
|
transcript={transcript}
|
|
hasOutput={hasOutput}
|
|
companyId={companyId}
|
|
/>
|
|
</div>
|
|
</div>
|
|
);
|
|
});
|