perf(ui): read the compact task index on the dashboard and refresh it once per window (#21413)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The board UI keeps company lists fresh from live task activity
events
> - The dashboard, the activity feed and board chat read the full
company task list, and every burst of task activity refetches it
> - One full list request costs the server 60 statements and 61-90 MB of
allocation for 500 tasks, and the staging server is saturated by
allocation and garbage collection
> - This pull request moves those three views to the compact task index
under its own key, and task activity refreshes that key at most once per
30 seconds
> - The benefit is about one tenth of the server allocation per open
dashboard tab while agents work, with the same rows on screen

## Linked Issues or Issue Description

**What existing behavior does this improve?**
The dashboard, the activity feed and board chat load the full company
task list. Every burst of task activity refetches that list.

**Subsystem affected**
Board UI: live updates, the dashboard, the activity feed and board chat.

**Current behavior**
The three views read the full company task list (`GET
/api/companies/:id/issues`, no view) with the key `["issues",
companyId]`. `LiveUpdatesProvider` invalidates that key on every task
activity event. Each open tab refetches the list at most every 5 seconds
while agents work. One request costs the server 60 statements and 61-90
MB of allocation for 500 tasks (local measurement). The staging server
allocates 57.6 MB per second and is saturated by garbage collection.

**Proposed behavior**
The views read only the fields they show, from the compact task index.
Task activity from other people refreshes the index at most once per
window. The viewer's own task changes refresh it at once.

**Reason and benefit**
Less server allocation and fewer statements per open tab, with the same
rows on screen.

**Breaking changes**
None. No API, schema or server change.

## What Changed

- The dashboard, the activity feed and board chat read `view=index`
under a new key `issues.listIndex(companyId)`.
- Task activity from other people refreshes that key at most once per 30
seconds (`TASK_INDEX_REFRESH_WINDOW_MS`). The viewer's own task activity
refreshes it at once. The first event after a quiet window refetches at
once. Later events in the window share one trailing refetch. Nothing
from the event goes into the cache. An unwatched index is marked stale
and refetches on the next mount. The user's own task changes still
invalidate the whole `["issues", companyId]` prefix immediately.
- The dashboard takes its ten recent tasks from the index, sorted by
updatedAt as before. It reads the status glyph fields for those ten
tasks from `issues/summaries`, because the index omits displayStatus and
blockerAttention.
- Toast and route lookups that read the full list cache also read the
index cache.
- `server/scripts/query-cost/` is the per-caller query cost harness used
for the measurements below. It records the caller of each query at
construction, the bytes allocated to build it, the rows it returns and
the statements sent.

## Verification

- `pnpm --filter @paperclipai/ui exec vitest run
src/context/LiveUpdatesProvider src/pages/Dashboard.test.ts`: 261 pass.
- Red/green: `LiveUpdatesProvider.task-index.test.ts` fails 6 of its
first 10 cases against the provider on master and passes with this
change. The tests cover a burst of events (4 index fetches in 90
seconds), the first event after a quiet window, a new task, a deleted
task, a hidden task, a status/title/priority change, an event for a task
outside the cache, an event for a task the viewer cannot read (nothing
from the event goes into the cache), the viewer's own activity
(refreshes at once), a task change made in this tab (refreshes at once
through the company list invalidation), the trailing refresh (the last
state shows when the window ends after activity stops), an unwatched
index, and the live-event batcher path.
- Other readers of `["issues", companyId]` (My Tasks, the task
properties picker, the command palette, task detail cache writes,
filtered and board-column lists) are not changed. The broad invalidation
excludes only the new index key, so they refresh as before. A hidden tab
keeps today's behavior: the live-event batcher still turns its refetches
into stale marks.
- Field parity (local, 1,500 tasks): the index returns the same 500 ids
as the full list, with 0 differences in id, identifier, title, status,
priority, createdAt, updatedAt and assigneeAgentId. It omits
displayStatus and blockerAttention on all 500 rows, so the recent rows
take those fields from summaries.
- Real browser (headless Chromium, Playwright) against a local server
from this branch, started with `test-drive --data-dir <scratch dir>`,
which sets `PAPERCLIP_HOME` and `PAPERCLIP_CONFIG` to that directory:
- Dashboard: the recent tasks and the "Tasks by Status" chart show the
new tasks. The page sent 0 full-list requests, 2 index requests and 1
summaries request.
- A task rename by this board user showed on the dashboard after 0.8
seconds, with 1 index refetch.
  - A second tab on the dashboard showed the same tasks.
- Board chat loaded with 0 full-list requests and 3 index requests. Its
activity feed showed the task names.
- Not checked in the browser: activity from another actor. An agent key
cannot edit a task without a heartbeat run context, so the window for
other people's activity is covered by the unit tests only.
- Nothing visual changed. The views keep their render code. Only their
data source changed.
- Cost per request (local, embedded PostgreSQL, Node 26, uninstrumented
pass, 5 requests each), measured:
`TSX_DISABLE_CACHE=1 UNITS=board ROUTES="issues list,issues list
view=index,issues/summaries (10 ids)" REPEAT=5 NODE_OPTIONS=--expose-gc
pnpm --filter @paperclipai/server exec tsx scripts/query-cost/drive.ts`

  | Request | Statements | Allocated |
  | --- | --- | --- |
  | full list | 60 | 61-90 MB |
  | view=index | 12 | 22-37 MB |
  | summaries, 10 ids | 23 | 2.8 MB |

  The ranges show the difference between runs.
- Estimate, not measured: per open dashboard tab with continuous task
activity, before about 13 MB/s (one full list per 5 seconds), after
about 1.3 MB/s (one index per 30 seconds and one 10-id summaries request
per 5 seconds). At the day-1 staging route rate (0.116 full lists per
second) that is about 6.5 MB/s less. The real rate of task activity
events on staging is not measured.

## Risks

- Freshness: the dashboard's task charts and recent-task order, the
activity feed's task names and board chat's task lookup can lag task
activity from other people by up to 30 seconds. The status glyphs of the
ten recent tasks stay on the 5-second lane. The viewer's own task
changes refresh at once. The window is one constant,
`TASK_INDEX_REFRESH_WINDOW_MS` in `LiveUpdatesProvider.tsx`, so a
different value is a one-line change.
- The index returns the same rows as the full list, so the charts count
the same tasks.
- The recent-task rows have no task description. The rows do not show
it.
- My Tasks, the task properties picker and the command palette still
read the full list. They are not in this change.

## Model Used

- Claude Opus 5.5 (claude-opus-5-5), Anthropic, with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (no
documentation describes this behavior)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed 2026-11-13 11:50:24 -06:00
1 parent bb253c6a99
commit b993702ae3
10 files changed
+795 -21

No files matched your search

+273
View File
@@ -0,0 +1,273 @@
import "./instrument.js";
import assert from "node:assert/strict";
import { randomBytes, randomUUID } from "node:crypto";
import type { AddressInfo } from "node:net";
import { writeFileSync } from "node:fs";
import express from "express";
import { completionContracts, createDb, heartbeatRuns, issues, startEmbeddedPostgresTestDatabase } from "../../../packages/db/src/index.js";
import { countStatements, diff, format, setEnabled, snapshot } from "./instrument.js";
import { seedReviewAttentionFixture } from "../../src/__tests__/helpers/review-attention-fixture.js";
import { seedAttentionSummaryVolume } from "../../src/__tests__/helpers/attention-summary-fixture.js";
process.env.PAPERCLIP_SECRETS_MASTER_KEY ??= randomBytes(32).toString("base64");
const only = (process.env.UNITS ?? "board,event").split(",");
const TASKS = Number(process.env.TASKS ?? 1500), AGENTS = Number(process.env.AGENTS ?? 40), REPEAT = Number(process.env.REPEAT ?? 3);
const keepAlive = setInterval(() => {}, 1_000);
const temporary = await startEmbeddedPostgresTestDatabase("query-cost-");
const db = createDb(temporary.connectionString);
countStatements(db as never);
const report: string[] = [];
const out = process.env.OUT ?? "query-cost.md";
async function measure(label: string, units: number, action: () => Promise<void>) {
await action();
(globalThis as { gc?: () => void }).gc?.();
const plain = snapshot();
for (let i = 0; i < units; i++) await action();
const uninstrumented = diff(plain);
setEnabled(true);
const before = snapshot();
for (let i = 0; i < units; i++) await action();
const result = diff(before);
setEnabled(false);
const text = `uninstrumented: statements ${(uninstrumented.statements / units).toFixed(1)}, allocated ${(uninstrumented.allocated / 1024 / units).toFixed(1)} KB, wall ${(uninstrumented.wallMs / units).toFixed(1)} ms per unit\n` + format(label, units, result);
report.push(text);
console.log(text + "\n");
writeFileSync(out, report.join("\n\n") + "\n");
}
try {
const fixture = await seedReviewAttentionFixture(db, { tasks: TASKS, agents: AGENTS, prefix: "SYN" });
await seedAttentionSummaryVolume(db, fixture);
const { companyId, userId } = fixture;
const agentId = fixture.agentIds[0]!;
if (only.includes("calibrate")) {
const { sql } = await import("drizzle-orm");
const v8 = await import("node:v8");
const alloc = () => (v8.getHeapStatistics() as unknown as { total_allocated_bytes: number }).total_allocated_bytes;
const lines = ["## read calibration (prepared, build excluded): KB allocated per execution", "| rows | fields | bytes/value | KB/exec | wire KB | bytes alloc per wire byte |", "| --- | --- | --- | --- | --- | --- |"];
for (const [rows, fields, width] of [[1, 1, 8], [1, 20, 8], [1, 80, 8], [50, 20, 8], [50, 20, 200], [500, 10, 40], [500, 3, 2000]] as const) {
const fieldsObj = Object.fromEntries(Array.from({ length: fields }, (_, i) => [`c${i}`, sql<string>`repeat('x', ${sql.raw(String(width))})`.as(`c${i}`)]));
const prepared = db.select(fieldsObj).from(sql`generate_series(1, ${sql.raw(String(rows))})`).prepare(`cal_${rows}_${fields}_${width}`);
for (let i = 0; i < 5; i++) await prepared.execute();
const start = alloc();
for (let i = 0; i < 30; i++) await prepared.execute();
const kb = (alloc() - start) / 30 / 1024, wire = rows * fields * (width + 4) / 1024;
lines.push(`| ${rows} | ${fields} | ${width} | ${kb.toFixed(1)} | ${wire.toFixed(1)} | ${(kb / wire).toFixed(1)} |`);
}
report.push(lines.join("\n")); console.log(lines.join("\n"));
}
if (only.includes("board")) {
const { attentionRoutes } = await import("../../src/routes/attention.js");
const { issueRoutes } = await import("../../src/routes/issues.js");
const { agentRoutes } = await import("../../src/routes/agents.js");
const { errorHandler } = await import("../../src/middleware/index.js");
const runIds: string[] = [];
for (const [index, issueId] of fixture.taskIds.slice(0, 12).entries()) {
const id = randomUUID();
runIds.push(id);
await db.insert(heartbeatRuns).values({ id, companyId, agentId: fixture.agentIds[index % fixture.agentIds.length]!, status: "running",
issueId, contextSnapshot: { issueId }, startedAt: new Date() });
}
const app = express();
app.use(express.json());
app.use((req, _res, next) => {
(req as unknown as { actor: unknown }).actor = { type: "board", source: "session", userId, companyIds: [companyId], isInstanceAdmin: false };
next();
});
app.use("/api", attentionRoutes(db));
app.use("/api", issueRoutes(db, {} as never));
app.use("/api", agentRoutes(db, {} as never));
app.use(errorHandler);
const server = app.listen(0, "127.0.0.1");
await new Promise(resolve => server.once("listening", resolve));
const base = `http://127.0.0.1:${(server.address() as AddressInfo).port}/api`;
const get = async (route: string) => {
const response = await fetch(`${base}${route}`);
const body = await response.text();
assert.equal(response.status, 200, `${route}: ${body.slice(0, 300)}`);
return body.length;
};
const ids = fixture.taskIds.slice(0, 50).join(",");
const routes: Record<string, string> = {
"attention/summary": `/companies/${companyId}/attention/summary`,
"issues list": `/companies/${companyId}/issues`,
"issues list view=index": `/companies/${companyId}/issues?view=index`,
"issues list view=rows": `/companies/${companyId}/issues?view=rows`,
"issues/summaries (50 ids)": `/companies/${companyId}/issues/summaries?ids=${ids}`,
"issues/summaries (10 ids)": `/companies/${companyId}/issues/summaries?ids=${fixture.taskIds.slice(0, 10).join(",")}`,
"live-runs": `/companies/${companyId}/live-runs`,
"agents list": `/companies/${companyId}/agents`,
"issue detail": `/issues/${fixture.taskIds[3]}`,
"heartbeat run detail": `/heartbeat-runs/${runIds[0]}`,
};
if (only.includes("fields")) {
const read = async (route: string) => JSON.parse(await (await fetch(`${base}${route}`)).text()) as Array<Record<string, unknown>>;
const full = await read(`/companies/${companyId}/issues`);
const index = await read(`/companies/${companyId}/issues?view=index`);
const rows = await read(`/companies/${companyId}/issues?view=rows&sortField=updated&sortDir=desc&limit=10`);
const fields = ["id", "identifier", "title", "status", "priority", "createdAt", "updatedAt", "assigneeAgentId", "displayStatus", "externalConversationState", "blockerAttention"];
const byId = new Map(full.map(row => [row.id, row]));
const diffs: Record<string, number> = {};
for (const [label, set] of [["index", index], ["rows10", rows]] as const) {
for (const row of set) for (const field of fields) {
const a = JSON.stringify(byId.get(row.id)?.[field] ?? null), b = JSON.stringify(row[field] ?? null);
if (a !== b) diffs[`${label}.${field}`] = (diffs[`${label}.${field}`] ?? 0) + 1;
}
}
const recentFull = [...full].sort((a, b) => Date.parse(String(b.updatedAt)) - Date.parse(String(a.updatedAt))).slice(0, 10).map(row => row.id);
console.log(JSON.stringify({ full: full.length, index: index.length, rows: rows.length,
sameIds: JSON.stringify(full.map(r => r.id).sort()) === JSON.stringify(index.map(r => r.id).sort()),
sameRecent: JSON.stringify(recentFull) === JSON.stringify(rows.map(r => r.id)), diffs,
fullHasDisplayStatus: full.filter(r => r.displayStatus !== undefined).length, fullBlocker: full.filter(r => r.blockerAttention).length }));
}
for (const [name, route] of Object.entries(routes).filter(([name]) => !process.env.ROUTES || process.env.ROUTES.split(",").includes(name))) {
try {
const bytes = await get(route);
await measure(`GET ${name} (body ${Math.round(bytes / 1024)} KB)`, REPEAT, async () => { await get(route); });
} catch (error) { console.log(`skip ${name}: ${(error as Error).message.slice(0, 300)}`); }
}
server.close();
}
if (only.includes("recovery")) {
const { assignedWorkSweep } = await import("../../src/services/recovery/assigned-work-sweep.js");
const sweep = assignedWorkSweep(db, { enqueueWakeup: async () => null, proveOwner: async () => ({ kind: "alive" }) as never });
const suspects = await sweep.listSuspects({ companyId }, null, 100);
console.log(`suspects ${suspects.length}`);
await measure("assigned-work sweep load (listSuspects page of 100)", 5, async () => { await sweep.listSuspects({ companyId }, null, 100); });
await measure("assigned-work sweep check of one task (checkIssue)", 25, async () => {
for (const suspect of suspects.slice(0, 1)) await sweep.checkIssue({ companyId, issueId: suspect.id });
});
const { loadNativeChatProgressScopes, reconcileNativeChatProgress } = await import("../../src/services/chat-native-progress.js");
await measure("chat native progress load + check", 5, async () => {
await loadNativeChatProgressScopes(db, { companyId } as never);
await reconcileNativeChatProgress(db, { scope: { companyId, issueId: fixture.taskIds[0]! }, limit: 100 });
});
}
if (only.includes("agent")) {
process.env.PAPERCLIP_AGENT_JWT_SECRET = randomBytes(32).toString("hex");
const { createLocalAgentJwt } = await import("../../src/agent-auth-jwt.js");
const { actorMiddleware } = await import("../../src/middleware/auth.js");
const { issueRoutes } = await import("../../src/routes/issues.js");
const { errorHandler } = await import("../../src/middleware/index.js");
const runId = randomUUID();
await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, status: "running", issueId: fixture.taskIds[3]!,
contextSnapshot: { issueId: fixture.taskIds[3]! }, startedAt: new Date() });
const token = createLocalAgentJwt(agentId, companyId, "codex_local", runId);
assert.ok(token, "agent token");
const app = express();
app.use(express.json());
app.use(actorMiddleware(db, { deploymentMode: "authenticated", localAgentProcessBoundary: null }));
app.get("/api/whoami", (req, res) => { res.json({ type: (req as unknown as { actor: { type: string } }).actor.type }); });
app.use("/api", issueRoutes(db, {} as never));
app.use(errorHandler);
const server = app.listen(0, "127.0.0.1");
await new Promise(resolve => server.once("listening", resolve));
const base = `http://127.0.0.1:${(server.address() as AddressInfo).port}/api`;
const get = async (route: string) => {
const response = await fetch(`${base}${route}`, { headers: { authorization: `Bearer ${token}`, "x-paperclip-run-id": runId } });
const body = await response.text();
assert.equal(response.status, 200, `${route}: ${body.slice(0, 300)}`);
return body;
};
assert.match(await get("/whoami"), /agent/);
await measure("agent request authentication only (actorMiddleware, agent JWT)", 20, async () => { await get("/whoami"); });
await measure("agent GET issues/:id (auth + route)", 5, async () => { await get(`/issues/${fixture.taskIds[3]}`); });
server.close();
}
if (only.includes("legacy")) {
const { createAgentLifecycle, configureAgentLifecycle } = await import("../../src/services/agent-lifecycle.js");
const { heartbeatService } = await import("../../src/services/heartbeat.js");
const { createDeliveryWorkCoordinator } = await import("../../src/services/delivery-work-coordinator.js");
const { enableDatabaseWorkNotifications } = await import("../../../packages/db/src/work-signals.js");
const { eq } = await import("drizzle-orm");
enableDatabaseWorkNotifications(db);
const lines = Number(process.env.LINES ?? 20), gapMs = Number(process.env.GAP_MS ?? 50);
const legacyAgentId = randomUUID(), legacyCompanyId = randomUUID();
const { companies } = await import("../../../packages/db/src/index.js");
await db.insert(companies).values({ id: legacyCompanyId, name: "Legacy", issuePrefix: `L${legacyCompanyId.slice(0, 6)}`, requireBoardApprovalForNewAgents: false, defaultResponsibleUserId: "legacy-user" });
await createAgentLifecycle(db).requestHire(legacyCompanyId, { id: legacyAgentId, name: "Legacy process", role: "engineer", status: "idle",
adapterType: "process", adapterConfig: { command: process.execPath,
args: ["-e", `let i=0;const t=setInterval(()=>{console.log("output line "+(++i)+" ".padEnd(80,"x"));if(i>=${lines}){clearInterval(t);}},${gapMs});`],
cwd: process.cwd() },
runtimeConfig: { heartbeat: { enabled: true, intervalSec: 0, wakeOnDemand: true } }, permissions: {} }, { runnerResolved: true });
const lifecycleWorker = configureAgentLifecycle(db, { requiredPluginIds: async () => [], runPlugin: async () => "complete", runHost: async () => "complete" });
for (let i = 0; i < 4; i++) await lifecycleWorker.process(legacyAgentId);
await lifecycleWorker.stop();
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, canReconcile: () => true, onError: error => console.log(`coordinator error ${String(error).slice(0, 200)}`) });
const heartbeat = heartbeatService(db, { runtimeEnv: { ...process.env, PAPERCLIP_IN_WORKTREE: undefined } });
const dispatch = heartbeat.registerRunDispatchWork(coordinator, () => true);
await dispatch.ready;
const runOnce = async () => {
const run = await heartbeat.wakeup(legacyAgentId, { source: "on_demand", reason: "manual", requestedByActorType: "system" });
if (!run) {
const { agents, agentWakeupRequests } = await import("../../../packages/db/src/index.js");
console.log(JSON.stringify({ agent: (await db.select({ status: agents.status, holds: agents.lifecycleHolds, state: agents.lifecycleState }).from(agents).where(eq(agents.id, legacyAgentId)))[0],
wakes: await db.select({ status: agentWakeupRequests.status, reason: agentWakeupRequests.reason, error: agentWakeupRequests.error }).from(agentWakeupRequests).where(eq(agentWakeupRequests.agentId, legacyAgentId)) }));
}
assert.ok(run, "wake accepted");
for (let i = 0; i < 600; i++) {
const [row] = await db.select({ status: heartbeatRuns.status }).from(heartbeatRuns).where(eq(heartbeatRuns.id, run.id));
if (row && !["queued", "running"].includes(row.status)) {
assert.equal(row.status, "succeeded", "a measured legacy run must succeed");
break;
}
assert.ok(i < 599, "a measured legacy run must finish within 60 seconds");
await new Promise(resolve => setTimeout(resolve, 100));
}
await heartbeat.drainActiveRunExecutions();
};
await measure(`legacy process run lifecycle (wake, queue, start, ${lines} output lines ${gapMs} ms apart, finish), includes the harness's 100 ms status poll`, 3, runOnce);
await dispatch.stop();
await coordinator.stop();
}
if (only.includes("event")) {
const { PaperclipControlPlanePort } = await import("../../src/services/native-runtime/paperclip-control-plane-port.js");
const [issue] = await db.insert(issues).values({ companyId, title: "Streaming", status: "in_progress", assigneeAgentId: agentId }).returning();
const contractSha = "query-cost";
const [contract] = await db.insert(completionContracts).values({ companyId, issueId: issue!.id, revision: 1,
schemaVersion: "paperclip.completion-contract.v1", policyVersion: "benchmark", risk: "standard",
completionAuthority: "server_arbiter", incompleteCriteriaPolicy: "preserve_non_terminal",
contractJson: { revision: "benchmark", objective: "Benchmark", criteria: [{ id: "objective", requirement: "Append" }] },
canonicalSha256: contractSha, createdByActorType: "system", createdByActorId: "benchmark" }).returning();
const runId = randomUUID(), sessionId = randomUUID(), runnerId = randomUUID();
await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, status: "running", runtimeMode: "native",
nativeIssueId: issue!.id, issueId: issue!.id, nativeSessionId: sessionId, runnerInstanceId: runnerId,
completionContractId: contract!.id, completionContractSha256: contractSha, contextSnapshot: { issueId: issue!.id } });
const identity = { companyId, issueId: issue!.id, runId, agentId, sessionId };
const port = new PaperclipControlPlanePort(db, { ...identity, completionContractId: contract!.id,
completionContractSha256: contractSha, sourceInstanceId: runnerId, controlPlaneSourceInstanceId: `control-${runId}` });
await port.openRun({ identity, backendKind: "mock", sourceInstanceId: runnerId } as never);
const text = randomBytes(72).toString("base64").slice(0, 96);
let ordinal = 0;
const next = async () => {
ordinal += 1;
const step = ordinal % 10;
const base = { schema: "paperclip.prp.event.v1" as const, sourceEventId: `${runnerId}:${ordinal}`, sourceSeq: ordinal,
sourceInstanceId: runnerId, sourceKind: "runner" as const, runId, normalizedSessionId: sessionId,
turnId: "turn-1", schemaVersion: 1 as const, priority: 1, emittedAt: new Date().toISOString() };
const event = step < 7 ? { ...base, itemId: "message-1", eventType: "item.delta", payload: { kind: "agentMessage", text } }
: step === 9 ? { ...base, itemId: `command-${ordinal}`, eventType: "item.completed",
payload: { kind: "commandExecution", item: { type: "command", id: `command-${ordinal}`, output: text } } }
: { ...base, itemId: `tool-${ordinal}`, eventType: step === 7 ? "tool.execution.started" : "tool.execution.completed",
payload: { schema: "paperclip.tool.execution.v1", executionId: `tool-${ordinal - (step === 7 ? 0 : 1)}`, transport: "builtin",
operation: "execute", name: "shell", target: null, namespace: null, readOnly: true,
status: step === 7 ? "running" : "completed", durationMs: step === 7 ? null : 12, exitCode: step === 7 ? null : 0,
progress: null, output: text, outputBytes: text.length, outputTruncated: false, outputDigest: `sha256:${"a".repeat(64)}` } };
await port.appendEvent(event as never);
};
await measure("native runner event via PaperclipControlPlanePort.appendEvent", 200, next);
}
} finally {
clearInterval(keepAlive);
await db.$client.end({ timeout: 1 }).catch(() => {});
await temporary.cleanup();
}
console.log(`report: ${out}`);
process.exit(0);
+216
View File
@@ -0,0 +1,216 @@
import v8 from "node:v8";
import { performance } from "node:perf_hooks";
import { PgDatabase, PgDeleteBase, PgInsertBase, PgInsertBuilder, PgSelectBase, PgSelectBuilder, PgUpdateBase, PgUpdateBuilder } from "drizzle-orm/pg-core";
import { PgRelationalQuery, RelationalQueryBuilder } from "drizzle-orm/pg-core/query-builders/query";
type Entry = {
key: string; operation: string; callers: string[]; builds: number; executions: number;
sqlBytes: number; buildBytes: number; buildMs: number; readBytes: number; readMs: number;
rows: number; resultBytes: number; fields: number; calibratedBytes?: number; calibratedMs?: number; sqlPrefix?: string;
};
const CALIBRATED = 3, LOOP = 40;
const allocated = () => (v8.getHeapStatistics() as unknown as { total_allocated_bytes: number }).total_allocated_bytes;
if (typeof allocated() !== "number") {
throw new Error(`query-cost needs a Node.js whose v8.getHeapStatistics() reports total_allocated_bytes (Node.js 26 or later); this is ${process.version}`);
}
const site = new WeakMap<object, { key: string; operation: string; callers: string[] }>();
export const entries = new Map<string, Entry>();
export const totals = { statements: 0, control: 0 };
let enabled = false;
let depth = 0;
export function setEnabled(value: boolean) { enabled = value; }
const root = process.cwd().replace(/\/server$/, "");
function callers(boundary: Function): string[] {
const holder: { stack?: string } = {};
const limit = Error.stackTraceLimit;
Error.stackTraceLimit = 40;
Error.captureStackTrace(holder, boundary);
Error.stackTraceLimit = limit;
const out: string[] = [];
for (const line of (holder.stack ?? "").split("\n").slice(1)) {
if (line.includes("node_modules") || line.includes("query-cost/") || line.includes("node:")) continue;
const match = line.match(/at (?:async )?(?:(\S+) \()?(?:file:\/\/)?([^()]+):(\d+):\d+\)?$/);
if (!match) continue;
const file = match[2]!.replace(root + "/", "");
if (file.startsWith("packages/db/src/client") || file.includes("lazy-transaction-queries")) continue;
out.push(`${file}:${match[3]}${match[1] ? ` ${match[1]}` : ""}`);
if (out.length === 3) break;
}
return out;
}
function tag(builder: object, operation: string, boundary: Function) {
if (!enabled || !builder || typeof builder !== "object") return;
const stack = callers(boundary);
site.set(builder, { key: `${operation} ${stack[0] ?? "unknown"}`, operation, callers: stack });
}
function entry(info: { key: string; operation: string; callers: string[] }): Entry {
let value = entries.get(info.key);
if (!value) {
value = { ...info, builds: 0, executions: 0, sqlBytes: 0, buildBytes: 0, buildMs: 0, readBytes: 0, readMs: 0, rows: 0, resultBytes: 0, fields: 0 };
entries.set(info.key, value);
}
return value;
}
function resultSize(value: unknown): { rows: number; bytes: number } {
const rows = Array.isArray(value) ? value : (value && typeof value === "object" && Array.isArray((value as { rows?: unknown }).rows)) ? (value as { rows: unknown[] }).rows : [];
let bytes = 0;
try { bytes = Buffer.byteLength(JSON.stringify(rows, (_k, v) => typeof v === "bigint" ? String(v) : v) ?? ""); } catch {}
return { rows: rows.length, bytes };
}
function record(e: Entry, value: unknown) {
const size = resultSize(value);
const rows = Array.isArray(value) ? value : [];
const width = e.fields || (rows[0] && typeof rows[0] === "object" ? Object.keys(rows[0] as object).length : 0);
e.rows += size.rows;
e.resultBytes += size.bytes;
e.readBytes += 40 * 1024 + width * 1024 + size.rows * width * 256 + size.bytes * 1.3;
}
function wrapPrepared(prepared: any, info: { key: string; operation: string; callers: string[] }) {
if (!prepared || typeof prepared.execute !== "function" || prepared.__qcost) return;
prepared.__qcost = true;
const original = prepared.execute;
prepared.execute = function (this: unknown, ...args: unknown[]) {
if (!enabled) return Reflect.apply(original, this, args);
const e = entry(info);
e.executions += 1;
const result = Reflect.apply(original, this, args) as Promise<unknown>;
return Promise.resolve(result).then(value => { record(e, value); return value; });
};
}
function wrapPrepare(proto: any, operation: string) {
const original = proto._prepare;
proto._prepare = function (this: object, ...args: unknown[]) {
if (!enabled || depth > 0) return Reflect.apply(original, this, args);
const info = site.get(this) ?? { key: `${operation} untagged`, operation, callers: [] };
depth += 1;
const startBytes = allocated(), startMs = performance.now();
let prepared: any;
try { prepared = Reflect.apply(original, this, args); }
finally { depth -= 1; }
let buildBytes = allocated() - startBytes, buildMs = performance.now() - startMs;
const e = entry(info);
if (e.builds < CALIBRATED) {
depth += 1;
try {
const loopBytes = allocated(), loopMs = performance.now();
for (let i = 0; i < LOOP; i++) Reflect.apply(original, this, args);
buildBytes = (allocated() - loopBytes) / LOOP; buildMs = (performance.now() - loopMs) / LOOP;
} catch {} finally { depth -= 1; }
e.calibratedBytes = ((e.calibratedBytes ?? 0) * e.builds + buildBytes) / (e.builds + 1);
e.calibratedMs = ((e.calibratedMs ?? 0) * e.builds + buildMs) / (e.builds + 1);
} else { buildBytes = e.calibratedBytes ?? buildBytes; buildMs = e.calibratedMs ?? buildMs; }
e.builds += 1;
e.buildBytes += buildBytes;
e.buildMs += buildMs;
e.sqlBytes += Buffer.byteLength(prepared?.queryString ?? "");
e.sqlPrefix ??= String(prepared?.queryString ?? "").slice(0, 200);
e.fields = Array.isArray(prepared?.fields) ? prepared.fields.length : 0;
wrapPrepared(prepared, info);
return prepared;
};
}
function wrapFactory(proto: any, method: string, operation: string) {
const original = proto[method];
proto[method] = function qcostFactory(this: unknown, ...args: unknown[]) {
const result = Reflect.apply(original, this, args);
tag(result, operation, qcostFactory);
return result;
};
}
wrapFactory(PgSelectBuilder.prototype, "from", "select");
wrapFactory(PgInsertBuilder.prototype, "values", "insert");
wrapFactory(PgInsertBuilder.prototype, "select", "insert");
wrapFactory(PgUpdateBuilder.prototype, "set", "update");
wrapFactory(PgDatabase.prototype, "delete", "delete");
wrapFactory(RelationalQueryBuilder.prototype, "findFirst", "relational");
wrapFactory(RelationalQueryBuilder.prototype, "findMany", "relational");
wrapPrepare(PgSelectBase.prototype, "select");
wrapPrepare(PgInsertBase.prototype, "insert");
wrapPrepare(PgUpdateBase.prototype, "update");
wrapPrepare(PgDeleteBase.prototype, "delete");
wrapPrepare(PgRelationalQuery.prototype, "relational");
const originalExecute = (PgDatabase.prototype as any).execute;
(PgDatabase.prototype as any).execute = function qcostExecute(this: any, query: unknown) {
if (!enabled || depth > 0) return Reflect.apply(originalExecute, this, [query]);
const stack = callers(qcostExecute);
const info = { key: `execute ${stack[0] ?? "unknown"}`, operation: "execute", callers: stack };
depth += 1;
const startBytes = allocated(), startMs = performance.now();
let raw: any;
try { raw = Reflect.apply(originalExecute, this, [query]); }
finally { depth -= 1; }
const e = entry(info);
let bytes = allocated() - startBytes, ms = performance.now() - startMs;
if (e.builds < CALIBRATED) {
depth += 1;
try {
const sequel = typeof query === "string" ? null : (query as { getSQL(): any }).getSQL();
const loopBytes = allocated(), loopMs = performance.now();
for (let i = 0; i < LOOP; i++) if (sequel) this.dialect.sqlToQuery(sequel);
bytes = (allocated() - loopBytes) / LOOP; ms = (performance.now() - loopMs) / LOOP;
} catch {} finally { depth -= 1; }
e.calibratedBytes = ((e.calibratedBytes ?? 0) * e.builds + bytes) / (e.builds + 1);
e.calibratedMs = ((e.calibratedMs ?? 0) * e.builds + ms) / (e.builds + 1);
} else { bytes = e.calibratedBytes ?? bytes; ms = e.calibratedMs ?? ms; }
e.builds += 1;
e.buildBytes += bytes;
e.buildMs += ms;
e.sqlBytes += Buffer.byteLength(raw?.query?.sql ?? "");
e.sqlPrefix ??= String(raw?.query?.sql ?? "").slice(0, 200);
const execute = raw.execute;
raw.execute = function (this: unknown, ...args: unknown[]) {
if (enabled) e.executions += 1;
return Promise.resolve(Reflect.apply(execute, this, args)).then(value => { if (enabled) record(e, value); return value; });
};
return raw;
};
export function countStatements(db: { $client: { options: { debug?: unknown } } }) {
db.$client.options.debug = (_connection: number, text: string) => {
totals.statements += 1;
if (/^\s*(begin|commit|rollback|savepoint|release)\b/i.test(text)) totals.control += 1;
};
}
export function snapshot() {
return { statements: totals.statements, control: totals.control, allocated: allocated(), at: performance.now(),
entries: new Map([...entries].map(([key, value]) => [key, { ...value }])) };
}
export function diff(before: ReturnType<typeof snapshot>) {
const rows: Entry[] = [];
for (const [key, value] of entries) {
const prior = before.entries.get(key);
const d = { ...value } as Entry;
if (prior) for (const field of ["builds", "executions", "sqlBytes", "buildBytes", "buildMs", "readBytes", "readMs", "rows", "resultBytes"] as const) d[field] -= prior[field];
if (d.builds || d.executions) rows.push(d);
}
rows.sort((a, b) => (b.buildBytes + b.readBytes) - (a.buildBytes + a.readBytes));
return { statements: totals.statements - before.statements, control: totals.control - before.control,
allocated: allocated() - before.allocated, wallMs: performance.now() - before.at, rows };
}
export function format(label: string, units: number, result: ReturnType<typeof diff>, top = Number(process.env.TOP ?? 25)) {
const kb = (n: number) => (n / 1024 / units).toFixed(1);
const lines = [`## ${label} (per unit, ${units} units)`,
`statements ${(result.statements / units).toFixed(1)} (control ${(result.control / units).toFixed(1)}), total allocated ${kb(result.allocated)} KB, built ${kb(result.rows.reduce((s, r) => s + r.buildBytes, 0))} KB, read ${kb(result.rows.reduce((s, r) => s + r.readBytes, 0))} KB, wall ${(result.wallMs / units).toFixed(1)} ms`,
"| family | calls | builds | SQL B/build | built KB/call | read KB/call (model) | rows/call | result KB/call | fields | build ms/call |",
"| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |"];
for (const row of result.rows.slice(0, top)) {
const calls = Math.max(row.executions, row.builds);
lines.push(`| ${row.key}${row.callers[1] ? ` <- ${row.callers.slice(1).join(" <- ")}` : ""} | ${(calls / units).toFixed(2)} | ${(row.builds / units).toFixed(2)} | ${row.builds ? Math.round(row.sqlBytes / row.builds) : 0} | ${(row.buildBytes / 1024 / Math.max(1, row.builds)).toFixed(1)} | ${(row.readBytes / 1024 / Math.max(1, row.executions)).toFixed(1)} | ${(row.rows / Math.max(1, row.executions)).toFixed(1)} | ${(row.resultBytes / 1024 / Math.max(1, row.executions)).toFixed(1)} | ${row.fields} | ${(row.buildMs / Math.max(1, row.builds)).toFixed(3)} |`);
}
if (process.env.SQL_PREFIX) for (const row of result.rows.slice(0, top)) lines.push(`- ${row.key}: \`${row.sqlPrefix}\``);
return lines.join("\n");
}
+2 -2
View File
@@ -369,8 +369,8 @@ export function ActivityFeed({ className }: ActivityFeedProps) {
// Fetch issues for name resolution
const { data: issues } = useQuery({
queryKey: queryKeys.issues.list(selectedCompanyId ?? ""),
queryFn: () => issuesApi.list(selectedCompanyId!),
queryKey: queryKeys.issues.listIndex(selectedCompanyId ?? ""),
queryFn: ({ signal }) => issuesApi.listCompact(selectedCompanyId!, { projection: "index" }, { signal }),
enabled: !!selectedCompanyId,
});
@@ -0,0 +1,177 @@
// @vitest-environment node
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { QueryClient, QueryObserver } from "@tanstack/react-query";
import type { Issue } from "@paperclipai/shared";
import { __liveUpdatesTestUtils } from "./LiveUpdatesProvider";
import { createCoalescingQueryClient } from "../lib/query-invalidation-batcher";
import { queryKeys } from "../lib/queryKeys";
const COMPANY = "company-1";
const ACTOR = { userId: "user-1", agentId: null };
const INDEX_KEY = queryKeys.issues.listIndex(COMPANY);
const WINDOW_MS = __liveUpdatesTestUtils.TASK_INDEX_REFRESH_WINDOW_MS;
function task(id: string, patch: Partial<Issue> = {}): Issue {
return { id, identifier: id.toUpperCase(), title: `Task ${id}`, status: "todo", ...patch } as Issue;
}
function serverList(initial: Issue[]) {
let rows = initial;
const queryFn = vi.fn(async () => rows);
return { queryFn, set: (next: Issue[]) => { rows = next; } };
}
function mount(client: QueryClient, queryKey: readonly unknown[], queryFn: () => Promise<unknown>) {
const observer = new QueryObserver(client, { queryKey, queryFn, staleTime: 60_000 });
return observer.subscribe(() => {});
}
function emit(client: QueryClient, payload: Record<string, unknown>) {
__liveUpdatesTestUtils.invalidateActivityQueries(client, COMPANY, { entityType: "issue", actorType: "agent", actorId: "agent-1", ...payload }, ACTOR);
}
async function settle(ms = 0) {
await vi.advanceTimersByTimeAsync(ms);
}
describe("LiveUpdatesProvider company task index", () => {
let client: QueryClient;
beforeEach(() => {
vi.useFakeTimers();
client = new QueryClient();
});
afterEach(() => {
client.clear();
vi.useRealTimers();
});
it("refetches the index at most once per window during a burst of task activity", async () => {
const index = serverList([task("t1")]);
const other = serverList([]);
const unmountIndex = mount(client, INDEX_KEY, index.queryFn);
const unmountOther = mount(client, [...queryKeys.issues.list(COMPANY), "descendants", "t1"], other.queryFn);
await settle(WINDOW_MS);
expect(index.queryFn).toHaveBeenCalledTimes(1);
for (let second = 0; second < 60; second += 1) {
emit(client, { entityId: "t1", action: "issue.comment_added" });
emit(client, { entityId: "t1", action: "issue.updated", details: { status: "in_progress" } });
await settle(1_000);
}
await settle(WINDOW_MS);
expect(index.queryFn).toHaveBeenCalledTimes(4);
expect(other.queryFn.mock.calls.length).toBeGreaterThan(60);
unmountIndex();
unmountOther();
});
it("refetches at once for the first event after a quiet window", async () => {
const index = serverList([task("t1")]);
const unmount = mount(client, INDEX_KEY, index.queryFn);
await settle(WINDOW_MS);
emit(client, { entityId: "t1", action: "issue.updated", details: { title: "Renamed" } });
await settle();
expect(index.queryFn).toHaveBeenCalledTimes(2);
unmount();
});
it.each([
["a created task", { entityId: "t3", action: "issue.created", details: { identifier: "T3" } }, [task("t1"), task("t2"), task("t3")]],
["a deleted task", { entityId: "t2", action: "issue.deleted", details: {} }, [task("t1")]],
["a hidden task", { entityId: "t2", action: "issue.updated", details: { hiddenAt: "2026-11-13T00:00:00.000Z" } }, [task("t1")]],
["a status, title and priority change", { entityId: "t1", action: "issue.updated", details: { status: "done" } },
[task("t1", { status: "done", title: "Renamed", priority: "high" }), task("t2")]],
["an event for a task outside the cache", { entityId: "t9", action: "issue.comment_added" }, [task("t1"), task("t2")]],
["an event for a task the viewer cannot read", { entityId: "private-1", action: "issue.updated", details: { title: "Secret" } }, [task("t1"), task("t2")]],
])("shows the server's list after %s, inside one window, with nothing taken from the event", async (_name, event, next) => {
const initial = [task("t1"), task("t2")];
const index = serverList(initial);
const unmount = mount(client, INDEX_KEY, index.queryFn);
await settle();
emit(client, { entityId: "t1", action: "issue.comment_added" });
await settle(1_000);
index.set(next);
emit(client, event);
await settle();
expect(client.getQueryData(INDEX_KEY)).toEqual(initial);
await settle(WINDOW_MS);
expect(client.getQueryData(INDEX_KEY)).toEqual(next);
expect(JSON.stringify(client.getQueryData(INDEX_KEY))).not.toContain("Secret");
unmount();
});
it("refreshes at once for the viewer's own task activity, even inside a window", async () => {
const index = serverList([task("t1")]);
const unmount = mount(client, INDEX_KEY, index.queryFn);
await settle();
emit(client, { entityId: "t1", action: "issue.comment_added" });
await settle(1_000);
index.set([task("t1"), task("t2")]);
__liveUpdatesTestUtils.invalidateActivityQueries(client, COMPANY,
{ entityType: "issue", entityId: "t2", action: "issue.created", actorType: "user", actorId: "user-1", details: { identifier: "T2" } }, ACTOR);
await settle();
expect(client.getQueryData(INDEX_KEY)).toEqual([task("t1"), task("t2")]);
unmount();
});
it("refreshes at once when this tab changes a task and invalidates the company task lists", async () => {
const index = serverList([task("t1")]);
const unmount = mount(client, INDEX_KEY, index.queryFn);
await settle();
emit(client, { entityId: "t1", action: "issue.comment_added" });
await settle(1_000);
index.set([task("t1", { status: "done" })]);
await client.invalidateQueries({ queryKey: queryKeys.issues.list(COMPANY) });
await settle();
expect(client.getQueryData(INDEX_KEY)).toEqual([task("t1", { status: "done" })]);
unmount();
});
it("never loses an event inside the window: the last state shows when the window ends after activity stops", async () => {
const index = serverList([task("t1")]);
const unmount = mount(client, INDEX_KEY, index.queryFn);
await settle(WINDOW_MS);
for (let second = 0; second < 10; second += 1) {
index.set([task("t1", { title: `Edit ${second}` })]);
emit(client, { entityId: "t1", action: "issue.updated", details: { title: `Edit ${second}` } });
await settle(1_000);
}
expect(client.getQueryData(INDEX_KEY)).not.toEqual([task("t1", { title: "Edit 9" })]);
await settle(WINDOW_MS);
expect(client.getQueryData(INDEX_KEY)).toEqual([task("t1", { title: "Edit 9" })]);
unmount();
});
it("marks an unwatched index stale without fetching, and a later mount refetches it", async () => {
const index = serverList([task("t1")]);
const unmount = mount(client, INDEX_KEY, index.queryFn);
await settle();
unmount();
emit(client, { entityId: "t1", action: "issue.updated", details: { status: "done" } });
await settle(WINDOW_MS);
expect(index.queryFn).toHaveBeenCalledTimes(1);
expect(client.getQueryState(INDEX_KEY)?.isInvalidated).toBe(true);
const remount = mount(client, INDEX_KEY, index.queryFn);
await settle();
expect(index.queryFn).toHaveBeenCalledTimes(2);
remount();
});
it("keeps the window when events arrive through the live-event batcher", async () => {
const index = serverList([task("t1")]);
const unmount = mount(client, INDEX_KEY, index.queryFn);
await settle(WINDOW_MS);
const batched = createCoalescingQueryClient(client, __liveUpdatesTestUtils.createLiveInvalidationBatcher(client, COMPANY));
for (let second = 0; second < 20; second += 1) {
__liveUpdatesTestUtils.invalidateActivityQueries(batched, COMPANY,
{ entityType: "issue", entityId: "t1", action: "issue.comment_added", actorType: "agent", actorId: "agent-1" }, ACTOR);
await settle(1_000);
}
await settle(WINDOW_MS);
expect(index.queryFn.mock.calls.length).toBeLessThanOrEqual(3);
expect(index.queryFn.mock.calls.length).toBeGreaterThanOrEqual(2);
unmount();
});
});
+43 -6
View File
@@ -315,9 +315,8 @@ function resolveIssueQueryRefs(
const detailIssue = queryClient.getQueryData<Issue>(
queryKeys.issues.detail(issueId),
);
const listIssues = queryClient.getQueryData<Issue[]>(
queryKeys.issues.list(companyId),
);
const listIssues = queryClient.getQueryData<Issue[]>(queryKeys.issues.list(companyId))
?? queryClient.getQueryData<Issue[]>(queryKeys.issues.listIndex(companyId));
const detailsIdentifier =
readString(details?.identifier) ?? readString(details?.issueIdentifier);
@@ -375,8 +374,8 @@ function resolveIssueToastContext(
const detailIssue = issueRefs
.map((ref) => queryClient.getQueryData<Issue>(queryKeys.issues.detail(ref)))
.find((issue): issue is Issue => !!issue);
const listIssue = queryClient
.getQueryData<Issue[]>(queryKeys.issues.list(companyId))
const listIssue = (queryClient.getQueryData<Issue[]>(queryKeys.issues.list(companyId))
?? queryClient.getQueryData<Issue[]>(queryKeys.issues.listIndex(companyId)))
?.find((issue) =>
issueRefs.some((ref) => issue.id === ref || issue.identifier === ref),
);
@@ -1066,6 +1065,42 @@ function refreshMentionPoolForActivity(
}, wait);
}
const TASK_INDEX_REFRESH_WINDOW_MS = 30_000;
const taskIndexRefreshes = new WeakMap<
object,
Map<string, { lastRequestedAt: number; timer: ReturnType<typeof setTimeout> | null }>
>();
function refreshTaskIndexForActivity(queryClient: QueryClient, companyId: string, ownActivity: boolean) {
const queryKey = queryKeys.issues.listIndex(companyId);
if (queryClient.getQueryData(queryKey) === undefined) return;
if (ownActivity) {
queryClient.invalidateQueries({ queryKey, exact: true });
return;
}
const query = queryClient.getQueryCache().find({ queryKey, exact: true });
if (!query) return;
if (query.getObserversCount() === 0) {
queryClient.invalidateQueries({ queryKey, exact: true, refetchType: "none" });
return;
}
let refresh = taskIndexRefreshes.get(queryClient);
if (!refresh) {
refresh = new Map();
taskIndexRefreshes.set(queryClient, refresh);
}
const state = refresh.get(companyId) ?? { lastRequestedAt: 0, timer: null };
refresh.set(companyId, state);
if (state.timer !== null) return;
const lastRefresh = Math.max(query.state.dataUpdatedAt, state.lastRequestedAt);
const wait = Math.max(0, lastRefresh + TASK_INDEX_REFRESH_WINDOW_MS - Date.now());
state.timer = setTimeout(() => {
state.timer = null;
state.lastRequestedAt = Date.now();
queryClient.invalidateQueries({ queryKey, exact: true });
}, wait);
}
const ISSUE_DOCUMENT_ACTIVITY_ACTIONS = new Set([
"issue.document_created",
"issue.document_updated",
@@ -1644,8 +1679,9 @@ function invalidateActivityQueries(
}
queryClient.invalidateQueries({
queryKey: queryKeys.issues.list(companyId),
predicate: excludingQueryKeys(queryKeys.issues.mentionPool(companyId)),
predicate: excludingQueryKeys(queryKeys.issues.mentionPool(companyId), queryKeys.issues.listIndex(companyId)),
});
refreshTaskIndexForActivity(queryClient, companyId, ownActorActivity);
refreshMentionPoolForActivity(
queryClient,
companyId,
@@ -2320,6 +2356,7 @@ export const __liveUpdatesTestUtils = {
createLiveInvalidationBatcher,
isAmbientCompanyQueryKey,
MENTION_POOL_REFRESH_WINDOW_MS,
TASK_INDEX_REFRESH_WINDOW_MS,
buildActivityToast,
applyRunLifecycleToCompanyLiveRuns,
buildAgentStatusToast,
+1
View File
@@ -323,6 +323,7 @@ export const queryKeys = {
["issues", id, "recovery-actions", actionId, runId] as const,
allRecoveryActions: (id: string) => ["issues", id, "recovery-actions"] as const,
list: (companyId: string) => ["issues", companyId] as const,
listIndex: (companyId: string) => ["issues", companyId, "list-index"] as const,
onboardingFirstTask: (companyId: string) =>
["issues", companyId, "onboarding-first-task"] as const,
starPromptDone: (companyId: string) =>
+2 -2
View File
@@ -20,7 +20,7 @@ import { BoardChat } from "./BoardChat";
const mockAgentsApi = vi.hoisted(() => ({ list: vi.fn() }));
const mockGoalsApi = vi.hoisted(() => ({ list: vi.fn() }));
const mockIssuesApi = vi.hoisted(() => ({
list: vi.fn(),
listCompact: vi.fn(),
listComments: vi.fn(),
listFeedbackVotes: vi.fn(),
}));
@@ -127,7 +127,7 @@ describe("BoardChat staged typing intro", () => {
mockGoalsApi.list.mockResolvedValue([
{ id: "goal-1", title: "Build affordable robots", status: "active" },
]);
mockIssuesApi.list.mockResolvedValue([BOARD_ISSUE]);
mockIssuesApi.listCompact.mockResolvedValue([BOARD_ISSUE]);
mockIssuesApi.listComments.mockResolvedValue([]);
mockIssuesApi.listFeedbackVotes.mockResolvedValue([]);
});
+2 -2
View File
@@ -286,8 +286,8 @@ export function BoardChat() {
// Find or detect the board operations issue
const { data: issues } = useQuery({
queryKey: queryKeys.issues.list(selectedCompanyId!),
queryFn: () => issuesApi.list(selectedCompanyId!),
queryKey: queryKeys.issues.listIndex(selectedCompanyId!),
queryFn: ({ signal }) => issuesApi.listCompact(selectedCompanyId!, { projection: "index" }, { signal }),
enabled: !!selectedCompanyId,
});
+35 -2
View File
@@ -1,6 +1,6 @@
import type { Agent } from "@paperclipai/shared";
import type { Agent, CompactIssue, IssueSummary } from "@paperclipai/shared";
import { describe, expect, it } from "vitest";
import { derivePausedAgentBanner } from "./Dashboard";
import { derivePausedAgentBanner, recentTaskIndexRows, withTaskSummaries } from "./Dashboard";
function agent(overrides: Partial<Agent>): Agent {
return {
@@ -43,3 +43,36 @@ describe("derivePausedAgentBanner", () => {
expect(banner).toBeNull();
});
});
describe("dashboard recent tasks from the task index", () => {
const row = (id: string, updatedAt: string): CompactIssue =>
({ id, identifier: id.toUpperCase(), title: id, status: "blocked", updatedAt, assigneeAgentId: "agent-1" }) as unknown as CompactIssue;
it("keeps the ten most recently updated tasks, newest first", () => {
const rows = Array.from({ length: 14 }, (_, index) => row(`t${index}`, new Date(Date.UTC(2026, 10, 1 + index)).toISOString()));
expect(recentTaskIndexRows(rows.reverse()).map(issue => issue.id)).toEqual(
["t13", "t12", "t11", "t10", "t9", "t8", "t7", "t6", "t5", "t4"]);
});
it("reads each task's update time once, in one pass over a large company", () => {
let reads = 0;
const rows = Array.from({ length: 5_000 }, (_, index) => {
const at = new Date(Date.UTC(2026, 0, 1) + ((index * 7919) % 5_000) * 60_000).toISOString();
return Object.defineProperty({ id: `t${index}` }, "updatedAt", { get: () => { reads += 1; return at; } }) as unknown as CompactIssue;
});
const recent = recentTaskIndexRows(rows);
expect(reads).toBe(5_000);
const sorted = [...rows].sort((a, b) => Date.parse(String(b.updatedAt)) - Date.parse(String(a.updatedAt))).slice(0, 10);
expect(recent.map(issue => issue.id)).toEqual(sorted.map(issue => issue.id));
});
it("takes the status glyph fields from the task summary and the rest from the index row", () => {
const summary = { id: "t1", displayStatus: "in_queue", externalConversationState: "waiting",
blockerAttention: { state: "covered" } } as unknown as IssueSummary;
const [merged] = withTaskSummaries([row("t1", "2026-11-01T00:00:00.000Z")], [summary]);
expect(merged).toMatchObject({ id: "t1", title: "t1", assigneeAgentId: "agent-1", displayStatus: "in_queue",
externalConversationState: "waiting", blockerAttention: { state: "covered" } });
const [plain] = withTaskSummaries([row("t2", "2026-11-01T00:00:00.000Z")], undefined);
expect(plain).not.toHaveProperty("displayStatus");
});
});
+44 -7
View File
@@ -1,4 +1,5 @@
import { companyProjectsQuery } from "@/lib/company-projects-query";
import { useAccountIdentity } from "@/api/companies-query";
import { AgentIdentity } from "../components/AgentIdentity";
import { TaskSummaryList } from "../components/TaskSummaryList";
import { useDesignSandbox } from "../context/DesignSandboxContext";
@@ -37,18 +38,43 @@ import { QueryErrorState, useQueryView } from "../components/QueryView";
import { Card } from "@/components/ui/card";
import { Button } from "@/components/ui/button";
import { InlineBanner } from "../components/InlineBanner";
import type { Agent, Issue } from "@paperclipai/shared";
import type { Agent, CompactIssue, Issue, IssueSummary } from "@paperclipai/shared";
import { PluginSlotOutlet } from "@/plugins/slots";
import { SmokeLabDashboardCard } from "../components/SmokeLabDashboardCard";
import { taskStatusIconProps } from "@/lib/task-status-presentation";
const DASHBOARD_ACTIVITY_LIMIT = 10;
const DASHBOARD_RECENT_TASK_LIMIT = 10;
function getRecentIssues(issues: Issue[]): Issue[] {
return [...issues]
.sort((a, b) => new Date(b.updatedAt).getTime() - new Date(a.updatedAt).getTime());
export function recentTaskIndexRows(issues: readonly CompactIssue[]): CompactIssue[] {
const recent: Array<{ issue: CompactIssue; at: number }> = [];
for (const issue of issues) {
const at = new Date(issue.updatedAt).getTime();
if (recent.length === DASHBOARD_RECENT_TASK_LIMIT && at <= recent[recent.length - 1]!.at) continue;
let index = recent.length;
while (index > 0 && recent[index - 1]!.at < at) index -= 1;
recent.splice(index, 0, { issue, at });
if (recent.length > DASHBOARD_RECENT_TASK_LIMIT) recent.pop();
}
return recent.map(entry => entry.issue);
}
export function withTaskSummaries(rows: readonly CompactIssue[], summaries: readonly IssueSummary[] | undefined): Issue[] {
const byId = new Map((summaries ?? []).map((summary) => [summary.id, summary]));
return rows.map((row) => {
const summary = byId.get(row.id);
return {
...row,
...(summary ? {
displayStatus: summary.displayStatus,
externalConversationState: summary.externalConversationState,
...(summary.blockerAttention ? { blockerAttention: summary.blockerAttention } : {}),
} : {}),
} as Issue;
});
}
export type PausedAgentBanner =
| { kind: "imported"; pausedImportedAgentIds: string[] }
| { kind: "all-paused" }
@@ -196,10 +222,18 @@ export function Dashboard() {
usePublishSharedQueryData(sharedActivity, activity, activityUpdatedAt);
const { data: issues } = useQuery({
queryKey: queryKeys.issues.list(selectedCompanyId!),
queryFn: () => issuesApi.list(selectedCompanyId!),
queryKey: queryKeys.issues.listIndex(selectedCompanyId!),
queryFn: ({ signal }) => issuesApi.listCompact(selectedCompanyId!, { projection: "index" }, { signal }),
enabled: !!selectedCompanyId,
});
const { userId: accountUserId, settled: accountSettled } = useAccountIdentity();
const recentIndexRows = useMemo(() => recentTaskIndexRows(issues ?? []), [issues]);
const recentIds = useMemo(() => recentIndexRows.map((issue) => issue.id).sort(), [recentIndexRows]);
const { data: recentSummaries } = useQuery({
queryKey: queryKeys.issues.summaryBatch(selectedCompanyId!, accountUserId, recentIds),
queryFn: ({ signal }) => issuesApi.summaries(selectedCompanyId!, recentIds, { signal }),
enabled: !!selectedCompanyId && accountSettled && recentIds.length > 0,
});
const { data: projects } = useQuery({
...companyProjectsQuery(selectedCompanyId!, { includeArchived: true }),
@@ -217,7 +251,10 @@ export function Dashboard() {
[companyMembers?.users],
);
const recentIssues = issues ? getRecentIssues(issues) : [];
const recentIssues = useMemo(
() => withTaskSummaries(recentIndexRows, recentSummaries),
[recentIndexRows, recentSummaries],
);
const recentActivity = useMemo(() => (activity ?? []).slice(0, 10), [activity]);
useEffect(() => {