mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-28 12:04:41 +01:00
perf(server): return only sidebar failure badge candidates (#17935)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/95/changes/paperclip/95 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The sidebar badge shows failures that need attention. > - The service reads the newest 200 runs before it selects one run per agent. > - Most of those rows cannot contribute to the failure count. > - This pull request selects failure candidates in SQL and returns four required fields. > - The benefit is less database response data with the same badge and access rules. ## Linked Issues or Issue Description Refs: #12770. Refs: #17756. Refs: #17683. These are related load, inbox, and badge reports. This implementation was written independently from the current service. It does not change which issue states count as failures. **What existing behavior does this improve?** The run read used by the company sidebar badge. **Current behavior** Every call transfers up to 200 rows with eight scalar fields. The application then selects the latest run per agent and applies status, Mine, dismissal, and access rules. **Proposed behavior** SQL selects the same latest run per agent from the same 200-row window. It then applies status and Mine rules. It returns only the run ID, scope, issue ID, and creation time. The service still checks dismissals and current issue access. **Reason and benefit** A mixed synthetic fixture reduces the response from 30,689 to 1,297 value bytes. The complete badge response stays equal. **Breaking changes** None. The endpoint response and the 200-run window stay the same. ## What Changed - Keep company and active-agent filters before the existing 200-row limit. - Rank each agent by the full creation timestamp in descending order, then by the observation ordinal. The timestamp order explicitly selects the newest run. Exact ties retain the first observed row. No ID tie-break is added. - Apply failure and Mine filters after ranking. A shared agent's success cannot expose an older failure. - Return four fields for dismissal and fresh issue authorization. Supply the already constrained company ID in the service. - Add real PostgreSQL regression fixtures and a service-generated SQL comparison harness. Document the command in `doc/DEVELOPING.md`. ## Verification - `pnpm -r typecheck` passed. `pnpm build` then passed in a separate sequential step. The normal Runner verifier checked 1,509 inputs and 1,555 outputs before reuse. - `PAPERCLIP_SIDEBAR_TEST_DATABASE_URL=<dedicated-test-url> pnpm --filter @paperclipai/server exec vitest run src/__tests__/sidebar-badges.test.ts` passed all 14 tests on PostgreSQL 17.11. - The 10 unchanged `inbox-dismissals.test.ts` route tests passed against the same real PostgreSQL server through a temporary local adapter for database startup and cleanup. The adapter was removed. The standard host run passed the 13 new tests and skipped those 10 embedded-PostgreSQL tests. - The first focused attempt could not import the unbuilt plugin SDK. The normal build supplied that package before the passing reruns. - `pnpm test:run` was attempted. It recorded 16 embedded-PostgreSQL startup failures in the unrelated native-session executor suite. Other database suites also reported startup failures and skipped. The owned command was then stopped with exit 143. This is incomplete local full-suite evidence, not a pass. Full remote CI passed on final head `df5b2c81af43c282d031cc37e6bc867054217df5`: 54 successful checks, 2 neutral checks, and 2 optional Storybook skips; no pending or failed checks. Fresh exact-head Greptile review is 5/5 with zero unresolved threads. - A fixture rollback test reproduced the review finding before the fix: a failed final approval insert left 2 companies, 44 agents, 2 issues, and 657 runs. After the fix, the same PostgreSQL constraint error leaves zero owned rows in all five tables. The test also confirms that more than 600 runs were written before the deliberate failure. - Fixture seeding now uses one transaction, including the submillisecond SQL insert. The 14 focused tests and all 10 benchmark cases pass after this repair. The server typecheck also passes. Production SQL is unchanged. - The final ordering repair passed all 14 real-PostgreSQL tests, all 10 pinned-baseline benchmark cases, and affected server typecheck and build, run sequentially. The plans now explicitly sort by agent ID, full creation time descending, then observation ordinal. Complete badge responses and candidate bindings remain equal. - One server-test shard was interrupted by its CI runner receiving a shutdown signal. Its test step was canceled without an assertion failure. A single targeted retry passed; the aggregate `ci / verify` gate also passed. No source change was needed for that interruption. - `git diff --check` passed. No schema, workflow, or lockfile changed. Reproduce the benchmark with current schema migrations and a dedicated synthetic database whose name ends in `_test`: ```sh PAPERCLIP_SIDEBAR_TEST_DATABASE_URL=<dedicated-test-url> PAPERCLIP_LOG_LEVEL=error SIDEBAR_REQUIRE_BYTE_REDUCTION=1 pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-sidebar-badges.ts --baseline-ref=bf3ef12f3a78d0b15a32ad57f7e16520fbf6cd5e ``` The harness loads the baseline service from that commit. Its authorization, privacy, and shared helper dependencies are unchanged. It captures the SQL each service executes. It compares complete badge responses and every returned candidate field. Fixtures cover more than 200 runs, shared agents, superseding successes, exact and submillisecond ties, Mine user variants, dismissal equality, private and deleted issues, missing issue bindings, terminated agents, and cross-company rows. Tests also cover absent actors, grants, revocation, deletion after a prior read, and index versus sort plans. Synthetic local SQL results from PostgreSQL 17.11 on Docker, Node 24.22.1, and postgres.js 3.4.9 follow. Each timing is the median of seven alternating warm samples after two warmups. The report includes all samples, ranges, settings, SQL, parameters, and `EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON)` plans. | Fixture | Rows before → after | Value bytes before → after | Reduction | Median ms before → after | | --- | --- | --- | --- | --- | | Mixed, company-wide | 200 → 18 | 30,689 → 1,297 | 95.8% | 1.361 → 1.305 | | Mixed, user-1 | 200 → 9 | 30,689 → 676 | 97.8% | 1.785 → 1.391 | | Mixed, local-board | 200 → 6 | 30,689 → 414 | 98.7% | 1.463 → 1.232 | | All failed, 200 distinct latest agents | 200 → 200 | 30,298 → 13,778 | 54.5% | 1.277 → 1.237 | | All succeeded | 200 → 0 | 30,898 → 0 | 100% | 1.095 → 0.898 | | Empty | 0 → 0 | 0 → 0 | Neutral | 0.703 → 1.002 | Value bytes mean UTF-8 non-null cell values before Drizzle decoding. They exclude PostgreSQL protocol framing and are not wire bytes. The primary claim is less response data. Warm timing ranges overlap. Earlier all-failed samples were slower, and the empty control remains slower. SQL adds ranking work. No production latency improvement is claimed. The plans keep the limit below both window operations. ## Risks - SQL adds two bounded window operations. The all-failed control transfers fewer bytes but can take more SQL time. - Per-agent ranking explicitly orders by full creation time descending. Exact timestamp ties retain the observation ordinal. Real PostgreSQL tests compare complete candidates with the original helper under index and sort plans. - Mine rules now also have a SQL form. Tests cover undefined, null, empty, named, and local-board users, including empty responsible-user IDs. - Issue access stays fresh and fail closed. No authorization result is cached. No migration or stored run change is required. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-6 through Codex. The exact serving model ID and context window are not exposed in this session. The model used reasoning, code execution, SQL inspection, and GitHub tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed
1 parent
dc8cbdb819
commit
09d4ca533e
5 files changed
+443
-9
No files matched your search
@@ -3382,6 +3382,35 @@ pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-workspace-prepar
|
||||
It compares actual preparation guards with synthetic history. Local timings are
|
||||
separate from Git materialization, environment acquisition, and provider startup.
|
||||
|
||||
### Sidebar badge run-query benchmark
|
||||
|
||||
The badge query keeps the newest 200 runs of nonterminated same-company agents.
|
||||
It selects each agent's first observed run before status or Mine filters. An
|
||||
observation ordinal preserves equal-time input order. Per-agent ranking orders
|
||||
by full creation time first, then that ordinal for exact ties. SQL returns only failure
|
||||
IDs, issue bindings, and creation times. Dismissals and fresh issue authorization
|
||||
still run in the service.
|
||||
|
||||
Run the synthetic PostgreSQL comparison with a pinned pre-change revision:
|
||||
|
||||
```sh
|
||||
PAPERCLIP_LOG_LEVEL=error SIDEBAR_REQUIRE_BYTE_REDUCTION=1 pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-sidebar-badges.ts --baseline-ref=bf3ef12f3a78d0b15a32ad57f7e16520fbf6cd5e
|
||||
```
|
||||
|
||||
The harness starts disposable embedded PostgreSQL by default. To use an existing
|
||||
isolated test database with current migrations, set
|
||||
`PAPERCLIP_SIDEBAR_TEST_DATABASE_URL` to a dedicated database whose name ends in
|
||||
`_test`. The harness removes only its own synthetic company fixtures. Do not use
|
||||
a development or hosted company database.
|
||||
|
||||
The report includes service-generated SQL, complete badge equality, candidate
|
||||
bindings, alternating warm samples, plans, and UTF-8 response-value bytes before
|
||||
Drizzle decoding. This byte count excludes protocol framing. The mixed fixture
|
||||
covers shared agents, superseding successes, timestamp ties, Mine users,
|
||||
dismissal equality, private and deleted issues, and more than 200 runs. The
|
||||
all-failed, all-succeeded, and empty controls expose neutral or slower paths.
|
||||
Timings describe local SQL work and do not establish hosted request latency.
|
||||
|
||||
### Budget accounting performance
|
||||
|
||||
Budget admission and settlement share the company accounting lock. They evaluate
|
||||
|
||||
@@ -0,0 +1,104 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { promises as fs } from "node:fs";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath, pathToFileURL } from "node:url";
|
||||
import { createDb, startEmbeddedPostgresTestDatabase, type Db } from "../packages/db/src/index.js";
|
||||
import { sidebarBadgeService } from "../server/src/services/sidebar-badges.js";
|
||||
import { captureSidebarQueries, cleanupSidebarFixture, observedSidebarFailures, seedSidebarFixture, type CapturedQuery } from "../server/src/__tests__/helpers/sidebar-badges-fixture.js";
|
||||
|
||||
const baselineRef = process.argv.find(arg => arg.startsWith("--baseline-ref="))?.slice(15);
|
||||
assert.ok(baselineRef, "Pass --baseline-ref=<pre-fix commit>");
|
||||
const baselineRevision = execFileSync("git", ["rev-parse", "--verify", `${baselineRef}^{commit}`], { encoding: "utf8" }).trim();
|
||||
const treatmentRevision = execFileSync("git", ["rev-parse", "HEAD"], { encoding: "utf8" }).trim();
|
||||
const externalUrl = process.env.PAPERCLIP_SIDEBAR_TEST_DATABASE_URL;
|
||||
assert.ok(!externalUrl || new URL(externalUrl).pathname.endsWith("_test"), "Use a dedicated synthetic *_test database");
|
||||
const keepAlive = setInterval(() => {}, 1000);
|
||||
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | undefined;
|
||||
let baselineDir: string | undefined;
|
||||
let db: Db | undefined;
|
||||
const fixtures: Awaited<ReturnType<typeof seedSidebarFixture>>[] = [];
|
||||
let failure: unknown;
|
||||
try {
|
||||
if (!externalUrl) database = await startEmbeddedPostgresTestDatabase("sidebar-benchmark-");
|
||||
db = createDb(externalUrl ?? database!.connectionString);
|
||||
const modulePath = "server/src/services/sidebar-badges.ts";
|
||||
const moduleUrl = new URL(`../${modulePath}`, import.meta.url);
|
||||
const source = execFileSync("git", ["show", `${baselineRevision}:${modulePath}`], { encoding: "utf8" });
|
||||
const rewritten = source.replace(/from "(\.{1,2}\/[^\"]+)"/g,
|
||||
(_match, specifier: string) => `from ${JSON.stringify(fileURLToPath(new URL(specifier, moduleUrl)))}`);
|
||||
baselineDir = await fs.mkdtemp(path.join(fileURLToPath(new URL("../server/scripts/", import.meta.url)), ".sidebar-baseline-"));
|
||||
const baselineFile = path.join(baselineDir, "sidebar-badges.ts");
|
||||
await fs.writeFile(baselineFile, rewritten);
|
||||
const baseline = await import(pathToFileURL(baselineFile).href) as typeof import("../server/src/services/sidebar-badges.js");
|
||||
const [{ version }] = await db.$client.unsafe("select version()");
|
||||
const settings = await db.$client.unsafe("select name, setting from pg_settings where name in ('work_mem', 'shared_buffers', 'jit', 'max_parallel_workers_per_gather')");
|
||||
const results = [];
|
||||
for (const mode of ["mixed", "all-failed", "all-succeeded", "empty"] as const) {
|
||||
const fixture = await seedSidebarFixture(db, mode);
|
||||
fixtures.push(fixture);
|
||||
await db.$client.unsafe("ANALYZE heartbeat_runs");
|
||||
await db.$client.unsafe("ANALYZE agents");
|
||||
const users = mode === "mixed" ? [undefined, null, "", "user-1", "user-2", "local-board", "nobody"] : [undefined];
|
||||
for (const currentUserId of users) {
|
||||
const queries = { baseline: [] as CapturedQuery[], treatment: [] as CapturedQuery[] };
|
||||
const before = baseline.sidebarBadgeService(captureSidebarQueries(db, queries.baseline));
|
||||
const after = sidebarBadgeService(captureSidebarQueries(db, queries.treatment));
|
||||
const options = { ...fixture.options, currentUserId };
|
||||
const baselineResponse = await before.get(fixture.companyId, options);
|
||||
const treatmentResponse = await after.get(fixture.companyId, options);
|
||||
assert.deepEqual(treatmentResponse, baselineResponse);
|
||||
assert.equal(queries.baseline.length, 1);
|
||||
assert.equal(queries.treatment.length, 1);
|
||||
const expected = await observedSidebarFailures(db, fixture.companyId, currentUserId);
|
||||
const normalize = (rows: unknown[][]) => rows.map(row => JSON.stringify(row)).sort();
|
||||
const baselineQuery = queries.baseline[0]!, treatmentQuery = queries.treatment[0]!;
|
||||
const treatmentRows = await db.$client.unsafe(treatmentQuery.sql, treatmentQuery.params as never[]).values();
|
||||
if (treatmentQuery.sql !== baselineQuery.sql) {
|
||||
assert.deepEqual(normalize(treatmentRows.map(row => [...row.slice(0, 3), new Date(String(row[3])).toISOString()])), normalize(expected.map(row => [row.id, row.scopeKind, row.issueId, row.createdAt.toISOString()])));
|
||||
}
|
||||
const samples = { baseline: [] as number[], treatment: [] as number[] };
|
||||
const rowsByVersion: Record<string, unknown[][]> = {};
|
||||
for (let iteration = 0; iteration < 9; iteration++) {
|
||||
for (const label of iteration % 2 ? ["treatment", "baseline"] as const : ["baseline", "treatment"] as const) {
|
||||
const query = label === "baseline" ? baselineQuery : treatmentQuery;
|
||||
const start = performance.now();
|
||||
rowsByVersion[label] = await db.$client.unsafe(query.sql, query.params as never[]).values();
|
||||
if (iteration >= 2) samples[label].push(performance.now() - start);
|
||||
}
|
||||
}
|
||||
const measurements = [];
|
||||
for (const label of ["baseline", "treatment"] as const) {
|
||||
const query = label === "baseline" ? baselineQuery : treatmentQuery;
|
||||
const rows = rowsByVersion[label]!;
|
||||
const responseValueBytes = rows.flat().reduce<number>((sum, value) => sum + (value == null ? 0 : Buffer.byteLength(typeof value === "string" ? value : String(value))), 0);
|
||||
const [plan] = await db.$client.unsafe(`EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON) ${query.sql}`, query.params as never[]);
|
||||
const sorted = [...samples[label]].sort((a, b) => a - b);
|
||||
measurements.push({ label, query, rows: rows.length, responseValueBytes, samplesMs: samples[label], medianMs: sorted[3], minMs: sorted[0], maxMs: sorted[6], plan: plan!["QUERY PLAN"] });
|
||||
}
|
||||
if (process.env.SIDEBAR_REQUIRE_BYTE_REDUCTION === "1" && mode !== "empty") {
|
||||
assert.ok(measurements[1]!.responseValueBytes < measurements[0]!.responseValueBytes);
|
||||
}
|
||||
results.push({ mode, currentUserId: currentUserId === undefined ? "<undefined>" : currentUserId, equalBadgeResponse: treatmentResponse, equalCandidateRows: expected.length, measurements });
|
||||
}
|
||||
}
|
||||
console.log(JSON.stringify({ baselineRevision, treatmentRevision,
|
||||
treatmentDiff: execFileSync("git", ["diff", "--", modulePath], { encoding: "utf8" }),
|
||||
dependencyPolicy: "Baseline imports unchanged current authorization, privacy, and shared helpers", version, settings,
|
||||
acceptance: "Identical complete badges and candidate bindings; fewer response-value bytes for nonempty fixtures. Timings are descriptive with no latency threshold.",
|
||||
byteBoundary: "UTF-8 non-null cell values before Drizzle decoding, excluding PostgreSQL protocol framing; not wire bytes",
|
||||
results }, null, 2));
|
||||
} catch (error) {
|
||||
failure = error;
|
||||
} finally {
|
||||
const errors: unknown[] = failure === undefined ? [] : [failure];
|
||||
async function cleanup(action: () => Promise<unknown> | undefined) {
|
||||
try { await action(); } catch (error) { errors.push(error); }
|
||||
}
|
||||
if (db) for (const fixture of fixtures) await cleanup(() => cleanupSidebarFixture(db!, fixture));
|
||||
await cleanup(() => db?.$client.end({ timeout: 0 }));
|
||||
await cleanup(() => database?.cleanup());
|
||||
await cleanup(() => baselineDir ? fs.rm(baselineDir, { recursive: true, force: true }) : undefined);
|
||||
clearInterval(keepAlive);
|
||||
if (errors.length) throw new AggregateError(errors, "Sidebar benchmark or cleanup failed");
|
||||
}
|
||||
@@ -0,0 +1,127 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { and, desc, eq, inArray, not, sql } from "drizzle-orm";
|
||||
import { agents, approvals, companies, heartbeatRuns, issues, type Db } from "@paperclipai/db";
|
||||
import { isHeartbeatRunVisibleInMine, latestFailedRunsByAgent } from "@paperclipai/shared";
|
||||
import type { sidebarBadgeService } from "../../services/sidebar-badges.js";
|
||||
|
||||
export type BadgeOptions = Parameters<ReturnType<typeof sidebarBadgeService>["get"]>[1];
|
||||
export type CapturedQuery = { sql: string; params: unknown[] };
|
||||
|
||||
export function captureSidebarQueries(db: Db, queries: CapturedQuery[]): Db {
|
||||
return new Proxy(db, { get(target, key) {
|
||||
if (key !== "select") return Reflect.get(target, key);
|
||||
return (...fields: Parameters<Db["select"]>) => {
|
||||
const builder = target.select(...fields), from = builder.from.bind(builder);
|
||||
builder.from = ((...args: Parameters<typeof from>) => {
|
||||
const selection = from(...args), execute = selection.execute.bind(selection);
|
||||
selection.execute = async () => {
|
||||
const query = selection.toSQL();
|
||||
if (query.sql.includes('"heartbeat_runs"')) queries.push(query);
|
||||
return execute();
|
||||
};
|
||||
return selection;
|
||||
}) as typeof builder.from;
|
||||
return builder;
|
||||
};
|
||||
}});
|
||||
}
|
||||
|
||||
export async function observedSidebarFailures(db: Db, companyId: string, currentUserId?: string | null) {
|
||||
const rows = await db.select({
|
||||
id: heartbeatRuns.id, agentId: heartbeatRuns.agentId, companyId: heartbeatRuns.companyId,
|
||||
scopeKind: heartbeatRuns.scopeKind, issueId: heartbeatRuns.issueId, status: heartbeatRuns.status,
|
||||
responsibleUserId: heartbeatRuns.responsibleUserId, createdAt: heartbeatRuns.createdAt,
|
||||
}).from(heartbeatRuns).innerJoin(agents, eq(heartbeatRuns.agentId, agents.id)).where(and(
|
||||
eq(heartbeatRuns.companyId, companyId), eq(agents.companyId, companyId), not(eq(agents.status, "terminated")),
|
||||
)).orderBy(desc(heartbeatRuns.createdAt)).limit(200);
|
||||
return latestFailedRunsByAgent(rows).filter(row => currentUserId === undefined || isHeartbeatRunVisibleInMine(row, currentUserId));
|
||||
}
|
||||
|
||||
export async function seedSidebarFixture(db: Db, mode: "mixed" | "all-failed" | "all-succeeded" | "empty" = "mixed") {
|
||||
return db.transaction(async tx => {
|
||||
const companyId = randomUUID(), foreignCompanyId = randomUUID();
|
||||
await tx.insert(companies).values([
|
||||
{ id: companyId, name: "Synthetic sidebar", issuePrefix: `B${companyId.slice(0, 7)}` },
|
||||
{ id: foreignCompanyId, name: "Foreign sidebar", issuePrefix: `F${foreignCompanyId.slice(0, 7)}` },
|
||||
]);
|
||||
const actorId = randomUUID();
|
||||
await tx.insert(agents).values({ id: actorId, companyId, name: "Reader", role: "engineer" });
|
||||
const actor = { type: "agent" as const, agentId: actorId, companyId };
|
||||
const privateIssueId = randomUUID(), publicIssueId = randomUUID(), deletedIssueId = randomUUID();
|
||||
await tx.insert(issues).values([
|
||||
{ id: privateIssueId, companyId, title: "Private", visibility: "private", privacyRootIssueId: privateIssueId, responsibleUserId: "other-user" },
|
||||
{ id: publicIssueId, companyId, title: "Public" },
|
||||
{ id: deletedIssueId, companyId, title: "Deleted" },
|
||||
]);
|
||||
const origin = Date.parse("2026-10-01T00:00:00.000Z");
|
||||
const agentIds = Array.from({ length: mode === "all-failed" ? 240 : 30 }, () => randomUUID());
|
||||
await tx.insert(agents).values(agentIds.map((id, index) => ({ id, companyId, name: `Agent ${index}`, role: "engineer" })));
|
||||
const runs: (typeof heartbeatRuns.$inferInsert)[] = [];
|
||||
if (mode !== "empty") {
|
||||
for (let index = 0; index < 420; index++) {
|
||||
const agentIndex = index % agentIds.length;
|
||||
runs.push({ id: randomUUID(), companyId, agentId: agentIds[agentIndex]!,
|
||||
status: mode === "all-failed" ? "failed" : mode === "all-succeeded" ? "succeeded" : ["failed", "timed_out", "succeeded", "running", "queued", "cancelled"][agentIndex % 6],
|
||||
responsibleUserId: ["user-1", "user-2", null, "", "local-board"][agentIndex % 5],
|
||||
createdAt: new Date(origin + index), invocationSource: "manual" });
|
||||
}
|
||||
}
|
||||
const special: Record<string, string> = {};
|
||||
if (mode === "mixed") {
|
||||
async function add(name: string, values: Partial<typeof heartbeatRuns.$inferInsert> = {}) {
|
||||
const agentId = randomUUID(), id = randomUUID();
|
||||
await tx.insert(agents).values({ id: agentId, companyId, name, role: "engineer" });
|
||||
runs.push({ id, companyId, agentId, status: "failed", responsibleUserId: "user-1", createdAt: new Date(origin + 1000 + runs.length), ...values });
|
||||
special[name] = id;
|
||||
return agentId;
|
||||
}
|
||||
await add("outside-window", { createdAt: new Date(origin - 1000) });
|
||||
const shared = await add("superseded");
|
||||
runs.push({ companyId, agentId: shared, status: "succeeded", responsibleUserId: "user-2", createdAt: new Date(origin + 2000) });
|
||||
const foreignOwner = await add("other-user-latest");
|
||||
runs.push({ companyId, agentId: foreignOwner, status: "failed", responsibleUserId: "user-2", createdAt: new Date(origin + 2001) });
|
||||
await add("private", { scopeKind: "issue", issueId: privateIssueId });
|
||||
await add("public", { scopeKind: "issue", issueId: publicIssueId });
|
||||
await add("deleted", { scopeKind: "issue", issueId: deletedIssueId });
|
||||
await add("missing", { scopeKind: "issue", issueId: null });
|
||||
await add("dismissed");
|
||||
for (const [name, status] of [["tied-failure-first", "failed"], ["tied-success-first", "succeeded"]] as const) {
|
||||
const agentId = await add(name, { status, createdAt: new Date(origin + 3000) });
|
||||
runs.push({ companyId, agentId, status: status === "failed" ? "succeeded" : "failed", responsibleUserId: "user-1", createdAt: new Date(origin + 3000) });
|
||||
}
|
||||
const terminated = randomUUID(), crossCompanyAgent = randomUUID();
|
||||
await tx.insert(agents).values([
|
||||
{ id: terminated, companyId, name: "Terminated", status: "terminated" },
|
||||
{ id: crossCompanyAgent, companyId: foreignCompanyId, name: "Foreign" },
|
||||
]);
|
||||
for (let index = 0; index < 220; index++) runs.push({ companyId, agentId: index % 2 ? terminated : crossCompanyAgent,
|
||||
status: "failed", responsibleUserId: "user-1", createdAt: new Date(origin + 10000 + index) });
|
||||
runs.push({ companyId: foreignCompanyId, agentId: crossCompanyAgent, status: "failed", createdAt: new Date(origin + 20000) });
|
||||
}
|
||||
if (runs.length) await tx.insert(heartbeatRuns).values(runs);
|
||||
if (mode === "mixed") {
|
||||
await tx.delete(issues).where(eq(issues.id, deletedIssueId));
|
||||
const microAgent = randomUUID();
|
||||
await tx.insert(agents).values({ id: microAgent, companyId, name: "Submillisecond" });
|
||||
await tx.execute(sql`insert into heartbeat_runs (company_id, agent_id, status, responsible_user_id, created_at) values
|
||||
(${companyId}, ${microAgent}, 'failed', 'user-1', '2026-10-01 00:00:04.000100+00'),
|
||||
(${companyId}, ${microAgent}, 'succeeded', 'user-2', '2026-10-01 00:00:04.000900+00')`);
|
||||
}
|
||||
const approvalId = randomUUID();
|
||||
await tx.insert(approvals).values({ id: approvalId, companyId, type: "hire_agent", status: "pending", payload: {}, updatedAt: new Date(origin) });
|
||||
const dismissals = new Map<string, number>([[`approval:${approvalId}`, origin]]);
|
||||
const dismissed = runs.find(row => row.id === special.dismissed);
|
||||
if (dismissed) dismissals.set(`run:${dismissed.id}`, (dismissed.createdAt as Date).getTime());
|
||||
return { companyId, foreignCompanyId, actor, actorId, privateIssueId, publicIssueId, special, dismissals,
|
||||
options: { dismissals, joinRequests: [{ id: "join", createdAt: new Date(origin), updatedAt: null }], unreadTouchedIssues: 2, actor } satisfies BadgeOptions };
|
||||
});
|
||||
}
|
||||
|
||||
export async function cleanupSidebarFixture(db: Db, fixture: Awaited<ReturnType<typeof seedSidebarFixture>>) {
|
||||
const companyIds = [fixture.companyId, fixture.foreignCompanyId];
|
||||
await db.delete(heartbeatRuns).where(inArray(heartbeatRuns.companyId, companyIds));
|
||||
await db.delete(approvals).where(inArray(approvals.companyId, companyIds));
|
||||
await db.delete(issues).where(inArray(issues.companyId, companyIds));
|
||||
await db.delete(agents).where(inArray(agents.companyId, companyIds));
|
||||
await db.delete(companies).where(inArray(companies.id, companyIds));
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||
import { count, eq, inArray } from "drizzle-orm";
|
||||
import { agents, approvals, companies, createDb, getEmbeddedPostgresTestSupport, heartbeatRuns, issueAccessGrants, issues, startEmbeddedPostgresTestDatabase, type Db } from "@paperclipai/db";
|
||||
import { sidebarBadgeService } from "../services/sidebar-badges.js";
|
||||
import { authorizationService } from "../services/authorization.js";
|
||||
import { getHeartbeatRunReadDecisions } from "../services/heartbeat-run-privacy.js";
|
||||
import { captureSidebarQueries, cleanupSidebarFixture, observedSidebarFailures, seedSidebarFixture, type BadgeOptions, type CapturedQuery } from "./helpers/sidebar-badges-fixture.js";
|
||||
|
||||
const externalUrl = process.env.PAPERCLIP_SIDEBAR_TEST_DATABASE_URL;
|
||||
const support = externalUrl ? { supported: true } : await getEmbeddedPostgresTestSupport();
|
||||
const describePostgres = support.supported ? describe : describe.skip;
|
||||
if (!support.supported) console.warn(`Skipping sidebar badge SQL tests: ${support.reason}`);
|
||||
|
||||
describePostgres("sidebar badge failure candidates", () => {
|
||||
let db: ReturnType<typeof createDb>;
|
||||
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | undefined;
|
||||
const fixtures: Awaited<ReturnType<typeof seedSidebarFixture>>[] = [];
|
||||
let fixture: Awaited<ReturnType<typeof seedSidebarFixture>>;
|
||||
|
||||
beforeAll(async () => {
|
||||
if (externalUrl && !new URL(externalUrl).pathname.endsWith("_test")) throw new Error("Use a dedicated synthetic *_test database");
|
||||
if (!externalUrl) database = await startEmbeddedPostgresTestDatabase("sidebar-badges-");
|
||||
db = createDb(externalUrl ?? database!.connectionString, { maxConnections: 1 });
|
||||
fixture = await seedSidebarFixture(db);
|
||||
fixtures.push(fixture);
|
||||
}, 90_000);
|
||||
|
||||
afterAll(async () => {
|
||||
const errors: unknown[] = [];
|
||||
async function cleanup(action: () => Promise<unknown> | undefined) {
|
||||
try { await action(); } catch (error) { errors.push(error); }
|
||||
}
|
||||
for (const entry of fixtures) await cleanup(() => cleanupSidebarFixture(db, entry));
|
||||
await cleanup(() => db?.$client.end({ timeout: 0 }));
|
||||
await cleanup(() => database?.cleanup());
|
||||
if (errors.length) throw new AggregateError(errors, "Sidebar test cleanup failed");
|
||||
});
|
||||
|
||||
async function verify(options?: BadgeOptions) {
|
||||
const queries: CapturedQuery[] = [];
|
||||
const actual = await sidebarBadgeService(captureSidebarQueries(db, queries)).get(fixture.companyId, options);
|
||||
const expectedCandidates = await observedSidebarFailures(db, fixture.companyId, options?.currentUserId);
|
||||
const candidates = await db.$client.unsafe(queries[0]!.sql, queries[0]!.params as never[]).values();
|
||||
const normalize = (rows: unknown[][]) => rows.map(row => JSON.stringify(row)).sort();
|
||||
expect(normalize(candidates.map(row => [...row.slice(0, 3), new Date(String(row[3])).toISOString()]))).toEqual(normalize(
|
||||
expectedCandidates.map(row => [row.id, row.scopeKind, row.issueId, row.createdAt.toISOString()]),
|
||||
));
|
||||
expect(expectedCandidates.map(row => row.id)).not.toContain(fixture.special.superseded);
|
||||
expect(expectedCandidates.map(row => row.id)).not.toContain(fixture.special["outside-window"]);
|
||||
const undismissed = expectedCandidates.filter(row => (options?.dismissals?.get(`run:${row.id}`) ?? -Infinity) < row.createdAt.getTime());
|
||||
const failedRuns = options?.actor
|
||||
? (await getHeartbeatRunReadDecisions(db, authorizationService(db), options.actor, undismissed)).filter(Boolean).length
|
||||
: undismissed.length;
|
||||
const approvals = options?.dismissals ? 0 : 1;
|
||||
const joinRequests = options?.joinRequests?.length ?? 0;
|
||||
expect(actual).toEqual({ inbox: approvals + failedRuns + joinRequests + (options?.unreadTouchedIssues ?? 0), approvals, failedRuns, joinRequests });
|
||||
return actual;
|
||||
}
|
||||
|
||||
it("rolls back all fixture rows when the final approval insert fails", async () => {
|
||||
let companyIds: string[] = [];
|
||||
let writtenRuns = 0;
|
||||
function failLastInsert<T extends Pick<Db, "insert" | "select">>(connection: T): T {
|
||||
return new Proxy(connection, { get(target, key) {
|
||||
if (key !== "insert") return Reflect.get(target, key);
|
||||
return (table: Parameters<Db["insert"]>[0]) => {
|
||||
if (table === companies) return { values: (values: (typeof companies.$inferInsert)[]) => {
|
||||
companyIds = values.map(row => row.id!);
|
||||
fixtures.push({ ...fixture, companyId: companyIds[0]!, foreignCompanyId: companyIds[1]! });
|
||||
return target.insert(companies).values(values);
|
||||
}};
|
||||
if (table === approvals) return { values: async (values: typeof approvals.$inferInsert) => {
|
||||
const [row] = await target.select({ value: count() }).from(heartbeatRuns).where(inArray(heartbeatRuns.companyId, companyIds));
|
||||
writtenRuns = row!.value;
|
||||
return target.insert(approvals).values({ ...values, payload: null as never });
|
||||
}};
|
||||
return target.insert(table);
|
||||
};
|
||||
}});
|
||||
}
|
||||
const failingDb = new Proxy(failLastInsert(db), { get(target, key) {
|
||||
if (key !== "transaction") return Reflect.get(target, key);
|
||||
return (callback: Parameters<Db["transaction"]>[0]) => db.transaction(tx => callback(failLastInsert(tx)));
|
||||
}});
|
||||
await expect(seedSidebarFixture(failingDb)).rejects.toMatchObject({ cause: { code: "23502" } });
|
||||
expect(companyIds).toHaveLength(2);
|
||||
expect(writtenRuns).toBeGreaterThan(600);
|
||||
const rows = await Promise.all([
|
||||
db.select({ value: count() }).from(companies).where(inArray(companies.id, companyIds)),
|
||||
db.select({ value: count() }).from(agents).where(inArray(agents.companyId, companyIds)),
|
||||
db.select({ value: count() }).from(issues).where(inArray(issues.companyId, companyIds)),
|
||||
db.select({ value: count() }).from(heartbeatRuns).where(inArray(heartbeatRuns.companyId, companyIds)),
|
||||
db.select({ value: count() }).from(approvals).where(inArray(approvals.companyId, companyIds)),
|
||||
]);
|
||||
expect(rows.map(([row]) => row!.value)).toEqual([0, 0, 0, 0, 0]);
|
||||
});
|
||||
|
||||
it.each([undefined, null, "", "user-1", "user-2", "local-board", "nobody"])("keeps the bounded latest run before Mine filtering for %s", async currentUserId => {
|
||||
await verify({ ...fixture.options, currentUserId });
|
||||
});
|
||||
|
||||
it("retains company-wide behavior without an actor or options", async () => {
|
||||
await verify();
|
||||
await verify({ ...fixture.options, actor: undefined });
|
||||
await verify({ ...fixture.options, actor: { type: "none" } });
|
||||
});
|
||||
|
||||
it("rechecks private issue grants and deletion on every request", async () => {
|
||||
const service = sidebarBadgeService(db);
|
||||
const options = { ...fixture.options, currentUserId: "user-1" };
|
||||
const denied = await service.get(fixture.companyId, options);
|
||||
const [grant] = await db.insert(issueAccessGrants).values({ issueId: fixture.privateIssueId, subjectType: "agent", subjectId: fixture.actorId, source: "explicit" }).returning();
|
||||
const allowed = await service.get(fixture.companyId, options);
|
||||
expect(allowed.failedRuns).toBe(denied.failedRuns + 1);
|
||||
await db.update(issueAccessGrants).set({ revokedAt: new Date() }).where(eq(issueAccessGrants.id, grant!.id));
|
||||
expect(await service.get(fixture.companyId, options)).toEqual(denied);
|
||||
await db.delete(issues).where(eq(issues.id, fixture.publicIssueId));
|
||||
expect((await service.get(fixture.companyId, options)).failedRuns).toBe(denied.failedRuns - 1);
|
||||
await verify(options);
|
||||
});
|
||||
|
||||
it("preserves equal-time input order with index and sort plans", async () => {
|
||||
await db.$client.unsafe("SET enable_indexscan = off");
|
||||
await db.$client.unsafe("SET enable_bitmapscan = off");
|
||||
try { await verify(fixture.options); }
|
||||
finally {
|
||||
await db.$client.unsafe("RESET enable_indexscan");
|
||||
await db.$client.unsafe("RESET enable_bitmapscan");
|
||||
}
|
||||
});
|
||||
|
||||
it.each(["all-failed", "all-succeeded", "empty"] as const)("handles the %s control", async mode => {
|
||||
const control = await seedSidebarFixture(db, mode);
|
||||
fixtures.push(control);
|
||||
const response = await sidebarBadgeService(db).get(control.companyId);
|
||||
const failedRuns = mode === "all-failed" ? 200 : 0;
|
||||
expect(response).toEqual({ inbox: failedRuns + 1, approvals: 1, failedRuns, joinRequests: 0 });
|
||||
});
|
||||
});
|
||||
@@ -1,7 +1,7 @@
|
||||
import { and, desc, eq, inArray, not, type SQL } from "drizzle-orm";
|
||||
import { and, desc, eq, inArray, isNull, not, or, sql, type SQL } from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { agents, approvals, heartbeatRuns } from "@paperclipai/db";
|
||||
import { isHeartbeatRunVisibleInMine, latestFailedRunsByAgent, type SidebarBadges } from "@paperclipai/shared";
|
||||
import type { SidebarBadges } from "@paperclipai/shared";
|
||||
import { authorizationService, type AuthorizationActor } from "./authorization.js";
|
||||
import { getHeartbeatRunReadDecisions } from "./heartbeat-run-privacy.js";
|
||||
|
||||
@@ -51,11 +51,10 @@ export function sidebarBadgeService(db: Db) {
|
||||
rows.filter((row) => !isDismissed(extra?.dismissals ?? new Map(), `approval:${row.id}`, row.updatedAt)).length
|
||||
);
|
||||
|
||||
const recentRuns = await db
|
||||
const recentRuns = db
|
||||
.select({
|
||||
id: heartbeatRuns.id,
|
||||
agentId: heartbeatRuns.agentId,
|
||||
companyId: heartbeatRuns.companyId,
|
||||
scopeKind: heartbeatRuns.scopeKind,
|
||||
issueId: heartbeatRuns.issueId,
|
||||
status: heartbeatRuns.status,
|
||||
@@ -72,14 +71,50 @@ export function sidebarBadgeService(db: Db) {
|
||||
),
|
||||
)
|
||||
.orderBy(desc(heartbeatRuns.createdAt))
|
||||
.limit(200);
|
||||
.limit(200)
|
||||
.as("recent_runs");
|
||||
|
||||
const failures = latestFailedRunsByAgent(recentRuns).filter((row) =>
|
||||
(extra?.currentUserId === undefined || isHeartbeatRunVisibleInMine(row, extra.currentUserId))
|
||||
&& !isDismissed(extra?.dismissals ?? new Map(), `run:${row.id}`, row.createdAt),
|
||||
const observedRuns = db.select({
|
||||
id: recentRuns.id,
|
||||
agentId: recentRuns.agentId,
|
||||
scopeKind: recentRuns.scopeKind,
|
||||
issueId: recentRuns.issueId,
|
||||
status: recentRuns.status,
|
||||
responsibleUserId: recentRuns.responsibleUserId,
|
||||
createdAt: recentRuns.createdAt,
|
||||
observedOrder: sql<number>`row_number() over ()`.as("observed_order"),
|
||||
}).from(recentRuns).as("observed_runs");
|
||||
const rankedRuns = db.select({
|
||||
id: observedRuns.id,
|
||||
scopeKind: observedRuns.scopeKind,
|
||||
issueId: observedRuns.issueId,
|
||||
status: observedRuns.status,
|
||||
responsibleUserId: observedRuns.responsibleUserId,
|
||||
createdAt: observedRuns.createdAt,
|
||||
agentRunRank: sql<number>`row_number() over (partition by ${observedRuns.agentId} order by ${observedRuns.createdAt} desc, ${observedRuns.observedOrder})`.as("agent_run_rank"),
|
||||
}).from(observedRuns).as("ranked_runs");
|
||||
const mineCondition = extra?.currentUserId === undefined
|
||||
? undefined
|
||||
: !extra.currentUserId
|
||||
? sql<boolean>`false`
|
||||
: extra.currentUserId === "local-board"
|
||||
? or(eq(rankedRuns.responsibleUserId, "local-board"), isNull(rankedRuns.responsibleUserId), eq(rankedRuns.responsibleUserId, ""))
|
||||
: eq(rankedRuns.responsibleUserId, extra.currentUserId);
|
||||
const candidates = await db.select({
|
||||
id: rankedRuns.id,
|
||||
scopeKind: rankedRuns.scopeKind,
|
||||
issueId: rankedRuns.issueId,
|
||||
createdAt: rankedRuns.createdAt,
|
||||
}).from(rankedRuns).where(and(
|
||||
eq(rankedRuns.agentRunRank, 1),
|
||||
inArray(rankedRuns.status, ["failed", "timed_out"]),
|
||||
mineCondition,
|
||||
));
|
||||
const failures = candidates.filter((row) =>
|
||||
!isDismissed(extra?.dismissals ?? new Map(), `run:${row.id}`, row.createdAt),
|
||||
);
|
||||
const failedRuns = extra?.actor
|
||||
? (await getHeartbeatRunReadDecisions(db, authorization, extra.actor, failures)).filter(Boolean).length
|
||||
? (await getHeartbeatRunReadDecisions(db, authorization, extra.actor, failures.map((row) => ({ ...row, companyId })))).filter(Boolean).length
|
||||
: failures.length;
|
||||
|
||||
const joinRequests = (extra?.joinRequests ?? []).filter((row) =>
|
||||
|
||||
Reference in new issue
Block a user