mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-20 09:50:48 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Hosted instances send real-user page-load traces to Honeycomb (#21156), so we can set page-load SLOs > - Staging data showed four measurement gaps: 140 of 147 hard loads were hidden tabs that became 30 s timeouts, a few timeouts lasted about 2 hours after the computer slept, hard loads had no API child spans, and list views on the tasks page could not be told apart > - These gaps make every average and percentile wrong, so the data cannot back an SLO yet > - This pull request ends hidden loads as `hidden`, caps timeouts at exactly 30 s and marks suspends, backfills hard-load API spans from Resource Timing, and records the list view from an allowlist > - The benefit is page-load numbers that describe what users see, with no cost on the critical path ## Linked Issues or Issue Description Refs #21156 **What is the issue?** Browser page-load tracing records misleading data. A load in a background tab never paints, so it always ends as a 30 s `timeout`. A timeout that fires late (after sleep) reports hours. Hard loads have no API spans because the SDK loads after the page settles. `?view=` is stripped, so all task list views share one route. **Steps to reproduce** Open the all-tasks page in a background tab on a traced instance. The `page.load` span is `outcome=timeout`, `was_hidden=true`, about 30.4 s (reproduced locally on master: 30,359 ms and 30,988 ms). **Expected behavior** A hidden load is counted separately and ends quickly. A timeout lasts exactly 30 s. A hard load shows its API requests. The list view is a dimension. ## What Changed - `page-load-recorder.ts`: new outcome `hidden`. A load that starts hidden, or hides before it is ready, ends at the moment the tab hid (same rule `web-vitals` uses for LCP/FCP). For a hard load, the controller reads Chromium `visibility-state` entries, so a hide before tracing started is caught. A redirect while still hidden renames the hidden span instead of starting a second one. - `page-load-recorder.ts`: any end observed after start + 30 s is a `timeout` that ends at exactly start + 30 s. `paperclip.page.suspended=true` marks an end more than 5 s past the deadline or a gap of more than 5 s between frames. - `controller.ts`: a `PerformanceObserver` (`type: resource`, `buffered: true`) keeps only this origin's `/api/` fetch/XHR entries (max 500) until the SDK loads. It does not depend on the 250-entry resource buffer. - `sdk.ts`: when the SDK loads, it creates `HTTP` child spans under each `page.load` that started before the SDK, for requests that started in the load window and before live fetch tracing began (no duplicates). The name is `HTTP` because Resource Timing has no method (OpenTelemetry's name for an unknown method). Status comes only from `responseStatus` when the browser exposes it. The URL goes through the same sanitizer as live fetch spans and becomes `paperclip.api.area`. Spans carry `paperclip.api.timing_source=resource_timing`. - `page-view.ts`: `paperclip.page.view` for `/:companyPrefix/issues` (values from `TASK_VIEW_KEYS`, `default` when missing) and `/:companyPrefix/decisions` (values from `ATTENTION_PERSONAL_VIEWS`, `mine` when missing). Any other value is `other`. - `span-sanitizer.ts`: allowlist adds `paperclip.page.suspended`, `paperclip.page.view`, `paperclip.api.timing_source`. - `doc/browser-performance-tracing.md`: outcome list, the meaning of "loaded" for hidden tabs, suspend handling, page views, backfilled API spans, attribute table, and example queries that filter `outcome = ready`. ## Verification - Red then green: 17 new tests failed on the old code and `page-view.test.ts` failed to import; all 50 tests in `ui/src/lib/browser-tracing` pass after the change. `PageLoadTracker.test.tsx` passes. `tsc --noEmit` for `ui` passes. - `pnpm verify` exits 2 (a doc is a global CI input), so CI supplies broad coverage. - Real browser (local dev server on an empty isolated instance, stand-in OTLP collector, headless Chromium, tracing config injected into the session response): - Background tab (raw CDP, `Target.createTarget background:true`): `outcome=hidden`, duration 0 ms. Master: `outcome=timeout`, 30,359 ms. - Tab covered 400 ms into the load: `outcome=hidden`, 434 ms. Master: `timeout`, 30,988 ms. - Hard load of the all-tasks page: `page.load` with 13 to 38 `HTTP` child spans (areas `companies`, `plugins`, `cli-auth`, and others, status 200 where exposed), same trace, parent = `page.load`. - `?view=mine`: `paperclip.page.view=mine`. - Leak scan of every exported body: no URL, host, company id, company prefix, user id, or query text. - Entry bundle: 3,822,045 → 3,823,991 B raw (+1.9 KB), 1,154,280 → 1,155,097 B gzip (+0.8 KB, 0.07%). Lazy SDK chunk: 84,322 → 85,229 B raw, 27,304 → 27,630 B gzip. ## Risks - Low risk. The change is limited to the operator-gated Observability path on hosted instances; self-hosted installs never start the controller. - Behavior shift for dashboards: hidden tabs are now `outcome=hidden` instead of `timeout`, so timeout rates drop. Queries should filter `outcome = ready`. - The observer adds one callback per batch of finished requests while tracing is on, until the SDK loads. It keeps at most 500 entries. - Backfilled spans do not join server traces because no `traceparent` was sent. The docs say so. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits, headless Chromium). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
173 lines
5.9 KiB
TypeScript
173 lines
5.9 KiB
TypeScript
// @vitest-environment jsdom
|
|
|
|
import { act } from "react";
|
|
import { createRoot, type Root } from "react-dom/client";
|
|
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
|
|
import { MemoryRouter, Route, Routes, useNavigate } from "react-router-dom";
|
|
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
|
import type { BrowserTracingConfig } from "@paperclipai/shared";
|
|
import type { PageLoadMeasurement, PageLoadRecorder } from "@/lib/browser-tracing/page-load-recorder";
|
|
import { resetPageSessionSampleForTest } from "@/lib/browser-tracing/controller";
|
|
import { PageLoadTracker } from "./PageLoadTracker";
|
|
|
|
const getSessionMock = vi.hoisted(() => vi.fn());
|
|
const sdkModuleImported = vi.hoisted(() => vi.fn());
|
|
const measurements = vi.hoisted(() => ({ begun: [] as PageLoadMeasurement[], ended: [] as PageLoadMeasurement[] }));
|
|
|
|
vi.mock("@/api/auth", () => ({
|
|
authApi: { getSession: () => getSessionMock() },
|
|
}));
|
|
|
|
// Records whether the tracing SDK chunk was ever imported; when it is, a
|
|
// fake sink stands in for OpenTelemetry.
|
|
vi.mock("@/lib/browser-tracing/sdk", () => {
|
|
sdkModuleImported();
|
|
return {
|
|
startTracingSdk: (_config: BrowserTracingConfig, recorder: PageLoadRecorder) => {
|
|
recorder.attachSink({
|
|
begin: (measurement) => measurements.begun.push({ ...measurement }),
|
|
end: (measurement) => measurements.ended.push({ ...measurement }),
|
|
});
|
|
return { forceFlush: async () => {}, shutdown: async () => {} };
|
|
},
|
|
};
|
|
});
|
|
|
|
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
|
(globalThis as any).IS_REACT_ACT_ENVIRONMENT = true;
|
|
|
|
const config: BrowserTracingConfig = {
|
|
tracesUrl: "https://api.honeycomb.io/v1/traces",
|
|
ingestKey: "hcaik_test",
|
|
serviceName: "paperclip-web",
|
|
environment: "test",
|
|
sampleRate: 1,
|
|
};
|
|
|
|
function session(browserTracing?: BrowserTracingConfig | null) {
|
|
return {
|
|
session: { id: "s", userId: "u" },
|
|
user: { id: "u", email: null, name: null, image: null },
|
|
sentryDsn: null,
|
|
...(browserTracing === undefined ? {} : { browserTracing }),
|
|
};
|
|
}
|
|
|
|
let navigateTo: ((path: string) => void) | null = null;
|
|
function NavigateHandle() {
|
|
navigateTo = useNavigate();
|
|
return null;
|
|
}
|
|
|
|
const Page = () => <div>page</div>;
|
|
const routes = (
|
|
<>
|
|
<Route path=":companyPrefix" element={<Page />} />
|
|
<Route path=":companyPrefix/issues" element={<Page />} />
|
|
<Route path=":companyPrefix/issues/:issueId" element={<Page />} />
|
|
</>
|
|
);
|
|
|
|
/**
|
|
* The page's clock starts at navigation start. A test process can be older
|
|
* than the 30 s page-load timeout, which would turn the hard load (start 0)
|
|
* into a timeout, so the clock is restarted for each test.
|
|
*/
|
|
function restartPageClock() {
|
|
const realNow = performance.now.bind(performance);
|
|
const origin = realNow();
|
|
vi.spyOn(performance, "now").mockImplementation(() => realNow() - origin);
|
|
}
|
|
|
|
describe("PageLoadTracker", () => {
|
|
let container: HTMLDivElement;
|
|
let root: Root | null = null;
|
|
let queryClient: QueryClient;
|
|
|
|
beforeEach(() => {
|
|
container = document.createElement("div");
|
|
document.body.appendChild(container);
|
|
queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } });
|
|
measurements.begun.length = 0;
|
|
measurements.ended.length = 0;
|
|
resetPageSessionSampleForTest();
|
|
window.history.replaceState(null, "", "/PAP/issues");
|
|
restartPageClock();
|
|
});
|
|
|
|
afterEach(() => {
|
|
act(() => root?.unmount());
|
|
root = null;
|
|
container.remove();
|
|
vi.clearAllMocks();
|
|
vi.mocked(performance.now).mockRestore?.();
|
|
});
|
|
|
|
async function render(initialPath: string) {
|
|
root = createRoot(container);
|
|
await act(async () => {
|
|
root!.render(
|
|
<QueryClientProvider client={queryClient}>
|
|
<MemoryRouter initialEntries={[initialPath]}>
|
|
<PageLoadTracker routes={routes} />
|
|
<NavigateHandle />
|
|
<Routes>{routes}</Routes>
|
|
</MemoryRouter>
|
|
</QueryClientProvider>,
|
|
);
|
|
});
|
|
}
|
|
|
|
async function settle(ms = 400) {
|
|
await act(async () => {
|
|
await new Promise((resolve) => window.setTimeout(resolve, ms));
|
|
});
|
|
}
|
|
|
|
it.each([
|
|
["the server sends null (self-hosted, local dev, or no endpoint)", session(null)],
|
|
["an older server omits the field", session()],
|
|
["the browser is signed out", null],
|
|
])("never imports the tracing SDK when %s", async (_label, payload) => {
|
|
getSessionMock.mockResolvedValue(payload);
|
|
await render("/PAP/issues");
|
|
await settle();
|
|
expect(sdkModuleImported).not.toHaveBeenCalled();
|
|
expect(measurements.begun).toHaveLength(0);
|
|
});
|
|
|
|
it("measures the hard load and a client-side navigation under route templates", async () => {
|
|
getSessionMock.mockResolvedValue(session(config));
|
|
await render("/PAP/issues");
|
|
await vi.waitFor(() => expect(measurements.ended).toHaveLength(1), { timeout: 3000 });
|
|
expect(sdkModuleImported).toHaveBeenCalledTimes(1);
|
|
expect(measurements.ended[0]).toMatchObject({
|
|
routeTemplate: "/:companyPrefix/issues",
|
|
loadType: "hard",
|
|
startTime: 0,
|
|
outcome: "ready",
|
|
});
|
|
|
|
await act(async () => navigateTo!("/PAP/issues/PAP-4242"));
|
|
await vi.waitFor(() => expect(measurements.ended).toHaveLength(2), { timeout: 3000 });
|
|
const navigation = measurements.ended[1]!;
|
|
expect(navigation).toMatchObject({
|
|
routeTemplate: "/:companyPrefix/issues/:issueId",
|
|
loadType: "client",
|
|
outcome: "ready",
|
|
readySignal: "idle",
|
|
});
|
|
expect(navigation.endTime).toBeGreaterThanOrEqual(navigation.startTime);
|
|
expect(JSON.stringify(measurements)).not.toContain("PAP-4242");
|
|
});
|
|
|
|
it("downloads nothing for an unsampled page session", async () => {
|
|
vi.spyOn(Math, "random").mockReturnValue(0.9);
|
|
getSessionMock.mockResolvedValue(session({ ...config, sampleRate: 0.5 }));
|
|
await render("/PAP/issues");
|
|
await settle();
|
|
expect(sdkModuleImported).not.toHaveBeenCalled();
|
|
vi.restoreAllMocks();
|
|
});
|
|
});
|