Files
PaperClipAI/ui/src/lib/browser-tracing/controller.ts
T
DottaandClaude Opus 5.5 51c81c537b fix(browser-tracing): end hidden loads, cap timeouts, backfill hard-load API spans, record list view (#21392)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Hosted instances send real-user page-load traces to Honeycomb
(#21156), so we can set page-load SLOs
> - Staging data showed four measurement gaps: 140 of 147 hard loads
were hidden tabs that became 30 s timeouts, a few timeouts lasted about
2 hours after the computer slept, hard loads had no API child spans, and
list views on the tasks page could not be told apart
> - These gaps make every average and percentile wrong, so the data
cannot back an SLO yet
> - This pull request ends hidden loads as `hidden`, caps timeouts at
exactly 30 s and marks suspends, backfills hard-load API spans from
Resource Timing, and records the list view from an allowlist
> - The benefit is page-load numbers that describe what users see, with
no cost on the critical path

## Linked Issues or Issue Description

Refs #21156

**What is the issue?**
Browser page-load tracing records misleading data. A load in a
background tab never paints, so it always ends as a 30 s `timeout`. A
timeout that fires late (after sleep) reports hours. Hard loads have no
API spans because the SDK loads after the page settles. `?view=` is
stripped, so all task list views share one route.

**Steps to reproduce**
Open the all-tasks page in a background tab on a traced instance. The
`page.load` span is `outcome=timeout`, `was_hidden=true`, about 30.4 s
(reproduced locally on master: 30,359 ms and 30,988 ms).

**Expected behavior**
A hidden load is counted separately and ends quickly. A timeout lasts
exactly 30 s. A hard load shows its API requests. The list view is a
dimension.

## What Changed

- `page-load-recorder.ts`: new outcome `hidden`. A load that starts
hidden, or hides before it is ready, ends at the moment the tab hid
(same rule `web-vitals` uses for LCP/FCP). For a hard load, the
controller reads Chromium `visibility-state` entries, so a hide before
tracing started is caught. A redirect while still hidden renames the
hidden span instead of starting a second one.
- `page-load-recorder.ts`: any end observed after start + 30 s is a
`timeout` that ends at exactly start + 30 s.
`paperclip.page.suspended=true` marks an end more than 5 s past the
deadline or a gap of more than 5 s between frames.
- `controller.ts`: a `PerformanceObserver` (`type: resource`, `buffered:
true`) keeps only this origin's `/api/` fetch/XHR entries (max 500)
until the SDK loads. It does not depend on the 250-entry resource
buffer.
- `sdk.ts`: when the SDK loads, it creates `HTTP` child spans under each
`page.load` that started before the SDK, for requests that started in
the load window and before live fetch tracing began (no duplicates). The
name is `HTTP` because Resource Timing has no method (OpenTelemetry's
name for an unknown method). Status comes only from `responseStatus`
when the browser exposes it. The URL goes through the same sanitizer as
live fetch spans and becomes `paperclip.api.area`. Spans carry
`paperclip.api.timing_source=resource_timing`.
- `page-view.ts`: `paperclip.page.view` for `/:companyPrefix/issues`
(values from `TASK_VIEW_KEYS`, `default` when missing) and
`/:companyPrefix/decisions` (values from `ATTENTION_PERSONAL_VIEWS`,
`mine` when missing). Any other value is `other`.
- `span-sanitizer.ts`: allowlist adds `paperclip.page.suspended`,
`paperclip.page.view`, `paperclip.api.timing_source`.
- `doc/browser-performance-tracing.md`: outcome list, the meaning of
"loaded" for hidden tabs, suspend handling, page views, backfilled API
spans, attribute table, and example queries that filter `outcome =
ready`.

## Verification

- Red then green: 17 new tests failed on the old code and
`page-view.test.ts` failed to import; all 50 tests in
`ui/src/lib/browser-tracing` pass after the change.
`PageLoadTracker.test.tsx` passes. `tsc --noEmit` for `ui` passes.
- `pnpm verify` exits 2 (a doc is a global CI input), so CI supplies
broad coverage.
- Real browser (local dev server on an empty isolated instance, stand-in
OTLP collector, headless Chromium, tracing config injected into the
session response):
- Background tab (raw CDP, `Target.createTarget background:true`):
`outcome=hidden`, duration 0 ms. Master: `outcome=timeout`, 30,359 ms.
- Tab covered 400 ms into the load: `outcome=hidden`, 434 ms. Master:
`timeout`, 30,988 ms.
- Hard load of the all-tasks page: `page.load` with 13 to 38 `HTTP`
child spans (areas `companies`, `plugins`, `cli-auth`, and others,
status 200 where exposed), same trace, parent = `page.load`.
  - `?view=mine`: `paperclip.page.view=mine`.
- Leak scan of every exported body: no URL, host, company id, company
prefix, user id, or query text.
- Entry bundle: 3,822,045 → 3,823,991 B raw (+1.9 KB), 1,154,280 →
1,155,097 B gzip (+0.8 KB, 0.07%). Lazy SDK chunk: 84,322 → 85,229 B
raw, 27,304 → 27,630 B gzip.

## Risks

- Low risk. The change is limited to the operator-gated Observability
path on hosted instances; self-hosted installs never start the
controller.
- Behavior shift for dashboards: hidden tabs are now `outcome=hidden`
instead of `timeout`, so timeout rates drop. Queries should filter
`outcome = ready`.
- The observer adds one callback per batch of finished requests while
tracing is on, until the SDK loads. It keeps at most 500 entries.
- Backfilled spans do not join server traces because no `traceparent`
was sent. The docs say so.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, headless Chromium).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 09:19:25 -06:00

279 lines
11 KiB
TypeScript

// Starts and stops browser performance tracing for one page session.
//
// Main-bundle code. It does nothing unless the authenticated session carries a
// `browserTracing` config, which the server sends only for a hosted Paperclip
// Cloud instance with an operator-set endpoint. When active it:
// - decides once per page session whether this session is sampled; an
// unsampled session does nothing else and downloads nothing,
// - measures the hard load and every client-side navigation
// (`page-load-recorder.ts`),
// - keeps the Resource Timing entries of this origin's `/api/` requests
// until the SDK loads, so the SDK can show the hard load's API requests,
// and
// - after the page settles, downloads the tracing SDK chunk (`sdk.ts`),
// which exports the measurements, Web Vitals, and API request spans.
// See doc/browser-performance-tracing.md.
import type { QueryClient } from "@tanstack/react-query";
import type { BrowserTracingConfig } from "@paperclipai/shared";
import { whenPageSettles } from "@/lib/when-page-settles";
import { readPageContentGates } from "./page-content-gates";
import { createPageLoadRecorder, PAGE_LOAD_TIMEOUT_MS, type PageLoadRecorder } from "./page-load-recorder";
import { pageViewFor } from "./page-view";
export type RouterNavigationType = "PUSH" | "POP" | "REPLACE";
export interface BrowserTracingController {
/** Called after React commits a new location. */
onLocationCommitted(pathname: string, navigationType: RouterNavigationType, routeTemplate: string): void;
stop(): void;
}
export interface TracingSdkHandle {
forceFlush(): Promise<void>;
shutdown(): Promise<void>;
}
/** Resource Timing entries for this origin's `/api/` requests, kept until the SDK takes them. */
export interface ApiResourceCollector {
entries(): readonly PerformanceResourceTiming[];
/** Stops observing and drops the entries. */
stop(): void;
}
export interface TracingSdkModule {
startTracingSdk(
config: BrowserTracingConfig,
recorder: PageLoadRecorder,
options?: { apiResources?: ApiResourceCollector },
): TracingSdkHandle;
}
export interface BrowserTracingControllerOptions {
queryClient: QueryClient;
initialPathname: string;
initialRouteTemplate: string;
/** Overridable for tests. */
loadSdk?: () => Promise<TracingSdkModule>;
random?: () => number;
/** Overridable for tests; defaults to waiting until the page settles. */
scheduleSdkLoad?: (start: () => void) => () => void;
/** Load the SDK by this time even if the page never settles. */
sdkLoadDeadlineMs?: number;
}
/**
* Load the SDK no later than this, measured from activation. A page whose
* requests hang never settles; its hard load then ends as `timeout` at 30 s
* after navigation start, and this deadline still exports it.
*/
export const SDK_LOAD_DEADLINE_MS = PAGE_LOAD_TIMEOUT_MS + 5_000;
let sessionSampled: boolean | null = null;
/** One decision per page session, so a sampled session is complete. */
export function isPageSessionSampled(sampleRate: number, random: () => number = Math.random): boolean {
if (sessionSampled === null) sessionSampled = random() < sampleRate;
return sessionSampled;
}
export function resetPageSessionSampleForTest(): void {
sessionSampled = null;
}
/** Any query that has no data yet and is fetching is the page's primary data. */
export function hasPendingPrimaryData(queryClient: QueryClient): boolean {
return queryClient.isFetching({ predicate: (query) => query.state.data === undefined }) > 0;
}
/** At most this many entries are kept; a hard load makes far fewer requests. */
const API_RESOURCE_LIMIT = 500;
/**
* Collects Resource Timing entries for `fetch`/XHR requests to `origin/api/`.
* `buffered: true` delivers the requests that finished before this ran (the
* hard load's session request, for example), and an observer keeps receiving
* entries after the browser's resource buffer (250 entries by default) is
* full. One observer callback per batch of finished requests; nothing else.
*/
export function observeApiResources(
origin: string,
Observer: typeof PerformanceObserver | null = typeof PerformanceObserver === "function" ? PerformanceObserver : null,
): ApiResourceCollector {
let kept: PerformanceResourceTiming[] = [];
if (!Observer || !(Observer.supportedEntryTypes ?? []).includes("resource")) {
return { entries: () => kept, stop: () => {} };
}
const prefix = `${origin}/api/`;
let observer: PerformanceObserver | null = null;
try {
observer = new Observer((list) => {
for (const entry of list.getEntries() as PerformanceResourceTiming[]) {
if (kept.length >= API_RESOURCE_LIMIT) break;
if ((entry.initiatorType === "fetch" || entry.initiatorType === "xmlhttprequest") && entry.name.startsWith(prefix)) {
kept.push(entry);
}
}
});
observer.observe({ type: "resource", buffered: true });
} catch {
observer = null;
}
return {
entries: () => kept,
stop() {
observer?.disconnect();
observer = null;
kept = [];
},
};
}
/**
* When the tab was first hidden since navigation start, or null if it has
* been visible throughout. Chromium records each visibility change as a
* `visibility-state` performance entry (including the initial state); where
* it does not, a tab that is hidden now counts as hidden from the start, as
* `web-vitals` assumes.
*/
function firstHiddenTime(): number | null {
try {
const entries = performance.getEntriesByType?.("visibility-state") ?? [];
const hidden = entries.find((entry) => entry.name === "hidden");
if (hidden) return hidden.startTime;
} catch {
// Not supported.
}
return document.visibilityState === "hidden" ? 0 : null;
}
function pathnameOf(url: string | URL | null | undefined): string | null {
if (url === null || url === undefined) return null;
try {
return new URL(String(url), window.location.href).pathname;
} catch {
return null;
}
}
/**
* React Router commits a navigation inside a transition, after any lazy route
* code has loaded. The history entry changes first, when the user acts, so
* the navigation start is taken from the history change.
*/
function trackHistoryChanges(now: () => number) {
let last: { pathname: string; time: number } | null = null;
const history = window.history;
const originalPush = history.pushState;
const originalReplace = history.replaceState;
const record = (url: string | URL | null | undefined) => {
const pathname = pathnameOf(url);
if (pathname) last = { pathname, time: now() };
};
history.pushState = function pushState(this: History, ...args: Parameters<History["pushState"]>) {
record(args[2]);
return originalPush.apply(this, args);
};
history.replaceState = function replaceState(this: History, ...args: Parameters<History["replaceState"]>) {
record(args[2]);
return originalReplace.apply(this, args);
};
const onPopState = (event: PopStateEvent) => {
last = { pathname: window.location.pathname, time: event.timeStamp || now() };
};
window.addEventListener("popstate", onPopState);
return {
takeStart(pathname: string): number | null {
const entry = last;
last = null;
return entry && entry.pathname === pathname ? entry.time : null;
},
stop() {
if (history.pushState !== originalPush) history.pushState = originalPush;
if (history.replaceState !== originalReplace) history.replaceState = originalReplace;
window.removeEventListener("popstate", onPopState);
},
};
}
export function startBrowserTracingController(
config: BrowserTracingConfig,
options: BrowserTracingControllerOptions,
): BrowserTracingController | null {
if (!isPageSessionSampled(config.sampleRate, options.random)) return null;
const now = () => performance.now();
const recorder = createPageLoadRecorder({
now,
requestFrame: (callback) => window.requestAnimationFrame(() => callback()),
cancelFrame: (handle) => window.cancelAnimationFrame(handle),
setTimer: (callback, ms) => window.setTimeout(callback, ms),
clearTimer: (handle) => window.clearTimeout(handle as number),
isDocumentHidden: () => document.visibilityState === "hidden",
hasPendingPrimaryData: () => hasPendingPrimaryData(options.queryClient),
readGates: readPageContentGates,
readView: (routeTemplate) => pageViewFor(routeTemplate, window.location.search),
});
const apiResources = observeApiResources(window.location.origin);
const history = trackHistoryChanges(now);
const onVisibilityChange = () => {
if (document.visibilityState === "hidden") recorder.noteHidden(now());
};
document.addEventListener("visibilitychange", onVisibilityChange);
// The hard load started at navigation start (time 0), before this code ran.
recorder.begin({ routeTemplate: options.initialRouteTemplate, loadType: "hard", startTime: 0, hiddenAt: firstHiddenTime() });
let lastPathname = options.initialPathname;
let stopped = false;
let sdk: TracingSdkHandle | null = null;
const loadSdk = options.loadSdk ?? (() => import("./sdk"));
const schedule = options.scheduleSdkLoad ?? ((start: () => void) => whenPageSettles(options.queryClient, start));
let loadStarted = false;
let cancelSettled: () => void = () => {};
const startLoad = () => {
if (loadStarted || stopped) return;
loadStarted = true;
cancelSettled();
window.clearTimeout(deadline);
void loadSdk()
.then((module) => {
if (stopped) return;
sdk = module.startTracingSdk(config, recorder, { apiResources });
})
.catch(() => {
// Tracing is optional; a failed chunk load must not affect the page.
apiResources.stop();
});
};
const deadline = window.setTimeout(startLoad, options.sdkLoadDeadlineMs ?? SDK_LOAD_DEADLINE_MS);
const cancelSchedule = schedule(startLoad);
if (loadStarted) cancelSchedule();
else cancelSettled = cancelSchedule;
const cancelScheduledLoad = () => {
cancelSettled();
window.clearTimeout(deadline);
};
return {
onLocationCommitted(pathname, navigationType, routeTemplate) {
if (stopped || pathname === lastPathname) return;
lastPathname = pathname;
const start = history.takeStart(pathname);
if (navigationType === "REPLACE" && recorder.redirect(routeTemplate)) return;
recorder.begin({ routeTemplate, loadType: "client", startTime: start ?? now() });
},
stop() {
if (stopped) return;
stopped = true;
cancelScheduledLoad();
history.stop();
document.removeEventListener("visibilitychange", onVisibilityChange);
recorder.stop();
apiResources.stop();
void sdk?.shutdown();
sdk = null;
},
};
}