mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 04:21:24 +01:00
fix(browser-tracing): end hidden loads, cap timeouts, backfill hard-load API spans, record list view (#21392)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Hosted instances send real-user page-load traces to Honeycomb (#21156), so we can set page-load SLOs > - Staging data showed four measurement gaps: 140 of 147 hard loads were hidden tabs that became 30 s timeouts, a few timeouts lasted about 2 hours after the computer slept, hard loads had no API child spans, and list views on the tasks page could not be told apart > - These gaps make every average and percentile wrong, so the data cannot back an SLO yet > - This pull request ends hidden loads as `hidden`, caps timeouts at exactly 30 s and marks suspends, backfills hard-load API spans from Resource Timing, and records the list view from an allowlist > - The benefit is page-load numbers that describe what users see, with no cost on the critical path ## Linked Issues or Issue Description Refs #21156 **What is the issue?** Browser page-load tracing records misleading data. A load in a background tab never paints, so it always ends as a 30 s `timeout`. A timeout that fires late (after sleep) reports hours. Hard loads have no API spans because the SDK loads after the page settles. `?view=` is stripped, so all task list views share one route. **Steps to reproduce** Open the all-tasks page in a background tab on a traced instance. The `page.load` span is `outcome=timeout`, `was_hidden=true`, about 30.4 s (reproduced locally on master: 30,359 ms and 30,988 ms). **Expected behavior** A hidden load is counted separately and ends quickly. A timeout lasts exactly 30 s. A hard load shows its API requests. The list view is a dimension. ## What Changed - `page-load-recorder.ts`: new outcome `hidden`. A load that starts hidden, or hides before it is ready, ends at the moment the tab hid (same rule `web-vitals` uses for LCP/FCP). For a hard load, the controller reads Chromium `visibility-state` entries, so a hide before tracing started is caught. A redirect while still hidden renames the hidden span instead of starting a second one. - `page-load-recorder.ts`: any end observed after start + 30 s is a `timeout` that ends at exactly start + 30 s. `paperclip.page.suspended=true` marks an end more than 5 s past the deadline or a gap of more than 5 s between frames. - `controller.ts`: a `PerformanceObserver` (`type: resource`, `buffered: true`) keeps only this origin's `/api/` fetch/XHR entries (max 500) until the SDK loads. It does not depend on the 250-entry resource buffer. - `sdk.ts`: when the SDK loads, it creates `HTTP` child spans under each `page.load` that started before the SDK, for requests that started in the load window and before live fetch tracing began (no duplicates). The name is `HTTP` because Resource Timing has no method (OpenTelemetry's name for an unknown method). Status comes only from `responseStatus` when the browser exposes it. The URL goes through the same sanitizer as live fetch spans and becomes `paperclip.api.area`. Spans carry `paperclip.api.timing_source=resource_timing`. - `page-view.ts`: `paperclip.page.view` for `/:companyPrefix/issues` (values from `TASK_VIEW_KEYS`, `default` when missing) and `/:companyPrefix/decisions` (values from `ATTENTION_PERSONAL_VIEWS`, `mine` when missing). Any other value is `other`. - `span-sanitizer.ts`: allowlist adds `paperclip.page.suspended`, `paperclip.page.view`, `paperclip.api.timing_source`. - `doc/browser-performance-tracing.md`: outcome list, the meaning of "loaded" for hidden tabs, suspend handling, page views, backfilled API spans, attribute table, and example queries that filter `outcome = ready`. ## Verification - Red then green: 17 new tests failed on the old code and `page-view.test.ts` failed to import; all 50 tests in `ui/src/lib/browser-tracing` pass after the change. `PageLoadTracker.test.tsx` passes. `tsc --noEmit` for `ui` passes. - `pnpm verify` exits 2 (a doc is a global CI input), so CI supplies broad coverage. - Real browser (local dev server on an empty isolated instance, stand-in OTLP collector, headless Chromium, tracing config injected into the session response): - Background tab (raw CDP, `Target.createTarget background:true`): `outcome=hidden`, duration 0 ms. Master: `outcome=timeout`, 30,359 ms. - Tab covered 400 ms into the load: `outcome=hidden`, 434 ms. Master: `timeout`, 30,988 ms. - Hard load of the all-tasks page: `page.load` with 13 to 38 `HTTP` child spans (areas `companies`, `plugins`, `cli-auth`, and others, status 200 where exposed), same trace, parent = `page.load`. - `?view=mine`: `paperclip.page.view=mine`. - Leak scan of every exported body: no URL, host, company id, company prefix, user id, or query text. - Entry bundle: 3,822,045 → 3,823,991 B raw (+1.9 KB), 1,154,280 → 1,155,097 B gzip (+0.8 KB, 0.07%). Lazy SDK chunk: 84,322 → 85,229 B raw, 27,304 → 27,630 B gzip. ## Risks - Low risk. The change is limited to the operator-gated Observability path on hosted instances; self-hosted installs never start the controller. - Behavior shift for dashboards: hidden tabs are now `outcome=hidden` instead of `timeout`, so timeout rates drop. Queries should filter `outcome = ready`. - The observer adds one callback per batch of finished requests while tracing is on, until the SDK loads. It keeps at most 500 entries. - Backfilled spans do not join server traces because no `traceparent` was sent. The docs say so. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits, headless Chromium). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
5c5a2fcebe
commit
51c81c537b
11 files changed
+909
-45
No files matched your search
@@ -63,7 +63,7 @@ for `0.25`), so Honeycomb's `COUNT` and percentiles are weighted correctly.
|
||||
|
||||
Volume per sampled session: one `page.load` span per navigation, up to three
|
||||
`web_vital.*` spans per page lifetime, and one span per same-origin `/api/`
|
||||
request made after the SDK loads.
|
||||
request made during a measured navigation or after the SDK loads.
|
||||
|
||||
## Load-speed cost
|
||||
|
||||
@@ -77,10 +77,22 @@ fetching, browser idle; `ui/src/lib/when-page-settles.ts`). If the page never
|
||||
settles (for example a request hangs), the chunk loads 35 seconds after tracing
|
||||
starts anyway, so a hard load that ended as `timeout` is still exported.
|
||||
|
||||
Because the SDK loads after the first page settles, API requests made during
|
||||
the hard load have no fetch spans. The hard load's `page.load` span is still
|
||||
recorded: measurements taken before the SDK loads are buffered and exported
|
||||
when it arrives.
|
||||
Because the SDK loads after the first page settles, it cannot trace the hard
|
||||
load's API requests live. Instead the main-bundle controller registers one
|
||||
`PerformanceObserver` for `resource` entries and keeps only this
|
||||
origin's `/api/` `fetch`/XHR entries, at most 500. `buffered: true` delivers
|
||||
the requests that finished before tracing started (it starts once the session
|
||||
request returns). After that, the observer receives every entry even when the
|
||||
browser's resource buffer (250 entries by default) is full; only entries that
|
||||
overflowed that buffer before tracing started are missed. When the SDK loads it turns
|
||||
those entries into child spans of the `page.load` they belong to (see
|
||||
[Backfilled API spans](#backfilled-api-spans)). The observer stops and the
|
||||
entries are dropped once no navigation that started before the SDK loaded can
|
||||
still claim them: every such navigation has ended, and either a navigation
|
||||
that started after the SDK loaded has begun or 60 seconds have passed (a
|
||||
navigation whose history change came before the SDK can commit after it).
|
||||
Measurements taken before the SDK loads are buffered and exported when it
|
||||
arrives.
|
||||
|
||||
## What "loaded" means
|
||||
|
||||
@@ -112,10 +124,35 @@ it renders to `PageLoadTracker`), so a new page is measured without page code.
|
||||
changes are not navigations. A history replace while a navigation is still
|
||||
open is a redirect and continues it.
|
||||
|
||||
Outcomes: `ready`; `interrupted` (another navigation started first); `timeout`
|
||||
(not ready after 30 seconds). A navigation open when the tab closes is not
|
||||
sent. `was_hidden = true` means the tab was hidden during the measurement;
|
||||
browsers pause rendering in hidden tabs, so exclude these from SLOs.
|
||||
Outcomes:
|
||||
|
||||
- `ready`: loaded, as defined above.
|
||||
- `interrupted`: another navigation started first.
|
||||
- `hidden`: the tab was hidden when the navigation started (a link opened in
|
||||
a background tab) or became hidden before the page was ready. The span ends
|
||||
at the moment the tab hid, so a load in a background tab has a duration of
|
||||
about 0. Browsers do not paint hidden tabs, so such a load could never
|
||||
become ready; this is the same rule `web-vitals` applies to LCP and FCP.
|
||||
For a hard load, Chromium's `visibility-state` performance entries give the
|
||||
time the tab first hid, even before tracing started; elsewhere a tab that is
|
||||
hidden when tracing starts counts as hidden from navigation start. A
|
||||
redirect while the tab is still hidden renames the hidden span rather than
|
||||
starting another one. `was_hidden` is `true` on these spans.
|
||||
- `timeout`: not ready 30 seconds after it started. The span always ends at
|
||||
exactly start + 30 s, however late the browser ran the timer. Any end
|
||||
observed after that deadline (for example a page that only became ready
|
||||
after the computer woke up) is also reported as this timeout.
|
||||
|
||||
`paperclip.page.suspended = true` marks a span during which the computer slept
|
||||
or the tab was frozen: the observed end was more than 5 seconds past the
|
||||
30 s deadline, or two consecutive animation frames were more than 5 seconds
|
||||
apart. Its duration does not describe the page; exclude it like `hidden`. A
|
||||
suspend that ends before the next frame is not detectable, but it can never
|
||||
stretch a span beyond 30 s.
|
||||
|
||||
A navigation open when the tab closes is not sent. For load-time percentiles
|
||||
and SLOs, filter on `outcome = ready`; that excludes hidden tabs, timeouts and
|
||||
interruptions, which you count separately.
|
||||
|
||||
Pages with an explicit content signal:
|
||||
|
||||
@@ -126,6 +163,23 @@ Pages with an explicit content signal:
|
||||
|
||||
An agent chat with no saved task has no content signal and uses the idle rule.
|
||||
|
||||
### Page views
|
||||
|
||||
Some pages show a different list depending on `?view=`. For those,
|
||||
`paperclip.page.view` records the view (`ui/src/lib/browser-tracing/page-view.ts`),
|
||||
read from the URL when the span ends, so it is the view the page resolved and
|
||||
wrote into the URL. Only names from the code's view registries are sent; any
|
||||
other value is `other`.
|
||||
|
||||
| Route | Values |
|
||||
| --- | --- |
|
||||
| `/:companyPrefix/issues` | The keys of `TASK_VIEW_KEYS` in `ui/src/lib/task-views.ts` (`mine`, `unread`, `blocked`, `recent`, `everything`, `needs_attention`, `all`, `active`, `backlog`, `done`), `default` when the URL has no view yet, else `other` |
|
||||
| `/:companyPrefix/decisions` | `ATTENTION_PERSONAL_VIEWS` (`mine`, `unassigned`, `company`, `archived`, `all`), `mine` when the URL has no view, else `other` |
|
||||
|
||||
Switching views on the same page changes only the query string, so it is not a
|
||||
new navigation and has no `page.load` span; the attribute describes the view a
|
||||
navigation landed on.
|
||||
|
||||
To add one, call `usePageContentReady(ready)` in the page component. Use it
|
||||
only when the idle rule is inaccurate for that page.
|
||||
|
||||
@@ -140,6 +194,30 @@ URLs, server addresses, status messages, and event attributes, is dropped.
|
||||
| `page.load` | One per hard load or client navigation. |
|
||||
| `web_vital.lcp`, `web_vital.inp`, `web_vital.cls` | Core Web Vitals, from Google's `web-vitals`, usually reported as the tab hides. Zero duration; the value is an attribute. |
|
||||
| `GET`, `POST`, … | A `fetch` to this origin's `/api/`. Its `traceparent` header joins the server's request span to the same trace. While a `page.load` span is open it is the parent, so one trace shows the navigation, its API calls, and (when server tracing is on) their server spans. |
|
||||
| `HTTP` | A request to this origin's `/api/` made before the SDK loaded, during a measured navigation (in practice the hard load), rebuilt from Resource Timing. See below. |
|
||||
|
||||
### Backfilled API spans
|
||||
|
||||
The hard load's API requests happen before the SDK exists. When the SDK loads,
|
||||
it creates one `HTTP` child span under that load's `page.load` for each
|
||||
same-origin `/api/` `fetch`/XHR Resource Timing entry that started inside the
|
||||
load's window (and before the SDK started tracing requests live, so nothing is
|
||||
counted twice). Each runs from the entry's `startTime` to its `responseEnd`.
|
||||
They differ from live fetch spans in three ways:
|
||||
|
||||
- The name is `HTTP`, OpenTelemetry's name for a request whose method is
|
||||
unknown: Resource Timing does not expose the method, so there is no
|
||||
`http.request.method`.
|
||||
- `http.response.status_code` is present only where the browser exposes
|
||||
`responseStatus` (Chromium); it is never guessed.
|
||||
- No `traceparent` was sent with these requests, so they do not join the
|
||||
server's spans. Use `paperclip.api.area` and the timings; for the server
|
||||
side, query server spans by time.
|
||||
|
||||
They carry `paperclip.api.timing_source = resource_timing`. The URL is reduced
|
||||
to `paperclip.api.area` by the same sanitizer as live fetch spans.
|
||||
|
||||
### Attributes
|
||||
|
||||
Attributes (the complete allowlist):
|
||||
|
||||
@@ -147,16 +225,19 @@ Attributes (the complete allowlist):
|
||||
| --- | --- | --- |
|
||||
| `paperclip.page.route` | all | Route pattern, for example `/:companyPrefix/issues/:issueId`; `(unmatched)` when no route matches. Built only from route definitions, never from the URL. On fetch spans: the route current when the request started. On `web_vital.lcp`: the hard load's route. |
|
||||
| `paperclip.page.load_type` | `page.load` | `hard`, `client` |
|
||||
| `paperclip.page.outcome` | `page.load` | `ready`, `interrupted`, `timeout` |
|
||||
| `paperclip.page.outcome` | `page.load` | `ready`, `interrupted`, `hidden`, `timeout` |
|
||||
| `paperclip.page.ready_signal` | `page.load` | `content`, `idle`, `none` |
|
||||
| `paperclip.page.was_hidden` | `page.load` | boolean |
|
||||
| `paperclip.page.suspended` | `page.load` | `true` when the computer slept or the tab froze during the measurement; absent otherwise |
|
||||
| `paperclip.page.view` | `page.load` on pages with views | The view from `?view=`, from a fixed list per page, else `other`; absent on other pages |
|
||||
| `paperclip.page.navigation_type` | hard `page.load` | `navigate`, `reload`, `back_forward`, `prerender` |
|
||||
| `paperclip.page.ttfb_ms`, `paperclip.page.dom_content_loaded_ms`, `paperclip.page.load_event_ms` | hard `page.load` | Navigation Timing milestones, ms from navigation start |
|
||||
| `paperclip.web_vital.name` | `web_vital.*` | `LCP`, `INP`, `CLS` |
|
||||
| `paperclip.web_vital.value` | `web_vital.*` | ms for LCP and INP; unitless for CLS |
|
||||
| `paperclip.web_vital.rating` | `web_vital.*` | `good`, `needs-improvement`, `poor` |
|
||||
| `paperclip.api.area` | fetch | First path segment after `/api/` when it is a fixed word (`issues`, `companies`, …), else `other`. Use the joined server span's route for detail. |
|
||||
| `http.request.method`, `http.response.status_code`, `error.type` | fetch | OpenTelemetry HTTP semantics |
|
||||
| `paperclip.api.timing_source` | backfilled `HTTP` | `resource_timing` |
|
||||
| `http.request.method`, `http.response.status_code`, `error.type` | fetch | OpenTelemetry HTTP semantics (backfilled spans: status only, where the browser exposes it) |
|
||||
| `SampleRate` | all, when sampling | 1-in-N weight |
|
||||
|
||||
Resource: `service.name`, `service.version` (the UI build commit),
|
||||
@@ -176,17 +257,20 @@ when server tracing is enabled ([observability.md](observability.md#enabling-tra
|
||||
## Example Honeycomb queries
|
||||
|
||||
Use the `paperclip-web` dataset (or your `service.name`) and the environment
|
||||
that holds browser data. Exclude hidden tabs and unfinished loads.
|
||||
that holds browser data. Load-time percentiles filter on `outcome = ready`,
|
||||
which leaves out hidden tabs, timeouts, and interruptions. Also leave out
|
||||
suspended spans: `paperclip.page.suspended` is only ever set to `true`, so
|
||||
filter on it not existing.
|
||||
|
||||
**P95 load time of the all-tasks page**
|
||||
**P95 load time of the all-tasks page, by view**
|
||||
|
||||
```text
|
||||
VISUALIZE P95(duration_ms), COUNT
|
||||
WHERE name = page.load
|
||||
paperclip.page.route = /:companyPrefix/issues
|
||||
paperclip.page.outcome = ready
|
||||
paperclip.page.was_hidden = false
|
||||
GROUP BY paperclip.page.load_type
|
||||
paperclip.page.suspended does-not-exist
|
||||
GROUP BY paperclip.page.load_type, paperclip.page.view
|
||||
```
|
||||
|
||||
**P95 load time of an individual task page**
|
||||
@@ -196,12 +280,22 @@ VISUALIZE P95(duration_ms), COUNT
|
||||
WHERE name = page.load
|
||||
paperclip.page.route = /:companyPrefix/issues/:issueId
|
||||
paperclip.page.outcome = ready
|
||||
paperclip.page.was_hidden = false
|
||||
paperclip.page.suspended does-not-exist
|
||||
GROUP BY paperclip.page.load_type
|
||||
```
|
||||
|
||||
**API requests during hard loads of the all-tasks page**
|
||||
|
||||
```text
|
||||
VISUALIZE P95(duration_ms), COUNT
|
||||
WHERE paperclip.api.timing_source = resource_timing
|
||||
paperclip.page.route = /:companyPrefix/issues
|
||||
GROUP BY paperclip.api.area
|
||||
```
|
||||
|
||||
Group by `service.version` to compare builds. For a timeout rate, count
|
||||
`paperclip.page.outcome = timeout` against all `page.load` spans for the route.
|
||||
`paperclip.page.outcome = timeout` against `page.load` spans for the route
|
||||
whose outcome is not `hidden`.
|
||||
For Web Vitals, `P75(paperclip.web_vital.value)` where
|
||||
`name = web_vital.lcp` (or `inp`, `cls`), grouped by `paperclip.page.route`.
|
||||
|
||||
@@ -211,11 +305,13 @@ For Web Vitals, `P75(paperclip.web_vital.value)` where
|
||||
```text
|
||||
IF(AND(EQUALS($name, "page.load"),
|
||||
EQUALS($paperclip.page.route, "/:companyPrefix/issues/:issueId"),
|
||||
EQUALS($paperclip.page.was_hidden, false)),
|
||||
IN($paperclip.page.outcome, "ready", "timeout"),
|
||||
NOT(EQUALS($paperclip.page.suspended, true))),
|
||||
AND(EQUALS($paperclip.page.outcome, "ready"), LTE($duration_ms, 2000)))
|
||||
```
|
||||
|
||||
The expression is null for other events, so they do not count.
|
||||
The expression is null for other events (including `hidden` and
|
||||
`interrupted` loads), so they do not count; a timeout counts as a failure.
|
||||
|
||||
## Verifying a deployment
|
||||
|
||||
|
||||
@@ -68,6 +68,17 @@ const routes = (
|
||||
</>
|
||||
);
|
||||
|
||||
/**
|
||||
* The page's clock starts at navigation start. A test process can be older
|
||||
* than the 30 s page-load timeout, which would turn the hard load (start 0)
|
||||
* into a timeout, so the clock is restarted for each test.
|
||||
*/
|
||||
function restartPageClock() {
|
||||
const realNow = performance.now.bind(performance);
|
||||
const origin = realNow();
|
||||
vi.spyOn(performance, "now").mockImplementation(() => realNow() - origin);
|
||||
}
|
||||
|
||||
describe("PageLoadTracker", () => {
|
||||
let container: HTMLDivElement;
|
||||
let root: Root | null = null;
|
||||
@@ -81,6 +92,7 @@ describe("PageLoadTracker", () => {
|
||||
measurements.ended.length = 0;
|
||||
resetPageSessionSampleForTest();
|
||||
window.history.replaceState(null, "", "/PAP/issues");
|
||||
restartPageClock();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
@@ -88,6 +100,7 @@ describe("PageLoadTracker", () => {
|
||||
root = null;
|
||||
container.remove();
|
||||
vi.clearAllMocks();
|
||||
vi.mocked(performance.now).mockRestore?.();
|
||||
});
|
||||
|
||||
async function render(initialPath: string) {
|
||||
|
||||
@@ -5,6 +5,7 @@ import { QueryClient } from "@tanstack/react-query";
|
||||
import type { BrowserTracingConfig } from "@paperclipai/shared";
|
||||
import type { PageLoadMeasurement } from "./page-load-recorder";
|
||||
import {
|
||||
observeApiResources,
|
||||
resetPageSessionSampleForTest,
|
||||
SDK_LOAD_DEADLINE_MS,
|
||||
startBrowserTracingController,
|
||||
@@ -19,14 +20,101 @@ const config: BrowserTracingConfig = {
|
||||
sampleRate: 1,
|
||||
};
|
||||
|
||||
/**
|
||||
* The page's clock starts at navigation start. A test process can be older
|
||||
* than the 30 s page-load timeout, which would turn the hard load (start 0)
|
||||
* into a timeout, so the clock is restarted for each test.
|
||||
*/
|
||||
function restartPageClock() {
|
||||
const realNow = performance.now.bind(performance);
|
||||
const origin = realNow();
|
||||
vi.spyOn(performance, "now").mockImplementation(() => realNow() - origin);
|
||||
}
|
||||
|
||||
describe("startBrowserTracingController", () => {
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout", "requestAnimationFrame", "cancelAnimationFrame"] });
|
||||
resetPageSessionSampleForTest();
|
||||
restartPageClock();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
vi.restoreAllMocks();
|
||||
setVisibility("visible");
|
||||
});
|
||||
|
||||
function setVisibility(state: DocumentVisibilityState) {
|
||||
Object.defineProperty(document, "visibilityState", { configurable: true, get: () => state });
|
||||
}
|
||||
|
||||
function hangingQueryClient() {
|
||||
const queryClient = new QueryClient();
|
||||
void queryClient.fetchQuery({ queryKey: ["hangs"], queryFn: () => new Promise(() => {}) });
|
||||
return queryClient;
|
||||
}
|
||||
|
||||
function recordingSdk(ended: PageLoadMeasurement[]) {
|
||||
return vi.fn(async (): Promise<TracingSdkModule> => ({
|
||||
startTracingSdk: (_config, recorder) => {
|
||||
recorder.attachSink({ begin: () => {}, end: (m) => ended.push({ ...m }) });
|
||||
return { forceFlush: async () => {}, shutdown: async () => {} };
|
||||
},
|
||||
}));
|
||||
}
|
||||
|
||||
it("ends a hard load in a background tab as hidden instead of a 30 s timeout", async () => {
|
||||
setVisibility("hidden");
|
||||
const ended: PageLoadMeasurement[] = [];
|
||||
const controller = startBrowserTracingController(config, {
|
||||
queryClient: hangingQueryClient(),
|
||||
initialPathname: "/PAP/issues",
|
||||
initialRouteTemplate: "/:companyPrefix/issues",
|
||||
loadSdk: recordingSdk(ended),
|
||||
scheduleSdkLoad: () => () => {},
|
||||
});
|
||||
await vi.advanceTimersByTimeAsync(SDK_LOAD_DEADLINE_MS);
|
||||
await vi.waitFor(() => expect(ended).toHaveLength(1));
|
||||
expect(ended[0]).toMatchObject({ loadType: "hard", outcome: "hidden", startTime: 0, endTime: 0, wasHidden: true });
|
||||
controller?.stop();
|
||||
});
|
||||
|
||||
it("ends an open load as hidden when the tab hides", async () => {
|
||||
const ended: PageLoadMeasurement[] = [];
|
||||
const controller = startBrowserTracingController(config, {
|
||||
queryClient: hangingQueryClient(),
|
||||
initialPathname: "/PAP/issues",
|
||||
initialRouteTemplate: "/:companyPrefix/issues",
|
||||
loadSdk: recordingSdk(ended),
|
||||
scheduleSdkLoad: () => () => {},
|
||||
});
|
||||
await vi.advanceTimersByTimeAsync(1_000);
|
||||
setVisibility("hidden");
|
||||
const hiddenAt = performance.now();
|
||||
document.dispatchEvent(new Event("visibilitychange"));
|
||||
await vi.advanceTimersByTimeAsync(SDK_LOAD_DEADLINE_MS);
|
||||
await vi.waitFor(() => expect(ended).toHaveLength(1));
|
||||
expect(ended[0]!.outcome).toBe("hidden");
|
||||
expect(ended[0]!.endTime).toBeGreaterThanOrEqual(hiddenAt);
|
||||
expect(ended[0]!.endTime).toBeLessThan(hiddenAt + 1_000);
|
||||
controller?.stop();
|
||||
});
|
||||
|
||||
it("records the all-tasks page view from the URL", async () => {
|
||||
window.history.replaceState(null, "", "/PAP/issues?view=mine&q=private-search");
|
||||
const ended: PageLoadMeasurement[] = [];
|
||||
const controller = startBrowserTracingController(config, {
|
||||
queryClient: hangingQueryClient(),
|
||||
initialPathname: "/PAP/issues",
|
||||
initialRouteTemplate: "/:companyPrefix/issues",
|
||||
loadSdk: recordingSdk(ended),
|
||||
scheduleSdkLoad: () => () => {},
|
||||
});
|
||||
await vi.advanceTimersByTimeAsync(SDK_LOAD_DEADLINE_MS);
|
||||
await vi.waitFor(() => expect(ended).toHaveLength(1));
|
||||
expect(ended[0]!.view).toBe("mine");
|
||||
controller?.stop();
|
||||
window.history.replaceState(null, "", "/");
|
||||
});
|
||||
|
||||
it("still loads the SDK and exports a timed-out hard load when the page never settles", async () => {
|
||||
@@ -79,3 +167,58 @@ describe("startBrowserTracingController", () => {
|
||||
controller?.stop();
|
||||
});
|
||||
});
|
||||
|
||||
describe("observeApiResources", () => {
|
||||
type Listener = (list: { getEntries(): PerformanceEntry[] }) => void;
|
||||
|
||||
function fakeObserver() {
|
||||
const state: { listener: Listener | null; options: PerformanceObserverInit | null; disconnected: boolean } = {
|
||||
listener: null,
|
||||
options: null,
|
||||
disconnected: false,
|
||||
};
|
||||
class FakeObserver {
|
||||
static supportedEntryTypes = ["resource"];
|
||||
constructor(listener: Listener) {
|
||||
state.listener = listener;
|
||||
}
|
||||
observe(options: PerformanceObserverInit) {
|
||||
state.options = options;
|
||||
}
|
||||
disconnect() {
|
||||
state.disconnected = true;
|
||||
}
|
||||
}
|
||||
return { state, Observer: FakeObserver as unknown as typeof PerformanceObserver };
|
||||
}
|
||||
|
||||
const entry = (name: string, initiatorType = "fetch") => ({ name, initiatorType, entryType: "resource", startTime: 5 }) as unknown as PerformanceEntry;
|
||||
|
||||
it("keeps only this origin's /api/ fetches, including ones already buffered", () => {
|
||||
const { state, Observer } = fakeObserver();
|
||||
const collector = observeApiResources("https://acme.paperclip.app", Observer);
|
||||
expect(state.options).toEqual({ type: "resource", buffered: true });
|
||||
state.listener!({
|
||||
getEntries: () => [
|
||||
entry("https://acme.paperclip.app/api/issues?view=mine"),
|
||||
entry("https://acme.paperclip.app/api/companies", "xmlhttprequest"),
|
||||
entry("https://acme.paperclip.app/assets/index.js", "script"),
|
||||
entry("https://acme.paperclip.app.evil.com/api/issues"),
|
||||
entry("https://api.honeycomb.io/v1/traces"),
|
||||
],
|
||||
});
|
||||
expect(collector.entries().map((e) => e.name)).toEqual([
|
||||
"https://acme.paperclip.app/api/issues?view=mine",
|
||||
"https://acme.paperclip.app/api/companies",
|
||||
]);
|
||||
collector.stop();
|
||||
expect(state.disconnected).toBe(true);
|
||||
expect(collector.entries()).toEqual([]);
|
||||
});
|
||||
|
||||
it("does nothing where resource timing is unavailable", () => {
|
||||
const collector = observeApiResources("https://acme.paperclip.app", null);
|
||||
expect(collector.entries()).toEqual([]);
|
||||
collector.stop();
|
||||
});
|
||||
});
|
||||
@@ -6,7 +6,10 @@
|
||||
// - decides once per page session whether this session is sampled; an
|
||||
// unsampled session does nothing else and downloads nothing,
|
||||
// - measures the hard load and every client-side navigation
|
||||
// (`page-load-recorder.ts`), and
|
||||
// (`page-load-recorder.ts`),
|
||||
// - keeps the Resource Timing entries of this origin's `/api/` requests
|
||||
// until the SDK loads, so the SDK can show the hard load's API requests,
|
||||
// and
|
||||
// - after the page settles, downloads the tracing SDK chunk (`sdk.ts`),
|
||||
// which exports the measurements, Web Vitals, and API request spans.
|
||||
// See doc/browser-performance-tracing.md.
|
||||
@@ -16,6 +19,7 @@ import type { BrowserTracingConfig } from "@paperclipai/shared";
|
||||
import { whenPageSettles } from "@/lib/when-page-settles";
|
||||
import { readPageContentGates } from "./page-content-gates";
|
||||
import { createPageLoadRecorder, PAGE_LOAD_TIMEOUT_MS, type PageLoadRecorder } from "./page-load-recorder";
|
||||
import { pageViewFor } from "./page-view";
|
||||
|
||||
export type RouterNavigationType = "PUSH" | "POP" | "REPLACE";
|
||||
|
||||
@@ -30,8 +34,19 @@ export interface TracingSdkHandle {
|
||||
shutdown(): Promise<void>;
|
||||
}
|
||||
|
||||
/** Resource Timing entries for this origin's `/api/` requests, kept until the SDK takes them. */
|
||||
export interface ApiResourceCollector {
|
||||
entries(): readonly PerformanceResourceTiming[];
|
||||
/** Stops observing and drops the entries. */
|
||||
stop(): void;
|
||||
}
|
||||
|
||||
export interface TracingSdkModule {
|
||||
startTracingSdk(config: BrowserTracingConfig, recorder: PageLoadRecorder): TracingSdkHandle;
|
||||
startTracingSdk(
|
||||
config: BrowserTracingConfig,
|
||||
recorder: PageLoadRecorder,
|
||||
options?: { apiResources?: ApiResourceCollector },
|
||||
): TracingSdkHandle;
|
||||
}
|
||||
|
||||
export interface BrowserTracingControllerOptions {
|
||||
@@ -71,6 +86,67 @@ export function hasPendingPrimaryData(queryClient: QueryClient): boolean {
|
||||
return queryClient.isFetching({ predicate: (query) => query.state.data === undefined }) > 0;
|
||||
}
|
||||
|
||||
/** At most this many entries are kept; a hard load makes far fewer requests. */
|
||||
const API_RESOURCE_LIMIT = 500;
|
||||
|
||||
/**
|
||||
* Collects Resource Timing entries for `fetch`/XHR requests to `origin/api/`.
|
||||
* `buffered: true` delivers the requests that finished before this ran (the
|
||||
* hard load's session request, for example), and an observer keeps receiving
|
||||
* entries after the browser's resource buffer (250 entries by default) is
|
||||
* full. One observer callback per batch of finished requests; nothing else.
|
||||
*/
|
||||
export function observeApiResources(
|
||||
origin: string,
|
||||
Observer: typeof PerformanceObserver | null = typeof PerformanceObserver === "function" ? PerformanceObserver : null,
|
||||
): ApiResourceCollector {
|
||||
let kept: PerformanceResourceTiming[] = [];
|
||||
if (!Observer || !(Observer.supportedEntryTypes ?? []).includes("resource")) {
|
||||
return { entries: () => kept, stop: () => {} };
|
||||
}
|
||||
const prefix = `${origin}/api/`;
|
||||
let observer: PerformanceObserver | null = null;
|
||||
try {
|
||||
observer = new Observer((list) => {
|
||||
for (const entry of list.getEntries() as PerformanceResourceTiming[]) {
|
||||
if (kept.length >= API_RESOURCE_LIMIT) break;
|
||||
if ((entry.initiatorType === "fetch" || entry.initiatorType === "xmlhttprequest") && entry.name.startsWith(prefix)) {
|
||||
kept.push(entry);
|
||||
}
|
||||
}
|
||||
});
|
||||
observer.observe({ type: "resource", buffered: true });
|
||||
} catch {
|
||||
observer = null;
|
||||
}
|
||||
return {
|
||||
entries: () => kept,
|
||||
stop() {
|
||||
observer?.disconnect();
|
||||
observer = null;
|
||||
kept = [];
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* When the tab was first hidden since navigation start, or null if it has
|
||||
* been visible throughout. Chromium records each visibility change as a
|
||||
* `visibility-state` performance entry (including the initial state); where
|
||||
* it does not, a tab that is hidden now counts as hidden from the start, as
|
||||
* `web-vitals` assumes.
|
||||
*/
|
||||
function firstHiddenTime(): number | null {
|
||||
try {
|
||||
const entries = performance.getEntriesByType?.("visibility-state") ?? [];
|
||||
const hidden = entries.find((entry) => entry.name === "hidden");
|
||||
if (hidden) return hidden.startTime;
|
||||
} catch {
|
||||
// Not supported.
|
||||
}
|
||||
return document.visibilityState === "hidden" ? 0 : null;
|
||||
}
|
||||
|
||||
function pathnameOf(url: string | URL | null | undefined): string | null {
|
||||
if (url === null || url === undefined) return null;
|
||||
try {
|
||||
@@ -136,15 +212,17 @@ export function startBrowserTracingController(
|
||||
isDocumentHidden: () => document.visibilityState === "hidden",
|
||||
hasPendingPrimaryData: () => hasPendingPrimaryData(options.queryClient),
|
||||
readGates: readPageContentGates,
|
||||
readView: (routeTemplate) => pageViewFor(routeTemplate, window.location.search),
|
||||
});
|
||||
const apiResources = observeApiResources(window.location.origin);
|
||||
const history = trackHistoryChanges(now);
|
||||
const onVisibilityChange = () => {
|
||||
if (document.visibilityState === "hidden") recorder.noteHidden();
|
||||
if (document.visibilityState === "hidden") recorder.noteHidden(now());
|
||||
};
|
||||
document.addEventListener("visibilitychange", onVisibilityChange);
|
||||
|
||||
// The hard load started at navigation start (time 0), before this code ran.
|
||||
recorder.begin({ routeTemplate: options.initialRouteTemplate, loadType: "hard", startTime: 0 });
|
||||
recorder.begin({ routeTemplate: options.initialRouteTemplate, loadType: "hard", startTime: 0, hiddenAt: firstHiddenTime() });
|
||||
let lastPathname = options.initialPathname;
|
||||
|
||||
let stopped = false;
|
||||
@@ -161,10 +239,11 @@ export function startBrowserTracingController(
|
||||
void loadSdk()
|
||||
.then((module) => {
|
||||
if (stopped) return;
|
||||
sdk = module.startTracingSdk(config, recorder);
|
||||
sdk = module.startTracingSdk(config, recorder, { apiResources });
|
||||
})
|
||||
.catch(() => {
|
||||
// Tracing is optional; a failed chunk load must not affect the page.
|
||||
apiResources.stop();
|
||||
});
|
||||
};
|
||||
const deadline = window.setTimeout(startLoad, options.sdkLoadDeadlineMs ?? SDK_LOAD_DEADLINE_MS);
|
||||
@@ -191,6 +270,7 @@ export function startBrowserTracingController(
|
||||
history.stop();
|
||||
document.removeEventListener("visibilitychange", onVisibilityChange);
|
||||
recorder.stop();
|
||||
apiResources.stop();
|
||||
void sdk?.shutdown();
|
||||
sdk = null;
|
||||
},
|
||||
|
||||
@@ -10,6 +10,7 @@ function createHarness() {
|
||||
const state = {
|
||||
pendingData: false,
|
||||
hidden: false,
|
||||
view: null as string | null,
|
||||
gates: { blockingCount: 0, contentCount: 0, pendingContentCount: 0, lastContentReadyAt: null } as PageContentGateSnapshot,
|
||||
};
|
||||
const recorder = createPageLoadRecorder({
|
||||
@@ -32,6 +33,7 @@ function createHarness() {
|
||||
isDocumentHidden: () => state.hidden,
|
||||
hasPendingPrimaryData: () => state.pendingData,
|
||||
readGates: () => state.gates,
|
||||
readView: () => state.view,
|
||||
});
|
||||
const ended: PageLoadMeasurement[] = [];
|
||||
const begun: PageLoadMeasurement[] = [];
|
||||
@@ -58,6 +60,10 @@ function createHarness() {
|
||||
}
|
||||
}
|
||||
},
|
||||
/** Move the clock without firing timers or frames, as a suspended computer does. */
|
||||
jump(ms: number) {
|
||||
time += ms;
|
||||
},
|
||||
advance(ms: number) {
|
||||
time += ms;
|
||||
for (const [id, timer] of timers) {
|
||||
@@ -132,6 +138,17 @@ describe("createPageLoadRecorder", () => {
|
||||
expect(harness.recorder.open()?.routeTemplate).toBe("/b");
|
||||
});
|
||||
|
||||
it("ends an interrupted measurement where the next one starts, so their windows never overlap", () => {
|
||||
const harness = createHarness();
|
||||
harness.recorder.attachSink(harness.sink);
|
||||
harness.state.pendingData = true;
|
||||
harness.recorder.begin({ routeTemplate: "/a", loadType: "client", startTime: 0 });
|
||||
harness.advance(250);
|
||||
// The router commits after the history change that started the navigation.
|
||||
harness.recorder.begin({ routeTemplate: "/b", loadType: "client", startTime: 200 });
|
||||
expect(harness.ended[0]).toMatchObject({ routeTemplate: "/a", outcome: "interrupted", endTime: 200 });
|
||||
});
|
||||
|
||||
it("continues a measurement through a redirect under the new route", () => {
|
||||
const harness = createHarness();
|
||||
harness.recorder.attachSink(harness.sink);
|
||||
@@ -154,14 +171,118 @@ describe("createPageLoadRecorder", () => {
|
||||
expect(harness.ended[0]).toMatchObject({ outcome: "timeout", endTime: 30_000, readySignal: null });
|
||||
});
|
||||
|
||||
it("marks a measurement whose tab was hidden", () => {
|
||||
it("ends a measurement as hidden, at the moment the tab hid, instead of waiting for frames that never come", () => {
|
||||
const harness = createHarness();
|
||||
harness.recorder.attachSink(harness.sink);
|
||||
harness.state.pendingData = true;
|
||||
harness.recorder.begin({ routeTemplate: "/a", loadType: "client", startTime: 0 });
|
||||
harness.recorder.noteHidden();
|
||||
harness.advance(400);
|
||||
harness.state.hidden = true;
|
||||
harness.recorder.noteHidden(400);
|
||||
expect(harness.ended).toEqual([expect.objectContaining({ outcome: "hidden", endTime: 400, readySignal: null, wasHidden: true })]);
|
||||
harness.advance(30_000);
|
||||
expect(harness.ended).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("ends a load that starts in a hidden tab immediately as hidden, never as a 30 s timeout", () => {
|
||||
const harness = createHarness();
|
||||
harness.recorder.attachSink(harness.sink);
|
||||
harness.state.hidden = true;
|
||||
harness.state.pendingData = true;
|
||||
harness.advance(900);
|
||||
harness.recorder.begin({ routeTemplate: "/", loadType: "hard", startTime: 0, hiddenAt: 0 });
|
||||
expect(harness.ended).toEqual([expect.objectContaining({ outcome: "hidden", endTime: 0, loadType: "hard" })]);
|
||||
harness.advance(30_000);
|
||||
expect(harness.ended).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("ends a hard load that was hidden before tracing started at the time it hid", () => {
|
||||
const harness = createHarness();
|
||||
harness.recorder.attachSink(harness.sink);
|
||||
harness.advance(900);
|
||||
// Visible again by now, but rendering was paused while it was hidden.
|
||||
harness.recorder.begin({ routeTemplate: "/a", loadType: "hard", startTime: 0, hiddenAt: 350 });
|
||||
expect(harness.ended[0]).toMatchObject({ outcome: "hidden", endTime: 350 });
|
||||
});
|
||||
|
||||
it("keeps a redirect after a hidden hard load on the same measurement", () => {
|
||||
const harness = createHarness();
|
||||
harness.state.hidden = true;
|
||||
harness.recorder.begin({ routeTemplate: "/", loadType: "hard", startTime: 0, hiddenAt: 0 });
|
||||
expect(harness.recorder.redirect("/:companyPrefix/dashboard")).toBe(true);
|
||||
harness.recorder.attachSink(harness.sink);
|
||||
expect(harness.ended).toEqual([expect.objectContaining({ routeTemplate: "/:companyPrefix/dashboard", outcome: "hidden" })]);
|
||||
expect(harness.recorder.hardLoadRouteTemplate()).toBe("/:companyPrefix/dashboard");
|
||||
});
|
||||
|
||||
it("ends a timeout at start + timeout however late the timer fires, and marks a suspend", () => {
|
||||
const harness = createHarness();
|
||||
harness.recorder.attachSink(harness.sink);
|
||||
harness.state.pendingData = true;
|
||||
harness.recorder.begin({ routeTemplate: "/slow", loadType: "hard", startTime: 0 });
|
||||
// The computer slept: the 30 s timer fires about two hours later.
|
||||
harness.advance(7_758_250);
|
||||
expect(harness.ended).toEqual([expect.objectContaining({ outcome: "timeout", endTime: 30_000, suspended: true })]);
|
||||
});
|
||||
|
||||
it("does not mark an ordinarily late timeout as suspended", () => {
|
||||
const harness = createHarness();
|
||||
harness.recorder.attachSink(harness.sink);
|
||||
harness.state.pendingData = true;
|
||||
harness.recorder.begin({ routeTemplate: "/slow", loadType: "client", startTime: 0 });
|
||||
harness.advance(30_400);
|
||||
expect(harness.ended[0]).toMatchObject({ outcome: "timeout", endTime: 30_000, suspended: false });
|
||||
});
|
||||
|
||||
it("reports a load that only became ready after the deadline, across a suspend, as a capped timeout", () => {
|
||||
const harness = createHarness();
|
||||
harness.recorder.attachSink(harness.sink);
|
||||
harness.state.pendingData = true;
|
||||
harness.recorder.begin({ routeTemplate: "/a", loadType: "client", startTime: 0 });
|
||||
harness.frame();
|
||||
harness.jump(2 * 60 * 60 * 1000);
|
||||
harness.state.pendingData = false;
|
||||
// Frames resume before the overdue timer runs.
|
||||
harness.frame();
|
||||
harness.frame();
|
||||
expect(harness.ended[0]).toMatchObject({ wasHidden: true });
|
||||
expect(harness.ended).toEqual([expect.objectContaining({ outcome: "timeout", endTime: 30_000, readySignal: null, suspended: true })]);
|
||||
});
|
||||
|
||||
it("marks a ready load that spanned a suspend shorter than the timeout", () => {
|
||||
const harness = createHarness();
|
||||
harness.recorder.attachSink(harness.sink);
|
||||
harness.state.pendingData = true;
|
||||
harness.recorder.begin({ routeTemplate: "/a", loadType: "client", startTime: 0 });
|
||||
harness.frame();
|
||||
harness.jump(20_000);
|
||||
harness.state.pendingData = false;
|
||||
harness.frame();
|
||||
harness.frame();
|
||||
expect(harness.ended[0]).toMatchObject({ outcome: "ready", suspended: true });
|
||||
});
|
||||
|
||||
it("records the page view, read again when the measurement ends", () => {
|
||||
const harness = createHarness();
|
||||
harness.recorder.attachSink(harness.sink);
|
||||
harness.state.view = "default";
|
||||
harness.recorder.begin({ routeTemplate: "/:companyPrefix/issues", loadType: "hard", startTime: 0 });
|
||||
expect(harness.begun[0]).toMatchObject({ view: "default" });
|
||||
// The page writes the resolved view into the URL while loading.
|
||||
harness.state.view = "mine";
|
||||
harness.frame();
|
||||
harness.frame();
|
||||
expect(harness.ended[0]).toMatchObject({ view: "mine", suspended: false });
|
||||
});
|
||||
|
||||
it("keeps the view an interrupted measurement had, not the next page's", () => {
|
||||
const harness = createHarness();
|
||||
harness.recorder.attachSink(harness.sink);
|
||||
harness.state.pendingData = true;
|
||||
harness.state.view = "mine";
|
||||
harness.recorder.begin({ routeTemplate: "/:companyPrefix/issues", loadType: "client", startTime: 0 });
|
||||
harness.state.view = null;
|
||||
harness.recorder.begin({ routeTemplate: "/b", loadType: "client", startTime: 10 });
|
||||
expect(harness.ended[0]).toMatchObject({ outcome: "interrupted", view: "mine" });
|
||||
});
|
||||
|
||||
it("buffers measurements that finish before the SDK attaches and replays them in order", () => {
|
||||
|
||||
@@ -12,13 +12,26 @@
|
||||
// (the end time is the latest gate's paint time), otherwise
|
||||
// 2b. no query that has no data yet is fetching (the end time is the first
|
||||
// such frame).
|
||||
// A new navigation interrupts an open one. A measurement that has not ended
|
||||
// after `timeoutMs` ends with outcome `timeout`.
|
||||
// A new navigation interrupts an open one.
|
||||
//
|
||||
// A measurement in a tab that is hidden when it starts, or that hides before
|
||||
// it is ready, ends with outcome `hidden` at the moment the tab hid. Browsers
|
||||
// stop painting hidden tabs, so it could never become ready; the same rule as
|
||||
// `web-vitals`, which stops reporting LCP and FCP once the page has been
|
||||
// hidden.
|
||||
//
|
||||
// A measurement that has not ended `timeoutMs` after it started ends with
|
||||
// outcome `timeout`, and its end time is exactly start + `timeoutMs`. Any end
|
||||
// observed after that deadline (a timer that fired late, a frame that ran
|
||||
// after the computer woke up) is reported as that timeout, never as a longer
|
||||
// duration. When the observed end, or a gap between two frames, is more than
|
||||
// `SUSPEND_GRACE_MS` late, the measurement is marked `suspended`: the
|
||||
// computer slept or the tab was frozen while it was open.
|
||||
|
||||
import type { PageContentGateSnapshot } from "./page-content-gates";
|
||||
|
||||
export type PageLoadType = "hard" | "client";
|
||||
export type PageLoadOutcome = "ready" | "timeout" | "interrupted";
|
||||
export type PageLoadOutcome = "ready" | "timeout" | "interrupted" | "hidden";
|
||||
export type PageLoadReadySignal = "content" | "idle";
|
||||
|
||||
export interface PageLoadMeasurement {
|
||||
@@ -32,6 +45,10 @@ export interface PageLoadMeasurement {
|
||||
readySignal: PageLoadReadySignal | null;
|
||||
/** The tab was hidden at some point during the measurement. */
|
||||
wasHidden: boolean;
|
||||
/** The computer slept or the tab froze while the measurement was open. */
|
||||
suspended: boolean;
|
||||
/** The page's `?view=`, from an allowlist (`page-view.ts`); null for pages without views. */
|
||||
view: string | null;
|
||||
}
|
||||
|
||||
/** Receives measurements; implemented by the lazily loaded tracing SDK. */
|
||||
@@ -49,6 +66,8 @@ export interface PageLoadRecorderDeps {
|
||||
isDocumentHidden(): boolean;
|
||||
hasPendingPrimaryData(): boolean;
|
||||
readGates(): PageContentGateSnapshot;
|
||||
/** The current page view for `routeTemplate` (`pageViewFor`). */
|
||||
readView?(routeTemplate: string): string | null;
|
||||
}
|
||||
|
||||
export interface PageLoadRecorderOptions {
|
||||
@@ -59,21 +78,29 @@ export interface PageLoadRecorderOptions {
|
||||
}
|
||||
|
||||
export interface PageLoadRecorder {
|
||||
begin(input: { routeTemplate: string; loadType: PageLoadType; startTime: number }): PageLoadMeasurement;
|
||||
/**
|
||||
* `hiddenAt`: when the tab was first hidden after `startTime`, if it was
|
||||
* hidden before this call (a hard load opened in a background tab).
|
||||
*/
|
||||
begin(input: { routeTemplate: string; loadType: PageLoadType; startTime: number; hiddenAt?: number | null }): PageLoadMeasurement;
|
||||
/**
|
||||
* A redirect (history replace) while a measurement is open continues it
|
||||
* under the new route. Returns false when no measurement is open.
|
||||
* under the new route, as does one right after a measurement that ended
|
||||
* hidden and has not been exported yet. Returns false otherwise.
|
||||
*/
|
||||
redirect(routeTemplate: string): boolean;
|
||||
open(): PageLoadMeasurement | null;
|
||||
currentRouteTemplate(): string | null;
|
||||
hardLoadRouteTemplate(): string | null;
|
||||
noteHidden(): void;
|
||||
/** The tab became hidden at `time`: an open measurement ends as `hidden`. */
|
||||
noteHidden(time: number): void;
|
||||
attachSink(sink: PageLoadSink): void;
|
||||
stop(): void;
|
||||
}
|
||||
|
||||
export const PAGE_LOAD_TIMEOUT_MS = 30_000;
|
||||
/** An end, or a frame, this much later than expected means the computer slept. */
|
||||
export const SUSPEND_GRACE_MS = 5_000;
|
||||
|
||||
export function createPageLoadRecorder(
|
||||
deps: PageLoadRecorderDeps,
|
||||
@@ -93,6 +120,8 @@ export function createPageLoadRecorder(
|
||||
let timer: unknown = null;
|
||||
let streak = 0;
|
||||
let streakStart = 0;
|
||||
let lastFrameAt: number | null = null;
|
||||
let lastEnded: PageLoadMeasurement | null = null;
|
||||
const buffered: PageLoadMeasurement[] = [];
|
||||
|
||||
function cancelScheduled() {
|
||||
@@ -102,14 +131,31 @@ export function createPageLoadRecorder(
|
||||
timer = null;
|
||||
}
|
||||
|
||||
function readView(routeTemplate: string): string | null {
|
||||
return deps.readView ? deps.readView(routeTemplate) : null;
|
||||
}
|
||||
|
||||
function finish(measurement: PageLoadMeasurement, outcome: PageLoadOutcome, endTime: number, signal: PageLoadReadySignal | null) {
|
||||
if (current === measurement) {
|
||||
current = null;
|
||||
cancelScheduled();
|
||||
}
|
||||
const deadline = measurement.startTime + timeoutMs;
|
||||
if (endTime > deadline) {
|
||||
// Not ready by the deadline, whatever happened after it.
|
||||
if (endTime - deadline > SUSPEND_GRACE_MS) measurement.suspended = true;
|
||||
outcome = "timeout";
|
||||
signal = null;
|
||||
endTime = deadline;
|
||||
}
|
||||
if (outcome === "hidden") measurement.wasHidden = true;
|
||||
// An interrupted measurement keeps the view it had: the URL already
|
||||
// belongs to the next page.
|
||||
if (outcome !== "interrupted") measurement.view = readView(measurement.routeTemplate);
|
||||
measurement.outcome = outcome;
|
||||
measurement.readySignal = signal;
|
||||
measurement.endTime = Math.max(measurement.startTime, endTime);
|
||||
lastEnded = measurement;
|
||||
if (measurement.loadType === "hard") hardLoadRoute = measurement.routeTemplate;
|
||||
if (sink) {
|
||||
sink.end(measurement);
|
||||
@@ -123,7 +169,13 @@ export function createPageLoadRecorder(
|
||||
frame = null;
|
||||
const measurement = current;
|
||||
if (!measurement || stopped) return;
|
||||
if (deps.isDocumentHidden()) measurement.wasHidden = true;
|
||||
const time = deps.now();
|
||||
if (deps.isDocumentHidden()) {
|
||||
finish(measurement, "hidden", time, null);
|
||||
return;
|
||||
}
|
||||
if (lastFrameAt !== null && time - lastFrameAt > SUSPEND_GRACE_MS) measurement.suspended = true;
|
||||
lastFrameAt = time;
|
||||
|
||||
const gates = deps.readGates();
|
||||
let signal: PageLoadReadySignal | null = null;
|
||||
@@ -138,7 +190,7 @@ export function createPageLoadRecorder(
|
||||
if (signal === null) {
|
||||
streak = 0;
|
||||
} else {
|
||||
if (streak === 0) streakStart = deps.now();
|
||||
if (streak === 0) streakStart = time;
|
||||
streak += 1;
|
||||
if (streak >= stableFrames) {
|
||||
const endTime = signal === "content" && gates.lastContentReadyAt !== null
|
||||
@@ -153,6 +205,7 @@ export function createPageLoadRecorder(
|
||||
|
||||
function schedule(measurement: PageLoadMeasurement) {
|
||||
streak = 0;
|
||||
lastFrameAt = null;
|
||||
frame = deps.requestFrame(tick);
|
||||
const remaining = Math.max(0, measurement.startTime + timeoutMs - deps.now());
|
||||
timer = deps.setTimer(() => {
|
||||
@@ -162,8 +215,11 @@ export function createPageLoadRecorder(
|
||||
}
|
||||
|
||||
return {
|
||||
begin({ routeTemplate, loadType, startTime }) {
|
||||
if (current) finish(current, "interrupted", deps.now(), null);
|
||||
begin({ routeTemplate, loadType, startTime, hiddenAt = null }) {
|
||||
// The open measurement ends where this one starts, so no request or
|
||||
// time is counted under both.
|
||||
if (current) finish(current, "interrupted", Math.min(deps.now(), startTime), null);
|
||||
const hidden = hiddenAt !== null || deps.isDocumentHidden();
|
||||
const measurement: PageLoadMeasurement = {
|
||||
id: nextId++,
|
||||
routeTemplate,
|
||||
@@ -172,18 +228,31 @@ export function createPageLoadRecorder(
|
||||
endTime: null,
|
||||
outcome: null,
|
||||
readySignal: null,
|
||||
wasHidden: deps.isDocumentHidden(),
|
||||
wasHidden: hidden,
|
||||
suspended: false,
|
||||
view: readView(routeTemplate),
|
||||
};
|
||||
latestRoute = routeTemplate;
|
||||
if (stopped) return measurement;
|
||||
current = measurement;
|
||||
sink?.begin(measurement);
|
||||
schedule(measurement);
|
||||
if (hidden) finish(measurement, "hidden", hiddenAt ?? startTime, null);
|
||||
else schedule(measurement);
|
||||
return measurement;
|
||||
},
|
||||
redirect(routeTemplate) {
|
||||
latestRoute = routeTemplate;
|
||||
if (!current) return false;
|
||||
if (!current) {
|
||||
// A load in a background tab ends as soon as it starts; its redirect
|
||||
// (for example `/` to the dashboard) renames it rather than starting
|
||||
// a second, also hidden, measurement.
|
||||
const ended = lastEnded;
|
||||
if (ended?.outcome !== "hidden" || !deps.isDocumentHidden() || !buffered.includes(ended)) return false;
|
||||
ended.routeTemplate = routeTemplate;
|
||||
ended.view = readView(routeTemplate);
|
||||
if (ended.loadType === "hard") hardLoadRoute = routeTemplate;
|
||||
return true;
|
||||
}
|
||||
current.routeTemplate = routeTemplate;
|
||||
streak = 0;
|
||||
return true;
|
||||
@@ -191,8 +260,8 @@ export function createPageLoadRecorder(
|
||||
open: () => current,
|
||||
currentRouteTemplate: () => latestRoute,
|
||||
hardLoadRouteTemplate: () => hardLoadRoute ?? (current?.loadType === "hard" ? current.routeTemplate : null),
|
||||
noteHidden() {
|
||||
if (current) current.wasHidden = true;
|
||||
noteHidden(time) {
|
||||
if (current) finish(current, "hidden", time, null);
|
||||
},
|
||||
attachSink(next) {
|
||||
if (stopped) return;
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { TASK_VIEW_KEYS } from "@/lib/task-views";
|
||||
import { pageViewFor } from "./page-view";
|
||||
|
||||
describe("pageViewFor", () => {
|
||||
it("reports the all-tasks page's view from ?view=", () => {
|
||||
expect(pageViewFor("/:companyPrefix/issues", "?view=mine")).toBe("mine");
|
||||
expect(pageViewFor("/:companyPrefix/issues", "?q=secret&view=needs_attention")).toBe("needs_attention");
|
||||
for (const key of TASK_VIEW_KEYS) expect(pageViewFor("/:companyPrefix/issues", `?view=${key}`)).toBe(key);
|
||||
});
|
||||
|
||||
it("never reports free text: an unknown view becomes other", () => {
|
||||
expect(pageViewFor("/:companyPrefix/issues", "?view=PAP-1234")).toBe("other");
|
||||
expect(pageViewFor("/:companyPrefix/issues", "?view=someone%40example.com")).toBe("other");
|
||||
expect(pageViewFor("/:companyPrefix/decisions", "?view=mine%20x")).toBe("other");
|
||||
});
|
||||
|
||||
it("reports the page's default when the URL names no view", () => {
|
||||
expect(pageViewFor("/:companyPrefix/issues", "")).toBe("default");
|
||||
expect(pageViewFor("/:companyPrefix/decisions", "?other=1")).toBe("mine");
|
||||
expect(pageViewFor("/:companyPrefix/decisions", "?view=unassigned")).toBe("unassigned");
|
||||
});
|
||||
|
||||
it("reports nothing for a page without views", () => {
|
||||
expect(pageViewFor("/:companyPrefix/issues/:issueId", "?view=mine")).toBeNull();
|
||||
expect(pageViewFor("/:companyPrefix/skills", "?view=catalog")).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,46 @@
|
||||
// The `paperclip.page.view` dimension: which view a page's `?view=` selects,
|
||||
// for the pages where that parameter picks a different list (for example
|
||||
// `/:companyPrefix/issues?view=mine`). Main-bundle code, no OpenTelemetry.
|
||||
//
|
||||
// Only fixed view names from the code's own view registries are reported. A
|
||||
// value that is not in the page's list becomes `other`, so a hand-edited URL
|
||||
// never puts free text into a span.
|
||||
|
||||
import { ATTENTION_PERSONAL_VIEWS } from "@paperclipai/shared";
|
||||
import { TASK_VIEW_KEYS, TASK_VIEW_PARAM } from "@/lib/task-views";
|
||||
import { ATTENTION_VIEW_PARAM } from "@/lib/attention-views";
|
||||
|
||||
/** Reported for a `?view=` value that is not one of the page's views. */
|
||||
export const OTHER_PAGE_VIEW = "other";
|
||||
|
||||
interface PageViewRule {
|
||||
param: string;
|
||||
views: ReadonlySet<string>;
|
||||
/** Reported when the URL has no `?view=`. */
|
||||
missing: string;
|
||||
}
|
||||
|
||||
const PAGE_VIEW_RULES: ReadonlyMap<string, PageViewRule> = new Map([
|
||||
// Tasks resolves a missing view from the last one used and writes it into
|
||||
// the URL, so a finished measurement usually sees the resolved view.
|
||||
["/:companyPrefix/issues", { param: TASK_VIEW_PARAM, views: new Set<string>(TASK_VIEW_KEYS), missing: "default" }],
|
||||
// Decisions shows "For you" (`mine`) when the URL names no view.
|
||||
["/:companyPrefix/decisions", { param: ATTENTION_VIEW_PARAM, views: new Set<string>(ATTENTION_PERSONAL_VIEWS), missing: "mine" }],
|
||||
]);
|
||||
|
||||
/**
|
||||
* The view for `routeTemplate` given the URL's query string, or null for a
|
||||
* page that has no views.
|
||||
*/
|
||||
export function pageViewFor(routeTemplate: string, search: string): string | null {
|
||||
const rule = PAGE_VIEW_RULES.get(routeTemplate);
|
||||
if (!rule) return null;
|
||||
let value: string | null;
|
||||
try {
|
||||
value = new URLSearchParams(search).get(rule.param);
|
||||
} catch {
|
||||
return OTHER_PAGE_VIEW;
|
||||
}
|
||||
if (value === null || value === "") return rule.missing;
|
||||
return rule.views.has(value) ? value : OTHER_PAGE_VIEW;
|
||||
}
|
||||
@@ -4,7 +4,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { InMemorySpanExporter } from "@opentelemetry/sdk-trace-web";
|
||||
import type { BrowserTracingConfig } from "@paperclipai/shared";
|
||||
import { createPageLoadRecorder, type PageLoadRecorder } from "./page-load-recorder";
|
||||
import { honeycombSampleRate, nonApiUrlPattern, startTracingSdk } from "./sdk";
|
||||
import { honeycombSampleRate, nonApiUrlPattern, pageLoadAttributes, startTracingSdk } from "./sdk";
|
||||
import { apiAreaFromUrl } from "./span-sanitizer";
|
||||
import type { TracingSdkHandle } from "./controller";
|
||||
|
||||
@@ -126,7 +126,186 @@ describe("startTracingSdk", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("hard-load API spans from Resource Timing", () => {
|
||||
let exporter: InMemorySpanExporter;
|
||||
let handle: TracingSdkHandle | null = null;
|
||||
|
||||
beforeEach(() => {
|
||||
exporter = new InMemorySpanExporter();
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await handle?.shutdown();
|
||||
handle = null;
|
||||
});
|
||||
|
||||
const origin = () => window.location.origin;
|
||||
const resource = (name: string, startTime: number, responseEnd: number, extra: Record<string, unknown> = {}) =>
|
||||
({ name, initiatorType: "fetch", entryType: "resource", startTime, responseEnd, ...extra }) as unknown as PerformanceResourceTiming;
|
||||
|
||||
it("backfills child spans under a hard load that finished before the SDK loaded", async () => {
|
||||
const recorder = idleRecorder();
|
||||
recorder.begin({ routeTemplate: "/:companyPrefix/issues", loadType: "hard", startTime: 0 });
|
||||
(recorder as PageLoadRecorder & { flushFrames(): void }).flushFrames();
|
||||
const loadEnd = performance.now();
|
||||
let stopped = false;
|
||||
const entries = [
|
||||
resource(`${origin()}/api/auth/get-session`, 0.5, 1, { responseStatus: 200 }),
|
||||
resource(`${origin()}/api/companies/0f6b6d1e-2c4f-4b5e/issues?view=mine&q=secret`, 1, 2, { responseStatus: 404 }),
|
||||
// Before the browser exposes a status, none is invented.
|
||||
resource(`${origin()}/api/issues/PAP-77`, 1.5, 3),
|
||||
// Started after the load ended: not part of it.
|
||||
resource(`${origin()}/api/issues`, loadEnd + 1, loadEnd + 2),
|
||||
];
|
||||
handle = startTracingSdk(config, recorder, {
|
||||
exporter,
|
||||
apiResources: { entries: () => entries, stop: () => { stopped = true; } },
|
||||
earlyResourceGraceMs: 0,
|
||||
});
|
||||
await handle.forceFlush();
|
||||
|
||||
const spans = exporter.getFinishedSpans();
|
||||
const pageSpan = spans.find((span) => span.name === "page.load")!;
|
||||
const children = spans.filter((span) => span.name !== "page.load");
|
||||
expect(children.map((span) => span.name)).toEqual(["HTTP", "HTTP", "HTTP"]);
|
||||
for (const child of children) {
|
||||
expect(child.parentSpanContext?.spanId).toBe(pageSpan.spanContext().spanId);
|
||||
expect(child.spanContext().traceId).toBe(pageSpan.spanContext().traceId);
|
||||
const serialized = JSON.stringify(child.attributes);
|
||||
expect(serialized).not.toContain("://");
|
||||
expect(serialized).not.toContain("0f6b6d1e");
|
||||
expect(serialized).not.toContain("PAP-77");
|
||||
expect(serialized).not.toContain("secret");
|
||||
expect(child.attributes["url.full"]).toBeUndefined();
|
||||
}
|
||||
expect(children.map((span) => span.attributes)).toEqual([
|
||||
{ "paperclip.api.area": "auth", "paperclip.api.timing_source": "resource_timing", "http.response.status_code": 200, "paperclip.page.route": "/:companyPrefix/issues", SampleRate: 4 },
|
||||
{ "paperclip.api.area": "companies", "paperclip.api.timing_source": "resource_timing", "http.response.status_code": 404, "paperclip.page.route": "/:companyPrefix/issues", SampleRate: 4 },
|
||||
{ "paperclip.api.area": "issues", "paperclip.api.timing_source": "resource_timing", "paperclip.page.route": "/:companyPrefix/issues", SampleRate: 4 },
|
||||
]);
|
||||
await new Promise((resolve) => setTimeout(resolve, 10));
|
||||
expect(stopped).toBe(true);
|
||||
});
|
||||
|
||||
it("backfills every buffered navigation, not only the first one replayed", async () => {
|
||||
const recorder = idleRecorder();
|
||||
const flush = (recorder as PageLoadRecorder & { flushFrames(): void }).flushFrames;
|
||||
recorder.begin({ routeTemplate: "/:companyPrefix/dashboard", loadType: "hard", startTime: 0 });
|
||||
flush();
|
||||
const clientStart = performance.now();
|
||||
recorder.begin({ routeTemplate: "/:companyPrefix/issues", loadType: "client", startTime: clientStart });
|
||||
flush();
|
||||
const clientEnd = performance.now();
|
||||
let entries: PerformanceResourceTiming[] = [
|
||||
resource(`${origin()}/api/companies`, 0.5, 1, { responseStatus: 200 }),
|
||||
resource(`${origin()}/api/issues`, clientStart, clientEnd, { responseStatus: 200 }),
|
||||
];
|
||||
handle = startTracingSdk(config, recorder, {
|
||||
exporter,
|
||||
// A real collector drops its entries when stopped.
|
||||
apiResources: { entries: () => entries, stop: () => { entries = []; } },
|
||||
});
|
||||
await handle.forceFlush();
|
||||
|
||||
const spans = exporter.getFinishedSpans();
|
||||
const pages = spans.filter((span) => span.name === "page.load");
|
||||
const childrenOf = (route: string) => {
|
||||
const page = pages.find((span) => span.attributes["paperclip.page.route"] === route)!;
|
||||
return spans.filter((span) => span.parentSpanContext?.spanId === page.spanContext().spanId).map((span) => span.attributes["paperclip.api.area"]);
|
||||
};
|
||||
expect(childrenOf("/:companyPrefix/dashboard")).toEqual(["companies"]);
|
||||
expect(childrenOf("/:companyPrefix/issues")).toEqual(["issues"]);
|
||||
});
|
||||
|
||||
it("keeps early entries for a navigation that started before the SDK loaded but commits after it", async () => {
|
||||
const recorder = idleRecorder();
|
||||
const flush = (recorder as PageLoadRecorder & { flushFrames(): void }).flushFrames;
|
||||
recorder.begin({ routeTemplate: "/:companyPrefix/dashboard", loadType: "hard", startTime: 0 });
|
||||
flush();
|
||||
// The user clicks (history change) just before the SDK finishes loading;
|
||||
// the route commits after it.
|
||||
const historyChange = performance.now();
|
||||
let entries: PerformanceResourceTiming[] = [
|
||||
resource(`${origin()}/api/companies`, 0.5, 1, { responseStatus: 200 }),
|
||||
resource(`${origin()}/api/issues`, historyChange, historyChange + 5, { responseStatus: 200 }),
|
||||
];
|
||||
let stopped = false;
|
||||
handle = startTracingSdk(config, recorder, {
|
||||
exporter,
|
||||
apiResources: { entries: () => entries, stop: () => { stopped = true; entries = []; } },
|
||||
});
|
||||
expect(stopped).toBe(false);
|
||||
recorder.begin({ routeTemplate: "/:companyPrefix/issues", loadType: "client", startTime: historyChange });
|
||||
flush();
|
||||
await handle.forceFlush();
|
||||
|
||||
const spans = exporter.getFinishedSpans();
|
||||
const childrenOf = (route: string) => {
|
||||
const page = spans.find((span) => span.name === "page.load" && span.attributes["paperclip.page.route"] === route)!;
|
||||
return spans.filter((span) => span.parentSpanContext?.spanId === page.spanContext().spanId).map((span) => span.attributes["paperclip.api.area"]);
|
||||
};
|
||||
expect(childrenOf("/:companyPrefix/dashboard")).toEqual(["companies"]);
|
||||
expect(childrenOf("/:companyPrefix/issues")).toEqual(["issues"]);
|
||||
// Released once a navigation that started after the SDK loaded begins.
|
||||
recorder.begin({ routeTemplate: "/:companyPrefix/agents", loadType: "client", startTime: performance.now() + 1 });
|
||||
expect(stopped).toBe(true);
|
||||
});
|
||||
|
||||
it("releases early entries after the grace period when no later navigation happens", async () => {
|
||||
const recorder = idleRecorder();
|
||||
recorder.begin({ routeTemplate: "/", loadType: "hard", startTime: 0 });
|
||||
(recorder as PageLoadRecorder & { flushFrames(): void }).flushFrames();
|
||||
let stopped = false;
|
||||
handle = startTracingSdk(config, recorder, {
|
||||
exporter,
|
||||
apiResources: { entries: () => [], stop: () => { stopped = true; } },
|
||||
earlyResourceGraceMs: 0,
|
||||
});
|
||||
expect(stopped).toBe(false);
|
||||
await new Promise((resolve) => setTimeout(resolve, 10));
|
||||
expect(stopped).toBe(true);
|
||||
});
|
||||
|
||||
it("does not backfill requests that the live fetch instrumentation already traces", async () => {
|
||||
const recorder = idleRecorder();
|
||||
recorder.begin({ routeTemplate: "/", loadType: "hard", startTime: 0 });
|
||||
const afterSdk = () => performance.now() + 1;
|
||||
const entries: PerformanceResourceTiming[] = [];
|
||||
handle = startTracingSdk(config, recorder, {
|
||||
exporter,
|
||||
apiResources: { entries: () => entries, stop: () => {} },
|
||||
});
|
||||
entries.push(resource(`${origin()}/api/issues`, afterSdk(), afterSdk() + 1));
|
||||
(recorder as PageLoadRecorder & { flushFrames(): void }).flushFrames();
|
||||
await handle.forceFlush();
|
||||
expect(exporter.getFinishedSpans().map((span) => span.name)).toEqual(["page.load"]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("browser tracing helpers", () => {
|
||||
it("reports hidden, suspended, and view attributes on page.load", () => {
|
||||
expect(pageLoadAttributes({
|
||||
id: 1,
|
||||
routeTemplate: "/:companyPrefix/issues",
|
||||
loadType: "hard",
|
||||
startTime: 0,
|
||||
endTime: 30_000,
|
||||
outcome: "timeout",
|
||||
readySignal: null,
|
||||
wasHidden: false,
|
||||
suspended: true,
|
||||
view: "mine",
|
||||
})).toEqual({
|
||||
"paperclip.page.route": "/:companyPrefix/issues",
|
||||
"paperclip.page.load_type": "hard",
|
||||
"paperclip.page.outcome": "timeout",
|
||||
"paperclip.page.ready_signal": "none",
|
||||
"paperclip.page.was_hidden": false,
|
||||
"paperclip.page.suspended": true,
|
||||
"paperclip.page.view": "mine",
|
||||
});
|
||||
});
|
||||
|
||||
it("ignores every URL outside this origin's API", () => {
|
||||
const pattern = nonApiUrlPattern("https://acme.paperclip.app");
|
||||
expect(pattern.test("https://acme.paperclip.app/api/issues")).toBe(false);
|
||||
|
||||
@@ -13,8 +13,11 @@
|
||||
// web_vital.lcp|inp|cls Core Web Vitals
|
||||
// GET|POST|… same-origin `/api/` fetches, with W3C trace context
|
||||
// so the server's request span joins the trace
|
||||
// HTTP same-origin `/api/` requests made before this chunk
|
||||
// loaded, rebuilt from Resource Timing under the
|
||||
// page.load they belong to (no method, no server join)
|
||||
|
||||
import { context, propagation, trace, ROOT_CONTEXT, type Context, type Span } from "@opentelemetry/api";
|
||||
import { context, propagation, trace, ROOT_CONTEXT, SpanKind, type Attributes, type Context, type Span } from "@opentelemetry/api";
|
||||
import {
|
||||
AlwaysOnSampler,
|
||||
BatchSpanProcessor,
|
||||
@@ -27,7 +30,7 @@ import { FetchInstrumentation } from "@opentelemetry/instrumentation-fetch";
|
||||
import { resourceFromAttributes } from "@opentelemetry/resources";
|
||||
import { onCLS, onINP, onLCP, type Metric } from "web-vitals";
|
||||
import type { BrowserTracingConfig } from "@paperclipai/shared";
|
||||
import type { TracingSdkHandle } from "./controller";
|
||||
import type { ApiResourceCollector, TracingSdkHandle } from "./controller";
|
||||
import type { PageLoadMeasurement, PageLoadRecorder } from "./page-load-recorder";
|
||||
import { OtlpFetchExporter } from "./otlp-fetch-exporter";
|
||||
import { SanitizingSpanExporter } from "./span-sanitizer";
|
||||
@@ -83,9 +86,46 @@ export function pageLoadAttributes(measurement: PageLoadMeasurement): Record<str
|
||||
"paperclip.page.outcome": measurement.outcome ?? "interrupted",
|
||||
"paperclip.page.ready_signal": measurement.readySignal ?? "none",
|
||||
"paperclip.page.was_hidden": measurement.wasHidden,
|
||||
...(measurement.suspended ? { "paperclip.page.suspended": true } : {}),
|
||||
...(measurement.view !== null ? { "paperclip.page.view": measurement.view } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
/** Span name for a request whose method is unknown (OpenTelemetry HTTP semantics). */
|
||||
export const UNKNOWN_METHOD_SPAN_NAME = "HTTP";
|
||||
|
||||
/**
|
||||
* Child spans for the `/api/` requests a page load made before the SDK
|
||||
* loaded, from their Resource Timing entries: requests that started inside
|
||||
* the load's window and before `before` (from then on the live fetch spans
|
||||
* cover them). Windows do not overlap (an interrupted load ends where the
|
||||
* next one starts), so each request becomes exactly one span. The URL goes into `url.full` only for the sanitizer, which
|
||||
* turns it into `paperclip.api.area` and drops it, as for live fetch spans.
|
||||
*/
|
||||
function backfillApiSpans(
|
||||
tracer: ReturnType<WebTracerProvider["getTracer"]>,
|
||||
pageSpan: Span,
|
||||
measurement: PageLoadMeasurement,
|
||||
entries: readonly PerformanceResourceTiming[],
|
||||
before: number,
|
||||
) {
|
||||
const windowEnd = Math.min(measurement.endTime ?? before, before);
|
||||
const parent = trace.setSpan(ROOT_CONTEXT, pageSpan);
|
||||
for (const entry of entries) {
|
||||
if (entry.startTime < measurement.startTime || entry.startTime >= windowEnd) continue;
|
||||
const attributes: Attributes = {
|
||||
"url.full": entry.name,
|
||||
"paperclip.api.timing_source": "resource_timing",
|
||||
"paperclip.page.route": measurement.routeTemplate,
|
||||
};
|
||||
// Only where the browser exposes it (Chromium); never guessed.
|
||||
const status = (entry as PerformanceResourceTiming & { responseStatus?: number }).responseStatus;
|
||||
if (typeof status === "number" && status > 0) attributes["http.response.status_code"] = status;
|
||||
const span = tracer.startSpan(UNKNOWN_METHOD_SPAN_NAME, { kind: SpanKind.CLIENT, startTime: entry.startTime, attributes }, parent);
|
||||
span.end(Math.max(entry.startTime, entry.responseEnd));
|
||||
}
|
||||
}
|
||||
|
||||
/** Honeycomb weights each event by its `SampleRate` (1 in N). */
|
||||
export function honeycombSampleRate(sampleRate: number): number | null {
|
||||
return sampleRate >= 1 ? null : Math.max(1, Math.round(1 / sampleRate));
|
||||
@@ -94,8 +134,19 @@ export function honeycombSampleRate(sampleRate: number): number | null {
|
||||
export interface TracingSdkOptions {
|
||||
/** Replaces the OTLP exporter (tests). Spans are still sanitized. */
|
||||
exporter?: SpanExporter;
|
||||
/** `/api/` requests recorded before the SDK loaded (`observeApiResources`). */
|
||||
apiResources?: ApiResourceCollector;
|
||||
/** Overrides `EARLY_RESOURCE_GRACE_MS` (tests). */
|
||||
earlyResourceGraceMs?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* After this long, a navigation whose history change preceded the SDK has
|
||||
* long since committed, so early entries no open measurement claims are
|
||||
* released even if the user never navigates again.
|
||||
*/
|
||||
export const EARLY_RESOURCE_GRACE_MS = 60_000;
|
||||
|
||||
export function startTracingSdk(
|
||||
config: BrowserTracingConfig,
|
||||
recorder: PageLoadRecorder,
|
||||
@@ -150,8 +201,30 @@ export function startTracingSdk(
|
||||
enabled: false,
|
||||
});
|
||||
fetchInstrumentation.setTracerProvider(provider);
|
||||
// Requests from here on get live fetch spans; earlier ones are backfilled.
|
||||
const liveFetchSince = performance.now();
|
||||
fetchInstrumentation.enable();
|
||||
|
||||
// Who may still claim the early Resource Timing entries: any measurement
|
||||
// that started before `liveFetchSince` and has not ended, including one not
|
||||
// yet begun (a navigation's start is its history change, which can precede
|
||||
// the SDK while its route commits after it). Navigation starts only move
|
||||
// forward, so once a measurement that started after `liveFetchSince` has
|
||||
// begun, or the grace period has passed, no new claimant can appear; the
|
||||
// entries are released when, in addition, no claimant is open.
|
||||
const apiResources = options.apiResources ?? null;
|
||||
const earlyOpen = new Set<number>();
|
||||
let noNewEarlyClaimants = false;
|
||||
const releaseIfUnclaimed = () => {
|
||||
if (apiResources && noNewEarlyClaimants && earlyOpen.size === 0) apiResources.stop();
|
||||
};
|
||||
const graceTimer = apiResources
|
||||
? window.setTimeout(() => {
|
||||
noNewEarlyClaimants = true;
|
||||
releaseIfUnclaimed();
|
||||
}, options.earlyResourceGraceMs ?? EARLY_RESOURCE_GRACE_MS)
|
||||
: undefined;
|
||||
|
||||
recorder.attachSink({
|
||||
begin(measurement) {
|
||||
const span = tracer.startSpan("page.load", {
|
||||
@@ -164,6 +237,12 @@ export function startTracingSdk(
|
||||
});
|
||||
pageSpans.set(measurement.id, span);
|
||||
if (measurement.endTime === null) openPageSpan = span;
|
||||
if (measurement.startTime < liveFetchSince) {
|
||||
earlyOpen.add(measurement.id);
|
||||
} else {
|
||||
noNewEarlyClaimants = true;
|
||||
releaseIfUnclaimed();
|
||||
}
|
||||
},
|
||||
end(measurement) {
|
||||
const span = pageSpans.get(measurement.id);
|
||||
@@ -172,7 +251,12 @@ export function startTracingSdk(
|
||||
if (openPageSpan === span) openPageSpan = null;
|
||||
span.setAttributes(pageLoadAttributes(measurement));
|
||||
if (measurement.loadType === "hard") span.setAttributes(navigationTimingAttributes());
|
||||
if (apiResources && measurement.startTime < liveFetchSince) {
|
||||
backfillApiSpans(tracer, span, measurement, apiResources.entries(), liveFetchSince);
|
||||
}
|
||||
span.end(measurement.endTime ?? performance.now());
|
||||
earlyOpen.delete(measurement.id);
|
||||
releaseIfUnclaimed();
|
||||
},
|
||||
});
|
||||
|
||||
@@ -207,6 +291,8 @@ export function startTracingSdk(
|
||||
pageSpans.clear();
|
||||
openPageSpan = null;
|
||||
fetchInstrumentation.disable();
|
||||
window.clearTimeout(graceTimer);
|
||||
apiResources?.stop();
|
||||
await provider.shutdown().catch(() => undefined);
|
||||
trace.disable();
|
||||
context.disable();
|
||||
|
||||
@@ -16,6 +16,8 @@ export const ALLOWED_SPAN_ATTRIBUTES: ReadonlySet<string> = new Set([
|
||||
"paperclip.page.outcome",
|
||||
"paperclip.page.ready_signal",
|
||||
"paperclip.page.was_hidden",
|
||||
"paperclip.page.suspended",
|
||||
"paperclip.page.view",
|
||||
"paperclip.page.navigation_type",
|
||||
"paperclip.page.ttfb_ms",
|
||||
"paperclip.page.dom_content_loaded_ms",
|
||||
@@ -24,6 +26,7 @@ export const ALLOWED_SPAN_ATTRIBUTES: ReadonlySet<string> = new Set([
|
||||
"paperclip.web_vital.value",
|
||||
"paperclip.web_vital.rating",
|
||||
"paperclip.api.area",
|
||||
"paperclip.api.timing_source",
|
||||
"http.request.method",
|
||||
"http.response.status_code",
|
||||
"error.type",
|
||||
|
||||
Reference in new issue
Block a user