Files
PaperClipAI/ui/src/api/client.ts
T
DottaandPaperclip e40de0c9e8 perf(ci): refresh test weights and narrow graph dependencies (#18464)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Contributors need fast, reliable CI feedback.
> - Diff-aware CI selects test files and partitions them into runner
jobs.
> - Stale cost estimates put several slow suites on the same runner.
Broad imports also couple small tests to large modules.
> - This PR refreshes test costs, runs independent graph discovery in
parallel, and narrows three dependency boundaries.
> - Selection safeguards and CODEOWNER-only AWS routing stay in force.

## Linked Issues or Issue Description

Refs #18398. This follows the audit of [PR #18427's verification
run](https://github.com/paperclipai/paperclip/actions/runs/41169978784).

The audit found 403 selected general-server files without timing
weights. Three suites measured at 232s, 122s and 116s shared one shard
because each received a roughly 2s fallback weight. Local graph
discovery across four projects took 43.7s serially and 21.7s with two
workers. A reverse-order repeat took 37.7s serially and 22.6s with two
workers. Both comparisons produced identical selection results. This is
a local measurement, not a claim about production CI speed.

## What Changed

- Refresh general-server costs for 1,285 of 1,309 current files and
serialized costs for all 163 files from two successful protected-master
Cloud readiness runs. Retain prior costs for unmeasured files, including
dedicated chat and native-runner suites.
- Record passing server file costs, including collection, setup and
environment preparation. A weekly hosted workflow validates trusted
source runs and opens a timing-only PR. It tests shard coverage, pauses
while a refresh PR is open, and never merges automatically.
- Discover at most two project graphs concurrently in isolated,
heap-limited Node processes. Cache repeated inventory reads within one
selection. Worker errors and timeouts retain the full task plan.
- Fetch only the merge commit and parents for the selector. A real
shallow-clone regression proves the complete diff still includes more
than 300 files, renames and deletions.
- Share the identical log viewer and run details between both
AgentDetail pages. The viewer test imports the shared implementation and
runs each of its 13 distinct cases once.
- Extract the pure API error class while preserving the API client's
public exports. Import delivery work signals through their existing
narrow DB export.

## Verification

- PASS: `pnpm -r typecheck` and `pnpm build`.
- PASS: 284 CI selection, timing, shallow-diff, sharding and routing
tests.
- PASS: 74 further cache, routing and graph-proof tests.
- PASS: 94 targeted UI assertions across the viewer, run invocation, API
errors/client and progress helpers. All 76 assertions in the 12 server
suites made independently selectable also pass.
- PASS: `pnpm check:token-gates`, module-boundary checks, YAML parsing
and `git diff --check`.
- PASS: [Full PR CI and required `ci / verify`
aggregate](https://github.com/paperclipai/paperclip/actions/runs/41206683345),
including all 12 general-server shards, all nine serialized shards,
workspace groups, chat suites, Runner lanes and build checks.
- PASS: [GitHub hosted real-workspace selector and cache/routing
qualification](https://github.com/paperclipai/paperclip/actions/runs/41206683205).
- PASS: Both Greptile and Paperclip reviews are 5/5 with no actionable
findings or inline comments on `7dc65c51b8`. The PR has no merge
conflicts.
- PASS: The production refresh parser accepted all 107 structured
file-cost records from a real successful server CI job, including
setup/loading cost.
- Local `pnpm test:run` completed all six chat shards successfully. The
duplicate monolithic local run was stopped after the complete remote
suite passed; it was not completed locally.

The focused log-viewer command improved from 18.84s (26 duplicate
variant cases) to 8.49s (all 13 distinct behavior cases). The dependency
audit confirms 12 server files and the API-error test no longer need
conservative always-run retention. Repartitioning the audited PR with
the new costs changes the slowest shard from 810s to 591s of summed
historical test execution (1,124 of 1,142 files measured). This is a
workload simulation, not measured new CI wall time.

## Risks

- Two graph processes use more peak memory. Each has a 2 GiB heap limit
and a three-minute timeout. Any failure preserves the original task
plan. Real workspace qualification runs in GitHub CI.
- Historical logs contain test/hook costs only. New reporter records add
loading/setup costs after merge. The manifests identify measurement
coverage and source runs. These weights change placement, never
permission to skip tests.
- Runner load adds timing noise. The refresh uses medians and requires
at least 90% measurement coverage in each lane.
- GitHub may require approval to run checks on a PR created with
`GITHUB_TOKEN`. Refreshes remain reviewable and do not merge themselves.
- The shared viewer is a mechanical extraction. Both pages retain their
existing exports, and the behavior assertions remain covered. AWS runner
authorization is unchanged.

## Model Used

OpenAI Codex, GPT-6, with repository inspection, reasoning, code
execution and GitHub tools. The exact deployment ID and context-window
size were not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub references)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused checks and six
chat shards; complete suite passed in CI as detailed above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-31 11:05:12 -05:00

287 lines
11 KiB
TypeScript

import { getPageVisibility, getVisibilityHeaderValue } from "@/lib/page-visibility";
import { tenantSessionRecovery } from "@/lib/tenant-session-recovery";
import { ApiError, apiErrorFromResponse } from "./api-error";
export { ApiError, apiErrorFromResponse, type ApiErrorOptions } from "./api-error";
import { readApiJson } from "./response";
const BASE = "/api";
// --- Dev/QA outage simulator ----------------------------------------------------
//
// Reproduces a deploy blip without a deploy. Dev and QA builds only:
//
// localStorage.paperclipSimulateOutage = "503:tenant_app_unavailable" // until removed
// localStorage.paperclipSimulateOutage = "503:tenant_app_unavailable@20" // for 20 seconds
// localStorage.paperclipSimulateOutage = "502" // non-JSON proxy page
// localStorage.paperclipSimulateOutage = "network" // fetch rejects (connection drop)
//
// Every same-origin API request made through `apiFetch` (the client, health,
// session) fails with the chosen shape. Remove the key to end the outage.
export const OUTAGE_SIMULATOR_STORAGE_KEY = "paperclipSimulateOutage";
export type SimulatedOutage =
| { kind: "network"; durationMs: number | null }
| { kind: "status"; status: number; code: string | null; durationMs: number | null };
export function parseSimulatedOutage(raw: string | null | undefined): SimulatedOutage | null {
const match = /^\s*(network|(\d{3})(?::([A-Za-z0-9_.-]+))?)(?:@(\d+(?:\.\d+)?))?\s*$/.exec(raw ?? "");
if (!match) return null;
const durationMs = match[4] ? Math.round(Number(match[4]) * 1000) : null;
if (match[1] === "network") return { kind: "network", durationMs };
const status = Number(match[2]);
if (status < 400 || status > 599) return null;
return { kind: "status", status, code: match[3] ?? null, durationMs };
}
function outageSimulatorEnabled(): boolean {
return import.meta.env.DEV || import.meta.env.MODE === "qa";
}
let simulatedOutageWindow: { raw: string; endsAt: number | null } | null = null;
/** The outage to simulate for the next request, or null. Never throws. */
export function activeSimulatedOutage(now = Date.now()): SimulatedOutage | null {
if (!outageSimulatorEnabled() || typeof window === "undefined") return null;
let raw: string | null = null;
try {
raw = window.localStorage?.getItem(OUTAGE_SIMULATOR_STORAGE_KEY) ?? null;
} catch {
return null;
}
const outage = parseSimulatedOutage(raw);
if (!raw || !outage) {
simulatedOutageWindow = null;
return null;
}
if (simulatedOutageWindow?.raw !== raw) {
simulatedOutageWindow = { raw, endsAt: outage.durationMs === null ? null : now + outage.durationMs };
}
if (simulatedOutageWindow.endsAt !== null && now >= simulatedOutageWindow.endsAt) {
simulatedOutageWindow = null;
try {
window.localStorage.removeItem(OUTAGE_SIMULATOR_STORAGE_KEY);
} catch {
// Storage can be unavailable; the outage still ends for this tab.
}
return null;
}
return outage;
}
function simulatedOutageResponse(outage: Extract<SimulatedOutage, { kind: "status" }>): Response {
if (!outage.code) {
// What a proxy serves while the app is down: an HTML page, not JSON.
return new Response("<!doctype html><title>Unavailable</title>", {
status: outage.status,
headers: { "Content-Type": "text/html" },
});
}
return new Response(JSON.stringify({ error: outage.code }), {
status: outage.status,
headers: { "Content-Type": "application/json" },
});
}
/**
* `fetch` for same-origin API calls. Identical to `fetch` except that the
* dev/QA outage simulator can intercept it.
*/
export async function apiFetch(input: string, init?: RequestInit): Promise<Response> {
const outage = activeSimulatedOutage();
if (outage) {
if (init?.signal?.aborted) throw abortError();
if (outage.kind === "network") throw new TypeError("Failed to fetch");
return simulatedOutageResponse(outage);
}
return fetch(input, init);
}
export interface RequestOptions {
/** Abort signal wired through to `fetch` and coalescing (per-caller). */
signal?: AbortSignal;
/** Extra request headers (e.g. the async-import opt-in). Mutations only. */
headers?: Record<string, string>;
/** The `fetch` cache mode. Use `"no-store"` for a response that must never
* come from the browser's HTTP cache. */
cache?: RequestCache;
}
function abortError(): DOMException {
return new DOMException("The operation was aborted.", "AbortError");
}
/**
* Non-authoritative observability hints (PAP-12556 / Phase 1). The server treats
* these as scheduling/telemetry only and never as security signals.
*/
function applyObservabilityHeaders(headers: Headers) {
if (headers.has("X-Paperclip-Tab-Visible")) return; // caller override wins
const visibility = getPageVisibility();
headers.set("X-Paperclip-Tab-Visible", getVisibilityHeaderValue(visibility));
if (typeof window !== "undefined" && window.location) {
headers.set("X-Paperclip-Route", window.location.pathname);
}
}
export async function requestResponse(path: string, init?: RequestInit): Promise<Response> {
const headers = new Headers(init?.headers ?? undefined);
const body = init?.body;
if (!(body instanceof FormData) && !headers.has("Content-Type")) {
headers.set("Content-Type", "application/json");
}
applyObservabilityHeaders(headers);
const res = await apiFetch(`${BASE}${path}`, {
credentials: "include",
...init,
headers,
});
if (!res.ok) {
const errorBody = await readApiJson(res);
const recovery = tenantSessionRecovery.recoverIfNeeded(res.status, errorBody);
if (recovery) return recovery;
throw apiErrorFromResponse(res, errorBody);
}
return res;
}
async function request<T>(path: string, init?: RequestInit): Promise<T> {
const res = await requestResponse(path, init);
if (res.status === 204) return undefined as T;
return readApiJson<T>(res);
}
// --- In-tab request coalescing for identical safe GETs -----------------------
//
// Multiple callers issuing the same GET while one is in flight share a single
// underlying fetch. Each caller keeps its own abort semantics: aborting one
// caller only cancels the shared fetch when *every* caller has aborted.
// Mutations are never coalesced.
interface InflightGet {
promise: Promise<unknown>;
controller: AbortController;
refs: Set<symbol>;
}
const inflightGets = new Map<string, InflightGet>();
function coalescedGet<T>(path: string, options?: RequestOptions): Promise<T> {
const signal = options?.signal;
if (signal?.aborted) return Promise.reject(abortError());
let entry = inflightGets.get(path);
if (!entry) {
const controller = new AbortController();
const promise = request<T>(path, {
method: "GET",
signal: controller.signal,
...(options?.cache ? { cache: options.cache } : {}),
});
const created: InflightGet = { promise, controller, refs: new Set() };
// Clear the shared entry once settled so later calls issue a fresh request.
promise.then(
() => {
if (inflightGets.get(path) === created) inflightGets.delete(path);
},
() => {
if (inflightGets.get(path) === created) inflightGets.delete(path);
},
);
inflightGets.set(path, created);
entry = created;
}
const activeEntry = entry;
const ref = Symbol("caller");
activeEntry.refs.add(ref);
const releaseRef = () => {
if (!activeEntry.refs.delete(ref)) return;
// Last caller gone before the fetch settled → abort the shared request.
if (activeEntry.refs.size === 0 && inflightGets.get(path) === activeEntry) {
inflightGets.delete(path);
activeEntry.controller.abort();
}
};
return new Promise<T>((resolve, reject) => {
const onAbort = () => {
signal?.removeEventListener("abort", onAbort);
releaseRef();
reject(abortError());
};
if (signal) signal.addEventListener("abort", onAbort);
activeEntry.promise.then(
(value) => {
signal?.removeEventListener("abort", onAbort);
activeEntry.refs.delete(ref);
resolve(value as T);
},
(err) => {
signal?.removeEventListener("abort", onAbort);
activeEntry.refs.delete(ref);
reject(err);
},
);
});
}
/**
* Stop later callers from joining the in-flight GET for `path`.
*
* Coalescing keys on the path alone, so a GET issued under one account's session
* can be joined by a caller that runs after the account changed — and handed the
* previous account's response. Detaching leaves that request to settle for the
* callers that asked for it, and makes the next call issue a fresh one. It does
* not abort, because those callers still want what they asked for.
*/
export function detachInflightGet(path: string): void {
inflightGets.delete(path);
}
function isRequestOptions(value: unknown): value is RequestOptions {
return typeof value === "object" && value !== null && "signal" in value;
}
export const api = {
get: <T>(path: string, options?: RequestOptions) => coalescedGet<T>(path, options),
post: <T>(path: string, body: unknown, options?: RequestOptions) =>
request<T>(path, {
method: "POST",
body: JSON.stringify(body),
signal: options?.signal,
...(options?.headers ? { headers: options.headers } : {}),
}),
postForm: <T>(path: string, body: FormData, options?: RequestOptions) =>
request<T>(path, {
method: "POST",
body,
signal: options?.signal,
// Never set Content-Type here — the browser sets multipart/form-data with
// the boundary. Extra headers (e.g. an async opt-in) may still ride along.
...(options?.headers ? { headers: options.headers } : {}),
}),
put: <T>(path: string, body: unknown, options?: RequestOptions) =>
request<T>(path, { method: "PUT", body: JSON.stringify(body), signal: options?.signal }),
/** Raw binary upload (e.g. one chunked import-transfer part); the body travels as-is. */
putRaw: <T>(path: string, body: Blob, options?: RequestOptions) =>
request<T>(path, {
method: "PUT",
body,
signal: options?.signal,
headers: { "Content-Type": "application/octet-stream", ...(options?.headers ?? {}) },
}),
patch: <T>(path: string, body: unknown, options?: RequestOptions) =>
request<T>(path, { method: "PATCH", body: JSON.stringify(body), signal: options?.signal }),
delete: <T>(path: string, bodyOrOptions?: unknown, options?: RequestOptions) => {
const requestOptions = isRequestOptions(bodyOrOptions) ? bodyOrOptions : options;
const body = bodyOrOptions === undefined || isRequestOptions(bodyOrOptions) ? undefined : JSON.stringify(bodyOrOptions);
return request<T>(path, { method: "DELETE", ...(body === undefined ? {} : { body }), signal: requestOptions?.signal });
},
deleteWithBody: <T>(path: string, body: unknown, options?: RequestOptions) =>
request<T>(path, { method: "DELETE", body: JSON.stringify(body), signal: options?.signal }),
};