mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-03 17:51:58 +01:00
## Thinking Path > - Paperclip manages AI agents and their access to saved connections. > - The Permissions page loads account details and permitted agent choices. > - Company-wide AI accounts repeat retained installation checks for every candidate agent. > - This makes target checks grow with the square of the roster size. > - The browser waits without a deadline, and connection names do not identify the provider login. > - This pull request batches direct AI checks and bounds loading with recovery controls. > - Account details load separately and show a provider email or an explicit profile-permission limit. ## Linked Issues or Issue Description **What happened?** Opening a saved AI account can leave Permissions on “Loading AI account…” for minutes. The account itself can be connected. The existing capability batch excludes AI connections, so a company-wide install repeats checks for every installed agent for every candidate. Saved names and Paperclip ownership also do not establish which provider login a credential uses. **Expected behavior** Direct AI accounts reuse retained-target checks within one descriptive request. The page shows loaded account controls or a bounded failure with Retry. Permissions shows the provider email and organization when the selected credential permits profile access. An unavailable profile must not block maintenance or disconnect a working inference credential. Writes still authorize current permissions. **Steps to reproduce** 1. Save an AI account in a company with several agents. 2. Install the account company-wide. 3. Open its Permissions page. 4. Delay the account-list or grants response to exercise the loading failure. The maintained benchmark reproduces database work with synthetic local data. Refs #17945, which added batching for non-AI connections. Searches found no open PR for this AI-account loading fix. ## What Changed - Include direct AI accounts in the existing board capability batch. Referenced pool members, delegated actors, complex selectors, and mutation authorization keep their existing evaluator. - Add opt-in per-caller GET deadlines to the API client. A timed-out caller detaches the stuck coalesced request so Retry starts a fresh fetch. Other callers retain their own cancellation and results. - Apply a 15-second deadline to AI account lists and grants reads. The permissions view retries transient failures once and reports other failures immediately. - Share the parent Permissions grants query with AI account details. Keep loaded controls visible after a background error and offer Retry. - Add a grant-scoped identity GET and OpenAPI contract. The service checks current membership and credential audience before it reads the selected credential. It projects only the provider email and organization; provider IDs, tokens, and raw errors stay private. - Show account details independently of maintenance controls. Anthropic profile denial has an explicit message and preserves connected status. Transient profile failures have a separate Retry action. Observations are cached for five minutes under the company, connection, grant, and grant revision. - Extend real PostgreSQL and authenticated HTTP coverage, client/component regressions, map guidance, and the existing benchmark's AI mode. - Address the Apex file-size finding by moving gallery identity lookup into the existing app display helper module. `AppDetail.tsx` is 984 lines; lookup behavior is unchanged. ## Verification Local source checkout on macOS, Node 25.6.1, pnpm 9.15.4, and disposable PostgreSQL 18.1. Provider calls use fixtures. No hosted deployment or provider qualification is claimed. - Passed `pnpm -r typecheck` and `pnpm build`. - Passed all 87 tests in `connection-management.integration.test.ts` and all 33 tests in `connection-install-capabilities.test.ts`. - Passed 196 AI connection, identity-provider, and OpenAPI tests: `ai-connections.test.ts`, `ai-account-identity.test.ts`, and `openapi-routes.test.ts`. - Passed 107 focused UI/client tests: `client.test.ts`, `ManagedAiConnectionDetails.test.tsx`, `AiConnectionAccountDetails.test.tsx`, and `AppDetail.test.tsx`. - After the Apex extraction, passed UI typecheck/build and 106 component tests across `AppDetail.test.tsx`, `AppNotConnected.test.tsx`, `ManagedAiConnectionDetails.test.tsx`, and `AiConnectionAccountDetails.test.tsx`. - The authenticated HTTP tests compare complete grants responses across human roles, for tool and AI connections. They verify a real install mutation succeeds before revocation and fails afterward. - A company-wide personal AI fixture checks 20 targets instead of 400. Pool-member permission loss still denies the complete pool description. - UI states covered: shared cached grants, stuck account/grants fetches, one automatic retry, terminal failure, manual recovery, authorization failure, background failure with retained controls, and unmount cancellation. Client tests also cover stalled response bodies and independent coalesced callers. - Identity coverage includes actual bearer-key/JWT reads, private-user and cross-company denial before provider access, revoked credentials, provider 401/403/429/500, and grant-change isolation. A real local HTTP fixture stalls the response body, verifies the shared 15-second deadline, and keeps unrelated HTTP requests responsive. The test shortens the native deadline to 100 ms. Profile permission denial leaves the saved account connected. - Passed `pnpm check:token-gates`, `node scripts/check-maps.mjs`, `node scripts/check-module-boundaries.mjs`, and `git diff --check`. - The full local `pnpm test:run` started with the loading-only change. All six chat shards passed. It recorded one failure in the unchanged `heartbeat-retry-scheduling.test.ts` quota-rejection case. That exact case passed a focused rerun on the final head. The initial failure cause remains unconfirmed and its local log is retained; this is not a full local-suite pass. The redundant local run was stopped with SIGINT after final-head CI completed successfully. - [Final-head CI](https://github.com/paperclipai/paperclip/actions/runs/41515556725) passed on `ead8d276d4a2023614fff54b3e9a858f28c3178b`: typecheck, build, all 12 server shards, all 3 chat shards, workspace suites, all 9 serialized suites, all 8 end-to-end shards, Runner checks, and the aggregate verify gate. Security checks passed. Optional Storybook checks were skipped by workflow policy. GitHub reports the branch mergeable without conflicts. - Apex reviewed head `ead8d276d4a2023614fff54b3e9a858f28c3178b` at 5/5, confirmed the extraction resolves its earlier file-size finding, and has no unresolved review threads. The existing benchmark ran for the loading-only change now recorded in commit `5f725b36fa52002545241a5a8b7a10602f0cc0ce`, with `--purpose=ai --baseline-ref=026fd9e3452`. The identity follow-up leaves that batch source unchanged. Two warmups and five alternating samples exercise each of eight fixture shapes. Complete positional capabilities, reasons, and operations match the baseline in every sample. | AI fixture | SQL calls before → after | Target reads before → after | | --- | ---: | ---: | | 1 agent, company install | 9 → 9 | 1 → 1 | | 32 agents, company install | 5,248 → 164 | 1,024 → 32 | | 100 agents, company install | 50,400 → 504 | 10,000 → 100 | | 100 agents, partial install | 13,175 → 503 | 2,575 → 100 | These counts measure the real service over local PostgreSQL. They do not predict hosted page latency. To reproduce the standalone benchmark, use a checkout of commit `5f725b36fa52002545241a5a8b7a10602f0cc0ce` and run the command below. Its isolation check intentionally rejects the later identity follow-up's additional runtime changes. ```sh node --import ./server/node_modules/tsx/dist/esm/index.mjs scripts/benchmark-connection-install-capabilities.ts --purpose=ai --baseline-ref=026fd9e3452 ``` Run the final-head regression suites from the repository root: ```sh cd server pnpm exec vitest run src/__tests__/connection-management.integration.test.ts src/__tests__/connection-install-capabilities.test.ts src/__tests__/ai-connections.test.ts src/services/ai-account-identity.test.ts src/__tests__/openapi-routes.test.ts cd ../ui pnpm exec vitest run src/api/client.test.ts src/components/ai-connections/ManagedAiConnectionDetails.test.tsx src/components/ai-connections/AiConnectionAccountDetails.test.tsx src/pages/apps/AppDetail.test.tsx ``` ## Risks Descriptive permissions reuse retained facts within one request. A concurrent change can leave a displayed choice stale; the next request and every write still authorize fresh. Pools with referenced members retain their existing cost. The account and grants clients now bound each caller's wait. A browser abort does not cancel an authorization read already admitted by the server. The independent profile request uses a fixed Anthropic origin, a bounded body, no redirects, and a 15-second deadline. Inference-only credentials can return no email; that is reported without guessing from a Paperclip owner or changing connection health. Profiles are fetched only for the caller's authorized selected grant. No ambient CLI identity, persistent identity cache, background polling, or billing-identity change is added. Other API reads keep their existing behavior. No migration is required. Staging has not been redeployed. ## Model Used OpenAI `gpt-6.1-sol` through the Codex harness in T3 Code, with xhigh reasoning, code editing, execution, and test tools. The host does not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
304 lines
11 KiB
TypeScript
304 lines
11 KiB
TypeScript
import { getPageVisibility, getVisibilityHeaderValue } from "@/lib/page-visibility";
|
|
import { tenantSessionRecovery } from "@/lib/tenant-session-recovery";
|
|
import { ApiError, apiErrorFromResponse } from "./api-error";
|
|
import { errorCopy } from "./errors";
|
|
export { ApiError, apiErrorFromResponse, type ApiErrorOptions } from "./api-error";
|
|
import { readApiJson } from "./response";
|
|
|
|
const BASE = "/api";
|
|
|
|
// --- Dev/QA outage simulator ----------------------------------------------------
|
|
//
|
|
// Reproduces a deploy blip without a deploy. Dev and QA builds only:
|
|
//
|
|
// localStorage.paperclipSimulateOutage = "503:tenant_app_unavailable" // until removed
|
|
// localStorage.paperclipSimulateOutage = "503:tenant_app_unavailable@20" // for 20 seconds
|
|
// localStorage.paperclipSimulateOutage = "502" // non-JSON proxy page
|
|
// localStorage.paperclipSimulateOutage = "network" // fetch rejects (connection drop)
|
|
//
|
|
// Every same-origin API request made through `apiFetch` (the client, health,
|
|
// session) fails with the chosen shape. Remove the key to end the outage.
|
|
|
|
export const OUTAGE_SIMULATOR_STORAGE_KEY = "paperclipSimulateOutage";
|
|
|
|
export type SimulatedOutage =
|
|
| { kind: "network"; durationMs: number | null }
|
|
| { kind: "status"; status: number; code: string | null; durationMs: number | null };
|
|
|
|
export function parseSimulatedOutage(raw: string | null | undefined): SimulatedOutage | null {
|
|
const match = /^\s*(network|(\d{3})(?::([A-Za-z0-9_.-]+))?)(?:@(\d+(?:\.\d+)?))?\s*$/.exec(raw ?? "");
|
|
if (!match) return null;
|
|
const durationMs = match[4] ? Math.round(Number(match[4]) * 1000) : null;
|
|
if (match[1] === "network") return { kind: "network", durationMs };
|
|
const status = Number(match[2]);
|
|
if (status < 400 || status > 599) return null;
|
|
return { kind: "status", status, code: match[3] ?? null, durationMs };
|
|
}
|
|
|
|
function outageSimulatorEnabled(): boolean {
|
|
return import.meta.env.DEV || import.meta.env.MODE === "qa";
|
|
}
|
|
|
|
let simulatedOutageWindow: { raw: string; endsAt: number | null } | null = null;
|
|
|
|
/** The outage to simulate for the next request, or null. Never throws. */
|
|
export function activeSimulatedOutage(now = Date.now()): SimulatedOutage | null {
|
|
if (!outageSimulatorEnabled() || typeof window === "undefined") return null;
|
|
let raw: string | null = null;
|
|
try {
|
|
raw = window.localStorage?.getItem(OUTAGE_SIMULATOR_STORAGE_KEY) ?? null;
|
|
} catch {
|
|
return null;
|
|
}
|
|
const outage = parseSimulatedOutage(raw);
|
|
if (!raw || !outage) {
|
|
simulatedOutageWindow = null;
|
|
return null;
|
|
}
|
|
if (simulatedOutageWindow?.raw !== raw) {
|
|
simulatedOutageWindow = { raw, endsAt: outage.durationMs === null ? null : now + outage.durationMs };
|
|
}
|
|
if (simulatedOutageWindow.endsAt !== null && now >= simulatedOutageWindow.endsAt) {
|
|
simulatedOutageWindow = null;
|
|
try {
|
|
window.localStorage.removeItem(OUTAGE_SIMULATOR_STORAGE_KEY);
|
|
} catch {
|
|
// Storage can be unavailable; the outage still ends for this tab.
|
|
}
|
|
return null;
|
|
}
|
|
return outage;
|
|
}
|
|
|
|
function simulatedOutageResponse(outage: Extract<SimulatedOutage, { kind: "status" }>): Response {
|
|
if (!outage.code) {
|
|
// What a proxy serves while the app is down: an HTML page, not JSON.
|
|
return new Response("<!doctype html><title>Unavailable</title>", {
|
|
status: outage.status,
|
|
headers: { "Content-Type": "text/html" },
|
|
});
|
|
}
|
|
return new Response(JSON.stringify({ error: outage.code }), {
|
|
status: outage.status,
|
|
headers: { "Content-Type": "application/json" },
|
|
});
|
|
}
|
|
|
|
/**
|
|
* `fetch` for same-origin API calls. Identical to `fetch` except that the
|
|
* dev/QA outage simulator can intercept it.
|
|
*/
|
|
export async function apiFetch(input: string, init?: RequestInit): Promise<Response> {
|
|
const outage = activeSimulatedOutage();
|
|
if (outage) {
|
|
if (init?.signal?.aborted) throw abortError();
|
|
if (outage.kind === "network") throw new TypeError("Failed to fetch");
|
|
return simulatedOutageResponse(outage);
|
|
}
|
|
return fetch(input, init);
|
|
}
|
|
|
|
export interface RequestOptions {
|
|
/** Per-caller GET deadline, including the response body. A retry starts a fresh fetch. */
|
|
timeoutMs?: number;
|
|
/** Abort signal wired through to `fetch` and coalescing (per-caller). */
|
|
signal?: AbortSignal;
|
|
/** Extra request headers (e.g. the async-import opt-in). Mutations only. */
|
|
headers?: Record<string, string>;
|
|
/** The `fetch` cache mode. Use `"no-store"` for a response that must never
|
|
* come from the browser's HTTP cache. */
|
|
cache?: RequestCache;
|
|
}
|
|
|
|
function abortError(): DOMException {
|
|
return new DOMException("The operation was aborted.", "AbortError");
|
|
}
|
|
|
|
/**
|
|
* Non-authoritative observability hints (PAP-12556 / Phase 1). The server treats
|
|
* these as scheduling/telemetry only and never as security signals.
|
|
*/
|
|
function applyObservabilityHeaders(headers: Headers) {
|
|
if (headers.has("X-Paperclip-Tab-Visible")) return; // caller override wins
|
|
const visibility = getPageVisibility();
|
|
headers.set("X-Paperclip-Tab-Visible", getVisibilityHeaderValue(visibility));
|
|
if (typeof window !== "undefined" && window.location) {
|
|
headers.set("X-Paperclip-Route", window.location.pathname);
|
|
}
|
|
}
|
|
|
|
export async function requestResponse(path: string, init?: RequestInit): Promise<Response> {
|
|
const headers = new Headers(init?.headers ?? undefined);
|
|
const body = init?.body;
|
|
if (!(body instanceof FormData) && !headers.has("Content-Type")) {
|
|
headers.set("Content-Type", "application/json");
|
|
}
|
|
applyObservabilityHeaders(headers);
|
|
|
|
const res = await apiFetch(`${BASE}${path}`, {
|
|
credentials: "include",
|
|
...init,
|
|
headers,
|
|
});
|
|
if (!res.ok) {
|
|
const errorBody = await readApiJson(res);
|
|
const recovery = tenantSessionRecovery.recoverIfNeeded(res.status, errorBody);
|
|
if (recovery) return recovery;
|
|
throw apiErrorFromResponse(res, errorBody);
|
|
}
|
|
return res;
|
|
}
|
|
|
|
async function request<T>(path: string, init?: RequestInit): Promise<T> {
|
|
const res = await requestResponse(path, init);
|
|
if (res.status === 204) return undefined as T;
|
|
return readApiJson<T>(res);
|
|
}
|
|
|
|
// --- In-tab request coalescing for identical safe GETs -----------------------
|
|
//
|
|
// Multiple callers issuing the same GET while one is in flight share a single
|
|
// underlying fetch. Each caller keeps its own abort semantics: aborting one
|
|
// caller only cancels the shared fetch when *every* caller has aborted.
|
|
// Mutations are never coalesced.
|
|
|
|
interface InflightGet {
|
|
promise: Promise<unknown>;
|
|
controller: AbortController;
|
|
refs: Set<symbol>;
|
|
}
|
|
|
|
const inflightGets = new Map<string, InflightGet>();
|
|
|
|
function coalescedGet<T>(path: string, options?: RequestOptions): Promise<T> {
|
|
const signal = options?.signal;
|
|
if (signal?.aborted) return Promise.reject(abortError());
|
|
|
|
let entry = inflightGets.get(path);
|
|
if (!entry) {
|
|
const controller = new AbortController();
|
|
const promise = request<T>(path, {
|
|
method: "GET",
|
|
signal: controller.signal,
|
|
...(options?.cache ? { cache: options.cache } : {}),
|
|
});
|
|
const created: InflightGet = { promise, controller, refs: new Set() };
|
|
// Clear the shared entry once settled so later calls issue a fresh request.
|
|
promise.then(
|
|
() => {
|
|
if (inflightGets.get(path) === created) inflightGets.delete(path);
|
|
},
|
|
() => {
|
|
if (inflightGets.get(path) === created) inflightGets.delete(path);
|
|
},
|
|
);
|
|
inflightGets.set(path, created);
|
|
entry = created;
|
|
}
|
|
|
|
const activeEntry = entry;
|
|
const ref = Symbol("caller");
|
|
activeEntry.refs.add(ref);
|
|
|
|
const releaseRef = () => {
|
|
if (!activeEntry.refs.delete(ref)) return;
|
|
// Last caller gone before the fetch settled → abort the shared request.
|
|
if (activeEntry.refs.size === 0) {
|
|
if (inflightGets.get(path) === activeEntry) inflightGets.delete(path);
|
|
activeEntry.controller.abort();
|
|
}
|
|
};
|
|
|
|
return new Promise<T>((resolve, reject) => {
|
|
let timer: ReturnType<typeof setTimeout> | undefined;
|
|
const cleanup = () => {
|
|
signal?.removeEventListener("abort", onAbort);
|
|
clearTimeout(timer);
|
|
};
|
|
const onAbort = () => {
|
|
cleanup();
|
|
releaseRef();
|
|
reject(abortError());
|
|
};
|
|
if (signal) signal.addEventListener("abort", onAbort);
|
|
if (options?.timeoutMs !== undefined) {
|
|
timer = setTimeout(() => {
|
|
cleanup();
|
|
// Other callers keep their request, but retries must not rejoin it.
|
|
if (inflightGets.get(path) === activeEntry) inflightGets.delete(path);
|
|
releaseRef();
|
|
reject(new ApiError(errorCopy("TIMEOUT").body, 408, { error: "TIMEOUT" }));
|
|
}, options.timeoutMs);
|
|
}
|
|
|
|
activeEntry.promise.then(
|
|
(value) => {
|
|
cleanup();
|
|
activeEntry.refs.delete(ref);
|
|
resolve(value as T);
|
|
},
|
|
(err) => {
|
|
cleanup();
|
|
activeEntry.refs.delete(ref);
|
|
reject(err);
|
|
},
|
|
);
|
|
});
|
|
}
|
|
|
|
/**
|
|
* Stop later callers from joining the in-flight GET for `path`.
|
|
*
|
|
* Coalescing keys on the path alone, so a GET issued under one account's session
|
|
* can be joined by a caller that runs after the account changed — and handed the
|
|
* previous account's response. Detaching leaves that request to settle for the
|
|
* callers that asked for it, and makes the next call issue a fresh one. It does
|
|
* not abort, because those callers still want what they asked for.
|
|
*/
|
|
export function detachInflightGet(path: string): void {
|
|
inflightGets.delete(path);
|
|
}
|
|
|
|
function isRequestOptions(value: unknown): value is RequestOptions {
|
|
return typeof value === "object" && value !== null && "signal" in value;
|
|
}
|
|
|
|
export const api = {
|
|
get: <T>(path: string, options?: RequestOptions) => coalescedGet<T>(path, options),
|
|
post: <T>(path: string, body: unknown, options?: RequestOptions) =>
|
|
request<T>(path, {
|
|
method: "POST",
|
|
body: JSON.stringify(body),
|
|
signal: options?.signal,
|
|
...(options?.headers ? { headers: options.headers } : {}),
|
|
}),
|
|
postForm: <T>(path: string, body: FormData, options?: RequestOptions) =>
|
|
request<T>(path, {
|
|
method: "POST",
|
|
body,
|
|
signal: options?.signal,
|
|
// Never set Content-Type here — the browser sets multipart/form-data with
|
|
// the boundary. Extra headers (e.g. an async opt-in) may still ride along.
|
|
...(options?.headers ? { headers: options.headers } : {}),
|
|
}),
|
|
put: <T>(path: string, body: unknown, options?: RequestOptions) =>
|
|
request<T>(path, { method: "PUT", body: JSON.stringify(body), signal: options?.signal }),
|
|
/** Raw binary upload (e.g. one chunked import-transfer part); the body travels as-is. */
|
|
putRaw: <T>(path: string, body: Blob, options?: RequestOptions) =>
|
|
request<T>(path, {
|
|
method: "PUT",
|
|
body,
|
|
signal: options?.signal,
|
|
headers: { "Content-Type": "application/octet-stream", ...(options?.headers ?? {}) },
|
|
}),
|
|
patch: <T>(path: string, body: unknown, options?: RequestOptions) =>
|
|
request<T>(path, { method: "PATCH", body: JSON.stringify(body), signal: options?.signal }),
|
|
delete: <T>(path: string, bodyOrOptions?: unknown, options?: RequestOptions) => {
|
|
const requestOptions = isRequestOptions(bodyOrOptions) ? bodyOrOptions : options;
|
|
const body = bodyOrOptions === undefined || isRequestOptions(bodyOrOptions) ? undefined : JSON.stringify(bodyOrOptions);
|
|
return request<T>(path, { method: "DELETE", ...(body === undefined ? {} : { body }), signal: requestOptions?.signal });
|
|
},
|
|
deleteWithBody: <T>(path: string, body: unknown, options?: RequestOptions) =>
|
|
request<T>(path, { method: "DELETE", body: JSON.stringify(body), signal: options?.signal }),
|
|
};
|