From b033cc124dd2a40805366b7c5b7b734858468a6f Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Wed, 11 Nov 2026 22:06:29 -0600 Subject: [PATCH] fix(evals): repair private human steps for webhook, device and key entry; name failed private steps (#21012) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat evaluation catalog checks agent replies and then the human step that follows, such as a person who reveals a key or approves a device > - Some human steps run in a separate private browser process, because they handle credentials. CI keeps the output of that process private on purpose > - Four cases failed only in that private step, so CI showed one fixed message and no cause > - I ran each private step locally against a dev server, where its output is visible, and found the cause of three failures > - This pull request fixes the harness and two product defects, and adds a closed failure record so that the next CI failure names its step > - The benefit is that these human steps work, and a private-step failure is diagnosable without any credential in logs ## Linked Issues or Issue Description Bug report (following `.github/ISSUE_TEMPLATE/bug_report.yml`). **What happened?** - `routines.webhook-trigger`: the private person opens the routine overview. The trigger cards are only in the Triggers section, so the step waits for a missing button. An unhandled response wait then stops the process. - `connection-setup.secure-api-key`: a key-protected MCP server answers 401 without a challenge header. Paperclip shows "Remote app returned HTTP 401" and does not ask for a key. The person cannot enter the key in the normal flow. - `assistant-connections.device-login`: an expired device code page shows "This item doesn't exist or was moved." The server sends "This code is expired, already decided, or invalid." in the OAuth `error_description` field, and the UI drops it. - `onboarding.new-company`: the private step reads back all saved state and then fails one proof check. CI does not say which check. **Expected behavior** - The person reveals the webhook key in the Triggers section and sends authenticated events. - A pasted MCP link that answers 401 shows the key field. - An expired device code says that the code is expired. - A private-step failure names its fixed step and error class, with no value. **Steps to reproduce** 1. Start a local dev server. 2. Create a routine with a bearer webhook, and run the private webhook worker. It times out on the "Edit webhook" button. 3. Paste the address of an MCP server that answers a plain 401 into Connectors > Add by link. No key field appears. 4. Open `/mcp-device?user_code=`. The page shows generic not-found text. **Paperclip version or commit** `master` (reproduced on origin/master before this change) **Deployment mode** Local dev server (embedded PGlite) and the CI chat-eval runner **Installation method** Source checkout (`pnpm dev`) ## What Changed - `integrator-webhook-private-worker.ts`: open `/triggers` (also for the setup-resume link), and attach a handler to the early response wait so a failed click reports its real step. - `server/.../catalog/discovery.ts`: a 401 with no `WWW-Authenticate` header, to a connection that sends no credential, is handled like a challenge without discoverable sign-in (`oauth_challenge`). The existing UI guidance then asks for a key. A 401 to a connection that already sends a credential keeps the old error. - `ui/src/api/errors.ts`: read the OAuth `error_description` field as a readable server message. - New `private-child-failure.ts`: a private child records only a fixed step name and an error class from a fixed list. The parent echoes a step only when it is in the caller's closed list. Error text, stdout and stderr stay private. - The webhook, private browser and identity children use it. Each has a closed step list file. - `identity-new-company-native-proof.ts`: each proof check has a fixed reason name. The private child reports `new-company-proof-` when a check fails. The accepted conditions do not change. - Tests: server red/green test for the bare 401, UI red/green test for `error_description`, unit tests for the failure record (no text leaks, closed list only, extra fields dropped). ## Verification - Local dev server (embedded PostgreSQL), private workers run directly with the real child process: - Webhook, ready and setup-pending triggers: before, timeout on "Edit webhook". After, receipt with invalid status 401, valid status 202 and a linked task. - Secure key entry with the synthetic notes provider: before, timeout on the key field. After, an active connection with health `ok` and 2 authorized `tools/list` calls. - Expired device page: before, `at step device-expired-page (TimeoutError)`. After, `expiredPageVisible: true`. - `server`: `vitest run src/__tests__/tool-access-service.test.ts -t "pasted link answers"` (fails before the discovery change with 502, passes after). - `ui`: `vitest run src/api/errors.test.ts` (fails before, passes after). - `pnpm test:e2e:runner:typecheck`, `pnpm test:e2e:runner:unit`, server `tsc --noEmit`. - `node tests/runner-e2e/source-sdk-public-inputs.mjs --check`: unchanged. - One paid run of the four cases on this branch. Results are in a PR comment. ## Risks - A plain 401 from an MCP link now returns 422 `oauth_challenge` instead of 502. The draft connection is removed, as for other challenges. Agents that call the connect API see the "needs sign-in" answer instead of a raw status. This only applies when the connection sends no credential. - Some API errors that carry `error_description` now show that text instead of generic copy. That text is written for people. - The failure record is opt-in per child and contains only constants. The parent ignores a record with any other field. - `onboarding.new-company` is not fixed here. Its native hello probe cannot start locally on macOS, so the failing check is not yet known. This change makes the next run name it. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use (shell, file edits, local browser automation through Playwright). Extended reasoning was on. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 --- .../src/__tests__/tool-access-service.test.ts | 70 +++++++++++++++ .../services/tool-access/catalog/discovery.ts | 24 ++++- .../external-private-browser-steps.ts | 8 ++ .../external-private-browser-worker.ts | 30 ++++--- .../external-private-browser.ts | 4 +- .../identity-new-company-native-proof.ts | 88 +++++++++++++------ .../identity-new-company-native.test.ts | 9 +- .../identity-new-company-native.ts | 30 ++++--- .../identity-private-browser-steps.ts | 8 ++ .../identity-private-browser-worker.ts | 14 ++- .../identity-private-browser.ts | 6 +- .../integrator-webhook-private-steps.ts | 3 + .../integrator-webhook-private-worker.ts | 23 +++-- .../integrator-webhook-private.ts | 4 +- .../private-child-failure.test.ts | 44 ++++++++++ .../private-child-failure.ts | 37 ++++++++ ui/src/api/errors.test.ts | 9 ++ ui/src/api/errors.ts | 5 +- 18 files changed, 346 insertions(+), 70 deletions(-) create mode 100644 tests/runner-e2e/chat-catalog-domains/external-private-browser-steps.ts create mode 100644 tests/runner-e2e/chat-catalog-domains/identity-private-browser-steps.ts create mode 100644 tests/runner-e2e/chat-catalog-domains/integrator-webhook-private-steps.ts create mode 100644 tests/runner-e2e/chat-catalog-domains/private-child-failure.test.ts create mode 100644 tests/runner-e2e/chat-catalog-domains/private-child-failure.ts diff --git a/server/src/__tests__/tool-access-service.test.ts b/server/src/__tests__/tool-access-service.test.ts index 69525cf08e0..583f24963f2 100644 --- a/server/src/__tests__/tool-access-service.test.ts +++ b/server/src/__tests__/tool-access-service.test.ts @@ -16625,6 +16625,76 @@ describeEmbeddedPostgres("tool access service", () => { await expect(db.select().from(toolConnections)).resolves.toHaveLength(0); }); + it("asks for a key when a pasted link answers 401 without a challenge header", async () => { + const company = await createCompany(db); + const app = createRouteApp(db); + vi.spyOn(globalThis, "fetch").mockResolvedValue({ + ok: false, + status: 401, + headers: { get: () => null }, + text: async () => "", + json: async () => ({}), + } as unknown as Response); + + const res = await request(app) + .post(`/api/companies/${company.id}/tools/apps/connect`) + .send({ + link: "https://key.example.test/mcp", + name: "Key app", + }); + + expect(res.status).toBe(422); + expect(res.body).toMatchObject({ + details: expect.objectContaining({ code: "oauth_challenge" }), + }); + }); + + it("keeps the raw rejection when a pasted link's own URL token is refused", async () => { + const company = await createCompany(db); + const app = createRouteApp(db); + vi.spyOn(globalThis, "fetch").mockResolvedValue({ + ok: false, + status: 401, + headers: { get: () => null }, + text: async () => "", + json: async () => ({}), + } as unknown as Response); + + const res = await request(app) + .post(`/api/companies/${company.id}/tools/apps/connect`) + .send({ + link: "https://key.example.test/mcp?token=expired-url-token", + name: "URL token app", + }); + + expect(res.body?.details?.code).not.toBe("oauth_challenge"); + expect(res.status).not.toBe(422); + }); + + it("still asks for a key when a pasted link carries only plain settings", async () => { + const company = await createCompany(db); + const app = createRouteApp(db); + vi.spyOn(globalThis, "fetch").mockResolvedValue({ + ok: false, + status: 401, + headers: { get: () => null }, + text: async () => "", + json: async () => ({}), + } as unknown as Response); + + const res = await request(app) + .post(`/api/companies/${company.id}/tools/apps/connect`) + .send({ + link: "https://key.example.test/mcp?region=us", + name: "Regional key app", + }); + + expect(res.status).toBe(422); + expect(res.body).toMatchObject({ + details: expect.objectContaining({ code: "oauth_challenge" }), + }); + }); + it.each(["catalog", "catalog/refresh", "health-check"])( "explains disabled Slack MCP access on %s without reporting a server error", async (path) => { diff --git a/server/src/services/tool-access/catalog/discovery.ts b/server/src/services/tool-access/catalog/discovery.ts index d2caf7abc17..e55771ef20a 100644 --- a/server/src/services/tool-access/catalog/discovery.ts +++ b/server/src/services/tool-access/catalog/discovery.ts @@ -35,6 +35,14 @@ import type { ConnectionSetupPolicy } from "../connections/setup-policy.js"; import type { ToolRemoteHttp } from "../remote-http.js"; import { normalizeToolDescriptor, type McpToolDescriptor } from "./descriptors.js"; import { googleWorkspaceServicesUnavailable, unsupportedToolConnectionTransport } from "./failures.js"; + +const URL_CREDENTIAL_PARAMETER = /token|secret|passw|api[-_]?key|^key$|auth|signature|^sig$|session|credential/i; + +/** Whether a sent URL carries a credential: user info, or a query parameter named like one. Plain settings such as `?region=us` do not. */ +export function urlCarriesCredential(url: URL): boolean { + if (url.username || url.password) return true; + return [...url.searchParams.keys()].some((key) => URL_CREDENTIAL_PARAMETER.test(key)); +} function asRecord(value: unknown): Record { if (value && typeof value === "object" && !Array.isArray(value)) return value as Record; @@ -263,9 +271,23 @@ export function createToolCatalogDiscovery(db: Db, { now, audit, secrets, creden ); } const authenticate = response.headers.get("www-authenticate") ?? ""; + // A bare 401 to a request that carried no credential is the server's + // answer to "does it need a key?". Treat it like a challenge whose sign-in + // Paperclip cannot discover, so the person is asked for a key instead of + // seeing a raw HTTP status. A request that carried a credential (stored + // secret, header, or a token or user info in the URL) keeps the raw + // rejection: that credential needs replacing, not a new key. + const sentUrl = new URL(endpoint); + const bareCredentialChallenge = + !authenticate.trim() && + connection.authKind === "none" && + connection.credentialSecretRefs.length === 0 && + Object.keys(headers).length === 0 && + !urlCarriesCredential(sentUrl); if ( response.status === 401 && - /bearer|oauth|authorization/i.test(authenticate) + (/bearer|oauth|authorization/i.test(authenticate) || + bareCredentialChallenge) ) { const endpoints = await discoverOAuthEndpoints( connection, diff --git a/tests/runner-e2e/chat-catalog-domains/external-private-browser-steps.ts b/tests/runner-e2e/chat-catalog-domains/external-private-browser-steps.ts new file mode 100644 index 00000000000..fcac6263677 --- /dev/null +++ b/tests/runner-e2e/chat-catalog-domains/external-private-browser-steps.ts @@ -0,0 +1,8 @@ +/** Closed failure steps the private browser child may report: each operation + * mode, plus finer steps for device approval and secure key entry. */ +export const PRIVATE_BROWSER_STEPS = ["request", "launch", + "secure-key", "canary", "gateway-token-rotation", "company-secret-create", "personal-secret-create", "grok-routine-configure", "slack-identity-confirm", + "slack-endpoint-configure", "slack-wizard-confirm", "slack-wizard-resume", "slack-endpoint-reconnect", "device-approve", "device-expired", + "device-session", "device-describe", "device-expired-page", "device-consent-page", + "secure-key-address", "secure-key-field", "secure-key-connect", "secure-key-ready"] as const; +export type PrivateBrowserStep = typeof PRIVATE_BROWSER_STEPS[number]; diff --git a/tests/runner-e2e/chat-catalog-domains/external-private-browser-worker.ts b/tests/runner-e2e/chat-catalog-domains/external-private-browser-worker.ts index a1638f47294..50157010d80 100644 --- a/tests/runner-e2e/chat-catalog-domains/external-private-browser-worker.ts +++ b/tests/runner-e2e/chat-catalog-domains/external-private-browser-worker.ts @@ -4,12 +4,15 @@ import { validDevicePrivateInput, type DevicePrivateInput } from "./external-dev import { readFile, writeFile } from "node:fs/promises"; import path from "node:path"; import { chromium, test, type Cookie } from "@playwright/test"; +import { recordPrivateChildFailure } from "./private-child-failure.js"; +import type { PrivateBrowserStep } from "./external-private-browser-steps.js"; import { validGrokRoutinePrivateInput, matchesConfiguredGrokRoutine, type GrokRoutinePrivateInput } from "./external-grok-private-contract.js"; import { validSlackIdentityInput, validSlackWizardIdentityInput, matchingSlackIdentityPreview, type SlackIdentityPrivateInput } from "./external-slack-identity-contract.js"; import { validSlackSetupPrivate, type SlackSetupPrivateInput } from "./external-slack-setup-private-contract.js"; interface Request { mode: "secure-key" | "canary" | "gateway-token-rotation" | "company-secret-create" | "personal-secret-create" | "grok-routine-configure" | "slack-identity-confirm" | "slack-endpoint-configure" | "slack-wizard-confirm" | "slack-wizard-resume" | "slack-endpoint-reconnect" | "device-approve" | "device-expired"; device?: DevicePrivateInput; slackSetup?:SlackSetupPrivateInput; slackIdentity?:SlackIdentityPrivateInput; grok?:GrokRoutinePrivateInput; companyId?:string; companyMetadata?:{name?:string;key?:string}; personal?:{definitionId:string;userId:string;viaMySecrets?:boolean}; gateway?: { id: string; oldTokenId: string; otherToken: string; endpointPath: string; readName: string; pendingTokenId?: string }; baseURL: string; setupURL: string; providerURL: string; credential: string; cookies: Cookie[]; executablePath?: string; channel?: string } /** Separate Node process: no Playwright Test worker, API-step listener or artifact * recorder exists. This module never enables tracing, video or screenshots. */ +let step: PrivateBrowserStep = "request"; async function main() { let attached = false; try { test.info(); attached = true; } catch { /* expected outside Test */ } if (attached) throw new Error("Private browser worker cannot run inside Playwright Test"); @@ -17,24 +20,24 @@ async function main() { if (!input.credential || input.credential.length > (input.mode === "grok-routine-configure" ? 16384 : 8192) || !["secure-key", "canary", "gateway-token-rotation", "company-secret-create", "personal-secret-create", "grok-routine-configure", "slack-identity-confirm", "slack-endpoint-configure", "slack-wizard-confirm", "slack-wizard-resume", "slack-endpoint-reconnect", "device-approve", "device-expired"].includes(input.mode)) throw new Error("Invalid private browser request"); const base = new URL(input.baseURL); if (base.hostname !== "127.0.0.1" || base.protocol !== "http:" || new URL(input.setupURL).origin !== base.origin) throw new Error("Private browser target is outside the disposable loopback instance"); - const browser = await chromium.launch({ headless: true, ...(input.executablePath ? { executablePath: input.executablePath } : {}), ...(input.channel ? { channel: input.channel } : {}) }); + step = "launch"; const browser = await chromium.launch({ headless: true, ...(input.executablePath ? { executablePath: input.executablePath } : {}), ...(input.channel ? { channel: input.channel } : {}) }); try { const context = await browser.newContext({ baseURL: base.origin }); - await context.addCookies(input.cookies); const page = await context.newPage(); page.setDefaultTimeout(30_000); + await context.addCookies(input.cookies); const page = await context.newPage(); page.setDefaultTimeout(30_000); step = input.mode; if (input.mode === "device-approve" || input.mode === "device-expired") { if (!validDevicePrivateInput(input.mode, input.baseURL, input.setupURL, input.companyId, input.device, input.credential) || !input.device) throw new Error("Invalid finite device verification request"); - const session = await context.request.get("/api/auth/get-session"); + step = "device-session"; const session = await context.request.get("/api/auth/get-session"); if (!session.ok() || (await session.json()).user?.id !== input.device.userId) throw new Error("Device UI requires the same signed-in fixture person"); const description = page.waitForResponse(response => { const url = new URL(response.url()); return url.origin === base.origin && url.pathname === "/api/mcp/device" && url.searchParams.get("user_code") === input.credential && response.request().method() === "GET"; }); - await page.goto(input.setupURL, { waitUntil: "domcontentloaded" }); + step = "device-describe"; await page.goto(input.setupURL, { waitUntil: "domcontentloaded" }); const described = await description; if (input.mode === "device-expired") { const body = await described.json(); if (described.status() !== 404 || body.error !== "invalid_request") throw new Error("Expired device UI did not receive the actual issuer rejection"); - await page.getByText(/expired|invalid/i).first().waitFor(); + step = "device-expired-page"; await page.getByText(/expired|invalid/i).first().waitFor(); if (await page.getByRole("button", { name: "Connect organization", exact: true }).isVisible()) throw new Error("Expired device still offers approval"); return { processIsolation: true, testRunnerAttached: false, tracingEnabled: false, videoEnabled: false, apiStepRecorder: false, deviceSessionVerified: true, expiredPageVisible: true }; } @@ -42,13 +45,13 @@ async function main() { const request = await described.json(); if (request.clientName !== "Paperclip disposable CLI" || request.requestedCompanyId !== input.companyId || request.requiresSignIn !== false || request.requestedWrite !== false || request.requestedConfigure !== false || request.agentConnection === true || !request.companies?.some((company: { id: string; name: string }) => company.id === input.companyId && company.name === input.device!.companyName)) throw new Error("Device consent does not match the exact owned read-only request"); - await page.getByRole("heading", { name: "Connect Paperclip disposable CLI to Paperclip", exact: true }).waitFor(); + step = "device-consent-page"; await page.getByRole("heading", { name: "Connect Paperclip disposable CLI to Paperclip", exact: true }).waitFor(); await page.getByText(input.credential.toUpperCase(), { exact: true }).waitFor(); await page.getByText(input.device.companyName, { exact: true }).waitFor(); await page.getByText("Read all of your Paperclip data", { exact: true }).waitFor(); if (await page.getByText("Write all of your Paperclip data", { exact: true }).isVisible()) throw new Error("Device read-only request unexpectedly offers write authority"); const approval = page.waitForResponse(response => new URL(response.url()).origin === base.origin && new URL(response.url()).pathname === "/api/mcp/device/consent" && response.request().method() === "POST"); - await page.getByRole("button", { name: "Connect organization", exact: true }).click(); + step = "device-approve"; await page.getByRole("button", { name: "Connect organization", exact: true }).click(); const approved = await approval, submitted = approved.request().postDataJSON(); if (!approved.ok() || (await approved.json()).status !== "approved" || submitted.userCode !== input.credential || submitted.companyId !== input.companyId || submitted.decision !== "approve" || submitted.allowWrites !== false || submitted.allowConfiguration !== false) throw new Error("Actual device UI did not approve exactly the selected read-only company/code"); await page.getByRole("heading", { name: "Access approved", exact: true }).waitFor(); @@ -292,20 +295,23 @@ async function main() { await writeFile(path.join(path.dirname(process.argv[2]!), "token.json"), JSON.stringify({id:replacement.id,token:replacement.token}), {mode:0o600}); return {processIsolation:true,testRunnerAttached:false,tracingEnabled:false,videoEnabled:false,apiStepRecorder:false,newTokenId:replacement.id,tokenUiVisible:true,newWorkedBeforeRevocation,oldWorkedUntilRevocation,revealedPending:!!g.pendingTokenId,oldRevoked:true,newWorksAfter:fresh.ok,oldStatusAfter:old.status,otherWorksAfter:other.ok}; } - await page.goto(input.setupURL, { waitUntil: "domcontentloaded" }); + step = "secure-key-address"; await page.goto(input.setupURL, { waitUntil: "domcontentloaded" }); await page.getByRole("textbox", { name: "MCP server URL", exact: true }).fill(input.providerURL); await page.getByRole("button", { name: "Continue", exact: true }).click(); await page.getByRole("button", { name: "Check link", exact: true }).click(); - const key = page.getByLabel("App key", { exact: true }); await key.waitFor({ state: "visible" }); + step = "secure-key-field"; const key = page.getByLabel("App key", { exact: true }); await key.waitFor({ state: "visible" }); const passwordInput = await key.getAttribute("type") === "password", autocompleteOff = await key.getAttribute("autocomplete") === "off"; - await key.fill(input.credential); + step = "secure-key-connect"; await key.fill(input.credential); const created = page.waitForResponse(response => new URL(response.url()).pathname.endsWith("/tools/apps/connect") && response.request().method() === "POST" && response.status() === 201); await page.getByRole("button", { name: "Check link", exact: true }).click(); const connectionId = (await (await created).json() as { connectionId: string }).connectionId; if (!/^[a-f0-9-]{36}$/i.test(connectionId)) throw new Error("Secure setup did not return a bounded connection identity"); - await page.getByRole("heading", { name: /is ready\./i }).waitFor({ state: "visible" }); + step = "secure-key-ready"; await page.getByRole("heading", { name: /is ready\./i }).waitFor({ state: "visible" }); const storageClean = await page.evaluate(secret => !JSON.stringify([Object.entries(localStorage), Object.entries(sessionStorage), location.href]).includes(secret), input.credential); await page.getByRole("button", { name: "View connection", exact: true }).click(); await key.waitFor({ state: "detached" }); return { processIsolation: true, testRunnerAttached: false, tracingEnabled: false, videoEnabled: false, apiStepRecorder: false, connectionId, challengeVisible: true, passwordInput, autocompleteOff, humanSubmittedSecurely: true, visibleReady: true, storageClean }; } finally { await browser.close(); } } -main().then(receipt => process.stdout.write(JSON.stringify(receipt))).catch(() => { process.stderr.write("Private browser operation failed; diagnostics withheld"); process.exitCode = 1; }); +main().then(receipt => process.stdout.write(JSON.stringify(receipt))).catch(async (error: unknown) => { + await recordPrivateChildFailure(path.dirname(process.argv[2]!), step, error); + process.stderr.write("Private browser operation failed; diagnostics withheld"); process.exitCode = 1; +}); diff --git a/tests/runner-e2e/chat-catalog-domains/external-private-browser.ts b/tests/runner-e2e/chat-catalog-domains/external-private-browser.ts index 466f78fd350..08e3d6d9579 100644 --- a/tests/runner-e2e/chat-catalog-domains/external-private-browser.ts +++ b/tests/runner-e2e/chat-catalog-domains/external-private-browser.ts @@ -7,6 +7,8 @@ import { fileURLToPath } from "node:url"; import { execFile } from "node:child_process"; import { promisify } from "node:util"; import { chromium, type Cookie } from "@playwright/test"; +import { readPrivateChildFailure } from "./private-child-failure.js"; +import { PRIVATE_BROWSER_STEPS } from "./external-private-browser-steps.js"; import type { GrokRoutinePrivateInput } from "./external-grok-private-contract.js"; import type { SlackIdentityPrivateInput } from "./external-slack-identity-contract.js"; import type { SlackSetupPrivateInput } from "./external-slack-setup-private-contract.js"; @@ -25,7 +27,7 @@ export async function privateBrowserEntry(input: { mode: "secure-key" | "canary" try { const result = await promisify(execFile)(process.execPath, ["--import", loader, worker, request], { timeout: 120_000, maxBuffer: 16_384, env: { PATH: process.env.PATH ?? "", NODE_ENV: "test", DO_NOT_TRACK: "1", HOME: directory, TMPDIR: directory } }); output = result.stdout; - } catch { throw new Error("Private browser entry failed; child output and credential-bearing request withheld"); } + } catch { throw new Error(`Private browser entry failed${await readPrivateChildFailure(directory, PRIVATE_BROWSER_STEPS)}; child output and credential-bearing request withheld`); } let receipt: PrivateBrowserReceipt; try { receipt = JSON.parse(output) as PrivateBrowserReceipt; } catch { throw new Error("Private browser returned invalid closed receipt"); } if (receipt.processIsolation !== true || receipt.testRunnerAttached !== false || receipt.tracingEnabled !== false || receipt.videoEnabled !== false || receipt.apiStepRecorder !== false || Object.keys(receipt).some(key => !["processIsolation", "testRunnerAttached", "tracingEnabled", "videoEnabled", "apiStepRecorder", "credentialEntered", "connectionId", "challengeVisible", "passwordInput", "autocompleteOff", "humanSubmittedSecurely", "visibleReady", "storageClean", "newTokenId", "tokenUiVisible", "newWorkedBeforeRevocation", "oldWorkedUntilRevocation", "revealedPending", "oldRevoked", "newWorksAfter", "oldStatusAfter", "otherWorksAfter", "requestedCompanyName", "requestedCompanyKey", "savedCompanyName", "savedCompanyKey", "secretId", "secretUiVisible", "bindingId", "routineConfigured", "routineUiVisible", "identityConfirmed", "identityUiVisible", "principalId", "paperclipUserId", "endpointId", "slackCredentialUiVisible", "activationUiVisible", "endpointActive", "identityWasAlreadyLinked", "credentialUiRejected", "deviceSessionVerified", "currentCodeVisible", "companyConsentVisible", "readScopeVisible", "humanApproved", "expiredPageVisible"].includes(key))) throw new Error("Private browser receipt exceeds its closed contract"); diff --git a/tests/runner-e2e/chat-catalog-domains/identity-new-company-native-proof.ts b/tests/runner-e2e/chat-catalog-domains/identity-new-company-native-proof.ts index 8c39ceaedb1..850734de2fd 100644 --- a/tests/runner-e2e/chat-catalog-domains/identity-new-company-native-proof.ts +++ b/tests/runner-e2e/chat-catalog-domains/identity-new-company-native-proof.ts @@ -1,5 +1,7 @@ import type { AiConnectionBinding, AiConnectionList } from "../../../packages/shared/src/ai-connections.js"; import type { ConnectionGrantsResponse, ToolConnectionInstall } from "../../../packages/shared/src/types/tool-access.js"; +import { AGENT_STATUSES } from "../../../packages/shared/src/constants.js"; +import { AGENT_LIFECYCLE_STATES } from "../../../packages/shared/src/types/agent-lifecycle.js"; export const NEW_COMPANY_NATIVE_MODEL = "openrouter/deepseek/deepseek-v4.1-flash-1017"; export const NEW_COMPANY_PROVIDER_MODEL = "deepseek/deepseek-v4.1-flash-1017"; @@ -37,6 +39,20 @@ export function nativeSetupRequest(input: { companyId: string; adapterType: stri binding: (body?.aiConnection ?? (body?.runtimeConfig as { aiConnection?: AiConnectionBinding } | undefined)?.aiConnection) as AiConnectionBinding | undefined, containsCredential: Boolean(credential && JSON.stringify(input.body).includes(credential)) }; } +// Step names use hyphens; product enum values may use underscores. +function agentLifecycleReason(state: string) { return `agent-lifecycle-${state.replaceAll("_", "-")}` as `agent-lifecycle-${string}`; } +function agentStatusReason(status: string) { return `agent-status-${status.replaceAll("_", "-")}` as `agent-status-${string}`; } +export const NATIVE_PROOF_FAILURE_REASONS = ["environment", "probe-count", "hire-count", + "probe-agent", "probe-environment", "probe-model", "probe-credential", "probe-binding", "probe-result", "probe-order", + "hire-agent", "hire-environment", "hire-model", "hire-credential", "hire-binding", "hire-result", + "agent-identity", "agent-model", "agent-environment", + ...[...AGENT_LIFECYCLE_STATES, "unknown"].map(state => agentLifecycleReason(state)), + ...[...AGENT_STATUSES, "unknown"].map(status => agentStatusReason(status)), "agent-heartbeat", "agent-binding", "responsible-user", + "account", "account-routing", "grant", "install"] as const; +export type NativeProofFailureReason = typeof NATIVE_PROOF_FAILURE_REASONS[number]; +export class NativeProofFailure extends Error { + constructor(readonly reason: NativeProofFailureReason, message: string) { super(message); } +} export function requireNativeFirstAgentProof(input: { company: NativeCompany; agent: NativeFirstAgent; humanId: string; environment: { id: string; driver: string }; savedSetupAgentId: string; probes: NativeSetupProbe[]; hires: Array; @@ -45,34 +61,50 @@ export function requireNativeFirstAgentProof(input: { }): void { const { company, agent, humanId, environment, savedSetupAgentId, probes, hires, connection, accounts, grants, installs } = input; const binding = agent.runtimeConfig.aiConnection; - const matches = (request: NativeSetupRequest) => request.companyId === company.id && request.agentId === savedSetupAgentId - && request.environmentId === environment.id && request.adapterType === "paperclip_runner" && request.provider === "opencode" - && request.model === NEW_COMPANY_NATIVE_MODEL && request.containsCredential === false - && request.binding?.provider === "openrouter" && request.binding.method === "api_key" && request.binding.mode === "delegated" - && request.binding.connectionId === connection.connectionId && request.binding.grantId === connection.grantId; - if (environment.driver !== "local" || !environment.id || probes.length !== 1 || hires.length !== 1 - || !matches(probes[0]!) || probes[0]!.httpStatus !== 200 || probes[0]!.resultAdapterType !== "paperclip_runner" || probes[0]!.status !== "pass" - || probes[0]!.checks.some(check => check.level === "error") || !probes[0]!.checks.some(check => check.code === "opencode_hello_probe_passed") - || !Number.isInteger(probes[0]!.sequence) || probes[0]!.sequence < 1 || !Number.isInteger(hires[0]!.sequence) || probes[0]!.sequence >= hires[0]!.sequence - || !matches(hires[0]!) || hires[0]!.httpStatus < 200 || hires[0]!.httpStatus >= 300 || hires[0]!.resultAgentId !== savedSetupAgentId) - throw new Error("New company requires one current same-agent native OpenCode account/model/local-target test and hire"); - if (agent.id !== savedSetupAgentId || agent.companyId !== company.id || agent.role !== "ceo" || agent.adapterType !== "paperclip_runner" - || agent.adapterConfig.provider !== "opencode" || agent.adapterConfig.model !== NEW_COMPANY_NATIVE_MODEL - || agent.defaultEnvironmentId !== environment.id || agent.lifecycleState !== "ready" || !["idle", "active"].includes(agent.status) - || agent.runtimeConfig.heartbeat?.enabled !== false - || binding?.provider !== "openrouter" || binding.method !== "api_key" || binding.mode !== "delegated" - || binding.connectionId !== connection.connectionId || binding.grantId !== connection.grantId - || company.defaultResponsibleUserId !== humanId || accounts.currentUserId !== humanId) - throw new Error("New company public first-agent identity/configuration differs from the tested setup"); + // Each failed check names a fixed reason, so the private child can report + // which one failed without exposing any value. + const check = (ok: boolean, reason: NativeProofFailureReason, message: string) => { if (!ok) throw new NativeProofFailure(reason, message); }; + const testAndHire = "New company requires one current same-agent native OpenCode account/model/local-target test and hire"; + const request = (value: NativeSetupRequest, prefix: "probe" | "hire") => { + check(value.companyId === company.id && value.agentId === savedSetupAgentId, `${prefix}-agent`, testAndHire); + check(value.environmentId === environment.id, `${prefix}-environment`, testAndHire); + check(value.adapterType === "paperclip_runner" && value.provider === "opencode" && value.model === NEW_COMPANY_NATIVE_MODEL, `${prefix}-model`, testAndHire); + check(value.containsCredential === false, `${prefix}-credential`, testAndHire); + check(value.binding?.provider === "openrouter" && value.binding.method === "api_key" && value.binding.mode === "delegated" + && value.binding.connectionId === connection.connectionId && value.binding.grantId === connection.grantId, `${prefix}-binding`, testAndHire); + }; + check(environment.driver === "local" && !!environment.id, "environment", testAndHire); + check(probes.length === 1, "probe-count", testAndHire); + check(hires.length === 1, "hire-count", testAndHire); + const probe = probes[0]!, hire = hires[0]!; + request(probe, "probe"); + check(probe.httpStatus === 200 && probe.resultAdapterType === "paperclip_runner" && probe.status === "pass" + && !probe.checks.some(check => check.level === "error") && probe.checks.some(check => check.code === "opencode_hello_probe_passed"), "probe-result", testAndHire); + check(Number.isInteger(probe.sequence) && probe.sequence >= 1 && Number.isInteger(hire.sequence) && probe.sequence < hire.sequence, "probe-order", testAndHire); + request(hire, "hire"); + check(hire.httpStatus >= 200 && hire.httpStatus < 300 && hire.resultAgentId === savedSetupAgentId, "hire-result", testAndHire); + const configuration = "New company public first-agent identity/configuration differs from the tested setup"; + check(agent.id === savedSetupAgentId && agent.companyId === company.id && agent.role === "ceo", "agent-identity", configuration); + check(agent.adapterType === "paperclip_runner" && agent.adapterConfig.provider === "opencode" && agent.adapterConfig.model === NEW_COMPANY_NATIVE_MODEL, "agent-model", configuration); + check(agent.defaultEnvironmentId === environment.id, "agent-environment", configuration); + // The observed state is reported only through the closed product enums. + const lifecycle = (AGENT_LIFECYCLE_STATES as readonly string[]).includes(agent.lifecycleState) ? agent.lifecycleState : "unknown"; + const status = (AGENT_STATUSES as readonly string[]).includes(agent.status) ? agent.status : "unknown"; + check(agent.lifecycleState === "ready", agentLifecycleReason(lifecycle), configuration); + check(["idle", "active"].includes(agent.status), agentStatusReason(status), configuration); + check(agent.runtimeConfig.heartbeat?.enabled === false, "agent-heartbeat", configuration); + check(binding?.provider === "openrouter" && binding.method === "api_key" && binding.mode === "delegated" + && binding.connectionId === connection.connectionId && binding.grantId === connection.grantId, "agent-binding", configuration); + check(company.defaultResponsibleUserId === humanId && accounts.currentUserId === humanId, "responsible-user", configuration); + const accountMessage = "New company account requires its actual personal grant and exact first-agent public installation"; const account = accounts.connections.find(row => row.id === connection.connectionId && row.grantId === connection.grantId); const grant = grants.grants.find(row => row.id === connection.grantId); - if (!account || account.companyId !== company.id || account.provider !== "openrouter" || account.method !== "api_key" - || account.status !== "connected" || account.ownership !== "personal" || account.ownerUserId !== humanId - || account.routing?.kind !== "openrouter" || account.routing.protocol !== "chat" || account.routing.auth !== "bearer" - || account.routing.models.length !== 1 || account.routing.models[0]?.id !== NEW_COMPANY_PROVIDER_MODEL - || grants.connection.id !== connection.connectionId || grants.currentUserId !== humanId || grant?.connectionId !== connection.connectionId - || grant.companyId !== company.id || grant.kind !== "user" || grant.subjectUserId !== humanId || grant.status !== "active" - || installs.length !== 1 || installs[0]?.companyId !== company.id || installs[0]?.connectionId !== connection.connectionId - || installs[0]?.targetType !== "agent" || installs[0]?.targetId !== agent.id) - throw new Error("New company account requires its actual personal grant and exact first-agent public installation"); + check(!!account && account.companyId === company.id && account.provider === "openrouter" && account.method === "api_key" + && account.status === "connected" && account.ownership === "personal" && account.ownerUserId === humanId, "account", accountMessage); + check(account!.routing?.kind === "openrouter" && account!.routing.protocol === "chat" && account!.routing.auth === "bearer" + && account!.routing.models.length === 1 && account!.routing.models[0]?.id === NEW_COMPANY_PROVIDER_MODEL, "account-routing", accountMessage); + check(grants.connection.id === connection.connectionId && grants.currentUserId === humanId && grant?.connectionId === connection.connectionId + && grant.companyId === company.id && grant.kind === "user" && grant.subjectUserId === humanId && grant.status === "active", "grant", accountMessage); + check(installs.length === 1 && installs[0]?.companyId === company.id && installs[0]?.connectionId === connection.connectionId + && installs[0]?.targetType === "agent" && installs[0]?.targetId === agent.id, "install", accountMessage); } diff --git a/tests/runner-e2e/chat-catalog-domains/identity-new-company-native.test.ts b/tests/runner-e2e/chat-catalog-domains/identity-new-company-native.test.ts index 957b19df23e..8b13339c9ea 100644 --- a/tests/runner-e2e/chat-catalog-domains/identity-new-company-native.test.ts +++ b/tests/runner-e2e/chat-catalog-domains/identity-new-company-native.test.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from "vitest"; import { readFileSync } from "node:fs"; -import { NEW_COMPANY_NATIVE_MODEL, NEW_COMPANY_PROVIDER_MODEL, nativeSetupRequest, requireNativeFirstAgentProof } from "./identity-new-company-native-proof.js"; +import { NEW_COMPANY_NATIVE_MODEL, NEW_COMPANY_PROVIDER_MODEL, NativeProofFailure, nativeSetupRequest, requireNativeFirstAgentProof } from "./identity-new-company-native-proof.js"; import { aiRoutingModel } from "../../../packages/shared/src/ai-provider-routing.js"; import { createAiConnectionSchema } from "../../../packages/shared/src/ai-connections.js"; import { resolveRouteOnboardingOptions } from "../../../ui/src/lib/onboarding-route.js"; @@ -24,6 +24,13 @@ function proof(): Proof { } describe("native first-agent public setup custody", () => { it("accepts the same saved CEO/account/local model test and hire", () => expect(() => requireNativeFirstAgentProof(proof())).not.toThrow()); + it("names a not-yet-ready agent by its closed lifecycle and status values", () => { + const reason = (mutate: (p: Proof) => void) => { const p = proof(); mutate(p); try { requireNativeFirstAgentProof(p); } catch (error) { return (error as NativeProofFailure).reason; } return null; }; + expect(reason(p => { p.agent.lifecycleState = "verifying"; })).toBe("agent-lifecycle-verifying"); + expect(reason(p => { p.agent.lifecycleState = "pending_approval"; })).toBe("agent-lifecycle-pending-approval"); + expect(reason(p => { p.agent.status = "error"; })).toBe("agent-status-error"); + expect(reason(p => { p.agent.lifecycleState = "not-a-state"; })).toBe("agent-lifecycle-unknown"); + }); for (const [name, mutate] of Object.entries<(p: Proof) => void>({ "remote target": p => { p.environment.driver = "remote"; }, "wrong selected target": p => { p.probes[0]!.environmentId = "other"; }, diff --git a/tests/runner-e2e/chat-catalog-domains/identity-new-company-native.ts b/tests/runner-e2e/chat-catalog-domains/identity-new-company-native.ts index 613271a62c5..d721d1c9b53 100644 --- a/tests/runner-e2e/chat-catalog-domains/identity-new-company-native.ts +++ b/tests/runner-e2e/chat-catalog-domains/identity-new-company-native.ts @@ -3,8 +3,9 @@ import { privateIdentityBrowser } from "./identity-private-browser.js"; import { pollUntil } from "../api.js"; import type { AiConnectionList } from "../../../packages/shared/src/ai-connections.js"; import type { ConnectionGrantsResponse, ToolConnectionInstall } from "../../../packages/shared/src/types/tool-access.js"; -import { NEW_COMPANY_NATIVE_MODEL, NEW_COMPANY_PROVIDER_MODEL, nativeSetupRequest, requireNativeFirstAgentProof, +import { NEW_COMPANY_NATIVE_MODEL, NEW_COMPANY_PROVIDER_MODEL, NativeProofFailure, nativeSetupRequest, requireNativeFirstAgentProof, type NativeCompany, type NativeFirstAgent, type NativeSetupProbe, type NativeSetupRequest, type NewCompanyNativeReceipt } from "./identity-new-company-native-proof.js"; +import type { IdentityPrivateStep } from "./identity-private-browser-steps.js"; export type { NewCompanyNativeReceipt } from "./identity-new-company-native-proof.js"; interface Input { page: Page; origin: string; companyName: string; agentName: string; credential: string; deadlineAt: number; @@ -23,7 +24,10 @@ export async function finishNativeNewCompany(input: Input): Promise; registerConnection(companyId: string, connectionId: string, grantId: string): Promise; + /** Names the fixed step in progress, for the child's closed failure record. */ + step?(name: IdentityPrivateStep): void; }): Promise { + const step = (name: IdentityPrivateStep) => input.step?.(name); let attached = false; try { test.info(); attached = true; } catch { /* standalone only */ } if (attached) throw new Error("Credential-bearing company setup requires the private child"); const { page, origin } = input; @@ -32,7 +36,7 @@ export async function runNativeNewCompanyInPrivateBrowser(input: Input & { const response = await page.context().request.get(origin + endpoint, { timeout: Math.min(30_000, remaining()) }); if (!response.ok()) throw new Error(`Public company setup read returned HTTP ${response.status()}`); return response.json() as Promise; }; - const session = await get<{ user: { id: string } }>("/api/auth/get-session"); + step("new-company-create"); const session = await get<{ user: { id: string } }>("/api/auth/get-session"); const before = await get("/api/companies"); await page.getByRole("button", { name: /^Open .+ (?:organization|workspace|company) switcher$/i }).click(); await page.getByRole("menuitem", { name: "Create organization", exact: true }).click(); @@ -47,7 +51,7 @@ export async function runNativeNewCompanyInPrivateBrowser(input: Input & { if ((await get(`/api/companies/${company.id}/agents`)).length) throw new Error("The wizard created an agent before the public route departure"); // Ordinary address-bar navigation to a supported public route. No hidden // dialog action, draft edit or forced localStorage state is used. - const publicSetupRoute = `/${company.issuePrefix}/agents/new`; + step("new-company-agent-route"); const publicSetupRoute = `/${company.issuePrefix}/agents/new`; await page.goto(new URL(publicSetupRoute, origin).href, { waitUntil: "domcontentloaded" }); await expect(page.getByTestId("onboarding-wizard")).not.toBeVisible(); await expect(page.getByRole("heading", { name: "Meet your next agent", exact: true })).toBeVisible(); @@ -58,7 +62,7 @@ export async function runNativeNewCompanyInPrivateBrowser(input: Input & { if (setupAgents.length !== 1 || setupAgents[0]?.name !== input.agentName || setupAgents[0]?.role !== "ceo" || setupAgents[0]?.lifecycleState === "ready") throw new Error("Public first-agent dialog did not save one setup-held CEO"); const provisional = setupAgents[0]!; - await page.getByRole("dialog").locator("summary").filter({ hasText: /^Advanced$/ }).click(); + step("new-company-configure"); await page.getByRole("dialog").locator("summary").filter({ hasText: /^Advanced$/ }).click(); await page.getByRole("button", { name: "Experimental harness", exact: true }).click(); await page.getByRole("option", { name: "OpenCode (Paperclip Runner)", exact: true }).click(); await page.getByRole("button", { name: "Configure agent", exact: true }).click(); @@ -101,7 +105,7 @@ export async function runNativeNewCompanyInPrivateBrowser(input: Input & { page.on("response", onResponse); const settleReads = async () => { await Promise.all([...reads].map(read => read.catch(() => {}))); if (errors.length) throw new Error("Public company setup response identity check failed"); }; try { - await page.getByRole("combobox", { name: "Connection", exact: true }).click(); + step("new-company-account"); await page.getByRole("combobox", { name: "Connection", exact: true }).click(); await page.getByRole("option", { name: "Connect an account…", exact: true }).click(); const accountDialog = page.getByRole("dialog").filter({ has: page.getByRole("heading", { name: "Connect account", exact: true }) }); await accountDialog.locator("summary").filter({ hasText: /^Advanced providers$/ }).click(); @@ -117,24 +121,28 @@ export async function runNativeNewCompanyInPrivateBrowser(input: Input & { await accountDialog.getByRole("button", { name: "Connect", exact: true }).click(); await expect(accountDialog).not.toBeVisible({ timeout: remaining() }); await settleReads(); if (!connection) throw new Error("Public new-company account creation receipt missing"); - await page.getByRole("button", { name: "Run test", exact: true }).click(); + step("new-company-test"); await page.getByRole("button", { name: "Run test", exact: true }).click(); await expect(page.getByRole("button", { name: "Test again", exact: true })).toBeVisible({ timeout: remaining() }); await settleReads(); if (probes.length !== 1 || probes[0]?.status !== "pass") throw new Error("The selected native account/model test did not pass"); - await page.getByRole("button", { name: "Finish setup", exact: true }).click(); - await expect(page.getByRole("button", { name: new RegExp(`^Assign ${input.agentName} a Task$`) })).toBeVisible({ timeout: remaining() }); await settleReads(); + step("new-company-hire"); await page.getByRole("button", { name: "Finish setup", exact: true }).click(); + // The button appears while Paperclip is still preparing the agent; a person + // assigns work only once it is enabled, when the agent is ready. + await expect(page.getByRole("button", { name: new RegExp(`^Assign ${input.agentName} a Task$`) })).toBeEnabled({ timeout: remaining() }); await settleReads(); } finally { page.off("response", onResponse); } - const agents = await get(`/api/companies/${company.id}/agents`); + step("new-company-readback"); const agents = await get(`/api/companies/${company.id}/agents`); if (agents.length !== 1 || agents[0]?.id !== provisional.id) throw new Error("Public setup duplicated or replaced the first agent"); const agent = await get(`/api/agents/${provisional.id}`); const responsibleCompany = await get(`/api/companies/${company.id}`); const accounts = await get(`/api/companies/${company.id}/ai-connections?agentId=${agent.id}`); const grants = await get(`/api/tool-connections/${connection!.connectionId}/grants`); const installs = await get<{ installs: ToolConnectionInstall[] }>(`/api/tool-connections/${connection!.connectionId}/installs`); - requireNativeFirstAgentProof({ company: responsibleCompany, agent, humanId: session.user.id, environment, savedSetupAgentId: provisional.id, probes, hires, connection: connection!, accounts, grants, installs: installs.installs }); + try { requireNativeFirstAgentProof({ company: responsibleCompany, agent, humanId: session.user.id, environment, savedSetupAgentId: provisional.id, probes, hires, connection: connection!, accounts, grants, installs: installs.installs }); } + catch (error) { if (error instanceof NativeProofFailure) step(`new-company-proof-${error.reason}`); throw error; } + step("new-company-credential-scan"); if (JSON.stringify(agent).includes(input.credential)) throw new Error("The public first-agent record exposes a credential; values withheld"); const draftContainsCredential = await page.evaluate(secret => Object.values(localStorage).some(value => String(value).includes(secret)), input.credential); if (draftContainsCredential) throw new Error("The public setup draft exposes a credential; values withheld"); - const settings = await get<{ enableAgentChat: boolean }>("/api/instance/settings/experimental"); + step("new-company-chat"); const settings = await get<{ enableAgentChat: boolean }>("/api/instance/settings/experimental"); if (!settings.enableAgentChat) throw new Error("The actual new-company chat entry is unavailable"); await page.goto(`${origin}/${company.issuePrefix}/chats/${agent.id}`, { waitUntil: "domcontentloaded" }); await expect(page.getByTestId("task-chat-composer-input")).toBeVisible(); diff --git a/tests/runner-e2e/chat-catalog-domains/identity-private-browser-steps.ts b/tests/runner-e2e/chat-catalog-domains/identity-private-browser-steps.ts new file mode 100644 index 00000000000..08cf8d5190d --- /dev/null +++ b/tests/runner-e2e/chat-catalog-domains/identity-private-browser-steps.ts @@ -0,0 +1,8 @@ +import { NATIVE_PROOF_FAILURE_REASONS } from "./identity-new-company-native-proof.js"; + +/** Closed failure steps the private identity child may report. */ +export const IDENTITY_PRIVATE_STEPS = ["request", "launch", "sign-in", "wizard", "new-company-native", "observability", "canary-failure", + "new-company-create", "new-company-agent-route", "new-company-configure", "new-company-account", "new-company-test", "new-company-hire", + "new-company-readback", "new-company-credential-scan", "new-company-chat", + ...NATIVE_PROOF_FAILURE_REASONS.map(reason => `new-company-proof-${reason}` as const)] as const; +export type IdentityPrivateStep = typeof IDENTITY_PRIVATE_STEPS[number]; diff --git a/tests/runner-e2e/chat-catalog-domains/identity-private-browser-worker.ts b/tests/runner-e2e/chat-catalog-domains/identity-private-browser-worker.ts index d6f4ff28b76..482e97ada9c 100644 --- a/tests/runner-e2e/chat-catalog-domains/identity-private-browser-worker.ts +++ b/tests/runner-e2e/chat-catalog-domains/identity-private-browser-worker.ts @@ -1,10 +1,13 @@ import { readFile, writeFile } from "node:fs/promises"; import path from "node:path"; import { chromium, test, expect } from "@playwright/test"; +import { recordPrivateChildFailure } from "./private-child-failure.js"; +import type { IdentityPrivateStep } from "./identity-private-browser-steps.js"; import type { PrivateIdentityOperation, PrivateIdentityReceipt } from "./identity-private-browser.js"; import { runIdentityWizardInPrivateBrowser } from "./identity-wizard.js"; import { runNativeNewCompanyInPrivateBrowser } from "./identity-new-company-native.js"; let privateValues: string[] = []; +let step: IdentityPrivateStep = "request"; async function main(): Promise { try { test.info(); throw new Error("Attached Test worker refused"); } catch (error) { if (error instanceof Error && error.message === "Attached Test worker refused") throw error; } const request = process.argv[2]!; const directory = path.dirname(request); @@ -13,10 +16,10 @@ async function main(): Promise { const origin = new URL(input.origin); if (origin.hostname !== "127.0.0.1" || origin.protocol !== "http:" || new URL(input.startPath, origin).origin !== origin.origin) throw new Error("Private identity target must be a disposable loopback instance"); if (!["sign-in", "wizard", "new-company-native", "observability", "canary-failure"].includes(input.kind)) throw new Error("Unknown finite identity operation"); - const browser = await chromium.launch({ headless: true, ...(input.executablePath ? { executablePath: input.executablePath } : {}), ...(input.channel ? { channel: input.channel } : {}) }); + step = "launch"; const browser = await chromium.launch({ headless: true, ...(input.executablePath ? { executablePath: input.executablePath } : {}), ...(input.channel ? { channel: input.channel } : {}) }); try { const context = await browser.newContext({ baseURL: origin.origin }); await context.addCookies(input.cookies); const page = await context.newPage(); page.setDefaultTimeout(30_000); - const receipt: PrivateIdentityReceipt = { operation: input.kind, completed: true, processIsolation: true, testRunnerAttached: false, apiStepRecorder: false, tracingEnabled: false, videoEnabled: false }; + step = input.kind; const receipt: PrivateIdentityReceipt = { operation: input.kind, completed: true, processIsolation: true, testRunnerAttached: false, apiStepRecorder: false, tracingEnabled: false, videoEnabled: false }; if (input.kind === "canary-failure") { await page.setContent(''); await page.getByLabel("Private credential", { exact: true }).fill(input.credential); @@ -37,7 +40,7 @@ async function main(): Promise { await writeFile(path.join(directory, "storage.json"), JSON.stringify(await context.storageState()), { mode: 0o600 }); } else if (input.kind === "new-company-native") { const connections: Array<{ companyId: string; connectionId: string; grantId: string }> = []; - receipt.newCompany = await runNativeNewCompanyInPrivateBrowser({ ...input, page, onCompany() {}, onConnection() {}, + receipt.newCompany = await runNativeNewCompanyInPrivateBrowser({ ...input, page, onCompany() {}, onConnection() {}, step(name) { step = name; }, async registerCompany(company) { await writeFile(path.join(directory, "company.json"), JSON.stringify({ id: company.id }), { mode: 0o600 }); }, async registerConnection(companyId, connectionId, grantId) { connections.push({ companyId, connectionId, grantId }); await writeFile(path.join(directory, "connections.json"), JSON.stringify(connections), { mode: 0o600 }); }, evidence: async () => {} }); } else if (input.kind === "wizard") { @@ -54,4 +57,7 @@ async function main(): Promise { return receipt; } finally { await browser.close(); } } -main().then(receipt => { const output = JSON.stringify(receipt); if (privateValues.some(value => [value, encodeURIComponent(value), Buffer.from(value).toString("base64")].some(encoded => output.includes(encoded)))) throw new Error("Private identity receipt contains a credential"); process.stdout.write(output); }).catch(() => { process.stderr.write("Private identity browser operation failed; details withheld"); process.exitCode = 1; }); +main().then(receipt => { const output = JSON.stringify(receipt); if (privateValues.some(value => [value, encodeURIComponent(value), Buffer.from(value).toString("base64")].some(encoded => output.includes(encoded)))) throw new Error("Private identity receipt contains a credential"); process.stdout.write(output); }).catch(async (error: unknown) => { + await recordPrivateChildFailure(path.dirname(process.argv[2]!), step, error); + process.stderr.write("Private identity browser operation failed; details withheld"); process.exitCode = 1; +}); diff --git a/tests/runner-e2e/chat-catalog-domains/identity-private-browser.ts b/tests/runner-e2e/chat-catalog-domains/identity-private-browser.ts index 3e4f0c47167..e228acf0c13 100644 --- a/tests/runner-e2e/chat-catalog-domains/identity-private-browser.ts +++ b/tests/runner-e2e/chat-catalog-domains/identity-private-browser.ts @@ -5,9 +5,11 @@ import { tmpdir } from "node:os"; import path from "node:path"; import { fileURLToPath } from "node:url"; import { chromium, type Cookie } from "@playwright/test"; +import { readPrivateChildFailure } from "./private-child-failure.js"; +import { IDENTITY_PRIVATE_STEPS } from "./identity-private-browser-steps.js"; import type { WizardReceipt, WizardHarness } from "./identity-wizard.js"; import type { NewCompanyNativeReceipt } from "./identity-new-company-native-proof.js"; -export const identityPrivateBrowserSources = ["chat-catalog-domains/identity-private-browser.ts", "chat-catalog-domains/identity-private-browser-worker.ts"]; +export const identityPrivateBrowserSources = ["chat-catalog-domains/identity-private-browser.ts", "chat-catalog-domains/identity-private-browser-worker.ts", "chat-catalog-domains/identity-private-browser-steps.ts", "chat-catalog-domains/private-child-failure.ts"]; export type PrivateIdentityOperation = | { kind: "sign-in"; origin: string; startPath: string; returnPath: string; email: string; password: string; cookies: Cookie[] } | { kind: "wizard"; origin: string; startPath: string; cookies: Cookie[]; credential: string; companyName: string; agentName: string; harness: WizardHarness; resume?: boolean; entry?: "empty" | "new-company"; deadlineAt: number } @@ -27,7 +29,7 @@ export async function privateIdentityBrowser(input: PrivateIdentityOperation, ca let output: string; try { output = (await promisify(execFile)(process.execPath, ["--import", loader, worker, request], { timeout: (input.kind === "wizard" || input.kind === "new-company-native") ? Math.max(1, Math.min(540_000, input.deadlineAt - Date.now())) : 120_000, maxBuffer: 32_768, env: { PATH: process.env.PATH ?? "", NODE_ENV: "test", DO_NOT_TRACK: "1", HOME: directory, TMPDIR: directory } })).stdout; } - catch { throw new Error("Private identity browser operation failed; child diagnostics withheld"); } + catch { throw new Error(`Private identity browser operation failed${await readPrivateChildFailure(directory, IDENTITY_PRIVATE_STEPS)}; child diagnostics withheld`); } let receipt: PrivateIdentityReceipt; try { receipt = JSON.parse(output) as PrivateIdentityReceipt; } catch { throw new Error("Invalid closed private identity receipt"); } if (receipt.operation !== input.kind || receipt.completed !== true || receipt.processIsolation !== true || receipt.testRunnerAttached !== false || receipt.apiStepRecorder !== false || receipt.tracingEnabled !== false || receipt.videoEnabled !== false diff --git a/tests/runner-e2e/chat-catalog-domains/integrator-webhook-private-steps.ts b/tests/runner-e2e/chat-catalog-domains/integrator-webhook-private-steps.ts new file mode 100644 index 00000000000..44984241f53 --- /dev/null +++ b/tests/runner-e2e/chat-catalog-domains/integrator-webhook-private-steps.ts @@ -0,0 +1,3 @@ +/** Closed failure steps the private webhook child may report. */ +export const WEBHOOK_PRIVATE_STEPS = ["request", "launch", "open-triggers", "reveal-key", "finish-setup", "invalid-delivery", "valid-delivery"] as const; +export type WebhookPrivateStep = typeof WEBHOOK_PRIVATE_STEPS[number]; diff --git a/tests/runner-e2e/chat-catalog-domains/integrator-webhook-private-worker.ts b/tests/runner-e2e/chat-catalog-domains/integrator-webhook-private-worker.ts index 137057c081b..9e22d2c6f96 100644 --- a/tests/runner-e2e/chat-catalog-domains/integrator-webhook-private-worker.ts +++ b/tests/runner-e2e/chat-catalog-domains/integrator-webhook-private-worker.ts @@ -2,29 +2,33 @@ import { readFile, writeFile } from "node:fs/promises"; import path from "node:path"; import { createHmac } from "node:crypto"; import { chromium, test, type Cookie } from "@playwright/test"; +import { recordPrivateChildFailure } from "./private-child-failure.js"; +import { WEBHOOK_PRIVATE_STEPS, type WebhookPrivateStep } from "./integrator-webhook-private-steps.js"; interface Request { baseURL: string; route: string; routineId: string; triggerId: string; setupPending: boolean; signingMode: string; nonce: string; cookies: Cookie[]; executablePath?: string; channel?: string } +let step: WebhookPrivateStep = WEBHOOK_PRIVATE_STEPS[0]; async function main() { let attached = false; try { test.info(); attached = true; } catch { /* no Test worker */ } if (attached) throw new Error("Private webhook driver must be outside Test"); const request = process.argv[2]!; const input = JSON.parse(await readFile(request, "utf8")) as Request; const base = new URL(input.baseURL); if (base.protocol !== "http:" || base.hostname !== "127.0.0.1" || !/^[a-f0-9-]{36}$/i.test(input.triggerId) || !/^[a-f0-9-]{36}$/i.test(input.routineId) || !input.route.endsWith(`/routines/${input.routineId}`)) throw new Error("Private webhook target is outside its exact disposable scope"); - const browser = await chromium.launch({ headless: true, ...(input.executablePath ? { executablePath: input.executablePath } : {}), ...(input.channel ? { channel: input.channel } : {}) }); + step = "launch"; const browser = await chromium.launch({ headless: true, ...(input.executablePath ? { executablePath: input.executablePath } : {}), ...(input.channel ? { channel: input.channel } : {}) }); try { const context = await browser.newContext({ baseURL: base.origin }); await context.addCookies(input.cookies); const page = await context.newPage(); page.setDefaultTimeout(30_000); - const url = new URL(input.route, base); if (input.setupPending) url.searchParams.set("triggerSetup", input.triggerId); + // Triggers live in their own routine section; the overview has no trigger cards. + step = "open-triggers"; const url = new URL(`${input.route}/triggers`, base); if (input.setupPending) url.searchParams.set("triggerSetup", input.triggerId); await page.goto(url.href, { waitUntil: "domcontentloaded" }); - const rotated = page.waitForResponse(response => new URL(response.url()).pathname === `/api/routine-triggers/${input.triggerId}/rotate-secret` && response.request().method() === "POST" && response.ok()); - if (input.setupPending) await page.getByRole("button", { name: "Generate new key", exact: true }).click(); + const rotated = page.waitForResponse(response => new URL(response.url()).pathname === `/api/routine-triggers/${input.triggerId}/rotate-secret` && response.request().method() === "POST" && response.ok()); rotated.catch(() => undefined); + step = "reveal-key"; if (input.setupPending) await page.getByRole("button", { name: "Generate new key", exact: true }).click(); else { const card = page.getByRole("region", { name: "Webhook trigger", exact: true }); await card.getByRole("button", { name: "Edit webhook", exact: true }).click(); await card.getByRole("button", { name: "Replace key", exact: true }).click(); await page.getByRole("dialog").getByRole("button", { name: "Replace key", exact: true }).click(); } const material = await (await rotated).json() as { secretMaterial: { webhookSecret: string; webhookUrl: string } }; const secret = material.secretMaterial.webhookSecret; const endpoint = new URL(material.secretMaterial.webhookUrl); if (!/^[a-f0-9]{48}$/i.test(secret) || endpoint.origin !== base.origin || !/^\/api\/routine-triggers\/public\/[^/]+\/fire$/.test(endpoint.pathname)) throw new Error("Unexpected one-time scoped webhook credential"); await page.locator("code").filter({ hasText: secret }).first().waitFor({ state: "visible" }); - if (input.setupPending) { await page.getByRole("button", { name: "Check connection", exact: true }).click(); + step = "finish-setup"; if (input.setupPending) { await page.getByRole("button", { name: "Check connection", exact: true }).click(); const finished = page.waitForResponse(response => new URL(response.url()).pathname === `/api/routine-triggers/${input.triggerId}` && response.request().method() === "PATCH" && response.ok()); await page.getByRole("button", { name: /^Finish (?:setup|without checking)$/ }).click(); await finished; } const cookie = input.cookies.map(value => `${value.name}=${value.value}`).join("; "); const readRuns = async () => { const response = await fetch(new URL(`/api/routines/${input.routineId}/runs`, base), { redirect: "error", headers: cookie ? { cookie } : {}, signal: AbortSignal.timeout(15_000) }); if (!response.ok) throw new Error("Invocation proof unavailable"); return await response.json() as Array<{ id: string }>; }; - const before = await readRuns(); const payload = JSON.stringify({ event: "synthetic.test-service", delivery: input.nonce }); + step = "invalid-delivery"; const before = await readRuns(); const payload = JSON.stringify({ event: "synthetic.test-service", delivery: input.nonce }); const bad = await fetch(endpoint, { method: "POST", redirect: "error", headers: { "content-type": "application/json", authorization: "Bearer deliberately-invalid-eval-key", "idempotency-key": `${input.nonce}-bad` }, body: payload, signal: AbortSignal.timeout(15_000) }); const invalidStatus = bad.status; await bad.body?.cancel(); const afterInvalid = await readRuns(); const headers: Record = { "content-type": "application/json", "idempotency-key": `${input.nonce}-valid` }; @@ -32,11 +36,14 @@ async function main() { else if (["github_hmac", "fireflies_hmac"].includes(input.signingMode)) headers[input.signingMode === "github_hmac" ? "x-hub-signature-256" : "x-hub-signature"] = `sha256=${createHmac("sha256", secret).update(payload).digest("hex")}`; else if (input.signingMode === "hmac_sha256") { const timestamp = String(Math.floor(Date.now() / 1000)); headers["x-paperclip-timestamp"] = timestamp; headers["x-paperclip-signature"] = `sha256=${createHmac("sha256", secret).update(`${timestamp}.${payload}`).digest("hex")}`; } else throw new Error("Webhook did not require authenticated delivery"); - const accepted = await fetch(endpoint, { method: "POST", redirect: "error", headers, body: payload, signal: AbortSignal.timeout(15_000) }); + step = "valid-delivery"; const accepted = await fetch(endpoint, { method: "POST", redirect: "error", headers, body: payload, signal: AbortSignal.timeout(15_000) }); const validStatus = accepted.status; if (validStatus !== 202) throw new Error("Authenticated synthetic delivery was not accepted"); const result = await accepted.json() as { id?: string; linkedIssueId?: string }; await writeFile(path.join(path.dirname(request), "secret.txt"), secret, { mode: 0o600 }); return { processIsolation: true, testRunnerAttached: false, credentialVisibleInHumanUI: true, setupFinished: true, invalidStatus, validStatus, invalidCreatedInvocation: afterInvalid.length !== before.length, invocationId: result.id, linkedIssueId: result.linkedIssueId, triggerId: input.triggerId }; } finally { await browser.close(); } } -main().then(receipt => process.stdout.write(JSON.stringify(receipt))).catch(() => { process.stderr.write("Private webhook operation failed; diagnostics withheld"); process.exitCode = 1; }); +main().then(receipt => process.stdout.write(JSON.stringify(receipt))).catch(async (error: unknown) => { + await recordPrivateChildFailure(path.dirname(process.argv[2]!), step, error); + process.stderr.write("Private webhook operation failed; diagnostics withheld"); process.exitCode = 1; +}); diff --git a/tests/runner-e2e/chat-catalog-domains/integrator-webhook-private.ts b/tests/runner-e2e/chat-catalog-domains/integrator-webhook-private.ts index 656cdcfefba..4f31da56fd9 100644 --- a/tests/runner-e2e/chat-catalog-domains/integrator-webhook-private.ts +++ b/tests/runner-e2e/chat-catalog-domains/integrator-webhook-private.ts @@ -5,6 +5,8 @@ import { fileURLToPath } from "node:url"; import { execFile } from "node:child_process"; import { promisify } from "node:util"; import { chromium, type Cookie } from "@playwright/test"; +import { readPrivateChildFailure } from "./private-child-failure.js"; +import { WEBHOOK_PRIVATE_STEPS } from "./integrator-webhook-private-steps.js"; export interface WebhookHumanReceipt { processIsolation: true; testRunnerAttached: false; credentialVisibleInHumanUI: true; setupFinished: true; invalidStatus: number; validStatus: number; invalidCreatedInvocation: boolean; invocationId: string; linkedIssueId: string; triggerId: string } export async function finishPrivateWebhook(input: { baseURL: string; route: string; routineId: string; triggerId: string; setupPending: boolean; signingMode: string; nonce: string; cookies: Cookie[] }, registerSecret: (value: string) => void): Promise { const dir = await mkdtemp(path.join(tmpdir(), "paperclip-routine-webhook-private-")); @@ -15,7 +17,7 @@ export async function finishPrivateWebhook(input: { baseURL: string; route: stri const worker = fileURLToPath(new URL("./integrator-webhook-private-worker.ts", import.meta.url)); const loader = fileURLToPath(new URL("../../../server/node_modules/tsx/dist/loader.mjs", import.meta.url)); let output: string; try { output = (await promisify(execFile)(process.execPath, ["--import", loader, worker, request], { timeout: 120_000, maxBuffer: 8192, env: { PATH: process.env.PATH ?? "", NODE_ENV: "test", DO_NOT_TRACK: "1", HOME: dir, TMPDIR: dir } })).stdout; } - catch { throw new Error("Private routine webhook setup failed; credential-bearing child diagnostics withheld"); } + catch { throw new Error(`Private routine webhook setup failed${await readPrivateChildFailure(dir, WEBHOOK_PRIVATE_STEPS)}; credential-bearing child diagnostics withheld`); } const secret = await readFile(path.join(dir, "secret.txt"), "utf8"); if (!/^[a-f0-9]{48}$/i.test(secret)) throw new Error("Invalid private one-time webhook credential handoff"); registerSecret(secret); const receipt = JSON.parse(output) as WebhookHumanReceipt; if (receipt.processIsolation !== true || receipt.testRunnerAttached !== false || receipt.credentialVisibleInHumanUI !== true || receipt.setupFinished !== true || receipt.triggerId !== input.triggerId diff --git a/tests/runner-e2e/chat-catalog-domains/private-child-failure.test.ts b/tests/runner-e2e/chat-catalog-domains/private-child-failure.test.ts new file mode 100644 index 00000000000..70f8d08b222 --- /dev/null +++ b/tests/runner-e2e/chat-catalog-domains/private-child-failure.test.ts @@ -0,0 +1,44 @@ +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import { afterEach, describe, expect, it } from "vitest"; +import { PRIVATE_CHILD_FAILURE_FILENAME, readPrivateChildFailure, recordPrivateChildFailure } from "./private-child-failure.js"; +import { WEBHOOK_PRIVATE_STEPS } from "./integrator-webhook-private-steps.js"; +import { PRIVATE_BROWSER_STEPS } from "./external-private-browser-steps.js"; +import { IDENTITY_PRIVATE_STEPS } from "./identity-private-browser-steps.js"; + +const secret = "a".repeat(48); +const directories: string[] = []; +async function directory() { const value = await mkdtemp(path.join(tmpdir(), "private-child-failure-")); directories.push(value); return value; } +afterEach(async () => { for (const value of directories.splice(0)) await rm(value, { recursive: true, force: true }); }); + +describe("private child failure record", () => { + it("reports only the fixed step and error class, never the error text", async () => { + const dir = await directory(); + const error = new Error(`locator.click: Timeout waiting for ${secret}`); error.name = "TimeoutError"; + await recordPrivateChildFailure(dir, "reveal-key", error); + const raw = await readFile(path.join(dir, PRIVATE_CHILD_FAILURE_FILENAME), "utf8"); + expect(raw).not.toContain(secret); + expect(await readPrivateChildFailure(dir, WEBHOOK_PRIVATE_STEPS)).toBe(" at step reveal-key (TimeoutError)"); + }); + + it("never echoes a step outside the caller's closed list or an unknown error class", async () => { + const dir = await directory(); + await writeFile(path.join(dir, PRIVATE_CHILD_FAILURE_FILENAME), JSON.stringify({ step: secret, errorClass: secret })); + expect(await readPrivateChildFailure(dir, WEBHOOK_PRIVATE_STEPS)).toBe(" at step unknown (Error)"); + const custom = new Error("x"); custom.name = secret; + const other = await directory(); await recordPrivateChildFailure(other, `step ${secret}`, custom); + expect(await readFile(path.join(other, PRIVATE_CHILD_FAILURE_FILENAME), "utf8")).not.toContain(secret); + }); + + it("drops a record with extra fields and reports nothing when the child left none", async () => { + const dir = await directory(); + expect(await readPrivateChildFailure(dir, WEBHOOK_PRIVATE_STEPS)).toBe(""); + await writeFile(path.join(dir, PRIVATE_CHILD_FAILURE_FILENAME), JSON.stringify({ step: "launch", errorClass: "Error", message: secret })); + expect(await readPrivateChildFailure(dir, WEBHOOK_PRIVATE_STEPS)).toBe(""); + }); + + it("keeps every declared step a credential-free constant", () => { + for (const step of [...WEBHOOK_PRIVATE_STEPS, ...PRIVATE_BROWSER_STEPS, ...IDENTITY_PRIVATE_STEPS]) expect(step).toMatch(/^[a-z][a-z0-9-]{0,63}$/); + }); +}); diff --git a/tests/runner-e2e/chat-catalog-domains/private-child-failure.ts b/tests/runner-e2e/chat-catalog-domains/private-child-failure.ts new file mode 100644 index 00000000000..ebfccd8e610 --- /dev/null +++ b/tests/runner-e2e/chat-catalog-domains/private-child-failure.ts @@ -0,0 +1,37 @@ +import { readFile, writeFile } from "node:fs/promises"; +import path from "node:path"; + +/** A credential-bearing child keeps its stdout, stderr and error text private. + * On failure it may record only which fixed step failed and the error's class, + * so a CI failure is diagnosable without exposing any value. */ +export const PRIVATE_CHILD_FAILURE_FILENAME = "failure.json"; +const ERROR_CLASSES = ["TimeoutError", "AbortError", "TypeError", "SyntaxError", "RangeError", "Error"] as const; +type ErrorClass = typeof ERROR_CLASSES[number]; +const STEP = /^[a-z][a-z0-9-]{0,63}$/; + +export function privateChildErrorClass(error: unknown): ErrorClass { + const name = error instanceof Error ? error.name : ""; + return (ERROR_CLASSES as readonly string[]).includes(name) ? name as ErrorClass : "Error"; +} + +/** Child side. `step` must be one of the caller's own constants. */ +export async function recordPrivateChildFailure(directory: string, step: string, error: unknown) { + try { + await writeFile(path.join(directory, PRIVATE_CHILD_FAILURE_FILENAME), + JSON.stringify({ step: STEP.test(step) ? step : "unknown", errorClass: privateChildErrorClass(error) }), { mode: 0o600, flag: "wx" }); + } catch { /* the parent still reports a withheld failure */ } +} + +/** Parent side. Returns text safe for public errors, or "" when the child left + * no valid record. Only steps from the caller's closed list are echoed. */ +export async function readPrivateChildFailure(directory: string, steps: readonly string[]): Promise { + try { + const raw = await readFile(path.join(directory, PRIVATE_CHILD_FAILURE_FILENAME), "utf8"); + if (raw.length > 256) return ""; + const value = JSON.parse(raw) as Record; + if (!value || typeof value !== "object" || Object.keys(value).some(key => key !== "step" && key !== "errorClass")) return ""; + const step = typeof value.step === "string" && steps.includes(value.step) ? value.step : "unknown"; + const errorClass = (ERROR_CLASSES as readonly unknown[]).includes(value.errorClass) ? value.errorClass as ErrorClass : "Error"; + return ` at step ${step} (${errorClass})`; + } catch { return ""; } +} diff --git a/ui/src/api/errors.test.ts b/ui/src/api/errors.test.ts index 79ce5a6f006..26f4238b8dd 100644 --- a/ui/src/api/errors.test.ts +++ b/ui/src/api/errors.test.ts @@ -122,6 +122,15 @@ describe("describeError", () => { }); }); + it("keeps an OAuth error description instead of generic not-found copy", () => { + const error = apiError(404, { + error: "invalid_request", + error_description: "This code is expired, already decided, or invalid. Start a new connection from your assistant.", + }); + expect(describeError(error).body).toBe("This code is expired, already decided, or invalid. Start a new connection from your assistant."); + expect(error.message).toBe("This code is expired, already decided, or invalid. Start a new connection from your assistant."); + }); + it("falls back to the kind's copy when the server sent a code or nothing", () => { expect(describeError(apiError(404, { error: "issue_not_found" })).body).toBe("This item doesn’t exist or was moved."); expect(describeError(apiError(403, null))).toMatchObject({ title: "Not allowed", retryable: false }); diff --git a/ui/src/api/errors.ts b/ui/src/api/errors.ts index 44c992dda01..a76be6093a9 100644 --- a/ui/src/api/errors.ts +++ b/ui/src/api/errors.ts @@ -185,7 +185,10 @@ export function isRawErrorCode(value: string): boolean { /** A server message that is safe to show: not a raw code, not a generic status line. */ function readableServerMessage(err: unknown): string | null { const error = record(err); - const candidates = [stringField(record(error?.body)?.message), stringField(record(error?.body)?.error)]; + const body = record(error?.body); + // OAuth endpoints (RFC 6749) put the readable sentence in `error_description` + // and a machine code in `error`. + const candidates = [stringField(body?.message), stringField(body?.error), stringField(body?.error_description)]; if (err instanceof Error) candidates.push(stringField(err.message)); for (const candidate of candidates) { if (!candidate) continue;