PaperClipAI/doc/connections/BRAINTRUST.md

Braintrust

Updated: 2026-10-21. Status: US OAuth connected and exercised against an authorized account; US API-key and both EU methods still require live account qualification. The remaining qualification gaps are listed below.

Braintrust is a P2 resource connection using Paperclip's existing remote MCP transport, vault, catalog, access profiles, policy evaluation, run-scoped gateway, and audit. Sign-in tokens are never reused as resource tokens; id.paperclip.ing never stores resource tokens; no connections hub on the ID service.

Methods and setup

The curated definition is authored in scripts/app-definition-overrides/braintrust.json and regenerated into the shared catalog. All methods use Streamable HTTP, S4 risk, and the same read/write capability profile.

Method key	Account region	Authentication	MCP resource
mcp-oauth	US	OAuth; recommended default	https://api.braintrust.dev/mcp
mcp-api-key	US	API key	https://api.braintrust.dev/mcp
mcp-eu-oauth	EU	OAuth	https://api-eu.braintrust.dev/mcp
mcp-eu-api-key	EU	API key	https://api-eu.braintrust.dev/mcp

Open Apps → Braintrust or /apps/connect?source=braintrust, choose access, then select the method for the account's region. OAuth registers a client dynamically and requests only mcp. No client ID or secret is entered manually. For an API key, create a separate key in Braintrust Settings → API keys, then paste it into Paperclip's password field. Paperclip projects the write-only credential as Authorization: Bearer <key>. See the official MCP setup guide.

Braintrust OAuth and API keys inherit their account's permissions. Paperclip cannot grant an issued key additional access. See Braintrust's authentication model. Use the normal personal or shared credential identity and agent selection. Personal invocation credentials remain owned by their user; shared credentials use the company vault. Client-registration credentials are separate from invocation credentials. Agents receive gateway access, never the provider key.

There is no Paperclip project filter: project IDs in tool arguments do not establish an access boundary. Resource authorization remains with Braintrust, while Paperclip enforces company, grant, agent, task, and action policy.

Self-hosted Braintrust uses Connect your own MCP server with the deployment's MCP URL. See the generic remote MCP guide. This connector does not configure Paperclip's Braintrust trace exporter.

Public protocol evidence

Unauthenticated probes on 2026-10-21 returned an HTTP 401 challenge from both regional MCP resources. Each challenge identified its own protected-resource document, with the exact /mcp resource and same-origin authorization server.

Metadata or operation	US	EU
Protected resource	https://api.braintrust.dev/.well-known/oauth-protected-resource/mcp	https://api-eu.braintrust.dev/.well-known/oauth-protected-resource/mcp
Authorization server	https://api.braintrust.dev/.well-known/oauth-authorization-server	https://api-eu.braintrust.dev/.well-known/oauth-authorization-server
Authorization	https://api.braintrust.dev/oauth/authorize	https://api-eu.braintrust.dev/oauth/authorize
Token	https://api.braintrust.dev/oauth/token	https://api-eu.braintrust.dev/oauth/token
Registration	https://api.braintrust.dev/oauth/register	https://api-eu.braintrust.dev/oauth/register

Both servers advertise authorization_code and refresh_token, scope mcp, PKCE S256 and plain, and token authentication client_secret_post, client_secret_basic, and none. Paperclip uses PKCE S256 and declares only ownershipModes: ["dcr"]. Neither metadata document advertised CIMD or a revocation endpoint. No dynamic registration, consent, token exchange, or authenticated tool discovery was performed by these public probes.

Actions and governance

The Braintrust connector owns the reviewed tool-risk inventory. The connector policy registry selects it and declares catalog review behavior; the shared connection service applies those policies. Retrieval tools are reads, including SQL, schema and experiment inspection, object lookup, trace inspection, chart previews, documentation search, and skill loading. Loading a provider skill is a governed read; it does not install a Paperclip skill.

edit_dataset_rows is destructive because rows can be permanently deleted. set_automation_status is destructive because activation can enable retention or delivery automation. Other mutations and execution tools are writes, including evaluations, evaluator/facet/preprocessor tests, dataset additions, patterns, dashboards, alerts, online scoring, and Topics processing. Evaluations and model-backed processing can incur provider costs. Unknown tools default to write. Provider annotations may raise risk but cannot downgrade reviewed mutations. The permission reviews record the scope and provider-permission rationale for every method.

Risk classification does not change the central action defaults: discovered actions initially use Paperclip's current Allowed defaults. Operators can set actions to Ask first or Off on Permissions. Existing decisions survive reconnect and refresh. After initial discovery, new or changed tools are quarantined for review; reconnect and refresh preserve that setting.

Recovery and removal

Save & exit, cancellation, Finish setup, and reconnect use the existing setup controller. Resume the saved connection and its exact regional method. An invalid or expired key needs replacement; OAuth access failures need fresh authorization. Do not switch regions to repair a credential error. An insufficient-permission failure requires the owner to authorize an account or key with the intended Braintrust access.

Remove the connection in Paperclip to clear its local credentials and tool access. Deactivate an API key in Braintrust separately to invalidate copies outside Paperclip. API keys are displayed once; expired keys require replacement, as described in the authentication documentation. OAuth provider-side revocation is unqualified: the public metadata advertises no revocation endpoint, so local removal must not be described as remote token invalidation.

Qualification record

Public metadata is protocol evidence, not proof of account authorization or tool execution. An existing signed-in US account was authorized for a temporary personal connection in an isolated source-checkout instance on 2026-10-21.

Method	Public regional metadata	Connect / authenticated discovery	Bounded read / disposable write	Real run gateway / refresh or reconnect	Local removal / provider revocation / secret inspection
mcp-oauth	pass, US	pass; 49 tools	pass; synthetic insert and readback	pass; process-agent runs and OAuth reconnect; token expiry/refresh not forced	removal and structural redaction pass; remote revocation unqualified
mcp-api-key	pass, US; does not validate a key	not run; no test key supplied	not run	not run	not run
mcp-eu-oauth	pass, EU	not run; available account uses US data plane	not run	not run	not run
mcp-eu-api-key	pass, EU; does not validate a key	not run; no EU account/key supplied	not run	not run	not run

The live US connection discovered 17 reads and 32 non-read actions. Provider annotations raised several write classifications to destructive; unreviewed names remained writes. list_recent_objects performed a bounded read. edit_dataset_rows created one synthetic row in a new disposable dataset, and sql_query read it back. Both ran through real process-agent heartbeats and run-scoped gateway sessions. The qualification harness delivered the gateway capability through a private temporary file; this does not qualify automatic credential delivery for a native CLI harness. No provider credential was sent to the process agent and no evaluation/model execution was invoked.

Catalog refresh and OAuth reconnect retained the same connection, US method, 49 actions, and changed-tool quarantine configuration. A subsequent bounded read succeeded. The provider returned openid profile email offline_access in the token response scope despite the authorization request containing only mcp; Paperclip retained that discrepancy as provider metadata. Access-token expiry and refresh were tested with fixtures, not forced on this live grant.

Cleanup deleted the synthetic row, verified an empty result, and removed the new empty dataset in Braintrust after explicit approval. Local connection removal revoked two vault secrets and one grant, removed the installation and 49 profile entries, and rejected a subsequent call with tool_not_found. Qualification gateway sessions were revoked and their private files deleted. The owned server and database were stopped and the disposable instance data was removed. Remote OAuth token invalidation remains unqualified because no revocation endpoint is advertised.

Connection/catalog API responses contained no raw credential fields. The isolated server log had no bearer values, access/refresh-token values, or OAuth callback codes under credential-shaped scans. This was a structural redaction check, not an exact decrypted-token search. Non-submitted key canaries were absent from browser storage during the separate UI walkthrough. Fixture tests provide exact canary redaction assertions. The sanitized qualification record separates live, browser, and simulated-provider evidence.

Qualify each method independently in an owned isolated instance. Record the revision, deployment, region, credential identity, discovered inventory, action decision, and audit correlation. Use synthetic resources for writes; prove Allowed, Ask-first, Off, quarantine, wrong-owner, and cross-company behavior through the real run-scoped gateway. Verify cancellation/resume and reconnect without duplicate connections or regional changes. Inspect API responses, browser storage, logs, and stored secret references without copying tokens or customer traces into evidence. Fixture-backed verification establishes Paperclip behavior only; record its commands/results separately from live provider proof, following the connector playbook.

Deterministic verification

On 2026-10-21, the Braintrust service fixture exercised all four methods with a simulated provider and the actual encrypted vault, database, catalog, policy, and run-scoped gateway. It covered OAuth DCR/PKCE and cancellation/resume, personal/shared credential ownership, allowed reads and writes, approval and Off decisions, schema-change quarantine, refresh/reconnect, wrong-owner and cross-company denial, revocation, and redaction. See braintrust-connection.test.ts and the connector-owned risk.ts.

The catalog suite validates all four reviewed contracts. Apps setup tests cover each region/authentication choice and retrying the saved EU OAuth connection. Brand validation checks the official local artwork; provenance is recorded in braintrust-brand-provenance-2026-10-21.json. These checks establish Paperclip behavior with fixtures, not live Braintrust account compatibility.

The isolated browser walkthrough passed 15 checks covering Apps and direct-link entry, all four method choices and reloads, password-field validation, cancellation, personal/shared access controls, and official light/dark artwork. Its first pass exposed missing pre-auth caveats: warnings were present in the definition but were not rendered by the setup form. The capability summary and key helper now show account permissions, costs, permanent deletion, and automation risks before authorization; the repeated walkthrough passed.