docs(connections): make the generic remote MCP route the documented baseline

Add GENERIC-REMOTE-MCP.md: the two operator routes, what the guided flow does with each kind of endpoint, header-safety rules, and the registration preference order with its client-binding and re-registration behaviour. Correct the connector playbook, which read as though a curated `AppDefinition` were required and as though `ownershipModes` gated registration generally. It now opens by saying a catalog entry is the optional branded convenience layer, and its DCR section describes all four registration tiers, the public-HTTPS requirement for CIMD, the issuer/resource/callback/company binding, and the path-aware discovery and `iss`/`resource` conformance the broker now implements. Note on the PostHog doc that the curated flow is preferable but not required — which is exactly what the acceptance demo has to establish. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-26 10:26:26 +01:00 · 2026-08-20 21:20:01 +00:00
1 parent 7e5898d69a
commit 8cbf83a097
@@ -4,6 +4,19 @@ This playbook is the repeatable template for adding a vendor to the Apps catalog

 Use it when Paperclip acts on an external system through a governed connection: a stored credential, a capability catalog, access profiles and policy rules, and audit. Inbound integrations, such as an external client acting on Paperclip, use gateway or webhook guidance instead.

+**A catalog entry is a convenience layer, not a prerequisite.** Since
+[PAP-17087](/PAP/issues/PAP-17087), an operator can connect any
+standards-compliant remote HTTP MCP server from **Connect your own MCP server**
+or **Paste a config** with no Paperclip code change at all — including servers
+that need browser sign-in. Those two routes are the documented baseline; see
+[Connecting any remote MCP server](./GENERIC-REMOTE-MCP.md).
+
+Write a catalog entry when Paperclip should *promote* a vendor: branding, tailored
+fields, field validation, scoped defaults, and support copy. A definition adds
+those conveniences and nothing else. It must not create a second connection,
+change ownership, or be necessary for health, catalog, or governance — a curated
+route and the generic route converge on the same connection and review pipeline.
+
 Every connector built with this playbook is a **plane P2** connection — a resource token in the instance vault, acquired via the connect broker, never a sign-in authenticator. Before writing a connector, read [Identity vs. connections](./README.md#identity-vs-connections) for the P1/P2/P3 boundary and the D7 standing rule (sign-in tokens are never reused as resource tokens; id.paperclip.ing never stores resource tokens; no connections hub on the ID service).

 ## Output
@@ -210,6 +223,15 @@ most; the broker resolves endpoints at connect time:
   `authorization_endpoint`, `token_endpoint`, and — when the vendor supports
   dynamic registration — `registration_endpoint`.

+For an issuer that has a path, step 3 is tried in both the RFC 8414 insertion
+form (`/.well-known/oauth-authorization-server<path>`) and the widely deployed
+OIDC suffix form (`<path>/.well-known/oauth-authorization-server`), and a
+document whose `issuer` disagrees with the issuer used to build the URL is
+discarded. Authorization, token, and refresh requests all carry the RFC 8707
+`resource` indicator naming the canonical MCP endpoint, and RFC 9207 `iss` is
+validated against the persisted expected issuer when the authorization server
+returns it.
+
 The broker implements this in `discoverOAuthEndpoints`
 (`server/src/services/tool-access.ts`), but discovery is **not**
 unconditional. `oauthEndpointsForConnection` resolves endpoints in this
@@ -259,6 +281,24 @@ registers a client on the fly and stores it on the connection:
  `customer` and `dcr` in the method's `ownershipModes` when the vendor
  supports both.

+Since [PAP-17087](/PAP/issues/PAP-17087), DCR is **one of four** registration
+tiers, and `ownershipModes` gates only the *curated* path. The broker resolves a
+client in this order: a deployment-preconfigured client, then a Client ID
+Metadata Document when the authorization server advertises one (requires a public
+HTTPS `PAPERCLIP_PUBLIC_URL`), then DCR, then client credentials the operator
+preregistered and pasted in. A URL-only connection with no `AppDefinition` may
+use the CIMD and DCR tiers too, but only after validated protected-resource and
+authorization-server discovery produced a metadata document. Registered client
+material is bound to the issuer, MCP resource URL, callback URI, and company;
+when a binding moves, a Paperclip-minted client re-registers and an
+operator-supplied one asks the operator to re-enter it. Full detail in
+[Connecting any remote MCP server](./GENERIC-REMOTE-MCP.md#how-sign-in-gets-a-client).
+
+For a curated entry, `ownershipModes` still decides whether Paperclip may
+dynamically register on that vendor's behalf: omit `dcr` for a vendor that must
+not be auto-registered, and the broker will not fall through to the generic
+registration path for it.
+
 **DCR needs neither Paperclip ID nor Paperclip Connect.** DCR is always
 instance-local (ratified in the PAP-14828 connector-service spec, section 10
 item 8.4: "DCR is always instance-local; the service has no DCR involvement").
@@ -0,0 +1,157 @@
+# Connecting any remote MCP server
+
+Paperclip can connect a standards-compliant remote HTTP MCP server without a
+Paperclip code change. A curated `AppDefinition` is a **convenience layer** —
+branding, tailored fields, scoped defaults, support copy — not a prerequisite.
+
+This is the documented baseline for connecting anything. Read
+[Connector playbook](./CONNECTOR-PLAYBOOK.md) when you want to add the branded
+convenience layer on top for a vendor Paperclip should promote.
+
+Accepted in the [generic remote MCP plan](/PAP/issues/PAP-17078#document-plan),
+implemented in [PAP-17087](/PAP/issues/PAP-17087).
+
+## The two routes
+
+| Route | Where | Use it when |
+| --- | --- | --- |
+| Guided URL | **Apps → Connect an app → Connect your own MCP server** | You have the server's address. Paperclip probes it and walks you through whatever it needs. |
+| Paste a config | **Advanced → Paste a config** | A README gave you an `mcpServers` snippet, or the server needs headers with names Paperclip could not guess. |
+
+Both routes normalize through the same backend contract, so auth discovery,
+secret handling, catalog refresh and review cannot diverge between them.
+
+**Advanced → Run your own** is a separate, higher-trust path for local stdio
+commands and is deliberately not covered here.
+
+Don't know the address or the headers? The question-mark control beside
+**Paste a config** gives you a request you can hand to an agent: it asks the
+agent to consult the vendor's current documentation and reply with one
+paste-ready `mcpServers` JSON object using credential *placeholders*, plus notes
+on how to obtain each credential. Paste only the JSON block back into Paperclip;
+Paperclip reads the header names from it and asks you for the values, which it
+stores as Paperclip secrets.
+
+## What the guided URL flow does
+
+After you paste an address and press **Check link**, Paperclip probes the
+endpoint and branches:
+
+| Endpoint says | You get |
+| --- | --- |
+| Nothing needed | Discovered actions, straight to review. |
+| Needs authorization, and publishes discoverable OAuth metadata | **Sign in to continue** — a browser sign-in at the provider. |
+| Needs authorization, but no discoverable sign-in | A prompt to add the key or headers its docs list, under **Advanced authentication**. |
+| Needs a client you registered yourself | A prompt for a client ID and secret. The draft connection is kept — you don't start over. |
+| Not a valid address / private network / unreachable | The specific problem and which field to change. |
+
+An unknown server is labelled **Unverified server** with its host shown, at every
+step through review, access and install. Reads are enabled for review;
+state-changing actions start off; newly discovered actions are quarantined until
+reviewed. That is the same treatment a curated connection gets.
+
+### Advanced authentication
+
+Collapsed by default. Open it when the server's docs are specific:
+
+- **No sign-in needed** — the server is open to anyone with the address.
+- **Key or token** — sent as an `Authorization` header.
+- **Custom headers** — for servers that name their own headers.
+- **Browser sign-in** — optionally with a client ID and secret you registered
+  yourself, for providers that require preregistration.
+
+Every value you enter becomes a Paperclip secret. Values are write-only: they
+never appear in stored config JSON, logs, activity details, API responses after
+write, or UI readback. Only header *names* are shown in review and diagnostics.
+
+Paperclip refuses to send header names it manages or that belong to the
+transport — `Host`, `Cookie`, `Content-Length`, `Transfer-Encoding`,
+hop-by-hop headers, and anything under `Proxy-*` or `Sec-*` — and rejects
+values containing line breaks or control characters. This is enforced in shared
+code (`packages/shared/src/mcp-remote-headers.ts`), checked at the API boundary,
+and re-checked in the service immediately before the header is projected onto a
+real request.
+
+## How sign-in gets a client
+
+You never choose this; Paperclip resolves it and the wizard shows none of it.
+Recorded here for security review and diagnostics. In preference order:
+
+1. **Deployment-preconfigured client.** `PAPERCLIP_TOOL_OAUTH_<PROVIDER>_CLIENT_ID`
+   / `_SECRET`, or the unsuffixed `PAPERCLIP_TOOL_OAUTH_CLIENT_ID` / `_SECRET`.
+   Always wins when set.
+2. **Client ID Metadata Document (CIMD).** When the authorization server
+   advertises `client_id_metadata_document_supported`, Paperclip presents the URL
+   of its own published metadata document as the `client_id`. Nothing is
+   registered. **Requires a public HTTPS base URL** (`PAPERCLIP_PUBLIC_URL`):
+   the authorization server has to fetch that document server-to-server, so
+   loopback and plain-HTTP deployments fall through to the next tier.
+   The document is served unauthenticated at `/api/tools/oauth/client-metadata`
+   and contains only this deployment's callback and the grant/response/auth
+   methods Paperclip uses — no company, connection or secret data.
+3. **Dynamic client registration (RFC 7591).** When the authorization server
+   advertises a `registration_endpoint`. Paperclip registers a public client
+   (`token_endpoint_auth_method: none`, `application_type: web`, PKCE S256).
+4. **Manual preregistered client.** The client ID and secret you paste under
+   **Advanced authentication → Browser sign-in**.
+
+A generic connection may register (tiers 2 and 3) **only after** validated
+protected-resource and authorization-server discovery actually produced a
+metadata document, and only on an explicit operator connect action. An endpoint
+that merely returns a 401 does not earn a registration.
+
+### Client binding
+
+Client material is bound to the authorization-server issuer, the MCP resource
+URL, the callback URI, and the company. If any of those change:
+
+- a Paperclip-minted client (CIMD or DCR) is **re-registered**;
+- a client you supplied yourself is **not** — Paperclip stops and asks you to
+  re-enter it, because it cannot register on your behalf in a console it does
+  not control.
+
+Credentials are never reused across issuers or across companies.
+
+### Protocol conformance
+
+- RFC 8707 `resource` on authorization, token, and refresh requests, naming the
+  canonical MCP endpoint (origin + path, no query or fragment), so the
+  authorization server can audience-restrict the token to that server.
+- RFC 9728 protected-resource discovery, path-aware first
+  (`/.well-known/oauth-protected-resource<path>`) then origin.
+- RFC 8414 authorization-server discovery for issuers with a path, in the
+  spec's insertion form (`/.well-known/oauth-authorization-server<path>`) and
+  the widely deployed OIDC suffix form (`<path>/.well-known/...`). A metadata
+  document whose `issuer` disagrees with the issuer used to build the discovery
+  URL is discarded.
+- RFC 9207 `iss` validated against the persisted expected issuer when the
+  authorization server returns it. A mismatch refuses the code rather than
+  exchanging it. An absent `iss` is tolerated — it is optional and widely
+  omitted.
+- PKCE S256, exact redirect/state binding, and SSRF/private-network and redirect
+  limits are unchanged from the curated path.
+
+The discovered auth kind, issuer, and resource are persisted on the connection,
+so refresh, reconnect, revoke and diagnostics all use the generic path instead of
+falling back to `authKind: none` semantics.
+
+## Curated definitions remain optional
+
+A curated definition matching a pasted endpoint is offered as a branded
+shortcut beside the generic form — never instead of it. A definition adds labels,
+logos, field validation, scoped defaults and support copy. It does **not** unlock
+a separate execution capability, and it must not create a second connection or
+change ownership.
+
+The bespoke [PostHog connection](./POSTHOG.md) is the worked example: it is the
+polished route for most users, and PostHog is also connectable generically
+through this page with either a key or browser sign-in.
+
+## Verifying
+
+Deterministic coverage lives in
+`server/src/__tests__/generic-mcp-connection.test.ts`, which stands up an
+in-process MCP server plus authorization server. It needs no network and no
+vendor credentials, and every case connects by URL without naming a gallery app.
+A credentialed vendor smoke (for example live PostHog OAuth) may be recorded by
+QA but is not required for deterministic verification.
@@ -10,6 +10,15 @@ Paperclip connects to PostHog's hosted MCP service at
 Paperclip does not silently fall back from OAuth to an API key. The selected
 method is saved on the connection and reused for reconnects.

+This curated connection is the polished route and is what most users should use:
+it provides branding, project selection, read-only/feature/tool/mode controls,
+field validation, and tailored guidance. None of it is *required* to reach
+PostHog's MCP server. Since [PAP-17087](/PAP/issues/PAP-17087), PostHog can also
+be connected generically from **Connect your own MCP server** by pasting
+`https://mcp.posthog.com/mcp` — with a personal API key, with explicit headers, or
+through browser sign-in — with no Paperclip-specific code involved. See
+[Connecting any remote MCP server](./GENERIC-REMOTE-MCP.md).
+
 ## Service involvement

 PostHog hosts both the MCP resource and OAuth authorization service. Paperclip
@@ -147,8 +147,12 @@ not own durable tokens.
 - [First-30 matrix](./FIRST-30-MATRIX.md) harvests the keeper from
  [PAP-2432](/PAP/issues/PAP-2432) and is the source matrix for connector
  playbook work.
+- [Connecting any remote MCP server](./GENERIC-REMOTE-MCP.md) is the baseline:
+  how an operator connects a standards-compliant remote MCP endpoint with no
+  Paperclip code change, and how sign-in resolves a client.
 - [Connector playbook](./CONNECTOR-PLAYBOOK.md) is the repeatable template for
-  adding a vendor as a catalog entry on Apps v2.
+  adding a vendor as a catalog entry on Apps v2 — the optional branded
+  convenience layer over the baseline above.
 - [MCP access governance](../MCP-ACCESS-GOVERNANCE.md) remains the operator
  runbook for the current gateway, profile, policy, approval, runtime, and audit
  APIs.