Files
PaperClipAI/doc/connections/GENERIC-REMOTE-MCP.md
DottaandPaperclip 839cac1343 fix: request supported offline access for generic MCP OAuth (#14950)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents use external MCP tools through the governed gateway.
> - Remote MCP connections can use OAuth access tokens that expire.
> - Some providers issue refresh tokens only after an offline-access
request and consent.
> - Resource scopes hid that identity-provider capability in the generic
connection flow.
> - This pull request requests supported offline access and tests expiry
through a local MCP server.
> - The benefit is continued tool access without another sign-in when
the provider permits refresh.

## Linked Issues or Issue Description

Related PR: #13447 addresses the same OAuth symptom together with
managed Codex configuration. This PR focuses on generic MCP OAuth. It
also covers consent, explicit scope overrides, legacy reconnects, exact
scope persistence, and real HTTP expiry tests.

**What happened?**

A generic MCP resource can advertise only its tool scopes. Its OAuth
server can separately advertise `offline_access`. Paperclip selected the
resource scopes and omitted the offline-access request. A provider could
then issue an access token without a refresh token. Tool access stopped
after the access token expired.

**Expected behavior**

Paperclip adds advertised offline access to the selected tool scopes
when the OAuth server does not exclude refresh tokens. It requests
consent and stores the scopes sent in the authorization request.
Existing connections can discover this capability when the user
reconnects. Providers without this capability keep their existing scope
behavior.

**Steps to reproduce**

1. Run `node scripts/mcp-fixtures/servers/oauth-refresh-fixture.mjs`
from the repository root.
2. Add its MCP URL as a generic connection on a local Paperclip
instance.
3. Approve the test consent page and call `read_status`.
4. Let the two-minute access token expire and call the tool again.
5. Before this fix, the connection needs another sign-in. With this fix,
the call refreshes the token and succeeds.

**Paperclip version or commit**

The integration regression reproduced the missing-refresh-token failure
on the parent of this PR's fix. The same test passes with the fix.

**Deployment mode**

Local development. Automated tests use a loopback HTTP MCP/OAuth server
and a disposable PostgreSQL database.

## What Changed

- Track offline-access capability separately from MCP tool scopes.
- Add supported offline access and consent for generic connections.
- Preserve the actual requested scopes through callback completion and
reconnect.
- Discover the capability for older connections with cached OAuth
endpoints.
- Add a reusable MCP/OAuth fixture with PKCE, token expiry, resource
binding, and refresh-token rotation.
- Test shared and personal gateway calls through two refresh rotations.
Cover scope selection, unsupported refresh, and legacy reconnects.
- Document the behavior and local test commands.

## Verification

- The two real HTTP expiry tests failed before the fix with
`oauth_refresh_missing` after the first token expired.
- Tests passed on the current head: 76 generic MCP regressions, all 365
tool-access service tests, and 4 fixture controls.
- Full workspace `pnpm -r typecheck` and `pnpm build` passed. Server
TypeScript checks also passed after the review fixes.
- All remote checks passed on
`d22909bb34e9d54478c0002077c498ffe105932d`. Greptile gave 5/5 with both
previous findings resolved. The complete local `pnpm test:run` is still
running.
- Run `node --test
scripts/mcp-fixtures/servers/oauth-refresh-fixture.test.mjs` for the
standalone provider controls.
- Run `pnpm exec vitest run
server/src/__tests__/generic-mcp-connection.test.ts` for the Paperclip
integration tests.

## Risks

- Users can see a consent prompt when a generic provider supports
offline access.
- The provider can still decline to issue a refresh token. Access works
until expiry, then the user must reconnect.
- A provider that advertises offline access but rejects the scope
produces an OAuth error. This PR does not add an automatic retry without
that scope.
- Existing grants without refresh tokens need another sign-in. The fix
does not change them in place.
- Curated Apps keep their reviewed scope and authorization-parameter
allowlists. No database migration is required.
- This simulation verifies the suspected failure. The reported internal
MCP server has not been tested.

## Model Used

- OpenAI Codex, based on GPT-6, with reasoning, tool use, and code
execution. The runtime did not expose a more specific model ID or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-02 14:13:59 -05:00

16 KiB

Connecting any remote MCP server

Paperclip can connect a standards-compliant remote HTTP MCP server without a Paperclip code change. A curated AppDefinition is a convenience layer — branding, tailored fields, scoped defaults, support copy — not a prerequisite.

This is the documented baseline for connecting anything. Read Connection authoring runbook when you want to add the branded convenience layer on top for a vendor Paperclip should promote.

Accepted in the generic remote MCP plan, implemented in PAP-17087.

The two routes

Route Where Use it when
Guided URL Apps → Connect an app → Connect your own MCP server You have the server's address. Paperclip probes it and walks you through whatever it needs.
Paste a config Advanced → Paste a config A README gave you an mcpServers snippet, or the server needs headers with names Paperclip could not guess.

Both routes normalize through the same backend contract, so auth discovery, secret handling, catalog refresh and review cannot diverge between them.

Advanced → Run your own is a separate, higher-trust path for local stdio commands and is deliberately not covered here.

Don't know the address or the headers? The question-mark control beside Paste a config gives you a request you can hand to an agent: it asks the agent to consult the vendor's current documentation and reply with one paste-ready mcpServers JSON object using credential placeholders, plus notes on how to obtain each credential. Paste only the JSON block back into Paperclip; Paperclip reads the header names from it and asks you for the values, which it stores as Paperclip secrets.

What the guided URL flow does

After you paste an address and press Check link, Paperclip probes the endpoint and branches:

Endpoint says You get
Nothing needed Discovered actions, straight to review.
Needs authorization, and publishes discoverable OAuth metadata Sign in to continue — a browser sign-in at the provider.
Needs authorization, but no discoverable sign-in A prompt to add the key or headers its docs list, under Advanced authentication.
Needs a client you registered yourself A prompt for a client ID and secret. The draft connection is kept — you don't start over.
Not a valid address / private network / unreachable The specific problem and which field to change.

An unknown server is labelled Unverified server with its host shown, at every step through review, access and install. Reads are enabled for review; state-changing actions start off; newly discovered actions are quarantined until reviewed. That is the same treatment a curated connection gets.

For Just me, the first probe of a new URL with no supplied credentials runs before a personal authorization exists. If the server requires OAuth, Paperclip creates the personal grant only after sign-in succeeds. If the server is public, Paperclip creates a personal grant with no credentials after the probe succeeds. That successful public probe saves the draft identity and its grant. A later catalog-refresh failure leaves them available for retry instead of undoing a grant that another setup attempt may already be using. Catalog and default-profile writes after that probe are atomic: a failure rolls back that step while retaining the established draft identity. Later health checks still require the user's authorization and return an actionable 422 error when it is missing.

Personal apps on shared agents

Installing a personal app on an agent does not require every user who runs that agent to connect the app. Runs start without probing optional apps or warning about another user's missing credentials. The installed app's cached tools stay available even when its shared health check needs attention.

Authorization happens when the agent calls an app tool. Paperclip uses the run's responsible user, never another user's personal grant. If that user has not connected the app, the tool returns user_authorization_required and adds an inline connection request. Unrelated work can continue without using the app. Disabled or uninstalled apps remain unavailable.

Optional assigned apps do not emit run-start connection warnings, including unavailable shared apps. Their health state and reconnect controls remain in Apps. An unrelated run does not need to act on that state.

Slack app access

If Slack reports that MCP access is disabled for the app, ask the Slack app owner to enable MCP access in the app's Slack settings, then refresh the connection. Signing in alone does not enable that app setting. Paperclip shows this as a setup error and keeps the connection available for retry. See Slack's MCP app requirements.

Advanced authentication

Collapsed by default. Open it when the server's docs are specific:

  • No sign-in needed — the server is open to anyone with the address.
  • Key or token — sent as an Authorization header.
  • Custom headers — for servers that name their own headers.
  • Browser sign-in — optionally with a client ID and secret you registered yourself, for providers that require preregistration.

Every value you enter becomes a Paperclip secret. Values are write-only: they never appear in stored config JSON, logs, activity details, API responses after write, or UI readback. Only header names are shown in review and diagnostics.

Paperclip refuses to send header names it manages or that belong to the transport — Host, Cookie, Content-Length, Transfer-Encoding, hop-by-hop headers, and anything under Proxy-* or Sec-* — and rejects values containing line breaks or control characters. This is enforced in shared code (packages/shared/src/mcp-remote-headers.ts), checked at the API boundary, and re-checked in the service immediately before the header is projected onto a real request.

How sign-in gets a client

You never choose this; Paperclip resolves it and the wizard shows none of it. Recorded here for security review and diagnostics. In preference order:

  1. Deployment-preconfigured client. PAPERCLIP_TOOL_OAUTH_<PROVIDER>_CLIENT_ID / _SECRET, or the unsuffixed PAPERCLIP_TOOL_OAUTH_CLIENT_ID / _SECRET. Always wins when set.
  2. Client ID Metadata Document (CIMD). When the authorization server advertises client_id_metadata_document_supported, Paperclip presents the URL of its own published metadata document as the client_id. Nothing is registered. Requires a public HTTPS base URL (PAPERCLIP_PUBLIC_URL): the authorization server has to fetch that document server-to-server, so loopback and plain-HTTP deployments fall through to the next tier. The document is served unauthenticated at /api/tools/oauth/client-metadata and contains only this deployment's callback and the grant/response/auth methods Paperclip uses — no company, connection or secret data.
  3. Dynamic client registration (RFC 7591). When the authorization server advertises a registration_endpoint. Paperclip registers a public client (token_endpoint_auth_method: none, application_type: web, PKCE S256).
  4. Manual preregistered client. The client ID and secret you paste under Advanced authentication → Browser sign-in.

A generic connection may register (tiers 2 and 3) only after validated protected-resource and authorization-server discovery actually produced a metadata document, and only on an explicit operator connect action. An endpoint that merely returns a 401 does not earn a registration.

Client binding

Client material is bound to the authorization-server issuer, the MCP resource URL, the callback URI, and the company. If any of those change:

  • a Paperclip-minted client (CIMD or DCR) is re-registered;
  • a client you supplied yourself is not — Paperclip stops and asks you to re-enter it, because it cannot register on your behalf in a console it does not control.

Credentials are never reused across issuers or across companies.

Endpoint addresses are validated before they are used

Every OAuth endpoint address is chosen by the remote server — in discovered metadata, in a WWW-Authenticate hint, in a pasted config, or in a gallery default — and the authorization endpoint additionally becomes a top-level browser navigation. All of them are parsed by one shared validator (checkOAuthEndpointUrl in @paperclipai/shared) and must be:

  • https:. Plain http: is refused, except for a loopback host under the local-development policy (the same policy that allows private remote endpoints), and except for this deployment's own origin.
  • Free of embedded credentials. https://accounts.google.com@evil.test/… reads as the wrong site to a human, so Paperclip refuses it.
  • Free of a fragment, and a well-formed absolute URL.

javascript:, data:, file: and friends are therefore refused before they can reach window.location. The board applies the same validator to the address it receives, so an unsafe value cannot pass the API boundary and then execute at the navigation boundary. A refusal is reported as oauth_<kind>_endpoint_rejected (422) and the unsafe value is never persisted on the connection.

An address that passes is still only an address: a valid HTTPS authorization page can be a phishing page. The redirect screen names the host you are being sent to, and the Unverified server label stays visible for an endpoint with no curated definition.

Protocol conformance

  • RFC 8707 resource on authorization, token, and refresh requests, naming the canonical MCP endpoint (origin + path, no query or fragment), so the authorization server can audience-restrict the token to that server.
  • RFC 9728 protected-resource discovery, path-aware first (/.well-known/oauth-protected-resource<path>) then origin.
  • RFC 8414 authorization-server discovery for issuers with a path, in the spec's insertion form (/.well-known/oauth-authorization-server<path>) and the widely deployed OIDC suffix form (<path>/.well-known/...). A metadata document whose issuer disagrees with the issuer used to build the discovery URL is discarded.
  • RFC 9207 iss validated against the persisted expected issuer when the authorization server returns it. A mismatch refuses the code rather than exchanging it. An absent iss is tolerated — it is optional and widely omitted.
  • PKCE S256, exact redirect/state binding, and SSRF/private-network and redirect limits are unchanged from the curated path.

The discovered auth kind, issuer, and resource are persisted on the connection, so refresh, reconnect, revoke and diagnostics all use the generic path instead of falling back to authKind: none semantics.

Staying connected after access-token expiry

For generic MCP connections, Paperclip adds offline_access to the selected tool scopes when the authorization server advertises it in scopes_supported and does not explicitly exclude the refresh-token grant. This works even when the MCP server's challenge or protected-resource metadata lists only tool scopes. Paperclip requests prompt=consent, stores the actual requested scopes for callback completion and reconnect, and refreshes access tokens using its existing encrypted-vault and rotating-token lease paths. Curated Apps continue to use their reviewed scope and authorization-parameter allowlists.

The provider still decides whether to issue a refresh token. Existing grants without one require another sign-in; reconnect also discovers refresh support for older generic connections that already cached their OAuth endpoints. Transport failures and provider-revoked refresh tokens are separate errors.

Local OAuth expiry simulation

Run the standalone MCP/OAuth test server with no external credentials:

node scripts/mcp-fixtures/servers/oauth-refresh-fixture.mjs

It binds only to 127.0.0.1 and prints an MCP URL. Add that URL through Apps → Connect an app → Connect your own MCP server on a local/private Paperclip instance with a loopback HTTP callback URL, approve the test consent page, and use read_status. Access tokens last two minutes; later calls should refresh without another sign-in. Stop the fixture with Ctrl-C; all provider state is in memory.

The fixture separates the MCP resource's mcp:read scope from the identity provider's offline_access scope. It issues refresh tokens only with offline consent and a registered refresh grant, enforces PKCE/client/resource binding, rejects expired access tokens, and rotates refresh tokens after every use. Diagnostics record only protocol events and scope names, never token values.

Run the fixture controls and Paperclip integration regressions:

node --test scripts/mcp-fixtures/servers/oauth-refresh-fixture.test.mjs
pnpm exec vitest run server/src/__tests__/generic-mcp-connection.test.ts

The integration tests use actual loopback HTTP and a disposable PostgreSQL database. They cover shared and personal sign-in, a successful gateway tool call, two expiry/refresh rotations without another sign-in, scope overrides, unsupported refresh, and reconnecting a legacy cached connection. Provider time and cached expiry timestamps are advanced explicitly, so no real-time wait is needed. The standalone controls prove that omitting offline scope or consent creates an access-only grant that loses MCP access after expiry.

Curated definitions remain optional

A curated definition matching a pasted endpoint is offered as a branded shortcut beside the generic form — never instead of it. A definition adds labels, logos, field validation, scoped defaults and support copy. It does not unlock a separate execution capability, and it must not create a second connection or change ownership.

The bespoke PostHog connection is the worked example: it is the polished route for most users, and PostHog is also connectable generically through this page with either a key or browser sign-in.

Verifying

Deterministic coverage lives in server/src/__tests__/generic-mcp-connection.test.ts uses a simulated MCP/OAuth provider and a real loopback HTTP server for the personal public-URL regression. It needs no vendor credentials. Tests create an isolated PostgreSQL database and close their servers after use. A credentialed vendor smoke (for example live PostHog OAuth) may be recorded by QA but is not required for deterministic verification.

Opt-in generic Notion live smoke

pnpm smoke:notion-generic-live exercises the generic Advanced → Paste a config route against https://mcp.notion.com/mcp. It is intentionally outside the normal unit, browser, and CI-required suites. Run it only against an already-running, browser-reachable HTTPS Paperclip instance with these bindings provided by the execution environment:

  • PAPERCLIP_E2E_BASE_URL, PAPERCLIP_E2E_EMAIL, and PAPERCLIP_DEV_LOGIN_PASSWORD for the target instance;
  • PAPERCLIP_API_URL, PAPERCLIP_API_KEY, PAPERCLIP_RUN_ID, and PAPERCLIP_TASK_ID for the control plane;
  • the approved on-demand secret binding access.notion_generic_flow_test_account, delivered by the agent secret API under its normalized key generic-flow-test-account, for the existing Notion test account.

If that account requires an emailed one-time code, also set AGENTMAIL_API_KEY and NOTION_AGENTMAIL_INBOX_ID for the already-configured forwarding inbox. The smoke loads the AgentMail SDK only after Notion presents the code challenge, accepts only a fresh authenticated Notion message, fills the code once in memory, and never records the message, address, or code.

Check the URL, health endpoint, and binding metadata without retrieving the credential value or opening a browser:

pnpm smoke:notion-generic-live -- --dry-run

The live command retrieves the credential only after the safe preflight and Paperclip login succeed. It disables trace, video, and HAR capture, takes only post-callback screenshots, enables and invokes only notion-get-self, proves notion-create-pages remains locally denied, and removes its uniquely named connection in a finally cleanup. Its summary.json and PNG files contain sanitized IDs, decisions, outcomes, and endpoint origins/paths only; they default to PAPERCLIP_RUN_SCRATCH_DIR, or to NOTION_EVIDENCE_DIR when set.

Run the credential-free harness checks with:

node --test scripts/smoke/notion-generic-live.test.mjs