From 4fdef9457f4c2836b316e09ec77a8110353b6bc0 Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Wed, 21 Oct 2026 22:21:43 -0500 Subject: [PATCH] refactor(tools): isolate credential ownership (#17161) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connections need durable credentials that survive authorization and token rotation. > - Storage, registration, refresh coordination and credential selection lived in the general tool service. > - Each has a different ownership boundary: secrets, process-wide registration, database leases and per-service refresh work. > - This pull request gives those responsibilities named modules and passes their narrow capabilities to consumers. > - The benefit is visible credential ownership while retaining current concurrency and failure behavior. ## Linked Issues or Issue Description Depends on #17159. Earlier stack: #17144 → #17146 → #17147 → #17155 → #17157 → #17159 → this PR. **What existing behavior does this improve?** Maintenance of OAuth credential persistence, refresh and provider credential access. **Current behavior** The 14,877-line service owns registration flights, refresh leases, secret writes and provider selection alongside app setup. **Proposed behavior** Dedicated owners handle credential storage, client registration, refresh leases, connection/grant refresh, GitHub continuity and Vercel credential access. Credential selection composes those owners. Authorization callbacks remain in the coordinator for the next extraction. **Reason and benefit** The entry point falls to 12,019 lines, a reduction of 2,858. Lease acquisition and cleanup can be reviewed together, independently of callback completion. ## What Changed - Extract credential owners and shared single-flight/connection-link primitives. The largest new module is 650 lines. - Preserve process-wide client-registration flights, per-service refresh maps and GitHub sweep state. - Preserve live cloud-connector lookup, options and environment reads; use the existing secret client and transaction-scoped clients. - Extend the real local HTTP/PostgreSQL OAuth fixture with concurrent refresh across two service instances and authenticated use of rotated credentials. ## Verification - The two extended HTTP integration cases pass before extraction for public and client-secret-basic clients. - After extraction, all 470 public service and HTTP integration cases pass, including refresh failure, abandoned leases, secret redaction and membership revocation cases. The fixture was then strengthened after review: a second expiry must use the persisted rotated refresh token, and the provider rejects the old token. Both cases pass before and after extraction; server typecheck also passes. - Server typecheck, module-boundary gate and diff whitespace checks pass. - All 233 pre-existing function/method bodies match by normalized syntax comparison. - Full typecheck/build will run at the stack handoff. The earlier interrupted local full suite remains unverified; all current-head CI gates pass, with Greptile 5/5 and no open review findings. - Prior-head CI failure retained: run 39708550066, general server shard 2/12, `native-session-executor.test.ts` case “preserves a real listening process and its marker when a health RPC times out” raised `runner_remote_process_identity_unavailable` / `process_identity_changed`. Both the fixture and runner implementation are unchanged. The isolated case passes without changing assertions or timeouts. The fresh full CI run passes for the test-only review fix; the prior failure remains recorded here. ## Risks - Changing state lifetime could permit duplicate refreshes or registrations. Existing concurrency cases and the real HTTP fixture exercise these paths. - A transaction-scoped secret write must stay inside its authority lock. Function bodies, await ordering and client arguments are preserved. - The fixture proves assembled local HTTP/database behavior, not live third-party qualification. - No schema, public API, policy or scheduling change is intended. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- .../2026-10-21-tool-access-boundaries.md | 7 + ...-access-oauth-protocol.integration.test.ts | 54 +- server/src/services/tool-access.ts | 2920 +---------------- .../services/tool-access/credential-access.ts | 301 ++ .../tool-access/github-grant-continuity.ts | 303 ++ .../src/services/tool-access/identifiers.ts | 11 + .../tool-access/oauth/client-registration.ts | 553 ++++ .../tool-access/oauth/client-settings.ts | 174 + .../tool-access/oauth/connection-refresh.ts | 395 +++ .../tool-access/oauth/credential-store.ts | 205 ++ .../tool-access/oauth/grant-refresh.ts | 650 ++++ .../tool-access/oauth/refresh-leases.ts | 299 ++ .../src/services/tool-access/single-flight.ts | 17 + .../tool-access/vercel-credentials.ts | 181 + 14 files changed, 3178 insertions(+), 2892 deletions(-) create mode 100644 server/src/services/tool-access/credential-access.ts create mode 100644 server/src/services/tool-access/github-grant-continuity.ts create mode 100644 server/src/services/tool-access/oauth/client-registration.ts create mode 100644 server/src/services/tool-access/oauth/client-settings.ts create mode 100644 server/src/services/tool-access/oauth/connection-refresh.ts create mode 100644 server/src/services/tool-access/oauth/credential-store.ts create mode 100644 server/src/services/tool-access/oauth/grant-refresh.ts create mode 100644 server/src/services/tool-access/oauth/refresh-leases.ts create mode 100644 server/src/services/tool-access/single-flight.ts create mode 100644 server/src/services/tool-access/vercel-credentials.ts diff --git a/doc/plans/2026-10-21-tool-access-boundaries.md b/doc/plans/2026-10-21-tool-access-boundaries.md index 0229029fbd3..dd555774538 100644 --- a/doc/plans/2026-10-21-tool-access-boundaries.md +++ b/doc/plans/2026-10-21-tool-access-boundaries.md @@ -189,3 +189,10 @@ The entry point is 14,877 lines / 600,156 bytes. Five destination modules total match normalized syntax. The public service and unsigned review-queue coverage pass before and after (469 combined cases). Server typechecking and module boundaries pass. No new supervisor, background loop or runtime cache is created. + +### Credential ownership + +- Moved OAuth client settings, credential persistence, client registration, refresh leases, connection/grant refresh, GitHub continuity, Vercel credential access and credential selection behind named factories. +- `tool-access.ts`: 14,877 → 12,019 lines. Registration single-flight remains process-wide; refresh maps and GitHub sweep state remain per service. The live cloud connector getter and existing secret client are passed through. +- The real HTTP/PostgreSQL OAuth fixture now also proves concurrent refresh across two service instances, one token exchange, durable rotated credentials and authenticated use of the new token. Its two cases passed before the move. +- After extraction, 470 public service/integration cases, server typecheck and module boundary checks pass. All 233 pre-existing function/method bodies match by normalized syntax comparison. diff --git a/server/src/__tests__/tool-access-oauth-protocol.integration.test.ts b/server/src/__tests__/tool-access-oauth-protocol.integration.test.ts index d6658e70be4..9223c214346 100644 --- a/server/src/__tests__/tool-access-oauth-protocol.integration.test.ts +++ b/server/src/__tests__/tool-access-oauth-protocol.integration.test.ts @@ -7,8 +7,9 @@ import { join } from "node:path"; import express from "express"; import request from "supertest"; import { afterAll, beforeAll, describe, expect, it, vi } from "vitest"; -import { companies, companyMemberships, createDb, toolCatalogEntries, toolConnections, toolOauthStates } from "@paperclipai/db"; +import { companies, companyMemberships, createDb, connectionGrants, toolCatalogEntries, toolConnections, toolOauthStates } from "@paperclipai/db"; import { eq } from "drizzle-orm"; +import { toolAccessService } from "../services/tool-access.js"; import { toolAccessRoutes } from "../routes/tool-access.js"; import { errorHandler } from "../middleware/index.js"; import { startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js"; @@ -45,6 +46,8 @@ describe("tool OAuth over local HTTP", () => { }); const calls: Array<{ path: string; authorization?: string; body: string }> = []; let origin: string; + let currentAccessToken = "fixture-access"; + let currentRefreshToken = "fixture-refresh"; const provider = createServer(async (req, res) => { let body = ""; for await (const chunk of req) body += chunk; @@ -73,11 +76,19 @@ describe("tool OAuth over local HTTP", () => { }); } if (path === "/token") { - return json({ access_token: "fixture-access", refresh_token: "fixture-refresh", + const refreshing = new URLSearchParams(body).get("grant_type") === "refresh_token"; + if (refreshing) { + if (new URLSearchParams(body).get("refresh_token") !== currentRefreshToken) { + return json({ error: "invalid_grant" }, 400); + } + currentAccessToken = "fixture-access-rotated"; + currentRefreshToken = "fixture-refresh-rotated"; + } + return json({ access_token: currentAccessToken, refresh_token: currentRefreshToken, expires_in: 3600, token_type: "Bearer", scope: "tools.read offline_access" }); } if (path === "/mcp") { - if (req.headers.authorization !== "Bearer fixture-access") { + if (req.headers.authorization !== `Bearer ${currentAccessToken}`) { res.setHeader("www-authenticate", `Bearer resource_metadata="${origin}/.well-known/oauth-protected-resource/mcp"`); return json({}, 401); } @@ -139,6 +150,43 @@ describe("tool OAuth over local HTTP", () => { expect(tokenBody.has("client_secret")).toBe(false); expect(calls.some((call) => call.path === "/mcp" && call.authorization === "Bearer fixture-access")).toBe(true); expect(calls.filter((call) => call.path === "/register")).toHaveLength(1); + + const [grant] = await db.select().from(connectionGrants).where(eq(connectionGrants.connectionId, connection.id)); + await db.update(connectionGrants).set({ providerTenant: { + ...grant.providerTenant, + oauth: { ...grant.providerTenant?.oauth, accessTokenExpiresAt: new Date(0).toISOString() }, + } }).where(eq(connectionGrants.id, grant.id)); + const options = { deploymentMode: "local_trusted" as const, deploymentExposure: "private" as const, paperclipCloudConnector: null }; + const first = toolAccessService(db, options); + const second = toolAccessService(db, options); + const refreshInput = { companyId: company.id, connectionId: connection.id, grantId: grant.id, actor: { actorType: "user" as const, actorId: userId } }; + const refreshed = await Promise.all([ + first.refreshOAuthGrantCredentials(refreshInput), + second.refreshOAuthGrantCredentials(refreshInput), + ]); + expect(refreshed.map(row => row.id)).toEqual([grant.id, grant.id]); + const exchanges = calls.filter(call => call.path === "/token"); + expect(exchanges).toHaveLength(2); + const refreshBody = new URLSearchParams(exchanges[1].body); + expect(refreshBody.get("grant_type")).toBe("refresh_token"); + expect(refreshBody.get("refresh_token")).toBe("fixture-refresh"); + expect(refreshBody.get("resource")).toBe(`${origin}/mcp`); + expect(exchanges[1].authorization).toBe(tokenCall.authorization); + const [persistedGrant] = await db.select().from(connectionGrants).where(eq(connectionGrants.id, grant.id)); + expect(persistedGrant.providerTenant?.oauth).not.toHaveProperty("refreshLease"); + expect(Date.parse(persistedGrant.providerTenant!.oauth!.accessTokenExpiresAt!)).toBeGreaterThan(Date.now()); + expect(JSON.stringify(persistedGrant)).not.toMatch(/fixture-(access|refresh|secret)/); + await first.checkHealth(connection.id, { actorType: "user", actorId: userId }); + expect(calls.some(call => call.path === "/mcp" && call.authorization === "Bearer fixture-access-rotated")).toBe(true); + await db.update(connectionGrants).set({ providerTenant: { + ...persistedGrant.providerTenant, + oauth: { ...persistedGrant.providerTenant?.oauth, accessTokenExpiresAt: new Date(0).toISOString() }, + } }).where(eq(connectionGrants.id, grant.id)); + await second.refreshOAuthGrantCredentials(refreshInput); + const allExchanges = calls.filter(call => call.path === "/token"); + expect(allExchanges).toHaveLength(3); + expect(new URLSearchParams(allExchanges[2].body).get("refresh_token")).toBe("fixture-refresh-rotated"); + expect(calls.filter(call => call.path === "/register")).toHaveLength(1); } finally { provider.closeAllConnections(); await new Promise((resolve, reject) => provider.close((error) => error ? reject(error) : resolve())); diff --git a/server/src/services/tool-access.ts b/server/src/services/tool-access.ts index db1c0088514..0bb68e89128 100644 --- a/server/src/services/tool-access.ts +++ b/server/src/services/tool-access.ts @@ -1,3 +1,14 @@ +import { createVercelCredentialService, vercelConnectHttpError, vercelCredentialFor, vercelConnectResourcesFor, redactedVercelGrant } from "./tool-access/vercel-credentials.js"; +import { createConnectionCredentialAccess } from "./tool-access/credential-access.js"; +import { singleFlight } from "./tool-access/single-flight.js"; +import { connectionSetupUrl, connectionReconnectUrl } from "./tool-access/identifiers.js"; +import { createOAuthCredentialStore, actorForSecret } from "./tool-access/oauth/credential-store.js"; +import { credentialScope, managedConnectorProfile } from "./tool-access/oauth/client-settings.js"; +import { createOAuthClientRegistration } from "./tool-access/oauth/client-registration.js"; +import { createOAuthRefreshLeases, withoutOAuthRefreshLease } from "./tool-access/oauth/refresh-leases.js"; +import { createLegacyOAuthRefresh } from "./tool-access/oauth/connection-refresh.js"; +import { createOAuthGrantRefresh } from "./tool-access/oauth/grant-refresh.js"; +import { createGitHubGrantContinuity } from "./tool-access/github-grant-continuity.js"; import { createToolRuntimeHealth } from "./tool-access/runtime/health.js"; import { createToolRuntimeSlots } from "./tool-access/runtime/slots.js"; import { createToolActivityService } from "./tool-access/activity/service.js"; @@ -10,14 +21,14 @@ import { createToolProfileResolver } from "./tool-access/profiles/resolution.js" import { toProfile, toProfileEntry, toProfileBinding, pendingNewToolsForProfile, profileCatalogColumns } from "./tool-access/profiles/projection.js"; import { lockManagedAppPolicies, lockToolAssignmentAgents, lockToolConnectionRuntimeInputs } from "./tool-access/assignment-locks.js"; export { lockToolConnectionRuntimeInputs } from "./tool-access/assignment-locks.js"; -import type { ActorInfo, DbTransaction, ToolAccessMutationDb } from "./tool-access/types.js"; +import type { ActorInfo, ToolAccessMutationDb } from "./tool-access/types.js"; import { normalizeKey, connectionUid } from "./tool-access/identifiers.js"; import { createOAuthEndpointPolicy } from "./tool-access/oauth/endpoint-policy.js"; import { createOAuthDiscovery } from "./tool-access/oauth/discovery.js"; import { createOAuthClientProtocol, storedOAuthTokenEndpointAuthMethod } from "./tool-access/oauth/client.js"; import { normalizeOAuthProviderError, oauthProviderErrorMessage } from "./tool-access/oauth/provider-errors.js"; -import { oauthConfig, normalizeOauthScopes, resolveGrantedOauthScopes, sameOAuthIssuer, originOf, isSmokeLabOAuthFixture } from "./tool-access/oauth/metadata.js"; -import { resolveOAuthClientIdMetadataDocumentUrl } from "./tool-access/oauth/client-metadata.js"; +import { oauthConfig, normalizeOauthScopes, resolveGrantedOauthScopes, sameOAuthIssuer, originOf } from "./tool-access/oauth/metadata.js"; + import type { OAuthProviderEndpoints, OAuthClientRegistrationSource } from "./tool-access/oauth/types.js"; export { OAUTH_CLIENT_ID_METADATA_DOCUMENT_PATH, resolveOAuthClientIdMetadataDocumentUrl, oauthClientIdMetadataDocument } from "./tool-access/oauth/client-metadata.js"; import { @@ -58,7 +69,7 @@ import { defaultConnectionAgentInstructions } from "@paperclipai/shared"; import { googleWorkspaceEffectiveScopes, GOOGLE_WORKSPACE_DEFAULT_SELECTION, googleWorkspaceSelectedScopes, type GoogleWorkspaceServiceSelection, GOOGLE_WORKSPACE_SERVICES, googleWorkspaceServiceGranted, googleWorkspaceToolName, googleWorkspaceToolTarget, googleWorkspaceGrantedScopes, isGoogleWorkspaceToolGranted, isGoogleWorkspaceScopeGrant, type GoogleWorkspaceService } from "@paperclipai/shared"; import { isInsufficientConnectionScope, INSUFFICIENT_CONNECTION_SCOPE_MESSAGE } from "./connection-permission-errors.js"; import { mcpDiscoveryHttpFailure, retainMcpConnectionFailure, withMcpConnectionFailure } from "./mcp-connection-failure.js"; -import { ASANA_CONNECTOR_SCOPES, isAsanaConnectorProfileId, type AsanaConnectorProfileId } from "@paperclipai/shared"; + import { BROWSER_USE_TOOLS } from "@paperclipai/shared"; import { browserUseClient, isBrowserUseConnection } from "./browser-use-client.js"; import { browserUseService } from "./browser-use.js"; @@ -74,7 +85,7 @@ import { import { canBrowseProjectRepositoryGrant, mergeProjectRepository } from "./project-repositories.js"; import { measureRunPhase } from "./execution-preparation-timing.js"; -import { measureRunTransaction } from "./execution-transaction-timing.js"; + import { createHash, randomBytes, randomUUID } from "node:crypto"; import { readFileSync } from "node:fs"; import { @@ -96,22 +107,8 @@ import { } from "drizzle-orm"; import type { Db } from "@paperclipai/db"; import { agents, connectionGrantMembers, connectionGrantDelegations, connectionGrants, connectionTokenIssuances, authUsers, companyMemberships, companySecretBindings, companySecrets, principalPermissionGrants, userSecretDefinitions, userSecretDeclarations, issueThreadInteractions, plugins, toolAccessAuditEvents, toolApplications, toolActionRequests, toolCatalogEntries, toolConnectionInstalls, toolConnectionAppSnapshots, toolConnectionAppSyncs, toolConnections, toolOauthStates, toolStdioCommandTemplates, toolCallEvents, toolInvocations, toolPolicies, toolGatewaySessions, toolMcpGateways, toolMcpGatewayTokens, toolProfileBindings, toolProfileEntries, toolProfiles, toolRuntimeSlots } from "@paperclipai/db"; -import type { AppDefinition, ConnectionGrantKind, CreateToolApplication, CreateToolConnection, ConnectToolApp, ConnectToolAppResult, CreateToolStdioCommandTemplate, FinishToolApp, FinishToolAppResult, FinalizeOAuthAccess, CreateToolProfileBindingForProfile, CreateToolProfileEntryForProfile, DeploymentExposure, DeploymentMode, ImportMcpJson, McpConnectionCredentialRef, McpJsonImportPreview, ToolApplication, ToolCatalogEntry, ToolCatalogRefreshResult, ToolConnection, ToolConnectionInstall, ToolConnectionInstallSnapshot, ToolConnectionRemovalResult, ToolConnectionRemovalSummary, ToolConnectionHealthCheckResult, ToolConnectionHealthStatus, ToolConnectionAuthKind, ToolConnectionCredentialPolicy, ToolConnectionCredentialSource, ToolConnectionTransport, ToolCredentialSecretRef, ToolOAuthStartResult, ToolAppsAttentionResponse, ToolAppMetadataPreflightResult, ToolAppConnectionActionSummary, ToolExampleInstallResult, ToolExampleSmokeCheck, ToolExampleSmokeResult, ToolExampleSummary, ToolProfileBinding, ToolProfileEntry, ToolPolicyDecision, ToolPolicy, ToolRiskLevel, ToolStdioCommandTemplate, UpdateToolApplication, UpdateToolConnection, PutToolConnectionInstalls, VercelConnectCredentialReference, VercelConnectGrantReference } from "@paperclipai/shared"; -import { - CLASS3_STATIC_LEASE_ALLOWLIST, - GITHUB_CONNECTOR_PROFILES, - GOOGLE_WORKSPACE_CONNECTOR_PROFILES, - connectionIntentPayloadSchema, - getAppDefinitionForUrl, - getAvailableConnectionMethods, - getConnectableAppDefinition, - isGitHubConnectorProfileId, - isGoogleWorkspaceConnectorProfileId, - isToolConnectionAttentionHealth, - recommendedDefaultsForApp, - type GitHubConnectorProfileId, - type GoogleWorkspaceConnectorProfileId, -} from "@paperclipai/shared"; +import type { AppDefinition, ConnectionGrantKind, CreateToolApplication, CreateToolConnection, ConnectToolApp, ConnectToolAppResult, CreateToolStdioCommandTemplate, FinishToolApp, FinishToolAppResult, FinalizeOAuthAccess, CreateToolProfileBindingForProfile, CreateToolProfileEntryForProfile, DeploymentExposure, DeploymentMode, ImportMcpJson, McpConnectionCredentialRef, McpJsonImportPreview, ToolApplication, ToolCatalogEntry, ToolCatalogRefreshResult, ToolConnection, ToolConnectionInstall, ToolConnectionInstallSnapshot, ToolConnectionRemovalResult, ToolConnectionRemovalSummary, ToolConnectionHealthCheckResult, ToolConnectionHealthStatus, ToolConnectionAuthKind, ToolConnectionCredentialPolicy, ToolConnectionCredentialSource, ToolConnectionTransport, ToolCredentialSecretRef, ToolOAuthStartResult, ToolAppsAttentionResponse, ToolAppMetadataPreflightResult, ToolAppConnectionActionSummary, ToolExampleInstallResult, ToolExampleSmokeCheck, ToolExampleSmokeResult, ToolExampleSummary, ToolProfileBinding, ToolProfileEntry, ToolPolicyDecision, ToolPolicy, ToolRiskLevel, ToolStdioCommandTemplate, UpdateToolApplication, UpdateToolConnection, PutToolConnectionInstalls, VercelConnectCredentialReference } from "@paperclipai/shared"; +import { CLASS3_STATIC_LEASE_ALLOWLIST, GOOGLE_WORKSPACE_CONNECTOR_PROFILES, connectionIntentPayloadSchema, getAppDefinitionForUrl, getAvailableConnectionMethods, getConnectableAppDefinition, isGoogleWorkspaceConnectorProfileId, isToolConnectionAttentionHealth, recommendedDefaultsForApp, type GoogleWorkspaceConnectorProfileId } from "@paperclipai/shared"; import { checkMcpRemoteHeaderName, checkMcpRemoteHeaderValue, @@ -146,7 +143,7 @@ import { splitRemoteUrlCredential, } from "./remote-url-credentials.js"; import { secretService } from "./secrets.js"; -import { connectionCredentialConfigPath as credentialRefConfigPath, connectionGrantCredentialRef, connectionSecretsUsedByOtherConsumers, resolveConnectionGrantSecret, validateConnectionGrantSecretOwnership, writeConnectionCredential } from "./connection-credentials.js"; +import { connectionCredentialConfigPath as credentialRefConfigPath, connectionSecretsUsedByOtherConsumers, validateConnectionGrantSecretOwnership, writeConnectionCredential } from "./connection-credentials.js"; import { agentmailApi } from "./agentmail-api.js"; import type { ConfigureRailwaySsh, RailwaySshSetup } from "@paperclipai/shared"; import { generateRailwaySshKey, RAILWAY_SSH_SECRET_PATH, validateRailwayKnownHosts } from "./railway-ssh.js"; @@ -159,15 +156,7 @@ import { gmailArchiveToolDescription } from "./gmail-archive-tool-policy.js"; import { createToolRuntimeSupervisor } from "./tool-runtime-supervisor.js"; import { isRetiredComposioConnection, RETIRED_COMPOSIO_MESSAGE } from "@paperclipai/shared"; -import { - appWithPaperclipCloudConnectorAvailability, - paperclipCloudConnectorCapabilitiesFromEnv, - createPaperclipCloudConnector, - isPaperclipCloudConnectorStrategy, - paperclipCloudConnectorConfigFromEnv, - PaperclipCloudConnectorError, - type PaperclipCloudConnector, -} from "./paperclip-cloud-connector.js"; +import { appWithPaperclipCloudConnectorAvailability, paperclipCloudConnectorCapabilitiesFromEnv, createPaperclipCloudConnector, isPaperclipCloudConnectorStrategy, paperclipCloudConnectorConfigFromEnv, type PaperclipCloudConnector } from "./paperclip-cloud-connector.js"; import { createVercelConnectClient, deriveVercelConnectSubject, @@ -180,27 +169,6 @@ import { } from "./vercel-connect.js"; const REMOTE_HTTP_REDIRECT_STATUSES = new Set([301, 302, 303, 307, 308]); const MAX_REMOTE_HTTP_REDIRECTS = 5; -const OAUTH_REFRESH_LEASE_MS = 120_000; -const OAUTH_REFRESH_LEASE_WAIT_MS = 30_000; -const OAUTH_REFRESH_LEASE_POLL_MS = 25; - -const oauthRegistrationFlights = new Map>(); - -async function singleFlight( - flights: Map>, - key: string, - operation: () => Promise, -): Promise { - const existing = flights.get(key) as Promise | undefined; - if (existing) return existing; - const pending = operation(); - flights.set(key, pending); - try { - return await pending; - } finally { - if (flights.get(key) === pending) flights.delete(key); - } -} type ToolAccessServiceOptions = { deploymentMode?: DeploymentMode; @@ -829,14 +797,6 @@ function toConnection(row: typeof toolConnections.$inferSelect): ToolConnection }; } -function redactedVercelGrant( - value: VercelConnectGrantReference | null | undefined, -): Omit | null { - if (!value) return null; - const { subjectId: _subjectId, ...redacted } = value; - return redacted; -} - function toConnectionGrant(row: typeof connectionGrants.$inferSelect) { return { ...row, @@ -1292,34 +1252,6 @@ export function isGoogleWorkspaceToolAllowed( ); } -type ManagedConnectorProfileId = - GoogleWorkspaceConnectorProfileId | GitHubConnectorProfileId | AsanaConnectorProfileId; - -function managedConnectorProfile(value: string | undefined): { - id: ManagedConnectorProfileId; - provider: "google" | "github" | "asana"; - scopes: readonly string[]; -} | null { - if (value && isAsanaConnectorProfileId(value)) { - return { id: value, provider: "asana", scopes: ASANA_CONNECTOR_SCOPES }; - } - if (value && isGoogleWorkspaceConnectorProfileId(value)) { - return { - id: value, - provider: "google", - scopes: GOOGLE_WORKSPACE_CONNECTOR_PROFILES[value].scopes, - }; - } - if (value && isGitHubConnectorProfileId(value)) { - return { - id: value, - provider: "github", - scopes: GITHUB_CONNECTOR_PROFILES[value].scopes, - }; - } - return null; -} - /** * Did this error come from the OAuth endpoint gate (PAP-17099)? Such a refusal * is Paperclip's own decision about an unsafe address, so it must keep its code @@ -1466,18 +1398,6 @@ function sanitizeHttpFailure(error: unknown): { }; } -function vercelConnectResourcesFor( - connection: typeof toolConnections.$inferSelect, -): string[] { - const resource = new URL(remoteEndpoint(connection.config)); - // OAuth resource indicators name the protected MCP resource, not the - // connection's transport-only filtering query. PostHog uses this canonical - // URL to select its MCP consent scope set. - resource.search = ""; - resource.hash = ""; - return [resource.toString()]; -} - function readStdioTemplateId(config: Record): string { const templateId = config.templateId; if (typeof templateId !== "string" || templateId.trim().length === 0) { @@ -1531,49 +1451,6 @@ export function toolAccessService( const composioSyncJobs = new Map>(); const composioSyncTtlMs = 5 * 60_000; const composioSyncLeaseMs = 4 * 60_000; - - async function resolvedRemoteEndpoint( - connection: typeof toolConnections.$inferSelect, - actor?: ActorInfo, - ): Promise { - const publicEndpoint = remoteEndpoint(connection.config); - const ref = connection.credentialRefs.find( - (candidate) => candidate.placement === "url", - ); - if (!ref) return publicEndpoint; - let value: string; - try { - const grant = await vaultGrantForConnection(connection, actor); - const grantRef = grant ? connectionGrantCredentialRef(grant, ref) : undefined; - if (grant && !grantRef) throw unprocessable("Your MCP URL credential is missing. Reconnect this connection.", { - code: "grant_credential_invalid", connectionId: connection.id, grantId: grant.id, - }); - value = grant && grantRef - ? (await resolveOAuthGrantSecret(connection, grant, grantRef, actor, undefined)).value - : await secrets.resolveSecretValue(connection.companyId, ref.secretId, ref.version ?? "latest", { - consumerType: "tool_connection", consumerId: connection.id, - configPath: REMOTE_URL_SECRET_CONFIG_PATH, - actorType: actor?.actorType ?? "system", actorId: actor?.actorId ?? null, - }); - } catch (error) { - if (error instanceof HttpError) throw error; - throw unprocessable("Your MCP URL credential could not be resolved. Reconnect this connection.", { - code: "mcp_remote_missing_secret", connectionId: connection.id, credential: REMOTE_URL_SECRET_CONFIG_PATH, - }); - } - if (!remoteUrlCredentialMatchesPublicUrl(publicEndpoint, value)) { - throw unprocessable( - "The stored MCP URL credential no longer matches this connection.", - { - code: "mcp_remote_url_credential_mismatch", - connectionId: connection.id, - }, - ); - } - return parseRemoteHttpEndpoint(value, (message, code) => - badRequest(message, { code }), - ).toString(); - } const policySvc = toolAccessPolicyService(db); const now = options.now ?? (() => new Date()); const audit = createToolAccessAudit(db); @@ -1605,175 +1482,33 @@ export function toolAccessService( : await paperclipCloudConnectorCapabilitiesFromEnv(); return appWithPaperclipCloudConnectorAvailability(app, profiles); } - let nextGitHubContinuitySweepAt = 0; const vercelConnect = options.vercelConnectClient === undefined ? createVercelConnectClient() : options.vercelConnectClient; const runtimeSupervisor = createToolRuntimeSupervisor(db, options); + const credentialStore = createOAuthCredentialStore(db, secrets); + const { syncCredentialBindings, annotateSavedOAuthClientSecret, oauthClientForConnection, createOrRotateOAuthSecret, getOAuthGrantRow, resolveOAuthGrantSecret } = credentialStore; + const ensureOAuthClient = createOAuthClientRegistration(db, { now, options, store: credentialStore, getConnectionRow, requestOAuthClientRegistration }); + const refreshLeases = createOAuthRefreshLeases(db, now, getConnectionRow, getOAuthGrantRow); + const maybeRefreshOAuthCredentials = createLegacyOAuthRefresh(db, { now, secrets, store: credentialStore, leases: refreshLeases, getConnectionRow, exchangeOAuthToken }); + const refreshOAuthGrantCredentials = createOAuthGrantRefresh(db, { now, store: credentialStore, leases: refreshLeases, currentCloudConnector, getConnectionRow, exchangeOAuthToken }); + const { refreshManagedGitHubGrantAccess, sweepGitHubConnectionContinuity } = createGitHubGrantContinuity(db, { now, currentCloudConnector, refreshOAuthGrantCredentials, resolveOAuthGrantSecret }); + const vercelCredentials = createVercelCredentialService(db, now, vercelConnect); + const { resolveVercelCredentialHeaders, vercelGrantForConnection } = vercelCredentials; + const credentialAccess = createConnectionCredentialAccess(db, { secrets, audit, vercel: vercelCredentials, resolveOAuthGrantSecret, maybeRefreshOAuthCredentials, refreshOAuthGrantCredentials }); + const { vaultGrantForConnection, resolveCredentialHeaders, resolvedRemoteEndpoint } = credentialAccess; const runtimeSlots = createToolRuntimeSlots(db, runtimeSupervisor); const { ensureRuntimeSlot } = runtimeSlots; const runtimeHealth = createToolRuntimeHealth(db, options, now, trustedRuntimeHost); const toolActivity = createToolActivityService(db, getConnectionRow); const toolActionReviewQueue = createToolActionReviewQueue(db); - // These maps remove duplicate work inside one service instance. OAuth also - // uses the database refresh lease below as its cross-process boundary. - const oauthRefreshFlights = new Map>(); - const oauthGrantRefreshFlights = new Map>(); const catalogRefreshFlights = new Map>(); const catalogCacheTtlMs = Math.max( 0, options.catalogCacheTtlMs ?? 15 * 60 * 1000, ); - function vercelConnectHttpError(error: unknown): HttpError { - if (error instanceof VercelConnectClientError) { - return new HttpError(error.status, error.message, { code: error.code }); - } - return new HttpError( - 502, - "Vercel Connect could not complete the credential request.", - { - code: "vercel_connect_request_failed", - }, - ); - } - - function vercelCredentialFor( - connection: typeof toolConnections.$inferSelect, - ): VercelConnectCredentialReference { - if ( - connection.credentialSource !== "vercel_connect" || - !connection.externalCredential - ) { - throw unprocessable("This connection does not use Vercel Connect", { - code: "vercel_connect_not_configured", - }); - } - return connection.externalCredential; - } - - async function resolveVercelCredentialHeaders( - connection: typeof toolConnections.$inferSelect, - grant: typeof connectionGrants.$inferSelect, - options: { forceRefresh?: boolean } = {}, - ): Promise> { - if (!vercelConnect) - throw vercelConnectHttpError( - new VercelConnectClientError("vercel_connect_unavailable", 503), - ); - const credential = vercelCredentialFor(connection); - const derived = deriveVercelConnectSubject({ - credential, - connectionId: connection.id, - companyId: connection.companyId, - grantKind: grant.kind, - subjectUserId: grant.subjectUserId, - }); - const request = vercelTokenRequest({ - credential, - grant, - connectionId: connection.id, - companyId: connection.companyId, - resources: vercelConnectResourcesFor(connection), - }); - try { - const token = await vercelConnect.getToken(request, options); - if ( - token.connector.id !== credential.connectorId && - token.connector.uid !== credential.connectorUid - ) { - throw new VercelConnectClientError( - "vercel_connect_request_failed", - 502, - ); - } - await db - .update(connectionGrants) - .set({ - externalCredential: vercelGrantReference({ - credential, - token, - subjectId: derived.subjectId, - verifiedAt: now(), - }), - status: "active", - revokedAt: null, - updatedAt: now(), - }) - .where( - and( - eq(connectionGrants.id, grant.id), - eq(connectionGrants.companyId, connection.companyId), - eq(connectionGrants.connectionId, connection.id), - ), - ); - return { - [credential.headerName]: `${credential.headerPrefix ?? ""}${token.token}`, - }; - } catch (error) { - if ( - error instanceof VercelConnectClientError && - error.code === "vercel_connect_authorization_required" - ) { - await db - .update(connectionGrants) - .set({ - status: "needs_reauthorization", - updatedAt: now(), - }) - .where( - and( - eq(connectionGrants.id, grant.id), - eq(connectionGrants.companyId, connection.companyId), - ), - ); - } - throw vercelConnectHttpError(error); - } - } - - async function vercelGrantForConnection( - connection: typeof toolConnections.$inferSelect, - actor?: ActorInfo, - ) { - const actorUserId = - actor?.actorType === "user" ? (actor.actorId ?? null) : null; - if (actorUserId) { - const [personal] = await db - .select() - .from(connectionGrants) - .where( - and( - eq(connectionGrants.companyId, connection.companyId), - eq(connectionGrants.connectionId, connection.id), - eq(connectionGrants.kind, "user"), - eq(connectionGrants.subjectUserId, actorUserId), - ), - ) - .limit(1); - if (personal) return personal; - } - const [organization] = await db - .select() - .from(connectionGrants) - .where( - and( - eq(connectionGrants.companyId, connection.companyId), - eq(connectionGrants.connectionId, connection.id), - eq(connectionGrants.kind, "organization"), - eq(connectionGrants.isDefault, true), - ), - ) - .limit(1); - if (!organization) { - throw conflict("This Vercel Connect identity has not been authorized", { - code: "vercel_connect_authorization_required", - }); - } - return organization; - } - function allowPrivateRemoteEndpoints() { return ( options.deploymentMode !== "authenticated" || @@ -2868,20 +2603,6 @@ export function toolAccessService( return created; } - /** - * @param grantSecretRefs Secret refs held by a grant rather than the connection - * row. A personal credential lives only on its user grant (PAP-17835), so it - * would otherwise have no `company_secret_bindings` row and drop out of - * secret projection and removal teardown. - */ - async function syncCredentialBindings( - connection: typeof toolConnections.$inferSelect, - grantSecretRefs: ToolCredentialSecretRef[] = [], - dbClient: ToolAccessMutationDb = db, - ) { - await syncConnectionCredentialBindings(db, connection, grantSecretRefs, dbClient); - } - /** * Split the secrets a connection points at into the ones it owns outright and * the ones another consumer still depends on. @@ -3457,225 +3178,6 @@ export function toolAccessService( }; } - async function vaultGrantForConnection( - connection: typeof toolConnections.$inferSelect, - actor?: ActorInfo, - ): Promise { - const actorUserId = - actor?.actorType === "user" ? (actor.actorId ?? null) : null; - if (actorUserId) { - const [personal] = await db - .select() - .from(connectionGrants) - .where( - and( - eq(connectionGrants.companyId, connection.companyId), - eq(connectionGrants.connectionId, connection.id), - eq(connectionGrants.kind, "user"), - eq(connectionGrants.subjectUserId, actorUserId), - ), - ) - .limit(1); - if (personal) { - if (personal.status !== "active") { - throw unprocessable("OAuth authorization must be reconnected", { - code: "oauth_reauthorization_required", - setupUrl: connectionSetupUrl(connection), - reconnectUrl: connectionReconnectUrl(connection), - }); - } - return personal; - } - } - if (!actorUserId && connection.credentialPolicy === "per_user") { - // A personal-only connection is intentionally fixed to one identity. - // Background health/catalog checks have no acting user, but may safely - // exercise that sole owner-bound grant without turning it into a shared - // credential or making it available to a different caller. - const personalGrants = await db - .select() - .from(connectionGrants) - .where( - and( - eq(connectionGrants.companyId, connection.companyId), - eq(connectionGrants.connectionId, connection.id), - eq(connectionGrants.kind, "user"), - eq(connectionGrants.status, "active"), - ), - ) - .limit(2); - if (personalGrants.length === 1) return personalGrants[0]!; - } - if (connection.credentialPolicy === "per_user") { - throw unprocessable( - "This connection needs the current user's authorization", - { - code: "user_authorization_required", - setupUrl: connectionSetupUrl(connection), - reconnectUrl: connectionReconnectUrl(connection), - }, - ); - } - const [organization] = await db - .select() - .from(connectionGrants) - .where( - and( - eq(connectionGrants.companyId, connection.companyId), - eq(connectionGrants.connectionId, connection.id), - eq(connectionGrants.kind, "organization"), - eq(connectionGrants.isDefault, true), - ), - ) - .limit(1); - if (organization?.status === "active") return organization; - return null; - } - - async function resolveCredentialHeaders( - connection: typeof toolConnections.$inferSelect, - actor?: ActorInfo, - options: { forceRefresh?: boolean } = {}, - ): Promise> { - if (connection.credentialSource === "vercel_connect") { - const grant = await vercelGrantForConnection(connection, actor); - return resolveVercelCredentialHeaders(connection, grant, options); - } - let grant: typeof connectionGrants.$inferSelect | null = null; - try { - if ( - connection.authKind === "oauth" || - connection.credentialPolicy !== "shared" - ) { - grant = await vaultGrantForConnection(connection, actor); - } - if (connection.authKind === "oauth" && grant) { - grant = await refreshOAuthGrantCredentials({ - companyId: connection.companyId, - connectionId: connection.id, - grantId: grant.id, - forceRefresh: options.forceRefresh, - actor, - }); - } else { - connection = await maybeRefreshOAuthCredentials(connection, actor); - } - } catch (error) { - const scope = credentialScope(connection); - await audit({ - companyId: connection.companyId, - connectionId: connection.id, - action: "tool_connection.credential_resolution", - outcome: "failure", - reasonCode: - error instanceof HttpError - ? String(asRecord(error.details).code ?? "oauth_refresh_failed") - : "oauth_refresh_failed", - details: { - credentialCount: connection.credentialRefs.length, - credentialSecretRefCount: connection.credentialSecretRefs.length, - credentialScopeType: scope.type, - credentialScopeHash: scope.hash, - setupUrl: connectionSetupUrl(connection), - reconnectUrl: connectionReconnectUrl(connection), - }, - }); - throw error; - } - const headers: Record = {}; - const scope = credentialScope(connection); - for (const ref of connection.credentialRefs) { - if (ref.placement !== "header") continue; - let value: string; - const configPath = credentialRefConfigPath(ref); - try { - const grantRef = grant ? connectionGrantCredentialRef(grant, ref) : undefined; - if (grant && !grantRef) throw unprocessable("Your credential is missing. Reconnect this connection.", { - code: "grant_credential_invalid", connectionId: connection.id, grantId: grant.id, credential: configPath, - }); - value = - grantRef && grant - ? ( - await resolveOAuthGrantSecret( - connection, - grant, - grantRef, - actor, - undefined, - ) - ).value - : await secrets.resolveSecretValue( - connection.companyId, - ref.secretId, - ref.version ?? "latest", - { - consumerType: "tool_connection", - consumerId: connection.id, - configPath, - actorType: "system", - }, - ); - } catch (error) { - await audit({ - companyId: connection.companyId, - connectionId: connection.id, - action: "tool_connection.credential_resolution", - outcome: "failure", - reasonCode: - error instanceof HttpError - ? String( - asRecord(error.details).code ?? "secret_resolution_failed", - ) - : "secret_resolution_failed", - details: { - credentialCount: connection.credentialRefs.length, - credentialScopeType: scope.type, - credentialScopeHash: scope.hash, - }, - }); - throw error; - } - if (ref.placement === "header") { - headers[ref.key] = `${ref.prefix ?? ""}${value}`; - } - } - const oauthAccessRef = grant?.credentialSecretRefs.find( - (ref) => ref.configPath === "oauth.access_token", - ); - if (oauthAccessRef && headers.Authorization === undefined) { - headers.Authorization = `Bearer ${ - ( - await resolveOAuthGrantSecret( - connection, - grant!, - oauthAccessRef, - actor, - undefined, - ) - ).value - }`; - } - if ( - connection.credentialRefs.length > 0 || - connection.credentialSecretRefs.length > 0 || - Object.keys(oauthConfig(connection)).length > 0 - ) { - await audit({ - companyId: connection.companyId, - connectionId: connection.id, - action: "tool_connection.credential_resolution", - outcome: "success", - details: { - credentialCount: connection.credentialRefs.length, - credentialSecretRefCount: connection.credentialSecretRefs.length, - credentialScopeType: scope.type, - credentialScopeHash: scope.hash, - }, - }); - } - return headers; - } - async function remoteTools( connection: typeof toolConnections.$inferSelect, credentialHeaders?: Record, @@ -5400,151 +4902,6 @@ export function toolAccessService( return fields; } - function actorForSecret( - actor?: ActorInfo, - ): { userId?: string | null; agentId?: string | null } | undefined { - if (actor?.actorType === "user") return { userId: actor.actorId ?? null }; - if (actor?.actorType === "agent") return { agentId: actor.actorId ?? null }; - return undefined; - } - - function oauthEnvName( - provider: string, - suffix: "CLIENT_ID" | "CLIENT_SECRET", - ) { - return `PAPERCLIP_TOOL_OAUTH_${provider.replace(/[^a-z0-9]+/gi, "_").toUpperCase()}_${suffix}`; - } - - function oauthClientConfig(provider: string) { - const clientIdEnv = oauthEnvName(provider, "CLIENT_ID"); - const clientSecretEnv = oauthEnvName(provider, "CLIENT_SECRET"); - return { - clientIdEnv, - clientSecretEnv, - clientId: - process.env[clientIdEnv] ?? - process.env.PAPERCLIP_TOOL_OAUTH_CLIENT_ID ?? - null, - clientSecret: - process.env[clientSecretEnv] ?? - process.env.PAPERCLIP_TOOL_OAUTH_CLIENT_SECRET ?? - null, - }; - } - - function configuredOAuthClientForConnection( - connection: typeof toolConnections.$inferSelect, - provider: string, - ) { - if (isSmokeLabOAuthFixture(connection) && provider === "smoke_lab") { - return { - clientIdEnv: "SMOKE_LAB_FIXED_CLIENT_ID", - clientSecretEnv: "SMOKE_LAB_FIXED_CLIENT_SECRET", - clientId: "paperclip-smoke-lab", - clientSecret: null, - }; - } - return oauthClientConfig(provider); - } - - async function annotateSavedOAuthClientSecret(connections: ToolConnection[], viewerUserId?: string) { - const personal = connections.filter((connection) => connection.authKind === "oauth" && connection.credentialPolicy === "per_user"); - const grants = viewerUserId && personal.length > 0 - ? await db.select({ connectionId: connectionGrants.connectionId, refs: connectionGrants.credentialSecretRefs }) - .from(connectionGrants) - .where(and( - inArray(connectionGrants.companyId, [...new Set(personal.map((connection) => connection.companyId))]), - inArray(connectionGrants.connectionId, personal.map((connection) => connection.id)), - eq(connectionGrants.kind, "user"), - eq(connectionGrants.subjectUserId, viewerUserId), - eq(connectionGrants.status, "active"), - )) - : []; - const savedPersonal = new Set(grants.filter((grant) => grant.refs.some((ref) => ref.configPath === "oauth.client_secret")).map((grant) => grant.connectionId)); - for (const connection of connections) { - connection.hasSavedOAuthClientSecret = connection.credentialPolicy === "per_user" - ? savedPersonal.has(connection.id) - : connection.credentialSecretRefs.some((ref) => ref.configPath === "oauth.client_secret"); - } - } - - async function oauthClientForConnection( - connection: typeof toolConnections.$inferSelect, - provider: string, - actor?: ActorInfo, - ) { - const configured = configuredOAuthClientForConnection(connection, provider); - if (configured.clientId) return configured; - const oauth = oauthConfig(connection); - const clientId = - typeof oauth.clientId === "string" && oauth.clientId.trim() - ? oauth.clientId.trim() - : null; - if (!clientId) return configured; - // CIMD clients and public DCR clients have no token-endpoint secret. A DCR - // authorization server may instead issue a confidential client (for - // example, `client_secret_basic`); in that case the registration secret is - // encrypted like every other provider credential and resolved only for the - // token endpoint. - const registrationSource = oauth.clientRegistrationSource; - const publicRegisteredClient = - registrationSource === "cimd" || - (registrationSource === "dcr" && - oauth.clientTokenEndpointAuthMethod === "none"); - let credentialSecretRefs = connection.credentialSecretRefs; - if ( - connection.credentialPolicy === "per_user" && - actor?.actorType === "user" && - actor.actorId && - !credentialSecretRefs.some( - (ref) => ref.configPath === "oauth.client_secret", - ) - ) { - const [personalGrant] = await db - .select({ - credentialSecretRefs: connectionGrants.credentialSecretRefs, - }) - .from(connectionGrants) - .where( - and( - eq(connectionGrants.companyId, connection.companyId), - eq(connectionGrants.connectionId, connection.id), - eq(connectionGrants.kind, "user"), - eq(connectionGrants.subjectUserId, actor.actorId), - eq(connectionGrants.status, "active"), - ), - ) - .limit(1); - credentialSecretRefs = - personalGrant?.credentialSecretRefs ?? credentialSecretRefs; - } - const clientSecretRef = publicRegisteredClient - ? undefined - : credentialSecretRefs.find( - (ref) => ref.configPath === "oauth.client_secret", - ); - const clientSecret = clientSecretRef - ? await secrets.resolveSecretValue( - connection.companyId, - clientSecretRef.secretId, - clientSecretRef.versionSelector ?? "latest", - { - consumerType: "tool_connection", - consumerId: connection.id, - configPath: "oauth.client_secret", - actorType: actor?.actorType ?? "system", - actorId: actor?.actorId ?? null, - }, - ) - : null; - return { - clientIdEnv: null, - clientSecretEnv: null, - clientId, - clientSecret, - }; - } - function base64UrlSha256(input: string) { return createHash("sha256").update(input).digest("base64url"); } @@ -5553,125 +4910,6 @@ export function toolAccessService( return randomBytes(bytes).toString("base64url"); } - function connectionSetupUrl(connection: typeof toolConnections.$inferSelect) { - return `/apps/${connection.id}/permissions`; - } - - function connectionReconnectUrl( - connection: typeof toolConnections.$inferSelect, - ) { - return `/apps/${connection.id}/permissions`; - } - - function credentialScope( - connection: typeof toolConnections.$inferSelect, - actor?: ActorInfo, - ) { - const configured = asRecord(oauthConfig(connection).credentialScope); - const type = - typeof configured.type === "string" - ? configured.type - : typeof configured.targetType === "string" - ? configured.targetType - : actor?.actorType === "agent" - ? "agent" - : actor?.actorType === "user" - ? "user" - : "company"; - const id = - typeof configured.id === "string" - ? configured.id - : typeof configured.targetId === "string" - ? configured.targetId - : (actor?.actorId ?? connection.companyId); - return { - type, - id, - hash: stableHash({ - companyId: connection.companyId, - connectionId: connection.id, - type, - id, - }), - }; - } - - function oauthSecretRef( - connection: typeof toolConnections.$inferSelect, - configPath: - "oauth.access_token" | "oauth.refresh_token" | "oauth.client_secret" | "railway.ssh_private_key", - ) { - return ( - connection.credentialSecretRefs.find( - (ref) => ref.configPath === configPath, - ) ?? null - ); - } - - function oauthExpiresAtMs( - connection: typeof toolConnections.$inferSelect, - ): number | null { - const expiresAt = oauthConfig(connection).expiresAt; - if (typeof expiresAt !== "string") return null; - const ms = Date.parse(expiresAt); - return Number.isFinite(ms) ? ms : null; - } - - async function oauthGalleryEntryForConnection( - connection: typeof toolConnections.$inferSelect, - ) { - const sourceTemplateKey = - typeof connection.config.sourceTemplateKey === "string" - ? connection.config.sourceTemplateKey - : null; - if (!sourceTemplateKey) - throw unprocessable( - "This app connection was not created from the app gallery", - ); - const galleryEntry = getConnectableAppDefinition(sourceTemplateKey); - if ( - !galleryEntry || - connectionMethodForConnection(galleryEntry, connection).auth !== "oauth" - ) { - throw unprocessable("This app connection does not use sign in"); - } - return galleryEntry; - } - - async function createOrRotateOAuthSecret( - input: { - companyId: string; - connection: typeof toolConnections.$inferSelect; - configPath: - "oauth.access_token" | "oauth.refresh_token" | "oauth.client_secret" | "railway.ssh_private_key"; - label: string; - value: string; - actor?: ActorInfo; - existingRefs?: typeof connectionGrants.$inferSelect.credentialSecretRefs; - ownerUserId?: string; - }, - context?: { - dbClient: Db | DbTransaction; - secretClient: ReturnType; - }, - ) { - const existing = input.existingRefs === undefined - ? oauthSecretRef(input.connection, input.configPath) - : input.existingRefs.find((ref) => ref.configPath === input.configPath); - const write = (client: Db | DbTransaction) => writeConnectionCredential(client, { - companyId: input.companyId, connectionName: input.connection.name, configPath: input.configPath, - label: input.label, value: input.value, existingRef: existing ?? undefined, actor: actorForSecret(input.actor), - // Registration authenticates the OAuth client, never the person invoking tools. - ownerUserId: input.configPath === "oauth.client_secret" ? undefined : input.ownerUserId, - definitionKey: input.ownerUserId ? `tool_oauth.${input.connection.id}.${input.configPath.replace(/[^a-z0-9_:-]+/gi, "_")}` : undefined, - }); - const { secret } = context ? await write(context.dbClient) : await db.transaction(write); - return { - secretId: secret.id, versionSelector: "latest" as const, configPath: input.configPath, - required: input.configPath === "oauth.access_token", label: input.label, - }; - } - function assertOAuthRedirectConstraints( app: AppDefinition | null, redirectUri: string, @@ -5737,2102 +4975,6 @@ export function toolAccessService( ); } - async function registerOAuthClient(input: { - connection: typeof toolConnections.$inferSelect; - endpoints: OAuthProviderEndpoints; - redirectUri: string; - actor?: ActorInfo; - }) { - const { clientId, clientSecret, tokenEndpointAuthMethod, clientIdIssuedAt, clientSecretExpiresAt } = await requestOAuthClientRegistration({ - endpoints: input.endpoints, - redirectUri: input.redirectUri, - }); - const existingClientSecretRef = oauthSecretRef( - input.connection, - "oauth.client_secret", - ); - const nextCredentialSecretRefs = - input.connection.credentialSecretRefs.filter( - (ref) => ref.configPath !== "oauth.client_secret", - ); - if (clientSecret) { - const clientSecretRef = await createOrRotateOAuthSecret({ - companyId: input.connection.companyId, - connection: input.connection, - configPath: "oauth.client_secret", - label: "OAuth client secret", - value: clientSecret, - actor: input.actor, - }); - nextCredentialSecretRefs.push(clientSecretRef); - } else if ( - existingClientSecretRef && - oauthConfig(input.connection).clientId === clientId - ) { - nextCredentialSecretRefs.push(existingClientSecretRef); - } - - const oauth = oauthConfig(input.connection); - const nextConfig = { - ...input.connection.config, - oauth: { - ...oauth, - provider: input.endpoints.provider, - authorizationUrl: input.endpoints.authorizationUrl, - tokenUrl: input.endpoints.tokenUrl, - registrationUrl: input.endpoints.registrationUrl, - metadataUrl: input.endpoints.metadataUrl ?? null, - scopes: input.endpoints.scopes, - offlineAccessSupported: input.endpoints.offlineAccessSupported, - grantTypesSupported: input.endpoints.grantTypesSupported ?? [], - codeChallengeMethodsSupported: - input.endpoints.codeChallengeMethodsSupported ?? [], - tokenEndpointAuthMethodsSupported: - input.endpoints.tokenEndpointAuthMethodsSupported ?? [], - issuer: input.endpoints.issuer ?? oauth.issuer ?? null, - resource: input.endpoints.resource ?? oauth.resource ?? null, - clientId, - clientRegistrationSource: "dcr" satisfies OAuthClientRegistrationSource, - clientTokenEndpointAuthMethod: tokenEndpointAuthMethod, - clientRedirectUri: input.redirectUri, - // Registered client material is only valid for the issuer/resource pair - // it was minted against. `assertOAuthClientBinding` re-registers when any - // of these move, so a re-pointed endpoint can never silently reuse a - // client another authorization server issued. - clientIssuer: input.endpoints.issuer ?? null, - clientResource: input.endpoints.resource ?? null, - clientCompanyId: input.connection.companyId, - clientIdIssuedAt, - clientSecretExpiresAt, - }, - }; - const [updated] = await db - .update(toolConnections) - .set({ - ownership: "dcr", - config: nextConfig, - transportConfig: nextConfig, - credentialSecretRefs: nextCredentialSecretRefs, - updatedAt: now(), - }) - .where( - and( - eq(toolConnections.id, input.connection.id), - eq(toolConnections.companyId, input.connection.companyId), - ), - ) - .returning(); - if (!updated) throw notFound("Tool connection not found"); - await syncCredentialBindings(updated); - return updated; - } - - /** - * Adopt a Client ID Metadata Document as this connection's client: the - * `client_id` *is* the https URL of Paperclip's published client metadata, so - * there is nothing to register with the authorization server. Still recorded on - * the connection so the issuer/resource/callback binding is enforced on reuse - * exactly like a dynamically registered client. - */ - async function adoptClientIdMetadataDocument(input: { - connection: typeof toolConnections.$inferSelect; - endpoints: OAuthProviderEndpoints; - redirectUri: string; - clientId: string; - }) { - const oauth = oauthConfig(input.connection); - const nextConfig = { - ...input.connection.config, - oauth: { - ...oauth, - provider: input.endpoints.provider, - authorizationUrl: input.endpoints.authorizationUrl, - tokenUrl: input.endpoints.tokenUrl, - registrationUrl: input.endpoints.registrationUrl ?? null, - metadataUrl: input.endpoints.metadataUrl ?? null, - scopes: input.endpoints.scopes, - offlineAccessSupported: input.endpoints.offlineAccessSupported, - grantTypesSupported: input.endpoints.grantTypesSupported ?? [], - codeChallengeMethodsSupported: - input.endpoints.codeChallengeMethodsSupported ?? [], - tokenEndpointAuthMethodsSupported: - input.endpoints.tokenEndpointAuthMethodsSupported ?? [], - clientIdMetadataDocumentSupported: true, - issuer: input.endpoints.issuer ?? oauth.issuer ?? null, - resource: input.endpoints.resource ?? oauth.resource ?? null, - clientId: input.clientId, - clientRegistrationSource: - "cimd" satisfies OAuthClientRegistrationSource, - clientTokenEndpointAuthMethod: "none", - clientRedirectUri: input.redirectUri, - clientIssuer: input.endpoints.issuer ?? null, - clientResource: input.endpoints.resource ?? null, - clientCompanyId: input.connection.companyId, - }, - }; - // A CIMD client has no secret. Drop any leftover one so a stale credential - // from an earlier registration can never be replayed against a new issuer. - const nextCredentialSecretRefs = - input.connection.credentialSecretRefs.filter( - (ref) => ref.configPath !== "oauth.client_secret", - ); - const [updated] = await db - .update(toolConnections) - .set({ - authKind: "oauth", - config: nextConfig, - transportConfig: nextConfig, - credentialSecretRefs: nextCredentialSecretRefs, - updatedAt: now(), - }) - .where( - and( - eq(toolConnections.id, input.connection.id), - eq(toolConnections.companyId, input.connection.companyId), - ), - ) - .returning(); - if (!updated) throw notFound("Tool connection not found"); - await syncCredentialBindings(updated); - return updated; - } - - /** - * How the client already stored on this connection was obtained. Anything - * unrecognised (including connections written before this field existed) reads - * as `manual`, which is the conservative answer: Paperclip will not silently - * re-register over client material it cannot prove it minted. - */ - function storedOAuthClientRegistrationSource( - connection: typeof toolConnections.$inferSelect, - ): OAuthClientRegistrationSource { - const source = oauthConfig(connection).clientRegistrationSource; - return source === "cimd" || source === "dcr" || source === "preconfigured" - ? source - : "manual"; - } - - /** - * Is the client material already stored on this connection still valid for the - * authorization server, MCP resource, callback URI and company we are about to - * use it with? Client credentials are minted against exactly one such tuple; - * reusing them across a moved binding would let a re-pointed endpoint borrow - * another server's registration. - */ - function reviewedOAuthClientBinding(connection: typeof toolConnections.$inferSelect, endpoints: OAuthProviderEndpoints) { - const oauth = oauthConfig(connection); - // Only repair the known Asana v1 discovery mistake. Retain company and - // callback checks, and never carry a secret to an arbitrary new issuer. - if (connection.config.sourceTemplateKey === "asana" - && connection.config.connectionMethodKey === "mcp-own-oauth" - && connection.config.url === "https://mcp.asana.com/v2/mcp" - && oauth.clientRegistrationSource === "manual" - && connection.ownership !== "dcr" - && endpoints.issuer === "https://app.asana.com" - && endpoints.authorizationUrl === "https://app.asana.com/-/oauth_authorize" - && endpoints.tokenUrl === "https://app.asana.com/-/oauth_token" - && endpoints.resource === "https://mcp.asana.com/v2/mcp" - && [undefined, null, "https://mcp.asana.com", "https://app.asana.com"].includes(oauth.clientIssuer as string | null | undefined) - && [undefined, null, "https://mcp.asana.com", "https://mcp.asana.com/v2/mcp"].includes(oauth.clientResource as string | null | undefined) - && (oauth.clientIssuer === "https://mcp.asana.com" || oauth.clientResource === "https://mcp.asana.com")) { - return { ...oauth, clientIssuer: endpoints.issuer, clientResource: endpoints.resource }; - } - return oauth; - } - - function oauthClientBindingMatches( - connection: typeof toolConnections.$inferSelect, - endpoints: OAuthProviderEndpoints, - redirectUri: string, - clientIdMetadataDocumentUrl: string | null, - ): boolean { - const oauth = reviewedOAuthClientBinding(connection, endpoints); - if (typeof oauth.clientId !== "string" || !oauth.clientId.trim()) - return false; - const source = - typeof oauth.clientRegistrationSource === "string" - ? oauth.clientRegistrationSource - : null; - // Older interrupted setup flows could accidentally round-trip a DCR client - // through the customer-client form and relabel it `manual`. Ownership is the - // durable proof that Paperclip minted that client. Force a fresh registration - // instead of preserving the damaged binding forever. - if (source === "manual" && connection.ownership === "dcr") return false; - // A URL client id that now resolves only to a private network is unusable by - // an external authorization server. Treat the stored binding as stale so a - // retry can replace it with a dynamically registered client. - if (source === "cimd" && oauth.clientId !== clientIdMetadataDocumentUrl) - return false; - // A manually preregistered client was registered by the operator against - // Paperclip's callback, so it has no recorded callback until first use. - const redirectMatches = - source === "manual" - ? oauth.clientRedirectUri === undefined || - oauth.clientRedirectUri === null || - oauth.clientRedirectUri === redirectUri - : oauth.clientRedirectUri === redirectUri; - if (!redirectMatches) return false; - if ( - typeof oauth.clientCompanyId === "string" && - oauth.clientCompanyId !== connection.companyId - ) - return false; - if ( - endpoints.issuer && - typeof oauth.clientIssuer === "string" && - oauth.clientIssuer && - !sameOAuthIssuer(oauth.clientIssuer, endpoints.issuer) - ) { - return false; - } - if ( - endpoints.resource && - typeof oauth.clientResource === "string" && - oauth.clientResource && - oauth.clientResource !== endpoints.resource - ) { - return false; - } - return true; - } - - /** - * Record the issuer/resource/callback/company a client is bound to, the first - * time that client is actually used. - * - * A dynamically registered or CIMD client is stamped at registration. A client - * the operator pasted in has no binding until now — without this, its - * `clientRedirectUri` would stay unset forever and `oauthClientBindingMatches` - * would keep accepting it for *any* callback, which is exactly the drift the - * binding check exists to catch. - */ - async function stampOAuthClientBinding( - connection: typeof toolConnections.$inferSelect, - endpoints: OAuthProviderEndpoints, - redirectUri: string, - ): Promise { - const originalOAuth = oauthConfig(connection); - const oauth = reviewedOAuthClientBinding(connection, endpoints); - const nextBinding = { - clientRedirectUri: redirectUri, - clientIssuer: - typeof oauth.clientIssuer === "string" && oauth.clientIssuer - ? oauth.clientIssuer - : (endpoints.issuer ?? null), - clientResource: - typeof oauth.clientResource === "string" && oauth.clientResource - ? oauth.clientResource - : (endpoints.resource ?? null), - clientCompanyId: - typeof oauth.clientCompanyId === "string" && oauth.clientCompanyId - ? oauth.clientCompanyId - : connection.companyId, - }; - const unchanged = Object.entries(nextBinding).every( - ([key, value]) => originalOAuth[key] === value, - ); - if (unchanged) return connection; - const nextConfig = { - ...connection.config, - oauth: { ...oauth, ...nextBinding }, - }; - const [updated] = await db - .update(toolConnections) - .set({ - config: nextConfig, - transportConfig: nextConfig, - updatedAt: now(), - }) - .where( - and( - eq(toolConnections.id, connection.id), - eq(toolConnections.companyId, connection.companyId), - ), - ) - .returning(); - return updated ?? connection; - } - - /** - * May Paperclip mint client material for this connection without an operator - * pasting client credentials? - * - * A curated app opts in through its `ownershipModes`. A generic remote MCP - * connection may register once — and only once — protected-resource and - * authorization-server discovery actually produced a metadata document; an - * endpoint that merely returned a 401 does not earn a registration. - * - * A curated method pinned to customer-owned clients still earns a registration - * when the provider's own metadata advertises one. `ownershipModes` is a - * point-in-time research snapshot, and providers add dynamic registration - * without telling us; live discovery is the better evidence of the two, so a - * stale catalog entry costs the operator a console detour rather than silently - * outranking what the server just said about itself. - */ - function canRegisterOAuthClientDynamically( - connection: typeof toolConnections.$inferSelect, - endpoints: OAuthProviderEndpoints, - galleryEntry: AppDefinition | null, - ): boolean { - if (galleryEntry) { - const method = connectionMethodForConnection(galleryEntry, connection); - if (method.ownershipModes.includes("dcr")) return true; - return method.auth === "oauth" && Boolean(endpoints.registrationUrl); - } - return ( - connection.transport === "mcp_remote" && Boolean(endpoints.metadataUrl) - ); - } - - async function ensureOAuthClient(input: { - connection: typeof toolConnections.$inferSelect; - endpoints: OAuthProviderEndpoints; - redirectUri: string; - galleryEntry: AppDefinition | null; - actor?: ActorInfo; - }) { - // 1. A client the deployment preconfigured for this issuer wins outright. - const configured = configuredOAuthClientForConnection( - input.connection, - input.endpoints.provider, - ); - if (configured.clientId) { - return { - connection: input.connection, - client: configured, - source: "preconfigured" as const, - }; - } - const method = input.galleryEntry - ? connectionMethodForConnection(input.galleryEntry, input.connection) - : null; - const dcrRequired = method?.oauthClientRegistration === "dcr"; - // A reviewed provider compatibility choice may require DCR even when the - // server also advertises CIMD. Do not resolve/adopt a metadata URL for that - // method; the default for every other method remains CIMD-first. - const metadataDocumentUrl = !dcrRequired && input.endpoints - .clientIdMetadataDocumentSupported - ? await resolveOAuthClientIdMetadataDocumentUrl( - input.redirectUri, - options.oauthClientMetadataLookup, - ) - : null; - // 2. Client material already bound to this issuer/resource/callback/company. - if ( - oauthClientBindingMatches( - input.connection, - input.endpoints, - input.redirectUri, - metadataDocumentUrl, - ) - ) { - const bound = await stampOAuthClientBinding( - input.connection, - input.endpoints, - input.redirectUri, - ); - return { - connection: bound, - client: await oauthClientForConnection( - bound, - input.endpoints.provider, - input.actor, - ), - source: storedOAuthClientRegistrationSource(bound), - }; - } - const oauth = oauthConfig(input.connection); - if ( - oauth.clientRegistrationSource === "manual" && - input.connection.ownership !== "dcr" && - typeof oauth.clientId === "string" && - oauth.clientId.trim() - ) { - // Paperclip cannot re-register on the operator's behalf: the credentials - // came from a console this deployment does not control. - throw unprocessable( - "This connection's sign-in details no longer match the server it points at. Re-enter the client ID and secret to continue.", - { code: "oauth_manual_client_rebinding_required" }, - ); - } - if ( - !canRegisterOAuthClientDynamically( - input.connection, - input.endpoints, - input.galleryEntry, - ) - ) { - throw unprocessable( - `OAuth client id is not configured for ${input.endpoints.provider}`, - { - code: "oauth_client_registration_unavailable", - }, - ); - } - if (dcrRequired && !input.endpoints.registrationUrl) { - throw unprocessable( - "This provider requires dynamic client registration, but its authorization server did not advertise a registration endpoint.", - { code: "oauth_dcr_not_supported" }, - ); - } - - const key = `${input.connection.id}:${input.redirectUri}`; - return singleFlight(oauthRegistrationFlights, key, async () => { - const latest = await getConnectionRow( - input.connection.id, - input.connection.companyId, - ); - const latestConfigured = configuredOAuthClientForConnection( - latest, - input.endpoints.provider, - ); - if (latestConfigured.clientId) { - return { - connection: latest, - client: latestConfigured, - source: "preconfigured" as const, - }; - } - if ( - oauthClientBindingMatches( - latest, - input.endpoints, - input.redirectUri, - metadataDocumentUrl, - ) - ) { - const bound = await stampOAuthClientBinding( - latest, - input.endpoints, - input.redirectUri, - ); - return { - connection: bound, - client: await oauthClientForConnection( - bound, - input.endpoints.provider, - input.actor, - ), - source: storedOAuthClientRegistrationSource(bound), - }; - } - // 3. By default, Client ID Metadata Documents need no registration call, - // so prefer them over DCR when the authorization server advertises - // support. Curated DCR opt-ins leave metadataDocumentUrl null above. - if (metadataDocumentUrl) { - const adopted = await adoptClientIdMetadataDocument({ - connection: latest, - endpoints: input.endpoints, - redirectUri: input.redirectUri, - clientId: metadataDocumentUrl, - }); - return { - connection: adopted, - client: await oauthClientForConnection( - adopted, - input.endpoints.provider, - input.actor, - ), - source: "cimd" as const, - }; - } - // 4. Dynamic client registration. - if (!input.endpoints.registrationUrl) { - throw unprocessable( - "This server needs sign-in details you create yourself. Add a client ID and secret under Advanced authentication.", - { code: "oauth_manual_client_required" }, - ); - } - const registered = await registerOAuthClient({ - connection: latest, - endpoints: input.endpoints, - redirectUri: input.redirectUri, - actor: input.actor, - }); - return { - connection: registered, - client: await oauthClientForConnection( - registered, - input.endpoints.provider, - input.actor, - ), - source: "dcr" as const, - }; - }); - } - - function withoutOAuthRefreshLease(oauth: Record) { - const { refreshLease: _refreshLease, ...rest } = oauth; - return rest; - } - - function oauthRefreshLeaseId( - connection: typeof toolConnections.$inferSelect, - ): string | null { - const lease = asRecord(oauthConfig(connection).refreshLease); - return typeof lease.id === "string" && lease.id ? lease.id : null; - } - - async function clearOAuthRefreshLease( - connection: typeof toolConnections.$inferSelect, - leaseId: string, - ) { - const latest = await getConnectionRow(connection.id, connection.companyId); - if (oauthRefreshLeaseId(latest) !== leaseId) return latest; - const nextConfig = { - ...latest.config, - oauth: withoutOAuthRefreshLease(oauthConfig(latest)), - }; - const [updated] = await db - .update(toolConnections) - .set({ - config: nextConfig, - transportConfig: nextConfig, - updatedAt: now(), - }) - .where( - and( - eq(toolConnections.id, latest.id), - eq(toolConnections.companyId, latest.companyId), - sql`${toolConnections.config} -> 'oauth' -> 'refreshLease' ->> 'id' = ${leaseId}`, - ), - ) - .returning(); - return updated ?? getConnectionRow(connection.id, connection.companyId); - } - - async function acquireOAuthRefreshLease( - connection: typeof toolConnections.$inferSelect, - ): Promise<{ - connection: typeof toolConnections.$inferSelect; - leaseId: string | null; - }> { - const waitDeadline = Date.now() + OAUTH_REFRESH_LEASE_WAIT_MS; - while (true) { - const latest = await getConnectionRow( - connection.id, - connection.companyId, - ); - const latestExpiresAtMs = oauthExpiresAtMs(latest); - if (latestExpiresAtMs && latestExpiresAtMs > Date.now() + 60_000) { - return { connection: latest, leaseId: null }; - } - - const oauth = oauthConfig(latest); - const currentLease = asRecord(oauth.refreshLease); - const currentLeaseExpiresAt = - typeof currentLease.expiresAt === "string" - ? Date.parse(currentLease.expiresAt) - : Number.NaN; - const currentLeaseId = - typeof currentLease.id === "string" && currentLease.id - ? currentLease.id - : null; - const leaseIsActive = - currentLeaseId !== null && - Number.isFinite(currentLeaseExpiresAt) && - currentLeaseExpiresAt > Date.now(); - if (!currentLeaseId) { - const leaseId = randomUUID(); - const claimedAt = now(); - const nextConfig = { - ...latest.config, - oauth: { - ...withoutOAuthRefreshLease(oauth), - refreshLease: { - id: leaseId, - expiresAt: new Date( - Date.now() + OAUTH_REFRESH_LEASE_MS, - ).toISOString(), - }, - }, - }; - const [claimed] = await db - .update(toolConnections) - .set({ - config: nextConfig, - transportConfig: nextConfig, - updatedAt: claimedAt, - }) - .where( - and( - eq(toolConnections.id, latest.id), - eq(toolConnections.companyId, latest.companyId), - sql`${toolConnections.config} = ${JSON.stringify(latest.config)}::jsonb`, - sql`${toolConnections.config} #>> '{oauth,refreshLease,id}' is null`, - ), - ) - .returning(); - if (claimed) return { connection: claimed, leaseId }; - } - - if (currentLeaseId && !leaseIsActive) { - throw new HttpError( - 422, - "The previous OAuth refresh did not finish. Reconnect this app before retrying.", - { - code: "oauth_refresh_outcome_unknown", - setupUrl: connectionSetupUrl(latest), - reconnectUrl: connectionReconnectUrl(latest), - }, - ); - } - - if (Date.now() >= waitDeadline) { - throw conflict("OAuth credential refresh is already in progress", { - code: "oauth_refresh_in_progress", - retryable: true, - }); - } - await new Promise((resolve) => - setTimeout(resolve, OAUTH_REFRESH_LEASE_POLL_MS), - ); - } - } - - async function markOAuthReauthorizationRequired( - connection: typeof toolConnections.$inferSelect, - guard: { - leaseId: string; - refreshSecretId: string; - refreshTokenVersion: number; - }, - ) { - const oauth = oauthConfig(connection); - const nextCredentialSecretRefs = connection.credentialSecretRefs.filter( - (ref) => - ref.configPath !== "oauth.access_token" && - ref.configPath !== "oauth.refresh_token", - ); - const nextCredentialRefs = connection.credentialRefs.filter( - (ref) => ref.name !== "oauth.access_token", - ); - const nextConfig = { - ...connection.config, - oauth: { - ...withoutOAuthRefreshLease(oauth), - expiresAt: null, - reauthorizationRequiredAt: now().toISOString(), - }, - }; - const [updated] = await db - .update(toolConnections) - .set({ - status: "draft", - enabled: false, - healthStatus: "error", - healthMessage: - "OAuth authorization expired. Reconnect this app to continue.", - lastError: "oauth_reauthorization_required", - config: nextConfig, - transportConfig: nextConfig, - credentialSecretRefs: nextCredentialSecretRefs, - credentialRefs: nextCredentialRefs, - updatedAt: now(), - }) - .where( - and( - eq(toolConnections.id, connection.id), - eq(toolConnections.companyId, connection.companyId), - sql`${toolConnections.config} -> 'oauth' -> 'refreshLease' ->> 'id' = ${guard.leaseId}`, - sql`exists ( - select 1 from ${companySecrets} - where ${companySecrets.id} = ${guard.refreshSecretId} - and ${companySecrets.companyId} = ${connection.companyId} - and ${companySecrets.latestVersion} = ${guard.refreshTokenVersion} - )`, - ), - ) - .returning(); - if (updated) { - await syncCredentialBindings(updated); - emitConnectionUpdated(updated, connection, "credential_refresh"); - } - return updated ?? null; - } - - async function refreshOAuthCredentials( - connection: typeof toolConnections.$inferSelect, - leaseId: string, - actor?: ActorInfo, - accessContext?: { - actorSource?: - | "local_implicit" - | "session" - | "board_key" - | "agent_key" - | "agent_jwt" - | "mcp_oauth" - | "cloud_tenant"; - issueId?: string | null; - heartbeatRunId?: string | null; - }, - ): Promise { - const oauth = oauthConfig(connection); - if ( - typeof oauth.tokenUrl !== "string" || - typeof oauth.provider !== "string" - ) - return connection; - const expiresAtMs = oauthExpiresAtMs(connection); - if (expiresAtMs && expiresAtMs > Date.now() + 60_000) return connection; - const grantType = - oauth.grantType === "client_credentials" || - oauth.clientCredentials === true - ? ("client_credentials" as const) - : ("refresh_token" as const); - const refreshRef = oauthSecretRef(connection, "oauth.refresh_token"); - if (grantType !== "client_credentials" && !refreshRef) { - throw new HttpError( - 422, - "OAuth credentials have expired and no refresh token is available", - { - code: "oauth_refresh_missing", - setupUrl: connectionSetupUrl(connection), - reconnectUrl: connectionReconnectUrl(connection), - }, - ); - } - const client = await oauthClientForConnection( - connection, - oauth.provider, - actor, - ); - if (!client.clientId) - throw unprocessable( - `OAuth client id is not configured for ${oauth.provider}`, - ); - const [refreshSecret] = refreshRef - ? await db - .select({ latestVersion: companySecrets.latestVersion }) - .from(companySecrets) - .where( - and( - eq(companySecrets.id, refreshRef.secretId), - eq(companySecrets.companyId, connection.companyId), - ), - ) - .limit(1) - : [undefined]; - const refreshTokenVersion = refreshSecret?.latestVersion ?? null; - const refreshToken = - refreshRef && refreshTokenVersion !== null - ? await secrets.resolveSecretValue( - connection.companyId, - refreshRef.secretId, - refreshTokenVersion, - { - consumerType: "tool_connection", - consumerId: connection.id, - configPath: "oauth.refresh_token", - actorType: actor?.actorType ?? "system", - actorId: actor?.actorId ?? null, - actorSource: accessContext?.actorSource, - issueId: accessContext?.issueId, - heartbeatRunId: accessContext?.heartbeatRunId, - }, - ) - : null; - let token: Awaited>; - try { - token = await exchangeOAuthToken({ - tokenUrl: oauth.tokenUrl, - clientId: client.clientId, - clientSecret: client.clientSecret, - tokenEndpointAuthMethod: storedOAuthTokenEndpointAuthMethod( - oauth, - client.clientSecret, - ), - grantType, - scopes: - normalizeOauthScopes(oauth.scopes).length > 0 - ? normalizeOauthScopes(oauth.scopes) - : normalizeOauthScopes(oauth.scope), - refreshToken, - // Refreshing must stay bound to the same MCP server the original grant - // named, or the authorization server may widen the token's audience. - resource: - typeof oauth.resource === "string" && oauth.resource - ? oauth.resource - : null, - }); - } catch (error) { - if ( - error instanceof HttpError && - asRecord(error.details).code === "oauth_reauthorization_required" - ) { - const marked = - refreshRef && refreshTokenVersion !== null - ? await markOAuthReauthorizationRequired(connection, { - leaseId, - refreshSecretId: refreshRef.secretId, - refreshTokenVersion, - }) - : null; - if (!marked) { - const latest = await getConnectionRow( - connection.id, - connection.companyId, - ); - const latestExpiresAtMs = oauthExpiresAtMs(latest); - if (latestExpiresAtMs && latestExpiresAtMs > Date.now() + 60_000) - return latest; - throw conflict( - "OAuth credentials changed while refresh was in progress. Retry the request.", - { - code: "oauth_refresh_superseded", - retryable: true, - }, - ); - } - throw new HttpError(error.status, error.message, { - ...asRecord(error.details), - setupUrl: connectionSetupUrl(connection), - reconnectUrl: connectionReconnectUrl(connection), - }); - } - throw error; - } - // Rotating providers invalidate the submitted refresh token immediately. - // Persist its replacement before the new access token can be returned to a - // caller, so a crash cannot leave the grant with only the consumed token. - let nextRefreshRef: Awaited< - ReturnType - > | null = null; - if (token.refreshToken) { - nextRefreshRef = await createOrRotateOAuthSecret({ - companyId: connection.companyId, - connection, - configPath: "oauth.refresh_token", - label: "OAuth refresh token", - value: token.refreshToken, - actor, - }); - } - const accessRef = await createOrRotateOAuthSecret({ - companyId: connection.companyId, - connection, - configPath: "oauth.access_token", - label: "OAuth access token", - value: token.accessToken, - actor, - }); - const nextCredentialSecretRefs = [ - ...connection.credentialSecretRefs.filter( - (ref) => - ref.configPath !== "oauth.access_token" && - (!nextRefreshRef || ref.configPath !== "oauth.refresh_token"), - ), - accessRef, - ...(nextRefreshRef ? [nextRefreshRef] : []), - ]; - const expiresAt = token.expiresIn - ? new Date(Date.now() + token.expiresIn * 1000).toISOString() - : null; - const nextConfig = { - ...connection.config, - oauth: { - ...withoutOAuthRefreshLease(oauth), - grantType: - grantType === "client_credentials" - ? grantType - : (oauth.grantType ?? "authorization_code"), - expiresAt, - scope: token.scope ?? oauth.scope ?? null, - tokenType: token.tokenType, - refreshedAt: new Date().toISOString(), - }, - providerMetadata: { - ...asRecord(connection.config.providerMetadata), - oauth: { - expiresAt, - scope: token.scope ?? oauth.scope ?? null, - tokenType: token.tokenType, - }, - }, - }; - const [updated] = await db - .update(toolConnections) - .set({ - config: nextConfig, - transportConfig: nextConfig, - credentialSecretRefs: nextCredentialSecretRefs, - credentialRefs: [ - ...connection.credentialRefs.filter( - (ref) => ref.name !== "oauth.access_token", - ), - { - name: "oauth.access_token", - secretId: accessRef.secretId, - version: "latest" as const, - placement: "header" as const, - key: "Authorization", - prefix: "Bearer ", - }, - ], - updatedAt: new Date(), - }) - .where( - and( - eq(toolConnections.id, connection.id), - eq(toolConnections.companyId, connection.companyId), - sql`${toolConnections.config} -> 'oauth' -> 'refreshLease' ->> 'id' = ${leaseId}`, - ), - ) - .returning(); - if (!updated) { - throw conflict( - "OAuth credentials changed while refresh was in progress. Retry the request.", - { - code: "oauth_refresh_superseded", - retryable: true, - }, - ); - } - const previousBindingKeys = new Set( - connection.credentialSecretRefs.map( - (ref) => `${ref.secretId}:${ref.configPath}`, - ), - ); - const nextBindingKeys = new Set( - nextCredentialSecretRefs.map( - (ref) => `${ref.secretId}:${ref.configPath}`, - ), - ); - const bindingsChanged = - previousBindingKeys.size !== nextBindingKeys.size || - [...previousBindingKeys].some((key) => !nextBindingKeys.has(key)); - if (bindingsChanged) await syncCredentialBindings(updated); - return updated; - } - - function oauthGrantConfig(grant: typeof connectionGrants.$inferSelect) { - return asRecord(asRecord(grant.providerTenant).oauth); - } - - function withoutOAuthGrantRefreshLease(oauth: Record) { - const { refreshLease: _refreshLease, ...rest } = oauth; - return rest; - } - - function oauthGrantExpiresAtMs( - grant: typeof connectionGrants.$inferSelect, - connection: typeof toolConnections.$inferSelect, - ): number | null { - const grantExpiresAt = oauthGrantConfig(grant).accessTokenExpiresAt; - const value = - typeof grantExpiresAt === "string" - ? grantExpiresAt - : oauthConfig(connection).expiresAt; - if (typeof value !== "string") return null; - const ms = Date.parse(value); - return Number.isFinite(ms) ? ms : null; - } - - async function getOAuthGrantRow(input: { - companyId: string; - connectionId: string; - grantId: string; - }) { - const [grant] = await db - .select() - .from(connectionGrants) - .where( - and( - eq(connectionGrants.id, input.grantId), - eq(connectionGrants.companyId, input.companyId), - eq(connectionGrants.connectionId, input.connectionId), - ), - ) - .limit(1); - if (!grant) throw notFound("Connection authorization not found"); - return grant; - } - - async function resolveOAuthGrantSecret( - connection: typeof toolConnections.$inferSelect, - grant: typeof connectionGrants.$inferSelect, - ref: ToolCredentialSecretRef, - actor: ActorInfo | undefined, - accessContext: - { issueId?: string | null; heartbeatRunId?: string | null } | undefined, - ) { - return resolveConnectionGrantSecret(db, connection, grant, ref, { - consumerType: "tool_connection", consumerId: connection.id, - actorType: actor?.actorType ?? "system", actorId: actor?.actorId ?? null, - issueId: accessContext?.issueId, heartbeatRunId: accessContext?.heartbeatRunId, - }); - } - - async function clearOAuthGrantRefreshLease( - grant: typeof connectionGrants.$inferSelect, - leaseId: string, - ) { - const latest = await getOAuthGrantRow({ - companyId: grant.companyId, - connectionId: grant.connectionId, - grantId: grant.id, - }); - const oauth = oauthGrantConfig(latest); - if (asRecord(oauth.refreshLease).id !== leaseId) return latest; - const providerTenant = { - ...(latest.providerTenant ?? {}), - oauth: withoutOAuthGrantRefreshLease(oauth), - }; - const [updated] = await db - .update(connectionGrants) - .set({ - providerTenant, - updatedAt: now(), - }) - .where( - and( - eq(connectionGrants.id, latest.id), - eq(connectionGrants.companyId, latest.companyId), - sql`${connectionGrants.providerTenant} -> 'oauth' -> 'refreshLease' ->> 'id' = ${leaseId}`, - ), - ) - .returning(); - return ( - updated ?? - getOAuthGrantRow({ - companyId: grant.companyId, - connectionId: grant.connectionId, - grantId: grant.id, - }) - ); - } - - async function acquireOAuthGrantRefreshLease( - connection: typeof toolConnections.$inferSelect, - grant: typeof connectionGrants.$inferSelect, - forceRefresh: boolean, - ): Promise<{ - grant: typeof connectionGrants.$inferSelect; - leaseId: string | null; - }> { - const waitDeadline = Date.now() + OAUTH_REFRESH_LEASE_WAIT_MS; - const initialUpdatedAt = grant.updatedAt.getTime(); - while (true) { - const latest = await getOAuthGrantRow({ - companyId: connection.companyId, - connectionId: connection.id, - grantId: grant.id, - }); - if (latest.status !== "active") { - throw unprocessable("OAuth authorization must be reconnected", { - code: "oauth_reauthorization_required", - setupUrl: connectionSetupUrl(connection), - reconnectUrl: connectionReconnectUrl(connection), - }); - } - const expiresAtMs = oauthGrantExpiresAtMs(latest, connection); - if ( - (!forceRefresh && - (expiresAtMs === null || expiresAtMs > Date.now() + 60_000)) || - (forceRefresh && - latest.updatedAt.getTime() > initialUpdatedAt && - !asRecord(oauthGrantConfig(latest).refreshLease).id) - ) { - return { grant: latest, leaseId: null }; - } - - const oauth = oauthGrantConfig(latest); - const currentLease = asRecord(oauth.refreshLease); - const currentLeaseId = - typeof currentLease.id === "string" && currentLease.id - ? currentLease.id - : null; - const currentLeaseExpiresAt = - typeof currentLease.expiresAt === "string" - ? Date.parse(currentLease.expiresAt) - : Number.NaN; - const leaseIsActive = - currentLeaseId !== null && - Number.isFinite(currentLeaseExpiresAt) && - currentLeaseExpiresAt > Date.now(); - if (!currentLeaseId) { - const leaseId = randomUUID(); - const providerTenant = { - ...(latest.providerTenant ?? {}), - oauth: { - ...withoutOAuthGrantRefreshLease(oauth), - refreshLease: { - id: leaseId, - expiresAt: new Date( - Date.now() + OAUTH_REFRESH_LEASE_MS, - ).toISOString(), - }, - }, - }; - const [claimed] = await db - .update(connectionGrants) - .set({ - providerTenant, - updatedAt: now(), - }) - .where( - and( - eq(connectionGrants.id, latest.id), - eq(connectionGrants.companyId, latest.companyId), - eq(connectionGrants.status, "active"), - sql`${connectionGrants.providerTenant} #>> '{oauth,refreshLease,id}' is null`, - ), - ) - .returning(); - if (claimed) return { grant: claimed, leaseId }; - } - if (currentLeaseId && !leaseIsActive) { - throw unprocessable( - "The previous OAuth refresh did not finish. Reconnect this app before retrying.", - { - code: "oauth_refresh_outcome_unknown", - setupUrl: connectionSetupUrl(connection), - reconnectUrl: connectionReconnectUrl(connection), - }, - ); - } - if (Date.now() >= waitDeadline) { - throw conflict("OAuth credential refresh is already in progress", { - code: "oauth_refresh_in_progress", - retryable: true, - }); - } - await new Promise((resolve) => - setTimeout(resolve, OAUTH_REFRESH_LEASE_POLL_MS), - ); - } - } - - async function refreshOAuthGrantCredentials(input: { - companyId: string; - connectionId: string; - grantId: string; - forceRefresh?: boolean; - actor?: ActorInfo; - issueId?: string | null; - heartbeatRunId?: string | null; - }): Promise { - const connection = await getConnectionRow( - input.connectionId, - input.companyId, - ); - const initialGrant = await getOAuthGrantRow(input); - const oauth = oauthConfig(connection); - const oauthProvider = - typeof oauth.provider === "string" ? oauth.provider : null; - const oauthTokenUrl = - typeof oauth.tokenUrl === "string" ? oauth.tokenUrl : null; - if ( - connection.authKind === "oauth" && - connection.credentialSource === "paperclip_vault" && - isPaperclipCloudConnectorStrategy(oauth.strategy) - ) { - const grantOauth = oauthGrantConfig(initialGrant); - const expiresAt = - typeof grantOauth.accessTokenExpiresAt === "string" - ? Date.parse(grantOauth.accessTokenExpiresAt) - : Number.NaN; - const refreshedAt = - typeof grantOauth.refreshedAt === "string" - ? Date.parse(grantOauth.refreshedAt) - : Number.NaN; - const rotationDue = - Number.isFinite(expiresAt) && - (!Number.isFinite(refreshedAt) || - refreshedAt <= Date.now() - 30 * 24 * 60 * 60_000); - const refreshDue = - Number.isFinite(expiresAt) && expiresAt <= Date.now() + 60 * 60_000; - // A GitHub App can deliberately issue a non-expiring ghu_ token. Its - // continuity is checked against /user below; only an expiring token pair - // enters this rotation path. - if (!input.forceRefresh && !refreshDue && !rotationDue) - return initialGrant; - if (!Number.isFinite(expiresAt)) return initialGrant; - - return singleFlight( - oauthGrantRefreshFlights, - initialGrant.id, - async () => { - const lease = await acquireOAuthGrantRefreshLease( - connection, - initialGrant, - true, - ); - if (!lease.leaseId) return lease.grant; - try { - const grant = lease.grant; - const profile = managedConnectorProfile( - typeof oauth.connectorProfile === "string" - ? oauth.connectorProfile - : undefined, - ); - const cloudConnector = currentCloudConnector(); - const connectorSubject = - grant.kind === "agent" && grant.subjectAgentId - ? `agent:${grant.subjectAgentId}` - : grant.kind === "user" && grant.subjectUserId - ? grant.subjectUserId - : typeof oauth.connectorSubjectUserId === "string" - ? oauth.connectorSubjectUserId - : null; - const accessRef = grant.credentialSecretRefs.find( - (ref) => ref.configPath === "oauth.access_token", - ); - const refreshRef = grant.credentialSecretRefs.find( - (ref) => ref.configPath === "oauth.refresh_token", - ); - if ( - !profile || - !cloudConnector || - !connectorSubject || - !accessRef || - !refreshRef - ) { - throw unprocessable("Managed authorization must be reconnected", { - code: "oauth_reauthorization_required", - setupUrl: connectionSetupUrl(connection), - reconnectUrl: connectionReconnectUrl(connection), - }); - } - const refreshSecret = await resolveOAuthGrantSecret( - connection, - grant, - refreshRef, - input.actor, - input, - ); - let credentials; - try { - credentials = await cloudConnector.refresh({ - subject: connectorSubject, - companyId: connection.companyId, - profile: profile.id, - refreshToken: refreshSecret.value, - }); - if (profile.id === "workspace.all") { - const requested = normalizeOauthScopes(oauthGrantConfig(grant).requestedScopes ?? oauthGrantConfig(grant).scopes); - credentials.scopes = googleWorkspaceEffectiveScopes(requested, credentials.scopes); - } - if (profile.id === "workspace.all" && !isGoogleWorkspaceScopeGrant(credentials.scopes)) { - throw new PaperclipCloudConnectorError("Google did not return verifiable permissions", "REAUTHORIZATION_REQUIRED"); - } - } catch (error) { - if ( - error instanceof PaperclipCloudConnectorError && - error.code === "REAUTHORIZATION_REQUIRED" - ) { - await db - .update(connectionGrants) - .set({ status: "needs_reauthorization", updatedAt: now() }) - .where( - and( - eq(connectionGrants.id, grant.id), - eq(connectionGrants.companyId, grant.companyId), - ), - ); - throw unprocessable( - "Managed authorization must be reconnected", - { - code: "oauth_reauthorization_required", - setupUrl: connectionSetupUrl(connection), - reconnectUrl: connectionReconnectUrl(connection), - }, - ); - } - throw error; - } - const credentialActor: ActorInfo | undefined = - grant.kind === "user" && grant.subjectUserId - ? { actorType: "user", actorId: grant.subjectUserId } - : input.actor; - const providerTenant = { - ...(grant.providerTenant ?? {}), - oauth: { - ...withoutOAuthGrantRefreshLease(oauthGrantConfig(grant)), - strategy: "paperclip_cloud_connector", - accessTokenExpiresAt: credentials.accessTokenExpiresAt, - scopes: credentials.scopes, - tokenType: credentials.tokenType, - refreshedAt: now().toISOString(), - ...(credentials.refreshTokenExpiresAt - ? { refreshTokenExpiresAt: credentials.refreshTokenExpiresAt } - : {}), - }, - }; - const updated = await db.transaction(async (tx) => { - const [current] = await tx.select({ id: connectionGrants.id }).from(connectionGrants) - .where(and(eq(connectionGrants.id, grant.id), eq(connectionGrants.companyId, grant.companyId), - eq(connectionGrants.status, "active"), - sql`${connectionGrants.providerTenant} -> 'oauth' -> 'refreshLease' ->> 'id' = ${lease.leaseId}`)) - .for("update"); - if (!current) throw conflict("OAuth credentials changed while refresh was in progress", { - code: "oauth_refresh_superseded", retryable: true, - }); - const txSecrets = secretService(tx); - await txSecrets.rotate( - accessRef.secretId, - { value: credentials.accessToken }, - actorForSecret(credentialActor), - ); - if (credentials.refreshToken) { - await txSecrets.rotate( - refreshRef.secretId, - { value: credentials.refreshToken }, - actorForSecret(credentialActor), - ); - } - const [committed] = await tx - .update(connectionGrants) - .set({ - providerTenant, - status: "active", - updatedAt: now(), - }) - .where( - and( - eq(connectionGrants.id, grant.id), - eq(connectionGrants.companyId, grant.companyId), - eq(connectionGrants.status, "active"), - sql`${connectionGrants.providerTenant} -> 'oauth' -> 'refreshLease' ->> 'id' = ${lease.leaseId}`, - ), - ) - .returning(); - return committed; - }); - if (!updated) - throw conflict( - "OAuth credentials changed while refresh was in progress", - { - code: "oauth_refresh_superseded", - retryable: true, - }, - ); - return updated; - } finally { - await clearOAuthGrantRefreshLease(lease.grant, lease.leaseId).catch( - () => undefined, - ); - } - }, - ); - } - if ( - connection.authKind !== "oauth" || - connection.credentialSource !== "paperclip_vault" || - !oauthTokenUrl || - !oauthProvider - ) { - return initialGrant; - } - const expiresAtMs = oauthGrantExpiresAtMs(initialGrant, connection); - if ( - !input.forceRefresh && - (expiresAtMs === null || expiresAtMs > Date.now() + 60_000) - ) { - return initialGrant; - } - - return singleFlight(oauthGrantRefreshFlights, initialGrant.id, async () => { - const lease = await acquireOAuthGrantRefreshLease( - connection, - initialGrant, - input.forceRefresh === true, - ); - if (!lease.leaseId) return lease.grant; - try { - const grant = lease.grant; - const refreshRef = grant.credentialSecretRefs.find( - (ref) => ref.configPath === "oauth.refresh_token", - ); - if (!refreshRef) { - throw unprocessable( - "OAuth credentials have expired and no refresh token is available", - { - code: "oauth_refresh_missing", - setupUrl: connectionSetupUrl(connection), - reconnectUrl: connectionReconnectUrl(connection), - }, - ); - } - const refreshSecret = await resolveOAuthGrantSecret( - connection, - grant, - refreshRef, - input.actor, - input, - ); - const credentialActor: ActorInfo | undefined = - grant.kind === "user" && grant.subjectUserId - ? { actorType: "user", actorId: grant.subjectUserId } - : input.actor; - const client = await oauthClientForConnection( - connection, - oauthProvider, - credentialActor, - ); - if (!client.clientId) - throw unprocessable( - `OAuth client id is not configured for ${oauthProvider}`, - ); - const grantOauth = oauthGrantConfig(grant); - let token: Awaited>; - try { - token = await exchangeOAuthToken({ - tokenUrl: oauthTokenUrl, - clientId: client.clientId, - clientSecret: client.clientSecret, - tokenEndpointAuthMethod: storedOAuthTokenEndpointAuthMethod( - oauth, - client.clientSecret, - ), - grantType: "refresh_token", - scopes: - normalizeOauthScopes(grantOauth.scopes).length > 0 - ? normalizeOauthScopes(grantOauth.scopes) - : normalizeOauthScopes(oauth.scopes).length > 0 - ? normalizeOauthScopes(oauth.scopes) - : normalizeOauthScopes(oauth.scope), - refreshToken: refreshSecret.value, - resource: - typeof oauth.resource === "string" && oauth.resource - ? oauth.resource - : null, - }); - } catch (error) { - if ( - error instanceof HttpError && - asRecord(error.details).code === "oauth_reauthorization_required" - ) { - const retainedCredentialSecretRefs = - grant.credentialSecretRefs.filter( - (ref) => - ref.configPath !== "oauth.access_token" && - ref.configPath !== "oauth.refresh_token", - ); - const providerTenant = { - ...(grant.providerTenant ?? {}), - oauth: { - ...withoutOAuthGrantRefreshLease(grantOauth), - accessTokenExpiresAt: undefined, - }, - }; - const [marked] = await db - .update(connectionGrants) - .set({ - status: "needs_reauthorization", - providerTenant, - credentialSecretRefs: retainedCredentialSecretRefs, - updatedAt: now(), - }) - .where( - and( - eq(connectionGrants.id, grant.id), - eq(connectionGrants.companyId, grant.companyId), - eq(connectionGrants.status, "active"), - sql`${connectionGrants.providerTenant} -> 'oauth' -> 'refreshLease' ->> 'id' = ${lease.leaseId}`, - sql`exists ( - select 1 from ${companySecrets} - where ${companySecrets.id} = ${refreshRef.secretId} - and ${companySecrets.companyId} = ${connection.companyId} - and ${companySecrets.latestVersion} = ${refreshSecret.latestVersion} - )`, - ), - ) - .returning(); - if (!marked) { - const latest = await getOAuthGrantRow(input); - const latestExpiresAt = oauthGrantExpiresAtMs(latest, connection); - if ( - latest.status === "active" && - latestExpiresAt && - latestExpiresAt > Date.now() + 60_000 - ) - return latest; - throw conflict( - "OAuth credentials changed while refresh was in progress. Retry the request.", - { - code: "oauth_refresh_superseded", - retryable: true, - }, - ); - } - if (marked.kind === "organization") { - const latestConnection = await getConnectionRow( - connection.id, - connection.companyId, - ); - const [reauthorizationRequired] = await db - .update(toolConnections) - .set({ - status: "draft", - enabled: false, - credentialSecretRefs: - latestConnection.credentialSecretRefs.filter( - (ref) => - ref.configPath !== "oauth.access_token" && - ref.configPath !== "oauth.refresh_token", - ), - credentialRefs: latestConnection.credentialRefs.filter( - (ref) => - ref.name !== "oauth.access_token" && - ref.name !== "oauth.refresh_token", - ), - updatedAt: now(), - }) - .where( - and( - eq(toolConnections.id, latestConnection.id), - eq(toolConnections.companyId, latestConnection.companyId), - ), - ) - .returning(); - await syncCredentialBindings(reauthorizationRequired); - emitConnectionUpdated( - reauthorizationRequired, - latestConnection, - "credential_refresh", - ); - } else { - const activeGrantRefs = await db - .select({ - refs: connectionGrants.credentialSecretRefs, - }) - .from(connectionGrants) - .where( - and( - eq(connectionGrants.companyId, connection.companyId), - eq(connectionGrants.connectionId, connection.id), - eq(connectionGrants.status, "active"), - ), - ); - await syncCredentialBindings( - connection, - activeGrantRefs.flatMap((row) => row.refs), - ); - } - throw new HttpError(error.status, error.message, { - ...asRecord(error.details), - setupUrl: connectionSetupUrl(connection), - reconnectUrl: connectionReconnectUrl(connection), - }); - } - throw error; - } - - if (connection.config.sourceTemplateKey === "google-workspace" - && !isGoogleWorkspaceScopeGrant(normalizeOauthScopes(token.scope))) { - await db.update(connectionGrants).set({ status: "needs_reauthorization", updatedAt: now() }) - .where(and(eq(connectionGrants.id, grant.id), eq(connectionGrants.companyId, connection.companyId))); - throw unprocessable("Google did not return verifiable permissions. Reconnect Google Workspace.", { code: "oauth_reauthorization_required" }); - } - let nextRefreshRef: ToolCredentialSecretRef | null = null; - if (token.refreshToken) { - nextRefreshRef = await createOrRotateOAuthSecret({ - companyId: connection.companyId, - connection, - configPath: "oauth.refresh_token", - label: "OAuth refresh token", - value: token.refreshToken, - actor: credentialActor, - existingRefs: grant.credentialSecretRefs, - ownerUserId: - grant.kind === "user" - ? (grant.subjectUserId ?? undefined) - : undefined, - }); - } - const accessRef = await createOrRotateOAuthSecret({ - companyId: connection.companyId, - connection, - configPath: "oauth.access_token", - label: "OAuth access token", - value: token.accessToken, - actor: credentialActor, - existingRefs: grant.credentialSecretRefs, - ownerUserId: - grant.kind === "user" - ? (grant.subjectUserId ?? undefined) - : undefined, - }); - const nextCredentialSecretRefs = [ - ...grant.credentialSecretRefs.filter( - (ref) => - ref.configPath !== "oauth.access_token" && - (!nextRefreshRef || ref.configPath !== "oauth.refresh_token"), - ), - accessRef, - ...(nextRefreshRef ? [nextRefreshRef] : []), - ]; - const expiresAt = token.expiresIn - ? new Date(Date.now() + token.expiresIn * 1000).toISOString() - : null; - // A refresh carries no fresh authorization request, so the baseline stays what - // Paperclip asked for when *this* grant was authorized. Judging the response against - // the grant's own scopes instead would let an over-grant that the provider re-asserts - // on every refresh read back as clean, because the widened grant would have become - // its own baseline. The connection-level list is only a fallback for grants created - // before the per-grant baseline existed — it is whichever callback ran last, so on a - // connection two users authorized with different scopes it is the wrong baseline for - // at least one of them. - const refreshed = resolveGrantedOauthScopes({ - tokenScope: token.scope, - requestedScopes: - grantOauth.requestedScopes ?? - oauth.scopes ?? - oauth.scope ?? - grantOauth.scopes, - previous: { - scopes: grantOauth.scopes, - scopeSource: grantOauth.scopeSource, - unrequestedScopes: grantOauth.unrequestedScopes, - }, - }); - if (connection.config.sourceTemplateKey === "google-workspace") { - refreshed.scopes = googleWorkspaceEffectiveScopes(normalizeOauthScopes(grantOauth.requestedScopes ?? grantOauth.scopes), refreshed.scopes); - } - const providerTenant = { - ...(grant.providerTenant ?? {}), - oauth: { - ...withoutOAuthGrantRefreshLease(grantOauth), - strategy: - typeof grantOauth.strategy === "string" - ? grantOauth.strategy - : "direct_oauth", - accessTokenExpiresAt: expiresAt ?? undefined, - scopes: refreshed.scopes, - scopeSource: refreshed.scopeSource, - unrequestedScopes: refreshed.unrequestedScopes, - tokenType: token.tokenType, - refreshedAt: now().toISOString(), - }, - }; - const [updated] = await db - .update(connectionGrants) - .set({ - providerTenant, - credentialSecretRefs: nextCredentialSecretRefs, - status: "active", - updatedAt: now(), - }) - .where( - and( - eq(connectionGrants.id, grant.id), - eq(connectionGrants.companyId, grant.companyId), - eq(connectionGrants.status, "active"), - sql`${connectionGrants.providerTenant} -> 'oauth' -> 'refreshLease' ->> 'id' = ${lease.leaseId}`, - ), - ) - .returning(); - if (!updated) { - throw conflict( - "OAuth credentials changed while refresh was in progress. Retry the request.", - { - code: "oauth_refresh_superseded", - retryable: true, - }, - ); - } - if (grant.kind === "organization") { - const latestConnection = await getConnectionRow( - connection.id, - connection.companyId, - ); - const latestOauth = oauthConfig(latestConnection); - const nextConfig = { - ...latestConnection.config, - oauth: { - ...withoutOAuthRefreshLease(latestOauth), - expiresAt, - scope: token.scope ?? latestOauth.scope ?? null, - tokenType: token.tokenType, - refreshedAt: now().toISOString(), - }, - providerMetadata: { - ...asRecord(latestConnection.config.providerMetadata), - oauth: { - expiresAt, - scope: token.scope ?? latestOauth.scope ?? null, - // Carry the same provenance the grant just recorded. Without it a connection - // read reverts to a bare scope with no source and no over-grant warning after - // the first ordinary refresh, while the grant read still has both. - scopeSource: refreshed.scopeSource, - unrequestedScopes: refreshed.unrequestedScopes, - tokenType: token.tokenType, - }, - }, - }; - await db - .update(toolConnections) - .set({ - config: nextConfig, - transportConfig: nextConfig, - updatedAt: now(), - }) - .where( - and( - eq(toolConnections.id, latestConnection.id), - eq(toolConnections.companyId, latestConnection.companyId), - ), - ); - } - const previousKeys = new Set( - grant.credentialSecretRefs.map( - (ref) => `${ref.secretId}:${ref.configPath}`, - ), - ); - const nextKeys = new Set( - nextCredentialSecretRefs.map( - (ref) => `${ref.secretId}:${ref.configPath}`, - ), - ); - if ( - previousKeys.size !== nextKeys.size || - [...previousKeys].some((key) => !nextKeys.has(key)) - ) { - const activeGrantRefs = await db - .select({ - refs: connectionGrants.credentialSecretRefs, - }) - .from(connectionGrants) - .where( - and( - eq(connectionGrants.companyId, connection.companyId), - eq(connectionGrants.connectionId, connection.id), - eq(connectionGrants.status, "active"), - ), - ); - await syncCredentialBindings( - connection, - activeGrantRefs.flatMap((row) => row.refs), - ); - } - return updated; - } finally { - await clearOAuthGrantRefreshLease(lease.grant, lease.leaseId).catch( - () => undefined, - ); - } - }); - } - - async function maybeRefreshOAuthCredentials( - connection: typeof toolConnections.$inferSelect, - actor?: ActorInfo, - accessContext?: { - actorSource?: - | "local_implicit" - | "session" - | "board_key" - | "agent_key" - | "agent_jwt" - | "mcp_oauth" - | "cloud_tenant"; - issueId?: string | null; - heartbeatRunId?: string | null; - }, - ): Promise { - const oauth = oauthConfig(connection); - if ( - typeof oauth.tokenUrl !== "string" || - typeof oauth.provider !== "string" - ) - return connection; - const expiresAtMs = oauthExpiresAtMs(connection); - if (expiresAtMs && expiresAtMs > Date.now() + 60_000) return connection; - return singleFlight(oauthRefreshFlights, connection.id, async () => { - const lease = await acquireOAuthRefreshLease(connection); - if (!lease.leaseId) return lease.connection; - try { - return await refreshOAuthCredentials( - lease.connection, - lease.leaseId, - actor, - accessContext, - ); - } finally { - await clearOAuthRefreshLease(lease.connection, lease.leaseId).catch( - () => undefined, - ); - } - }); - } - - async function refreshManagedGitHubGrantAccess( - connection: typeof toolConnections.$inferSelect, - initialGrant: typeof connectionGrants.$inferSelect, - actor?: ActorInfo, - ) { - return measureRunPhase("connectors.github.refresh", async () => { - try { - return await refreshManagedGitHubGrantAccessOnce( - connection, - initialGrant, - actor, - ); - } catch (error) { - if ( - !(error instanceof HttpError) || - asRecord(error.details).code !== "github_access_changed" - ) - throw error; - const [current] = await measureRunPhase("connectors.github.grants", () => db - .select() - .from(connectionGrants) - .where( - and( - eq(connectionGrants.id, initialGrant.id), - eq(connectionGrants.companyId, connection.companyId), - eq(connectionGrants.connectionId, connection.id), - ), - )); - if (!current || current.status !== "active") throw error; - return refreshManagedGitHubGrantAccessOnce(connection, current, actor); - } - }); - } - - async function refreshManagedGitHubGrantAccessOnce( - connection: typeof toolConnections.$inferSelect, - initialGrant: typeof connectionGrants.$inferSelect, - actor?: ActorInfo, - ) { - let grant = await measureRunPhase("connectors.github.credentials", () => refreshOAuthGrantCredentials({ - companyId: connection.companyId, - connectionId: connection.id, - grantId: initialGrant.id, - actor, - })); - const resolveAccessToken = async () => measureRunPhase("connectors.github.credentials", async () => { - const accessRef = grant.credentialSecretRefs.find( - (ref) => ref.configPath === "oauth.access_token", - ); - if (!accessRef) - throw unprocessable("GitHub authorization must be reconnected", { - code: "oauth_reauthorization_required", - }); - return ( - await resolveOAuthGrantSecret( - connection, - grant, - accessRef, - actor, - undefined, - ) - ).value; - }); - let metadata; - let accessToken = await resolveAccessToken(); - try { - metadata = await measureRunPhase("connectors.github.repositories", () => loadGitHubGrantMetadata( - accessToken, - fetch, - grant.providerTenant?.github?.appSlug, - )); - } catch (error) { - const providerCode = - error instanceof HttpError ? asRecord(error.details).code : null; - if (providerCode !== "oauth_reauthorization_required") throw error; - // GitHub may invalidate an access token before its recorded expiry. If an - // expiring token pair exists, rotate it once under the same durable lease - // and CAS path, then repeat /user before surfacing reconnect-required. - grant = await measureRunPhase("connectors.github.credentials", () => refreshOAuthGrantCredentials({ - companyId: connection.companyId, - connectionId: connection.id, - grantId: initialGrant.id, - forceRefresh: true, - actor, - })); - accessToken = await resolveAccessToken(); - try { - metadata = await measureRunPhase("connectors.github.repositories", () => loadGitHubGrantMetadata( - accessToken, - fetch, - grant.providerTenant?.github?.appSlug, - )); - } catch (retryError) { - const retryCode = - retryError instanceof HttpError - ? asRecord(retryError.details).code - : null; - if (retryCode === "oauth_reauthorization_required") { - await measureRunPhase("connectors.github.grant_write", () => db - .update(connectionGrants) - .set({ status: "needs_reauthorization", updatedAt: now() }) - .where( - and( - eq(connectionGrants.id, grant.id), - eq(connectionGrants.companyId, grant.companyId), - ), - )); - } - throw retryError; - } - } - const { updated, previousGitHub } = await measureRunTransaction(db, "connectors.github.refresh", async (tx) => { - const [currentGrant] = await measureRunPhase("connectors.github.grant_lock", () => tx - .select() - .from(connectionGrants) - .where( - and( - eq(connectionGrants.id, grant.id), - eq(connectionGrants.companyId, grant.companyId), - ), - ) - .for("update") - .limit(1)); - if (!currentGrant || currentGrant.status === "revoked") - throw notFound("GitHub authorization not found"); - const previousGitHub = currentGrant.providerTenant?.github; - const initialGitHub = grant.providerTenant?.github; - // No lock is held during provider requests. Reject a snapshot if another - // refresh or webhook changed access while those requests were in flight. - if ( - previousGitHub?.accessRevision !== initialGitHub?.accessRevision || - previousGitHub?.lastWebhookAt !== initialGitHub?.lastWebhookAt || - previousGitHub?.lastAccessRefreshAt !== - initialGitHub?.lastAccessRefreshAt - ) { - throw conflict("GitHub access changed during refresh. Try again.", { - code: "github_access_changed", - }); - } - const providerTenant = { - ...(currentGrant.providerTenant ?? {}), - github: { - ...metadata, - ...(previousGitHub?.lastWebhookAt - ? { lastWebhookAt: previousGitHub.lastWebhookAt } - : {}), - webhookHealth: - previousGitHub?.webhookHealth ?? metadata.webhookHealth, - }, - }; - const [updated] = await measureRunPhase("connectors.github.grant_write", () => tx - .update(connectionGrants) - .set({ - providerTenant, - status: "active", - updatedAt: now(), - }) - .where( - and( - eq(connectionGrants.id, grant.id), - eq(connectionGrants.companyId, grant.companyId), - ), - ) - .returning()); - if (!updated) throw notFound("GitHub authorization not found"); - return { updated, previousGitHub }; - }); - - const cloudConnector = currentCloudConnector(); - const subject = - updated.kind === "agent" && updated.subjectAgentId - ? `agent:${updated.subjectAgentId}` - : updated.kind === "user" && updated.subjectUserId - ? updated.subjectUserId - : null; - if (cloudConnector && subject) { - const previous = new Set(previousGitHub?.installationIds ?? []); - const current = new Set(metadata.installationIds); - await measureRunPhase("connectors.github.webhooks", () => Promise.all([ - ...metadata.installationIds.map((installationId) => - cloudConnector.setWebhookBinding({ - subject, - companyId: connection.companyId, - id: `${updated.id}_${installationId}`, - installationId, - connectionId: connection.id, - grantId: updated.id, - active: true, - accessToken, - }), - ), - ...[...previous] - .filter((installationId) => !current.has(installationId)) - .map((installationId) => - cloudConnector.setWebhookBinding({ - subject, - companyId: connection.companyId, - id: `${updated.id}_${installationId}`, - installationId, - connectionId: connection.id, - grantId: updated.id, - active: false, - }), - ), - ])); - } - return updated; - } - - async function sweepGitHubConnectionContinuity() { - if (now().getTime() < nextGitHubContinuitySweepAt) { - return { checked: 0, due: 0, refreshed: 0, failed: 0 }; - } - nextGitHubContinuitySweepAt = now().getTime() + 60 * 60_000; - const cutoff = now().getTime() - 30 * 24 * 60 * 60_000; - const rows = await db - .select({ grant: connectionGrants, connection: toolConnections }) - .from(connectionGrants) - .innerJoin( - toolConnections, - and( - eq(toolConnections.id, connectionGrants.connectionId), - eq(toolConnections.companyId, connectionGrants.companyId), - ), - ) - .where( - and( - eq(connectionGrants.status, "active"), - eq(toolConnections.status, "active"), - eq(toolConnections.enabled, true), - ), - ); - const due = rows.filter(({ grant, connection }) => { - const config = asRecord(connection.config); - const oauth = asRecord(config.oauth); - if ( - config.sourceTemplateKey !== "github" || - oauth.connectorProfile !== "github.code" || - !grant.providerTenant?.github - ) - return false; - const expiresAt = grant.providerTenant.oauth?.accessTokenExpiresAt; - const refreshedAt = grant.providerTenant.oauth?.refreshedAt - ? Date.parse(grant.providerTenant.oauth.refreshedAt) - : Number.NaN; - const accessCheckedAt = grant.providerTenant.github.lastAccessRefreshAt - ? Date.parse(grant.providerTenant.github.lastAccessRefreshAt) - : Number.NaN; - if (typeof expiresAt === "string") { - const expiry = Date.parse(expiresAt); - return ( - !Number.isFinite(expiry) || - expiry <= now().getTime() + 60 * 60_000 || - !Number.isFinite(refreshedAt) || - refreshedAt <= cutoff - ); - } - return !Number.isFinite(accessCheckedAt) || accessCheckedAt <= cutoff; - }); - let refreshed = 0; - let failed = 0; - for (const row of due) { - try { - await refreshManagedGitHubGrantAccess(row.connection, row.grant, { - actorType: "system", - actorId: "github-connection-continuity", - }); - refreshed += 1; - } catch { - failed += 1; - } - } - return { checked: rows.length, due: due.length, refreshed, failed }; - } - function policyNameForApp( connection: typeof toolConnections.$inferSelect, entry: typeof toolCatalogEntries.$inferSelect, diff --git a/server/src/services/tool-access/credential-access.ts b/server/src/services/tool-access/credential-access.ts new file mode 100644 index 00000000000..63d0919ddff --- /dev/null +++ b/server/src/services/tool-access/credential-access.ts @@ -0,0 +1,301 @@ +import type { ActorInfo } from "./types.js"; +import { oauthConfig } from "./oauth/metadata.js"; +import { remoteEndpoint } from "./connection-methods.js"; +import { and, eq } from "drizzle-orm"; +import type { Db } from "@paperclipai/db"; +import { connectionGrants, toolConnections } from "@paperclipai/db"; +import { badRequest, HttpError, unprocessable } from "../../errors.js"; +import { parseRemoteHttpEndpoint } from "../remote-http-endpoint-guard.js"; +import { REMOTE_URL_SECRET_CONFIG_PATH, remoteUrlCredentialMatchesPublicUrl } from "../remote-url-credentials.js"; +import { secretService } from "../secrets.js"; +import { connectionCredentialConfigPath as credentialRefConfigPath, connectionGrantCredentialRef } from "../connection-credentials.js"; + +import type { ToolAccessAudit } from "./audit.js"; +import type { VercelCredentialService } from "./vercel-credentials.js"; +import type { OAuthCredentialStore } from "./oauth/credential-store.js"; +import type { createLegacyOAuthRefresh } from "./oauth/connection-refresh.js"; +import type { createOAuthGrantRefresh } from "./oauth/grant-refresh.js"; +import { credentialScope } from "./oauth/client-settings.js"; +import { connectionSetupUrl, connectionReconnectUrl } from "./identifiers.js"; +function asRecord(value: unknown): Record { + if (value && typeof value === "object" && !Array.isArray(value)) + return value as Record; + return {}; +} + +type CredentialAccessDependencies = { + secrets: ReturnType; + audit: ToolAccessAudit; + vercel: VercelCredentialService; + resolveOAuthGrantSecret: OAuthCredentialStore["resolveOAuthGrantSecret"]; + maybeRefreshOAuthCredentials: ReturnType; + refreshOAuthGrantCredentials: ReturnType; +}; + +export function createConnectionCredentialAccess(db: Db, { secrets, audit, vercel, resolveOAuthGrantSecret, maybeRefreshOAuthCredentials, refreshOAuthGrantCredentials }: CredentialAccessDependencies) { + const { resolveVercelCredentialHeaders, vercelGrantForConnection } = vercel; + async function vaultGrantForConnection( + connection: typeof toolConnections.$inferSelect, + actor?: ActorInfo, + ): Promise { + const actorUserId = + actor?.actorType === "user" ? (actor.actorId ?? null) : null; + if (actorUserId) { + const [personal] = await db + .select() + .from(connectionGrants) + .where( + and( + eq(connectionGrants.companyId, connection.companyId), + eq(connectionGrants.connectionId, connection.id), + eq(connectionGrants.kind, "user"), + eq(connectionGrants.subjectUserId, actorUserId), + ), + ) + .limit(1); + if (personal) { + if (personal.status !== "active") { + throw unprocessable("OAuth authorization must be reconnected", { + code: "oauth_reauthorization_required", + setupUrl: connectionSetupUrl(connection), + reconnectUrl: connectionReconnectUrl(connection), + }); + } + return personal; + } + } + if (!actorUserId && connection.credentialPolicy === "per_user") { + // A personal-only connection is intentionally fixed to one identity. + // Background health/catalog checks have no acting user, but may safely + // exercise that sole owner-bound grant without turning it into a shared + // credential or making it available to a different caller. + const personalGrants = await db + .select() + .from(connectionGrants) + .where( + and( + eq(connectionGrants.companyId, connection.companyId), + eq(connectionGrants.connectionId, connection.id), + eq(connectionGrants.kind, "user"), + eq(connectionGrants.status, "active"), + ), + ) + .limit(2); + if (personalGrants.length === 1) return personalGrants[0]!; + } + if (connection.credentialPolicy === "per_user") { + throw unprocessable( + "This connection needs the current user's authorization", + { + code: "user_authorization_required", + setupUrl: connectionSetupUrl(connection), + reconnectUrl: connectionReconnectUrl(connection), + }, + ); + } + const [organization] = await db + .select() + .from(connectionGrants) + .where( + and( + eq(connectionGrants.companyId, connection.companyId), + eq(connectionGrants.connectionId, connection.id), + eq(connectionGrants.kind, "organization"), + eq(connectionGrants.isDefault, true), + ), + ) + .limit(1); + if (organization?.status === "active") return organization; + return null; + } + + async function resolveCredentialHeaders( + connection: typeof toolConnections.$inferSelect, + actor?: ActorInfo, + options: { forceRefresh?: boolean } = {}, + ): Promise> { + if (connection.credentialSource === "vercel_connect") { + const grant = await vercelGrantForConnection(connection, actor); + return resolveVercelCredentialHeaders(connection, grant, options); + } + let grant: typeof connectionGrants.$inferSelect | null = null; + try { + if ( + connection.authKind === "oauth" || + connection.credentialPolicy !== "shared" + ) { + grant = await vaultGrantForConnection(connection, actor); + } + if (connection.authKind === "oauth" && grant) { + grant = await refreshOAuthGrantCredentials({ + companyId: connection.companyId, + connectionId: connection.id, + grantId: grant.id, + forceRefresh: options.forceRefresh, + actor, + }); + } else { + connection = await maybeRefreshOAuthCredentials(connection, actor); + } + } catch (error) { + const scope = credentialScope(connection); + await audit({ + companyId: connection.companyId, + connectionId: connection.id, + action: "tool_connection.credential_resolution", + outcome: "failure", + reasonCode: + error instanceof HttpError + ? String(asRecord(error.details).code ?? "oauth_refresh_failed") + : "oauth_refresh_failed", + details: { + credentialCount: connection.credentialRefs.length, + credentialSecretRefCount: connection.credentialSecretRefs.length, + credentialScopeType: scope.type, + credentialScopeHash: scope.hash, + setupUrl: connectionSetupUrl(connection), + reconnectUrl: connectionReconnectUrl(connection), + }, + }); + throw error; + } + const headers: Record = {}; + const scope = credentialScope(connection); + for (const ref of connection.credentialRefs) { + if (ref.placement !== "header") continue; + let value: string; + const configPath = credentialRefConfigPath(ref); + try { + const grantRef = grant ? connectionGrantCredentialRef(grant, ref) : undefined; + if (grant && !grantRef) throw unprocessable("Your credential is missing. Reconnect this connection.", { + code: "grant_credential_invalid", connectionId: connection.id, grantId: grant.id, credential: configPath, + }); + value = + grantRef && grant + ? ( + await resolveOAuthGrantSecret( + connection, + grant, + grantRef, + actor, + undefined, + ) + ).value + : await secrets.resolveSecretValue( + connection.companyId, + ref.secretId, + ref.version ?? "latest", + { + consumerType: "tool_connection", + consumerId: connection.id, + configPath, + actorType: "system", + }, + ); + } catch (error) { + await audit({ + companyId: connection.companyId, + connectionId: connection.id, + action: "tool_connection.credential_resolution", + outcome: "failure", + reasonCode: + error instanceof HttpError + ? String( + asRecord(error.details).code ?? "secret_resolution_failed", + ) + : "secret_resolution_failed", + details: { + credentialCount: connection.credentialRefs.length, + credentialScopeType: scope.type, + credentialScopeHash: scope.hash, + }, + }); + throw error; + } + if (ref.placement === "header") { + headers[ref.key] = `${ref.prefix ?? ""}${value}`; + } + } + const oauthAccessRef = grant?.credentialSecretRefs.find( + (ref) => ref.configPath === "oauth.access_token", + ); + if (oauthAccessRef && headers.Authorization === undefined) { + headers.Authorization = `Bearer ${ + ( + await resolveOAuthGrantSecret( + connection, + grant!, + oauthAccessRef, + actor, + undefined, + ) + ).value + }`; + } + if ( + connection.credentialRefs.length > 0 || + connection.credentialSecretRefs.length > 0 || + Object.keys(oauthConfig(connection)).length > 0 + ) { + await audit({ + companyId: connection.companyId, + connectionId: connection.id, + action: "tool_connection.credential_resolution", + outcome: "success", + details: { + credentialCount: connection.credentialRefs.length, + credentialSecretRefCount: connection.credentialSecretRefs.length, + credentialScopeType: scope.type, + credentialScopeHash: scope.hash, + }, + }); + } + return headers; + } + + async function resolvedRemoteEndpoint( + connection: typeof toolConnections.$inferSelect, + actor?: ActorInfo, + ): Promise { + const publicEndpoint = remoteEndpoint(connection.config); + const ref = connection.credentialRefs.find( + (candidate) => candidate.placement === "url", + ); + if (!ref) return publicEndpoint; + let value: string; + try { + const grant = await vaultGrantForConnection(connection, actor); + const grantRef = grant ? connectionGrantCredentialRef(grant, ref) : undefined; + if (grant && !grantRef) throw unprocessable("Your MCP URL credential is missing. Reconnect this connection.", { + code: "grant_credential_invalid", connectionId: connection.id, grantId: grant.id, + }); + value = grant && grantRef + ? (await resolveOAuthGrantSecret(connection, grant, grantRef, actor, undefined)).value + : await secrets.resolveSecretValue(connection.companyId, ref.secretId, ref.version ?? "latest", { + consumerType: "tool_connection", consumerId: connection.id, + configPath: REMOTE_URL_SECRET_CONFIG_PATH, + actorType: actor?.actorType ?? "system", actorId: actor?.actorId ?? null, + }); + } catch (error) { + if (error instanceof HttpError) throw error; + throw unprocessable("Your MCP URL credential could not be resolved. Reconnect this connection.", { + code: "mcp_remote_missing_secret", connectionId: connection.id, credential: REMOTE_URL_SECRET_CONFIG_PATH, + }); + } + if (!remoteUrlCredentialMatchesPublicUrl(publicEndpoint, value)) { + throw unprocessable( + "The stored MCP URL credential no longer matches this connection.", + { + code: "mcp_remote_url_credential_mismatch", + connectionId: connection.id, + }, + ); + } + return parseRemoteHttpEndpoint(value, (message, code) => + badRequest(message, { code }), + ).toString(); + } + return { vaultGrantForConnection, resolveCredentialHeaders, resolvedRemoteEndpoint }; +} + +export type ConnectionCredentialAccess = ReturnType; diff --git a/server/src/services/tool-access/github-grant-continuity.ts b/server/src/services/tool-access/github-grant-continuity.ts new file mode 100644 index 00000000000..31691bd7167 --- /dev/null +++ b/server/src/services/tool-access/github-grant-continuity.ts @@ -0,0 +1,303 @@ +import type { ActorInfo } from "./types.js"; +import { loadGitHubGrantMetadata } from "./github-access-discovery.js"; +import { measureRunPhase } from "../execution-preparation-timing.js"; +import { measureRunTransaction } from "../execution-transaction-timing.js"; +import { and, eq } from "drizzle-orm"; +import type { Db } from "@paperclipai/db"; +import { connectionGrants, toolConnections } from "@paperclipai/db"; +import { conflict, HttpError, notFound, unprocessable } from "../../errors.js"; +import { type PaperclipCloudConnector } from "../paperclip-cloud-connector.js"; + +import type { createOAuthGrantRefresh } from "./oauth/grant-refresh.js"; +import type { OAuthCredentialStore } from "./oauth/credential-store.js"; +function asRecord(value: unknown): Record { + if (value && typeof value === "object" && !Array.isArray(value)) + return value as Record; + return {}; +} + +type GitHubContinuityDependencies = { + now: () => Date; + currentCloudConnector: () => PaperclipCloudConnector | null; + refreshOAuthGrantCredentials: ReturnType; + resolveOAuthGrantSecret: OAuthCredentialStore["resolveOAuthGrantSecret"]; +}; + +export function createGitHubGrantContinuity(db: Db, { now, currentCloudConnector, refreshOAuthGrantCredentials, resolveOAuthGrantSecret }: GitHubContinuityDependencies) { + let nextGitHubContinuitySweepAt = 0; + async function refreshManagedGitHubGrantAccess( + connection: typeof toolConnections.$inferSelect, + initialGrant: typeof connectionGrants.$inferSelect, + actor?: ActorInfo, + ) { + return measureRunPhase("connectors.github.refresh", async () => { + try { + return await refreshManagedGitHubGrantAccessOnce( + connection, + initialGrant, + actor, + ); + } catch (error) { + if ( + !(error instanceof HttpError) || + asRecord(error.details).code !== "github_access_changed" + ) + throw error; + const [current] = await measureRunPhase("connectors.github.grants", () => db + .select() + .from(connectionGrants) + .where( + and( + eq(connectionGrants.id, initialGrant.id), + eq(connectionGrants.companyId, connection.companyId), + eq(connectionGrants.connectionId, connection.id), + ), + )); + if (!current || current.status !== "active") throw error; + return refreshManagedGitHubGrantAccessOnce(connection, current, actor); + } + }); + } + + async function refreshManagedGitHubGrantAccessOnce( + connection: typeof toolConnections.$inferSelect, + initialGrant: typeof connectionGrants.$inferSelect, + actor?: ActorInfo, + ) { + let grant = await measureRunPhase("connectors.github.credentials", () => refreshOAuthGrantCredentials({ + companyId: connection.companyId, + connectionId: connection.id, + grantId: initialGrant.id, + actor, + })); + const resolveAccessToken = async () => measureRunPhase("connectors.github.credentials", async () => { + const accessRef = grant.credentialSecretRefs.find( + (ref) => ref.configPath === "oauth.access_token", + ); + if (!accessRef) + throw unprocessable("GitHub authorization must be reconnected", { + code: "oauth_reauthorization_required", + }); + return ( + await resolveOAuthGrantSecret( + connection, + grant, + accessRef, + actor, + undefined, + ) + ).value; + }); + let metadata; + let accessToken = await resolveAccessToken(); + try { + metadata = await measureRunPhase("connectors.github.repositories", () => loadGitHubGrantMetadata( + accessToken, + fetch, + grant.providerTenant?.github?.appSlug, + )); + } catch (error) { + const providerCode = + error instanceof HttpError ? asRecord(error.details).code : null; + if (providerCode !== "oauth_reauthorization_required") throw error; + // GitHub may invalidate an access token before its recorded expiry. If an + // expiring token pair exists, rotate it once under the same durable lease + // and CAS path, then repeat /user before surfacing reconnect-required. + grant = await measureRunPhase("connectors.github.credentials", () => refreshOAuthGrantCredentials({ + companyId: connection.companyId, + connectionId: connection.id, + grantId: initialGrant.id, + forceRefresh: true, + actor, + })); + accessToken = await resolveAccessToken(); + try { + metadata = await measureRunPhase("connectors.github.repositories", () => loadGitHubGrantMetadata( + accessToken, + fetch, + grant.providerTenant?.github?.appSlug, + )); + } catch (retryError) { + const retryCode = + retryError instanceof HttpError + ? asRecord(retryError.details).code + : null; + if (retryCode === "oauth_reauthorization_required") { + await measureRunPhase("connectors.github.grant_write", () => db + .update(connectionGrants) + .set({ status: "needs_reauthorization", updatedAt: now() }) + .where( + and( + eq(connectionGrants.id, grant.id), + eq(connectionGrants.companyId, grant.companyId), + ), + )); + } + throw retryError; + } + } + const { updated, previousGitHub } = await measureRunTransaction(db, "connectors.github.refresh", async (tx) => { + const [currentGrant] = await measureRunPhase("connectors.github.grant_lock", () => tx + .select() + .from(connectionGrants) + .where( + and( + eq(connectionGrants.id, grant.id), + eq(connectionGrants.companyId, grant.companyId), + ), + ) + .for("update") + .limit(1)); + if (!currentGrant || currentGrant.status === "revoked") + throw notFound("GitHub authorization not found"); + const previousGitHub = currentGrant.providerTenant?.github; + const initialGitHub = grant.providerTenant?.github; + // No lock is held during provider requests. Reject a snapshot if another + // refresh or webhook changed access while those requests were in flight. + if ( + previousGitHub?.accessRevision !== initialGitHub?.accessRevision || + previousGitHub?.lastWebhookAt !== initialGitHub?.lastWebhookAt || + previousGitHub?.lastAccessRefreshAt !== + initialGitHub?.lastAccessRefreshAt + ) { + throw conflict("GitHub access changed during refresh. Try again.", { + code: "github_access_changed", + }); + } + const providerTenant = { + ...(currentGrant.providerTenant ?? {}), + github: { + ...metadata, + ...(previousGitHub?.lastWebhookAt + ? { lastWebhookAt: previousGitHub.lastWebhookAt } + : {}), + webhookHealth: + previousGitHub?.webhookHealth ?? metadata.webhookHealth, + }, + }; + const [updated] = await measureRunPhase("connectors.github.grant_write", () => tx + .update(connectionGrants) + .set({ + providerTenant, + status: "active", + updatedAt: now(), + }) + .where( + and( + eq(connectionGrants.id, grant.id), + eq(connectionGrants.companyId, grant.companyId), + ), + ) + .returning()); + if (!updated) throw notFound("GitHub authorization not found"); + return { updated, previousGitHub }; + }); + + const cloudConnector = currentCloudConnector(); + const subject = + updated.kind === "agent" && updated.subjectAgentId + ? `agent:${updated.subjectAgentId}` + : updated.kind === "user" && updated.subjectUserId + ? updated.subjectUserId + : null; + if (cloudConnector && subject) { + const previous = new Set(previousGitHub?.installationIds ?? []); + const current = new Set(metadata.installationIds); + await measureRunPhase("connectors.github.webhooks", () => Promise.all([ + ...metadata.installationIds.map((installationId) => + cloudConnector.setWebhookBinding({ + subject, + companyId: connection.companyId, + id: `${updated.id}_${installationId}`, + installationId, + connectionId: connection.id, + grantId: updated.id, + active: true, + accessToken, + }), + ), + ...[...previous] + .filter((installationId) => !current.has(installationId)) + .map((installationId) => + cloudConnector.setWebhookBinding({ + subject, + companyId: connection.companyId, + id: `${updated.id}_${installationId}`, + installationId, + connectionId: connection.id, + grantId: updated.id, + active: false, + }), + ), + ])); + } + return updated; + } + + async function sweepGitHubConnectionContinuity() { + if (now().getTime() < nextGitHubContinuitySweepAt) { + return { checked: 0, due: 0, refreshed: 0, failed: 0 }; + } + nextGitHubContinuitySweepAt = now().getTime() + 60 * 60_000; + const cutoff = now().getTime() - 30 * 24 * 60 * 60_000; + const rows = await db + .select({ grant: connectionGrants, connection: toolConnections }) + .from(connectionGrants) + .innerJoin( + toolConnections, + and( + eq(toolConnections.id, connectionGrants.connectionId), + eq(toolConnections.companyId, connectionGrants.companyId), + ), + ) + .where( + and( + eq(connectionGrants.status, "active"), + eq(toolConnections.status, "active"), + eq(toolConnections.enabled, true), + ), + ); + const due = rows.filter(({ grant, connection }) => { + const config = asRecord(connection.config); + const oauth = asRecord(config.oauth); + if ( + config.sourceTemplateKey !== "github" || + oauth.connectorProfile !== "github.code" || + !grant.providerTenant?.github + ) + return false; + const expiresAt = grant.providerTenant.oauth?.accessTokenExpiresAt; + const refreshedAt = grant.providerTenant.oauth?.refreshedAt + ? Date.parse(grant.providerTenant.oauth.refreshedAt) + : Number.NaN; + const accessCheckedAt = grant.providerTenant.github.lastAccessRefreshAt + ? Date.parse(grant.providerTenant.github.lastAccessRefreshAt) + : Number.NaN; + if (typeof expiresAt === "string") { + const expiry = Date.parse(expiresAt); + return ( + !Number.isFinite(expiry) || + expiry <= now().getTime() + 60 * 60_000 || + !Number.isFinite(refreshedAt) || + refreshedAt <= cutoff + ); + } + return !Number.isFinite(accessCheckedAt) || accessCheckedAt <= cutoff; + }); + let refreshed = 0; + let failed = 0; + for (const row of due) { + try { + await refreshManagedGitHubGrantAccess(row.connection, row.grant, { + actorType: "system", + actorId: "github-connection-continuity", + }); + refreshed += 1; + } catch { + failed += 1; + } + } + return { checked: rows.length, due: due.length, refreshed, failed }; + } + return { refreshManagedGitHubGrantAccess, sweepGitHubConnectionContinuity }; +} diff --git a/server/src/services/tool-access/identifiers.ts b/server/src/services/tool-access/identifiers.ts index 33f8bfcc4d1..1b61515eeba 100644 --- a/server/src/services/tool-access/identifiers.ts +++ b/server/src/services/tool-access/identifiers.ts @@ -1,3 +1,4 @@ +import type { toolConnections } from "@paperclipai/db"; export function normalizeKey(input: string) { @@ -14,3 +15,13 @@ export function normalizeKey(input: string) { export function connectionUid(namespace: string, name: string, connectionId: string) { return `${normalizeKey(namespace)}/${normalizeKey(name)}-${connectionId.slice(0, 8)}`; } + +export function connectionSetupUrl(connection: typeof toolConnections.$inferSelect) { + return `/apps/${connection.id}/permissions`; +} + +export function connectionReconnectUrl( + connection: typeof toolConnections.$inferSelect, +) { + return `/apps/${connection.id}/permissions`; +} diff --git a/server/src/services/tool-access/oauth/client-registration.ts b/server/src/services/tool-access/oauth/client-registration.ts new file mode 100644 index 00000000000..50841892b10 --- /dev/null +++ b/server/src/services/tool-access/oauth/client-registration.ts @@ -0,0 +1,553 @@ +import type { ActorInfo } from "../types.js"; +import { createOAuthClientProtocol } from "./client.js"; +import { oauthConfig, sameOAuthIssuer } from "./metadata.js"; +import { resolveOAuthClientIdMetadataDocumentUrl } from "./client-metadata.js"; +import type { OAuthProviderEndpoints, OAuthClientRegistrationSource } from "./types.js"; +import { connectionMethodForConnection } from "../connection-methods.js"; +import { and, eq } from "drizzle-orm"; +import type { Db } from "@paperclipai/db"; +import { toolConnections } from "@paperclipai/db"; +import type { AppDefinition } from "@paperclipai/shared"; +import { notFound, unprocessable } from "../../../errors.js"; +import { type RemoteHttpEndpointLookup } from "../../remote-http-endpoint-guard.js"; + +import type { OAuthCredentialStore } from "./credential-store.js"; +import { singleFlight } from "../single-flight.js"; +import { configuredOAuthClientForConnection, oauthSecretRef } from "./client-settings.js"; +const oauthRegistrationFlights = new Map>(); + +type RegistrationDependencies = { + now: () => Date; + options: { oauthClientMetadataLookup?: RemoteHttpEndpointLookup }; + store: Pick; + getConnectionRow(idOrUid: string, companyId?: string): Promise; + requestOAuthClientRegistration: ReturnType["requestOAuthClientRegistration"]; +}; + +export function createOAuthClientRegistration(db: Db, { now, options, store, getConnectionRow, requestOAuthClientRegistration }: RegistrationDependencies) { + const { createOrRotateOAuthSecret, oauthClientForConnection, syncCredentialBindings } = store; + async function registerOAuthClient(input: { + connection: typeof toolConnections.$inferSelect; + endpoints: OAuthProviderEndpoints; + redirectUri: string; + actor?: ActorInfo; + }) { + const { clientId, clientSecret, tokenEndpointAuthMethod, clientIdIssuedAt, clientSecretExpiresAt } = await requestOAuthClientRegistration({ + endpoints: input.endpoints, + redirectUri: input.redirectUri, + }); + const existingClientSecretRef = oauthSecretRef( + input.connection, + "oauth.client_secret", + ); + const nextCredentialSecretRefs = + input.connection.credentialSecretRefs.filter( + (ref) => ref.configPath !== "oauth.client_secret", + ); + if (clientSecret) { + const clientSecretRef = await createOrRotateOAuthSecret({ + companyId: input.connection.companyId, + connection: input.connection, + configPath: "oauth.client_secret", + label: "OAuth client secret", + value: clientSecret, + actor: input.actor, + }); + nextCredentialSecretRefs.push(clientSecretRef); + } else if ( + existingClientSecretRef && + oauthConfig(input.connection).clientId === clientId + ) { + nextCredentialSecretRefs.push(existingClientSecretRef); + } + + const oauth = oauthConfig(input.connection); + const nextConfig = { + ...input.connection.config, + oauth: { + ...oauth, + provider: input.endpoints.provider, + authorizationUrl: input.endpoints.authorizationUrl, + tokenUrl: input.endpoints.tokenUrl, + registrationUrl: input.endpoints.registrationUrl, + metadataUrl: input.endpoints.metadataUrl ?? null, + scopes: input.endpoints.scopes, + offlineAccessSupported: input.endpoints.offlineAccessSupported, + grantTypesSupported: input.endpoints.grantTypesSupported ?? [], + codeChallengeMethodsSupported: + input.endpoints.codeChallengeMethodsSupported ?? [], + tokenEndpointAuthMethodsSupported: + input.endpoints.tokenEndpointAuthMethodsSupported ?? [], + issuer: input.endpoints.issuer ?? oauth.issuer ?? null, + resource: input.endpoints.resource ?? oauth.resource ?? null, + clientId, + clientRegistrationSource: "dcr" satisfies OAuthClientRegistrationSource, + clientTokenEndpointAuthMethod: tokenEndpointAuthMethod, + clientRedirectUri: input.redirectUri, + // Registered client material is only valid for the issuer/resource pair + // it was minted against. `assertOAuthClientBinding` re-registers when any + // of these move, so a re-pointed endpoint can never silently reuse a + // client another authorization server issued. + clientIssuer: input.endpoints.issuer ?? null, + clientResource: input.endpoints.resource ?? null, + clientCompanyId: input.connection.companyId, + clientIdIssuedAt, + clientSecretExpiresAt, + }, + }; + const [updated] = await db + .update(toolConnections) + .set({ + ownership: "dcr", + config: nextConfig, + transportConfig: nextConfig, + credentialSecretRefs: nextCredentialSecretRefs, + updatedAt: now(), + }) + .where( + and( + eq(toolConnections.id, input.connection.id), + eq(toolConnections.companyId, input.connection.companyId), + ), + ) + .returning(); + if (!updated) throw notFound("Tool connection not found"); + await syncCredentialBindings(updated); + return updated; + } + + /** + * Adopt a Client ID Metadata Document as this connection's client: the + * `client_id` *is* the https URL of Paperclip's published client metadata, so + * there is nothing to register with the authorization server. Still recorded on + * the connection so the issuer/resource/callback binding is enforced on reuse + * exactly like a dynamically registered client. + */ +async function adoptClientIdMetadataDocument(input: { + connection: typeof toolConnections.$inferSelect; + endpoints: OAuthProviderEndpoints; + redirectUri: string; + clientId: string; + }) { + const oauth = oauthConfig(input.connection); + const nextConfig = { + ...input.connection.config, + oauth: { + ...oauth, + provider: input.endpoints.provider, + authorizationUrl: input.endpoints.authorizationUrl, + tokenUrl: input.endpoints.tokenUrl, + registrationUrl: input.endpoints.registrationUrl ?? null, + metadataUrl: input.endpoints.metadataUrl ?? null, + scopes: input.endpoints.scopes, + offlineAccessSupported: input.endpoints.offlineAccessSupported, + grantTypesSupported: input.endpoints.grantTypesSupported ?? [], + codeChallengeMethodsSupported: + input.endpoints.codeChallengeMethodsSupported ?? [], + tokenEndpointAuthMethodsSupported: + input.endpoints.tokenEndpointAuthMethodsSupported ?? [], + clientIdMetadataDocumentSupported: true, + issuer: input.endpoints.issuer ?? oauth.issuer ?? null, + resource: input.endpoints.resource ?? oauth.resource ?? null, + clientId: input.clientId, + clientRegistrationSource: + "cimd" satisfies OAuthClientRegistrationSource, + clientTokenEndpointAuthMethod: "none", + clientRedirectUri: input.redirectUri, + clientIssuer: input.endpoints.issuer ?? null, + clientResource: input.endpoints.resource ?? null, + clientCompanyId: input.connection.companyId, + }, + }; + // A CIMD client has no secret. Drop any leftover one so a stale credential + // from an earlier registration can never be replayed against a new issuer. + const nextCredentialSecretRefs = + input.connection.credentialSecretRefs.filter( + (ref) => ref.configPath !== "oauth.client_secret", + ); + const [updated] = await db + .update(toolConnections) + .set({ + authKind: "oauth", + config: nextConfig, + transportConfig: nextConfig, + credentialSecretRefs: nextCredentialSecretRefs, + updatedAt: now(), + }) + .where( + and( + eq(toolConnections.id, input.connection.id), + eq(toolConnections.companyId, input.connection.companyId), + ), + ) + .returning(); + if (!updated) throw notFound("Tool connection not found"); + await syncCredentialBindings(updated); + return updated; + } + + /** + * How the client already stored on this connection was obtained. Anything + * unrecognised (including connections written before this field existed) reads + * as `manual`, which is the conservative answer: Paperclip will not silently + * re-register over client material it cannot prove it minted. + */ +function storedOAuthClientRegistrationSource( + connection: typeof toolConnections.$inferSelect, + ): OAuthClientRegistrationSource { + const source = oauthConfig(connection).clientRegistrationSource; + return source === "cimd" || source === "dcr" || source === "preconfigured" + ? source + : "manual"; + } + + /** + * Is the client material already stored on this connection still valid for the + * authorization server, MCP resource, callback URI and company we are about to + * use it with? Client credentials are minted against exactly one such tuple; + * reusing them across a moved binding would let a re-pointed endpoint borrow + * another server's registration. + */ +function reviewedOAuthClientBinding(connection: typeof toolConnections.$inferSelect, endpoints: OAuthProviderEndpoints) { + const oauth = oauthConfig(connection); + // Only repair the known Asana v1 discovery mistake. Retain company and + // callback checks, and never carry a secret to an arbitrary new issuer. + if (connection.config.sourceTemplateKey === "asana" + && connection.config.connectionMethodKey === "mcp-own-oauth" + && connection.config.url === "https://mcp.asana.com/v2/mcp" + && oauth.clientRegistrationSource === "manual" + && connection.ownership !== "dcr" + && endpoints.issuer === "https://app.asana.com" + && endpoints.authorizationUrl === "https://app.asana.com/-/oauth_authorize" + && endpoints.tokenUrl === "https://app.asana.com/-/oauth_token" + && endpoints.resource === "https://mcp.asana.com/v2/mcp" + && [undefined, null, "https://mcp.asana.com", "https://app.asana.com"].includes(oauth.clientIssuer as string | null | undefined) + && [undefined, null, "https://mcp.asana.com", "https://mcp.asana.com/v2/mcp"].includes(oauth.clientResource as string | null | undefined) + && (oauth.clientIssuer === "https://mcp.asana.com" || oauth.clientResource === "https://mcp.asana.com")) { + return { ...oauth, clientIssuer: endpoints.issuer, clientResource: endpoints.resource }; + } + return oauth; + } + + function oauthClientBindingMatches( + connection: typeof toolConnections.$inferSelect, + endpoints: OAuthProviderEndpoints, + redirectUri: string, + clientIdMetadataDocumentUrl: string | null, + ): boolean { + const oauth = reviewedOAuthClientBinding(connection, endpoints); + if (typeof oauth.clientId !== "string" || !oauth.clientId.trim()) + return false; + const source = + typeof oauth.clientRegistrationSource === "string" + ? oauth.clientRegistrationSource + : null; + // Older interrupted setup flows could accidentally round-trip a DCR client + // through the customer-client form and relabel it `manual`. Ownership is the + // durable proof that Paperclip minted that client. Force a fresh registration + // instead of preserving the damaged binding forever. + if (source === "manual" && connection.ownership === "dcr") return false; + // A URL client id that now resolves only to a private network is unusable by + // an external authorization server. Treat the stored binding as stale so a + // retry can replace it with a dynamically registered client. + if (source === "cimd" && oauth.clientId !== clientIdMetadataDocumentUrl) + return false; + // A manually preregistered client was registered by the operator against + // Paperclip's callback, so it has no recorded callback until first use. + const redirectMatches = + source === "manual" + ? oauth.clientRedirectUri === undefined || + oauth.clientRedirectUri === null || + oauth.clientRedirectUri === redirectUri + : oauth.clientRedirectUri === redirectUri; + if (!redirectMatches) return false; + if ( + typeof oauth.clientCompanyId === "string" && + oauth.clientCompanyId !== connection.companyId + ) + return false; + if ( + endpoints.issuer && + typeof oauth.clientIssuer === "string" && + oauth.clientIssuer && + !sameOAuthIssuer(oauth.clientIssuer, endpoints.issuer) + ) { + return false; + } + if ( + endpoints.resource && + typeof oauth.clientResource === "string" && + oauth.clientResource && + oauth.clientResource !== endpoints.resource + ) { + return false; + } + return true; + } + + /** + * Record the issuer/resource/callback/company a client is bound to, the first + * time that client is actually used. + * + * A dynamically registered or CIMD client is stamped at registration. A client + * the operator pasted in has no binding until now — without this, its + * `clientRedirectUri` would stay unset forever and `oauthClientBindingMatches` + * would keep accepting it for *any* callback, which is exactly the drift the + * binding check exists to catch. + */ +async function stampOAuthClientBinding( + connection: typeof toolConnections.$inferSelect, + endpoints: OAuthProviderEndpoints, + redirectUri: string, + ): Promise { + const originalOAuth = oauthConfig(connection); + const oauth = reviewedOAuthClientBinding(connection, endpoints); + const nextBinding = { + clientRedirectUri: redirectUri, + clientIssuer: + typeof oauth.clientIssuer === "string" && oauth.clientIssuer + ? oauth.clientIssuer + : (endpoints.issuer ?? null), + clientResource: + typeof oauth.clientResource === "string" && oauth.clientResource + ? oauth.clientResource + : (endpoints.resource ?? null), + clientCompanyId: + typeof oauth.clientCompanyId === "string" && oauth.clientCompanyId + ? oauth.clientCompanyId + : connection.companyId, + }; + const unchanged = Object.entries(nextBinding).every( + ([key, value]) => originalOAuth[key] === value, + ); + if (unchanged) return connection; + const nextConfig = { + ...connection.config, + oauth: { ...oauth, ...nextBinding }, + }; + const [updated] = await db + .update(toolConnections) + .set({ + config: nextConfig, + transportConfig: nextConfig, + updatedAt: now(), + }) + .where( + and( + eq(toolConnections.id, connection.id), + eq(toolConnections.companyId, connection.companyId), + ), + ) + .returning(); + return updated ?? connection; + } + + /** + * May Paperclip mint client material for this connection without an operator + * pasting client credentials? + * + * A curated app opts in through its `ownershipModes`. A generic remote MCP + * connection may register once — and only once — protected-resource and + * authorization-server discovery actually produced a metadata document; an + * endpoint that merely returned a 401 does not earn a registration. + * + * A curated method pinned to customer-owned clients still earns a registration + * when the provider's own metadata advertises one. `ownershipModes` is a + * point-in-time research snapshot, and providers add dynamic registration + * without telling us; live discovery is the better evidence of the two, so a + * stale catalog entry costs the operator a console detour rather than silently + * outranking what the server just said about itself. + */ +function canRegisterOAuthClientDynamically( + connection: typeof toolConnections.$inferSelect, + endpoints: OAuthProviderEndpoints, + galleryEntry: AppDefinition | null, + ): boolean { + if (galleryEntry) { + const method = connectionMethodForConnection(galleryEntry, connection); + if (method.ownershipModes.includes("dcr")) return true; + return method.auth === "oauth" && Boolean(endpoints.registrationUrl); + } + return ( + connection.transport === "mcp_remote" && Boolean(endpoints.metadataUrl) + ); + } + + async function ensureOAuthClient(input: { + connection: typeof toolConnections.$inferSelect; + endpoints: OAuthProviderEndpoints; + redirectUri: string; + galleryEntry: AppDefinition | null; + actor?: ActorInfo; + }) { + // 1. A client the deployment preconfigured for this issuer wins outright. + const configured = configuredOAuthClientForConnection( + input.connection, + input.endpoints.provider, + ); + if (configured.clientId) { + return { + connection: input.connection, + client: configured, + source: "preconfigured" as const, + }; + } + const method = input.galleryEntry + ? connectionMethodForConnection(input.galleryEntry, input.connection) + : null; + const dcrRequired = method?.oauthClientRegistration === "dcr"; + // A reviewed provider compatibility choice may require DCR even when the + // server also advertises CIMD. Do not resolve/adopt a metadata URL for that + // method; the default for every other method remains CIMD-first. + const metadataDocumentUrl = !dcrRequired && input.endpoints + .clientIdMetadataDocumentSupported + ? await resolveOAuthClientIdMetadataDocumentUrl( + input.redirectUri, + options.oauthClientMetadataLookup, + ) + : null; + // 2. Client material already bound to this issuer/resource/callback/company. + if ( + oauthClientBindingMatches( + input.connection, + input.endpoints, + input.redirectUri, + metadataDocumentUrl, + ) + ) { + const bound = await stampOAuthClientBinding( + input.connection, + input.endpoints, + input.redirectUri, + ); + return { + connection: bound, + client: await oauthClientForConnection( + bound, + input.endpoints.provider, + input.actor, + ), + source: storedOAuthClientRegistrationSource(bound), + }; + } + const oauth = oauthConfig(input.connection); + if ( + oauth.clientRegistrationSource === "manual" && + input.connection.ownership !== "dcr" && + typeof oauth.clientId === "string" && + oauth.clientId.trim() + ) { + // Paperclip cannot re-register on the operator's behalf: the credentials + // came from a console this deployment does not control. + throw unprocessable( + "This connection's sign-in details no longer match the server it points at. Re-enter the client ID and secret to continue.", + { code: "oauth_manual_client_rebinding_required" }, + ); + } + if ( + !canRegisterOAuthClientDynamically( + input.connection, + input.endpoints, + input.galleryEntry, + ) + ) { + throw unprocessable( + `OAuth client id is not configured for ${input.endpoints.provider}`, + { + code: "oauth_client_registration_unavailable", + }, + ); + } + if (dcrRequired && !input.endpoints.registrationUrl) { + throw unprocessable( + "This provider requires dynamic client registration, but its authorization server did not advertise a registration endpoint.", + { code: "oauth_dcr_not_supported" }, + ); + } + + const key = `${input.connection.id}:${input.redirectUri}`; + return singleFlight(oauthRegistrationFlights, key, async () => { + const latest = await getConnectionRow( + input.connection.id, + input.connection.companyId, + ); + const latestConfigured = configuredOAuthClientForConnection( + latest, + input.endpoints.provider, + ); + if (latestConfigured.clientId) { + return { + connection: latest, + client: latestConfigured, + source: "preconfigured" as const, + }; + } + if ( + oauthClientBindingMatches( + latest, + input.endpoints, + input.redirectUri, + metadataDocumentUrl, + ) + ) { + const bound = await stampOAuthClientBinding( + latest, + input.endpoints, + input.redirectUri, + ); + return { + connection: bound, + client: await oauthClientForConnection( + bound, + input.endpoints.provider, + input.actor, + ), + source: storedOAuthClientRegistrationSource(bound), + }; + } + // 3. By default, Client ID Metadata Documents need no registration call, + // so prefer them over DCR when the authorization server advertises + // support. Curated DCR opt-ins leave metadataDocumentUrl null above. + if (metadataDocumentUrl) { + const adopted = await adoptClientIdMetadataDocument({ + connection: latest, + endpoints: input.endpoints, + redirectUri: input.redirectUri, + clientId: metadataDocumentUrl, + }); + return { + connection: adopted, + client: await oauthClientForConnection( + adopted, + input.endpoints.provider, + input.actor, + ), + source: "cimd" as const, + }; + } + // 4. Dynamic client registration. + if (!input.endpoints.registrationUrl) { + throw unprocessable( + "This server needs sign-in details you create yourself. Add a client ID and secret under Advanced authentication.", + { code: "oauth_manual_client_required" }, + ); + } + const registered = await registerOAuthClient({ + connection: latest, + endpoints: input.endpoints, + redirectUri: input.redirectUri, + actor: input.actor, + }); + return { + connection: registered, + client: await oauthClientForConnection( + registered, + input.endpoints.provider, + input.actor, + ), + source: "dcr" as const, + }; + }); + } + return ensureOAuthClient; +} diff --git a/server/src/services/tool-access/oauth/client-settings.ts b/server/src/services/tool-access/oauth/client-settings.ts new file mode 100644 index 00000000000..1fea19bafd7 --- /dev/null +++ b/server/src/services/tool-access/oauth/client-settings.ts @@ -0,0 +1,174 @@ +import type { ActorInfo } from "../types.js"; +import { oauthConfig, isSmokeLabOAuthFixture } from "./metadata.js"; +import { connectionMethodForConnection } from "../connection-methods.js"; +import { stableHash } from "../../tool-catalog-hash.js"; +import { ASANA_CONNECTOR_SCOPES, isAsanaConnectorProfileId, type AsanaConnectorProfileId } from "@paperclipai/shared"; +import { connectionGrants, toolConnections } from "@paperclipai/db"; +import { GITHUB_CONNECTOR_PROFILES, GOOGLE_WORKSPACE_CONNECTOR_PROFILES, getConnectableAppDefinition, isGitHubConnectorProfileId, isGoogleWorkspaceConnectorProfileId, type GitHubConnectorProfileId, type GoogleWorkspaceConnectorProfileId } from "@paperclipai/shared"; +import { unprocessable } from "../../../errors.js"; + +function asRecord(value: unknown): Record { + if (value && typeof value === "object" && !Array.isArray(value)) + return value as Record; + return {}; +} + +export function oauthEnvName( + provider: string, + suffix: "CLIENT_ID" | "CLIENT_SECRET", +) { + return `PAPERCLIP_TOOL_OAUTH_${provider.replace(/[^a-z0-9]+/gi, "_").toUpperCase()}_${suffix}`; +} + +export function oauthClientConfig(provider: string) { + const clientIdEnv = oauthEnvName(provider, "CLIENT_ID"); + const clientSecretEnv = oauthEnvName(provider, "CLIENT_SECRET"); + return { + clientIdEnv, + clientSecretEnv, + clientId: + process.env[clientIdEnv] ?? + process.env.PAPERCLIP_TOOL_OAUTH_CLIENT_ID ?? + null, + clientSecret: + process.env[clientSecretEnv] ?? + process.env.PAPERCLIP_TOOL_OAUTH_CLIENT_SECRET ?? + null, + }; +} + +export function configuredOAuthClientForConnection( + connection: typeof toolConnections.$inferSelect, + provider: string, +) { + if (isSmokeLabOAuthFixture(connection) && provider === "smoke_lab") { + return { + clientIdEnv: "SMOKE_LAB_FIXED_CLIENT_ID", + clientSecretEnv: "SMOKE_LAB_FIXED_CLIENT_SECRET", + clientId: "paperclip-smoke-lab", + clientSecret: null, + }; + } + return oauthClientConfig(provider); +} + +export function credentialScope( + connection: typeof toolConnections.$inferSelect, + actor?: ActorInfo, +) { + const configured = asRecord(oauthConfig(connection).credentialScope); + const type = + typeof configured.type === "string" + ? configured.type + : typeof configured.targetType === "string" + ? configured.targetType + : actor?.actorType === "agent" + ? "agent" + : actor?.actorType === "user" + ? "user" + : "company"; + const id = + typeof configured.id === "string" + ? configured.id + : typeof configured.targetId === "string" + ? configured.targetId + : (actor?.actorId ?? connection.companyId); + return { + type, + id, + hash: stableHash({ + companyId: connection.companyId, + connectionId: connection.id, + type, + id, + }), + }; +} + +export function oauthSecretRef( + connection: typeof toolConnections.$inferSelect, + configPath: + "oauth.access_token" | "oauth.refresh_token" | "oauth.client_secret" | "railway.ssh_private_key", +) { + return ( + connection.credentialSecretRefs.find( + (ref) => ref.configPath === configPath, + ) ?? null + ); +} + +export function oauthExpiresAtMs( + connection: typeof toolConnections.$inferSelect, +): number | null { + const expiresAt = oauthConfig(connection).expiresAt; + if (typeof expiresAt !== "string") return null; + const ms = Date.parse(expiresAt); + return Number.isFinite(ms) ? ms : null; +} + +export async function oauthGalleryEntryForConnection( + connection: typeof toolConnections.$inferSelect, +) { + const sourceTemplateKey = + typeof connection.config.sourceTemplateKey === "string" + ? connection.config.sourceTemplateKey + : null; + if (!sourceTemplateKey) + throw unprocessable( + "This app connection was not created from the app gallery", + ); + const galleryEntry = getConnectableAppDefinition(sourceTemplateKey); + if ( + !galleryEntry || + connectionMethodForConnection(galleryEntry, connection).auth !== "oauth" + ) { + throw unprocessable("This app connection does not use sign in"); + } + return galleryEntry; +} + +export type ManagedConnectorProfileId = + GoogleWorkspaceConnectorProfileId | GitHubConnectorProfileId | AsanaConnectorProfileId; + +export function managedConnectorProfile(value: string | undefined): { + id: ManagedConnectorProfileId; + provider: "google" | "github" | "asana"; + scopes: readonly string[]; +} | null { + if (value && isAsanaConnectorProfileId(value)) { + return { id: value, provider: "asana", scopes: ASANA_CONNECTOR_SCOPES }; + } + if (value && isGoogleWorkspaceConnectorProfileId(value)) { + return { + id: value, + provider: "google", + scopes: GOOGLE_WORKSPACE_CONNECTOR_PROFILES[value].scopes, + }; + } + if (value && isGitHubConnectorProfileId(value)) { + return { + id: value, + provider: "github", + scopes: GITHUB_CONNECTOR_PROFILES[value].scopes, + }; + } + return null; +} + +export function oauthGrantConfig(grant: typeof connectionGrants.$inferSelect) { + return asRecord(asRecord(grant.providerTenant).oauth); +} + +export function oauthGrantExpiresAtMs( + grant: typeof connectionGrants.$inferSelect, + connection: typeof toolConnections.$inferSelect, +): number | null { + const grantExpiresAt = oauthGrantConfig(grant).accessTokenExpiresAt; + const value = + typeof grantExpiresAt === "string" + ? grantExpiresAt + : oauthConfig(connection).expiresAt; + if (typeof value !== "string") return null; + const ms = Date.parse(value); + return Number.isFinite(ms) ? ms : null; +} diff --git a/server/src/services/tool-access/oauth/connection-refresh.ts b/server/src/services/tool-access/oauth/connection-refresh.ts new file mode 100644 index 00000000000..959d0619c00 --- /dev/null +++ b/server/src/services/tool-access/oauth/connection-refresh.ts @@ -0,0 +1,395 @@ +import type { ActorInfo } from "../types.js"; +import { createOAuthClientProtocol, storedOAuthTokenEndpointAuthMethod } from "./client.js"; +import { oauthConfig, normalizeOauthScopes } from "./metadata.js"; +import { emitConnectionUpdated } from "../../connector-telemetry.js"; +import { and, eq, sql } from "drizzle-orm"; +import type { Db } from "@paperclipai/db"; +import { companySecrets, toolConnections } from "@paperclipai/db"; +import { conflict, HttpError, unprocessable } from "../../../errors.js"; +import { secretService } from "../../secrets.js"; + +import type { OAuthCredentialStore } from "./credential-store.js"; +import { withoutOAuthRefreshLease, type OAuthRefreshLeases } from "./refresh-leases.js"; +import { oauthExpiresAtMs, oauthSecretRef } from "./client-settings.js"; +import { connectionSetupUrl, connectionReconnectUrl } from "../identifiers.js"; +import { singleFlight } from "../single-flight.js"; +function asRecord(value: unknown): Record { + if (value && typeof value === "object" && !Array.isArray(value)) + return value as Record; + return {}; +} + +type LegacyRefreshDependencies = { + now: () => Date; + secrets: ReturnType; + store: Pick; + leases: Pick; + getConnectionRow(idOrUid: string, companyId?: string): Promise; + exchangeOAuthToken: ReturnType["exchangeOAuthToken"]; +}; + +export function createLegacyOAuthRefresh(db: Db, { now, secrets, store, leases, getConnectionRow, exchangeOAuthToken }: LegacyRefreshDependencies) { + // These maps remove duplicate work inside one service instance. OAuth also +// uses the database refresh lease below as its cross-process boundary. +const oauthRefreshFlights = new Map>(); + const { createOrRotateOAuthSecret, oauthClientForConnection, syncCredentialBindings } = store; + const { acquireOAuthRefreshLease, clearOAuthRefreshLease } = leases; + async function markOAuthReauthorizationRequired( + connection: typeof toolConnections.$inferSelect, + guard: { + leaseId: string; + refreshSecretId: string; + refreshTokenVersion: number; + }, + ) { + const oauth = oauthConfig(connection); + const nextCredentialSecretRefs = connection.credentialSecretRefs.filter( + (ref) => + ref.configPath !== "oauth.access_token" && + ref.configPath !== "oauth.refresh_token", + ); + const nextCredentialRefs = connection.credentialRefs.filter( + (ref) => ref.name !== "oauth.access_token", + ); + const nextConfig = { + ...connection.config, + oauth: { + ...withoutOAuthRefreshLease(oauth), + expiresAt: null, + reauthorizationRequiredAt: now().toISOString(), + }, + }; + const [updated] = await db + .update(toolConnections) + .set({ + status: "draft", + enabled: false, + healthStatus: "error", + healthMessage: + "OAuth authorization expired. Reconnect this app to continue.", + lastError: "oauth_reauthorization_required", + config: nextConfig, + transportConfig: nextConfig, + credentialSecretRefs: nextCredentialSecretRefs, + credentialRefs: nextCredentialRefs, + updatedAt: now(), + }) + .where( + and( + eq(toolConnections.id, connection.id), + eq(toolConnections.companyId, connection.companyId), + sql`${toolConnections.config} -> 'oauth' -> 'refreshLease' ->> 'id' = ${guard.leaseId}`, + sql`exists ( + select 1 from ${companySecrets} + where ${companySecrets.id} = ${guard.refreshSecretId} + and ${companySecrets.companyId} = ${connection.companyId} + and ${companySecrets.latestVersion} = ${guard.refreshTokenVersion} + )`, + ), + ) + .returning(); + if (updated) { + await syncCredentialBindings(updated); + emitConnectionUpdated(updated, connection, "credential_refresh"); + } + return updated ?? null; + } + + async function refreshOAuthCredentials( + connection: typeof toolConnections.$inferSelect, + leaseId: string, + actor?: ActorInfo, + accessContext?: { + actorSource?: + | "local_implicit" + | "session" + | "board_key" + | "agent_key" + | "agent_jwt" + | "mcp_oauth" + | "cloud_tenant"; + issueId?: string | null; + heartbeatRunId?: string | null; + }, + ): Promise { + const oauth = oauthConfig(connection); + if ( + typeof oauth.tokenUrl !== "string" || + typeof oauth.provider !== "string" + ) + return connection; + const expiresAtMs = oauthExpiresAtMs(connection); + if (expiresAtMs && expiresAtMs > Date.now() + 60_000) return connection; + const grantType = + oauth.grantType === "client_credentials" || + oauth.clientCredentials === true + ? ("client_credentials" as const) + : ("refresh_token" as const); + const refreshRef = oauthSecretRef(connection, "oauth.refresh_token"); + if (grantType !== "client_credentials" && !refreshRef) { + throw new HttpError( + 422, + "OAuth credentials have expired and no refresh token is available", + { + code: "oauth_refresh_missing", + setupUrl: connectionSetupUrl(connection), + reconnectUrl: connectionReconnectUrl(connection), + }, + ); + } + const client = await oauthClientForConnection( + connection, + oauth.provider, + actor, + ); + if (!client.clientId) + throw unprocessable( + `OAuth client id is not configured for ${oauth.provider}`, + ); + const [refreshSecret] = refreshRef + ? await db + .select({ latestVersion: companySecrets.latestVersion }) + .from(companySecrets) + .where( + and( + eq(companySecrets.id, refreshRef.secretId), + eq(companySecrets.companyId, connection.companyId), + ), + ) + .limit(1) + : [undefined]; + const refreshTokenVersion = refreshSecret?.latestVersion ?? null; + const refreshToken = + refreshRef && refreshTokenVersion !== null + ? await secrets.resolveSecretValue( + connection.companyId, + refreshRef.secretId, + refreshTokenVersion, + { + consumerType: "tool_connection", + consumerId: connection.id, + configPath: "oauth.refresh_token", + actorType: actor?.actorType ?? "system", + actorId: actor?.actorId ?? null, + actorSource: accessContext?.actorSource, + issueId: accessContext?.issueId, + heartbeatRunId: accessContext?.heartbeatRunId, + }, + ) + : null; + let token: Awaited>; + try { + token = await exchangeOAuthToken({ + tokenUrl: oauth.tokenUrl, + clientId: client.clientId, + clientSecret: client.clientSecret, + tokenEndpointAuthMethod: storedOAuthTokenEndpointAuthMethod( + oauth, + client.clientSecret, + ), + grantType, + scopes: + normalizeOauthScopes(oauth.scopes).length > 0 + ? normalizeOauthScopes(oauth.scopes) + : normalizeOauthScopes(oauth.scope), + refreshToken, + // Refreshing must stay bound to the same MCP server the original grant + // named, or the authorization server may widen the token's audience. + resource: + typeof oauth.resource === "string" && oauth.resource + ? oauth.resource + : null, + }); + } catch (error) { + if ( + error instanceof HttpError && + asRecord(error.details).code === "oauth_reauthorization_required" + ) { + const marked = + refreshRef && refreshTokenVersion !== null + ? await markOAuthReauthorizationRequired(connection, { + leaseId, + refreshSecretId: refreshRef.secretId, + refreshTokenVersion, + }) + : null; + if (!marked) { + const latest = await getConnectionRow( + connection.id, + connection.companyId, + ); + const latestExpiresAtMs = oauthExpiresAtMs(latest); + if (latestExpiresAtMs && latestExpiresAtMs > Date.now() + 60_000) + return latest; + throw conflict( + "OAuth credentials changed while refresh was in progress. Retry the request.", + { + code: "oauth_refresh_superseded", + retryable: true, + }, + ); + } + throw new HttpError(error.status, error.message, { + ...asRecord(error.details), + setupUrl: connectionSetupUrl(connection), + reconnectUrl: connectionReconnectUrl(connection), + }); + } + throw error; + } + // Rotating providers invalidate the submitted refresh token immediately. + // Persist its replacement before the new access token can be returned to a + // caller, so a crash cannot leave the grant with only the consumed token. + let nextRefreshRef: Awaited< + ReturnType + > | null = null; + if (token.refreshToken) { + nextRefreshRef = await createOrRotateOAuthSecret({ + companyId: connection.companyId, + connection, + configPath: "oauth.refresh_token", + label: "OAuth refresh token", + value: token.refreshToken, + actor, + }); + } + const accessRef = await createOrRotateOAuthSecret({ + companyId: connection.companyId, + connection, + configPath: "oauth.access_token", + label: "OAuth access token", + value: token.accessToken, + actor, + }); + const nextCredentialSecretRefs = [ + ...connection.credentialSecretRefs.filter( + (ref) => + ref.configPath !== "oauth.access_token" && + (!nextRefreshRef || ref.configPath !== "oauth.refresh_token"), + ), + accessRef, + ...(nextRefreshRef ? [nextRefreshRef] : []), + ]; + const expiresAt = token.expiresIn + ? new Date(Date.now() + token.expiresIn * 1000).toISOString() + : null; + const nextConfig = { + ...connection.config, + oauth: { + ...withoutOAuthRefreshLease(oauth), + grantType: + grantType === "client_credentials" + ? grantType + : (oauth.grantType ?? "authorization_code"), + expiresAt, + scope: token.scope ?? oauth.scope ?? null, + tokenType: token.tokenType, + refreshedAt: new Date().toISOString(), + }, + providerMetadata: { + ...asRecord(connection.config.providerMetadata), + oauth: { + expiresAt, + scope: token.scope ?? oauth.scope ?? null, + tokenType: token.tokenType, + }, + }, + }; + const [updated] = await db + .update(toolConnections) + .set({ + config: nextConfig, + transportConfig: nextConfig, + credentialSecretRefs: nextCredentialSecretRefs, + credentialRefs: [ + ...connection.credentialRefs.filter( + (ref) => ref.name !== "oauth.access_token", + ), + { + name: "oauth.access_token", + secretId: accessRef.secretId, + version: "latest" as const, + placement: "header" as const, + key: "Authorization", + prefix: "Bearer ", + }, + ], + updatedAt: new Date(), + }) + .where( + and( + eq(toolConnections.id, connection.id), + eq(toolConnections.companyId, connection.companyId), + sql`${toolConnections.config} -> 'oauth' -> 'refreshLease' ->> 'id' = ${leaseId}`, + ), + ) + .returning(); + if (!updated) { + throw conflict( + "OAuth credentials changed while refresh was in progress. Retry the request.", + { + code: "oauth_refresh_superseded", + retryable: true, + }, + ); + } + const previousBindingKeys = new Set( + connection.credentialSecretRefs.map( + (ref) => `${ref.secretId}:${ref.configPath}`, + ), + ); + const nextBindingKeys = new Set( + nextCredentialSecretRefs.map( + (ref) => `${ref.secretId}:${ref.configPath}`, + ), + ); + const bindingsChanged = + previousBindingKeys.size !== nextBindingKeys.size || + [...previousBindingKeys].some((key) => !nextBindingKeys.has(key)); + if (bindingsChanged) await syncCredentialBindings(updated); + return updated; + } + + async function maybeRefreshOAuthCredentials( + connection: typeof toolConnections.$inferSelect, + actor?: ActorInfo, + accessContext?: { + actorSource?: + | "local_implicit" + | "session" + | "board_key" + | "agent_key" + | "agent_jwt" + | "mcp_oauth" + | "cloud_tenant"; + issueId?: string | null; + heartbeatRunId?: string | null; + }, + ): Promise { + const oauth = oauthConfig(connection); + if ( + typeof oauth.tokenUrl !== "string" || + typeof oauth.provider !== "string" + ) + return connection; + const expiresAtMs = oauthExpiresAtMs(connection); + if (expiresAtMs && expiresAtMs > Date.now() + 60_000) return connection; + return singleFlight(oauthRefreshFlights, connection.id, async () => { + const lease = await acquireOAuthRefreshLease(connection); + if (!lease.leaseId) return lease.connection; + try { + return await refreshOAuthCredentials( + lease.connection, + lease.leaseId, + actor, + accessContext, + ); + } finally { + await clearOAuthRefreshLease(lease.connection, lease.leaseId).catch( + () => undefined, + ); + } + }); + } + return maybeRefreshOAuthCredentials; +} diff --git a/server/src/services/tool-access/oauth/credential-store.ts b/server/src/services/tool-access/oauth/credential-store.ts new file mode 100644 index 00000000000..9e786c891f9 --- /dev/null +++ b/server/src/services/tool-access/oauth/credential-store.ts @@ -0,0 +1,205 @@ +import type { ActorInfo, DbTransaction, ToolAccessMutationDb } from "../types.js"; +import { oauthConfig } from "./metadata.js"; +import { syncConnectionCredentialBindings } from "../../connection-credential-bindings.js"; +import { and, eq, inArray } from "drizzle-orm"; +import type { Db } from "@paperclipai/db"; +import { connectionGrants, toolConnections } from "@paperclipai/db"; +import type { ToolConnection, ToolCredentialSecretRef } from "@paperclipai/shared"; +import { notFound } from "../../../errors.js"; +import { secretService } from "../../secrets.js"; +import { resolveConnectionGrantSecret, writeConnectionCredential } from "../../connection-credentials.js"; + +import { configuredOAuthClientForConnection, oauthSecretRef } from "./client-settings.js"; +export function actorForSecret( + actor?: ActorInfo, +): { userId?: string | null; agentId?: string | null } | undefined { + if (actor?.actorType === "user") return { userId: actor.actorId ?? null }; + if (actor?.actorType === "agent") return { agentId: actor.actorId ?? null }; + return undefined; +} + +export function createOAuthCredentialStore(db: Db, secrets: ReturnType) { + /** + * @param grantSecretRefs Secret refs held by a grant rather than the connection + * row. A personal credential lives only on its user grant (PAP-17835), so it + * would otherwise have no `company_secret_bindings` row and drop out of + * secret projection and removal teardown. + */ +async function syncCredentialBindings( + connection: typeof toolConnections.$inferSelect, + grantSecretRefs: ToolCredentialSecretRef[] = [], + dbClient: ToolAccessMutationDb = db, + ) { + await syncConnectionCredentialBindings(db, connection, grantSecretRefs, dbClient); + } + + async function annotateSavedOAuthClientSecret(connections: ToolConnection[], viewerUserId?: string) { + const personal = connections.filter((connection) => connection.authKind === "oauth" && connection.credentialPolicy === "per_user"); + const grants = viewerUserId && personal.length > 0 + ? await db.select({ connectionId: connectionGrants.connectionId, refs: connectionGrants.credentialSecretRefs }) + .from(connectionGrants) + .where(and( + inArray(connectionGrants.companyId, [...new Set(personal.map((connection) => connection.companyId))]), + inArray(connectionGrants.connectionId, personal.map((connection) => connection.id)), + eq(connectionGrants.kind, "user"), + eq(connectionGrants.subjectUserId, viewerUserId), + eq(connectionGrants.status, "active"), + )) + : []; + const savedPersonal = new Set(grants.filter((grant) => grant.refs.some((ref) => ref.configPath === "oauth.client_secret")).map((grant) => grant.connectionId)); + for (const connection of connections) { + connection.hasSavedOAuthClientSecret = connection.credentialPolicy === "per_user" + ? savedPersonal.has(connection.id) + : connection.credentialSecretRefs.some((ref) => ref.configPath === "oauth.client_secret"); + } + } + + async function oauthClientForConnection( + connection: typeof toolConnections.$inferSelect, + provider: string, + actor?: ActorInfo, + ) { + const configured = configuredOAuthClientForConnection(connection, provider); + if (configured.clientId) return configured; + const oauth = oauthConfig(connection); + const clientId = + typeof oauth.clientId === "string" && oauth.clientId.trim() + ? oauth.clientId.trim() + : null; + if (!clientId) return configured; + // CIMD clients and public DCR clients have no token-endpoint secret. A DCR + // authorization server may instead issue a confidential client (for + // example, `client_secret_basic`); in that case the registration secret is + // encrypted like every other provider credential and resolved only for the + // token endpoint. + const registrationSource = oauth.clientRegistrationSource; + const publicRegisteredClient = + registrationSource === "cimd" || + (registrationSource === "dcr" && + oauth.clientTokenEndpointAuthMethod === "none"); + let credentialSecretRefs = connection.credentialSecretRefs; + if ( + connection.credentialPolicy === "per_user" && + actor?.actorType === "user" && + actor.actorId && + !credentialSecretRefs.some( + (ref) => ref.configPath === "oauth.client_secret", + ) + ) { + const [personalGrant] = await db + .select({ + credentialSecretRefs: connectionGrants.credentialSecretRefs, + }) + .from(connectionGrants) + .where( + and( + eq(connectionGrants.companyId, connection.companyId), + eq(connectionGrants.connectionId, connection.id), + eq(connectionGrants.kind, "user"), + eq(connectionGrants.subjectUserId, actor.actorId), + eq(connectionGrants.status, "active"), + ), + ) + .limit(1); + credentialSecretRefs = + personalGrant?.credentialSecretRefs ?? credentialSecretRefs; + } + const clientSecretRef = publicRegisteredClient + ? undefined + : credentialSecretRefs.find( + (ref) => ref.configPath === "oauth.client_secret", + ); + const clientSecret = clientSecretRef + ? await secrets.resolveSecretValue( + connection.companyId, + clientSecretRef.secretId, + clientSecretRef.versionSelector ?? "latest", + { + consumerType: "tool_connection", + consumerId: connection.id, + configPath: "oauth.client_secret", + actorType: actor?.actorType ?? "system", + actorId: actor?.actorId ?? null, + }, + ) + : null; + return { + clientIdEnv: null, + clientSecretEnv: null, + clientId, + clientSecret, + }; + } + + async function createOrRotateOAuthSecret( + input: { + companyId: string; + connection: typeof toolConnections.$inferSelect; + configPath: + "oauth.access_token" | "oauth.refresh_token" | "oauth.client_secret" | "railway.ssh_private_key"; + label: string; + value: string; + actor?: ActorInfo; + existingRefs?: typeof connectionGrants.$inferSelect.credentialSecretRefs; + ownerUserId?: string; + }, + context?: { + dbClient: Db | DbTransaction; + secretClient: ReturnType; + }, + ) { + const existing = input.existingRefs === undefined + ? oauthSecretRef(input.connection, input.configPath) + : input.existingRefs.find((ref) => ref.configPath === input.configPath); + const write = (client: Db | DbTransaction) => writeConnectionCredential(client, { + companyId: input.companyId, connectionName: input.connection.name, configPath: input.configPath, + label: input.label, value: input.value, existingRef: existing ?? undefined, actor: actorForSecret(input.actor), + // Registration authenticates the OAuth client, never the person invoking tools. + ownerUserId: input.configPath === "oauth.client_secret" ? undefined : input.ownerUserId, + definitionKey: input.ownerUserId ? `tool_oauth.${input.connection.id}.${input.configPath.replace(/[^a-z0-9_:-]+/gi, "_")}` : undefined, + }); + const { secret } = context ? await write(context.dbClient) : await db.transaction(write); + return { + secretId: secret.id, versionSelector: "latest" as const, configPath: input.configPath, + required: input.configPath === "oauth.access_token", label: input.label, + }; + } + + async function getOAuthGrantRow(input: { + companyId: string; + connectionId: string; + grantId: string; + }) { + const [grant] = await db + .select() + .from(connectionGrants) + .where( + and( + eq(connectionGrants.id, input.grantId), + eq(connectionGrants.companyId, input.companyId), + eq(connectionGrants.connectionId, input.connectionId), + ), + ) + .limit(1); + if (!grant) throw notFound("Connection authorization not found"); + return grant; + } + + async function resolveOAuthGrantSecret( + connection: typeof toolConnections.$inferSelect, + grant: typeof connectionGrants.$inferSelect, + ref: ToolCredentialSecretRef, + actor: ActorInfo | undefined, + accessContext: + { issueId?: string | null; heartbeatRunId?: string | null } | undefined, + ) { + return resolveConnectionGrantSecret(db, connection, grant, ref, { + consumerType: "tool_connection", consumerId: connection.id, + actorType: actor?.actorType ?? "system", actorId: actor?.actorId ?? null, + issueId: accessContext?.issueId, heartbeatRunId: accessContext?.heartbeatRunId, + }); + } + return { syncCredentialBindings, annotateSavedOAuthClientSecret, oauthClientForConnection, createOrRotateOAuthSecret, getOAuthGrantRow, resolveOAuthGrantSecret }; +} + +export type OAuthCredentialStore = ReturnType; diff --git a/server/src/services/tool-access/oauth/grant-refresh.ts b/server/src/services/tool-access/oauth/grant-refresh.ts new file mode 100644 index 00000000000..4a3748f6d83 --- /dev/null +++ b/server/src/services/tool-access/oauth/grant-refresh.ts @@ -0,0 +1,650 @@ +import type { ActorInfo } from "../types.js"; +import { createOAuthClientProtocol, storedOAuthTokenEndpointAuthMethod } from "./client.js"; +import { oauthConfig, normalizeOauthScopes, resolveGrantedOauthScopes } from "./metadata.js"; +import { googleWorkspaceEffectiveScopes, isGoogleWorkspaceScopeGrant } from "@paperclipai/shared"; +import { emitConnectionUpdated } from "../../connector-telemetry.js"; +import { and, eq, sql } from "drizzle-orm"; +import type { Db } from "@paperclipai/db"; +import { connectionGrants, companySecrets, toolConnections } from "@paperclipai/db"; +import type { ToolCredentialSecretRef } from "@paperclipai/shared"; +import { conflict, HttpError, unprocessable } from "../../../errors.js"; +import { secretService } from "../../secrets.js"; +import { isPaperclipCloudConnectorStrategy, PaperclipCloudConnectorError, type PaperclipCloudConnector } from "../../paperclip-cloud-connector.js"; + +import { actorForSecret, type OAuthCredentialStore } from "./credential-store.js"; +import { withoutOAuthRefreshLease, withoutOAuthGrantRefreshLease, type OAuthRefreshLeases } from "./refresh-leases.js"; +import { managedConnectorProfile, oauthGrantConfig, oauthGrantExpiresAtMs } from "./client-settings.js"; +import { connectionSetupUrl, connectionReconnectUrl } from "../identifiers.js"; +import { singleFlight } from "../single-flight.js"; +function asRecord(value: unknown): Record { + if (value && typeof value === "object" && !Array.isArray(value)) + return value as Record; + return {}; +} + +type GrantRefreshDependencies = { + now: () => Date; + store: Pick; + leases: Pick; + currentCloudConnector: () => PaperclipCloudConnector | null; + getConnectionRow(idOrUid: string, companyId?: string): Promise; + exchangeOAuthToken: ReturnType["exchangeOAuthToken"]; +}; + +export function createOAuthGrantRefresh(db: Db, { now, store, leases, currentCloudConnector, getConnectionRow, exchangeOAuthToken }: GrantRefreshDependencies) { + const oauthGrantRefreshFlights = new Map>(); + const { getOAuthGrantRow, resolveOAuthGrantSecret, createOrRotateOAuthSecret, oauthClientForConnection, syncCredentialBindings } = store; + const { acquireOAuthGrantRefreshLease, clearOAuthGrantRefreshLease } = leases; + async function refreshOAuthGrantCredentials(input: { + companyId: string; + connectionId: string; + grantId: string; + forceRefresh?: boolean; + actor?: ActorInfo; + issueId?: string | null; + heartbeatRunId?: string | null; + }): Promise { + const connection = await getConnectionRow( + input.connectionId, + input.companyId, + ); + const initialGrant = await getOAuthGrantRow(input); + const oauth = oauthConfig(connection); + const oauthProvider = + typeof oauth.provider === "string" ? oauth.provider : null; + const oauthTokenUrl = + typeof oauth.tokenUrl === "string" ? oauth.tokenUrl : null; + if ( + connection.authKind === "oauth" && + connection.credentialSource === "paperclip_vault" && + isPaperclipCloudConnectorStrategy(oauth.strategy) + ) { + const grantOauth = oauthGrantConfig(initialGrant); + const expiresAt = + typeof grantOauth.accessTokenExpiresAt === "string" + ? Date.parse(grantOauth.accessTokenExpiresAt) + : Number.NaN; + const refreshedAt = + typeof grantOauth.refreshedAt === "string" + ? Date.parse(grantOauth.refreshedAt) + : Number.NaN; + const rotationDue = + Number.isFinite(expiresAt) && + (!Number.isFinite(refreshedAt) || + refreshedAt <= Date.now() - 30 * 24 * 60 * 60_000); + const refreshDue = + Number.isFinite(expiresAt) && expiresAt <= Date.now() + 60 * 60_000; + // A GitHub App can deliberately issue a non-expiring ghu_ token. Its + // continuity is checked against /user below; only an expiring token pair + // enters this rotation path. + if (!input.forceRefresh && !refreshDue && !rotationDue) + return initialGrant; + if (!Number.isFinite(expiresAt)) return initialGrant; + + return singleFlight( + oauthGrantRefreshFlights, + initialGrant.id, + async () => { + const lease = await acquireOAuthGrantRefreshLease( + connection, + initialGrant, + true, + ); + if (!lease.leaseId) return lease.grant; + try { + const grant = lease.grant; + const profile = managedConnectorProfile( + typeof oauth.connectorProfile === "string" + ? oauth.connectorProfile + : undefined, + ); + const cloudConnector = currentCloudConnector(); + const connectorSubject = + grant.kind === "agent" && grant.subjectAgentId + ? `agent:${grant.subjectAgentId}` + : grant.kind === "user" && grant.subjectUserId + ? grant.subjectUserId + : typeof oauth.connectorSubjectUserId === "string" + ? oauth.connectorSubjectUserId + : null; + const accessRef = grant.credentialSecretRefs.find( + (ref) => ref.configPath === "oauth.access_token", + ); + const refreshRef = grant.credentialSecretRefs.find( + (ref) => ref.configPath === "oauth.refresh_token", + ); + if ( + !profile || + !cloudConnector || + !connectorSubject || + !accessRef || + !refreshRef + ) { + throw unprocessable("Managed authorization must be reconnected", { + code: "oauth_reauthorization_required", + setupUrl: connectionSetupUrl(connection), + reconnectUrl: connectionReconnectUrl(connection), + }); + } + const refreshSecret = await resolveOAuthGrantSecret( + connection, + grant, + refreshRef, + input.actor, + input, + ); + let credentials; + try { + credentials = await cloudConnector.refresh({ + subject: connectorSubject, + companyId: connection.companyId, + profile: profile.id, + refreshToken: refreshSecret.value, + }); + if (profile.id === "workspace.all") { + const requested = normalizeOauthScopes(oauthGrantConfig(grant).requestedScopes ?? oauthGrantConfig(grant).scopes); + credentials.scopes = googleWorkspaceEffectiveScopes(requested, credentials.scopes); + } + if (profile.id === "workspace.all" && !isGoogleWorkspaceScopeGrant(credentials.scopes)) { + throw new PaperclipCloudConnectorError("Google did not return verifiable permissions", "REAUTHORIZATION_REQUIRED"); + } + } catch (error) { + if ( + error instanceof PaperclipCloudConnectorError && + error.code === "REAUTHORIZATION_REQUIRED" + ) { + await db + .update(connectionGrants) + .set({ status: "needs_reauthorization", updatedAt: now() }) + .where( + and( + eq(connectionGrants.id, grant.id), + eq(connectionGrants.companyId, grant.companyId), + ), + ); + throw unprocessable( + "Managed authorization must be reconnected", + { + code: "oauth_reauthorization_required", + setupUrl: connectionSetupUrl(connection), + reconnectUrl: connectionReconnectUrl(connection), + }, + ); + } + throw error; + } + const credentialActor: ActorInfo | undefined = + grant.kind === "user" && grant.subjectUserId + ? { actorType: "user", actorId: grant.subjectUserId } + : input.actor; + const providerTenant = { + ...(grant.providerTenant ?? {}), + oauth: { + ...withoutOAuthGrantRefreshLease(oauthGrantConfig(grant)), + strategy: "paperclip_cloud_connector", + accessTokenExpiresAt: credentials.accessTokenExpiresAt, + scopes: credentials.scopes, + tokenType: credentials.tokenType, + refreshedAt: now().toISOString(), + ...(credentials.refreshTokenExpiresAt + ? { refreshTokenExpiresAt: credentials.refreshTokenExpiresAt } + : {}), + }, + }; + const updated = await db.transaction(async (tx) => { + const [current] = await tx.select({ id: connectionGrants.id }).from(connectionGrants) + .where(and(eq(connectionGrants.id, grant.id), eq(connectionGrants.companyId, grant.companyId), + eq(connectionGrants.status, "active"), + sql`${connectionGrants.providerTenant} -> 'oauth' -> 'refreshLease' ->> 'id' = ${lease.leaseId}`)) + .for("update"); + if (!current) throw conflict("OAuth credentials changed while refresh was in progress", { + code: "oauth_refresh_superseded", retryable: true, + }); + const txSecrets = secretService(tx); + await txSecrets.rotate( + accessRef.secretId, + { value: credentials.accessToken }, + actorForSecret(credentialActor), + ); + if (credentials.refreshToken) { + await txSecrets.rotate( + refreshRef.secretId, + { value: credentials.refreshToken }, + actorForSecret(credentialActor), + ); + } + const [committed] = await tx + .update(connectionGrants) + .set({ + providerTenant, + status: "active", + updatedAt: now(), + }) + .where( + and( + eq(connectionGrants.id, grant.id), + eq(connectionGrants.companyId, grant.companyId), + eq(connectionGrants.status, "active"), + sql`${connectionGrants.providerTenant} -> 'oauth' -> 'refreshLease' ->> 'id' = ${lease.leaseId}`, + ), + ) + .returning(); + return committed; + }); + if (!updated) + throw conflict( + "OAuth credentials changed while refresh was in progress", + { + code: "oauth_refresh_superseded", + retryable: true, + }, + ); + return updated; + } finally { + await clearOAuthGrantRefreshLease(lease.grant, lease.leaseId).catch( + () => undefined, + ); + } + }, + ); + } + if ( + connection.authKind !== "oauth" || + connection.credentialSource !== "paperclip_vault" || + !oauthTokenUrl || + !oauthProvider + ) { + return initialGrant; + } + const expiresAtMs = oauthGrantExpiresAtMs(initialGrant, connection); + if ( + !input.forceRefresh && + (expiresAtMs === null || expiresAtMs > Date.now() + 60_000) + ) { + return initialGrant; + } + + return singleFlight(oauthGrantRefreshFlights, initialGrant.id, async () => { + const lease = await acquireOAuthGrantRefreshLease( + connection, + initialGrant, + input.forceRefresh === true, + ); + if (!lease.leaseId) return lease.grant; + try { + const grant = lease.grant; + const refreshRef = grant.credentialSecretRefs.find( + (ref) => ref.configPath === "oauth.refresh_token", + ); + if (!refreshRef) { + throw unprocessable( + "OAuth credentials have expired and no refresh token is available", + { + code: "oauth_refresh_missing", + setupUrl: connectionSetupUrl(connection), + reconnectUrl: connectionReconnectUrl(connection), + }, + ); + } + const refreshSecret = await resolveOAuthGrantSecret( + connection, + grant, + refreshRef, + input.actor, + input, + ); + const credentialActor: ActorInfo | undefined = + grant.kind === "user" && grant.subjectUserId + ? { actorType: "user", actorId: grant.subjectUserId } + : input.actor; + const client = await oauthClientForConnection( + connection, + oauthProvider, + credentialActor, + ); + if (!client.clientId) + throw unprocessable( + `OAuth client id is not configured for ${oauthProvider}`, + ); + const grantOauth = oauthGrantConfig(grant); + let token: Awaited>; + try { + token = await exchangeOAuthToken({ + tokenUrl: oauthTokenUrl, + clientId: client.clientId, + clientSecret: client.clientSecret, + tokenEndpointAuthMethod: storedOAuthTokenEndpointAuthMethod( + oauth, + client.clientSecret, + ), + grantType: "refresh_token", + scopes: + normalizeOauthScopes(grantOauth.scopes).length > 0 + ? normalizeOauthScopes(grantOauth.scopes) + : normalizeOauthScopes(oauth.scopes).length > 0 + ? normalizeOauthScopes(oauth.scopes) + : normalizeOauthScopes(oauth.scope), + refreshToken: refreshSecret.value, + resource: + typeof oauth.resource === "string" && oauth.resource + ? oauth.resource + : null, + }); + } catch (error) { + if ( + error instanceof HttpError && + asRecord(error.details).code === "oauth_reauthorization_required" + ) { + const retainedCredentialSecretRefs = + grant.credentialSecretRefs.filter( + (ref) => + ref.configPath !== "oauth.access_token" && + ref.configPath !== "oauth.refresh_token", + ); + const providerTenant = { + ...(grant.providerTenant ?? {}), + oauth: { + ...withoutOAuthGrantRefreshLease(grantOauth), + accessTokenExpiresAt: undefined, + }, + }; + const [marked] = await db + .update(connectionGrants) + .set({ + status: "needs_reauthorization", + providerTenant, + credentialSecretRefs: retainedCredentialSecretRefs, + updatedAt: now(), + }) + .where( + and( + eq(connectionGrants.id, grant.id), + eq(connectionGrants.companyId, grant.companyId), + eq(connectionGrants.status, "active"), + sql`${connectionGrants.providerTenant} -> 'oauth' -> 'refreshLease' ->> 'id' = ${lease.leaseId}`, + sql`exists ( + select 1 from ${companySecrets} + where ${companySecrets.id} = ${refreshRef.secretId} + and ${companySecrets.companyId} = ${connection.companyId} + and ${companySecrets.latestVersion} = ${refreshSecret.latestVersion} + )`, + ), + ) + .returning(); + if (!marked) { + const latest = await getOAuthGrantRow(input); + const latestExpiresAt = oauthGrantExpiresAtMs(latest, connection); + if ( + latest.status === "active" && + latestExpiresAt && + latestExpiresAt > Date.now() + 60_000 + ) + return latest; + throw conflict( + "OAuth credentials changed while refresh was in progress. Retry the request.", + { + code: "oauth_refresh_superseded", + retryable: true, + }, + ); + } + if (marked.kind === "organization") { + const latestConnection = await getConnectionRow( + connection.id, + connection.companyId, + ); + const [reauthorizationRequired] = await db + .update(toolConnections) + .set({ + status: "draft", + enabled: false, + credentialSecretRefs: + latestConnection.credentialSecretRefs.filter( + (ref) => + ref.configPath !== "oauth.access_token" && + ref.configPath !== "oauth.refresh_token", + ), + credentialRefs: latestConnection.credentialRefs.filter( + (ref) => + ref.name !== "oauth.access_token" && + ref.name !== "oauth.refresh_token", + ), + updatedAt: now(), + }) + .where( + and( + eq(toolConnections.id, latestConnection.id), + eq(toolConnections.companyId, latestConnection.companyId), + ), + ) + .returning(); + await syncCredentialBindings(reauthorizationRequired); + emitConnectionUpdated( + reauthorizationRequired, + latestConnection, + "credential_refresh", + ); + } else { + const activeGrantRefs = await db + .select({ + refs: connectionGrants.credentialSecretRefs, + }) + .from(connectionGrants) + .where( + and( + eq(connectionGrants.companyId, connection.companyId), + eq(connectionGrants.connectionId, connection.id), + eq(connectionGrants.status, "active"), + ), + ); + await syncCredentialBindings( + connection, + activeGrantRefs.flatMap((row) => row.refs), + ); + } + throw new HttpError(error.status, error.message, { + ...asRecord(error.details), + setupUrl: connectionSetupUrl(connection), + reconnectUrl: connectionReconnectUrl(connection), + }); + } + throw error; + } + + if (connection.config.sourceTemplateKey === "google-workspace" + && !isGoogleWorkspaceScopeGrant(normalizeOauthScopes(token.scope))) { + await db.update(connectionGrants).set({ status: "needs_reauthorization", updatedAt: now() }) + .where(and(eq(connectionGrants.id, grant.id), eq(connectionGrants.companyId, connection.companyId))); + throw unprocessable("Google did not return verifiable permissions. Reconnect Google Workspace.", { code: "oauth_reauthorization_required" }); + } + let nextRefreshRef: ToolCredentialSecretRef | null = null; + if (token.refreshToken) { + nextRefreshRef = await createOrRotateOAuthSecret({ + companyId: connection.companyId, + connection, + configPath: "oauth.refresh_token", + label: "OAuth refresh token", + value: token.refreshToken, + actor: credentialActor, + existingRefs: grant.credentialSecretRefs, + ownerUserId: + grant.kind === "user" + ? (grant.subjectUserId ?? undefined) + : undefined, + }); + } + const accessRef = await createOrRotateOAuthSecret({ + companyId: connection.companyId, + connection, + configPath: "oauth.access_token", + label: "OAuth access token", + value: token.accessToken, + actor: credentialActor, + existingRefs: grant.credentialSecretRefs, + ownerUserId: + grant.kind === "user" + ? (grant.subjectUserId ?? undefined) + : undefined, + }); + const nextCredentialSecretRefs = [ + ...grant.credentialSecretRefs.filter( + (ref) => + ref.configPath !== "oauth.access_token" && + (!nextRefreshRef || ref.configPath !== "oauth.refresh_token"), + ), + accessRef, + ...(nextRefreshRef ? [nextRefreshRef] : []), + ]; + const expiresAt = token.expiresIn + ? new Date(Date.now() + token.expiresIn * 1000).toISOString() + : null; + // A refresh carries no fresh authorization request, so the baseline stays what + // Paperclip asked for when *this* grant was authorized. Judging the response against + // the grant's own scopes instead would let an over-grant that the provider re-asserts + // on every refresh read back as clean, because the widened grant would have become + // its own baseline. The connection-level list is only a fallback for grants created + // before the per-grant baseline existed — it is whichever callback ran last, so on a + // connection two users authorized with different scopes it is the wrong baseline for + // at least one of them. + const refreshed = resolveGrantedOauthScopes({ + tokenScope: token.scope, + requestedScopes: + grantOauth.requestedScopes ?? + oauth.scopes ?? + oauth.scope ?? + grantOauth.scopes, + previous: { + scopes: grantOauth.scopes, + scopeSource: grantOauth.scopeSource, + unrequestedScopes: grantOauth.unrequestedScopes, + }, + }); + if (connection.config.sourceTemplateKey === "google-workspace") { + refreshed.scopes = googleWorkspaceEffectiveScopes(normalizeOauthScopes(grantOauth.requestedScopes ?? grantOauth.scopes), refreshed.scopes); + } + const providerTenant = { + ...(grant.providerTenant ?? {}), + oauth: { + ...withoutOAuthGrantRefreshLease(grantOauth), + strategy: + typeof grantOauth.strategy === "string" + ? grantOauth.strategy + : "direct_oauth", + accessTokenExpiresAt: expiresAt ?? undefined, + scopes: refreshed.scopes, + scopeSource: refreshed.scopeSource, + unrequestedScopes: refreshed.unrequestedScopes, + tokenType: token.tokenType, + refreshedAt: now().toISOString(), + }, + }; + const [updated] = await db + .update(connectionGrants) + .set({ + providerTenant, + credentialSecretRefs: nextCredentialSecretRefs, + status: "active", + updatedAt: now(), + }) + .where( + and( + eq(connectionGrants.id, grant.id), + eq(connectionGrants.companyId, grant.companyId), + eq(connectionGrants.status, "active"), + sql`${connectionGrants.providerTenant} -> 'oauth' -> 'refreshLease' ->> 'id' = ${lease.leaseId}`, + ), + ) + .returning(); + if (!updated) { + throw conflict( + "OAuth credentials changed while refresh was in progress. Retry the request.", + { + code: "oauth_refresh_superseded", + retryable: true, + }, + ); + } + if (grant.kind === "organization") { + const latestConnection = await getConnectionRow( + connection.id, + connection.companyId, + ); + const latestOauth = oauthConfig(latestConnection); + const nextConfig = { + ...latestConnection.config, + oauth: { + ...withoutOAuthRefreshLease(latestOauth), + expiresAt, + scope: token.scope ?? latestOauth.scope ?? null, + tokenType: token.tokenType, + refreshedAt: now().toISOString(), + }, + providerMetadata: { + ...asRecord(latestConnection.config.providerMetadata), + oauth: { + expiresAt, + scope: token.scope ?? latestOauth.scope ?? null, + // Carry the same provenance the grant just recorded. Without it a connection + // read reverts to a bare scope with no source and no over-grant warning after + // the first ordinary refresh, while the grant read still has both. + scopeSource: refreshed.scopeSource, + unrequestedScopes: refreshed.unrequestedScopes, + tokenType: token.tokenType, + }, + }, + }; + await db + .update(toolConnections) + .set({ + config: nextConfig, + transportConfig: nextConfig, + updatedAt: now(), + }) + .where( + and( + eq(toolConnections.id, latestConnection.id), + eq(toolConnections.companyId, latestConnection.companyId), + ), + ); + } + const previousKeys = new Set( + grant.credentialSecretRefs.map( + (ref) => `${ref.secretId}:${ref.configPath}`, + ), + ); + const nextKeys = new Set( + nextCredentialSecretRefs.map( + (ref) => `${ref.secretId}:${ref.configPath}`, + ), + ); + if ( + previousKeys.size !== nextKeys.size || + [...previousKeys].some((key) => !nextKeys.has(key)) + ) { + const activeGrantRefs = await db + .select({ + refs: connectionGrants.credentialSecretRefs, + }) + .from(connectionGrants) + .where( + and( + eq(connectionGrants.companyId, connection.companyId), + eq(connectionGrants.connectionId, connection.id), + eq(connectionGrants.status, "active"), + ), + ); + await syncCredentialBindings( + connection, + activeGrantRefs.flatMap((row) => row.refs), + ); + } + return updated; + } finally { + await clearOAuthGrantRefreshLease(lease.grant, lease.leaseId).catch( + () => undefined, + ); + } + }); + } + return refreshOAuthGrantCredentials; +} diff --git a/server/src/services/tool-access/oauth/refresh-leases.ts b/server/src/services/tool-access/oauth/refresh-leases.ts new file mode 100644 index 00000000000..8250da26e65 --- /dev/null +++ b/server/src/services/tool-access/oauth/refresh-leases.ts @@ -0,0 +1,299 @@ +import { oauthConfig } from "./metadata.js"; +import { randomUUID } from "node:crypto"; +import { and, eq, sql } from "drizzle-orm"; +import type { Db } from "@paperclipai/db"; +import { connectionGrants, toolConnections } from "@paperclipai/db"; +import { conflict, HttpError, unprocessable } from "../../../errors.js"; +import type { OAuthCredentialStore } from "./credential-store.js"; +import { oauthExpiresAtMs, oauthGrantConfig, oauthGrantExpiresAtMs } from "./client-settings.js"; +import { connectionSetupUrl, connectionReconnectUrl } from "../identifiers.js"; +function asRecord(value: unknown): Record { + if (value && typeof value === "object" && !Array.isArray(value)) + return value as Record; + return {}; +} + +const OAUTH_REFRESH_LEASE_MS = 120_000; + +const OAUTH_REFRESH_LEASE_WAIT_MS = 30_000; + +const OAUTH_REFRESH_LEASE_POLL_MS = 25; + +export function withoutOAuthRefreshLease(oauth: Record) { + const { refreshLease: _refreshLease, ...rest } = oauth; + return rest; +} + +export function withoutOAuthGrantRefreshLease(oauth: Record) { + const { refreshLease: _refreshLease, ...rest } = oauth; + return rest; +} + +export function oauthRefreshLeaseId( + connection: typeof toolConnections.$inferSelect, +): string | null { + const lease = asRecord(oauthConfig(connection).refreshLease); + return typeof lease.id === "string" && lease.id ? lease.id : null; +} + +export function createOAuthRefreshLeases(db: Db, now: () => Date, getConnectionRow: (idOrUid: string, companyId?: string) => Promise, getOAuthGrantRow: OAuthCredentialStore["getOAuthGrantRow"]) { + async function clearOAuthRefreshLease( + connection: typeof toolConnections.$inferSelect, + leaseId: string, + ) { + const latest = await getConnectionRow(connection.id, connection.companyId); + if (oauthRefreshLeaseId(latest) !== leaseId) return latest; + const nextConfig = { + ...latest.config, + oauth: withoutOAuthRefreshLease(oauthConfig(latest)), + }; + const [updated] = await db + .update(toolConnections) + .set({ + config: nextConfig, + transportConfig: nextConfig, + updatedAt: now(), + }) + .where( + and( + eq(toolConnections.id, latest.id), + eq(toolConnections.companyId, latest.companyId), + sql`${toolConnections.config} -> 'oauth' -> 'refreshLease' ->> 'id' = ${leaseId}`, + ), + ) + .returning(); + return updated ?? getConnectionRow(connection.id, connection.companyId); + } + + async function acquireOAuthRefreshLease( + connection: typeof toolConnections.$inferSelect, + ): Promise<{ + connection: typeof toolConnections.$inferSelect; + leaseId: string | null; + }> { + const waitDeadline = Date.now() + OAUTH_REFRESH_LEASE_WAIT_MS; + while (true) { + const latest = await getConnectionRow( + connection.id, + connection.companyId, + ); + const latestExpiresAtMs = oauthExpiresAtMs(latest); + if (latestExpiresAtMs && latestExpiresAtMs > Date.now() + 60_000) { + return { connection: latest, leaseId: null }; + } + + const oauth = oauthConfig(latest); + const currentLease = asRecord(oauth.refreshLease); + const currentLeaseExpiresAt = + typeof currentLease.expiresAt === "string" + ? Date.parse(currentLease.expiresAt) + : Number.NaN; + const currentLeaseId = + typeof currentLease.id === "string" && currentLease.id + ? currentLease.id + : null; + const leaseIsActive = + currentLeaseId !== null && + Number.isFinite(currentLeaseExpiresAt) && + currentLeaseExpiresAt > Date.now(); + if (!currentLeaseId) { + const leaseId = randomUUID(); + const claimedAt = now(); + const nextConfig = { + ...latest.config, + oauth: { + ...withoutOAuthRefreshLease(oauth), + refreshLease: { + id: leaseId, + expiresAt: new Date( + Date.now() + OAUTH_REFRESH_LEASE_MS, + ).toISOString(), + }, + }, + }; + const [claimed] = await db + .update(toolConnections) + .set({ + config: nextConfig, + transportConfig: nextConfig, + updatedAt: claimedAt, + }) + .where( + and( + eq(toolConnections.id, latest.id), + eq(toolConnections.companyId, latest.companyId), + sql`${toolConnections.config} = ${JSON.stringify(latest.config)}::jsonb`, + sql`${toolConnections.config} #>> '{oauth,refreshLease,id}' is null`, + ), + ) + .returning(); + if (claimed) return { connection: claimed, leaseId }; + } + + if (currentLeaseId && !leaseIsActive) { + throw new HttpError( + 422, + "The previous OAuth refresh did not finish. Reconnect this app before retrying.", + { + code: "oauth_refresh_outcome_unknown", + setupUrl: connectionSetupUrl(latest), + reconnectUrl: connectionReconnectUrl(latest), + }, + ); + } + + if (Date.now() >= waitDeadline) { + throw conflict("OAuth credential refresh is already in progress", { + code: "oauth_refresh_in_progress", + retryable: true, + }); + } + await new Promise((resolve) => + setTimeout(resolve, OAUTH_REFRESH_LEASE_POLL_MS), + ); + } + } + + async function clearOAuthGrantRefreshLease( + grant: typeof connectionGrants.$inferSelect, + leaseId: string, + ) { + const latest = await getOAuthGrantRow({ + companyId: grant.companyId, + connectionId: grant.connectionId, + grantId: grant.id, + }); + const oauth = oauthGrantConfig(latest); + if (asRecord(oauth.refreshLease).id !== leaseId) return latest; + const providerTenant = { + ...(latest.providerTenant ?? {}), + oauth: withoutOAuthGrantRefreshLease(oauth), + }; + const [updated] = await db + .update(connectionGrants) + .set({ + providerTenant, + updatedAt: now(), + }) + .where( + and( + eq(connectionGrants.id, latest.id), + eq(connectionGrants.companyId, latest.companyId), + sql`${connectionGrants.providerTenant} -> 'oauth' -> 'refreshLease' ->> 'id' = ${leaseId}`, + ), + ) + .returning(); + return ( + updated ?? + getOAuthGrantRow({ + companyId: grant.companyId, + connectionId: grant.connectionId, + grantId: grant.id, + }) + ); + } + + async function acquireOAuthGrantRefreshLease( + connection: typeof toolConnections.$inferSelect, + grant: typeof connectionGrants.$inferSelect, + forceRefresh: boolean, + ): Promise<{ + grant: typeof connectionGrants.$inferSelect; + leaseId: string | null; + }> { + const waitDeadline = Date.now() + OAUTH_REFRESH_LEASE_WAIT_MS; + const initialUpdatedAt = grant.updatedAt.getTime(); + while (true) { + const latest = await getOAuthGrantRow({ + companyId: connection.companyId, + connectionId: connection.id, + grantId: grant.id, + }); + if (latest.status !== "active") { + throw unprocessable("OAuth authorization must be reconnected", { + code: "oauth_reauthorization_required", + setupUrl: connectionSetupUrl(connection), + reconnectUrl: connectionReconnectUrl(connection), + }); + } + const expiresAtMs = oauthGrantExpiresAtMs(latest, connection); + if ( + (!forceRefresh && + (expiresAtMs === null || expiresAtMs > Date.now() + 60_000)) || + (forceRefresh && + latest.updatedAt.getTime() > initialUpdatedAt && + !asRecord(oauthGrantConfig(latest).refreshLease).id) + ) { + return { grant: latest, leaseId: null }; + } + + const oauth = oauthGrantConfig(latest); + const currentLease = asRecord(oauth.refreshLease); + const currentLeaseId = + typeof currentLease.id === "string" && currentLease.id + ? currentLease.id + : null; + const currentLeaseExpiresAt = + typeof currentLease.expiresAt === "string" + ? Date.parse(currentLease.expiresAt) + : Number.NaN; + const leaseIsActive = + currentLeaseId !== null && + Number.isFinite(currentLeaseExpiresAt) && + currentLeaseExpiresAt > Date.now(); + if (!currentLeaseId) { + const leaseId = randomUUID(); + const providerTenant = { + ...(latest.providerTenant ?? {}), + oauth: { + ...withoutOAuthGrantRefreshLease(oauth), + refreshLease: { + id: leaseId, + expiresAt: new Date( + Date.now() + OAUTH_REFRESH_LEASE_MS, + ).toISOString(), + }, + }, + }; + const [claimed] = await db + .update(connectionGrants) + .set({ + providerTenant, + updatedAt: now(), + }) + .where( + and( + eq(connectionGrants.id, latest.id), + eq(connectionGrants.companyId, latest.companyId), + eq(connectionGrants.status, "active"), + sql`${connectionGrants.providerTenant} #>> '{oauth,refreshLease,id}' is null`, + ), + ) + .returning(); + if (claimed) return { grant: claimed, leaseId }; + } + if (currentLeaseId && !leaseIsActive) { + throw unprocessable( + "The previous OAuth refresh did not finish. Reconnect this app before retrying.", + { + code: "oauth_refresh_outcome_unknown", + setupUrl: connectionSetupUrl(connection), + reconnectUrl: connectionReconnectUrl(connection), + }, + ); + } + if (Date.now() >= waitDeadline) { + throw conflict("OAuth credential refresh is already in progress", { + code: "oauth_refresh_in_progress", + retryable: true, + }); + } + await new Promise((resolve) => + setTimeout(resolve, OAUTH_REFRESH_LEASE_POLL_MS), + ); + } + } + return { clearOAuthRefreshLease, acquireOAuthRefreshLease, clearOAuthGrantRefreshLease, acquireOAuthGrantRefreshLease }; +} + +export type OAuthRefreshLeases = ReturnType; diff --git a/server/src/services/tool-access/single-flight.ts b/server/src/services/tool-access/single-flight.ts new file mode 100644 index 00000000000..e45bb8aac30 --- /dev/null +++ b/server/src/services/tool-access/single-flight.ts @@ -0,0 +1,17 @@ + + +export async function singleFlight( + flights: Map>, + key: string, + operation: () => Promise, +): Promise { + const existing = flights.get(key) as Promise | undefined; + if (existing) return existing; + const pending = operation(); + flights.set(key, pending); + try { + return await pending; + } finally { + if (flights.get(key) === pending) flights.delete(key); + } +} diff --git a/server/src/services/tool-access/vercel-credentials.ts b/server/src/services/tool-access/vercel-credentials.ts new file mode 100644 index 00000000000..10bb87628dc --- /dev/null +++ b/server/src/services/tool-access/vercel-credentials.ts @@ -0,0 +1,181 @@ +import type { ActorInfo } from "./types.js"; +import { remoteEndpoint } from "./connection-methods.js"; +import { and, eq } from "drizzle-orm"; +import type { Db } from "@paperclipai/db"; +import { connectionGrants, toolConnections } from "@paperclipai/db"; +import type { VercelConnectCredentialReference, VercelConnectGrantReference } from "@paperclipai/shared"; +import { conflict, HttpError, unprocessable } from "../../errors.js"; +import { deriveVercelConnectSubject, vercelGrantReference, vercelTokenRequest, VercelConnectClientError, type VercelConnectClient } from "../vercel-connect.js"; + +export function vercelConnectHttpError(error: unknown): HttpError { + if (error instanceof VercelConnectClientError) { + return new HttpError(error.status, error.message, { code: error.code }); + } + return new HttpError( + 502, + "Vercel Connect could not complete the credential request.", + { + code: "vercel_connect_request_failed", + }, + ); +} + +export function vercelCredentialFor( + connection: typeof toolConnections.$inferSelect, +): VercelConnectCredentialReference { + if ( + connection.credentialSource !== "vercel_connect" || + !connection.externalCredential + ) { + throw unprocessable("This connection does not use Vercel Connect", { + code: "vercel_connect_not_configured", + }); + } + return connection.externalCredential; +} + +export function vercelConnectResourcesFor( + connection: typeof toolConnections.$inferSelect, +): string[] { + const resource = new URL(remoteEndpoint(connection.config)); + // OAuth resource indicators name the protected MCP resource, not the + // connection's transport-only filtering query. PostHog uses this canonical + // URL to select its MCP consent scope set. + resource.search = ""; + resource.hash = ""; + return [resource.toString()]; +} + +export function redactedVercelGrant( + value: VercelConnectGrantReference | null | undefined, +): Omit | null { + if (!value) return null; + const { subjectId: _subjectId, ...redacted } = value; + return redacted; +} + +export function createVercelCredentialService(db: Db, now: () => Date, vercelConnect: VercelConnectClient | null) { + async function resolveVercelCredentialHeaders( + connection: typeof toolConnections.$inferSelect, + grant: typeof connectionGrants.$inferSelect, + options: { forceRefresh?: boolean } = {}, + ): Promise> { + if (!vercelConnect) + throw vercelConnectHttpError( + new VercelConnectClientError("vercel_connect_unavailable", 503), + ); + const credential = vercelCredentialFor(connection); + const derived = deriveVercelConnectSubject({ + credential, + connectionId: connection.id, + companyId: connection.companyId, + grantKind: grant.kind, + subjectUserId: grant.subjectUserId, + }); + const request = vercelTokenRequest({ + credential, + grant, + connectionId: connection.id, + companyId: connection.companyId, + resources: vercelConnectResourcesFor(connection), + }); + try { + const token = await vercelConnect.getToken(request, options); + if ( + token.connector.id !== credential.connectorId && + token.connector.uid !== credential.connectorUid + ) { + throw new VercelConnectClientError( + "vercel_connect_request_failed", + 502, + ); + } + await db + .update(connectionGrants) + .set({ + externalCredential: vercelGrantReference({ + credential, + token, + subjectId: derived.subjectId, + verifiedAt: now(), + }), + status: "active", + revokedAt: null, + updatedAt: now(), + }) + .where( + and( + eq(connectionGrants.id, grant.id), + eq(connectionGrants.companyId, connection.companyId), + eq(connectionGrants.connectionId, connection.id), + ), + ); + return { + [credential.headerName]: `${credential.headerPrefix ?? ""}${token.token}`, + }; + } catch (error) { + if ( + error instanceof VercelConnectClientError && + error.code === "vercel_connect_authorization_required" + ) { + await db + .update(connectionGrants) + .set({ + status: "needs_reauthorization", + updatedAt: now(), + }) + .where( + and( + eq(connectionGrants.id, grant.id), + eq(connectionGrants.companyId, connection.companyId), + ), + ); + } + throw vercelConnectHttpError(error); + } + } + + async function vercelGrantForConnection( + connection: typeof toolConnections.$inferSelect, + actor?: ActorInfo, + ) { + const actorUserId = + actor?.actorType === "user" ? (actor.actorId ?? null) : null; + if (actorUserId) { + const [personal] = await db + .select() + .from(connectionGrants) + .where( + and( + eq(connectionGrants.companyId, connection.companyId), + eq(connectionGrants.connectionId, connection.id), + eq(connectionGrants.kind, "user"), + eq(connectionGrants.subjectUserId, actorUserId), + ), + ) + .limit(1); + if (personal) return personal; + } + const [organization] = await db + .select() + .from(connectionGrants) + .where( + and( + eq(connectionGrants.companyId, connection.companyId), + eq(connectionGrants.connectionId, connection.id), + eq(connectionGrants.kind, "organization"), + eq(connectionGrants.isDefault, true), + ), + ) + .limit(1); + if (!organization) { + throw conflict("This Vercel Connect identity has not been authorized", { + code: "vercel_connect_authorization_required", + }); + } + return organization; + } + return { resolveVercelCredentialHeaders, vercelGrantForConnection }; +} + +export type VercelCredentialService = ReturnType;