Files
PaperClipAI/server/src/services/tool-access/credential-access.ts
T
DottaandPaperclip 4fdef9457f refactor(tools): isolate credential ownership (#17161)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connections need durable credentials that survive authorization and
token rotation.
> - Storage, registration, refresh coordination and credential selection
lived in the general tool service.
> - Each has a different ownership boundary: secrets, process-wide
registration, database leases and per-service refresh work.
> - This pull request gives those responsibilities named modules and
passes their narrow capabilities to consumers.
> - The benefit is visible credential ownership while retaining current
concurrency and failure behavior.

## Linked Issues or Issue Description

Depends on #17159. Earlier stack: #17144 → #17146 → #17147 → #17155 →
#17157 → #17159 → this PR.

**What existing behavior does this improve?**

Maintenance of OAuth credential persistence, refresh and provider
credential access.

**Current behavior**

The 14,877-line service owns registration flights, refresh leases,
secret writes and provider selection alongside app setup.

**Proposed behavior**

Dedicated owners handle credential storage, client registration, refresh
leases, connection/grant refresh, GitHub continuity and Vercel
credential access. Credential selection composes those owners.
Authorization callbacks remain in the coordinator for the next
extraction.

**Reason and benefit**

The entry point falls to 12,019 lines, a reduction of 2,858. Lease
acquisition and cleanup can be reviewed together, independently of
callback completion.

## What Changed

- Extract credential owners and shared single-flight/connection-link
primitives. The largest new module is 650 lines.
- Preserve process-wide client-registration flights, per-service refresh
maps and GitHub sweep state.
- Preserve live cloud-connector lookup, options and environment reads;
use the existing secret client and transaction-scoped clients.
- Extend the real local HTTP/PostgreSQL OAuth fixture with concurrent
refresh across two service instances and authenticated use of rotated
credentials.

## Verification

- The two extended HTTP integration cases pass before extraction for
public and client-secret-basic clients.
- After extraction, all 470 public service and HTTP integration cases
pass, including refresh failure, abandoned leases, secret redaction and
membership revocation cases. The fixture was then strengthened after
review: a second expiry must use the persisted rotated refresh token,
and the provider rejects the old token. Both cases pass before and after
extraction; server typecheck also passes.
- Server typecheck, module-boundary gate and diff whitespace checks
pass.
- All 233 pre-existing function/method bodies match by normalized syntax
comparison.
- Full typecheck/build will run at the stack handoff. The earlier
interrupted local full suite remains unverified; all current-head CI
gates pass, with Greptile 5/5 and no open review findings.

- Prior-head CI failure retained: run 39708550066, general server shard
2/12, `native-session-executor.test.ts` case “preserves a real listening
process and its marker when a health RPC times out” raised
`runner_remote_process_identity_unavailable` /
`process_identity_changed`. Both the fixture and runner implementation
are unchanged. The isolated case passes without changing assertions or
timeouts. The fresh full CI run passes for the test-only review fix; the
prior failure remains recorded here.

## Risks

- Changing state lifetime could permit duplicate refreshes or
registrations. Existing concurrency cases and the real HTTP fixture
exercise these paths.
- A transaction-scoped secret write must stay inside its authority lock.
Function bodies, await ordering and client arguments are preserved.
- The fixture proves assembled local HTTP/database behavior, not live
third-party qualification.
- No schema, public API, policy or scheduling change is intended.

## Model Used

OpenAI Codex, GPT-6 family, with repository tools and code execution.
The exact serving model ID and context-window size are not exposed in
this session.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-21 22:21:43 -05:00

302 lines
12 KiB
TypeScript

import type { ActorInfo } from "./types.js";
import { oauthConfig } from "./oauth/metadata.js";
import { remoteEndpoint } from "./connection-methods.js";
import { and, eq } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import { connectionGrants, toolConnections } from "@paperclipai/db";
import { badRequest, HttpError, unprocessable } from "../../errors.js";
import { parseRemoteHttpEndpoint } from "../remote-http-endpoint-guard.js";
import { REMOTE_URL_SECRET_CONFIG_PATH, remoteUrlCredentialMatchesPublicUrl } from "../remote-url-credentials.js";
import { secretService } from "../secrets.js";
import { connectionCredentialConfigPath as credentialRefConfigPath, connectionGrantCredentialRef } from "../connection-credentials.js";
import type { ToolAccessAudit } from "./audit.js";
import type { VercelCredentialService } from "./vercel-credentials.js";
import type { OAuthCredentialStore } from "./oauth/credential-store.js";
import type { createLegacyOAuthRefresh } from "./oauth/connection-refresh.js";
import type { createOAuthGrantRefresh } from "./oauth/grant-refresh.js";
import { credentialScope } from "./oauth/client-settings.js";
import { connectionSetupUrl, connectionReconnectUrl } from "./identifiers.js";
function asRecord(value: unknown): Record<string, unknown> {
if (value && typeof value === "object" && !Array.isArray(value))
return value as Record<string, unknown>;
return {};
}
type CredentialAccessDependencies = {
secrets: ReturnType<typeof secretService>;
audit: ToolAccessAudit;
vercel: VercelCredentialService;
resolveOAuthGrantSecret: OAuthCredentialStore["resolveOAuthGrantSecret"];
maybeRefreshOAuthCredentials: ReturnType<typeof createLegacyOAuthRefresh>;
refreshOAuthGrantCredentials: ReturnType<typeof createOAuthGrantRefresh>;
};
export function createConnectionCredentialAccess(db: Db, { secrets, audit, vercel, resolveOAuthGrantSecret, maybeRefreshOAuthCredentials, refreshOAuthGrantCredentials }: CredentialAccessDependencies) {
const { resolveVercelCredentialHeaders, vercelGrantForConnection } = vercel;
async function vaultGrantForConnection(
connection: typeof toolConnections.$inferSelect,
actor?: ActorInfo,
): Promise<typeof connectionGrants.$inferSelect | null> {
const actorUserId =
actor?.actorType === "user" ? (actor.actorId ?? null) : null;
if (actorUserId) {
const [personal] = await db
.select()
.from(connectionGrants)
.where(
and(
eq(connectionGrants.companyId, connection.companyId),
eq(connectionGrants.connectionId, connection.id),
eq(connectionGrants.kind, "user"),
eq(connectionGrants.subjectUserId, actorUserId),
),
)
.limit(1);
if (personal) {
if (personal.status !== "active") {
throw unprocessable("OAuth authorization must be reconnected", {
code: "oauth_reauthorization_required",
setupUrl: connectionSetupUrl(connection),
reconnectUrl: connectionReconnectUrl(connection),
});
}
return personal;
}
}
if (!actorUserId && connection.credentialPolicy === "per_user") {
// A personal-only connection is intentionally fixed to one identity.
// Background health/catalog checks have no acting user, but may safely
// exercise that sole owner-bound grant without turning it into a shared
// credential or making it available to a different caller.
const personalGrants = await db
.select()
.from(connectionGrants)
.where(
and(
eq(connectionGrants.companyId, connection.companyId),
eq(connectionGrants.connectionId, connection.id),
eq(connectionGrants.kind, "user"),
eq(connectionGrants.status, "active"),
),
)
.limit(2);
if (personalGrants.length === 1) return personalGrants[0]!;
}
if (connection.credentialPolicy === "per_user") {
throw unprocessable(
"This connection needs the current user's authorization",
{
code: "user_authorization_required",
setupUrl: connectionSetupUrl(connection),
reconnectUrl: connectionReconnectUrl(connection),
},
);
}
const [organization] = await db
.select()
.from(connectionGrants)
.where(
and(
eq(connectionGrants.companyId, connection.companyId),
eq(connectionGrants.connectionId, connection.id),
eq(connectionGrants.kind, "organization"),
eq(connectionGrants.isDefault, true),
),
)
.limit(1);
if (organization?.status === "active") return organization;
return null;
}
async function resolveCredentialHeaders(
connection: typeof toolConnections.$inferSelect,
actor?: ActorInfo,
options: { forceRefresh?: boolean } = {},
): Promise<Record<string, string>> {
if (connection.credentialSource === "vercel_connect") {
const grant = await vercelGrantForConnection(connection, actor);
return resolveVercelCredentialHeaders(connection, grant, options);
}
let grant: typeof connectionGrants.$inferSelect | null = null;
try {
if (
connection.authKind === "oauth" ||
connection.credentialPolicy !== "shared"
) {
grant = await vaultGrantForConnection(connection, actor);
}
if (connection.authKind === "oauth" && grant) {
grant = await refreshOAuthGrantCredentials({
companyId: connection.companyId,
connectionId: connection.id,
grantId: grant.id,
forceRefresh: options.forceRefresh,
actor,
});
} else {
connection = await maybeRefreshOAuthCredentials(connection, actor);
}
} catch (error) {
const scope = credentialScope(connection);
await audit({
companyId: connection.companyId,
connectionId: connection.id,
action: "tool_connection.credential_resolution",
outcome: "failure",
reasonCode:
error instanceof HttpError
? String(asRecord(error.details).code ?? "oauth_refresh_failed")
: "oauth_refresh_failed",
details: {
credentialCount: connection.credentialRefs.length,
credentialSecretRefCount: connection.credentialSecretRefs.length,
credentialScopeType: scope.type,
credentialScopeHash: scope.hash,
setupUrl: connectionSetupUrl(connection),
reconnectUrl: connectionReconnectUrl(connection),
},
});
throw error;
}
const headers: Record<string, string> = {};
const scope = credentialScope(connection);
for (const ref of connection.credentialRefs) {
if (ref.placement !== "header") continue;
let value: string;
const configPath = credentialRefConfigPath(ref);
try {
const grantRef = grant ? connectionGrantCredentialRef(grant, ref) : undefined;
if (grant && !grantRef) throw unprocessable("Your credential is missing. Reconnect this connection.", {
code: "grant_credential_invalid", connectionId: connection.id, grantId: grant.id, credential: configPath,
});
value =
grantRef && grant
? (
await resolveOAuthGrantSecret(
connection,
grant,
grantRef,
actor,
undefined,
)
).value
: await secrets.resolveSecretValue(
connection.companyId,
ref.secretId,
ref.version ?? "latest",
{
consumerType: "tool_connection",
consumerId: connection.id,
configPath,
actorType: "system",
},
);
} catch (error) {
await audit({
companyId: connection.companyId,
connectionId: connection.id,
action: "tool_connection.credential_resolution",
outcome: "failure",
reasonCode:
error instanceof HttpError
? String(
asRecord(error.details).code ?? "secret_resolution_failed",
)
: "secret_resolution_failed",
details: {
credentialCount: connection.credentialRefs.length,
credentialScopeType: scope.type,
credentialScopeHash: scope.hash,
},
});
throw error;
}
if (ref.placement === "header") {
headers[ref.key] = `${ref.prefix ?? ""}${value}`;
}
}
const oauthAccessRef = grant?.credentialSecretRefs.find(
(ref) => ref.configPath === "oauth.access_token",
);
if (oauthAccessRef && headers.Authorization === undefined) {
headers.Authorization = `Bearer ${
(
await resolveOAuthGrantSecret(
connection,
grant!,
oauthAccessRef,
actor,
undefined,
)
).value
}`;
}
if (
connection.credentialRefs.length > 0 ||
connection.credentialSecretRefs.length > 0 ||
Object.keys(oauthConfig(connection)).length > 0
) {
await audit({
companyId: connection.companyId,
connectionId: connection.id,
action: "tool_connection.credential_resolution",
outcome: "success",
details: {
credentialCount: connection.credentialRefs.length,
credentialSecretRefCount: connection.credentialSecretRefs.length,
credentialScopeType: scope.type,
credentialScopeHash: scope.hash,
},
});
}
return headers;
}
async function resolvedRemoteEndpoint(
connection: typeof toolConnections.$inferSelect,
actor?: ActorInfo,
): Promise<string> {
const publicEndpoint = remoteEndpoint(connection.config);
const ref = connection.credentialRefs.find(
(candidate) => candidate.placement === "url",
);
if (!ref) return publicEndpoint;
let value: string;
try {
const grant = await vaultGrantForConnection(connection, actor);
const grantRef = grant ? connectionGrantCredentialRef(grant, ref) : undefined;
if (grant && !grantRef) throw unprocessable("Your MCP URL credential is missing. Reconnect this connection.", {
code: "grant_credential_invalid", connectionId: connection.id, grantId: grant.id,
});
value = grant && grantRef
? (await resolveOAuthGrantSecret(connection, grant, grantRef, actor, undefined)).value
: await secrets.resolveSecretValue(connection.companyId, ref.secretId, ref.version ?? "latest", {
consumerType: "tool_connection", consumerId: connection.id,
configPath: REMOTE_URL_SECRET_CONFIG_PATH,
actorType: actor?.actorType ?? "system", actorId: actor?.actorId ?? null,
});
} catch (error) {
if (error instanceof HttpError) throw error;
throw unprocessable("Your MCP URL credential could not be resolved. Reconnect this connection.", {
code: "mcp_remote_missing_secret", connectionId: connection.id, credential: REMOTE_URL_SECRET_CONFIG_PATH,
});
}
if (!remoteUrlCredentialMatchesPublicUrl(publicEndpoint, value)) {
throw unprocessable(
"The stored MCP URL credential no longer matches this connection.",
{
code: "mcp_remote_url_credential_mismatch",
connectionId: connection.id,
},
);
}
return parseRemoteHttpEndpoint(value, (message, code) =>
badRequest(message, { code }),
).toString();
}
return { vaultGrantForConnection, resolveCredentialHeaders, resolvedRemoteEndpoint };
}
export type ConnectionCredentialAccess = ReturnType<typeof createConnectionCredentialAccess>;