mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-21 10:52:06 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connections need durable credentials that survive authorization and token rotation. > - Storage, registration, refresh coordination and credential selection lived in the general tool service. > - Each has a different ownership boundary: secrets, process-wide registration, database leases and per-service refresh work. > - This pull request gives those responsibilities named modules and passes their narrow capabilities to consumers. > - The benefit is visible credential ownership while retaining current concurrency and failure behavior. ## Linked Issues or Issue Description Depends on #17159. Earlier stack: #17144 → #17146 → #17147 → #17155 → #17157 → #17159 → this PR. **What existing behavior does this improve?** Maintenance of OAuth credential persistence, refresh and provider credential access. **Current behavior** The 14,877-line service owns registration flights, refresh leases, secret writes and provider selection alongside app setup. **Proposed behavior** Dedicated owners handle credential storage, client registration, refresh leases, connection/grant refresh, GitHub continuity and Vercel credential access. Credential selection composes those owners. Authorization callbacks remain in the coordinator for the next extraction. **Reason and benefit** The entry point falls to 12,019 lines, a reduction of 2,858. Lease acquisition and cleanup can be reviewed together, independently of callback completion. ## What Changed - Extract credential owners and shared single-flight/connection-link primitives. The largest new module is 650 lines. - Preserve process-wide client-registration flights, per-service refresh maps and GitHub sweep state. - Preserve live cloud-connector lookup, options and environment reads; use the existing secret client and transaction-scoped clients. - Extend the real local HTTP/PostgreSQL OAuth fixture with concurrent refresh across two service instances and authenticated use of rotated credentials. ## Verification - The two extended HTTP integration cases pass before extraction for public and client-secret-basic clients. - After extraction, all 470 public service and HTTP integration cases pass, including refresh failure, abandoned leases, secret redaction and membership revocation cases. The fixture was then strengthened after review: a second expiry must use the persisted rotated refresh token, and the provider rejects the old token. Both cases pass before and after extraction; server typecheck also passes. - Server typecheck, module-boundary gate and diff whitespace checks pass. - All 233 pre-existing function/method bodies match by normalized syntax comparison. - Full typecheck/build will run at the stack handoff. The earlier interrupted local full suite remains unverified; all current-head CI gates pass, with Greptile 5/5 and no open review findings. - Prior-head CI failure retained: run 39708550066, general server shard 2/12, `native-session-executor.test.ts` case “preserves a real listening process and its marker when a health RPC times out” raised `runner_remote_process_identity_unavailable` / `process_identity_changed`. Both the fixture and runner implementation are unchanged. The isolated case passes without changing assertions or timeouts. The fresh full CI run passes for the test-only review fix; the prior failure remains recorded here. ## Risks - Changing state lifetime could permit duplicate refreshes or registrations. Existing concurrency cases and the real HTTP fixture exercise these paths. - A transaction-scoped secret write must stay inside its authority lock. Function bodies, await ordering and client arguments are preserved. - The fixture proves assembled local HTTP/database behavior, not live third-party qualification. - No schema, public API, policy or scheduling change is intended. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
175 lines
5.5 KiB
TypeScript
175 lines
5.5 KiB
TypeScript
import type { ActorInfo } from "../types.js";
|
|
import { oauthConfig, isSmokeLabOAuthFixture } from "./metadata.js";
|
|
import { connectionMethodForConnection } from "../connection-methods.js";
|
|
import { stableHash } from "../../tool-catalog-hash.js";
|
|
import { ASANA_CONNECTOR_SCOPES, isAsanaConnectorProfileId, type AsanaConnectorProfileId } from "@paperclipai/shared";
|
|
import { connectionGrants, toolConnections } from "@paperclipai/db";
|
|
import { GITHUB_CONNECTOR_PROFILES, GOOGLE_WORKSPACE_CONNECTOR_PROFILES, getConnectableAppDefinition, isGitHubConnectorProfileId, isGoogleWorkspaceConnectorProfileId, type GitHubConnectorProfileId, type GoogleWorkspaceConnectorProfileId } from "@paperclipai/shared";
|
|
import { unprocessable } from "../../../errors.js";
|
|
|
|
function asRecord(value: unknown): Record<string, unknown> {
|
|
if (value && typeof value === "object" && !Array.isArray(value))
|
|
return value as Record<string, unknown>;
|
|
return {};
|
|
}
|
|
|
|
export function oauthEnvName(
|
|
provider: string,
|
|
suffix: "CLIENT_ID" | "CLIENT_SECRET",
|
|
) {
|
|
return `PAPERCLIP_TOOL_OAUTH_${provider.replace(/[^a-z0-9]+/gi, "_").toUpperCase()}_${suffix}`;
|
|
}
|
|
|
|
export function oauthClientConfig(provider: string) {
|
|
const clientIdEnv = oauthEnvName(provider, "CLIENT_ID");
|
|
const clientSecretEnv = oauthEnvName(provider, "CLIENT_SECRET");
|
|
return {
|
|
clientIdEnv,
|
|
clientSecretEnv,
|
|
clientId:
|
|
process.env[clientIdEnv] ??
|
|
process.env.PAPERCLIP_TOOL_OAUTH_CLIENT_ID ??
|
|
null,
|
|
clientSecret:
|
|
process.env[clientSecretEnv] ??
|
|
process.env.PAPERCLIP_TOOL_OAUTH_CLIENT_SECRET ??
|
|
null,
|
|
};
|
|
}
|
|
|
|
export function configuredOAuthClientForConnection(
|
|
connection: typeof toolConnections.$inferSelect,
|
|
provider: string,
|
|
) {
|
|
if (isSmokeLabOAuthFixture(connection) && provider === "smoke_lab") {
|
|
return {
|
|
clientIdEnv: "SMOKE_LAB_FIXED_CLIENT_ID",
|
|
clientSecretEnv: "SMOKE_LAB_FIXED_CLIENT_SECRET",
|
|
clientId: "paperclip-smoke-lab",
|
|
clientSecret: null,
|
|
};
|
|
}
|
|
return oauthClientConfig(provider);
|
|
}
|
|
|
|
export function credentialScope(
|
|
connection: typeof toolConnections.$inferSelect,
|
|
actor?: ActorInfo,
|
|
) {
|
|
const configured = asRecord(oauthConfig(connection).credentialScope);
|
|
const type =
|
|
typeof configured.type === "string"
|
|
? configured.type
|
|
: typeof configured.targetType === "string"
|
|
? configured.targetType
|
|
: actor?.actorType === "agent"
|
|
? "agent"
|
|
: actor?.actorType === "user"
|
|
? "user"
|
|
: "company";
|
|
const id =
|
|
typeof configured.id === "string"
|
|
? configured.id
|
|
: typeof configured.targetId === "string"
|
|
? configured.targetId
|
|
: (actor?.actorId ?? connection.companyId);
|
|
return {
|
|
type,
|
|
id,
|
|
hash: stableHash({
|
|
companyId: connection.companyId,
|
|
connectionId: connection.id,
|
|
type,
|
|
id,
|
|
}),
|
|
};
|
|
}
|
|
|
|
export function oauthSecretRef(
|
|
connection: typeof toolConnections.$inferSelect,
|
|
configPath:
|
|
"oauth.access_token" | "oauth.refresh_token" | "oauth.client_secret" | "railway.ssh_private_key",
|
|
) {
|
|
return (
|
|
connection.credentialSecretRefs.find(
|
|
(ref) => ref.configPath === configPath,
|
|
) ?? null
|
|
);
|
|
}
|
|
|
|
export function oauthExpiresAtMs(
|
|
connection: typeof toolConnections.$inferSelect,
|
|
): number | null {
|
|
const expiresAt = oauthConfig(connection).expiresAt;
|
|
if (typeof expiresAt !== "string") return null;
|
|
const ms = Date.parse(expiresAt);
|
|
return Number.isFinite(ms) ? ms : null;
|
|
}
|
|
|
|
export async function oauthGalleryEntryForConnection(
|
|
connection: typeof toolConnections.$inferSelect,
|
|
) {
|
|
const sourceTemplateKey =
|
|
typeof connection.config.sourceTemplateKey === "string"
|
|
? connection.config.sourceTemplateKey
|
|
: null;
|
|
if (!sourceTemplateKey)
|
|
throw unprocessable(
|
|
"This app connection was not created from the app gallery",
|
|
);
|
|
const galleryEntry = getConnectableAppDefinition(sourceTemplateKey);
|
|
if (
|
|
!galleryEntry ||
|
|
connectionMethodForConnection(galleryEntry, connection).auth !== "oauth"
|
|
) {
|
|
throw unprocessable("This app connection does not use sign in");
|
|
}
|
|
return galleryEntry;
|
|
}
|
|
|
|
export type ManagedConnectorProfileId =
|
|
GoogleWorkspaceConnectorProfileId | GitHubConnectorProfileId | AsanaConnectorProfileId;
|
|
|
|
export function managedConnectorProfile(value: string | undefined): {
|
|
id: ManagedConnectorProfileId;
|
|
provider: "google" | "github" | "asana";
|
|
scopes: readonly string[];
|
|
} | null {
|
|
if (value && isAsanaConnectorProfileId(value)) {
|
|
return { id: value, provider: "asana", scopes: ASANA_CONNECTOR_SCOPES };
|
|
}
|
|
if (value && isGoogleWorkspaceConnectorProfileId(value)) {
|
|
return {
|
|
id: value,
|
|
provider: "google",
|
|
scopes: GOOGLE_WORKSPACE_CONNECTOR_PROFILES[value].scopes,
|
|
};
|
|
}
|
|
if (value && isGitHubConnectorProfileId(value)) {
|
|
return {
|
|
id: value,
|
|
provider: "github",
|
|
scopes: GITHUB_CONNECTOR_PROFILES[value].scopes,
|
|
};
|
|
}
|
|
return null;
|
|
}
|
|
|
|
export function oauthGrantConfig(grant: typeof connectionGrants.$inferSelect) {
|
|
return asRecord(asRecord(grant.providerTenant).oauth);
|
|
}
|
|
|
|
export function oauthGrantExpiresAtMs(
|
|
grant: typeof connectionGrants.$inferSelect,
|
|
connection: typeof toolConnections.$inferSelect,
|
|
): number | null {
|
|
const grantExpiresAt = oauthGrantConfig(grant).accessTokenExpiresAt;
|
|
const value =
|
|
typeof grantExpiresAt === "string"
|
|
? grantExpiresAt
|
|
: oauthConfig(connection).expiresAt;
|
|
if (typeof value !== "string") return null;
|
|
const ms = Date.parse(value);
|
|
return Number.isFinite(ms) ? ms : null;
|
|
}
|