Files
PaperClipAI/server/src/services/tool-access/github-grant-continuity.ts
T
DottaandPaperclip 4fdef9457f refactor(tools): isolate credential ownership (#17161)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connections need durable credentials that survive authorization and
token rotation.
> - Storage, registration, refresh coordination and credential selection
lived in the general tool service.
> - Each has a different ownership boundary: secrets, process-wide
registration, database leases and per-service refresh work.
> - This pull request gives those responsibilities named modules and
passes their narrow capabilities to consumers.
> - The benefit is visible credential ownership while retaining current
concurrency and failure behavior.

## Linked Issues or Issue Description

Depends on #17159. Earlier stack: #17144 → #17146 → #17147 → #17155 →
#17157 → #17159 → this PR.

**What existing behavior does this improve?**

Maintenance of OAuth credential persistence, refresh and provider
credential access.

**Current behavior**

The 14,877-line service owns registration flights, refresh leases,
secret writes and provider selection alongside app setup.

**Proposed behavior**

Dedicated owners handle credential storage, client registration, refresh
leases, connection/grant refresh, GitHub continuity and Vercel
credential access. Credential selection composes those owners.
Authorization callbacks remain in the coordinator for the next
extraction.

**Reason and benefit**

The entry point falls to 12,019 lines, a reduction of 2,858. Lease
acquisition and cleanup can be reviewed together, independently of
callback completion.

## What Changed

- Extract credential owners and shared single-flight/connection-link
primitives. The largest new module is 650 lines.
- Preserve process-wide client-registration flights, per-service refresh
maps and GitHub sweep state.
- Preserve live cloud-connector lookup, options and environment reads;
use the existing secret client and transaction-scoped clients.
- Extend the real local HTTP/PostgreSQL OAuth fixture with concurrent
refresh across two service instances and authenticated use of rotated
credentials.

## Verification

- The two extended HTTP integration cases pass before extraction for
public and client-secret-basic clients.
- After extraction, all 470 public service and HTTP integration cases
pass, including refresh failure, abandoned leases, secret redaction and
membership revocation cases. The fixture was then strengthened after
review: a second expiry must use the persisted rotated refresh token,
and the provider rejects the old token. Both cases pass before and after
extraction; server typecheck also passes.
- Server typecheck, module-boundary gate and diff whitespace checks
pass.
- All 233 pre-existing function/method bodies match by normalized syntax
comparison.
- Full typecheck/build will run at the stack handoff. The earlier
interrupted local full suite remains unverified; all current-head CI
gates pass, with Greptile 5/5 and no open review findings.

- Prior-head CI failure retained: run 39708550066, general server shard
2/12, `native-session-executor.test.ts` case “preserves a real listening
process and its marker when a health RPC times out” raised
`runner_remote_process_identity_unavailable` /
`process_identity_changed`. Both the fixture and runner implementation
are unchanged. The isolated case passes without changing assertions or
timeouts. The fresh full CI run passes for the test-only review fix; the
prior failure remains recorded here.

## Risks

- Changing state lifetime could permit duplicate refreshes or
registrations. Existing concurrency cases and the real HTTP fixture
exercise these paths.
- A transaction-scoped secret write must stay inside its authority lock.
Function bodies, await ordering and client arguments are preserved.
- The fixture proves assembled local HTTP/database behavior, not live
third-party qualification.
- No schema, public API, policy or scheduling change is intended.

## Model Used

OpenAI Codex, GPT-6 family, with repository tools and code execution.
The exact serving model ID and context-window size are not exposed in
this session.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-21 22:21:43 -05:00

304 lines
11 KiB
TypeScript

import type { ActorInfo } from "./types.js";
import { loadGitHubGrantMetadata } from "./github-access-discovery.js";
import { measureRunPhase } from "../execution-preparation-timing.js";
import { measureRunTransaction } from "../execution-transaction-timing.js";
import { and, eq } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import { connectionGrants, toolConnections } from "@paperclipai/db";
import { conflict, HttpError, notFound, unprocessable } from "../../errors.js";
import { type PaperclipCloudConnector } from "../paperclip-cloud-connector.js";
import type { createOAuthGrantRefresh } from "./oauth/grant-refresh.js";
import type { OAuthCredentialStore } from "./oauth/credential-store.js";
function asRecord(value: unknown): Record<string, unknown> {
if (value && typeof value === "object" && !Array.isArray(value))
return value as Record<string, unknown>;
return {};
}
type GitHubContinuityDependencies = {
now: () => Date;
currentCloudConnector: () => PaperclipCloudConnector | null;
refreshOAuthGrantCredentials: ReturnType<typeof createOAuthGrantRefresh>;
resolveOAuthGrantSecret: OAuthCredentialStore["resolveOAuthGrantSecret"];
};
export function createGitHubGrantContinuity(db: Db, { now, currentCloudConnector, refreshOAuthGrantCredentials, resolveOAuthGrantSecret }: GitHubContinuityDependencies) {
let nextGitHubContinuitySweepAt = 0;
async function refreshManagedGitHubGrantAccess(
connection: typeof toolConnections.$inferSelect,
initialGrant: typeof connectionGrants.$inferSelect,
actor?: ActorInfo,
) {
return measureRunPhase("connectors.github.refresh", async () => {
try {
return await refreshManagedGitHubGrantAccessOnce(
connection,
initialGrant,
actor,
);
} catch (error) {
if (
!(error instanceof HttpError) ||
asRecord(error.details).code !== "github_access_changed"
)
throw error;
const [current] = await measureRunPhase("connectors.github.grants", () => db
.select()
.from(connectionGrants)
.where(
and(
eq(connectionGrants.id, initialGrant.id),
eq(connectionGrants.companyId, connection.companyId),
eq(connectionGrants.connectionId, connection.id),
),
));
if (!current || current.status !== "active") throw error;
return refreshManagedGitHubGrantAccessOnce(connection, current, actor);
}
});
}
async function refreshManagedGitHubGrantAccessOnce(
connection: typeof toolConnections.$inferSelect,
initialGrant: typeof connectionGrants.$inferSelect,
actor?: ActorInfo,
) {
let grant = await measureRunPhase("connectors.github.credentials", () => refreshOAuthGrantCredentials({
companyId: connection.companyId,
connectionId: connection.id,
grantId: initialGrant.id,
actor,
}));
const resolveAccessToken = async () => measureRunPhase("connectors.github.credentials", async () => {
const accessRef = grant.credentialSecretRefs.find(
(ref) => ref.configPath === "oauth.access_token",
);
if (!accessRef)
throw unprocessable("GitHub authorization must be reconnected", {
code: "oauth_reauthorization_required",
});
return (
await resolveOAuthGrantSecret(
connection,
grant,
accessRef,
actor,
undefined,
)
).value;
});
let metadata;
let accessToken = await resolveAccessToken();
try {
metadata = await measureRunPhase("connectors.github.repositories", () => loadGitHubGrantMetadata(
accessToken,
fetch,
grant.providerTenant?.github?.appSlug,
));
} catch (error) {
const providerCode =
error instanceof HttpError ? asRecord(error.details).code : null;
if (providerCode !== "oauth_reauthorization_required") throw error;
// GitHub may invalidate an access token before its recorded expiry. If an
// expiring token pair exists, rotate it once under the same durable lease
// and CAS path, then repeat /user before surfacing reconnect-required.
grant = await measureRunPhase("connectors.github.credentials", () => refreshOAuthGrantCredentials({
companyId: connection.companyId,
connectionId: connection.id,
grantId: initialGrant.id,
forceRefresh: true,
actor,
}));
accessToken = await resolveAccessToken();
try {
metadata = await measureRunPhase("connectors.github.repositories", () => loadGitHubGrantMetadata(
accessToken,
fetch,
grant.providerTenant?.github?.appSlug,
));
} catch (retryError) {
const retryCode =
retryError instanceof HttpError
? asRecord(retryError.details).code
: null;
if (retryCode === "oauth_reauthorization_required") {
await measureRunPhase("connectors.github.grant_write", () => db
.update(connectionGrants)
.set({ status: "needs_reauthorization", updatedAt: now() })
.where(
and(
eq(connectionGrants.id, grant.id),
eq(connectionGrants.companyId, grant.companyId),
),
));
}
throw retryError;
}
}
const { updated, previousGitHub } = await measureRunTransaction(db, "connectors.github.refresh", async (tx) => {
const [currentGrant] = await measureRunPhase("connectors.github.grant_lock", () => tx
.select()
.from(connectionGrants)
.where(
and(
eq(connectionGrants.id, grant.id),
eq(connectionGrants.companyId, grant.companyId),
),
)
.for("update")
.limit(1));
if (!currentGrant || currentGrant.status === "revoked")
throw notFound("GitHub authorization not found");
const previousGitHub = currentGrant.providerTenant?.github;
const initialGitHub = grant.providerTenant?.github;
// No lock is held during provider requests. Reject a snapshot if another
// refresh or webhook changed access while those requests were in flight.
if (
previousGitHub?.accessRevision !== initialGitHub?.accessRevision ||
previousGitHub?.lastWebhookAt !== initialGitHub?.lastWebhookAt ||
previousGitHub?.lastAccessRefreshAt !==
initialGitHub?.lastAccessRefreshAt
) {
throw conflict("GitHub access changed during refresh. Try again.", {
code: "github_access_changed",
});
}
const providerTenant = {
...(currentGrant.providerTenant ?? {}),
github: {
...metadata,
...(previousGitHub?.lastWebhookAt
? { lastWebhookAt: previousGitHub.lastWebhookAt }
: {}),
webhookHealth:
previousGitHub?.webhookHealth ?? metadata.webhookHealth,
},
};
const [updated] = await measureRunPhase("connectors.github.grant_write", () => tx
.update(connectionGrants)
.set({
providerTenant,
status: "active",
updatedAt: now(),
})
.where(
and(
eq(connectionGrants.id, grant.id),
eq(connectionGrants.companyId, grant.companyId),
),
)
.returning());
if (!updated) throw notFound("GitHub authorization not found");
return { updated, previousGitHub };
});
const cloudConnector = currentCloudConnector();
const subject =
updated.kind === "agent" && updated.subjectAgentId
? `agent:${updated.subjectAgentId}`
: updated.kind === "user" && updated.subjectUserId
? updated.subjectUserId
: null;
if (cloudConnector && subject) {
const previous = new Set(previousGitHub?.installationIds ?? []);
const current = new Set(metadata.installationIds);
await measureRunPhase("connectors.github.webhooks", () => Promise.all([
...metadata.installationIds.map((installationId) =>
cloudConnector.setWebhookBinding({
subject,
companyId: connection.companyId,
id: `${updated.id}_${installationId}`,
installationId,
connectionId: connection.id,
grantId: updated.id,
active: true,
accessToken,
}),
),
...[...previous]
.filter((installationId) => !current.has(installationId))
.map((installationId) =>
cloudConnector.setWebhookBinding({
subject,
companyId: connection.companyId,
id: `${updated.id}_${installationId}`,
installationId,
connectionId: connection.id,
grantId: updated.id,
active: false,
}),
),
]));
}
return updated;
}
async function sweepGitHubConnectionContinuity() {
if (now().getTime() < nextGitHubContinuitySweepAt) {
return { checked: 0, due: 0, refreshed: 0, failed: 0 };
}
nextGitHubContinuitySweepAt = now().getTime() + 60 * 60_000;
const cutoff = now().getTime() - 30 * 24 * 60 * 60_000;
const rows = await db
.select({ grant: connectionGrants, connection: toolConnections })
.from(connectionGrants)
.innerJoin(
toolConnections,
and(
eq(toolConnections.id, connectionGrants.connectionId),
eq(toolConnections.companyId, connectionGrants.companyId),
),
)
.where(
and(
eq(connectionGrants.status, "active"),
eq(toolConnections.status, "active"),
eq(toolConnections.enabled, true),
),
);
const due = rows.filter(({ grant, connection }) => {
const config = asRecord(connection.config);
const oauth = asRecord(config.oauth);
if (
config.sourceTemplateKey !== "github" ||
oauth.connectorProfile !== "github.code" ||
!grant.providerTenant?.github
)
return false;
const expiresAt = grant.providerTenant.oauth?.accessTokenExpiresAt;
const refreshedAt = grant.providerTenant.oauth?.refreshedAt
? Date.parse(grant.providerTenant.oauth.refreshedAt)
: Number.NaN;
const accessCheckedAt = grant.providerTenant.github.lastAccessRefreshAt
? Date.parse(grant.providerTenant.github.lastAccessRefreshAt)
: Number.NaN;
if (typeof expiresAt === "string") {
const expiry = Date.parse(expiresAt);
return (
!Number.isFinite(expiry) ||
expiry <= now().getTime() + 60 * 60_000 ||
!Number.isFinite(refreshedAt) ||
refreshedAt <= cutoff
);
}
return !Number.isFinite(accessCheckedAt) || accessCheckedAt <= cutoff;
});
let refreshed = 0;
let failed = 0;
for (const row of due) {
try {
await refreshManagedGitHubGrantAccess(row.connection, row.grant, {
actorType: "system",
actorId: "github-connection-continuity",
});
refreshed += 1;
} catch {
failed += 1;
}
}
return { checked: rows.length, due: due.length, refreshed, failed };
}
return { refreshManagedGitHubGrantAccess, sweepGitHubConnectionContinuity };
}