mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-24 10:13:11 +02:00
refactor(tools): isolate GitHub access discovery (#17144)
## Thinking Path > - Paperclip manages AI agents and their access to external tools. > - Tool connections discover repositories through GitHub account grants. > - The tool-access service mixes GitHub response parsing with grant authorization and persistence. > - These responsibilities have different inputs and failure rules. > - This PR puts GitHub discovery in one provider module. > - The service keeps current authority checks and stored grant updates. ## Linked Issues or Issue Description **What existing behavior does this improve?** The ownership and review of GitHub access discovery in the tool-access service. **Current behavior** The 21,525-line service contains GitHub pagination, account metadata validation, and grant lifecycle code. **Proposed behavior** A GitHub module returns normalized access metadata. Existing service exports remain compatible. This is the first PR in a three-part refactor stack. No behavior change is intended. A search found no existing tool-access refactor PR or issue. Stack order: this PR, #17146, then #17147. Each PR targets its predecessor. ## What Changed - Move six functions into `tool-access/github-access-discovery.ts`. Their bodies are unchanged. - Keep grant selection, credential resolution, refresh retry, membership locks, revision checks, and webhook updates in the service. - Add public-entry characterization tests for HTTP error classification and invalid pagination. - Record five candidate boundaries and the initial three-PR scope in the dated plan. - Reduce `tool-access.ts` from 21,525 lines / 829,519 bytes to 21,237 lines / 820,116 bytes. The new module is 294 lines / 9,722 bytes. ## Verification - Before and after: `pnpm exec vitest run server/src/__tests__/github-grant-metadata.test.ts server/src/__tests__/project-repositories.test.ts` — 13 tests pass. - Before and after: `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts -t 'GitHub|github'` — 11 selected tests pass with disposable PostgreSQL; 455 cases are outside this selection. - `pnpm --filter @paperclipai/server typecheck` passes. - Exact text comparison confirms all six moved bodies match the base. `git diff --check` passes. - The initial baseline attempt failed because the plugin SDK had not been built. The normal workspace prerequisite build resolved it before behavioral validation. - Final-stack local verification: `pnpm -r --workspace-concurrency=2 typecheck` and `pnpm build` pass. CI is green at `07e78dd1b1c26e0bd4234006766444c71e234451`. Greptile is 5/5 with no actionable findings. - Full local-suite limitation: `pnpm test:run` was interrupted after 79 minutes during the serial general-server phase, after CI had completed the full test matrix. Before interruption, it flagged the unchanged `company-search-service.test.ts` suite (26 database cases skipped). One isolated rerun passed all 28 cases without code changes. The original log and a pre-interruption copy are preserved. The original failure cause was not reproduced. The full local suite and its later groups are not claimed as passed; the complete CI matrix is green on this stack. ## Risks - Import wiring could break existing consumers. Existing imports remain supported and all four production call sites are unchanged. - Provider requests retain their ordering, timeout, fixed GitHub origin, and response redaction. The new module has no database dependency or shared mutable state. - Tests simulate GitHub responses. They do not qualify live GitHub access. ## Model Used OpenAI Codex, GPT-6 family, with code execution and repository tools. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed
1 parent
0081863463
commit
9db3a5726e
4 files changed
+365
-290
No files matched your search
@@ -0,0 +1,46 @@
|
||||
# Tool access boundaries
|
||||
|
||||
Baseline: `efc2ddc7bf19ec319b21cd294e71e5a4932c16f4`.
|
||||
`server/src/services/tool-access.ts` has 21,525 lines and 829,519 bytes.
|
||||
Imports account for 293 lines. The service factory spans 18,366 lines,
|
||||
including its public method implementations and service wiring.
|
||||
The largest functions are connection setup (1,468 lines), OAuth callback
|
||||
completion (644), OAuth start (613), and grant refresh (611).
|
||||
|
||||
The first batch contains three dependent PRs. Each PR targets its predecessor.
|
||||
Existing exports remain compatible. No branch is merged as part of this batch.
|
||||
|
||||
| Order | Responsibility and destination | Estimated move / net reduction | Contract | State and effects | Verification |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 1 | GitHub access discovery, `services/tool-access/github-access-discovery.ts` | 290 / 288 lines | Token or headers and request function to normalized repository/account metadata | GitHub pagination and response validation; no stored grants or shared state | Existing public exports, paginated PAT and managed access, malformed responses, retry classification, GitHub service integration |
|
||||
| 2 | Connection method configuration and projection, `services/tool-access/connection-methods.ts` | About 330 / 310 lines | Reviewed method and raw settings to validated settings; connection settings and tool input to projected headers/schema/arguments | Pure transforms; provider-specific argument policy stays in its existing module | Configuration rejection, managed defaults, header safety, legacy methods, setup and discovery integration |
|
||||
| 3 | OAuth protocol handling, `services/tool-access/oauth/` | About 1,500 / 1,400 lines | Guarded requests and connection metadata to discovered endpoints; registration request to validated client material | Network interpretation and endpoint policy; persistence and refresh leases stay in the coordinator | Existing discovery, DCR, CIMD, scope, redaction, issuer and endpoint-security tests; real local protocol fixture |
|
||||
| 4 | Profile resolution and review, `services/tool-access/profiles/` | About 1,500 / 1,400 lines | Company, agent, selectors and current catalog to effective profiles and review decisions | Scoped queries and profile mutations; preserve assignment locks and caller transactions | Binding precedence, exclusions, company isolation, new-tool review and lightweight projections |
|
||||
| 5 | Connection token brokering, `services/tool-access/token-broker/` | About 1,400 / 1,300 lines | Authorized run and token request to issuance result | Run/membership locks, rate limit, credential exchange, hashed issuance audit | Revocation races, scope restriction, unavailable/disabled broker, exchange failure and secret redaction |
|
||||
|
||||
Rows 4 and 5 are follow-up candidates, not part of the initial batch. Reassess
|
||||
their contracts after the first three cuts. Keep authorization, current grant
|
||||
checks, company isolation, audit, and transaction ownership with their existing
|
||||
owners. Do not pass the full service context into a new module.
|
||||
|
||||
Existing extracted owners include tool-access policy, profile entry validation,
|
||||
binding precedence, credential bindings, MCP transport and endpoint guards,
|
||||
runtime supervision, managed Cloud connectors, Vercel Connect, and individual
|
||||
provider policies. New modules import those owners directly and never import
|
||||
the compatibility service entry point.
|
||||
|
||||
Run relevant tests before and after each move, with one database test worker.
|
||||
Compare moved bodies against the base. Run repository typecheck, full tests,
|
||||
and build at the batch handoff. Record setup failures separately from behavior
|
||||
failures, and leave unavailable checks unverified.
|
||||
|
||||
## First extraction
|
||||
|
||||
GitHub access discovery preserves all six moved function bodies exactly.
|
||||
The service retains grant selection, credential resolution, refresh retry,
|
||||
membership locking, revision checks, persistence, and webhook updates.
|
||||
The original service becomes 21,237 lines / 820,116 bytes; the new module is
|
||||
294 lines / 9,722 bytes. Public-path tests pass before and after: 13 focused
|
||||
tests and 11 selected GitHub service cases using disposable PostgreSQL.
|
||||
The initial test attempt could not resolve the unbuilt plugin SDK; building
|
||||
the existing workspace prerequisites resolved that setup failure.
|
||||
@@ -92,4 +92,27 @@ describe("GitHub grant metadata", () => {
|
||||
details: expect.objectContaining({ code: "github_access_check_failed" }),
|
||||
});
|
||||
});
|
||||
|
||||
it.each([
|
||||
[401, "oauth_reauthorization_required"],
|
||||
[503, "github_access_check_failed"],
|
||||
])("preserves the retry decision for HTTP %s without reflecting provider text", async (status, code) => {
|
||||
const request = vi.fn<typeof fetch>().mockResolvedValue(new Response("private provider detail", { status }));
|
||||
await expect(loadGitHubGrantMetadata("ghu_secret", request)).rejects.toMatchObject({
|
||||
status: 422,
|
||||
message: "GitHub could not verify this account's installation access",
|
||||
details: { code },
|
||||
});
|
||||
expect(request).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("rejects an empty next page before reporting an incomplete access snapshot", async () => {
|
||||
const request = vi.fn<typeof fetch>()
|
||||
.mockResolvedValueOnce(json({ id: 42, login: "octocat" }))
|
||||
.mockResolvedValueOnce(json({ installations: [] }, true));
|
||||
await expect(loadGitHubGrantMetadata("ghu_secret", request)).rejects.toMatchObject({
|
||||
details: { code: "github_bad_response" },
|
||||
});
|
||||
expect(request).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
});
|
||||
@@ -1,3 +1,5 @@
|
||||
import { loadGitHubGrantMetadata, loadGitHubTokenRepositories } from "./tool-access/github-access-discovery.js";
|
||||
export { loadGitHubGrantMetadata, loadGitHubTokenRepositories } from "./tool-access/github-access-discovery.js";
|
||||
import { getAppDefinitionForApplicationKey, isConnectorFeatureEnabled } from "@paperclipai/shared";
|
||||
import { stableHash, descriptorHash } from "./tool-catalog-hash.js";
|
||||
import { assertProfileEntryInputs } from "./tool-profile-entry-validation.js";
|
||||
@@ -2602,296 +2604,6 @@ function managedConnectorProfile(value: string | undefined): {
|
||||
return null;
|
||||
}
|
||||
|
||||
function gitHubRepositoryOwnerType(repository: Record<string, unknown>) {
|
||||
const type = recordValue(repository.owner) ? repository.owner.type : undefined;
|
||||
return type === "Organization" ? "organization" as const
|
||||
: type === "User" ? "personal" as const : undefined;
|
||||
}
|
||||
|
||||
export async function loadGitHubTokenRepositories(
|
||||
headers: Record<string, string>,
|
||||
request: typeof fetch = fetch,
|
||||
) {
|
||||
const repositories: Array<{
|
||||
id: string;
|
||||
fullName: string;
|
||||
ownerType?: "personal" | "organization";
|
||||
private?: boolean;
|
||||
}> = [];
|
||||
for (let page = 1; ; page += 1) {
|
||||
const response = await request(
|
||||
`https://api.github.com/user/repos?per_page=100&page=${page}`,
|
||||
{
|
||||
headers: {
|
||||
...headers,
|
||||
accept: "application/vnd.github+json",
|
||||
"user-agent": "Paperclip",
|
||||
"x-github-api-version": "2022-11-28",
|
||||
},
|
||||
signal: AbortSignal.timeout(15_000),
|
||||
},
|
||||
);
|
||||
if (!response.ok)
|
||||
throw unprocessable(
|
||||
"Could not load GitHub repositories. Reconnect GitHub and try again.",
|
||||
);
|
||||
const rows: unknown = await response.json();
|
||||
if (!Array.isArray(rows))
|
||||
throw unprocessable("GitHub returned invalid repositories");
|
||||
for (const row of rows) {
|
||||
if (
|
||||
!recordValue(row) ||
|
||||
!githubId(row.id) ||
|
||||
typeof row.full_name !== "string" ||
|
||||
!/^[A-Za-z0-9][A-Za-z0-9-]*\/(?!\.{1,2}$)[A-Za-z0-9_.-]+$/.test(
|
||||
row.full_name,
|
||||
)
|
||||
) {
|
||||
throw unprocessable("GitHub returned invalid repository metadata");
|
||||
}
|
||||
repositories.push({
|
||||
id: githubId(row.id)!,
|
||||
fullName: row.full_name,
|
||||
...(gitHubRepositoryOwnerType(row) ? { ownerType: gitHubRepositoryOwnerType(row) } : {}),
|
||||
...(typeof row.private === "boolean" ? { private: row.private } : {}),
|
||||
});
|
||||
}
|
||||
if (!/;\s*rel="next"/.test(response.headers.get("link") ?? ""))
|
||||
return repositories;
|
||||
if (!rows.length) throw unprocessable("GitHub returned invalid pagination");
|
||||
}
|
||||
}
|
||||
|
||||
export async function loadGitHubGrantMetadata(
|
||||
accessToken: string,
|
||||
request: typeof fetch = fetch,
|
||||
appSlug?: string,
|
||||
): Promise<{
|
||||
userId: string;
|
||||
login: string;
|
||||
avatarUrl?: string;
|
||||
installationCount: number;
|
||||
repositoryCount: number;
|
||||
repositorySelection: "all" | "selected" | "mixed" | "none";
|
||||
installationIds: string[];
|
||||
installationOwnerLogins: string[];
|
||||
repositories: Array<{
|
||||
id: string;
|
||||
fullName: string;
|
||||
installationId: string;
|
||||
ownerType?: "personal" | "organization";
|
||||
private?: boolean;
|
||||
}>;
|
||||
installationUrl: string;
|
||||
managementUrl: string;
|
||||
appSlug?: string;
|
||||
accessRevision: string;
|
||||
lastAccessRefreshAt: string;
|
||||
webhookHealth: "pending";
|
||||
}> {
|
||||
let resolvedAppSlug = appSlug;
|
||||
const accessRefreshStartedAt = new Date().toISOString();
|
||||
const github = async (
|
||||
path: string,
|
||||
): Promise<{ data: Record<string, unknown>; hasNext: boolean }> => {
|
||||
const response = await request(`https://api.github.com${path}`, {
|
||||
headers: {
|
||||
accept: "application/vnd.github+json",
|
||||
authorization: `Bearer ${accessToken}`,
|
||||
"user-agent": "Paperclip",
|
||||
"x-github-api-version": "2022-11-28",
|
||||
},
|
||||
signal: AbortSignal.timeout(15_000),
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw unprocessable(
|
||||
"GitHub could not verify this account's installation access",
|
||||
{
|
||||
code:
|
||||
response.status === 401
|
||||
? "oauth_reauthorization_required"
|
||||
: "github_access_check_failed",
|
||||
},
|
||||
);
|
||||
}
|
||||
const value = (await response.json()) as unknown;
|
||||
if (!recordValue(value))
|
||||
throw unprocessable("GitHub returned invalid account metadata", {
|
||||
code: "github_bad_response",
|
||||
});
|
||||
return {
|
||||
data: value,
|
||||
hasNext: /;\s*rel="next"/.test(response.headers.get("link") ?? ""),
|
||||
};
|
||||
};
|
||||
const list = async (
|
||||
path: string,
|
||||
key: string,
|
||||
): Promise<Record<string, unknown>[]> => {
|
||||
const items: Record<string, unknown>[] = [];
|
||||
for (let page = 1; ; page += 1) {
|
||||
const { data, hasNext } = await github(
|
||||
`${path}?per_page=100&page=${page}`,
|
||||
);
|
||||
const batch = data[key];
|
||||
if (
|
||||
!Array.isArray(batch) ||
|
||||
!batch.every(recordValue) ||
|
||||
(hasNext && batch.length === 0)
|
||||
) {
|
||||
throw unprocessable("GitHub returned invalid access metadata", {
|
||||
code: "github_bad_response",
|
||||
});
|
||||
}
|
||||
items.push(...batch);
|
||||
if (!hasNext) return items;
|
||||
}
|
||||
};
|
||||
const { data: user } = await github("/user");
|
||||
const userId = githubId(user.id);
|
||||
const login = typeof user.login === "string" ? user.login : null;
|
||||
if (!userId || !login)
|
||||
throw unprocessable("GitHub returned invalid account metadata", {
|
||||
code: "github_bad_response",
|
||||
});
|
||||
const installations = await list("/user/installations", "installations");
|
||||
const installationIds: string[] = [];
|
||||
const owners = new Set<string>();
|
||||
const selections = new Set<"all" | "selected">();
|
||||
const managementUrls = new Set<string>();
|
||||
const repositories = new Map<
|
||||
string,
|
||||
{ id: string; fullName: string; installationId: string; ownerType?: "personal" | "organization"; private?: boolean }
|
||||
>();
|
||||
for (const installation of installations) {
|
||||
const installationId = githubId(installation.id);
|
||||
if (!installationId) continue;
|
||||
installationIds.push(installationId);
|
||||
// Older grants predate the broker's appSlug field. GitHub's installation
|
||||
// response identifies this token's app without choosing an environment.
|
||||
if (
|
||||
!resolvedAppSlug &&
|
||||
typeof installation.app_slug === "string" &&
|
||||
/^[a-z0-9-]{1,100}$/.test(installation.app_slug)
|
||||
) {
|
||||
resolvedAppSlug = installation.app_slug;
|
||||
}
|
||||
if (
|
||||
installation.repository_selection === "all" ||
|
||||
installation.repository_selection === "selected"
|
||||
) {
|
||||
selections.add(installation.repository_selection);
|
||||
}
|
||||
const account = recordValue(installation.account)
|
||||
? installation.account
|
||||
: null;
|
||||
if (typeof account?.login === "string") owners.add(account.login);
|
||||
const managementUrl = githubInstallationManagementUrl(
|
||||
installation.html_url,
|
||||
);
|
||||
if (managementUrl) managementUrls.add(managementUrl);
|
||||
for (const repository of await list(
|
||||
`/user/installations/${installationId}/repositories`,
|
||||
"repositories",
|
||||
)) {
|
||||
const id = githubId(repository.id);
|
||||
const fullName =
|
||||
typeof repository.full_name === "string" ? repository.full_name : "";
|
||||
if (
|
||||
!id ||
|
||||
!/^[A-Za-z0-9][A-Za-z0-9-]*\/(?!\.{1,2}$)[A-Za-z0-9_.-]+$/.test(
|
||||
fullName,
|
||||
)
|
||||
) {
|
||||
throw unprocessable("GitHub returned invalid repository metadata", {
|
||||
code: "github_bad_response",
|
||||
});
|
||||
}
|
||||
repositories.set(id, {
|
||||
id,
|
||||
fullName,
|
||||
installationId,
|
||||
...(gitHubRepositoryOwnerType(repository) ? { ownerType: gitHubRepositoryOwnerType(repository) } : {}),
|
||||
...(typeof repository.private === "boolean"
|
||||
? { private: repository.private }
|
||||
: {}),
|
||||
});
|
||||
}
|
||||
}
|
||||
const repositoryCount = repositories.size;
|
||||
if (installationIds.length === 0 || repositoryCount === 0) {
|
||||
const installationUrl = resolvedAppSlug
|
||||
? `https://github.com/apps/${resolvedAppSlug}/installations/new`
|
||||
: "https://github.com/settings/installations";
|
||||
throw unprocessable(
|
||||
"GitHub access is required. Install Paperclip and grant at least one repository before refreshing access.",
|
||||
{
|
||||
code: "github_installation_required",
|
||||
installationUrl,
|
||||
managementUrl: "https://github.com/settings/installations",
|
||||
},
|
||||
);
|
||||
}
|
||||
const installationUrl = resolvedAppSlug
|
||||
? `https://github.com/apps/${resolvedAppSlug}/installations/new`
|
||||
: "https://github.com/settings/installations";
|
||||
return {
|
||||
userId,
|
||||
login,
|
||||
...(typeof user.avatar_url === "string"
|
||||
? { avatarUrl: user.avatar_url }
|
||||
: {}),
|
||||
installationCount: installationIds.length,
|
||||
repositoryCount,
|
||||
repositorySelection:
|
||||
selections.size > 1
|
||||
? "mixed"
|
||||
: (selections.values().next().value ?? "none"),
|
||||
installationIds,
|
||||
installationOwnerLogins: [...owners],
|
||||
repositories: [...repositories.values()].sort((a, b) =>
|
||||
a.fullName.localeCompare(b.fullName),
|
||||
),
|
||||
installationUrl,
|
||||
managementUrl:
|
||||
managementUrls.size === 1
|
||||
? managementUrls.values().next().value!
|
||||
: "https://github.com/settings/installations",
|
||||
...(resolvedAppSlug ? { appSlug: resolvedAppSlug } : {}),
|
||||
accessRevision: randomUUID(),
|
||||
lastAccessRefreshAt: accessRefreshStartedAt,
|
||||
webhookHealth: "pending",
|
||||
};
|
||||
}
|
||||
|
||||
function githubInstallationManagementUrl(value: unknown): string | null {
|
||||
if (typeof value !== "string" || value.length > 2_000) return null;
|
||||
try {
|
||||
const url = new URL(value);
|
||||
if (
|
||||
url.protocol !== "https:" ||
|
||||
url.hostname.toLowerCase() !== "github.com"
|
||||
)
|
||||
return null;
|
||||
return url.pathname.includes("/settings/installations/")
|
||||
? url.toString()
|
||||
: null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function githubId(value: unknown): string | null {
|
||||
if (typeof value === "number" && Number.isSafeInteger(value) && value > 0)
|
||||
return String(value);
|
||||
if (typeof value === "string" && /^[1-9][0-9]{0,30}$/.test(value))
|
||||
return value;
|
||||
return null;
|
||||
}
|
||||
|
||||
function recordValue(value: unknown): value is Record<string, unknown> {
|
||||
return typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
|
||||
@@ -0,0 +1,294 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { unprocessable } from "../../errors.js";
|
||||
|
||||
function gitHubRepositoryOwnerType(repository: Record<string, unknown>) {
|
||||
const type = recordValue(repository.owner) ? repository.owner.type : undefined;
|
||||
return type === "Organization" ? "organization" as const
|
||||
: type === "User" ? "personal" as const : undefined;
|
||||
}
|
||||
|
||||
export async function loadGitHubTokenRepositories(
|
||||
headers: Record<string, string>,
|
||||
request: typeof fetch = fetch,
|
||||
) {
|
||||
const repositories: Array<{
|
||||
id: string;
|
||||
fullName: string;
|
||||
ownerType?: "personal" | "organization";
|
||||
private?: boolean;
|
||||
}> = [];
|
||||
for (let page = 1; ; page += 1) {
|
||||
const response = await request(
|
||||
`https://api.github.com/user/repos?per_page=100&page=${page}`,
|
||||
{
|
||||
headers: {
|
||||
...headers,
|
||||
accept: "application/vnd.github+json",
|
||||
"user-agent": "Paperclip",
|
||||
"x-github-api-version": "2022-11-28",
|
||||
},
|
||||
signal: AbortSignal.timeout(15_000),
|
||||
},
|
||||
);
|
||||
if (!response.ok)
|
||||
throw unprocessable(
|
||||
"Could not load GitHub repositories. Reconnect GitHub and try again.",
|
||||
);
|
||||
const rows: unknown = await response.json();
|
||||
if (!Array.isArray(rows))
|
||||
throw unprocessable("GitHub returned invalid repositories");
|
||||
for (const row of rows) {
|
||||
if (
|
||||
!recordValue(row) ||
|
||||
!githubId(row.id) ||
|
||||
typeof row.full_name !== "string" ||
|
||||
!/^[A-Za-z0-9][A-Za-z0-9-]*\/(?!\.{1,2}$)[A-Za-z0-9_.-]+$/.test(
|
||||
row.full_name,
|
||||
)
|
||||
) {
|
||||
throw unprocessable("GitHub returned invalid repository metadata");
|
||||
}
|
||||
repositories.push({
|
||||
id: githubId(row.id)!,
|
||||
fullName: row.full_name,
|
||||
...(gitHubRepositoryOwnerType(row) ? { ownerType: gitHubRepositoryOwnerType(row) } : {}),
|
||||
...(typeof row.private === "boolean" ? { private: row.private } : {}),
|
||||
});
|
||||
}
|
||||
if (!/;\s*rel="next"/.test(response.headers.get("link") ?? ""))
|
||||
return repositories;
|
||||
if (!rows.length) throw unprocessable("GitHub returned invalid pagination");
|
||||
}
|
||||
}
|
||||
|
||||
export async function loadGitHubGrantMetadata(
|
||||
accessToken: string,
|
||||
request: typeof fetch = fetch,
|
||||
appSlug?: string,
|
||||
): Promise<{
|
||||
userId: string;
|
||||
login: string;
|
||||
avatarUrl?: string;
|
||||
installationCount: number;
|
||||
repositoryCount: number;
|
||||
repositorySelection: "all" | "selected" | "mixed" | "none";
|
||||
installationIds: string[];
|
||||
installationOwnerLogins: string[];
|
||||
repositories: Array<{
|
||||
id: string;
|
||||
fullName: string;
|
||||
installationId: string;
|
||||
ownerType?: "personal" | "organization";
|
||||
private?: boolean;
|
||||
}>;
|
||||
installationUrl: string;
|
||||
managementUrl: string;
|
||||
appSlug?: string;
|
||||
accessRevision: string;
|
||||
lastAccessRefreshAt: string;
|
||||
webhookHealth: "pending";
|
||||
}> {
|
||||
let resolvedAppSlug = appSlug;
|
||||
const accessRefreshStartedAt = new Date().toISOString();
|
||||
const github = async (
|
||||
path: string,
|
||||
): Promise<{ data: Record<string, unknown>; hasNext: boolean }> => {
|
||||
const response = await request(`https://api.github.com${path}`, {
|
||||
headers: {
|
||||
accept: "application/vnd.github+json",
|
||||
authorization: `Bearer ${accessToken}`,
|
||||
"user-agent": "Paperclip",
|
||||
"x-github-api-version": "2022-11-28",
|
||||
},
|
||||
signal: AbortSignal.timeout(15_000),
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw unprocessable(
|
||||
"GitHub could not verify this account's installation access",
|
||||
{
|
||||
code:
|
||||
response.status === 401
|
||||
? "oauth_reauthorization_required"
|
||||
: "github_access_check_failed",
|
||||
},
|
||||
);
|
||||
}
|
||||
const value = (await response.json()) as unknown;
|
||||
if (!recordValue(value))
|
||||
throw unprocessable("GitHub returned invalid account metadata", {
|
||||
code: "github_bad_response",
|
||||
});
|
||||
return {
|
||||
data: value,
|
||||
hasNext: /;\s*rel="next"/.test(response.headers.get("link") ?? ""),
|
||||
};
|
||||
};
|
||||
const list = async (
|
||||
path: string,
|
||||
key: string,
|
||||
): Promise<Record<string, unknown>[]> => {
|
||||
const items: Record<string, unknown>[] = [];
|
||||
for (let page = 1; ; page += 1) {
|
||||
const { data, hasNext } = await github(
|
||||
`${path}?per_page=100&page=${page}`,
|
||||
);
|
||||
const batch = data[key];
|
||||
if (
|
||||
!Array.isArray(batch) ||
|
||||
!batch.every(recordValue) ||
|
||||
(hasNext && batch.length === 0)
|
||||
) {
|
||||
throw unprocessable("GitHub returned invalid access metadata", {
|
||||
code: "github_bad_response",
|
||||
});
|
||||
}
|
||||
items.push(...batch);
|
||||
if (!hasNext) return items;
|
||||
}
|
||||
};
|
||||
const { data: user } = await github("/user");
|
||||
const userId = githubId(user.id);
|
||||
const login = typeof user.login === "string" ? user.login : null;
|
||||
if (!userId || !login)
|
||||
throw unprocessable("GitHub returned invalid account metadata", {
|
||||
code: "github_bad_response",
|
||||
});
|
||||
const installations = await list("/user/installations", "installations");
|
||||
const installationIds: string[] = [];
|
||||
const owners = new Set<string>();
|
||||
const selections = new Set<"all" | "selected">();
|
||||
const managementUrls = new Set<string>();
|
||||
const repositories = new Map<
|
||||
string,
|
||||
{ id: string; fullName: string; installationId: string; ownerType?: "personal" | "organization"; private?: boolean }
|
||||
>();
|
||||
for (const installation of installations) {
|
||||
const installationId = githubId(installation.id);
|
||||
if (!installationId) continue;
|
||||
installationIds.push(installationId);
|
||||
// Older grants predate the broker's appSlug field. GitHub's installation
|
||||
// response identifies this token's app without choosing an environment.
|
||||
if (
|
||||
!resolvedAppSlug &&
|
||||
typeof installation.app_slug === "string" &&
|
||||
/^[a-z0-9-]{1,100}$/.test(installation.app_slug)
|
||||
) {
|
||||
resolvedAppSlug = installation.app_slug;
|
||||
}
|
||||
if (
|
||||
installation.repository_selection === "all" ||
|
||||
installation.repository_selection === "selected"
|
||||
) {
|
||||
selections.add(installation.repository_selection);
|
||||
}
|
||||
const account = recordValue(installation.account)
|
||||
? installation.account
|
||||
: null;
|
||||
if (typeof account?.login === "string") owners.add(account.login);
|
||||
const managementUrl = githubInstallationManagementUrl(
|
||||
installation.html_url,
|
||||
);
|
||||
if (managementUrl) managementUrls.add(managementUrl);
|
||||
for (const repository of await list(
|
||||
`/user/installations/${installationId}/repositories`,
|
||||
"repositories",
|
||||
)) {
|
||||
const id = githubId(repository.id);
|
||||
const fullName =
|
||||
typeof repository.full_name === "string" ? repository.full_name : "";
|
||||
if (
|
||||
!id ||
|
||||
!/^[A-Za-z0-9][A-Za-z0-9-]*\/(?!\.{1,2}$)[A-Za-z0-9_.-]+$/.test(
|
||||
fullName,
|
||||
)
|
||||
) {
|
||||
throw unprocessable("GitHub returned invalid repository metadata", {
|
||||
code: "github_bad_response",
|
||||
});
|
||||
}
|
||||
repositories.set(id, {
|
||||
id,
|
||||
fullName,
|
||||
installationId,
|
||||
...(gitHubRepositoryOwnerType(repository) ? { ownerType: gitHubRepositoryOwnerType(repository) } : {}),
|
||||
...(typeof repository.private === "boolean"
|
||||
? { private: repository.private }
|
||||
: {}),
|
||||
});
|
||||
}
|
||||
}
|
||||
const repositoryCount = repositories.size;
|
||||
if (installationIds.length === 0 || repositoryCount === 0) {
|
||||
const installationUrl = resolvedAppSlug
|
||||
? `https://github.com/apps/${resolvedAppSlug}/installations/new`
|
||||
: "https://github.com/settings/installations";
|
||||
throw unprocessable(
|
||||
"GitHub access is required. Install Paperclip and grant at least one repository before refreshing access.",
|
||||
{
|
||||
code: "github_installation_required",
|
||||
installationUrl,
|
||||
managementUrl: "https://github.com/settings/installations",
|
||||
},
|
||||
);
|
||||
}
|
||||
const installationUrl = resolvedAppSlug
|
||||
? `https://github.com/apps/${resolvedAppSlug}/installations/new`
|
||||
: "https://github.com/settings/installations";
|
||||
return {
|
||||
userId,
|
||||
login,
|
||||
...(typeof user.avatar_url === "string"
|
||||
? { avatarUrl: user.avatar_url }
|
||||
: {}),
|
||||
installationCount: installationIds.length,
|
||||
repositoryCount,
|
||||
repositorySelection:
|
||||
selections.size > 1
|
||||
? "mixed"
|
||||
: (selections.values().next().value ?? "none"),
|
||||
installationIds,
|
||||
installationOwnerLogins: [...owners],
|
||||
repositories: [...repositories.values()].sort((a, b) =>
|
||||
a.fullName.localeCompare(b.fullName),
|
||||
),
|
||||
installationUrl,
|
||||
managementUrl:
|
||||
managementUrls.size === 1
|
||||
? managementUrls.values().next().value!
|
||||
: "https://github.com/settings/installations",
|
||||
...(resolvedAppSlug ? { appSlug: resolvedAppSlug } : {}),
|
||||
accessRevision: randomUUID(),
|
||||
lastAccessRefreshAt: accessRefreshStartedAt,
|
||||
webhookHealth: "pending",
|
||||
};
|
||||
}
|
||||
|
||||
function githubInstallationManagementUrl(value: unknown): string | null {
|
||||
if (typeof value !== "string" || value.length > 2_000) return null;
|
||||
try {
|
||||
const url = new URL(value);
|
||||
if (
|
||||
url.protocol !== "https:" ||
|
||||
url.hostname.toLowerCase() !== "github.com"
|
||||
)
|
||||
return null;
|
||||
return url.pathname.includes("/settings/installations/")
|
||||
? url.toString()
|
||||
: null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function githubId(value: unknown): string | null {
|
||||
if (typeof value === "number" && Number.isSafeInteger(value) && value > 0)
|
||||
return String(value);
|
||||
if (typeof value === "string" && /^[1-9][0-9]{0,30}$/.test(value))
|
||||
return value;
|
||||
return null;
|
||||
}
|
||||
|
||||
function recordValue(value: unknown): value is Record<string, unknown> {
|
||||
return typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user