mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-13 07:12:00 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task conversations contain links to pull requests and other work. > - A URL alone cannot tell the operator whether a PR is open, merged, or blocked by conflicts. > - Fetching every link at render time wastes API calls and can expose private data through a shared cache. > - This pull request adds a demand-driven links service and a GitHub PR adapter. > - Each viewer and company has a separate cache. Repeated links share one request and result. > - Operators get context when they hover or focus a link. Normal links stay useful when a provider is unavailable. ## Linked Issues or Issue Description **Subsystem affected** Task Markdown links, shared link metadata, and the server GitHub API boundary. **Problem or motivation** PR links have compact labels and neutral icons. They do not provide a rich hover card or a common status source across repeated mentions. A background resolver cannot borrow a person's GitHub connection because it has no viewer identity. **Proposed solution** Recognize supported links locally. Request metadata after a delayed hover or keyboard focus. Show a card at the first line of the link, aligned with its icon. Share the result across all mentions in the current viewer and company. Keep the exact original navigation URL. The first adapter supports GitHub pull requests. Open is green, merged is purple, and merge conflicts are orange. Draft, closed, unknown, loading and unavailable states remain quiet. **Alternatives considered** Eager page scans would spend API quota on links the operator does not inspect. Company-wide caching of personal GitHub results would cross the credential boundary. This change uses neither approach. **Roadmap alignment** ROADMAP.md was checked. This extends existing task links and the GitHub resolver. It adds no deployment adapter or new background worker. Searches of open and closed PRs for link previews and PR hover cards found no matching implementation. ## What Changed - Add host-owned link recognition and a normalized preview contract. The service selects an adapter by provider. - Add a board-only company read endpoint. It checks company access before cache reads and sends a private, no-store response. - Use the current viewer's eligible GitHub connection. A configured connection that cannot supply credentials fails closed. With no managed connection, reuse existing company credentials or public access. Instance credentials retain the public-repository check. - Reuse the GitHub external-object resolver for API transport and response parsing. Add title and mergeability to its normalized snapshot without changing its existing state contract. - Coalesce requests by company, viewer and canonical PR URL. Bound cached entries and concurrent work. Cache success for one minute and failures for a short period or the provider retry window. - Add an abortable request deadline. Timed-out work stays in the concurrency budget until it settles. No periodic worker or idle polling is added. - Add a shared browser query and anchored preview card. Repeated Markdown links observe the same result. There are no automatic retries, window-focus refreshes, toasts or connection prompts. - Add ten production-component Storybook examples and real pointer/keyboard browser checks. Document the API and verification route. ## Verification - Linux source checkout: 233 focused tests passed in nine suites. They cover recognition, fixed API destinations, provider states, access failures, credential selection, viewer/company isolation, request coalescing, expiry, cache bounds, concurrency bounds and mounted OpenAPI coverage. - A live public GitHub smoke returned this PR's correct title and open state. Two equivalent URLs shared one real network request. Personal connection access was not part of that smoke. - A controlled HTTP server stalls a real response body. The test proves cancellation while another lookup completes. - UI typecheck and the server layered typecheck passed. The UI production build passed. - Token gates and query policy gates passed. All 30 map/catalog support checks passed. The three new server test files also passed a focused TypeScript check. - All 27 focused browser checks passed on the reviewed source tree. They cover light/dark themes, narrow layouts, open/merged/conflict/draft/closed/loading/unavailable/failed states, fixed positioning during pointer movement, exact navigation and keyboard access. A real browser HTTP interception counts one deferred lookup for two equivalent PR links. - The first browser attempt lacked Chromium. A later combined build/test attempt exhausted the sandbox's 8 GiB memory limit and killed Storybook. Those attempts supply no pass evidence. Chromium is installed, and the final browser checks run with the build stopped. One initial Storybook dependency reload interrupted a streaming check. All eight existing inline-link cases passed on the warmed server, followed by a complete 27-case pass. - Synthetic Storybook pointer events did not exercise the native hover boundary. The assertions now run through real Playwright pointer and keyboard actions. The fixture stories retain the production components. - Review fixes preserve the Draft label when GitHub reports both draft and conflicts. They also preserve neutral link colors in dark mode. The 25 affected service checks pass, including the added combined-state regression. Browser checks compare unknown, draft and closed links with the muted theme color. - The first CI policy run rejected a literal retry opt-out. The code now uses the existing explicit zero-retry helper. A new test-spy type error was also corrected. These failures are fixed; their earlier runs remain failed. - `pnpm verify --explain` returned exit 2 because the browser test path selects broad CI. No full local suite is claimed. The final CI run supplies that coverage. - The previous head `437913c797963aa113bb82e2ae8b836969317ef9` passed CI and received a 5/5 Greptile score. All four review threads are resolved. Master then advanced. The current head `ae2a1c2adbd61a621cca9bdd160561952613df6e` resolves the source-manifest conflict against master `9b21dcbdda01296349f3cfe0a3a6891e79df6130`. Fresh CI and review are running. - The previous CI run found a lost class on decorated PR links and a stale SDK source manifest. Both are fixed. The existing CommentThread integration test and 26 focused link service tests pass. All 45 SDK source/privacy tests pass. All eight source-manifest helper tests pass. The manifest helper changed only source identities and added the two new shared files. Runtime, compiler, parser and library pins stay unchanged. - A later review found that expired timeouts could reuse a settled promise. The service now removes settled requests from its shared-request map and counts unfinished provider work separately. The new regression failed before the fix and passes after it. The concurrency-bound and real stalled-HTTP tests also pass. - UI typecheck and token gates pass after these fixes. The browser rerun needed Chromium in the new session. That first attempt did not exercise the UI. The complete 27-case browser rerun passed before the master merge with retries disabled. All 27 browser cases also passed on the merge revision with retries disabled. - After the master merge, all 84 focused link, OpenAPI and integration tests pass. All 55 SDK source/privacy/manifest tests pass. UI typecheck and token gates pass again. The source manifest retains master's reviewed runtime pins and includes both sets of source changes. GitHub confirms the PR can merge cleanly. The preflight helper rebuilt the shared package and plugin SDK; Cargo is unavailable locally. Final CI supplies Rust coverage. - The latest master update changed the reviewed report-lock policy but left the SDK lock hash at its previous value. The SDK source/evidence checks reproduced the mismatch. This change sets only the SDK lock hash to the authenticated output of the existing master policy and patch. Compiler, parser and Zod pins stay unchanged. All 56 SDK source/evidence/manifest tests now pass. All 84 focused link/API/integration tests, UI typecheck and token gates also pass on the updated source. ## Risks - This cache is process-local. Separate server replicas can make separate GitHub requests. - A cached result can remain visible for its short lifetime after GitHub changes. A later hover refreshes expired data. - GitHub can report unknown mergeability. The UI shows conflicts only when GitHub explicitly reports them. - Personal preview data stays in viewer/company caches. It is not written to company external-object records or persisted browser query storage. - Stories simulate the API. They do not prove a live GitHub login or private-repository access. - No schema migration is required. ## Model Used OpenAI Codex, based on GPT-6. The runtime does not expose the exact serving model ID or context-window limit. Used reasoning, repository inspection, source editing, shell execution and browser tools. No subagents were used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
653 lines
26 KiB
TypeScript
653 lines
26 KiB
TypeScript
import { createHash } from "node:crypto";
|
|
import type { Db } from "@paperclipai/db";
|
|
import type { ExternalObjectCanonicalUrl } from "@paperclipai/shared";
|
|
import { DEFAULT_GITHUB_TOKEN_SECRET_NAMES, resolveGitHubApiCredential, type GitHubApiCredential } from "./git-credentials.js";
|
|
import { ghFetch, gitHubApiBase } from "./github-fetch.js";
|
|
import type {
|
|
ExternalObjectDetection,
|
|
ExternalObjectDetector,
|
|
ExternalObjectResolver,
|
|
ExternalObjectResolverSnapshot,
|
|
ExternalObjectResolveResult,
|
|
} from "./external-objects.js";
|
|
|
|
type FetchLike = (url: string, init?: RequestInit) => Promise<Response>;
|
|
|
|
type GitHubObjectResolver = Omit<ExternalObjectResolver, "resolve"> & {
|
|
resolve(input: {
|
|
companyId: string;
|
|
object: { externalId: string; sanitizedCanonicalUrl: string | null };
|
|
signal?: AbortSignal;
|
|
}): Promise<ExternalObjectResolveResult>;
|
|
};
|
|
|
|
export interface GitHubExternalObjectProviderOptions {
|
|
fetch?: FetchLike;
|
|
requestTimeoutMs?: number;
|
|
tokenProvider?: (companyId: string) => Promise<string | null> | string | null;
|
|
/**
|
|
* Resolve the credential and its source. Prefer this over `tokenProvider` when the
|
|
* caller must treat a server-environment token differently from a company credential.
|
|
*/
|
|
credentialProvider?: (companyId: string) => Promise<GitHubApiCredential | null> | GitHubApiCredential | null;
|
|
secretNames?: readonly string[];
|
|
}
|
|
|
|
interface GitHubObjectIdentity {
|
|
host: string;
|
|
owner: string;
|
|
repo: string;
|
|
number: number;
|
|
objectType: "pull_request" | "issue";
|
|
pathKind: "pull" | "issues";
|
|
}
|
|
|
|
const GITHUB_OBJECT_TTL_SECONDS = 300;
|
|
|
|
function isGitHubHost(host: string) {
|
|
const h = host.toLowerCase();
|
|
return h === "github.com" || h === "www.github.com";
|
|
}
|
|
|
|
function asRecord(value: unknown): Record<string, unknown> | null {
|
|
return value && typeof value === "object" && !Array.isArray(value) ? value as Record<string, unknown> : null;
|
|
}
|
|
|
|
function asString(value: unknown) {
|
|
return typeof value === "string" ? value : null;
|
|
}
|
|
|
|
function asBoolean(value: unknown) {
|
|
return typeof value === "boolean" ? value : null;
|
|
}
|
|
|
|
function asNonNegativeInteger(value: unknown) {
|
|
return typeof value === "number" && Number.isSafeInteger(value) && value >= 0 ? value : null;
|
|
}
|
|
|
|
function asNestedString(record: Record<string, unknown>, key: string, nestedKey: string) {
|
|
const nested = asRecord(record[key]);
|
|
return nested ? asString(nested[nestedKey]) : null;
|
|
}
|
|
|
|
function normalizeCredential(credential: GitHubApiCredential | null | undefined): GitHubApiCredential | null {
|
|
const token = credential?.token?.trim();
|
|
if (!token) return null;
|
|
return { token, source: credential!.source, secretName: credential!.secretName ?? null };
|
|
}
|
|
|
|
function parseGitHubCanonicalUrl(canonical: ExternalObjectCanonicalUrl): GitHubObjectIdentity | null {
|
|
if (canonical.canonicalIdentity.scheme !== "https") return null;
|
|
const host = canonical.canonicalIdentity.host.toLowerCase();
|
|
if (!isGitHubHost(host)) return null;
|
|
|
|
const parts = canonical.canonicalIdentity.path.split("/").filter(Boolean);
|
|
if (parts.length !== 4) return null;
|
|
const [owner, repo, kind, rawNumber] = parts;
|
|
if (!owner || !repo || !kind || !rawNumber) return null;
|
|
if (!/^[A-Za-z0-9_.-]+$/.test(owner) || !/^[A-Za-z0-9_.-]+$/.test(repo)) return null;
|
|
if (!/^[1-9][0-9]*$/.test(rawNumber)) return null;
|
|
if (kind !== "pull" && kind !== "issues") return null;
|
|
|
|
return {
|
|
host: host === "www.github.com" ? "github.com" : host,
|
|
owner,
|
|
repo,
|
|
number: Number(rawNumber),
|
|
pathKind: kind,
|
|
objectType: kind === "pull" ? "pull_request" : "issue",
|
|
};
|
|
}
|
|
|
|
function parseGitHubObject(object: { externalId: string; sanitizedCanonicalUrl: string | null }): GitHubObjectIdentity | null {
|
|
const match = /^([^/]+)\/([^/]+)#(pull|issues)\/([1-9][0-9]*)$/.exec(object.externalId);
|
|
if (!match) return null;
|
|
let host = "github.com";
|
|
if (object.sanitizedCanonicalUrl) {
|
|
try {
|
|
const url = new URL(object.sanitizedCanonicalUrl);
|
|
if (isGitHubHost(url.hostname)) host = url.hostname === "www.github.com" ? "github.com" : url.hostname;
|
|
} catch {
|
|
return null;
|
|
}
|
|
}
|
|
return {
|
|
host,
|
|
owner: match[1]!,
|
|
repo: match[2]!,
|
|
pathKind: match[3] as "pull" | "issues",
|
|
number: Number(match[4]),
|
|
objectType: match[3] === "pull" ? "pull_request" : "issue",
|
|
};
|
|
}
|
|
|
|
function externalIdFor(identity: GitHubObjectIdentity) {
|
|
return `${identity.owner.toLowerCase()}/${identity.repo.toLowerCase()}#${identity.pathKind}/${identity.number}`;
|
|
}
|
|
|
|
function displayTitleFor(identity: GitHubObjectIdentity) {
|
|
return `${identity.owner}/${identity.repo}#${identity.number}`;
|
|
}
|
|
|
|
function displayKeyFor(identity: Pick<GitHubObjectIdentity, "objectType">) {
|
|
return identity.objectType === "pull_request" ? "GitHub Pull Request" : "GitHub Issue";
|
|
}
|
|
|
|
function retryAfterSeconds(response: Response) {
|
|
const retryAfter = response.headers.get("retry-after");
|
|
if (retryAfter && /^[0-9]+$/.test(retryAfter)) return Number(retryAfter);
|
|
|
|
const reset = response.headers.get("x-ratelimit-reset");
|
|
if (reset && /^[0-9]+$/.test(reset)) {
|
|
return Math.max(1, Number(reset) - Math.floor(Date.now() / 1000));
|
|
}
|
|
|
|
return 300;
|
|
}
|
|
|
|
function rateLimitedResult(authenticated: boolean, retryAfterSeconds: number): ExternalObjectResolveResult {
|
|
return {
|
|
ok: false,
|
|
liveness: "rate_limited",
|
|
errorCode: "github_rate_limited",
|
|
errorMessage: authenticated
|
|
? "GitHub rate limit reached while refreshing this object. It will retry after the limit resets."
|
|
: "GitHub rate limit reached while refreshing this object because no GitHub credential is configured, so requests are anonymous. Add a GitHub token to raise the limit.",
|
|
retryAfterSeconds,
|
|
};
|
|
}
|
|
|
|
function failureFromGitHubResponse(response: Response, authenticated: boolean): ExternalObjectResolveResult | null {
|
|
if (response.status === 401) {
|
|
return {
|
|
ok: false,
|
|
liveness: "auth_required",
|
|
errorCode: "github_auth_required",
|
|
errorMessage: "GitHub authentication is required to refresh this object.",
|
|
retryAfterSeconds: retryAfterSeconds(response),
|
|
};
|
|
}
|
|
|
|
if (response.status === 403) {
|
|
const rateLimitRemaining = response.headers.get("x-ratelimit-remaining");
|
|
if (rateLimitRemaining === "0") {
|
|
return rateLimitedResult(authenticated, retryAfterSeconds(response));
|
|
}
|
|
return {
|
|
ok: false,
|
|
liveness: "auth_required",
|
|
errorCode: "github_forbidden",
|
|
errorMessage: "GitHub rejected the configured credentials for this object.",
|
|
retryAfterSeconds: retryAfterSeconds(response),
|
|
};
|
|
}
|
|
|
|
if (response.status === 429) {
|
|
return rateLimitedResult(authenticated, retryAfterSeconds(response));
|
|
}
|
|
|
|
if (response.status >= 500) {
|
|
return {
|
|
ok: false,
|
|
liveness: "unreachable",
|
|
errorCode: "github_unreachable",
|
|
errorMessage: `GitHub returned HTTP ${response.status} while refreshing this object.`,
|
|
retryAfterSeconds: retryAfterSeconds(response),
|
|
};
|
|
}
|
|
|
|
return null;
|
|
}
|
|
|
|
function notFoundSnapshot(identity: GitHubObjectIdentity, etag: string | null): ExternalObjectResolverSnapshot {
|
|
return {
|
|
displayKey: displayKeyFor(identity),
|
|
iconKey: "github",
|
|
displayTitle: displayTitleFor(identity),
|
|
statusKey: "not_found",
|
|
statusLabel: "Not found",
|
|
statusIconKey: "archive",
|
|
statusCategory: "archived",
|
|
statusTone: "muted",
|
|
isTerminal: true,
|
|
etag,
|
|
ttlSeconds: GITHUB_OBJECT_TTL_SECONDS,
|
|
data: {
|
|
provider: "github",
|
|
owner: identity.owner,
|
|
repo: identity.repo,
|
|
number: identity.number,
|
|
notFound: true,
|
|
// The repository was probed with the configured credentials and found
|
|
// to be visible, so this not-found verdict is definitive, not a stale
|
|
// guess from a time when credentials could not see the repo. The due
|
|
// sweep uses this flag to avoid re-polling a confirmed-deleted object
|
|
// forever and exhausting the company's GitHub API budget.
|
|
probeVerified: true,
|
|
},
|
|
};
|
|
}
|
|
|
|
/**
|
|
* Probe `/repos/{owner}/{repo}` to classify a 404 on a pull request or issue, and to
|
|
* learn whether the repository is private.
|
|
*
|
|
* GitHub returns 404 both when the object does not exist and when the repository
|
|
* exists but the credentials cannot see it (private repos are deliberately hidden
|
|
* behind 404). This probes the repo metadata with the same credentials to tell the
|
|
* two apart:
|
|
*
|
|
* - "visible": the repo returns 2xx — the referenced object really is gone. The
|
|
* response also carries the repo's `private` flag.
|
|
* - "hidden": the repo returns 404/403 — the credentials lack access to the repo,
|
|
* so the object status cannot be determined and should not be frozen as "Not found".
|
|
* - "rate_limited": the probe was rejected by a GitHub rate limit — retry later, with a
|
|
* distinct state so a quota problem is not reported as a network failure.
|
|
* - "uncertain": the probe itself failed (network/other) — retry later.
|
|
*/
|
|
type RepoProbeOutcome = {
|
|
visibility: "visible" | "hidden" | "rate_limited" | "uncertain";
|
|
isPrivate: boolean | null;
|
|
};
|
|
|
|
async function probeRepoVisibility(
|
|
identity: GitHubObjectIdentity,
|
|
opts: { fetch: FetchLike; headers?: Record<string, string> | undefined; signal?: AbortSignal },
|
|
): Promise<RepoProbeOutcome> {
|
|
try {
|
|
const probe = await opts.fetch(
|
|
`${gitHubApiBase(identity.host)}/repos/${encodeURIComponent(identity.owner)}/${encodeURIComponent(identity.repo)}`,
|
|
{ headers: opts.headers, signal: opts.signal },
|
|
);
|
|
if (probe.ok) {
|
|
const body = await safeJson(probe);
|
|
return { visibility: "visible", isPrivate: typeof body?.private === "boolean" ? body.private : null };
|
|
}
|
|
if (probe.status === 404) return { visibility: "hidden", isPrivate: null };
|
|
if (probe.status === 429) return { visibility: "rate_limited", isPrivate: null };
|
|
if (probe.status === 403) {
|
|
// GitHub can return 403 for both "credentials lack access" and "rate limit
|
|
// reached". A rate-limited probe cannot tell the two apart, so treat it as
|
|
// a rate limit (retry later) instead of implying missing access.
|
|
const rateLimitRemaining = probe.headers.get("x-ratelimit-remaining");
|
|
if (rateLimitRemaining === "0") return { visibility: "rate_limited", isPrivate: null };
|
|
return { visibility: "hidden", isPrivate: null };
|
|
}
|
|
return { visibility: "uncertain", isPrivate: null };
|
|
} catch {
|
|
return { visibility: "uncertain", isPrivate: null };
|
|
}
|
|
}
|
|
|
|
function pullRequestSnapshot(identity: GitHubObjectIdentity, body: Record<string, unknown>, etag: string | null): ExternalObjectResolverSnapshot {
|
|
const title = asString(body.title);
|
|
const state = asString(body.state) ?? "unknown";
|
|
const draft = asBoolean(body.draft) ?? false;
|
|
const merged = (asBoolean(body.merged) ?? false) || Boolean(asString(body.merged_at));
|
|
const authorLogin = asNestedString(body, "user", "login");
|
|
const headRef = asNestedString(body, "head", "ref");
|
|
const headSha = asNestedString(body, "head", "sha");
|
|
const baseRef = asNestedString(body, "base", "ref");
|
|
const reviewDecision = asString(body.review_decision);
|
|
const additions = asNonNegativeInteger(body.additions);
|
|
const deletions = asNonNegativeInteger(body.deletions);
|
|
const changedFiles = asNonNegativeInteger(body.changed_files);
|
|
|
|
let statusKey = state;
|
|
let statusLabel = state === "open" ? "Open" : state === "closed" ? "Closed" : "Unknown";
|
|
let statusCategory: ExternalObjectResolverSnapshot["statusCategory"] = state === "open" ? "open" : "unknown";
|
|
let statusTone: ExternalObjectResolverSnapshot["statusTone"] = state === "open" ? "info" : "neutral";
|
|
let isTerminal = false;
|
|
|
|
if (merged) {
|
|
statusKey = "merged";
|
|
statusLabel = "Merged";
|
|
statusCategory = "succeeded";
|
|
statusTone = "success";
|
|
isTerminal = true;
|
|
} else if (state === "closed") {
|
|
statusKey = "closed";
|
|
statusLabel = "Closed";
|
|
statusCategory = "closed";
|
|
statusTone = "muted";
|
|
isTerminal = true;
|
|
} else if (draft) {
|
|
statusKey = "draft";
|
|
statusLabel = "Draft";
|
|
statusCategory = "waiting";
|
|
statusTone = "warning";
|
|
}
|
|
|
|
return {
|
|
displayKey: displayKeyFor(identity),
|
|
iconKey: "github",
|
|
displayTitle: title ? `${displayTitleFor(identity)}: ${title}` : displayTitleFor(identity),
|
|
statusKey,
|
|
statusLabel,
|
|
statusIconKey: merged
|
|
? "git-merge"
|
|
: state === "closed"
|
|
? "x-circle"
|
|
: draft
|
|
? "clock"
|
|
: "git-pull-request",
|
|
statusCategory,
|
|
statusTone,
|
|
isTerminal,
|
|
remoteVersion: asString(body.updated_at),
|
|
etag,
|
|
ttlSeconds: GITHUB_OBJECT_TTL_SECONDS,
|
|
data: {
|
|
provider: "github",
|
|
owner: identity.owner,
|
|
repo: identity.repo,
|
|
number: identity.number,
|
|
state,
|
|
merged,
|
|
draft,
|
|
title,
|
|
mergeable: asBoolean(body.mergeable),
|
|
...(authorLogin ? { authorLogin } : {}),
|
|
...(headRef ? { headRef } : {}),
|
|
...(headSha ? { headSha } : {}),
|
|
...(baseRef ? { baseRef } : {}),
|
|
...(reviewDecision ? { reviewDecision } : {}),
|
|
...(additions !== null ? { additions } : {}),
|
|
...(deletions !== null ? { deletions } : {}),
|
|
...(changedFiles !== null ? { changedFiles } : {}),
|
|
},
|
|
};
|
|
}
|
|
|
|
function issueSnapshot(identity: GitHubObjectIdentity, body: Record<string, unknown>, etag: string | null): ExternalObjectResolverSnapshot {
|
|
const title = asString(body.title);
|
|
const state = asString(body.state) ?? "unknown";
|
|
const stateReason = asString(body.state_reason);
|
|
const authorLogin = asNestedString(body, "user", "login");
|
|
const statusKey = state === "closed" && stateReason ? `closed_${stateReason}` : state;
|
|
const statusLabel = state === "closed"
|
|
? stateReason
|
|
? `Closed: ${stateReason.replace(/_/g, " ")}`
|
|
: "Closed"
|
|
: state === "open"
|
|
? "Open"
|
|
: "Unknown";
|
|
|
|
return {
|
|
displayKey: displayKeyFor(identity),
|
|
iconKey: "github",
|
|
displayTitle: title ? `${displayTitleFor(identity)}: ${title}` : displayTitleFor(identity),
|
|
statusKey,
|
|
statusLabel,
|
|
statusIconKey: state === "closed" ? "circle" : "circle-dot",
|
|
statusCategory: state === "open" ? "open" : state === "closed" ? "closed" : "unknown",
|
|
statusTone: state === "open" ? "info" : state === "closed" ? "muted" : "neutral",
|
|
isTerminal: state === "closed",
|
|
remoteVersion: asString(body.updated_at),
|
|
etag,
|
|
ttlSeconds: GITHUB_OBJECT_TTL_SECONDS,
|
|
data: {
|
|
provider: "github",
|
|
owner: identity.owner,
|
|
repo: identity.repo,
|
|
number: identity.number,
|
|
state,
|
|
...(stateReason ? { stateReason } : {}),
|
|
...(authorLogin ? { authorLogin } : {}),
|
|
},
|
|
};
|
|
}
|
|
|
|
async function safeJson(response: Response) {
|
|
try {
|
|
return asRecord(await response.json());
|
|
} catch {
|
|
return null;
|
|
}
|
|
}
|
|
|
|
export function createGitHubExternalObjectProvider(
|
|
db: Db,
|
|
opts: GitHubExternalObjectProviderOptions = {},
|
|
): { detector: ExternalObjectDetector; resolvers: GitHubObjectResolver[] } {
|
|
const fetchImpl = opts.fetch ?? ghFetch;
|
|
const secretNames = opts.secretNames ?? DEFAULT_GITHUB_TOKEN_SECRET_NAMES;
|
|
// `credentialProvider` is the preferred seam; `tokenProvider` stays for callers that
|
|
// only care about the token string. A token from `tokenProvider` is treated as
|
|
// company-scoped, because the caller supplied it explicitly.
|
|
const credentialProvider: (companyId: string) => Promise<GitHubApiCredential | null> =
|
|
opts.credentialProvider
|
|
? async (companyId: string) => normalizeCredential(await opts.credentialProvider!(companyId))
|
|
: Object.prototype.hasOwnProperty.call(opts, "tokenProvider") && opts.tokenProvider !== undefined
|
|
? async (companyId: string) => {
|
|
const raw = typeof opts.tokenProvider === "function" ? await opts.tokenProvider(companyId) : opts.tokenProvider;
|
|
const token = raw?.trim();
|
|
return token ? { token, source: "company_secret", secretName: null } : null;
|
|
}
|
|
: (companyId: string) => resolveGitHubApiCredential(db, companyId, { secretNames });
|
|
// Cache repository-visibility probe results so many objects in the same repo
|
|
// share a single probe instead of each issuing its own request every refresh
|
|
// cycle. Only determined outcomes ("visible"/"hidden") are cached; uncertain
|
|
// probes are retried on the next cycle. The cache is keyed per company+repo+
|
|
// credential fingerprint so a verdict obtained under one token is never
|
|
// reused after the company rotates its credentials — that would freeze a
|
|
// reference as "probe-verified not-found" using an out-of-date result.
|
|
|
|
const probeCache = new Map<string, { outcome: RepoProbeOutcome; expiresAt: number }>();
|
|
async function cachedProbeRepoVisibility(
|
|
companyId: string,
|
|
identity: GitHubObjectIdentity,
|
|
headers: Record<string, string>,
|
|
signal: AbortSignal,
|
|
): Promise<RepoProbeOutcome> {
|
|
const credentialFingerprint = createHash("sha256").update(headers.authorization ?? "anonymous").digest("hex");
|
|
const key = `${companyId}\u0000${credentialFingerprint}\u0000${identity.owner.toLowerCase()}\u0000${identity.repo.toLowerCase()}`;
|
|
const cached = probeCache.get(key);
|
|
if (cached && cached.expiresAt > Date.now()) return cached.outcome;
|
|
const outcome = await probeRepoVisibility(identity, { fetch: fetchImpl, headers, signal });
|
|
if (outcome.visibility === "visible" || outcome.visibility === "hidden") {
|
|
probeCache.set(key, { outcome, expiresAt: Date.now() + GITHUB_OBJECT_TTL_SECONDS * 1000 });
|
|
}
|
|
return outcome;
|
|
}
|
|
|
|
const detector: ExternalObjectDetector = {
|
|
key: "github",
|
|
detect({ urls }): ExternalObjectDetection[] {
|
|
return urls.flatMap((canonical) => {
|
|
const identity = parseGitHubCanonicalUrl(canonical);
|
|
if (!identity) return [];
|
|
return [{
|
|
canonical,
|
|
detectorKey: "github",
|
|
providerKey: "github",
|
|
objectType: identity.objectType,
|
|
externalId: externalIdFor(identity),
|
|
displayKey: displayKeyFor(identity),
|
|
iconKey: "github",
|
|
displayTitle: displayTitleFor(identity),
|
|
confidence: "exact",
|
|
}];
|
|
});
|
|
},
|
|
};
|
|
|
|
function resolver(objectType: GitHubObjectIdentity["objectType"]): GitHubObjectResolver {
|
|
return {
|
|
providerKey: "github",
|
|
objectType,
|
|
async resolve({ companyId, object, signal: ownerSignal }) {
|
|
const deadline = AbortSignal.timeout(opts.requestTimeoutMs ?? 10_000);
|
|
const signal = ownerSignal ? AbortSignal.any([ownerSignal, deadline]) : deadline;
|
|
const identity = parseGitHubObject(object);
|
|
if (!identity || identity.objectType !== objectType) {
|
|
return {
|
|
ok: false,
|
|
liveness: "unreachable",
|
|
errorCode: "github_invalid_identity",
|
|
errorMessage: "GitHub object identity is invalid.",
|
|
retryAfterSeconds: GITHUB_OBJECT_TTL_SECONDS,
|
|
};
|
|
}
|
|
|
|
let credential: GitHubApiCredential | null = null;
|
|
try {
|
|
credential = await credentialProvider(companyId);
|
|
} catch {
|
|
return {
|
|
ok: false,
|
|
liveness: "auth_required",
|
|
errorCode: "github_token_unavailable",
|
|
errorMessage: "Configured GitHub credentials could not be resolved.",
|
|
retryAfterSeconds: GITHUB_OBJECT_TTL_SECONDS,
|
|
};
|
|
}
|
|
const token = credential?.token?.trim() || null;
|
|
const authenticated = Boolean(token);
|
|
// A server-environment token is instance-wide, so it can see repositories this
|
|
// company cannot. Only a company-scoped credential may resolve a private
|
|
// repository for company display; a server token is limited to public repos.
|
|
const companyScoped = credential?.source !== "server_env";
|
|
const headers: Record<string, string> = {
|
|
accept: "application/vnd.github+json",
|
|
"user-agent": "paperclip-external-object-resolver",
|
|
"x-github-api-version": "2022-11-28",
|
|
};
|
|
if (token) headers.authorization = `Bearer ${token}`;
|
|
|
|
if (!companyScoped) {
|
|
// A server-environment token is instance-wide, so it can read repositories
|
|
// this company cannot. Before reading with it, verify that the repository is
|
|
// public RIGHT NOW. Never serve this authorization decision from the probe
|
|
// cache: a repository can become private inside the cache TTL, and a cached
|
|
// "public" verdict must not authorize a later read of its private contents.
|
|
// The cache is still used below to classify a 404, where a stale verdict can
|
|
// only make the object look deleted — it can never expose private data.
|
|
const probe = await probeRepoVisibility(identity, { fetch: fetchImpl, headers, signal });
|
|
if (probe.visibility === "hidden") {
|
|
return {
|
|
ok: false,
|
|
liveness: "auth_required",
|
|
errorCode: "github_repo_not_accessible",
|
|
errorMessage: `GitHub returned not-found for ${identity.owner}/${identity.repo}. The repository may be private, deleted, or the configured GitHub credentials may lack access to it.`,
|
|
retryAfterSeconds: GITHUB_OBJECT_TTL_SECONDS,
|
|
};
|
|
}
|
|
if (probe.visibility === "rate_limited") {
|
|
return rateLimitedResult(authenticated, GITHUB_OBJECT_TTL_SECONDS);
|
|
}
|
|
if (probe.visibility === "uncertain") {
|
|
return {
|
|
ok: false,
|
|
liveness: "unreachable",
|
|
errorCode: "github_repo_probe_failed",
|
|
errorMessage: "GitHub could not verify this repository while checking this object.",
|
|
retryAfterSeconds: GITHUB_OBJECT_TTL_SECONDS,
|
|
};
|
|
}
|
|
// Require positive proof that the repository is public before reading
|
|
// with the instance-wide server token. A private verdict and a probe
|
|
// response that does not confirm public visibility both fail closed, so
|
|
// the server token can never fetch private details for a company that
|
|
// did not authorize it.
|
|
if (probe.isPrivate !== false) {
|
|
return {
|
|
ok: false,
|
|
liveness: "auth_required",
|
|
errorCode: "github_repo_requires_company_token",
|
|
errorMessage: probe.isPrivate === true
|
|
? `The ${identity.owner}/${identity.repo} repository is private. Add a GitHub token for this company to refresh this object.`
|
|
: `The ${identity.owner}/${identity.repo} repository visibility could not be confirmed as public. Add a GitHub token for this company to refresh this object.`,
|
|
retryAfterSeconds: GITHUB_OBJECT_TTL_SECONDS,
|
|
};
|
|
}
|
|
}
|
|
|
|
const apiKind = objectType === "pull_request" ? "pulls" : "issues";
|
|
const url = `${gitHubApiBase(identity.host)}/repos/${encodeURIComponent(identity.owner)}/${encodeURIComponent(identity.repo)}/${apiKind}/${identity.number}`;
|
|
|
|
let response: Response;
|
|
try {
|
|
response = await fetchImpl(url, { headers, signal });
|
|
} catch {
|
|
return {
|
|
ok: false,
|
|
liveness: "unreachable",
|
|
errorCode: "github_fetch_failed",
|
|
errorMessage: "GitHub could not be reached while refreshing this object.",
|
|
retryAfterSeconds: GITHUB_OBJECT_TTL_SECONDS,
|
|
};
|
|
}
|
|
|
|
const etag = response.headers.get("etag");
|
|
if (response.status === 404) {
|
|
// GitHub deliberately returns 404 for repositories the credentials cannot
|
|
// see, to avoid leaking that a private repo exists. A 404 therefore means
|
|
// either "object deleted/never existed" or "repo private / no access".
|
|
// Probe the repository metadata to classify: if the repo itself is visible
|
|
// the object truly is gone; if it is not, surface credentials-needed
|
|
// instead of permanently freezing a reference as "Not found".
|
|
const probe = await cachedProbeRepoVisibility(companyId, identity, headers, signal);
|
|
if (probe.visibility === "visible") {
|
|
return { ok: true, snapshot: notFoundSnapshot(identity, etag) };
|
|
}
|
|
if (probe.visibility === "hidden") {
|
|
return {
|
|
ok: false,
|
|
liveness: "auth_required",
|
|
errorCode: "github_repo_not_accessible",
|
|
errorMessage: `GitHub returned not-found for ${identity.owner}/${identity.repo}. The repository may be private, deleted, or the configured GitHub credentials may lack access to it.`,
|
|
retryAfterSeconds: GITHUB_OBJECT_TTL_SECONDS,
|
|
};
|
|
}
|
|
if (probe.visibility === "rate_limited") {
|
|
return rateLimitedResult(authenticated, GITHUB_OBJECT_TTL_SECONDS);
|
|
}
|
|
return {
|
|
ok: false,
|
|
liveness: "unreachable",
|
|
errorCode: "github_repo_probe_failed",
|
|
errorMessage: "GitHub could not verify this repository while checking this object.",
|
|
retryAfterSeconds: GITHUB_OBJECT_TTL_SECONDS,
|
|
};
|
|
}
|
|
|
|
const failure = failureFromGitHubResponse(response, authenticated);
|
|
if (failure) return failure;
|
|
if (!response.ok) {
|
|
return {
|
|
ok: false,
|
|
liveness: "unreachable",
|
|
errorCode: "github_unexpected_response",
|
|
errorMessage: `GitHub returned HTTP ${response.status} while refreshing this object.`,
|
|
retryAfterSeconds: GITHUB_OBJECT_TTL_SECONDS,
|
|
};
|
|
}
|
|
|
|
const body = await safeJson(response);
|
|
if (!body) {
|
|
return {
|
|
ok: false,
|
|
liveness: "unreachable",
|
|
errorCode: "github_invalid_response",
|
|
errorMessage: "GitHub returned an invalid object response.",
|
|
retryAfterSeconds: GITHUB_OBJECT_TTL_SECONDS,
|
|
};
|
|
}
|
|
|
|
return {
|
|
ok: true,
|
|
snapshot: objectType === "pull_request"
|
|
? pullRequestSnapshot(identity, body, etag)
|
|
: issueSnapshot(identity, body, etag),
|
|
};
|
|
},
|
|
};
|
|
}
|
|
|
|
return {
|
|
detector,
|
|
resolvers: [resolver("pull_request"), resolver("issue")],
|
|
};
|
|
}
|