mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-15 18:56:05 +01:00
feat(links): add cached GitHub pull request previews (#20593)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task conversations contain links to pull requests and other work. > - A URL alone cannot tell the operator whether a PR is open, merged, or blocked by conflicts. > - Fetching every link at render time wastes API calls and can expose private data through a shared cache. > - This pull request adds a demand-driven links service and a GitHub PR adapter. > - Each viewer and company has a separate cache. Repeated links share one request and result. > - Operators get context when they hover or focus a link. Normal links stay useful when a provider is unavailable. ## Linked Issues or Issue Description **Subsystem affected** Task Markdown links, shared link metadata, and the server GitHub API boundary. **Problem or motivation** PR links have compact labels and neutral icons. They do not provide a rich hover card or a common status source across repeated mentions. A background resolver cannot borrow a person's GitHub connection because it has no viewer identity. **Proposed solution** Recognize supported links locally. Request metadata after a delayed hover or keyboard focus. Show a card at the first line of the link, aligned with its icon. Share the result across all mentions in the current viewer and company. Keep the exact original navigation URL. The first adapter supports GitHub pull requests. Open is green, merged is purple, and merge conflicts are orange. Draft, closed, unknown, loading and unavailable states remain quiet. **Alternatives considered** Eager page scans would spend API quota on links the operator does not inspect. Company-wide caching of personal GitHub results would cross the credential boundary. This change uses neither approach. **Roadmap alignment** ROADMAP.md was checked. This extends existing task links and the GitHub resolver. It adds no deployment adapter or new background worker. Searches of open and closed PRs for link previews and PR hover cards found no matching implementation. ## What Changed - Add host-owned link recognition and a normalized preview contract. The service selects an adapter by provider. - Add a board-only company read endpoint. It checks company access before cache reads and sends a private, no-store response. - Use the current viewer's eligible GitHub connection. A configured connection that cannot supply credentials fails closed. With no managed connection, reuse existing company credentials or public access. Instance credentials retain the public-repository check. - Reuse the GitHub external-object resolver for API transport and response parsing. Add title and mergeability to its normalized snapshot without changing its existing state contract. - Coalesce requests by company, viewer and canonical PR URL. Bound cached entries and concurrent work. Cache success for one minute and failures for a short period or the provider retry window. - Add an abortable request deadline. Timed-out work stays in the concurrency budget until it settles. No periodic worker or idle polling is added. - Add a shared browser query and anchored preview card. Repeated Markdown links observe the same result. There are no automatic retries, window-focus refreshes, toasts or connection prompts. - Add ten production-component Storybook examples and real pointer/keyboard browser checks. Document the API and verification route. ## Verification - Linux source checkout: 233 focused tests passed in nine suites. They cover recognition, fixed API destinations, provider states, access failures, credential selection, viewer/company isolation, request coalescing, expiry, cache bounds, concurrency bounds and mounted OpenAPI coverage. - A live public GitHub smoke returned this PR's correct title and open state. Two equivalent URLs shared one real network request. Personal connection access was not part of that smoke. - A controlled HTTP server stalls a real response body. The test proves cancellation while another lookup completes. - UI typecheck and the server layered typecheck passed. The UI production build passed. - Token gates and query policy gates passed. All 30 map/catalog support checks passed. The three new server test files also passed a focused TypeScript check. - All 27 focused browser checks passed on the reviewed source tree. They cover light/dark themes, narrow layouts, open/merged/conflict/draft/closed/loading/unavailable/failed states, fixed positioning during pointer movement, exact navigation and keyboard access. A real browser HTTP interception counts one deferred lookup for two equivalent PR links. - The first browser attempt lacked Chromium. A later combined build/test attempt exhausted the sandbox's 8 GiB memory limit and killed Storybook. Those attempts supply no pass evidence. Chromium is installed, and the final browser checks run with the build stopped. One initial Storybook dependency reload interrupted a streaming check. All eight existing inline-link cases passed on the warmed server, followed by a complete 27-case pass. - Synthetic Storybook pointer events did not exercise the native hover boundary. The assertions now run through real Playwright pointer and keyboard actions. The fixture stories retain the production components. - Review fixes preserve the Draft label when GitHub reports both draft and conflicts. They also preserve neutral link colors in dark mode. The 25 affected service checks pass, including the added combined-state regression. Browser checks compare unknown, draft and closed links with the muted theme color. - The first CI policy run rejected a literal retry opt-out. The code now uses the existing explicit zero-retry helper. A new test-spy type error was also corrected. These failures are fixed; their earlier runs remain failed. - `pnpm verify --explain` returned exit 2 because the browser test path selects broad CI. No full local suite is claimed. The final CI run supplies that coverage. - The previous head `437913c797963aa113bb82e2ae8b836969317ef9` passed CI and received a 5/5 Greptile score. All four review threads are resolved. Master then advanced. The current head `ae2a1c2adbd61a621cca9bdd160561952613df6e` resolves the source-manifest conflict against master `9b21dcbdda01296349f3cfe0a3a6891e79df6130`. Fresh CI and review are running. - The previous CI run found a lost class on decorated PR links and a stale SDK source manifest. Both are fixed. The existing CommentThread integration test and 26 focused link service tests pass. All 45 SDK source/privacy tests pass. All eight source-manifest helper tests pass. The manifest helper changed only source identities and added the two new shared files. Runtime, compiler, parser and library pins stay unchanged. - A later review found that expired timeouts could reuse a settled promise. The service now removes settled requests from its shared-request map and counts unfinished provider work separately. The new regression failed before the fix and passes after it. The concurrency-bound and real stalled-HTTP tests also pass. - UI typecheck and token gates pass after these fixes. The browser rerun needed Chromium in the new session. That first attempt did not exercise the UI. The complete 27-case browser rerun passed before the master merge with retries disabled. All 27 browser cases also passed on the merge revision with retries disabled. - After the master merge, all 84 focused link, OpenAPI and integration tests pass. All 55 SDK source/privacy/manifest tests pass. UI typecheck and token gates pass again. The source manifest retains master's reviewed runtime pins and includes both sets of source changes. GitHub confirms the PR can merge cleanly. The preflight helper rebuilt the shared package and plugin SDK; Cargo is unavailable locally. Final CI supplies Rust coverage. - The latest master update changed the reviewed report-lock policy but left the SDK lock hash at its previous value. The SDK source/evidence checks reproduced the mismatch. This change sets only the SDK lock hash to the authenticated output of the existing master policy and patch. Compiler, parser and Zod pins stay unchanged. All 56 SDK source/evidence/manifest tests now pass. All 84 focused link/API/integration tests, UI typecheck and token gates also pass on the updated source. ## Risks - This cache is process-local. Separate server replicas can make separate GitHub requests. - A cached result can remain visible for its short lifetime after GitHub changes. A later hover refreshes expired data. - GitHub can report unknown mergeability. The UI shows conflicts only when GitHub explicitly reports them. - Personal preview data stays in viewer/company caches. It is not written to company external-object records or persisted browser query storage. - Stories simulate the API. They do not prove a live GitHub login or private-repository access. - No schema migration is required. ## Model Used OpenAI Codex, based on GPT-6. The runtime does not expose the exact serving model ID or context-window limit. Used reasoning, repository inspection, source editing, shell execution and browser tools. No subagents were used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
161ee3b9a2
commit
89ec4e7614
25 files changed
+855
-19
No files matched your search
@@ -0,0 +1,59 @@
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { recognizeLinkPreview } from "@paperclipai/shared";
|
||||
import { createGitHubLinkPreviewProvider, resolveLinkPreviewGitHubCredential } from "../services/github-link-preview.js";
|
||||
import { resolveGitHubApiCredential, resolveManagedGitHubCredential } from "../services/git-credentials.js";
|
||||
|
||||
vi.mock("../services/git-credentials.js", async original => ({
|
||||
...await original<typeof import("../services/git-credentials.js")>(),
|
||||
resolveManagedGitHubCredential: vi.fn(), resolveGitHubApiCredential: vi.fn(),
|
||||
}));
|
||||
vi.mock("../services/secrets.js", () => ({ secretService: () => ({}) }));
|
||||
const db = null as unknown as Db;
|
||||
const scope = { companyId: "company-a", userId: "alice" };
|
||||
const target = recognizeLinkPreview("https://github.com/acme/web/pull/42")!;
|
||||
beforeEach(() => vi.resetAllMocks());
|
||||
|
||||
describe("GitHub link preview boundary", () => {
|
||||
it("uses only the current viewer's managed identity with no standing delegation", async () => {
|
||||
vi.mocked(resolveManagedGitHubCredential).mockResolvedValue({ configured: true, credential: { token: "token", source: "managed_connection", secretName: null } });
|
||||
expect(await resolveLinkPreviewGitHubCredential(db, scope)).toEqual({ token: "token", source: "managed_connection", secretName: null });
|
||||
expect(resolveManagedGitHubCredential).toHaveBeenCalledWith(db, {}, scope.companyId, { responsibleUserId: "alice", allowStandingDelegation: false });
|
||||
expect(resolveGitHubApiCredential).not.toHaveBeenCalled();
|
||||
});
|
||||
it("does not bypass a configured connection that cannot supply credentials", async () => {
|
||||
vi.mocked(resolveManagedGitHubCredential).mockResolvedValue({ configured: true });
|
||||
await expect(resolveLinkPreviewGitHubCredential(db, scope)).rejects.toThrow();
|
||||
expect(resolveGitHubApiCredential).not.toHaveBeenCalled();
|
||||
});
|
||||
it("uses the existing company/public fallback only without a managed connection", async () => {
|
||||
vi.mocked(resolveManagedGitHubCredential).mockResolvedValue({ configured: false });
|
||||
vi.mocked(resolveGitHubApiCredential).mockResolvedValue(null);
|
||||
expect(await resolveLinkPreviewGitHubCredential(db, scope)).toBeNull();
|
||||
expect(resolveGitHubApiCredential).toHaveBeenCalledWith(db, scope.companyId);
|
||||
});
|
||||
it.each([
|
||||
[{ state: "open" }, "open"], [{ state: "open", mergeable: false }, "conflicts"],
|
||||
[{ state: "open", mergeable: null }, "open"], [{ state: "open", draft: true }, "draft"],
|
||||
[{ state: "open", draft: true, mergeable: false }, "draft"],
|
||||
[{ state: "closed", merged: true }, "merged"], [{ state: "closed" }, "closed"],
|
||||
])("normalizes state %j as %s without fetching checks or reviews", async (fields, expected) => {
|
||||
const fetcher = vi.fn(async () => Response.json({ title: "A change", updated_at: "2026-11-10T12:00:00Z", user: { login: "alice" }, ...fields }));
|
||||
const provider = createGitHubLinkPreviewProvider(db, { fetch: fetcher, credentialProvider: () => null });
|
||||
const result = await provider.resolve(target, scope, new AbortController().signal);
|
||||
expect(result.preview).toMatchObject({ title: "A change", state: expected, author: "alice", target });
|
||||
expect(fetcher).toHaveBeenCalledTimes(1);
|
||||
expect(fetcher).toHaveBeenCalledWith("https://api.github.com/repos/acme/web/pulls/42", expect.objectContaining({ signal: expect.any(AbortSignal) }));
|
||||
});
|
||||
it.each([401, 403, 429, 500])("returns a quiet absence for GitHub HTTP %s", async status => {
|
||||
const provider = createGitHubLinkPreviewProvider(db, { fetch: async () => new Response(null, { status }), credentialProvider: () => null });
|
||||
expect((await provider.resolve(target, scope, new AbortController().signal)).preview).toBeNull();
|
||||
});
|
||||
it("retains the instance-token public-repository boundary", async () => {
|
||||
const fetcher = vi.fn(async () => Response.json({ private: true }));
|
||||
const provider = createGitHubLinkPreviewProvider(db, { fetch: fetcher, credentialProvider: () => ({ token: "instance", source: "server_env", secretName: null }) });
|
||||
expect((await provider.resolve(target, scope, new AbortController().signal)).preview).toBeNull();
|
||||
expect(fetcher).toHaveBeenCalledTimes(1);
|
||||
expect(fetcher).toHaveBeenCalledWith("https://api.github.com/repos/acme/web", expect.any(Object));
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,48 @@
|
||||
import express from "express";
|
||||
import request from "supertest";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { linkPreviewRoutes } from "../routes/link-previews.js";
|
||||
import { HttpError } from "../errors.js";
|
||||
|
||||
function app(actor: express.Request["actor"]) {
|
||||
const get = vi.fn(async () => ({ preview: null, maxAgeMs: 60_000 }));
|
||||
const server = express();
|
||||
server.use((req, _res, next) => { req.actor = actor; next(); });
|
||||
server.use(linkPreviewRoutes(null as unknown as Db, { get }));
|
||||
server.use((error: HttpError, _req: express.Request, res: express.Response, _next: express.NextFunction) => {
|
||||
res.status(error.status ?? 500).json({ error: error.message });
|
||||
});
|
||||
return { server, get };
|
||||
}
|
||||
const board = { type: "board", source: "session", userId: "alice", companyIds: ["company-a"] } as const;
|
||||
const path = "/companies/company-a/link-preview";
|
||||
|
||||
describe("link preview routes", () => {
|
||||
it("passes the authenticated viewer and disables shared HTTP caching", async () => {
|
||||
const { server, get } = app({ ...board, companyIds: [...board.companyIds] });
|
||||
const response = await request(server).get(path).query({ url: "https://github.com/acme/web/pull/42" });
|
||||
expect(response.status).toBe(200);
|
||||
expect(response.headers["cache-control"]).toBe("private, no-store");
|
||||
expect(get).toHaveBeenCalledWith({ companyId: "company-a", userId: "alice" }, "https://github.com/acme/web/pull/42");
|
||||
});
|
||||
it("checks company access before consulting any cache or provider", async () => {
|
||||
const { server, get } = app({ ...board, companyIds: ["company-b"] });
|
||||
expect((await request(server).get(path).query({ url: "https://github.com/acme/web/pull/42" })).status).toBe(403);
|
||||
expect(get).not.toHaveBeenCalled();
|
||||
});
|
||||
it("rejects agents and unauthenticated requests", async () => {
|
||||
for (const actor of [{ type: "none" }, { type: "agent", agentId: "agent-a", companyId: "company-a" }] as express.Request["actor"][]) {
|
||||
const { server, get } = app(actor);
|
||||
expect([401, 403]).toContain((await request(server).get(path).query({ url: "https://github.com/acme/web/pull/42" })).status);
|
||||
expect(get).not.toHaveBeenCalled();
|
||||
}
|
||||
});
|
||||
it("bounds and validates the URL input", async () => {
|
||||
const { server, get } = app({ ...board, companyIds: [...board.companyIds] });
|
||||
for (const query of [{}, { url: "x".repeat(2049) }, { url: ["a", "b"] }]) {
|
||||
expect((await request(server).get(path).query(query)).status).toBe(400);
|
||||
}
|
||||
expect(get).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,113 @@
|
||||
import { createServer } from "node:http";
|
||||
import { once } from "node:events";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { createLinkPreviewService, type LinkPreviewProvider } from "../services/link-previews.js";
|
||||
|
||||
const scope = { companyId: "company-a", userId: "alice" };
|
||||
const url = "https://github.com/acme/web/pull/42";
|
||||
const unavailable = { preview: null, maxAgeMs: 60_000 };
|
||||
const provider = (resolve = vi.fn(async () => unavailable)): LinkPreviewProvider => ({ key: "github", resolve });
|
||||
afterEach(() => vi.useRealTimers());
|
||||
|
||||
describe("demand-driven link preview cache", () => {
|
||||
it("does no work until requested and coalesces equivalent URLs", async () => {
|
||||
const resolve = vi.fn(async () => unavailable);
|
||||
const service = createLinkPreviewService([provider(resolve)]);
|
||||
expect(resolve).not.toHaveBeenCalled();
|
||||
await Promise.all([
|
||||
service.get(scope, url), service.get(scope, "https://www.github.com/ACME/Web/pull/42/files?x=1#review"),
|
||||
service.get(scope, url),
|
||||
]);
|
||||
expect(resolve).toHaveBeenCalledTimes(1);
|
||||
await service.get(scope, url);
|
||||
expect(resolve).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
it("separates viewers, companies, and resources", async () => {
|
||||
const resolve = vi.fn(async () => unavailable);
|
||||
const service = createLinkPreviewService([provider(resolve)]);
|
||||
await service.get(scope, url);
|
||||
await service.get({ ...scope, userId: "bob" }, url);
|
||||
await service.get({ ...scope, companyId: "company-b" }, url);
|
||||
await service.get(scope, `${url}1`);
|
||||
expect(resolve).toHaveBeenCalledTimes(4);
|
||||
});
|
||||
it("caches failures, then refreshes only on a new request after expiry", async () => {
|
||||
let now = 0;
|
||||
const resolve = vi.fn(async () => { throw new Error("no access"); });
|
||||
const service = createLinkPreviewService([provider(resolve)], { now: () => now });
|
||||
expect(await service.get(scope, url)).toEqual(unavailable);
|
||||
now = 30_000;
|
||||
expect((await service.get(scope, url)).maxAgeMs).toBe(30_000);
|
||||
now = 61_000;
|
||||
expect(resolve).toHaveBeenCalledTimes(1);
|
||||
await service.get(scope, url);
|
||||
expect(resolve).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
it("bounds cached resources and does not resolve unsupported hosts", async () => {
|
||||
const resolve = vi.fn(async () => unavailable);
|
||||
const service = createLinkPreviewService([provider(resolve)], { maxEntries: 2 });
|
||||
await service.get(scope, "https://localhost/private");
|
||||
expect(resolve).not.toHaveBeenCalled();
|
||||
for (const suffix of ["", "1", "2", ""]) await service.get(scope, url + suffix);
|
||||
expect(resolve).toHaveBeenCalledTimes(4);
|
||||
});
|
||||
it("holds timed-out underlying work inside the concurrency limit", async () => {
|
||||
vi.useFakeTimers();
|
||||
let finish!: (value: typeof unavailable) => void;
|
||||
const resolve = vi.fn(() => new Promise<typeof unavailable>(done => { finish = done; }));
|
||||
const service = createLinkPreviewService([provider(resolve)], { timeoutMs: 20, maxConcurrent: 1 });
|
||||
const first = service.get(scope, url);
|
||||
await vi.advanceTimersByTimeAsync(21);
|
||||
expect(await first).toEqual(unavailable);
|
||||
await service.get(scope, url + "1");
|
||||
expect(resolve).toHaveBeenCalledTimes(1);
|
||||
finish(unavailable);
|
||||
await vi.advanceTimersByTimeAsync(1);
|
||||
const next = service.get(scope, url + "1");
|
||||
await vi.advanceTimersByTimeAsync(21);
|
||||
await next;
|
||||
expect(resolve).toHaveBeenCalledTimes(2);
|
||||
finish(unavailable);
|
||||
});
|
||||
it("refreshes an expired timeout while the earlier operation ignores cancellation", async () => {
|
||||
vi.useFakeTimers();
|
||||
let finish!: (value: typeof unavailable) => void;
|
||||
const resolve = vi.fn(async () => unavailable).mockImplementationOnce(
|
||||
() => new Promise<typeof unavailable>(done => { finish = done; }),
|
||||
);
|
||||
const service = createLinkPreviewService([provider(resolve)], { timeoutMs: 20, maxConcurrent: 2 });
|
||||
const first = service.get(scope, url);
|
||||
await vi.advanceTimersByTimeAsync(21);
|
||||
expect(await first).toEqual(unavailable);
|
||||
await service.get(scope, url);
|
||||
expect(resolve).toHaveBeenCalledTimes(1);
|
||||
await vi.advanceTimersByTimeAsync(60_001);
|
||||
await service.get(scope, url);
|
||||
expect(resolve).toHaveBeenCalledTimes(2);
|
||||
finish(unavailable);
|
||||
});
|
||||
it("aborts a real stalled HTTP read while an unrelated lookup remains responsive", async () => {
|
||||
const server = createServer((_req, res) => { res.writeHead(200, { "content-type": "application/json" }); res.write("{"); });
|
||||
server.listen(0, "127.0.0.1");
|
||||
await once(server, "listening");
|
||||
const address = server.address() as { port: number };
|
||||
let aborted = false;
|
||||
const service = createLinkPreviewService([{ key: "github", resolve: async (target, _scope, signal) => {
|
||||
if (target.canonicalUrl.endsWith("/421")) return unavailable;
|
||||
try {
|
||||
const response = await fetch(`http://127.0.0.1:${address.port}`, { signal });
|
||||
await response.json();
|
||||
} catch { aborted = signal.aborted; }
|
||||
return unavailable;
|
||||
} }], { timeoutMs: 100 });
|
||||
try {
|
||||
const slow = service.get(scope, url);
|
||||
expect(await service.get(scope, url + "1")).toEqual(unavailable);
|
||||
await slow;
|
||||
await vi.waitFor(() => expect(aborted).toBe(true));
|
||||
} finally {
|
||||
server.closeAllConnections();
|
||||
await new Promise<void>(done => server.close(() => done()));
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -78,6 +78,7 @@ const apiPrefixes: Record<string, string> = {
|
||||
"issues.ts": "/api",
|
||||
"issue-tree-control.ts": "/api",
|
||||
"llms.ts": "/api",
|
||||
"link-previews.ts": "/api",
|
||||
"managed-agent-profiles.ts": "/api",
|
||||
"onboarding-seed.ts": "/api",
|
||||
"openapi.ts": "/api",
|
||||
|
||||
@@ -121,6 +121,7 @@ import { environmentService } from "./services/environments.js";
|
||||
import { environmentRuntimeService } from "./services/environment-runtime.js";
|
||||
import { projectRoutes } from "./routes/projects.js";
|
||||
import { issueRoutes } from "./routes/issues.js";
|
||||
import { linkPreviewRoutes } from "./routes/link-previews.js";
|
||||
import { issueTreeControlRoutes } from "./routes/issue-tree-control.js";
|
||||
import { caseRoutes } from "./routes/cases.js";
|
||||
import { caseHandoffRoutes } from "./routes/case-handoffs.js";
|
||||
@@ -920,6 +921,7 @@ export async function createApp(
|
||||
api.use(decisionModelRoutes(db));
|
||||
api.use(agentObservabilityRoutes(db, { deploymentMode: opts.deploymentMode, deploymentExposure: opts.deploymentExposure }));
|
||||
api.use(fastResponseRoutes(db));
|
||||
api.use(linkPreviewRoutes(db));
|
||||
api.use(promptInjectionPreventionRoutes(db));
|
||||
api.use(aiConnectionRoutes(db, { deploymentMode: opts.deploymentMode, deploymentExposure: opts.deploymentExposure, trustedLocalStdioRuntimeHost, pluginWorkerManager: workerManager }));
|
||||
api.use(
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
import { Router } from "express";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { badRequest } from "../errors.js";
|
||||
import { assertBoard, assertCompanyAccess } from "./authz.js";
|
||||
import { createLinkPreviewService } from "../services/link-previews.js";
|
||||
import { createGitHubLinkPreviewProvider } from "../services/github-link-preview.js";
|
||||
|
||||
export function linkPreviewRoutes(db: Db, service = createLinkPreviewService([createGitHubLinkPreviewProvider(db)])) {
|
||||
const router = Router();
|
||||
router.get("/companies/:companyId/link-preview", async (req, res) => {
|
||||
const companyId = req.params.companyId as string;
|
||||
assertCompanyAccess(req, companyId);
|
||||
assertBoard(req);
|
||||
const href = req.query.url;
|
||||
if (typeof href !== "string" || href.length > 2048) throw badRequest("A link URL is required");
|
||||
// Personal repository metadata must never be stored by a proxy or shared browser cache.
|
||||
res.set("Cache-Control", "private, no-store");
|
||||
res.json(await service.get({ companyId, userId: req.actor.userId ?? null }, href));
|
||||
});
|
||||
return router;
|
||||
}
|
||||
@@ -10151,6 +10151,26 @@ registry.registerPath({
|
||||
responses: { 200: r.ok(), 401: r.unauthorized, 404: r.notFound },
|
||||
});
|
||||
|
||||
registry.registerPath({
|
||||
method: "get",
|
||||
path: "/api/companies/{companyId}/link-preview",
|
||||
tags: ["companies"],
|
||||
summary: "Read optional link metadata for the current board user",
|
||||
description: "Company access and board authentication are required. This read uses a viewer-scoped cache and returns a quiet null preview when provider access is unavailable. It does not change tasks or schedule background work.",
|
||||
request: { params: z.object({ companyId: z.string() }), query: z.object({ url: z.string().max(2048) }) },
|
||||
responses: {
|
||||
200: r.ok(z.object({
|
||||
preview: z.object({
|
||||
target: z.object({ provider: z.string(), kind: z.string(), canonicalUrl: z.string(), label: z.string() }),
|
||||
title: z.string(), state: z.enum(["unknown", "open", "draft", "merged", "closed", "conflicts"]),
|
||||
author: z.string().nullable(), updatedAt: z.string().nullable(),
|
||||
}).nullable(),
|
||||
maxAgeMs: z.number(),
|
||||
})),
|
||||
400: r.badRequest, 401: r.unauthorized, 403: r.forbidden,
|
||||
},
|
||||
});
|
||||
|
||||
registry.registerPath({
|
||||
method: "get",
|
||||
path: "/api/issues/{id}/external-objects",
|
||||
|
||||
@@ -200,7 +200,7 @@ type GitCredentialSecretsDeps = {
|
||||
resolveUserSecretValue?: SecretServiceLike["resolveUserSecretValue"];
|
||||
};
|
||||
|
||||
export type GitHubApiCredentialSource = "company_secret" | "server_env";
|
||||
export type GitHubApiCredentialSource = "managed_connection" | "company_secret" | "server_env";
|
||||
|
||||
export type GitHubApiCredential = {
|
||||
token: string;
|
||||
|
||||
@@ -13,6 +13,14 @@ import type {
|
||||
|
||||
type FetchLike = (url: string, init?: RequestInit) => Promise<Response>;
|
||||
|
||||
type GitHubObjectResolver = Omit<ExternalObjectResolver, "resolve"> & {
|
||||
resolve(input: {
|
||||
companyId: string;
|
||||
object: { externalId: string; sanitizedCanonicalUrl: string | null };
|
||||
signal?: AbortSignal;
|
||||
}): Promise<ExternalObjectResolveResult>;
|
||||
};
|
||||
|
||||
export interface GitHubExternalObjectProviderOptions {
|
||||
fetch?: FetchLike;
|
||||
requestTimeoutMs?: number;
|
||||
@@ -337,6 +345,8 @@ function pullRequestSnapshot(identity: GitHubObjectIdentity, body: Record<string
|
||||
state,
|
||||
merged,
|
||||
draft,
|
||||
title,
|
||||
mergeable: asBoolean(body.mergeable),
|
||||
...(authorLogin ? { authorLogin } : {}),
|
||||
...(headRef ? { headRef } : {}),
|
||||
...(headSha ? { headSha } : {}),
|
||||
@@ -399,7 +409,7 @@ async function safeJson(response: Response) {
|
||||
export function createGitHubExternalObjectProvider(
|
||||
db: Db,
|
||||
opts: GitHubExternalObjectProviderOptions = {},
|
||||
): { detector: ExternalObjectDetector; resolvers: ExternalObjectResolver[] } {
|
||||
): { detector: ExternalObjectDetector; resolvers: GitHubObjectResolver[] } {
|
||||
const fetchImpl = opts.fetch ?? ghFetch;
|
||||
const secretNames = opts.secretNames ?? DEFAULT_GITHUB_TOKEN_SECRET_NAMES;
|
||||
// `credentialProvider` is the preferred seam; `tokenProvider` stays for callers that
|
||||
@@ -462,7 +472,7 @@ export function createGitHubExternalObjectProvider(
|
||||
},
|
||||
};
|
||||
|
||||
function resolver(objectType: GitHubObjectIdentity["objectType"]): ExternalObjectResolver {
|
||||
function resolver(objectType: GitHubObjectIdentity["objectType"]): GitHubObjectResolver {
|
||||
return {
|
||||
providerKey: "github",
|
||||
objectType,
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import type { LinkPreviewState } from "@paperclipai/shared";
|
||||
import { createGitHubExternalObjectProvider, type GitHubExternalObjectProviderOptions } from "./github-external-object-provider.js";
|
||||
import { resolveGitHubApiCredential, resolveManagedGitHubCredential } from "./git-credentials.js";
|
||||
import { secretService } from "./secrets.js";
|
||||
import type { LinkPreviewProvider, LinkPreviewScope } from "./link-previews.js";
|
||||
|
||||
/** Only the viewer's own connection is eligible. Never borrow the task assignee's identity. */
|
||||
export async function resolveLinkPreviewGitHubCredential(db: Db, scope: LinkPreviewScope) {
|
||||
if (scope.userId) {
|
||||
const managed = await resolveManagedGitHubCredential(db, secretService(db), scope.companyId, {
|
||||
responsibleUserId: scope.userId, allowStandingDelegation: false,
|
||||
});
|
||||
if (managed.configured) {
|
||||
if (!managed.credential) throw new Error("GitHub connection unavailable");
|
||||
return { token: managed.credential.token, source: "managed_connection" as const, secretName: null };
|
||||
}
|
||||
}
|
||||
return resolveGitHubApiCredential(db, scope.companyId);
|
||||
}
|
||||
|
||||
export function createGitHubLinkPreviewProvider(db: Db, options?: GitHubExternalObjectProviderOptions): LinkPreviewProvider {
|
||||
return {
|
||||
key: "github",
|
||||
async resolve(target, scope, signal) {
|
||||
const [, owner, repo, , number] = new URL(target.canonicalUrl).pathname.split("/");
|
||||
const resolver = createGitHubExternalObjectProvider(db, {
|
||||
...options,
|
||||
credentialProvider: options?.credentialProvider ?? (() => resolveLinkPreviewGitHubCredential(db, scope)),
|
||||
}).resolvers.find(candidate => candidate.objectType === "pull_request")!;
|
||||
const result = await resolver.resolve({ companyId: scope.companyId, signal, object: {
|
||||
externalId: `${owner}/${repo}#pull/${number}`,
|
||||
sanitizedCanonicalUrl: target.canonicalUrl,
|
||||
} });
|
||||
if (!result.ok) return { preview: null, maxAgeMs: Math.max(60_000, (result.retryAfterSeconds ?? 60) * 1_000) };
|
||||
const snapshot = result.snapshot;
|
||||
const data = snapshot.data;
|
||||
if (!data || typeof data.title !== "string" || !data.title.trim()) return { preview: null, maxAgeMs: 60_000 };
|
||||
const state: LinkPreviewState = data.merged === true ? "merged"
|
||||
: data.state === "closed" ? "closed"
|
||||
: data.state !== "open" ? "unknown"
|
||||
: data.draft === true ? "draft"
|
||||
: data.mergeable === false ? "conflicts" : "open";
|
||||
return {
|
||||
preview: {
|
||||
target, title: data.title, state,
|
||||
author: typeof data.authorLogin === "string" ? data.authorLogin : null,
|
||||
updatedAt: snapshot.remoteVersion ?? null,
|
||||
},
|
||||
maxAgeMs: 60_000,
|
||||
};
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
import type { LinkPreviewResult, LinkPreviewTarget } from "@paperclipai/shared";
|
||||
import { recognizeLinkPreview } from "@paperclipai/shared";
|
||||
|
||||
export interface LinkPreviewScope { companyId: string; userId: string | null }
|
||||
export interface LinkPreviewProvider {
|
||||
key: string;
|
||||
resolve(target: LinkPreviewTarget, scope: LinkPreviewScope, signal: AbortSignal): Promise<LinkPreviewResult>;
|
||||
}
|
||||
|
||||
const UNAVAILABLE: LinkPreviewResult = { preview: null, maxAgeMs: 60_000 };
|
||||
|
||||
/** Request-driven and process-local. No DB writes, sweep, timers while idle, or cross-viewer reuse. */
|
||||
export function createLinkPreviewService(providers: readonly LinkPreviewProvider[], options: {
|
||||
now?: () => number; timeoutMs?: number; maxEntries?: number; maxConcurrent?: number;
|
||||
} = {}) {
|
||||
const now = options.now ?? Date.now;
|
||||
const cache = new Map<string, { result: LinkPreviewResult; expiresAt: number }>();
|
||||
const inFlight = new Map<string, Promise<LinkPreviewResult>>();
|
||||
let activeOperations = 0;
|
||||
const byKey = new Map(providers.map(provider => [provider.key, provider]));
|
||||
|
||||
return {
|
||||
async get(scope: LinkPreviewScope, href: string): Promise<LinkPreviewResult> {
|
||||
const target = recognizeLinkPreview(href);
|
||||
const provider = target && byKey.get(target.provider);
|
||||
if (!target || !provider) return UNAVAILABLE;
|
||||
const key = JSON.stringify([scope.companyId, scope.userId, target.canonicalUrl]);
|
||||
const cached = cache.get(key);
|
||||
if (cached && cached.expiresAt > now()) {
|
||||
return { ...cached.result, maxAgeMs: cached.expiresAt - now() };
|
||||
}
|
||||
const pending = inFlight.get(key);
|
||||
if (pending) return pending;
|
||||
if (activeOperations >= (options.maxConcurrent ?? 32)) return UNAVAILABLE;
|
||||
|
||||
const controller = new AbortController();
|
||||
let timer: ReturnType<typeof setTimeout>;
|
||||
const deadline = new Promise<LinkPreviewResult>(resolve => {
|
||||
timer = setTimeout(() => { controller.abort(); resolve(UNAVAILABLE); }, options.timeoutMs ?? 5_000);
|
||||
});
|
||||
// Keep an underlying operation in the concurrency budget until it settles,
|
||||
// even if a credential backend ignores cancellation after our deadline.
|
||||
activeOperations++;
|
||||
const operation = Promise.resolve().then(() => provider.resolve(target, scope, controller.signal))
|
||||
.catch(() => UNAVAILABLE).finally(() => { activeOperations--; });
|
||||
const request = Promise.race([operation, deadline]).then(result => {
|
||||
const bounded = { ...result, maxAgeMs: Math.max(1_000, Math.min(result.maxAgeMs, 60 * 60_000)) };
|
||||
cache.delete(key);
|
||||
while (cache.size >= (options.maxEntries ?? 1_000)) cache.delete(cache.keys().next().value!);
|
||||
cache.set(key, { result: bounded, expiresAt: now() + bounded.maxAgeMs });
|
||||
return bounded;
|
||||
}).finally(() => {
|
||||
clearTimeout(timer!);
|
||||
if (inFlight.get(key) === request) inFlight.delete(key);
|
||||
});
|
||||
inFlight.set(key, request);
|
||||
return request;
|
||||
},
|
||||
};
|
||||
}
|
||||
Reference in new issue
Block a user