mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-28 12:04:41 +01:00
<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/11/changes/paperclip/11
<!-- entire-trail-link-end -->
## Thinking Path
> - Paperclip is the open source app that people use to manage AI agents
for work.
> - Paperclip shows the status of external objects, such as GitHub pull
requests and issues.
> - A background sweep refreshes these objects and resolves a GitHub
credential.
> - The sweep read a credential only from a company secret with a
well-known name.
> - An instance without that secret made anonymous GitHub API calls.
Anonymous calls are limited to 60 per hour.
> - When the limit was reached, the sweep showed "unreachable" for
objects that it could not refresh.
> - A server environment token is instance-wide, so it must not expose a
private repository that a company cannot see.
> - This pull request reads a server token for public repositories,
reports a rate limit as its own state, and shows a warning on the pill.
> - The benefit is an accurate status, fewer misleading failures, and no
cross-company private data.
## Linked Issues or Issue Description
No public GitHub issue exists. The sections below describe the bug.
**What happened?**
The board showed "unreachable" for many GitHub links, even after a
related fix was deployed.
**Expected behavior**
The board should show "unreachable" only for real network failures. It
should show a rate limit as its own state and retry later.
**Steps to reproduce**
1. Use an instance with no GitHub token company secret and no server
token.
2. Let the external-object sweep make more than 60 anonymous GitHub
requests in one hour.
3. Open a task with GitHub links.
**Paperclip version or commit**
master (commit a67a18c at report time).
**Deployment mode**
Self-hosted server.
## What Changed
- Add `resolveGitHubApiCredential` in
`server/src/services/git-credentials.ts`. It reads a company secret with
a well-known name, then falls back to the server environment token
(`GITHUB_TOKEN`/`GH_TOKEN`), and it reports the credential `source`.
- Use the source in the GitHub external-object provider and in
`github-commit-details.ts`. A `server_env` token is limited to public
repositories. A private repository resolves only with a company
credential; otherwise the object reports `auth_required` and does not
fetch the private object.
- Add a `rate_limited` liveness state in
`packages/shared/src/constants.ts`.
- Classify a 403 with `x-ratelimit-remaining: 0`, a 429 response, and a
rate-limited repo probe as `rate_limited` instead of `unreachable`.
- Show `rate_limited` in the UI filter, icon, label, tone, overlay, and
badge maps.
- Make `ExternalObjectPill` and `ExternalObjectStatusIcon` prefer a
liveness warning icon and tone, so a rate-limited object shows a warning
even when the last status was healthy.
- Add tests for the credential source, the public-only server token, the
company-scoped private repository, and the rate-limited pill tone.
## Verification
- `vitest run server/src/__tests__/external-objects-service.test.ts
server/src/__tests__/git-credentials.test.ts
server/src/__tests__/github-commit-details.test.ts
server/src/__tests__/work-products.test.ts` — pass, with the new scoping
and credential-source tests.
- `vitest run ui/src/lib/external-objects.test.ts
ui/src/lib/issue-filters.test.ts
ui/src/components/ExternalObjectPill.test.tsx
ui/src/components/MarkdownExternalLink.test.tsx` — pass, with the new
rate-limited pill test.
- `pnpm --filter @paperclipai/shared typecheck` — pass. The full
monorepo `tsc` (server and ui) is killed by the 4GB memory cap of this
sandbox; the authoritative CI Typecheck gate runs it instead.
- One test in `git-credentials.test.ts` fails in this sandbox only. The
test "credential helper execution (real git, no network) ... env-carried
token" reads a global GitHub credential helper in the sandbox. The test
fails the same way on the base without this change.
## Risks
- Low risk. The change adds a liveness state and a credential source. It
does not change existing terminal verdicts.
- A server-environment token cannot resolve a private repository. This
is intentional. An operator who needs private repositories must add a
GitHub token as a company secret. The object then reports
`auth_required` with that instruction.
- `rate_limited` is a new value in `EXTERNAL_OBJECT_LIVENESS_STATES`. An
old client ignores an unknown value.
## Model Used
- Provider: DeepSeek (OpenRouter).
- Exact model ID: `openrouter/deepseek/deepseek-v4.1-flash`.
- Capabilities: tool use and code execution in the Paperclip runner.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for similar open and closed PRs and
confirmed this is not a duplicate
- [x] I have either (a) linked existing issues with `Fixes:` / `Closes`
/ `Refs:` OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>
168 lines
6.3 KiB
TypeScript
168 lines
6.3 KiB
TypeScript
import type { Db } from "@paperclipai/db";
|
|
import {
|
|
DEFAULT_GITHUB_TOKEN_SECRET_NAMES,
|
|
resolveGitHubApiCredential,
|
|
type GitHubApiCredential,
|
|
} from "./git-credentials.js";
|
|
import { ghFetch, gitHubApiBase } from "./github-fetch.js";
|
|
|
|
type FetchLike = (url: string, init?: RequestInit) => Promise<Response>;
|
|
|
|
export type GitHubCommitReference = {
|
|
host: "github.com";
|
|
owner: string;
|
|
repo: string;
|
|
sha: string;
|
|
};
|
|
|
|
export type GitHubCommitDiffDetails = {
|
|
additions: number;
|
|
deletions: number;
|
|
changedFiles: number;
|
|
};
|
|
|
|
export type GitHubCommitDiffDetailsResolver = (
|
|
companyId: string,
|
|
reference: GitHubCommitReference,
|
|
) => Promise<GitHubCommitDiffDetails | null>;
|
|
|
|
export interface GitHubCommitDetailsResolverOptions {
|
|
fetch?: FetchLike;
|
|
tokenProvider?: (companyId: string) => Promise<string | null> | string | null;
|
|
credentialProvider?: (companyId: string) => Promise<GitHubApiCredential | null> | GitHubApiCredential | null;
|
|
secretNames?: readonly string[];
|
|
}
|
|
|
|
const GITHUB_COMMIT_URL_PATTERN = /https:\/\/(?:www\.)?github\.com\/([A-Za-z0-9_.-]+)\/([A-Za-z0-9_.-]+)\/commit\/([0-9a-f]{7,64})\b/i;
|
|
const GITHUB_COMMIT_SHORTHAND_PATTERN = /^([A-Za-z0-9_.-]+)\/([A-Za-z0-9_.-]+)@([0-9a-f]{7,64})$/i;
|
|
|
|
function nonNegativeInteger(value: unknown): number | null {
|
|
return typeof value === "number" && Number.isSafeInteger(value) && value >= 0 ? value : null;
|
|
}
|
|
|
|
function record(value: unknown): Record<string, unknown> | null {
|
|
return value && typeof value === "object" && !Array.isArray(value)
|
|
? value as Record<string, unknown>
|
|
: null;
|
|
}
|
|
|
|
function normalizeCredential(credential: GitHubApiCredential | null | undefined): GitHubApiCredential | null {
|
|
const token = credential?.token?.trim();
|
|
if (!token) return null;
|
|
return { token, source: credential!.source, secretName: credential!.secretName ?? null };
|
|
}
|
|
|
|
export function extractGitHubCommitReference(values: readonly unknown[]): GitHubCommitReference | null {
|
|
for (const value of values) {
|
|
if (typeof value !== "string" || value.length === 0) continue;
|
|
const match = GITHUB_COMMIT_URL_PATTERN.exec(value) ?? GITHUB_COMMIT_SHORTHAND_PATTERN.exec(value);
|
|
if (!match) continue;
|
|
return {
|
|
host: "github.com",
|
|
owner: match[1]!,
|
|
repo: match[2]!,
|
|
sha: match[3]!,
|
|
};
|
|
}
|
|
return null;
|
|
}
|
|
|
|
async function defaultCredentialProvider(db: Db, companyId: string, secretNames: readonly string[]) {
|
|
return resolveGitHubApiCredential(db, companyId, { secretNames });
|
|
}
|
|
|
|
/** Return the repo's `private` flag, or null when the probe does not answer. */
|
|
async function probeRepoIsPrivate(
|
|
reference: GitHubCommitReference,
|
|
headers: Record<string, string>,
|
|
fetchImpl: FetchLike,
|
|
): Promise<boolean | null> {
|
|
try {
|
|
const response = await fetchImpl(
|
|
`${gitHubApiBase(reference.host)}/repos/${encodeURIComponent(reference.owner)}/${encodeURIComponent(reference.repo)}`,
|
|
{ headers },
|
|
);
|
|
if (!response.ok) return null;
|
|
const body = record(await response.json());
|
|
return typeof body?.private === "boolean" ? body.private : null;
|
|
} catch {
|
|
return null;
|
|
}
|
|
}
|
|
|
|
function hasNextPage(response: Response): boolean {
|
|
const link = response.headers.get("link");
|
|
if (!link) return false;
|
|
for (const part of link.split(",")) {
|
|
const match = /^\s*<[^>]+>;\s*rel="([^"]+)"\s*$/.exec(part);
|
|
if (match?.[1]?.split(/\s+/).includes("next")) return true;
|
|
}
|
|
return false;
|
|
}
|
|
|
|
export function createGitHubCommitDiffDetailsResolver(
|
|
db: Db,
|
|
opts: GitHubCommitDetailsResolverOptions = {},
|
|
): GitHubCommitDiffDetailsResolver {
|
|
const fetchImpl = opts.fetch ?? ghFetch;
|
|
const secretNames = opts.secretNames ?? DEFAULT_GITHUB_TOKEN_SECRET_NAMES;
|
|
const credentialProvider: (companyId: string) => Promise<GitHubApiCredential | null> =
|
|
opts.credentialProvider
|
|
? async (companyId: string) => normalizeCredential(await opts.credentialProvider!(companyId))
|
|
: Object.prototype.hasOwnProperty.call(opts, "tokenProvider") && opts.tokenProvider !== undefined
|
|
? async (companyId: string) => {
|
|
const raw = typeof opts.tokenProvider === "function" ? await opts.tokenProvider(companyId) : opts.tokenProvider;
|
|
const token = raw?.trim();
|
|
return token ? { token, source: "company_secret", secretName: null } : null;
|
|
}
|
|
: (companyId: string) => defaultCredentialProvider(db, companyId, secretNames);
|
|
|
|
return async (companyId, reference) => {
|
|
try {
|
|
const credential = await credentialProvider(companyId);
|
|
const token = credential?.token?.trim() || null;
|
|
const headers: Record<string, string> = {
|
|
accept: "application/vnd.github+json",
|
|
"user-agent": "paperclip-work-product-resolver",
|
|
"x-github-api-version": "2022-11-28",
|
|
};
|
|
if (token) headers.authorization = `Bearer ${token}`;
|
|
|
|
// A server-environment token is instance-wide. Do not use it to read a private
|
|
// repository on behalf of one company; only a company-scoped credential may.
|
|
if (credential?.source === "server_env") {
|
|
const isPrivate = await probeRepoIsPrivate(reference, headers, fetchImpl);
|
|
if (isPrivate !== false) return null;
|
|
}
|
|
|
|
const commitUrl = `${gitHubApiBase(reference.host)}/repos/${encodeURIComponent(reference.owner)}/${encodeURIComponent(reference.repo)}/commits/${encodeURIComponent(reference.sha)}`;
|
|
let additions: number | null = null;
|
|
let deletions: number | null = null;
|
|
let changedFiles = 0;
|
|
let pages = 0;
|
|
let morePages = true;
|
|
while (morePages && pages < 30) {
|
|
const url = `${commitUrl}?per_page=100&page=${pages + 1}`;
|
|
const response = await fetchImpl(url, { headers });
|
|
if (!response.ok) return null;
|
|
const body = record(await response.json());
|
|
if (!body) return null;
|
|
if (pages === 0) {
|
|
const stats = record(body.stats);
|
|
additions = nonNegativeInteger(stats?.additions);
|
|
deletions = nonNegativeInteger(stats?.deletions);
|
|
if (additions === null || deletions === null) return null;
|
|
}
|
|
if (!Array.isArray(body.files)) return null;
|
|
changedFiles += body.files.length;
|
|
morePages = hasNextPage(response);
|
|
pages += 1;
|
|
}
|
|
if (morePages || additions === null || deletions === null) return null;
|
|
return { additions, deletions, changedFiles };
|
|
} catch {
|
|
return null;
|
|
}
|
|
};
|
|
}
|