mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-29 21:07:40 +01:00
<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/11/changes/paperclip/11
<!-- entire-trail-link-end -->
## Thinking Path
> - Paperclip is the open source app that people use to manage AI agents
for work.
> - Paperclip shows the status of external objects, such as GitHub pull
requests and issues.
> - A background sweep refreshes these objects and resolves a GitHub
credential.
> - The sweep read a credential only from a company secret with a
well-known name.
> - An instance without that secret made anonymous GitHub API calls.
Anonymous calls are limited to 60 per hour.
> - When the limit was reached, the sweep showed "unreachable" for
objects that it could not refresh.
> - A server environment token is instance-wide, so it must not expose a
private repository that a company cannot see.
> - This pull request reads a server token for public repositories,
reports a rate limit as its own state, and shows a warning on the pill.
> - The benefit is an accurate status, fewer misleading failures, and no
cross-company private data.
## Linked Issues or Issue Description
No public GitHub issue exists. The sections below describe the bug.
**What happened?**
The board showed "unreachable" for many GitHub links, even after a
related fix was deployed.
**Expected behavior**
The board should show "unreachable" only for real network failures. It
should show a rate limit as its own state and retry later.
**Steps to reproduce**
1. Use an instance with no GitHub token company secret and no server
token.
2. Let the external-object sweep make more than 60 anonymous GitHub
requests in one hour.
3. Open a task with GitHub links.
**Paperclip version or commit**
master (commit a67a18c at report time).
**Deployment mode**
Self-hosted server.
## What Changed
- Add `resolveGitHubApiCredential` in
`server/src/services/git-credentials.ts`. It reads a company secret with
a well-known name, then falls back to the server environment token
(`GITHUB_TOKEN`/`GH_TOKEN`), and it reports the credential `source`.
- Use the source in the GitHub external-object provider and in
`github-commit-details.ts`. A `server_env` token is limited to public
repositories. A private repository resolves only with a company
credential; otherwise the object reports `auth_required` and does not
fetch the private object.
- Add a `rate_limited` liveness state in
`packages/shared/src/constants.ts`.
- Classify a 403 with `x-ratelimit-remaining: 0`, a 429 response, and a
rate-limited repo probe as `rate_limited` instead of `unreachable`.
- Show `rate_limited` in the UI filter, icon, label, tone, overlay, and
badge maps.
- Make `ExternalObjectPill` and `ExternalObjectStatusIcon` prefer a
liveness warning icon and tone, so a rate-limited object shows a warning
even when the last status was healthy.
- Add tests for the credential source, the public-only server token, the
company-scoped private repository, and the rate-limited pill tone.
## Verification
- `vitest run server/src/__tests__/external-objects-service.test.ts
server/src/__tests__/git-credentials.test.ts
server/src/__tests__/github-commit-details.test.ts
server/src/__tests__/work-products.test.ts` — pass, with the new scoping
and credential-source tests.
- `vitest run ui/src/lib/external-objects.test.ts
ui/src/lib/issue-filters.test.ts
ui/src/components/ExternalObjectPill.test.tsx
ui/src/components/MarkdownExternalLink.test.tsx` — pass, with the new
rate-limited pill test.
- `pnpm --filter @paperclipai/shared typecheck` — pass. The full
monorepo `tsc` (server and ui) is killed by the 4GB memory cap of this
sandbox; the authoritative CI Typecheck gate runs it instead.
- One test in `git-credentials.test.ts` fails in this sandbox only. The
test "credential helper execution (real git, no network) ... env-carried
token" reads a global GitHub credential helper in the sandbox. The test
fails the same way on the base without this change.
## Risks
- Low risk. The change adds a liveness state and a credential source. It
does not change existing terminal verdicts.
- A server-environment token cannot resolve a private repository. This
is intentional. An operator who needs private repositories must add a
GitHub token as a company secret. The object then reports
`auth_required` with that instruction.
- `rate_limited` is a new value in `EXTERNAL_OBJECT_LIVENESS_STATES`. An
old client ignores an unknown value.
## Model Used
- Provider: DeepSeek (OpenRouter).
- Exact model ID: `openrouter/deepseek/deepseek-v4.1-flash`.
- Capabilities: tool use and code execution in the Paperclip runner.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for similar open and closed PRs and
confirmed this is not a duplicate
- [x] I have either (a) linked existing issues with `Fixes:` / `Closes`
/ `Refs:` OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>
1438 lines
53 KiB
TypeScript
1438 lines
53 KiB
TypeScript
import { randomUUID } from "node:crypto";
|
|
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
|
|
import { eq } from "drizzle-orm";
|
|
import {
|
|
companies,
|
|
createDb,
|
|
activityLog,
|
|
externalObjectMentions,
|
|
externalObjects,
|
|
issueComments,
|
|
issues,
|
|
plugins,
|
|
} from "@paperclipai/db";
|
|
import {
|
|
getEmbeddedPostgresTestSupport,
|
|
startEmbeddedPostgresTestDatabase,
|
|
} from "./helpers/embedded-postgres.js";
|
|
import {
|
|
createExternalObjectDetectorRegistry,
|
|
createExternalObjectResolverRegistry,
|
|
externalObjectService,
|
|
type ExternalObjectResolver,
|
|
} from "../services/external-objects.js";
|
|
import { canonicalizeExternalObjectUrl } from "@paperclipai/shared/external-objects-server";
|
|
import type { PaperclipPluginManifestV1 } from "@paperclipai/shared";
|
|
import type { PluginWorkerManager } from "../services/plugin-worker-manager.js";
|
|
import { createGitHubExternalObjectProvider } from "../services/github-external-object-provider.js";
|
|
|
|
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
|
|
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
|
|
|
|
if (!embeddedPostgresSupport.supported) {
|
|
console.warn(
|
|
`Skipping embedded Postgres external object tests on this host: ${embeddedPostgresSupport.reason ?? "unsupported environment"}`,
|
|
);
|
|
}
|
|
|
|
describe("external object registries", () => {
|
|
it("lets provider detectors claim urls before the generic fallback", async () => {
|
|
const canonical = canonicalizeExternalObjectUrl("https://github.com/acme/app/pull/42");
|
|
if (!canonical) throw new Error("expected canonical url");
|
|
const registry = createExternalObjectDetectorRegistry([
|
|
{
|
|
key: "github",
|
|
detect: ({ urls }) =>
|
|
urls.map((url) => ({
|
|
canonical: url,
|
|
detectorKey: "github",
|
|
providerKey: "github",
|
|
objectType: "pull_request",
|
|
externalId: "acme/app#42",
|
|
confidence: "exact",
|
|
})),
|
|
},
|
|
]);
|
|
|
|
const detections = await registry.detect({
|
|
companyId: "company-1",
|
|
urls: [canonical],
|
|
sourceContext: {
|
|
companyId: "company-1",
|
|
sourceIssueId: "issue-1",
|
|
sourceKind: "description",
|
|
sourceRecordId: null,
|
|
documentKey: null,
|
|
propertyKey: null,
|
|
},
|
|
});
|
|
|
|
expect(detections).toHaveLength(1);
|
|
expect(detections[0]).toMatchObject({
|
|
providerKey: "github",
|
|
objectType: "pull_request",
|
|
externalId: "acme/app#42",
|
|
});
|
|
});
|
|
|
|
it("falls back to generic url objects when no provider detector claims a url", async () => {
|
|
const canonical = canonicalizeExternalObjectUrl("https://example.com/path?token=secret#frag");
|
|
if (!canonical) throw new Error("expected canonical url");
|
|
const registry = createExternalObjectDetectorRegistry([]);
|
|
|
|
const detections = await registry.detect({
|
|
companyId: "company-1",
|
|
urls: [canonical],
|
|
sourceContext: {
|
|
companyId: "company-1",
|
|
sourceIssueId: "issue-1",
|
|
sourceKind: "description",
|
|
sourceRecordId: null,
|
|
documentKey: null,
|
|
propertyKey: null,
|
|
},
|
|
});
|
|
|
|
expect(detections[0]).toMatchObject({
|
|
providerKey: "url",
|
|
objectType: "link",
|
|
externalId: canonical.canonicalIdentityHash,
|
|
displayTitle: "https://example.com/path",
|
|
});
|
|
});
|
|
|
|
it("matches resolvers by provider and optional object type", () => {
|
|
const fallbackResolver: ExternalObjectResolver = {
|
|
providerKey: "github",
|
|
resolve: async () => ({
|
|
ok: true,
|
|
snapshot: { statusCategory: "unknown", statusTone: "neutral" },
|
|
}),
|
|
};
|
|
const pullRequestResolver: ExternalObjectResolver = {
|
|
providerKey: "github",
|
|
objectType: "pull_request",
|
|
resolve: async () => ({
|
|
ok: true,
|
|
snapshot: { statusCategory: "open", statusTone: "info" },
|
|
}),
|
|
};
|
|
const registry = createExternalObjectResolverRegistry([pullRequestResolver, fallbackResolver]);
|
|
|
|
expect(registry.find({ providerKey: "github", objectType: "pull_request" })).toBe(pullRequestResolver);
|
|
expect(registry.find({ providerKey: "github", objectType: "issue" })).toBe(fallbackResolver);
|
|
expect(registry.find({ providerKey: "linear", objectType: "issue" })).toBeNull();
|
|
});
|
|
});
|
|
|
|
describe("GitHub external object provider", () => {
|
|
function githubObject(path: string, objectType: "pull_request" | "issue") {
|
|
const canonical = canonicalizeExternalObjectUrl(`https://github.com/acme/app/${path}`);
|
|
if (!canonical) throw new Error("expected canonical url");
|
|
return {
|
|
id: randomUUID(),
|
|
companyId: "company-1",
|
|
providerKey: "github",
|
|
objectType,
|
|
externalId: `acme/app#${path}`,
|
|
sanitizedCanonicalUrl: canonical.sanitizedCanonicalUrl,
|
|
canonicalIdentityHash: canonical.canonicalIdentityHash,
|
|
displayTitle: "acme/app#42",
|
|
statusKey: null,
|
|
statusLabel: null,
|
|
statusCategory: "unknown",
|
|
statusTone: "neutral",
|
|
liveness: "unknown",
|
|
isTerminal: false,
|
|
data: {},
|
|
remoteVersion: null,
|
|
etag: null,
|
|
} as any;
|
|
}
|
|
|
|
function response(body: Record<string, unknown>, init: ResponseInit = {}) {
|
|
return new Response(JSON.stringify(body), {
|
|
status: 200,
|
|
headers: { "content-type": "application/json", etag: '"etag-1"', ...(init.headers ?? {}) },
|
|
...init,
|
|
});
|
|
}
|
|
|
|
it("detects GitHub pull request and issue URLs before the generic fallback", async () => {
|
|
const provider = createGitHubExternalObjectProvider({} as any, { tokenProvider: null });
|
|
const pr = canonicalizeExternalObjectUrl("https://github.com/Acme/App/pull/42?token=secret#discussion");
|
|
const issue = canonicalizeExternalObjectUrl("https://github.com/Acme/App/issues/7");
|
|
const other = canonicalizeExternalObjectUrl("https://example.com/Acme/App/pull/42");
|
|
if (!pr || !issue || !other) throw new Error("expected canonical urls");
|
|
|
|
const detections = await provider.detector.detect({
|
|
companyId: "company-1",
|
|
urls: [pr, issue, other],
|
|
sourceContext: {
|
|
companyId: "company-1",
|
|
sourceIssueId: "issue-1",
|
|
sourceKind: "description",
|
|
sourceRecordId: null,
|
|
documentKey: null,
|
|
propertyKey: null,
|
|
},
|
|
});
|
|
|
|
expect(detections).toEqual([
|
|
expect.objectContaining({
|
|
providerKey: "github",
|
|
objectType: "pull_request",
|
|
externalId: "acme/app#pull/42",
|
|
displayKey: "GitHub Pull Request",
|
|
iconKey: "github",
|
|
displayTitle: "Acme/App#42",
|
|
}),
|
|
expect.objectContaining({
|
|
providerKey: "github",
|
|
objectType: "issue",
|
|
externalId: "acme/app#issues/7",
|
|
displayKey: "GitHub Issue",
|
|
iconKey: "github",
|
|
displayTitle: "Acme/App#7",
|
|
}),
|
|
]);
|
|
expect(JSON.stringify(detections)).not.toContain("secret");
|
|
});
|
|
|
|
it.each([
|
|
[
|
|
"open",
|
|
{ state: "open", draft: false, merged: false, title: "Ship it", updated_at: "2026-04-24T01:02:03Z" },
|
|
{ statusKey: "open", statusLabel: "Open", statusIconKey: "git-pull-request", statusCategory: "open", statusTone: "info", isTerminal: false },
|
|
],
|
|
[
|
|
"draft",
|
|
{ state: "open", draft: true, merged: false, title: "WIP", updated_at: "2026-04-24T01:02:03Z" },
|
|
{ statusKey: "draft", statusLabel: "Draft", statusIconKey: "clock", statusCategory: "waiting", statusTone: "warning", isTerminal: false },
|
|
],
|
|
[
|
|
"closed",
|
|
{ state: "closed", draft: false, merged: false, title: "Closed PR", updated_at: "2026-04-24T01:02:03Z" },
|
|
{ statusKey: "closed", statusLabel: "Closed", statusIconKey: "x-circle", statusCategory: "closed", statusTone: "muted", isTerminal: true },
|
|
],
|
|
[
|
|
"merged",
|
|
{ state: "closed", draft: false, merged: true, title: "Merged PR", updated_at: "2026-04-24T01:02:03Z" },
|
|
{ statusKey: "merged", statusLabel: "Merged", statusIconKey: "git-merge", statusCategory: "succeeded", statusTone: "success", isTerminal: true },
|
|
],
|
|
])("resolves a %s pull request snapshot", async (_name, body, expected) => {
|
|
const fetch = vi.fn(async () => response(body));
|
|
const provider = createGitHubExternalObjectProvider({} as any, { fetch, tokenProvider: null });
|
|
const resolver = provider.resolvers.find((entry) => entry.objectType === "pull_request")!;
|
|
|
|
const result = await resolver.resolve({
|
|
companyId: "company-1",
|
|
object: githubObject("pull/42", "pull_request"),
|
|
});
|
|
|
|
expect(fetch).toHaveBeenCalledWith(
|
|
"https://api.github.com/repos/acme/app/pulls/42",
|
|
expect.objectContaining({
|
|
headers: expect.not.objectContaining({ authorization: expect.any(String) }),
|
|
}),
|
|
);
|
|
expect(result).toEqual({
|
|
ok: true,
|
|
snapshot: expect.objectContaining({
|
|
...expected,
|
|
displayKey: "GitHub Pull Request",
|
|
iconKey: "github",
|
|
displayTitle: expect.stringContaining(String(body.title)),
|
|
remoteVersion: "2026-04-24T01:02:03Z",
|
|
etag: '"etag-1"',
|
|
data: expect.objectContaining({
|
|
provider: "github",
|
|
owner: "acme",
|
|
repo: "app",
|
|
number: 42,
|
|
}),
|
|
}),
|
|
});
|
|
expect(JSON.stringify(result)).not.toContain("authorization");
|
|
});
|
|
|
|
it.each([
|
|
[
|
|
"open",
|
|
{ state: "open", title: "Issue", state_reason: null, updated_at: "2026-04-24T01:02:03Z" },
|
|
{ statusKey: "open", statusLabel: "Open", statusIconKey: "circle-dot", statusCategory: "open", statusTone: "info", isTerminal: false },
|
|
],
|
|
[
|
|
"closed",
|
|
{ state: "closed", title: "Issue", state_reason: "completed", updated_at: "2026-04-24T01:02:03Z" },
|
|
{ statusKey: "closed_completed", statusLabel: "Closed: completed", statusIconKey: "circle", statusCategory: "closed", statusTone: "muted", isTerminal: true },
|
|
],
|
|
])("resolves a %s issue snapshot", async (_name, body, expected) => {
|
|
const fetch = vi.fn(async () => response(body));
|
|
const provider = createGitHubExternalObjectProvider({} as any, { fetch, tokenProvider: null });
|
|
const resolver = provider.resolvers.find((entry) => entry.objectType === "issue")!;
|
|
|
|
const result = await resolver.resolve({
|
|
companyId: "company-1",
|
|
object: githubObject("issues/42", "issue"),
|
|
});
|
|
|
|
expect(fetch).toHaveBeenCalledWith("https://api.github.com/repos/acme/app/issues/42", expect.any(Object));
|
|
expect(result).toEqual({
|
|
ok: true,
|
|
snapshot: expect.objectContaining({
|
|
...expected,
|
|
displayKey: "GitHub Issue",
|
|
iconKey: "github",
|
|
data: expect.objectContaining({
|
|
provider: "github",
|
|
owner: "acme",
|
|
repo: "app",
|
|
number: 42,
|
|
}),
|
|
}),
|
|
});
|
|
});
|
|
|
|
it("uses a configured token without storing it in the resolved snapshot", async () => {
|
|
const fetch = vi.fn(async (_url: string, init?: RequestInit) => {
|
|
expect(init?.headers).toEqual(expect.objectContaining({ authorization: "Bearer ghp_secret" }));
|
|
return response({ state: "open", draft: false, merged: false, title: "Private PR" });
|
|
});
|
|
const provider = createGitHubExternalObjectProvider({} as any, {
|
|
fetch,
|
|
tokenProvider: async () => "ghp_secret",
|
|
});
|
|
const resolver = provider.resolvers.find((entry) => entry.objectType === "pull_request")!;
|
|
|
|
const result = await resolver.resolve({
|
|
companyId: "company-1",
|
|
object: githubObject("pull/42", "pull_request"),
|
|
});
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(JSON.stringify(result)).not.toContain("ghp_secret");
|
|
});
|
|
|
|
it.each([
|
|
[
|
|
"auth-required",
|
|
new Response("", { status: 401 }),
|
|
{ ok: false, liveness: "auth_required", errorCode: "github_auth_required" },
|
|
],
|
|
[
|
|
"rate-limit",
|
|
new Response("", {
|
|
status: 403,
|
|
headers: { "x-ratelimit-remaining": "0", "x-ratelimit-reset": String(Math.floor(Date.now() / 1000) + 120) },
|
|
}),
|
|
{ ok: false, liveness: "rate_limited", errorCode: "github_rate_limited" },
|
|
],
|
|
])("maps %s responses to provider-safe results", async (_name, githubResponse, expected) => {
|
|
const provider = createGitHubExternalObjectProvider({} as any, {
|
|
fetch: async () => githubResponse,
|
|
tokenProvider: null,
|
|
});
|
|
const resolver = provider.resolvers.find((entry) => entry.objectType === "pull_request")!;
|
|
|
|
const result = await resolver.resolve({
|
|
companyId: "company-1",
|
|
object: githubObject("pull/42", "pull_request"),
|
|
});
|
|
|
|
expect(result).toEqual(expect.objectContaining(expected));
|
|
expect(JSON.stringify(result)).not.toContain("http");
|
|
});
|
|
|
|
it("classifies a 404 as not-found when the repository is visible to the credentials", async () => {
|
|
const fetch = vi.fn(async (url: string) => {
|
|
if (url.endsWith("/repos/acme/app")) {
|
|
return new Response(JSON.stringify({ full_name: "acme/app" }), {
|
|
status: 200,
|
|
headers: { "content-type": "application/json" },
|
|
});
|
|
}
|
|
return new Response("", { status: 404, headers: { etag: '"missing"' } });
|
|
});
|
|
const provider = createGitHubExternalObjectProvider({} as any, { fetch, tokenProvider: null });
|
|
const resolver = provider.resolvers.find((entry) => entry.objectType === "pull_request")!;
|
|
|
|
const result = await resolver.resolve({
|
|
companyId: "company-1",
|
|
object: githubObject("pull/42", "pull_request"),
|
|
});
|
|
|
|
expect(fetch).toHaveBeenCalledWith(
|
|
"https://api.github.com/repos/acme/app",
|
|
expect.objectContaining({ headers: expect.not.objectContaining({ authorization: expect.any(String) }) }),
|
|
);
|
|
expect(result).toEqual({
|
|
ok: true,
|
|
snapshot: expect.objectContaining({
|
|
displayKey: "GitHub Pull Request",
|
|
iconKey: "github",
|
|
statusKey: "not_found",
|
|
statusIconKey: "archive",
|
|
statusCategory: "archived",
|
|
statusTone: "muted",
|
|
isTerminal: true,
|
|
data: expect.objectContaining({ provider: "github", notFound: true, probeVerified: true }),
|
|
}),
|
|
});
|
|
expect(JSON.stringify(result)).not.toContain("http");
|
|
});
|
|
|
|
it("caches repository-visibility probes per company and repo", async () => {
|
|
let probeCalls = 0;
|
|
const fetch = vi.fn(async (url: string) => {
|
|
if (url.endsWith("/repos/acme/app")) {
|
|
probeCalls += 1;
|
|
return new Response(JSON.stringify({ full_name: "acme/app" }), {
|
|
status: 200,
|
|
headers: { "content-type": "application/json" },
|
|
});
|
|
}
|
|
return new Response("", { status: 404, headers: { etag: '"missing"' } });
|
|
});
|
|
const provider = createGitHubExternalObjectProvider({} as any, { fetch, tokenProvider: null });
|
|
const resolver = provider.resolvers.find((entry) => entry.objectType === "pull_request")!;
|
|
|
|
await resolver.resolve({ companyId: "company-1", object: githubObject("pull/42", "pull_request") });
|
|
await resolver.resolve({ companyId: "company-1", object: githubObject("pull/99", "pull_request") });
|
|
|
|
expect(probeCalls).toBe(1);
|
|
expect(fetch.mock.calls.filter(([url]) => String(url).endsWith("/repos/acme/app"))).toHaveLength(1);
|
|
});
|
|
|
|
it("does not reuse a cached probe verdict across credential changes", async () => {
|
|
const fetch = vi.fn(async (url: string, init?: RequestInit) => {
|
|
if (url.endsWith("/repos/acme/app")) {
|
|
const auth = (init?.headers as Record<string, string> | undefined)?.authorization;
|
|
if (auth === "Bearer token-A" || auth === "Bearer token-B") {
|
|
return new Response(JSON.stringify({ full_name: "acme/app" }), {
|
|
status: 200,
|
|
headers: { "content-type": "application/json" },
|
|
});
|
|
}
|
|
return new Response("", { status: 404 });
|
|
}
|
|
return new Response("", { status: 404, headers: { etag: '"missing"' } });
|
|
});
|
|
let token: string | null = "token-A";
|
|
const provider = createGitHubExternalObjectProvider({} as any, {
|
|
fetch,
|
|
tokenProvider: async () => token,
|
|
});
|
|
const resolver = provider.resolvers.find((entry) => entry.objectType === "pull_request")!;
|
|
|
|
const visibleVerdict = await resolver.resolve({ companyId: "company-1", object: githubObject("pull/42", "pull_request") });
|
|
|
|
// Same repo, same credentials: probe result is reused.
|
|
await resolver.resolve({ companyId: "company-1", object: githubObject("pull/99", "pull_request") });
|
|
const probesUnderTokenA = fetch.mock.calls.filter(([url]) => String(url).endsWith("/repos/acme/app"));
|
|
expect(probesUnderTokenA).toHaveLength(1);
|
|
|
|
// Credential now only has access to a different repo — cache must not
|
|
// serve the stale "visible" verdict, or this object would be frozen as a
|
|
// probe-verified not-found forever.
|
|
token = "other-token";
|
|
const hiddenVerdict = await resolver.resolve({ companyId: "company-1", object: githubObject("pull/10", "pull_request") });
|
|
|
|
expect(visibleVerdict).toEqual(
|
|
expect.objectContaining({ ok: true, snapshot: expect.objectContaining({ isTerminal: true }) }),
|
|
);
|
|
expect(hiddenVerdict).toEqual(
|
|
expect.objectContaining({ ok: false, liveness: "auth_required", errorCode: "github_repo_not_accessible" }),
|
|
);
|
|
expect(fetch.mock.calls.filter(([url]) => String(url).endsWith("/repos/acme/app"))).toHaveLength(2);
|
|
});
|
|
|
|
it("classifies a 404 as auth-required when the repository itself is hidden", async () => {
|
|
const fetch = vi.fn(async (url: string) => {
|
|
if (url.endsWith("/repos/acme/app")) {
|
|
return new Response("", { status: 404 });
|
|
}
|
|
return new Response("", { status: 404, headers: { etag: '"missing"' } });
|
|
});
|
|
const provider = createGitHubExternalObjectProvider({} as any, { fetch, tokenProvider: null });
|
|
const resolver = provider.resolvers.find((entry) => entry.objectType === "pull_request")!;
|
|
|
|
const result = await resolver.resolve({
|
|
companyId: "company-1",
|
|
object: githubObject("pull/42", "pull_request"),
|
|
});
|
|
|
|
expect(result).toEqual(
|
|
expect.objectContaining({
|
|
ok: false,
|
|
liveness: "auth_required",
|
|
errorCode: "github_repo_not_accessible",
|
|
errorMessage: expect.stringContaining("acme/app"),
|
|
}),
|
|
);
|
|
expect(JSON.stringify(result)).not.toContain("http");
|
|
});
|
|
|
|
it("classifies a 404 as unreachable when the repository probe itself fails", async () => {
|
|
const fetch = vi.fn(async (url: string) => {
|
|
if (url.endsWith("/repos/acme/app")) {
|
|
throw new Error("network down");
|
|
}
|
|
return new Response("", { status: 404, headers: { etag: '"missing"' } });
|
|
});
|
|
const provider = createGitHubExternalObjectProvider({} as any, { fetch, tokenProvider: null });
|
|
const resolver = provider.resolvers.find((entry) => entry.objectType === "pull_request")!;
|
|
|
|
const result = await resolver.resolve({
|
|
companyId: "company-1",
|
|
object: githubObject("pull/42", "pull_request"),
|
|
});
|
|
|
|
expect(result).toEqual(
|
|
expect.objectContaining({
|
|
ok: false,
|
|
liveness: "unreachable",
|
|
errorCode: "github_repo_probe_failed",
|
|
}),
|
|
);
|
|
expect(JSON.stringify(result)).not.toContain("http");
|
|
});
|
|
|
|
it("classifies a 404 as rate-limited when the repository probe is rate-limited", async () => {
|
|
const fetch = vi.fn(async (url: string) => {
|
|
if (url.endsWith("/repos/acme/app")) {
|
|
return new Response("", {
|
|
status: 403,
|
|
headers: { "x-ratelimit-remaining": "0", "x-ratelimit-reset": String(Math.floor(Date.now() / 1000) + 120) },
|
|
});
|
|
}
|
|
return new Response("", { status: 404, headers: { etag: '"missing"' } });
|
|
});
|
|
const provider = createGitHubExternalObjectProvider({} as any, { fetch, tokenProvider: null });
|
|
const resolver = provider.resolvers.find((entry) => entry.objectType === "pull_request")!;
|
|
|
|
const result = await resolver.resolve({
|
|
companyId: "company-1",
|
|
object: githubObject("pull/42", "pull_request"),
|
|
});
|
|
|
|
expect(result).toEqual(
|
|
expect.objectContaining({
|
|
ok: false,
|
|
liveness: "rate_limited",
|
|
errorCode: "github_rate_limited",
|
|
}),
|
|
);
|
|
expect(result).not.toEqual(
|
|
expect.objectContaining({ ok: false, liveness: "auth_required", errorCode: "github_repo_not_accessible" }),
|
|
);
|
|
expect(JSON.stringify(result)).not.toContain("http");
|
|
});
|
|
|
|
it("classifies an authenticated rate limit as rate-limited rather than unreachable", async () => {
|
|
const fetch = vi.fn(async () => new Response("", {
|
|
status: 403,
|
|
headers: { "x-ratelimit-remaining": "0", "x-ratelimit-reset": String(Math.floor(Date.now() / 1000) + 120) },
|
|
}));
|
|
const provider = createGitHubExternalObjectProvider({} as any, {
|
|
fetch,
|
|
tokenProvider: async () => "ghp_secret",
|
|
});
|
|
const resolver = provider.resolvers.find((entry) => entry.objectType === "pull_request")!;
|
|
|
|
const result = await resolver.resolve({
|
|
companyId: "company-1",
|
|
object: githubObject("pull/42", "pull_request"),
|
|
});
|
|
|
|
expect(result).toEqual(
|
|
expect.objectContaining({ ok: false, liveness: "rate_limited", errorCode: "github_rate_limited" }),
|
|
);
|
|
expect(JSON.stringify(result)).not.toContain("ghp_secret");
|
|
});
|
|
|
|
it("classifies a 429 as rate-limited", async () => {
|
|
const fetch = vi.fn(async () => new Response("", {
|
|
status: 429,
|
|
headers: { "retry-after": "60" },
|
|
}));
|
|
const provider = createGitHubExternalObjectProvider({} as any, { fetch, tokenProvider: null });
|
|
const resolver = provider.resolvers.find((entry) => entry.objectType === "pull_request")!;
|
|
|
|
const result = await resolver.resolve({
|
|
companyId: "company-1",
|
|
object: githubObject("pull/42", "pull_request"),
|
|
});
|
|
|
|
expect(result).toEqual(
|
|
expect.objectContaining({ ok: false, liveness: "rate_limited", errorCode: "github_rate_limited" }),
|
|
);
|
|
});
|
|
|
|
it("limits a server-environment token to public repositories and never fetches a private object", async () => {
|
|
const fetch = vi.fn(async (url: string) => {
|
|
if (url.endsWith("/repos/acme/app")) return response({ private: true, full_name: "acme/app" });
|
|
return response({ state: "open", draft: false, merged: false, title: "Private PR" });
|
|
});
|
|
const provider = createGitHubExternalObjectProvider({} as any, {
|
|
fetch,
|
|
credentialProvider: async () => ({ token: "env-token", source: "server_env", secretName: null }),
|
|
});
|
|
const resolver = provider.resolvers.find((entry) => entry.objectType === "pull_request")!;
|
|
|
|
const result = await resolver.resolve({
|
|
companyId: "company-1",
|
|
object: githubObject("pull/42", "pull_request"),
|
|
});
|
|
|
|
expect(result).toEqual(
|
|
expect.objectContaining({
|
|
ok: false,
|
|
liveness: "auth_required",
|
|
errorCode: "github_repo_requires_company_token",
|
|
}),
|
|
);
|
|
expect(fetch).not.toHaveBeenCalledWith(expect.stringContaining("/pulls/42"), expect.anything());
|
|
expect(JSON.stringify(result)).not.toContain("env-token");
|
|
});
|
|
|
|
it("fails closed when a server-environment token probe does not confirm the repository is public", async () => {
|
|
const fetch = vi.fn(async (url: string) => {
|
|
// No `private` flag: public visibility is not positively confirmed.
|
|
if (url.endsWith("/repos/acme/app")) return response({ full_name: "acme/app" });
|
|
return response({ state: "open", draft: false, merged: false, title: "Unconfirmed PR" });
|
|
});
|
|
const provider = createGitHubExternalObjectProvider({} as any, {
|
|
fetch,
|
|
credentialProvider: async () => ({ token: "env-token", source: "server_env", secretName: null }),
|
|
});
|
|
const resolver = provider.resolvers.find((entry) => entry.objectType === "pull_request")!;
|
|
|
|
const result = await resolver.resolve({
|
|
companyId: "company-1",
|
|
object: githubObject("pull/42", "pull_request"),
|
|
});
|
|
|
|
expect(result).toEqual(
|
|
expect.objectContaining({
|
|
ok: false,
|
|
liveness: "auth_required",
|
|
errorCode: "github_repo_requires_company_token",
|
|
}),
|
|
);
|
|
expect(fetch).not.toHaveBeenCalledWith(expect.stringContaining("/pulls/42"), expect.anything());
|
|
expect(JSON.stringify(result)).not.toContain("env-token");
|
|
});
|
|
|
|
it("uses a server-environment token to resolve a public repository", async () => {
|
|
const fetch = vi.fn(async (url: string) => {
|
|
if (url.endsWith("/repos/acme/app")) return response({ private: false });
|
|
return response({ state: "open", draft: false, merged: false, title: "Public PR" });
|
|
});
|
|
const provider = createGitHubExternalObjectProvider({} as any, {
|
|
fetch,
|
|
credentialProvider: async () => ({ token: "env-token", source: "server_env", secretName: null }),
|
|
});
|
|
const resolver = provider.resolvers.find((entry) => entry.objectType === "pull_request")!;
|
|
|
|
const result = await resolver.resolve({
|
|
companyId: "company-1",
|
|
object: githubObject("pull/42", "pull_request"),
|
|
});
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(fetch).toHaveBeenCalledWith("https://api.github.com/repos/acme/app/pulls/42", expect.anything());
|
|
});
|
|
|
|
it("re-verifies public access for a server-environment token instead of reusing a cached verdict", async () => {
|
|
let repoPrivate = false;
|
|
const fetch = vi.fn(async (url: string) => {
|
|
if (url.endsWith("/repos/acme/app")) return response({ private: repoPrivate });
|
|
return response({ state: "open", draft: false, merged: false, title: "acme/app PR" });
|
|
});
|
|
const provider = createGitHubExternalObjectProvider({} as any, {
|
|
fetch,
|
|
credentialProvider: async () => ({ token: "env-token", source: "server_env", secretName: null }),
|
|
});
|
|
const resolver = provider.resolvers.find((entry) => entry.objectType === "pull_request")!;
|
|
|
|
// First refresh while the repository is public: the object resolves.
|
|
const publicResult = await resolver.resolve({
|
|
companyId: "company-1",
|
|
object: githubObject("pull/42", "pull_request"),
|
|
});
|
|
expect(publicResult.ok).toBe(true);
|
|
|
|
// The repository becomes private inside the probe-cache TTL. The server token
|
|
// still has access, but this company did not authorize it, so the second object
|
|
// must not reuse the cached "public" verdict to read private details.
|
|
repoPrivate = true;
|
|
const privateResult = await resolver.resolve({
|
|
companyId: "company-1",
|
|
object: githubObject("pull/43", "pull_request"),
|
|
});
|
|
|
|
expect(privateResult).toEqual(
|
|
expect.objectContaining({
|
|
ok: false,
|
|
liveness: "auth_required",
|
|
errorCode: "github_repo_requires_company_token",
|
|
}),
|
|
);
|
|
expect(fetch).not.toHaveBeenCalledWith(expect.stringContaining("/pulls/43"), expect.anything());
|
|
expect(JSON.stringify(privateResult)).not.toContain("env-token");
|
|
});
|
|
|
|
it("lets a company-scoped token resolve a private repository", async () => {
|
|
const fetch = vi.fn(async (url: string) => {
|
|
if (url.endsWith("/repos/acme/app")) return response({ private: true });
|
|
return response({ state: "open", draft: false, merged: false, title: "Private PR" });
|
|
});
|
|
const provider = createGitHubExternalObjectProvider({} as any, {
|
|
fetch,
|
|
credentialProvider: async () => ({ token: "company-token", source: "company_secret", secretName: "GITHUB_TOKEN" }),
|
|
});
|
|
const resolver = provider.resolvers.find((entry) => entry.objectType === "pull_request")!;
|
|
|
|
const result = await resolver.resolve({
|
|
companyId: "company-1",
|
|
object: githubObject("pull/42", "pull_request"),
|
|
});
|
|
|
|
expect(result.ok).toBe(true);
|
|
expect(fetch).toHaveBeenCalledWith("https://api.github.com/repos/acme/app/pulls/42", expect.anything());
|
|
});
|
|
});
|
|
|
|
describeEmbeddedPostgres("externalObjectService", () => {
|
|
let db!: ReturnType<typeof createDb>;
|
|
let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
|
|
|
|
beforeAll(async () => {
|
|
tempDb = await startEmbeddedPostgresTestDatabase("paperclip-external-objects-");
|
|
db = createDb(tempDb.connectionString);
|
|
}, 20_000);
|
|
|
|
afterEach(async () => {
|
|
await db.delete(activityLog);
|
|
await db.delete(externalObjectMentions);
|
|
await db.delete(externalObjects);
|
|
await db.delete(issueComments);
|
|
await db.delete(issues);
|
|
await db.delete(plugins);
|
|
await db.delete(companies);
|
|
vi.restoreAllMocks();
|
|
});
|
|
|
|
afterAll(async () => {
|
|
await tempDb?.cleanup();
|
|
});
|
|
|
|
async function createIssue(companyId = randomUUID()) {
|
|
const issueId = randomUUID();
|
|
await db.insert(companies).values({
|
|
id: companyId,
|
|
name: "Paperclip",
|
|
issuePrefix: `E${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
|
|
requireBoardApprovalForNewAgents: false,
|
|
});
|
|
await db.insert(issues).values({
|
|
id: issueId,
|
|
companyId,
|
|
identifier: `PAP-${companyId.replace(/-/g, "").slice(0, 12).toUpperCase()}`,
|
|
title: "External refs",
|
|
description: "Track https://github.com/acme/app/pull/42?token=secret#discussion twice https://github.com/acme/app/pull/42.",
|
|
status: "todo",
|
|
priority: "medium",
|
|
});
|
|
return { companyId, issueId };
|
|
}
|
|
|
|
it("syncs sanitized, deduped mentions without storing secret-bearing urls", async () => {
|
|
const { companyId, issueId } = await createIssue();
|
|
const svc = externalObjectService(db);
|
|
|
|
await svc.syncIssue(issueId);
|
|
|
|
const [objectRows, mentionRows] = await Promise.all([
|
|
db.select().from(externalObjects),
|
|
db.select().from(externalObjectMentions),
|
|
]);
|
|
expect(objectRows).toHaveLength(1);
|
|
expect(objectRows[0]).toMatchObject({
|
|
companyId,
|
|
providerKey: "github",
|
|
objectType: "pull_request",
|
|
externalId: "acme/app#pull/42",
|
|
sanitizedCanonicalUrl: "https://github.com/acme/app/pull/42",
|
|
liveness: "unknown",
|
|
statusCategory: "unknown",
|
|
});
|
|
expect(JSON.stringify(objectRows[0])).not.toContain("secret");
|
|
expect(mentionRows).toHaveLength(1);
|
|
expect(mentionRows[0]).toMatchObject({
|
|
companyId,
|
|
sourceIssueId: issueId,
|
|
sourceKind: "description",
|
|
sanitizedDisplayUrl: "https://github.com/acme/app/pull/42",
|
|
matchedTextRedacted: "https://github.com/acme/app/pull/42",
|
|
});
|
|
});
|
|
|
|
it("no-ops detection and summaries when external objects are disabled", async () => {
|
|
const { issueId } = await createIssue();
|
|
const svc = externalObjectService(db, { enabled: false });
|
|
|
|
await svc.syncIssue(issueId);
|
|
|
|
const [objectRows, mentionRows, summary] = await Promise.all([
|
|
db.select().from(externalObjects),
|
|
db.select().from(externalObjectMentions),
|
|
svc.getIssueSummary(issueId),
|
|
]);
|
|
expect(objectRows).toHaveLength(0);
|
|
expect(mentionRows).toHaveLength(0);
|
|
expect(summary).toMatchObject({
|
|
total: 0,
|
|
byStatusCategory: {},
|
|
byLiveness: {},
|
|
highestSeverity: "neutral",
|
|
objects: [],
|
|
});
|
|
});
|
|
|
|
it("preserves last-known status when resolver reports auth and unreachable failures", async () => {
|
|
const { companyId, issueId } = await createIssue();
|
|
const resolver: ExternalObjectResolver = {
|
|
providerKey: "url",
|
|
objectType: "link",
|
|
resolve: vi
|
|
.fn()
|
|
.mockResolvedValueOnce({
|
|
ok: true,
|
|
snapshot: {
|
|
statusCategory: "open",
|
|
statusTone: "info",
|
|
statusKey: "open",
|
|
statusLabel: "Open",
|
|
ttlSeconds: 1,
|
|
},
|
|
})
|
|
.mockResolvedValueOnce({
|
|
ok: false,
|
|
liveness: "auth_required",
|
|
errorCode: "auth_required",
|
|
errorMessage: "token=secret failed for https://github.com/acme/app/pull/42?token=secret",
|
|
retryAfterSeconds: 60,
|
|
})
|
|
.mockResolvedValueOnce({
|
|
ok: false,
|
|
liveness: "unreachable",
|
|
errorCode: "network",
|
|
errorMessage: "GET https://github.com/acme/app/pull/42 failed",
|
|
retryAfterSeconds: 60,
|
|
}),
|
|
};
|
|
const svc = externalObjectService(db, { resolvers: [resolver], github: false });
|
|
await svc.syncIssue(issueId);
|
|
const object = await db.select().from(externalObjects).then((rows) => rows[0]!);
|
|
|
|
await svc.refreshObject(object.id, { companyId, force: true });
|
|
await svc.refreshObject(object.id, { companyId, force: true });
|
|
|
|
const authFailure = await db.select().from(externalObjects).then((rows) => rows[0]!);
|
|
expect(authFailure.lastErrorMessage).toContain("token=[redacted]");
|
|
expect(authFailure.lastErrorMessage).not.toContain("secret");
|
|
|
|
await svc.refreshObject(object.id, { companyId, force: true });
|
|
|
|
const updated = await db.select().from(externalObjects).then((rows) => rows[0]!);
|
|
expect(updated.statusCategory).toBe("open");
|
|
expect(updated.statusLabel).toBe("Open");
|
|
expect(updated.liveness).toBe("unreachable");
|
|
expect(updated.lastErrorMessage).toContain("[redacted-url]");
|
|
expect(updated.lastErrorMessage).not.toContain("secret");
|
|
});
|
|
|
|
it("schedules newly detected objects for automatic refresh", async () => {
|
|
const { companyId, issueId } = await createIssue();
|
|
const resolve = vi.fn(async () => ({
|
|
ok: true as const,
|
|
snapshot: {
|
|
statusCategory: "open" as const,
|
|
statusTone: "info" as const,
|
|
statusKey: "open",
|
|
statusLabel: "Open",
|
|
ttlSeconds: 300,
|
|
},
|
|
}));
|
|
const resolver: ExternalObjectResolver = {
|
|
providerKey: "url",
|
|
objectType: "link",
|
|
resolve,
|
|
};
|
|
const svc = externalObjectService(db, { resolvers: [resolver], github: false });
|
|
await svc.syncIssue(issueId);
|
|
const object = await db.select().from(externalObjects).then((rows) => rows[0]!);
|
|
|
|
expect(object.nextRefreshAt).toBeInstanceOf(Date);
|
|
|
|
const refreshed = await svc.refreshDueObjects(companyId, 50, new Date(Date.now() + 1_000));
|
|
|
|
expect(refreshed).toHaveLength(1);
|
|
expect(resolve).toHaveBeenCalledTimes(1);
|
|
});
|
|
|
|
it("coalesces automatic and manual refreshes for the same object", async () => {
|
|
const { companyId, issueId } = await createIssue();
|
|
let markResolverStarted!: () => void;
|
|
const resolverStarted = new Promise<void>((resolve) => {
|
|
markResolverStarted = resolve;
|
|
});
|
|
let finishResolver!: () => void;
|
|
const resolverCanFinish = new Promise<void>((resolve) => {
|
|
finishResolver = resolve;
|
|
});
|
|
const resolve = vi.fn(async () => {
|
|
markResolverStarted();
|
|
await resolverCanFinish;
|
|
return {
|
|
ok: true as const,
|
|
snapshot: {
|
|
statusCategory: "open" as const,
|
|
statusTone: "info" as const,
|
|
statusKey: "open",
|
|
statusLabel: "Open",
|
|
ttlSeconds: 300,
|
|
},
|
|
};
|
|
});
|
|
const resolver: ExternalObjectResolver = {
|
|
providerKey: "url",
|
|
objectType: "link",
|
|
resolve,
|
|
};
|
|
const svc = externalObjectService(db, { resolvers: [resolver], github: false });
|
|
await svc.syncIssue(issueId);
|
|
const object = await db.select().from(externalObjects).then((rows) => rows[0]!);
|
|
|
|
const dueRefresh = svc.refreshDueObjects(companyId, 50, new Date(Date.now() + 1_000));
|
|
await resolverStarted;
|
|
const manualRefresh = svc.refreshObject(object.id, { companyId, force: true });
|
|
finishResolver();
|
|
|
|
const [dueResults, manualResult] = await Promise.all([dueRefresh, manualRefresh]);
|
|
|
|
expect(dueResults).toHaveLength(1);
|
|
expect(dueResults[0]?.refreshed).toBe(true);
|
|
expect(manualResult.refreshed).toBe(true);
|
|
expect(manualResult.object.statusLabel).toBe("Open");
|
|
expect(resolve).toHaveBeenCalledTimes(1);
|
|
});
|
|
|
|
it("prevents duplicate refreshes across service instances", async () => {
|
|
const { companyId, issueId } = await createIssue();
|
|
let markResolverStarted!: () => void;
|
|
const resolverStarted = new Promise<void>((resolve) => {
|
|
markResolverStarted = resolve;
|
|
});
|
|
let finishResolver!: () => void;
|
|
const resolverCanFinish = new Promise<void>((resolve) => {
|
|
finishResolver = resolve;
|
|
});
|
|
const resolve = vi.fn(async () => {
|
|
markResolverStarted();
|
|
await resolverCanFinish;
|
|
return {
|
|
ok: true as const,
|
|
snapshot: {
|
|
statusCategory: "open" as const,
|
|
statusTone: "info" as const,
|
|
statusKey: "open",
|
|
statusLabel: "Open",
|
|
ttlSeconds: 300,
|
|
},
|
|
};
|
|
});
|
|
const resolver: ExternalObjectResolver = {
|
|
providerKey: "url",
|
|
objectType: "link",
|
|
resolve,
|
|
};
|
|
const scheduledService = externalObjectService(db, { resolvers: [resolver], github: false });
|
|
const manualService = externalObjectService(db, { resolvers: [resolver], github: false });
|
|
await scheduledService.syncIssue(issueId);
|
|
const object = await db.select().from(externalObjects).then((rows) => rows[0]!);
|
|
|
|
const dueRefresh = scheduledService.refreshDueObjects(companyId, 50, new Date(Date.now() + 1_000));
|
|
await resolverStarted;
|
|
const manualRefresh = await manualService.refreshObject(object.id, { companyId, force: true });
|
|
finishResolver();
|
|
const dueResults = await dueRefresh;
|
|
|
|
expect(dueResults).toHaveLength(1);
|
|
expect(dueResults[0]?.refreshed).toBe(true);
|
|
expect(manualRefresh.refreshed).toBe(false);
|
|
expect(manualRefresh.reason).toBe("refresh_in_progress");
|
|
expect(manualRefresh.object).not.toHaveProperty("refreshToken");
|
|
expect(resolve).toHaveBeenCalledTimes(1);
|
|
await expect(
|
|
db.select().from(externalObjects).then((rows) => rows[0]?.refreshStartedAt ?? null),
|
|
).resolves.toBeNull();
|
|
});
|
|
|
|
it("does not let a stale refresh overwrite a replacement claimant", async () => {
|
|
const { companyId, issueId } = await createIssue();
|
|
let markResolverStarted!: () => void;
|
|
const resolverStarted = new Promise<void>((resolve) => {
|
|
markResolverStarted = resolve;
|
|
});
|
|
let finishResolver!: () => void;
|
|
const resolverCanFinish = new Promise<void>((resolve) => {
|
|
finishResolver = resolve;
|
|
});
|
|
const slowResolve = vi.fn(async () => {
|
|
markResolverStarted();
|
|
await resolverCanFinish;
|
|
return {
|
|
ok: true as const,
|
|
snapshot: {
|
|
statusCategory: "open" as const,
|
|
statusTone: "info" as const,
|
|
statusKey: "open",
|
|
statusLabel: "Open",
|
|
ttlSeconds: 300,
|
|
},
|
|
};
|
|
});
|
|
const replacementResolve = vi.fn(async () => ({
|
|
ok: true as const,
|
|
snapshot: {
|
|
statusCategory: "closed" as const,
|
|
statusTone: "muted" as const,
|
|
statusKey: "closed",
|
|
statusLabel: "Closed",
|
|
ttlSeconds: 300,
|
|
},
|
|
}));
|
|
const scheduledService = externalObjectService(db, {
|
|
resolvers: [{ providerKey: "url", objectType: "link", resolve: slowResolve }],
|
|
github: false,
|
|
});
|
|
const replacementService = externalObjectService(db, {
|
|
resolvers: [{ providerKey: "url", objectType: "link", resolve: replacementResolve }],
|
|
github: false,
|
|
});
|
|
await scheduledService.syncIssue(issueId);
|
|
const object = await db.select().from(externalObjects).then((rows) => rows[0]!);
|
|
const claimedAt = new Date(Date.now() + 1_000);
|
|
|
|
const dueRefresh = scheduledService.refreshDueObjects(companyId, 50, claimedAt);
|
|
await resolverStarted;
|
|
const replacementResult = await replacementService.refreshObject(object.id, {
|
|
companyId,
|
|
force: true,
|
|
now: new Date(claimedAt.getTime() + 301_000),
|
|
});
|
|
finishResolver();
|
|
const dueResults = await dueRefresh;
|
|
const finalObject = await db.select().from(externalObjects).then((rows) => rows[0]!);
|
|
|
|
expect(replacementResult.refreshed).toBe(true);
|
|
expect(dueResults[0]).toMatchObject({ refreshed: false, reason: "refresh_superseded" });
|
|
expect(finalObject.statusLabel).toBe("Closed");
|
|
expect(finalObject.refreshStartedAt).toBeNull();
|
|
expect(finalObject.refreshToken).toBeNull();
|
|
expect(slowResolve).toHaveBeenCalledTimes(1);
|
|
expect(replacementResolve).toHaveBeenCalledTimes(1);
|
|
});
|
|
|
|
it("refreshes due objects for active companies only", async () => {
|
|
const active = await createIssue();
|
|
const paused = await createIssue();
|
|
await db
|
|
.update(companies)
|
|
.set({ status: "paused" })
|
|
.where(eq(companies.id, paused.companyId));
|
|
const resolve = vi.fn(async () => ({
|
|
ok: true as const,
|
|
snapshot: {
|
|
statusCategory: "open" as const,
|
|
statusTone: "info" as const,
|
|
statusKey: "open",
|
|
statusLabel: "Open",
|
|
ttlSeconds: 300,
|
|
},
|
|
}));
|
|
const resolver: ExternalObjectResolver = {
|
|
providerKey: "url",
|
|
objectType: "link",
|
|
resolve,
|
|
};
|
|
const svc = externalObjectService(db, { resolvers: [resolver], github: false });
|
|
await svc.syncIssue(active.issueId);
|
|
await svc.syncIssue(paused.issueId);
|
|
|
|
const result = await svc.refreshDueObjectsForActiveCompanies(50, new Date(Date.now() + 1_000));
|
|
|
|
expect(result).toEqual({ companies: 1, checked: 1, refreshed: 1 });
|
|
expect(resolve).toHaveBeenCalledTimes(1);
|
|
const rows = await db.select().from(externalObjects);
|
|
const activeObject = rows.find((row) => row.companyId === active.companyId);
|
|
const pausedObject = rows.find((row) => row.companyId === paused.companyId);
|
|
expect(activeObject?.statusLabel).toBe("Open");
|
|
expect(pausedObject?.statusLabel).toBeNull();
|
|
});
|
|
|
|
it("removes comment mentions when a synced comment is hard-deleted", async () => {
|
|
const { companyId, issueId } = await createIssue();
|
|
const commentId = randomUUID();
|
|
await db.insert(issueComments).values({
|
|
id: commentId,
|
|
companyId,
|
|
issueId,
|
|
authorType: "user",
|
|
authorUserId: "local-board",
|
|
body: "See https://github.com/acme/app/issues/88",
|
|
});
|
|
const svc = externalObjectService(db, { github: false });
|
|
|
|
await svc.syncComment(commentId);
|
|
expect(await db.select().from(externalObjectMentions)).toHaveLength(1);
|
|
|
|
await db.delete(issueComments).where(eq(issueComments.id, commentId));
|
|
await svc.syncComment(commentId);
|
|
|
|
expect(await db.select().from(externalObjectMentions)).toHaveLength(0);
|
|
});
|
|
|
|
it("skips terminal objects when refreshing due objects", async () => {
|
|
const { companyId, issueId } = await createIssue();
|
|
const resolve = vi.fn(async () => ({
|
|
ok: true as const,
|
|
snapshot: {
|
|
statusCategory: "closed" as const,
|
|
statusTone: "muted" as const,
|
|
statusKey: "closed",
|
|
statusLabel: "Closed",
|
|
isTerminal: true,
|
|
ttlSeconds: 1,
|
|
},
|
|
}));
|
|
const resolver: ExternalObjectResolver = {
|
|
providerKey: "url",
|
|
objectType: "link",
|
|
resolve,
|
|
};
|
|
const svc = externalObjectService(db, { resolvers: [resolver], github: false });
|
|
await svc.syncIssue(issueId);
|
|
const object = await db.select().from(externalObjects).then((rows) => rows[0]!);
|
|
|
|
await svc.refreshObject(object.id, { companyId, force: true });
|
|
await db
|
|
.update(externalObjects)
|
|
.set({ nextRefreshAt: new Date(0) })
|
|
.where(eq(externalObjects.id, object.id));
|
|
|
|
const refreshed = await svc.refreshDueObjects(companyId);
|
|
|
|
expect(refreshed).toEqual([]);
|
|
expect(resolve).toHaveBeenCalledTimes(1);
|
|
});
|
|
|
|
it("keeps terminal objects fresh-for-display after their refresh window passes", async () => {
|
|
const { companyId, issueId } = await createIssue();
|
|
const resolve = vi.fn(async () => ({
|
|
ok: true as const,
|
|
snapshot: {
|
|
statusCategory: "closed" as const,
|
|
statusTone: "muted" as const,
|
|
statusKey: "closed",
|
|
statusLabel: "Closed",
|
|
isTerminal: true,
|
|
ttlSeconds: 1,
|
|
},
|
|
}));
|
|
const svc = externalObjectService(db, {
|
|
resolvers: [{ providerKey: "url", objectType: "link", resolve }],
|
|
github: false,
|
|
});
|
|
await svc.syncIssue(issueId);
|
|
const object = await db.select().from(externalObjects).then((rows) => rows[0]!);
|
|
await svc.refreshObject(object.id, { companyId, force: true });
|
|
// Expire the refresh window so the display path computes liveness from
|
|
// scratch. Terminal objects must still show as fresh, never as stale.
|
|
await db
|
|
.update(externalObjects)
|
|
.set({ nextRefreshAt: new Date(0) })
|
|
.where(eq(externalObjects.id, object.id));
|
|
|
|
const summary = await svc.getIssueSummary(issueId);
|
|
const displayed = summary.objects[0];
|
|
expect(displayed?.liveness).toBe("fresh");
|
|
expect(displayed?.isTerminal).toBe(true);
|
|
|
|
const summaryLater = await svc.getIssueSummaries(companyId, [issueId]).then((map) => map.get(issueId));
|
|
expect(summaryLater?.objects[0]?.liveness).toBe("fresh");
|
|
expect(summaryLater?.staleCount).toBe(0);
|
|
expect(summaryLater?.byLiveness?.stale ?? 0).toBe(0);
|
|
});
|
|
|
|
it("re-checks previously frozen GitHub not-found objects so they can self-heal", async () => {
|
|
const { companyId } = await createIssue();
|
|
const canonical = canonicalizeExternalObjectUrl("https://github.com/acme/app/pull/42");
|
|
if (!canonical) throw new Error("expected canonical url");
|
|
const inserted = await db.insert(externalObjects).values({
|
|
companyId,
|
|
providerKey: "github",
|
|
objectType: "pull_request",
|
|
externalId: "acme/app#pull/42",
|
|
sanitizedCanonicalUrl: canonical.sanitizedCanonicalUrl,
|
|
canonicalIdentityHash: canonical.canonicalIdentityHash,
|
|
displayTitle: "acme/app#42",
|
|
statusKey: "not_found",
|
|
statusLabel: "Not found",
|
|
statusCategory: "archived",
|
|
statusTone: "muted",
|
|
liveness: "fresh",
|
|
isTerminal: true,
|
|
nextRefreshAt: new Date(0),
|
|
data: { provider: "github", notFound: true },
|
|
}).returning();
|
|
const object = inserted[0]!;
|
|
|
|
const snapshot = {
|
|
statusKey: "open" as const,
|
|
statusLabel: "Open" as const,
|
|
statusCategory: "open" as const,
|
|
statusTone: "info" as const,
|
|
statusIconKey: "git-pull-request" as const,
|
|
displayTitle: "acme/app#42",
|
|
isTerminal: false,
|
|
ttlSeconds: 300,
|
|
data: { provider: "github", owner: "acme", repo: "app", number: 42 },
|
|
};
|
|
const resolver: ExternalObjectResolver = {
|
|
providerKey: "github",
|
|
objectType: "pull_request",
|
|
resolve: vi.fn(async () => ({ ok: true as const, snapshot })),
|
|
};
|
|
const svc = externalObjectService(db, { resolvers: [resolver], github: false });
|
|
expect(object.isTerminal).toBe(true);
|
|
|
|
const refreshed = await svc.refreshDueObjects(companyId);
|
|
expect(refreshed).toHaveLength(1);
|
|
expect(refreshed[0]?.refreshed).toBe(true);
|
|
|
|
const updated = await db.select().from(externalObjects).then((rows) => rows[0]!);
|
|
expect(updated.statusLabel).toBe("Open");
|
|
expect(updated.isTerminal).toBe(false);
|
|
expect(updated.liveness).toBe("fresh");
|
|
});
|
|
|
|
it("does not re-check probe-verified GitHub not-found objects (deleted references are terminal)", async () => {
|
|
const { companyId } = await createIssue();
|
|
const canonical = canonicalizeExternalObjectUrl("https://github.com/acme/app/pull/42");
|
|
if (!canonical) throw new Error("expected canonical url");
|
|
await db.insert(externalObjects).values({
|
|
companyId,
|
|
providerKey: "github",
|
|
objectType: "pull_request",
|
|
externalId: "acme/app#pull/42",
|
|
sanitizedCanonicalUrl: canonical.sanitizedCanonicalUrl,
|
|
canonicalIdentityHash: canonical.canonicalIdentityHash,
|
|
displayTitle: "acme/app#42",
|
|
statusKey: "not_found",
|
|
statusLabel: "Not found",
|
|
statusCategory: "archived",
|
|
statusTone: "muted",
|
|
liveness: "fresh",
|
|
isTerminal: true,
|
|
nextRefreshAt: new Date(0),
|
|
data: { provider: "github", notFound: true, probeVerified: true },
|
|
});
|
|
const resolver: ExternalObjectResolver = {
|
|
providerKey: "github",
|
|
objectType: "pull_request",
|
|
resolve: vi.fn(async () => ({
|
|
ok: true as const,
|
|
snapshot: {
|
|
statusKey: "open",
|
|
statusLabel: "Open",
|
|
statusCategory: "open" as const,
|
|
statusTone: "info" as const,
|
|
isTerminal: false,
|
|
ttlSeconds: 300,
|
|
},
|
|
})),
|
|
};
|
|
const svc = externalObjectService(db, { resolvers: [resolver], github: false });
|
|
|
|
const refreshed = await svc.refreshDueObjects(companyId);
|
|
|
|
expect(refreshed).toEqual([]);
|
|
expect(resolver.resolve).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("does not re-check terminal objects that are not GitHub not-found verdicts", async () => {
|
|
const { companyId } = await createIssue();
|
|
const canonical = canonicalizeExternalObjectUrl("https://github.com/acme/app/pull/42");
|
|
if (!canonical) throw new Error("expected canonical url");
|
|
await db.insert(externalObjects).values({
|
|
companyId,
|
|
providerKey: "github",
|
|
objectType: "pull_request",
|
|
externalId: "acme/app#pull/42",
|
|
sanitizedCanonicalUrl: canonical.sanitizedCanonicalUrl,
|
|
canonicalIdentityHash: canonical.canonicalIdentityHash,
|
|
displayTitle: "acme/app#42",
|
|
statusKey: "merged",
|
|
statusLabel: "Merged",
|
|
statusCategory: "succeeded",
|
|
statusTone: "success",
|
|
liveness: "fresh",
|
|
isTerminal: true,
|
|
nextRefreshAt: new Date(0),
|
|
});
|
|
const resolver: ExternalObjectResolver = {
|
|
providerKey: "github",
|
|
objectType: "pull_request",
|
|
resolve: vi.fn(async () => ({
|
|
ok: true as const,
|
|
snapshot: {
|
|
statusKey: "open",
|
|
statusLabel: "Open",
|
|
statusCategory: "open" as const,
|
|
statusTone: "info" as const,
|
|
isTerminal: false,
|
|
ttlSeconds: 300,
|
|
},
|
|
})),
|
|
};
|
|
const svc = externalObjectService(db, { resolvers: [resolver], github: false });
|
|
|
|
const refreshed = await svc.refreshDueObjects(companyId);
|
|
|
|
expect(refreshed).toEqual([]);
|
|
expect(resolver.resolve).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("keeps external object identities company-scoped for duplicate urls", async () => {
|
|
const first = await createIssue();
|
|
const second = await createIssue();
|
|
const svc = externalObjectService(db);
|
|
|
|
await svc.syncIssue(first.issueId);
|
|
await svc.syncIssue(second.issueId);
|
|
|
|
const objectRows = await db.select().from(externalObjects);
|
|
expect(objectRows).toHaveLength(2);
|
|
expect(new Set(objectRows.map((row) => row.companyId))).toEqual(new Set([first.companyId, second.companyId]));
|
|
expect(new Set(objectRows.map((row) => row.canonicalIdentityHash)).size).toBe(1);
|
|
});
|
|
|
|
it("uses a mock plugin provider to detect and resolve non-GitHub objects", async () => {
|
|
const { companyId, issueId } = await createIssue();
|
|
await db
|
|
.update(issues)
|
|
.set({
|
|
description: "Track https://mock.example/tickets/123?secret=drop",
|
|
})
|
|
.where(eq(issues.id, issueId));
|
|
|
|
const manifest: PaperclipPluginManifestV1 = {
|
|
id: "paperclip.mock-object-provider",
|
|
apiVersion: 1,
|
|
version: "1.0.0",
|
|
displayName: "Mock Object Provider",
|
|
description: "Detects mock tracker tickets",
|
|
author: "Paperclip",
|
|
categories: ["connector"],
|
|
capabilities: ["external.objects.detect", "external.objects.read"],
|
|
entrypoints: { worker: "dist/worker.js" },
|
|
objectReferences: [
|
|
{
|
|
providerKey: "mocktracker",
|
|
displayName: "Mock Tracker",
|
|
objectTypes: ["ticket"],
|
|
urlPatterns: ["https://mock.example/tickets/:id"],
|
|
},
|
|
],
|
|
};
|
|
const [plugin] = await db.insert(plugins).values({
|
|
pluginKey: manifest.id,
|
|
packageName: "@paperclip/mock-object-provider",
|
|
version: manifest.version,
|
|
apiVersion: 1,
|
|
categories: manifest.categories,
|
|
manifestJson: manifest,
|
|
status: "ready",
|
|
installOrder: 1,
|
|
}).returning();
|
|
|
|
const workerManager = {
|
|
call: vi.fn(async (pluginId: string, method: string, params: any) => {
|
|
expect(pluginId).toBe(plugin!.id);
|
|
if (method === "detectExternalObjects") {
|
|
return {
|
|
detections: params.urls.map((url: any) => ({
|
|
urlIdentityHash: url.canonicalIdentityHash,
|
|
providerKey: "mocktracker",
|
|
objectType: "ticket",
|
|
externalId: "MOCK-123",
|
|
displayKey: "Mock Ticket",
|
|
iconKey: "circle-dot",
|
|
displayTitle: "Mock ticket 123",
|
|
confidence: "exact",
|
|
})),
|
|
};
|
|
}
|
|
if (method === "resolveExternalObject") {
|
|
return {
|
|
ok: true,
|
|
snapshot: {
|
|
displayKey: "Mock Ticket",
|
|
iconKey: "circle-dot",
|
|
displayTitle: `Resolved ${params.externalId}`,
|
|
statusKey: "ready",
|
|
statusLabel: "Ready",
|
|
statusIconKey: "check-circle",
|
|
statusCategory: "succeeded",
|
|
statusTone: "success",
|
|
ttlSeconds: 300,
|
|
},
|
|
};
|
|
}
|
|
throw new Error(`unexpected method ${method}`);
|
|
}),
|
|
} as unknown as PluginWorkerManager;
|
|
|
|
const svc = externalObjectService(db, { pluginWorkerManager: workerManager });
|
|
await svc.syncIssue(issueId);
|
|
|
|
const object = await db.select().from(externalObjects).then((rows) => rows[0]!);
|
|
expect(object).toMatchObject({
|
|
companyId,
|
|
providerKey: "mocktracker",
|
|
objectType: "ticket",
|
|
externalId: "MOCK-123",
|
|
displayKey: "Mock Ticket",
|
|
iconKey: "circle-dot",
|
|
pluginId: plugin!.id,
|
|
sanitizedCanonicalUrl: "https://mock.example/tickets/123",
|
|
});
|
|
expect(JSON.stringify(object)).not.toContain("secret");
|
|
|
|
const refreshed = await svc.refreshObject(object.id, { companyId, force: true });
|
|
expect(refreshed.object).toMatchObject({
|
|
displayKey: "Mock Ticket",
|
|
iconKey: "circle-dot",
|
|
displayTitle: "Resolved MOCK-123",
|
|
statusIconKey: "check-circle",
|
|
statusCategory: "succeeded",
|
|
statusTone: "success",
|
|
liveness: "fresh",
|
|
});
|
|
});
|
|
});
|