mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 13:01:05 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent setup lists the models that an OpenRouter key can select. The server gets this list from the public OpenRouter catalog. > - The catalog request has its own 10 second deadline. When the deadline expired, fetch rejected with a raw `TimeoutError` (a `DOMException`). > - The error handler answered 500 "Internal server error" and sent a crash report with no first-party stack frames. The user did not see the existing retry guidance. > - A slow or unavailable third-party catalog is not a fault in the Paperclip server. It must not create a crash report. > - This pull request maps catalog outages to a 503 response with the existing retry message and a stable code. Only those errors skip crash reporting. > - The benefit: users see "Retry or enter a model ID manually", and Sentry does not get noise from a provider outage. ## Linked Issues or Issue Description **What happened?** A staging server answered `GET /api/companies/:companyId/adapters/opencode_local/models` with `500 — The operation was aborted due to timeout`. At the same time, other outbound requests from the host were slow. Sentry recorded `TimeoutError: The operation was aborted due to timeout`. The event had only Node internal frames (`AbortSignal.timeout`). **Expected behavior** The route returns a clear "catalog unavailable" response. The UI shows the retry guidance. A third-party outage does not create a crash report. **Steps to reproduce** 1. Make `https://openrouter.ai/api/v1/models` slow (no answer for more than 10 seconds), or make it unreachable. 2. Request the OpenCode model list with `provider=openrouter`. 3. See a 500 response and a Sentry `TimeoutError` event. **Paperclip version or commit** `e23549cd9c631e4595a3213e03d0c960360452c6` I searched for duplicate PRs. #17083, #17961 and #10290 change the OpenRouter catalog contents or model validation. They do not change how the server handles a catalog timeout. ## What Changed - `server/src/services/openrouter-models.ts`: The catalog now maps three failure types to `HttpError(503)` with code `openrouter_catalog_unavailable` and the existing retry message: its own deadline, a classified transport failure (uses `classifyRemoteConnectionError`), and a 408, 429 or 5xx status. A module-local `WeakSet` records only the errors that it creates for these outages. `isOpenRouterCatalogUnavailable()` reads that record. - Other non-OK statuses become a 502 with code `openrouter_catalog_error`. These are still reported. Malformed JSON and an invalid catalog shape are also still reported. - `server/src/middleware/error-handler.ts`: The handler does not send a crash report for an error from `isOpenRouterCatalogUnavailable()`. It still answers 503 with the message and code. The new check is next to the existing MCP connection-failure exclusion. - Tests: unit tests for the deadline, DNS, 429 and 5xx cases, and for the failures that stay reportable. A route-level test sends a stalled catalog request through the real error handler. It checks for a 503 response and no crash report. It also checks that a different 503 is still reported. ## Verification - `vitest run server/src/services/openrouter-models.test.ts`: 7 tests pass. I ran this file alone because the full workspace install did not fit in the agent sandbox disk. - Red check: with the original `openrouter-models.ts`, 5 of the 7 tests fail. The deadline test receives `DOMException TimeoutError`, which is the same error as the production event. - `server/src/__tests__/openrouter-catalog-error-handler.test.ts` needs the full workspace dependencies. CI runs it. ## Risks - Low risk. Successful responses do not change. The cache and in-flight deduplication do not change. - The failure status changes from 500 to 503 (or 502) for the three routes that read this catalog: adapter models, fast-response models and prompt-injection-prevention models. The UI already shows `error.message`, so users now see the retry guidance in place of "Internal server error". - The crash-report exclusion is narrow. It applies only to error objects that this module created after an actual outage. A copied message or code does not match. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), in the Claude Code agent harness with tool use (shell, file edits, Sentry, Railway and GitHub connectors). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
82 lines
4.6 KiB
TypeScript
82 lines
4.6 KiB
TypeScript
import { afterEach, expect, it, vi } from "vitest";
|
|
|
|
afterEach(() => { vi.unstubAllGlobals(); vi.resetModules(); });
|
|
|
|
it("lists and caches public OpenRouter models in popularity order without sending credentials", async () => {
|
|
const fetch = vi.fn().mockResolvedValue({ ok: true, json: async () => ({ data: [
|
|
{ id: "z-ai/glm-5", name: "Z.AI GLM" },
|
|
{ id: "anthropic/claude-sonnet-4.5", name: "Claude Sonnet" }, { id: 42 },
|
|
] }) });
|
|
vi.stubGlobal("fetch", fetch);
|
|
const { listOpenRouterModels } = await import("./openrouter-models.js");
|
|
expect(await listOpenRouterModels()).toEqual([
|
|
{ id: "openrouter/z-ai/glm-5", label: "Z.AI GLM" },
|
|
{ id: "openrouter/anthropic/claude-sonnet-4.5", label: "Claude Sonnet" },
|
|
]);
|
|
await listOpenRouterModels();
|
|
expect(fetch).toHaveBeenCalledTimes(1);
|
|
expect(fetch).toHaveBeenCalledWith("https://openrouter.ai/api/v1/models?sort=most-popular", { signal: expect.any(AbortSignal) });
|
|
await listOpenRouterModels(true);
|
|
expect(fetch).toHaveBeenCalledTimes(2);
|
|
});
|
|
|
|
it("allows retry after a public catalog failure", async () => {
|
|
const fetch = vi.fn().mockResolvedValueOnce({ ok: false }).mockResolvedValueOnce({ ok: true, json: async () => ({ data: [] }) });
|
|
vi.stubGlobal("fetch", fetch);
|
|
const { listOpenRouterModels } = await import("./openrouter-models.js");
|
|
await expect(listOpenRouterModels()).rejects.toThrow("Retry or enter a model ID manually");
|
|
await expect(listOpenRouterModels()).resolves.toEqual([]);
|
|
});
|
|
|
|
/** A fetch that only settles when its signal fires, as a stalled upstream does. */
|
|
function stalledFetch() {
|
|
return vi.fn((_url: string, init: { signal: AbortSignal }) => new Promise((_resolve, reject) => {
|
|
init.signal.addEventListener("abort", () => reject(init.signal.reason));
|
|
}));
|
|
}
|
|
|
|
function shortCatalogDeadline() {
|
|
const timeout = AbortSignal.timeout.bind(AbortSignal);
|
|
return vi.spyOn(AbortSignal, "timeout").mockImplementation(() => timeout(5));
|
|
}
|
|
|
|
it("answers its own catalog deadline as an unavailable provider, not a raw TimeoutError", async () => {
|
|
const deadline = shortCatalogDeadline();
|
|
vi.stubGlobal("fetch", stalledFetch());
|
|
const { listOpenRouterModels, isOpenRouterCatalogUnavailable } = await import("./openrouter-models.js");
|
|
const error = await listOpenRouterModels().catch((err: unknown) => err);
|
|
deadline.mockRestore();
|
|
expect(error).toMatchObject({ status: 503, message: "Could not load OpenRouter models. Retry or enter a model ID manually.",
|
|
details: { code: "openrouter_catalog_unavailable" } });
|
|
expect(isOpenRouterCatalogUnavailable(error)).toBe(true);
|
|
});
|
|
|
|
it.each([
|
|
["a DNS failure", () => Promise.reject(Object.assign(new TypeError("fetch failed"), { cause: Object.assign(new Error("getaddrinfo ENOTFOUND openrouter.ai"), { code: "ENOTFOUND" }) }))],
|
|
["a rate limit", () => Promise.resolve({ ok: false, status: 429 })],
|
|
["a provider server error", () => Promise.resolve({ ok: false, status: 503 })],
|
|
])("marks %s as an unavailable catalog", async (_label, response) => {
|
|
vi.stubGlobal("fetch", vi.fn(response));
|
|
const { listOpenRouterModels, isOpenRouterCatalogUnavailable } = await import("./openrouter-models.js");
|
|
const error = await listOpenRouterModels().catch((err: unknown) => err);
|
|
expect(error).toMatchObject({ status: 503, details: { code: "openrouter_catalog_unavailable" } });
|
|
expect(isOpenRouterCatalogUnavailable(error)).toBe(true);
|
|
});
|
|
|
|
it("keeps contract failures and unrelated errors reportable", async () => {
|
|
const { HttpError } = await import("../errors.js");
|
|
vi.stubGlobal("fetch", vi.fn()
|
|
.mockResolvedValueOnce({ ok: false, status: 404 })
|
|
.mockResolvedValueOnce({ ok: true, json: async () => { throw new SyntaxError("Unexpected token <"); } })
|
|
.mockResolvedValueOnce({ ok: true, json: async () => ({ data: null }) }));
|
|
const { listOpenRouterModels, isOpenRouterCatalogUnavailable } = await import("./openrouter-models.js");
|
|
const notFound = await listOpenRouterModels().catch((err: unknown) => err);
|
|
expect(notFound).toMatchObject({ status: 502, details: { code: "openrouter_catalog_error", status: 404 } });
|
|
expect(isOpenRouterCatalogUnavailable(notFound)).toBe(false);
|
|
const malformed = await listOpenRouterModels().catch((err: unknown) => err);
|
|
expect(malformed).toBeInstanceOf(SyntaxError);
|
|
expect(isOpenRouterCatalogUnavailable(malformed)).toBe(false);
|
|
await expect(listOpenRouterModels()).rejects.toThrow("OpenRouter returned an invalid model catalog.");
|
|
expect(isOpenRouterCatalogUnavailable(new HttpError(503, "Could not load OpenRouter models. Retry or enter a model ID manually.", { code: "openrouter_catalog_unavailable" }))).toBe(false);
|
|
});
|