Files
PaperClipAI/server/src/__tests__/adapter-model-refresh-routes.test.ts
T
DottaandPaperclip 4d31588dea test(server): load model route setup before test deadlines (#18828)
## Thinking Path

> - Paperclip uses native agents and local provider adapters to do work.
> - The server model route must keep provider catalogs separate.
> - Its tests use the real HTTP route with controlled service and
adapter fixtures.
> - Each test reset the module graph and imported the route again inside
short test deadlines.
> - Slow imports timed out. Unfinished work could then change mock call
counts in a later case.
> - This pull request loads the stable route graph during collection and
keeps each case's fixture resets.

## Linked Issues or Issue Description

Refs #18816 (affected required CI run).

**What happened?**

The pending Kimi model-route test exceeded its 15-second deadline in
required Linux CI. The unchanged test file also failed locally: the
first setup hook exceeded 30 seconds, later tests exceeded 15 seconds,
and the last OpenCode case saw two calls instead of one after unfinished
work reached the shared mock.

**Expected behavior**

Route import and transformation must finish before the case deadline
starts. Each case must still test the real HTTP route with its own mock
state, environment and profile values, and adapter registry cleanup.

**Steps to reproduce**

Run `pnpm exec vitest run --project @paperclipai/server
server/src/__tests__/adapter-model-refresh-routes.test.ts` with a cold
module graph. On a loaded worker, repeated module resets and imports can
exceed the hook and test deadlines.

## What Changed

- Register stable service mocks once and load the real route and
middleware during test collection.
- Remove per-case module invalidation. Keep all five original HTTP
assertions, fixture resets, profile and environment restoration, and
adapter unregister operations.
- Keep the existing hook and test deadlines. Product source code does
not change. No user-facing documentation changes are needed.

## Verification

- The unchanged file reproduced the hook timeout, test timeouts, and
leaked call-count failure locally.
- All five original cases pass with the fix. The cold graph takes about
47 seconds during collection; the cases complete within the unchanged
deadlines.
- All five cases also pass at the same head in a shuffled order with
seed `20261103`.
- Module-boundary and diff checks pass.
- Full repository typecheck, tests, and build run in required CI. No
hosted product behavior is claimed from this test-only change.
- Current-head CI attempt 2 failed in the typecheck harness because the
trusted master workflow called `scripts/typecheck-test-projects.mjs`,
which the older source head did not contain. The log is retained. A
normal merge of master supplies the helper from #18738. All five
original HTTP cases pass again on that merged baseline. Fresh CI and
exact-head reviews passed at `86f0bf27267fe3f44ab813fb0a57ebbb94bae7fe`.

- The refreshed run passed all server checks but one browser shard
failed: `task-reload.spec.ts:63` waited five seconds for a task heading
after the intentional bundle failure. The trace showed the saved task
breadcrumb and properties, with the conversation still loading. The
exact failure log and trace were retained. One targeted retry of that
shard and its aggregate passed without source changes. This establishes
recovery of the check; the original browser loading delay remains
unexplained.
- Final exact-head checks: 52 success, two skipped, and two neutral.
Both Greptile and Paperclip reviews are 5/5. No review threads remain.

## Risks

- Stable imports share module state across cases. The test retains
explicit mock resets, adapter unregister, and profile/environment
restoration.
- The affected model route reads profile qualification and environment
fixtures when it handles the request. These cases do not require a new
module instance for each environment change.
- No runtime or schema changes. No test deadline increase.

## Model Used

OpenAI `gpt-6.1-sol`, Extra High reasoning. The model used repository
tools, code execution, and test output. The host does not expose the
context window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-03 13:50:15 -06:00

277 lines
9.7 KiB
TypeScript

import express from "express";
import request from "supertest";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { models as openCodeFallbackModels } from "@paperclipai/adapter-opencode-local";
import type { ServerAdapterModule } from "../adapters/index.js";
vi.mock("acpx/runtime", () => ({
createAcpRuntime: vi.fn(),
createAgentRegistry: vi.fn(),
createRuntimeStore: vi.fn(),
isAcpRuntimeError: vi.fn(() => false),
}));
const mockAccessService = vi.hoisted(() => ({
canUser: vi.fn(),
hasPermission: vi.fn(),
ensureMembership: vi.fn(),
setPrincipalPermission: vi.fn(),
}));
const mockCompanySkillService = vi.hoisted(() => ({
listRuntimeSkillEntries: vi.fn(),
resolveRequestedSkillKeys: vi.fn(),
}));
const mockSecretService = vi.hoisted(() => ({
normalizeAdapterConfigForPersistence: vi.fn(async (_companyId: string, config: Record<string, unknown>) => config),
resolveAdapterConfigForRuntime: vi.fn(async (_companyId: string, config: Record<string, unknown>) => ({ config })),
}));
const mockEnvironmentService = vi.hoisted(() => ({
getById: vi.fn(),
}));
const mockListOpenCodeModels = vi.hoisted(() => vi.fn());
const mockAgentInstructionsService = vi.hoisted(() => ({
materializeManagedBundle: vi.fn(),
getBundle: vi.fn(),
readFile: vi.fn(),
updateBundle: vi.fn(),
writeFile: vi.fn(),
deleteFile: vi.fn(),
exportFiles: vi.fn(),
ensureManagedBundle: vi.fn(),
}));
const mockBudgetService = vi.hoisted(() => ({
upsertPolicy: vi.fn(),
}));
const mockHeartbeatService = vi.hoisted(() => ({
cancelActiveForAgent: vi.fn(),
}));
const mockIssueApprovalService = vi.hoisted(() => ({
linkManyForApproval: vi.fn(),
}));
const mockApprovalService = vi.hoisted(() => ({
create: vi.fn(),
getById: vi.fn(),
}));
const mockInstanceSettingsService = vi.hoisted(() => ({
getGeneral: vi.fn(async () => ({ censorUsernameInLogs: false })),
}));
const mockLogActivity = vi.hoisted(() => vi.fn());
function registerModuleMocks() {
vi.doMock("@paperclipai/adapter-opencode-local/server", async () => {
const actual = await vi.importActual<typeof import("@paperclipai/adapter-opencode-local/server")>("@paperclipai/adapter-opencode-local/server");
return {
...actual,
listOpenCodeModels: mockListOpenCodeModels,
};
});
vi.doMock("../services/index.js", () => ({
agentService: () => ({}),
agentInstructionsService: () => mockAgentInstructionsService,
accessService: () => mockAccessService,
approvalService: () => mockApprovalService,
builtInAgentService: () => ({ ensureCompanyDefaultAgentGrants: vi.fn() }),
companySkillService: () => mockCompanySkillService,
budgetService: () => mockBudgetService,
heartbeatService: () => mockHeartbeatService,
issueApprovalService: () => mockIssueApprovalService,
issueService: () => ({}),
logActivity: mockLogActivity,
secretService: () => mockSecretService,
syncInstructionsBundleConfigFromFilePath: vi.fn((_agent, config) => config),
workspaceOperationService: () => ({}),
}));
vi.doMock("../services/instance-settings.js", () => ({
instanceSettingsService: () => mockInstanceSettingsService,
}));
vi.doMock("../services/environments.js", () => ({
environmentService: () => mockEnvironmentService,
}));
}
registerModuleMocks();
const [{ agentRoutes }, { errorHandler }] = await Promise.all([
vi.importActual<typeof import("../routes/agents.js")>("../routes/agents.js"),
vi.importActual<typeof import("../middleware/index.js")>("../middleware/index.js"),
]);
const refreshableAdapterType = "refreshable_adapter_route_test";
async function createApp() {
const app = express();
app.use(express.json());
app.use((req, _res, next) => {
(req as any).actor = {
type: "board",
userId: "local-board",
companyIds: ["company-1"],
source: "local_implicit",
isInstanceAdmin: false,
};
next();
});
app.use("/api", agentRoutes({} as any));
app.use(errorHandler);
return app;
}
async function requestApp(
app: express.Express,
buildRequest: (baseUrl: string) => request.Test,
) {
const { createServer } = await vi.importActual<typeof import("node:http")>("node:http");
const server = createServer(app);
try {
await new Promise<void>((resolve) => {
server.listen(0, "127.0.0.1", resolve);
});
const address = server.address();
if (!address || typeof address === "string") {
throw new Error("Expected HTTP server to listen on a TCP port");
}
return await buildRequest(`http://127.0.0.1:${address.port}`);
} finally {
if (server.listening) {
await new Promise<void>((resolve, reject) => {
server.close((error) => {
if (error) reject(error);
else resolve();
});
});
}
}
}
async function unregisterTestAdapter(type: string) {
const { unregisterServerAdapter } = await import("../adapters/index.js");
unregisterServerAdapter(type);
}
describe("adapter model refresh route", () => {
beforeEach(async () => {
vi.clearAllMocks();
mockCompanySkillService.listRuntimeSkillEntries.mockResolvedValue([]);
mockCompanySkillService.resolveRequestedSkillKeys.mockResolvedValue([]);
mockAccessService.canUser.mockResolvedValue(true);
mockAccessService.hasPermission.mockResolvedValue(true);
mockAccessService.ensureMembership.mockResolvedValue(undefined);
mockAccessService.setPrincipalPermission.mockResolvedValue(undefined);
mockLogActivity.mockResolvedValue(undefined);
mockEnvironmentService.getById.mockReset();
mockEnvironmentService.getById.mockResolvedValue(null);
mockListOpenCodeModels.mockReset();
mockListOpenCodeModels.mockResolvedValue([{ id: "dynamic-opencode-model", label: "dynamic-opencode-model" }]);
await unregisterTestAdapter(refreshableAdapterType);
});
afterEach(async () => {
await unregisterTestAdapter(refreshableAdapterType);
});
it.each([
{ qualification: "pending", qualified: false, status: 422 },
{ qualification: "released", qualified: true, status: 200 },
])("keeps $qualification Kimi model selection separate from other provider catalogs", async ({ qualified, status }) => {
const { PAPERCLIP_RUNNER_ACPX_PROFILES } = await import("@paperclipai/adapter-utils");
const kimi = PAPERCLIP_RUNNER_ACPX_PROFILES.find(profile => profile.value === "kimi")! as { qualified: boolean };
const previousQualified = kimi.qualified;
const previousAdmission = process.env.PAPERCLIP_ACPX_CANDIDATE_ADMISSION;
try {
kimi.qualified = qualified;
delete process.env.PAPERCLIP_ACPX_CANDIDATE_ADMISSION;
const app = await createApp();
const res = await requestApp(app, baseUrl =>
request(baseUrl).get("/api/companies/company-1/adapters/paperclip_runner/models?provider=acpx&acpxAgent=kimi&refresh=1"),
);
expect(res.status, JSON.stringify(res.body)).toBe(status);
if (qualified) expect(res.body).toEqual([]);
else expect(res.body).toMatchObject({ code: "paperclip_runner_acpx_agent_unavailable" });
expect(mockListOpenCodeModels).not.toHaveBeenCalled();
} finally {
kimi.qualified = previousQualified;
if (previousAdmission === undefined) delete process.env.PAPERCLIP_ACPX_CANDIDATE_ADMISSION;
else process.env.PAPERCLIP_ACPX_CANDIDATE_ADMISSION = previousAdmission;
}
});
it("uses refreshModels when refresh=1 is requested", async () => {
const listModels = vi.fn(async () => [{ id: "stale-model", label: "stale-model" }]);
const refreshModels = vi.fn(async () => [{ id: "fresh-model", label: "fresh-model" }]);
const { registerServerAdapter } = await import("../adapters/index.js");
const adapter: ServerAdapterModule = {
type: refreshableAdapterType,
execute: async () => ({ exitCode: 0, signal: null, timedOut: false }),
testEnvironment: async () => ({
adapterType: refreshableAdapterType,
status: "pass",
checks: [],
testedAt: new Date(0).toISOString(),
}),
listModels,
refreshModels,
};
registerServerAdapter(adapter);
const app = await createApp();
const res = await requestApp(app, (baseUrl) =>
request(baseUrl).get(`/api/companies/company-1/adapters/${refreshableAdapterType}/models?refresh=1`),
);
expect(res.status, JSON.stringify(res.body)).toBe(200);
expect(res.body).toEqual([{ id: "fresh-model", label: "fresh-model" }]);
expect(refreshModels).toHaveBeenCalledTimes(1);
expect(listModels).not.toHaveBeenCalled();
});
it("skips OpenCode model discovery for non-local environments", async () => {
mockEnvironmentService.getById.mockResolvedValue({
id: "env-1",
companyId: "company-1",
name: "Remote SSH",
driver: "ssh",
config: {},
});
const app = await createApp();
const res = await requestApp(app, (baseUrl) =>
request(baseUrl).get("/api/companies/company-1/adapters/opencode_local/models?environmentId=env-1"),
);
expect(res.status, JSON.stringify(res.body)).toBe(200);
expect(res.body).toEqual(openCodeFallbackModels);
expect(mockListOpenCodeModels).not.toHaveBeenCalled();
});
it("keeps OpenCode model discovery enabled for local environments", async () => {
mockEnvironmentService.getById.mockResolvedValue({
id: "env-1",
companyId: "company-1",
name: "Local",
driver: "local",
config: {},
});
const app = await createApp();
const res = await requestApp(app, (baseUrl) =>
request(baseUrl).get("/api/companies/company-1/adapters/opencode_local/models?environmentId=env-1"),
);
expect(res.status, JSON.stringify(res.body)).toBe(200);
expect(res.body).toEqual([{ id: "dynamic-opencode-model", label: "dynamic-opencode-model" }]);
expect(mockListOpenCodeModels).toHaveBeenCalledTimes(1);
});
});