Files
PaperClipAI/server/src/__tests__/adapter-models.test.ts
T
Devin Foley bb7d040894 Switch OpenCode to explicit static/local-aware model selection (#5117)
> **Stacked PR (part 4 of 7).** Depends on:
  - PR #5114
  - PR #5115
  - PR #5116
> Diff against `master` includes commits from earlier PRs in the stack —
the new commit in this PR is the topmost one.

## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies
> - When creating an OpenCode-local agent, Paperclip currently validates
> `adapterConfig.model` against the *Paperclip host's* `opencode models`
output
> - SSH testing surfaced that this blocks creating an OpenCode agent for
an SSH
> environment: the model that exists on the SSH target isn't visible to
the
> host, so creation fails with "OpenCode requires `adapterConfig.model`
in
> provider/model format" even when the operator picked a real remote
model
> - The initial direction was environment-aware model discovery; the
final
> decision was to keep OpenCode on the same explicit-model pattern as
other
> adapters (default + curated list + manual override) and stop blocking
>   creation on host-side discovery
> - This PR does both: the adapter-models endpoint now accepts
`environmentId` and
> probes against the target environment, and the create-time hard gate
is
> replaced by `requireOpenCodeModelId` which validates `provider/model`
*format*
> without requiring host-local discovery. Test/run-time still surfaces
real
>   auth/availability problems
> - The benefit is that operators can create OpenCode agents for remote
> environments without out-of-band setup, and the model picker in the UI
>   reflects the actually-targeted environment

## What Changed

- Added `requireOpenCodeModelId(input)` in
`opencode-local/src/server/models.ts`,
  exported it from the adapter index
- `ensureOpenCodeModelConfiguredAndAvailable` now delegates the format
check to
  `requireOpenCodeModelId`
- `agentsApi.adapterModels(companyId, adapterType, { environmentId })`
now accepts
  an environment ID and passes it as a query parameter
- `queryKeys.agents.adapterModels` now keys on `(companyId, adapterType,
environmentId)`
- `server/src/routes/agents.ts` reads and validates the new query
parameter,
  forwarding it to the adapter's model probe
- `AgentConfigForm.tsx` and `OnboardingWizard.tsx` build the model query
key from
the currently selected default environment ID and disable autodetect for
  `opencode_local` (model selection is explicit)
- `NewAgent.tsx` simplified — no longer special-cases OpenCode
autodetect
- `company-portability.ts` no longer needs OpenCode-specific autodetect
handling
- Tests added/updated:
  `adapter-model-refresh-routes.test.ts`, `adapter-models.test.ts`,
`agent-permissions-routes.test.ts`,
`opencode-local/src/server/models.test.ts`

## Verification

- `pnpm --filter @paperclipai/server test -- adapter-models
adapter-model-refresh agent-permissions`
- `pnpm --filter @paperclipai/adapter-opencode-local test`
- `pnpm --filter @paperclipai/ui test -- AgentConfigForm
OnboardingWizard NewAgent`
- Manual QA in browser:
1. Boot Paperclip on Tailscale-bound port (so it's reachable from
another
machine), create an OpenCode-local agent, switch the default environment
between two installed sandboxes, and confirm the model list refreshes
     per-environment
  2. Submit with a malformed `provider/model` string and verify the new
     `requireOpenCodeModelId` error surfaces
- Before/after screenshots attached for `AgentConfigForm` model picker

## Risks

- Behavioural shift: switching default environment now triggers a model
refetch.
Should be cheap but introduces a new UI loading state for OpenCode
users.
- Removing dynamic autodetect for OpenCode: if any user configured an
agent
without specifying `model` and relied on autodetect populating it, that
agent
will now fail at submit time. Mitigation: validation error is explicit
and
  actionable.
- New query string parameter on `/api/companies/:id/adapter-models` —
older
clients that omit it still work (parameter is optional and defaults to
null).

## Model Used

- OpenAI GPT-5.4 (reasoning effort: high) via Codex CLI
- Provider: OpenAI
- Used to author the code changes in this PR

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [ ] I have updated relevant documentation to reflect my changes — N/A
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-05-03 13:01:34 -07:00

139 lines
4.8 KiB
TypeScript

import { beforeEach, describe, expect, it, vi } from "vitest";
import { models as codexFallbackModels } from "@paperclipai/adapter-codex-local";
import { models as cursorFallbackModels } from "@paperclipai/adapter-cursor-local";
import { models as opencodeFallbackModels } from "@paperclipai/adapter-opencode-local";
import { resetOpenCodeModelsCacheForTests } from "@paperclipai/adapter-opencode-local/server";
import { listAdapterModels, refreshAdapterModels } from "../adapters/index.js";
import { resetCodexModelsCacheForTests } from "../adapters/codex-models.js";
import { resetCursorModelsCacheForTests, setCursorModelsRunnerForTests } from "../adapters/cursor-models.js";
vi.mock("acpx/runtime", () => ({
createAcpRuntime: vi.fn(),
createAgentRegistry: vi.fn(),
createRuntimeStore: vi.fn(),
isAcpRuntimeError: vi.fn(() => false),
}));
describe("adapter model listing", () => {
beforeEach(() => {
delete process.env.OPENAI_API_KEY;
delete process.env.PAPERCLIP_OPENCODE_COMMAND;
resetCodexModelsCacheForTests();
resetCursorModelsCacheForTests();
setCursorModelsRunnerForTests(null);
resetOpenCodeModelsCacheForTests();
vi.restoreAllMocks();
});
it("returns an empty list for unknown adapters", async () => {
const models = await listAdapterModels("unknown_adapter");
expect(models).toEqual([]);
});
it("returns codex fallback models when no OpenAI key is available", async () => {
const fetchSpy = vi.spyOn(globalThis, "fetch");
const models = await listAdapterModels("codex_local");
expect(models).toEqual(codexFallbackModels);
expect(fetchSpy).not.toHaveBeenCalled();
});
it("loads codex models dynamically and merges fallback options", async () => {
process.env.OPENAI_API_KEY = "sk-test";
const fetchSpy = vi.spyOn(globalThis, "fetch").mockResolvedValue({
ok: true,
json: async () => ({
data: [
{ id: "gpt-5-pro" },
{ id: "gpt-5" },
],
}),
} as Response);
const first = await listAdapterModels("codex_local");
const second = await listAdapterModels("codex_local");
expect(fetchSpy).toHaveBeenCalledTimes(1);
expect(first).toEqual(second);
expect(first.some((model) => model.id === "gpt-5-pro")).toBe(true);
expect(first.some((model) => model.id === "codex-mini-latest")).toBe(true);
});
it("refreshes cached codex models on demand", async () => {
process.env.OPENAI_API_KEY = "sk-test";
const fetchSpy = vi.spyOn(globalThis, "fetch")
.mockResolvedValueOnce({
ok: true,
json: async () => ({
data: [{ id: "gpt-5" }],
}),
} as Response)
.mockResolvedValueOnce({
ok: true,
json: async () => ({
data: [{ id: "gpt-5.5" }],
}),
} as Response);
const initial = await listAdapterModels("codex_local");
const refreshed = await refreshAdapterModels("codex_local");
expect(fetchSpy).toHaveBeenCalledTimes(2);
expect(initial.some((model) => model.id === "gpt-5")).toBe(true);
expect(refreshed.some((model) => model.id === "gpt-5.5")).toBe(true);
});
it("falls back to static codex models when OpenAI model discovery fails", async () => {
process.env.OPENAI_API_KEY = "sk-test";
vi.spyOn(globalThis, "fetch").mockResolvedValue({
ok: false,
status: 401,
json: async () => ({}),
} as Response);
const models = await listAdapterModels("codex_local");
expect(models).toEqual(codexFallbackModels);
});
it("returns cursor fallback models when CLI discovery is unavailable", async () => {
setCursorModelsRunnerForTests(() => ({
status: null,
stdout: "",
stderr: "",
hasError: true,
}));
const models = await listAdapterModels("cursor");
expect(models).toEqual(cursorFallbackModels);
});
it("returns opencode fallback models including gpt-5.4", async () => {
process.env.PAPERCLIP_OPENCODE_COMMAND = "__paperclip_missing_opencode_command__";
const models = await listAdapterModels("opencode_local");
expect(models).toEqual(opencodeFallbackModels);
});
it("loads cursor models dynamically and caches them", async () => {
const runner = vi.fn(() => ({
status: 0,
stdout: "Available models: auto, composer-1.5, gpt-5.3-codex-high, sonnet-4.6",
stderr: "",
hasError: false,
}));
setCursorModelsRunnerForTests(runner);
const first = await listAdapterModels("cursor");
const second = await listAdapterModels("cursor");
expect(runner).toHaveBeenCalledTimes(1);
expect(first).toEqual(second);
expect(first.some((model) => model.id === "auto")).toBe(true);
expect(first.some((model) => model.id === "gpt-5.3-codex-high")).toBe(true);
expect(first.some((model) => model.id === "composer-1")).toBe(true);
});
});