mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-28 04:02:18 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents run through adapters; the `opencode-local` adapter shells out to the OpenCode CLI and, before each run, does a pre-flight `opencode models` **availability probe** to fail fast on a misconfigured `provider/model`. > - That probe was written to **throw on any probe failure** — a timeout, a non-zero exit, or a transient `Unexpected error` from the CLI — which aborts the whole heartbeat run. > - In practice the CLI probe fails transiently (provider hiccup, cold cache, momentary CLI error). When that happens *after* the agent has already done its work, the run dies before its terminal disposition is written, so the platform reopens the issue and re-runs it — a spurious crash/re-run loop that affects every agent on the OpenCode adapter. > - This PR makes the probe **non-fatal when it cannot run**: it warns and proceeds with the configured model, letting the real invocation be authoritative. > - It deliberately **keeps** the genuine guard: when the probe *succeeds* and the configured model is absent from a non-empty list, it still throws (this is what catches misconfigured slugs). > - The benefit is that a best-effort pre-flight check can no longer take down an otherwise-healthy run, while the useful misconfiguration guard is retained. ## Linked Issues or Issue Description No public GitHub issue exists; describing inline (bug). **What happened:** an OpenCode-adapter agent run terminated at the adapter level with `` `opencode models` failed: Unexpected error ``. The failure landed after the agent had produced its work, so the terminal-status update never applied and the run was reopened and re-executed. **Expected:** a transient failure of the `opencode models` availability *probe* should not abort the run — the probe is a best-effort pre-flight guard, not a gate. **Actual:** the probe threw on timeout / non-zero exit / empty output, aborting the run and discarding the completed work + disposition. **Scope:** both the local (`models.ts`) and remote/SSH (`execute.ts`) probe paths; affects any agent on the `opencode_local` adapter. Related PRs (context / prior art): - Refs #5119 — added the remote execution-target model-probe validation this PR softens. - Refs #3291 — closed prior attempt to make the `opencode_local` model probe non-blocking (at agent-create time; different entry point). - Refs #8014 — related open work raising the probe timeout (20s → 60s); complementary, not overlapping. ## What Changed - `models.ts` (`ensureOpenCodeModelConfiguredAndAvailable`): if discovery throws (probe can't run) or returns an empty list, **warn and proceed** with the configured model instead of throwing. The "model present in a non-empty list" check is unchanged and still throws when the configured model is genuinely absent. - `execute.ts` (`ensureRemoteOpenCodeModelConfiguredAndAvailable`): remote probe **timeout / non-zero exit / empty output** now warn and return (proceed) instead of throwing. The remote model-absent guard still throws. - `models.test.ts`: the local "discovery cannot run" case now asserts the probe **proceeds** with the configured model (was: asserts it rejects). - `execute.test.ts`: added remote regression tests — non-zero exit, timeout, and empty output all proceed; a successful probe missing the configured model still rejects. ## Verification ```bash pnpm --filter @paperclipai/adapter-opencode-local typecheck # clean # opencode-local server suite (default 5s per-test timeout is too tight for the # heavy SSH tests on some machines; use a realistic timeout): node node_modules/.pnpm/vitest@*/node_modules/vitest/vitest.mjs run \ packages/adapters/opencode-local/src/server/models.test.ts \ packages/adapters/opencode-local/src/server/execute.test.ts \ packages/adapters/opencode-local/src/server/execute.remote.test.ts \ --testTimeout=45000 ``` Result: typecheck clean; all opencode-local server tests pass, including the new remote fail-open tests and the retained "model unavailable on the remote target" guard test. ## Risks - **Fail-open behavior (intentional).** When the probe can't run, a genuinely misconfigured model is no longer caught at pre-flight — it surfaces at the real invocation instead. This is the accepted tradeoff: the probe is best-effort, and the real invocation is authoritative. The high-value guard (probe succeeds + model absent from a non-empty list) is retained, so the common misconfiguration — a bad `provider/model` slug — is still caught. - No API, schema, or migration changes. Behavior change is confined to the two probe helpers. Low risk overall. ## Model Used Anthropic **Claude Opus 4.8** (`claude-opus-4-8`), used via Claude Code with agentic tool use (repo search, file editing, shell/code execution) and extended reasoning. Used to diagnose the crash, implement the fix, and write the tests; the change was reviewed before submission. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change (`fix/opencode-model-probe-non-fatal`) and contains no internal ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes — N/A (internal adapter behavior; no user-facing docs affected) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green (functional gates: tests/build/e2e/typecheck/security). Review/Greptile gate re-running after this update. - [ ] Greptile is 5/5 with no open P2s — re-triggered after addressing both P2s (remote test coverage + this template-complete description) - [x] I will address all Greptile and reviewer comments before requesting merge
129 lines
4.5 KiB
TypeScript
129 lines
4.5 KiB
TypeScript
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
|
|
|
vi.mock("@paperclipai/adapter-utils/execution-target", async (importOriginal) => {
|
|
const actual = (await importOriginal()) as Record<string, unknown>;
|
|
return { ...actual, runAdapterExecutionTargetProcess: vi.fn() };
|
|
});
|
|
|
|
import { ensureRemoteOpenCodeModelConfiguredAndAvailable } from "./execute.js";
|
|
import { runAdapterExecutionTargetProcess } from "@paperclipai/adapter-utils/execution-target";
|
|
|
|
const runProcessMock = vi.mocked(runAdapterExecutionTargetProcess);
|
|
|
|
function probeResult(overrides: Record<string, unknown>) {
|
|
return {
|
|
exitCode: 0,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout: "",
|
|
stderr: "",
|
|
pid: 123,
|
|
startedAt: new Date().toISOString(),
|
|
...overrides,
|
|
} as never;
|
|
}
|
|
|
|
describe("ensureRemoteOpenCodeModelConfiguredAndAvailable", () => {
|
|
afterEach(() => {
|
|
delete process.env.OPENCODE_ALLOW_ALL_MODELS;
|
|
});
|
|
|
|
// The remote/sandbox execution path must honour OPENCODE_ALLOW_ALL_MODELS just
|
|
// like the local path: gateway-routed models (e.g. anthropic/<gateway>/<model>
|
|
// via Bifrost) never appear in `opencode models`, so the availability probe
|
|
// must be skipped. The early return happens before the executionTarget is ever
|
|
// touched, so a bogus target proves the probe was not run.
|
|
const bogusTarget = {} as never;
|
|
|
|
it("skips the remote availability probe when OPENCODE_ALLOW_ALL_MODELS is set in the run env", async () => {
|
|
await expect(
|
|
ensureRemoteOpenCodeModelConfiguredAndAvailable({
|
|
runId: "run-1",
|
|
executionTarget: bogusTarget,
|
|
command: "opencode",
|
|
model: "anthropic/tensorix/deepseek/deepseek-chat-v3.1",
|
|
cwd: "/tmp",
|
|
env: { OPENCODE_ALLOW_ALL_MODELS: "true" },
|
|
timeoutSec: 30,
|
|
graceSec: 5,
|
|
}),
|
|
).resolves.toBeUndefined();
|
|
});
|
|
|
|
it("honours OPENCODE_ALLOW_ALL_MODELS from the process env", async () => {
|
|
process.env.OPENCODE_ALLOW_ALL_MODELS = "1";
|
|
await expect(
|
|
ensureRemoteOpenCodeModelConfiguredAndAvailable({
|
|
runId: "run-2",
|
|
executionTarget: bogusTarget,
|
|
command: "opencode",
|
|
model: "anthropic/tensorix/deepseek/deepseek-chat-v3.1",
|
|
cwd: "/tmp",
|
|
env: {},
|
|
timeoutSec: 30,
|
|
graceSec: 5,
|
|
}),
|
|
).resolves.toBeUndefined();
|
|
});
|
|
|
|
it("still enforces provider/model format even when the bypass flag is set", async () => {
|
|
await expect(
|
|
ensureRemoteOpenCodeModelConfiguredAndAvailable({
|
|
runId: "run-3",
|
|
executionTarget: bogusTarget,
|
|
command: "opencode",
|
|
model: "",
|
|
cwd: "/tmp",
|
|
env: { OPENCODE_ALLOW_ALL_MODELS: "true" },
|
|
timeoutSec: 30,
|
|
graceSec: 5,
|
|
}),
|
|
).rejects.toThrow();
|
|
});
|
|
});
|
|
|
|
describe("ensureRemoteOpenCodeModelConfiguredAndAvailable — probe is non-fatal when it cannot run", () => {
|
|
const target = { kind: "remote", transport: "ssh" } as never;
|
|
const base = {
|
|
runId: "run-probe",
|
|
executionTarget: target,
|
|
command: "opencode",
|
|
cwd: "/tmp",
|
|
env: {} as Record<string, string>,
|
|
timeoutSec: 30,
|
|
graceSec: 5,
|
|
};
|
|
|
|
beforeEach(() => {
|
|
runProcessMock.mockReset();
|
|
});
|
|
|
|
it("proceeds when the remote probe exits non-zero (e.g. a transient `Unexpected error`)", async () => {
|
|
runProcessMock.mockResolvedValueOnce(probeResult({ exitCode: 1, stderr: "Unexpected error" }));
|
|
await expect(
|
|
ensureRemoteOpenCodeModelConfiguredAndAvailable({ ...base, model: "openai/gpt-5" }),
|
|
).resolves.toBeUndefined();
|
|
});
|
|
|
|
it("proceeds when the remote probe times out", async () => {
|
|
runProcessMock.mockResolvedValueOnce(probeResult({ timedOut: true, exitCode: null }));
|
|
await expect(
|
|
ensureRemoteOpenCodeModelConfiguredAndAvailable({ ...base, model: "openai/gpt-5" }),
|
|
).resolves.toBeUndefined();
|
|
});
|
|
|
|
it("proceeds when the remote probe returns no models", async () => {
|
|
runProcessMock.mockResolvedValueOnce(probeResult({ exitCode: 0, stdout: "" }));
|
|
await expect(
|
|
ensureRemoteOpenCodeModelConfiguredAndAvailable({ ...base, model: "openai/gpt-5" }),
|
|
).resolves.toBeUndefined();
|
|
});
|
|
|
|
it("still rejects when the probe succeeds but the configured model is absent (guard retained)", async () => {
|
|
runProcessMock.mockResolvedValueOnce(probeResult({ exitCode: 0, stdout: "openai/gpt-4.1\n" }));
|
|
await expect(
|
|
ensureRemoteOpenCodeModelConfiguredAndAvailable({ ...base, model: "openai/gpt-5" }),
|
|
).rejects.toThrow("Configured OpenCode model is unavailable on the remote execution target");
|
|
});
|
|
});
|