mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-01 06:42:06 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Codex adapter and the native runner run Codex in local hosts and in managed sandboxes, and the model catalog lists the models an operator can select > - The Codex backend accepts a new model with ChatGPT sign-in only from a recent enough Codex CLI. An older CLI fails every turn with `The '<model>' model is not supported when using Codex with a ChatGPT account.` > - After #14942 added `gpt-6.1-sol`, an operator selected it for an agent in a managed Daytona sandbox. The sandbox image still shipped Codex 0.156.0. The connection test and runs failed with that sentence, which reads like an account problem > - Paperclip only checked the Codex compatibility window (`>=0.149.0 <0.161.0`), so 0.156.0 passed and the failure surfaced from the backend without a cause > - This pull request records the verified Codex CLI floor for each gated model, compares the installed `codex --version` with that floor in the environment Test and in the remote runner, and names the backend rejection when it still happens > - The benefit is a precise, actionable message ("gpt-6.1-sol requires Codex CLI 0.159.0 or newer; detected 0.156.0; promote a sandbox image with Codex 0.160.0") instead of an opaque 400, and no doomed hello probe ## Linked Issues or Issue Description Refs #14942 **Bug: selecting `gpt-6.1-sol` in a managed sandbox fails with a ChatGPT account error.** **Steps to reproduce** 1. Promote a sandbox image that ships Codex CLI 0.156.0. 2. Create a Codex agent with a ChatGPT sign-in connection, select `gpt-6.1-sol`, and run the environment Test or a task in that sandbox. **Expected behavior** The Test or the run tells the operator that the Codex CLI in the sandbox is older than the model needs. **Actual behavior** The Test reports `codex_hello_probe_failed` with `{"type":"error","status":400,"error":{"type":"invalid_request_error","message":"The 'gpt-6.1-sol' model is not supported when using Codex with a ChatGPT account."}}`. A run fails with "The selected model is not supported by the current ChatGPT connection." **Evidence** - Codex 0.156.1 and older are rejected for `gpt-6.1-sol` with ChatGPT sign-in: https://github.com/openai/codex/issues/49396 - Codex 0.159.0 and 0.159.2 are accepted: https://github.com/openai/codex/issues/49464 and https://github.com/decolua/9router/issues/4471 (same error, fixed by raising the client version identity from 0.155.0 to 0.159.0) - Codex 0.157.0 release notes: "Add GPT-6 Sol and Luna to the model catalog": https://github.com/openai/codex/releases/tag/rust-v0.157.0 - GPT-6.1 Sol is included for Plus, Pro, Business, Enterprise, and Edu with ChatGPT sign-in: https://learn.chatgpt.com/docs/models ## What Changed - `packages/adapters/codex-local/src/index.ts`: add `minimumCodexCliVersionForModel` (`gpt-6.1-sol` → 0.159.0, `gpt-6-sol` and `gpt-6-luna` → 0.157.0, with the sources above), `parseCodexCliVersionOutput`, `codexCliVersionAtLeast`, and `CODEX_CHATGPT_MODEL_REJECTION_RE`. Models without a verified floor return `null`, so nothing probes them. The agent configuration doc describes the floors. - `packages/adapters/codex-local/src/server/cli-version.ts` (new): run `codex --version` where the run would execute it (local, SSH, or sandbox), compare it with the model floor, and build the `codex_cli_version_compatible` / `codex_cli_version_incompatible` checks. Map the backend rejection sentence to `codex_hello_probe_model_rejected` with the detected CLI version and a hint that separates a stale CLI from a plan that does not include the model. - `packages/adapters/codex-local/src/server/test.ts` (CLI lane Test): run the version check before the hello probe when the model has a floor. Skip the hello probe when the CLI is too old (`codex_hello_probe_skipped_cli_version`). When the probe still fails with the backend sentence, report `codex_hello_probe_model_rejected` instead of the generic `codex_hello_probe_failed`. The new codes do not match the auth-failure patterns, so a managed connection is not invalidated. - `packages/adapters/codex-local/src/server/acp.ts` (ACP lane Test): for remote targets, run the same version check against the shared `codex` the ACP server spawns. - `server/src/services/native-runtime/native-session-executor.ts`: after the compatibility-window check, compare the remote Codex with the configured model's floor and fail with `runner_remote_provider_artifact_incompatible: <model> requires Codex <floor> or newer with ChatGPT sign-in, received <version> from the sandbox image; promote a sandbox image with Codex 0.160.0 or configure PAPERCLIP_RUNNER_REMOTE_CODEX_NPM_SPEC=...`. When a preinstalled Codex fails this check and an npm spec is configured, the existing fallback installs the pinned release. - Tests: adapter metadata, CLI-lane Test (too old, compatible, no floor, backend rejection), ACP-lane Test (too old, compatible, no floor), and the remote runner floor (nine version/model cases). - `doc/adapter-model-audit-2026-10-02.md`: record the floors and the reason. No catalog entry, default model, saved agent configuration, pin, lockfile, or image definition changes. ## Verification Local (Node 25.9, pnpm 9.15.4 via corepack): ```sh pnpm --filter @paperclipai/adapter-codex-local typecheck pnpm --filter @paperclipai/adapter-codex-local exec vitest run # 81 passed pnpm --filter @paperclipai/server exec vitest run \ src/services/native-runtime/native-session-executor.test.ts \ src/services/native-runtime/codex-runtime-compatibility.test.ts -t Codex # 84 passed ``` Also run locally: the full `native-session-executor.test.ts` file (533 passed) and the full Codex adapter suite (81 passed). Follow-up commit `9fd4d3b25` (Greptile P2: the ACP-lane version probe ignored the agent's configured env): the probe now receives the adapter's string-valued `env` entries, the same ones `buildCodexAcpConfig` hands to remote ACP runs, so a `PATH` override selects the same `codex` for the Test as for the run. New test covers a `PATH` + `CODEX_HOME` override and a dropped non-string entry. Re-run on that head: `pnpm --filter @paperclipai/adapter-codex-local typecheck` clean; full Codex adapter suite 503 passed (31 files). Not run here: `pnpm -r typecheck` for `server` (the direct `tsc --noEmit` was killed by the sandbox memory cap; the adapter package typecheck passes and CI covers the server), `pnpm build`, browser suites, and a live sandbox probe. No UI files changed, so token gates do not apply. Manual check for a reviewer: set an agent's Codex model to `gpt-6.1-sol`, point its environment at a sandbox whose `codex --version` prints `codex-cli 0.156.0`, and run the environment Test. The result contains `codex_cli_version_incompatible` with `Detected Codex CLI 0.156.0.` and no hello probe check. With Codex 0.160.0 the Test contains `codex_cli_version_compatible` and runs the hello probe. ## Risks - A floor is enforced for all authentication modes, because the runner does not know the credential method at verification time. An OpenAI API key on Codex 0.157.0 or 0.158.0 with `gpt-6.1-sol` is now rejected before launch. Paperclip pins Codex 0.160.0 everywhere, so this only affects installs that lag the pin, and the message names the fix. - The 0.159.0 floor for `gpt-6.1-sol` is the oldest stable release verified to work; 0.157.0 and 0.158.0 were not verified either way. If OpenAI accepts an older client, the floor can be lowered in one table. - The `codex --version` probe adds one short process run to the Test only for models with a floor. - Rollback: revert this pull request. No data or configuration migrates. ## Model Used Claude Fable 5.1 (`claude-fable-5-1`), Anthropic, with extended thinking, tool use, and web research, operating as a Paperclip agent through Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Bender (Fable) <noreply@paperclip.ing> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
183 lines
6.3 KiB
TypeScript
183 lines
6.3 KiB
TypeScript
import fs from "node:fs/promises";
|
|
import os from "node:os";
|
|
import path from "node:path";
|
|
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
|
import type { AdapterExecutionTarget } from "@paperclipai/adapter-utils/execution-target";
|
|
|
|
const { ensureAdapterExecutionTargetCommandResolvable, runAdapterExecutionTargetProcess } = vi.hoisted(() => ({
|
|
ensureAdapterExecutionTargetCommandResolvable: vi.fn(async () => {}),
|
|
runAdapterExecutionTargetProcess: vi.fn(async () => ({
|
|
exitCode: 0,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout: "codex-cli 0.156.0\n",
|
|
stderr: "",
|
|
pid: 1,
|
|
startedAt: new Date().toISOString(),
|
|
})),
|
|
}));
|
|
|
|
vi.mock("@paperclipai/adapter-utils/execution-target", async () => {
|
|
const actual = await vi.importActual<typeof import("@paperclipai/adapter-utils/execution-target")>(
|
|
"@paperclipai/adapter-utils/execution-target",
|
|
);
|
|
return { ...actual, ensureAdapterExecutionTargetCommandResolvable, runAdapterExecutionTargetProcess };
|
|
});
|
|
|
|
import { testCodexAcpEnvironment } from "./acp.js";
|
|
|
|
describe("codex ACP lane model CLI floor", () => {
|
|
let cwd: string;
|
|
|
|
beforeEach(async () => {
|
|
cwd = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-acp-version-"));
|
|
vi.stubEnv("CODEX_HOME", await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-acp-home-")));
|
|
vi.stubEnv("PAPERCLIP_HOME", await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-acp-instance-")));
|
|
vi.stubEnv("PAPERCLIP_INSTANCE_ID", "default");
|
|
});
|
|
|
|
afterEach(async () => {
|
|
vi.clearAllMocks();
|
|
vi.unstubAllEnvs();
|
|
await fs.rm(cwd, { recursive: true, force: true }).catch(() => undefined);
|
|
});
|
|
|
|
function sandboxTarget(): AdapterExecutionTarget {
|
|
return {
|
|
kind: "remote",
|
|
transport: "sandbox",
|
|
providerKey: "daytona",
|
|
remoteCwd: "/remote/workspace",
|
|
runner: {
|
|
execute: async () => ({
|
|
exitCode: 0,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout: "",
|
|
stderr: "",
|
|
pid: null,
|
|
startedAt: new Date().toISOString(),
|
|
}),
|
|
},
|
|
};
|
|
}
|
|
|
|
it("fails the sandbox Test when the shared codex is older than the model's floor", async () => {
|
|
const result = await testCodexAcpEnvironment({
|
|
companyId: "company-1",
|
|
adapterType: "codex_local",
|
|
config: { cwd, model: "gpt-6.1-sol", env: { OPENAI_API_KEY: "sk-test" } },
|
|
executionTarget: sandboxTarget(),
|
|
environmentName: "QA Daytona",
|
|
});
|
|
|
|
expect(result.status).toBe("fail");
|
|
expect(result.checks.find((check) => check.code === "codex_cli_version_incompatible")).toMatchObject({
|
|
level: "error",
|
|
message: "gpt-6.1-sol requires Codex CLI 0.159.0 or newer with ChatGPT sign-in.",
|
|
detail: "Detected Codex CLI 0.156.0.",
|
|
});
|
|
expect(runAdapterExecutionTargetProcess).toHaveBeenCalledTimes(1);
|
|
const call = runAdapterExecutionTargetProcess.mock.calls[0] as unknown as [
|
|
string,
|
|
AdapterExecutionTarget,
|
|
string,
|
|
string[],
|
|
{ env: Record<string, string> },
|
|
];
|
|
expect(call[1]).toMatchObject({ kind: "remote", transport: "sandbox" });
|
|
expect(call[2]).toBe("codex");
|
|
expect(call[3]).toEqual(["--version"]);
|
|
expect(call[4].env).toEqual({ OPENAI_API_KEY: "sk-test" });
|
|
});
|
|
|
|
it("probes the codex the agent's configured env selects, not the image default", async () => {
|
|
runAdapterExecutionTargetProcess.mockResolvedValueOnce({
|
|
exitCode: 0,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout: "codex-cli 0.160.0\n",
|
|
stderr: "",
|
|
pid: 1,
|
|
startedAt: new Date().toISOString(),
|
|
});
|
|
|
|
const result = await testCodexAcpEnvironment({
|
|
companyId: "company-1",
|
|
adapterType: "codex_local",
|
|
config: {
|
|
cwd,
|
|
model: "gpt-6.1-sol",
|
|
env: {
|
|
OPENAI_API_KEY: "sk-test",
|
|
PATH: "/opt/codex-0.160.0/bin:/usr/local/bin:/usr/bin",
|
|
CODEX_HOME: "/remote/codex-home",
|
|
PAPERCLIP_CODEX_ACP_NETWORK_ACCESS: false,
|
|
},
|
|
},
|
|
executionTarget: sandboxTarget(),
|
|
environmentName: "QA Daytona",
|
|
});
|
|
|
|
expect(result.status).toBe("pass");
|
|
expect(result.checks.find((check) => check.code === "codex_cli_version_compatible")).toMatchObject({
|
|
message: "Codex CLI 0.160.0 satisfies the 0.159.0 minimum for gpt-6.1-sol.",
|
|
});
|
|
expect(runAdapterExecutionTargetProcess).toHaveBeenCalledTimes(1);
|
|
const call = runAdapterExecutionTargetProcess.mock.calls[0] as unknown as [
|
|
string,
|
|
AdapterExecutionTarget,
|
|
string,
|
|
string[],
|
|
{ env: Record<string, string> },
|
|
];
|
|
expect(call[3]).toEqual(["--version"]);
|
|
// String-valued entries are forwarded exactly as the ACP run receives them;
|
|
// non-string values are dropped the same way buildCodexAcpConfig consumers do.
|
|
expect(call[4].env).toEqual({
|
|
OPENAI_API_KEY: "sk-test",
|
|
PATH: "/opt/codex-0.160.0/bin:/usr/local/bin:/usr/bin",
|
|
CODEX_HOME: "/remote/codex-home",
|
|
});
|
|
});
|
|
|
|
it("passes the sandbox Test when the shared codex satisfies the floor", async () => {
|
|
runAdapterExecutionTargetProcess.mockResolvedValueOnce({
|
|
exitCode: 0,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout: "codex-cli 0.160.0\n",
|
|
stderr: "",
|
|
pid: 1,
|
|
startedAt: new Date().toISOString(),
|
|
});
|
|
|
|
const result = await testCodexAcpEnvironment({
|
|
companyId: "company-1",
|
|
adapterType: "codex_local",
|
|
config: { cwd, model: "gpt-6.1-sol", env: { OPENAI_API_KEY: "sk-test" } },
|
|
executionTarget: sandboxTarget(),
|
|
environmentName: "QA Daytona",
|
|
});
|
|
|
|
expect(result.status).toBe("pass");
|
|
expect(result.checks.find((check) => check.code === "codex_cli_version_compatible")).toMatchObject({
|
|
message: "Codex CLI 0.160.0 satisfies the 0.159.0 minimum for gpt-6.1-sol.",
|
|
});
|
|
});
|
|
|
|
it("does not probe the sandbox for models without a verified floor", async () => {
|
|
const result = await testCodexAcpEnvironment({
|
|
companyId: "company-1",
|
|
adapterType: "codex_local",
|
|
config: { cwd, model: "gpt-5.6-sol", env: { OPENAI_API_KEY: "sk-test" } },
|
|
executionTarget: sandboxTarget(),
|
|
environmentName: "QA Daytona",
|
|
});
|
|
|
|
expect(result.status).toBe("pass");
|
|
expect(runAdapterExecutionTargetProcess).not.toHaveBeenCalled();
|
|
expect(result.checks.some((check) => check.code.startsWith("codex_cli_version_"))).toBe(false);
|
|
});
|
|
});
|