mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-22 16:51:28 +02:00
fix(codex): gate models on the Codex CLI floor before the ChatGPT backend rejects them (#15399)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Codex adapter and the native runner run Codex in local hosts and in managed sandboxes, and the model catalog lists the models an operator can select > - The Codex backend accepts a new model with ChatGPT sign-in only from a recent enough Codex CLI. An older CLI fails every turn with `The '<model>' model is not supported when using Codex with a ChatGPT account.` > - After #14942 added `gpt-6.1-sol`, an operator selected it for an agent in a managed Daytona sandbox. The sandbox image still shipped Codex 0.156.0. The connection test and runs failed with that sentence, which reads like an account problem > - Paperclip only checked the Codex compatibility window (`>=0.149.0 <0.161.0`), so 0.156.0 passed and the failure surfaced from the backend without a cause > - This pull request records the verified Codex CLI floor for each gated model, compares the installed `codex --version` with that floor in the environment Test and in the remote runner, and names the backend rejection when it still happens > - The benefit is a precise, actionable message ("gpt-6.1-sol requires Codex CLI 0.159.0 or newer; detected 0.156.0; promote a sandbox image with Codex 0.160.0") instead of an opaque 400, and no doomed hello probe ## Linked Issues or Issue Description Refs #14942 **Bug: selecting `gpt-6.1-sol` in a managed sandbox fails with a ChatGPT account error.** **Steps to reproduce** 1. Promote a sandbox image that ships Codex CLI 0.156.0. 2. Create a Codex agent with a ChatGPT sign-in connection, select `gpt-6.1-sol`, and run the environment Test or a task in that sandbox. **Expected behavior** The Test or the run tells the operator that the Codex CLI in the sandbox is older than the model needs. **Actual behavior** The Test reports `codex_hello_probe_failed` with `{"type":"error","status":400,"error":{"type":"invalid_request_error","message":"The 'gpt-6.1-sol' model is not supported when using Codex with a ChatGPT account."}}`. A run fails with "The selected model is not supported by the current ChatGPT connection." **Evidence** - Codex 0.156.1 and older are rejected for `gpt-6.1-sol` with ChatGPT sign-in: https://github.com/openai/codex/issues/49396 - Codex 0.159.0 and 0.159.2 are accepted: https://github.com/openai/codex/issues/49464 and https://github.com/decolua/9router/issues/4471 (same error, fixed by raising the client version identity from 0.155.0 to 0.159.0) - Codex 0.157.0 release notes: "Add GPT-6 Sol and Luna to the model catalog": https://github.com/openai/codex/releases/tag/rust-v0.157.0 - GPT-6.1 Sol is included for Plus, Pro, Business, Enterprise, and Edu with ChatGPT sign-in: https://learn.chatgpt.com/docs/models ## What Changed - `packages/adapters/codex-local/src/index.ts`: add `minimumCodexCliVersionForModel` (`gpt-6.1-sol` → 0.159.0, `gpt-6-sol` and `gpt-6-luna` → 0.157.0, with the sources above), `parseCodexCliVersionOutput`, `codexCliVersionAtLeast`, and `CODEX_CHATGPT_MODEL_REJECTION_RE`. Models without a verified floor return `null`, so nothing probes them. The agent configuration doc describes the floors. - `packages/adapters/codex-local/src/server/cli-version.ts` (new): run `codex --version` where the run would execute it (local, SSH, or sandbox), compare it with the model floor, and build the `codex_cli_version_compatible` / `codex_cli_version_incompatible` checks. Map the backend rejection sentence to `codex_hello_probe_model_rejected` with the detected CLI version and a hint that separates a stale CLI from a plan that does not include the model. - `packages/adapters/codex-local/src/server/test.ts` (CLI lane Test): run the version check before the hello probe when the model has a floor. Skip the hello probe when the CLI is too old (`codex_hello_probe_skipped_cli_version`). When the probe still fails with the backend sentence, report `codex_hello_probe_model_rejected` instead of the generic `codex_hello_probe_failed`. The new codes do not match the auth-failure patterns, so a managed connection is not invalidated. - `packages/adapters/codex-local/src/server/acp.ts` (ACP lane Test): for remote targets, run the same version check against the shared `codex` the ACP server spawns. - `server/src/services/native-runtime/native-session-executor.ts`: after the compatibility-window check, compare the remote Codex with the configured model's floor and fail with `runner_remote_provider_artifact_incompatible: <model> requires Codex <floor> or newer with ChatGPT sign-in, received <version> from the sandbox image; promote a sandbox image with Codex 0.160.0 or configure PAPERCLIP_RUNNER_REMOTE_CODEX_NPM_SPEC=...`. When a preinstalled Codex fails this check and an npm spec is configured, the existing fallback installs the pinned release. - Tests: adapter metadata, CLI-lane Test (too old, compatible, no floor, backend rejection), ACP-lane Test (too old, compatible, no floor), and the remote runner floor (nine version/model cases). - `doc/adapter-model-audit-2026-10-02.md`: record the floors and the reason. No catalog entry, default model, saved agent configuration, pin, lockfile, or image definition changes. ## Verification Local (Node 25.9, pnpm 9.15.4 via corepack): ```sh pnpm --filter @paperclipai/adapter-codex-local typecheck pnpm --filter @paperclipai/adapter-codex-local exec vitest run # 81 passed pnpm --filter @paperclipai/server exec vitest run \ src/services/native-runtime/native-session-executor.test.ts \ src/services/native-runtime/codex-runtime-compatibility.test.ts -t Codex # 84 passed ``` Also run locally: the full `native-session-executor.test.ts` file (533 passed) and the full Codex adapter suite (81 passed). Follow-up commit `9fd4d3b25` (Greptile P2: the ACP-lane version probe ignored the agent's configured env): the probe now receives the adapter's string-valued `env` entries, the same ones `buildCodexAcpConfig` hands to remote ACP runs, so a `PATH` override selects the same `codex` for the Test as for the run. New test covers a `PATH` + `CODEX_HOME` override and a dropped non-string entry. Re-run on that head: `pnpm --filter @paperclipai/adapter-codex-local typecheck` clean; full Codex adapter suite 503 passed (31 files). Not run here: `pnpm -r typecheck` for `server` (the direct `tsc --noEmit` was killed by the sandbox memory cap; the adapter package typecheck passes and CI covers the server), `pnpm build`, browser suites, and a live sandbox probe. No UI files changed, so token gates do not apply. Manual check for a reviewer: set an agent's Codex model to `gpt-6.1-sol`, point its environment at a sandbox whose `codex --version` prints `codex-cli 0.156.0`, and run the environment Test. The result contains `codex_cli_version_incompatible` with `Detected Codex CLI 0.156.0.` and no hello probe check. With Codex 0.160.0 the Test contains `codex_cli_version_compatible` and runs the hello probe. ## Risks - A floor is enforced for all authentication modes, because the runner does not know the credential method at verification time. An OpenAI API key on Codex 0.157.0 or 0.158.0 with `gpt-6.1-sol` is now rejected before launch. Paperclip pins Codex 0.160.0 everywhere, so this only affects installs that lag the pin, and the message names the fix. - The 0.159.0 floor for `gpt-6.1-sol` is the oldest stable release verified to work; 0.157.0 and 0.158.0 were not verified either way. If OpenAI accepts an older client, the floor can be lowered in one table. - The `codex --version` probe adds one short process run to the Test only for models with a floor. - Rollback: revert this pull request. No data or configuration migrates. ## Model Used Claude Fable 5.1 (`claude-fable-5-1`), Anthropic, with extended thinking, tool use, and web research, operating as a Paperclip agent through Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Bender (Fable) <noreply@paperclip.ing> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
10 files changed
+699
-3
No files matched your search
@@ -11968,6 +11968,95 @@ describe("runnerd provider runtime wiring", () => {
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ version: "0.156.0", model: "gpt-6.1-sol", floor: "0.159.0" },
|
||||
{ version: "0.158.0", model: "gpt-6.1-sol", floor: "0.159.0" },
|
||||
{ version: "0.156.0", model: "gpt-6-sol", floor: "0.157.0" },
|
||||
{ version: "0.156.0", model: "gpt-6-luna", floor: "0.157.0" },
|
||||
{ version: "0.159.0", model: "gpt-6.1-sol", floor: null },
|
||||
{ version: "0.160.0", model: "gpt-6.1-sol", floor: null },
|
||||
{ version: "0.157.0", model: "gpt-6-sol", floor: null },
|
||||
{ version: "0.156.0", model: "gpt-5.6-sol", floor: null },
|
||||
{ version: "0.156.0", model: null, floor: null },
|
||||
])("rejects a sandbox Codex $version below the ChatGPT floor for $model before launch (floor=$floor)", async ({ version, model, floor }) => {
|
||||
// A preinstalled Codex inside the compatibility window can still be too
|
||||
// old for the configured model: the ChatGPT backend rejects every turn
|
||||
// with "not supported when using Codex with a ChatGPT account". The
|
||||
// verifier names the stale image instead of letting the run fail as an
|
||||
// account problem.
|
||||
const gatedExecution = {
|
||||
...execution,
|
||||
provider: { kind: "codex", model, approvalPolicy: "never" },
|
||||
binding: { ...execution.binding, runId: `run-codex-floor-${version}-${model ?? "default"}` },
|
||||
} as NativeExecutionInputV1;
|
||||
const onLog = vi.fn(async () => undefined);
|
||||
const syncIn = vi.fn(async () => undefined);
|
||||
const remoteExecute = vi.fn(
|
||||
async (command: { command: string; args?: string[] }) => {
|
||||
let stdout = "";
|
||||
const script = command.args?.[1] ?? "";
|
||||
if (command.args?.[0] === "--build-metadata") {
|
||||
stdout = JSON.stringify({
|
||||
schema: "paperclip-runner/runnerd-build-metadata/v1",
|
||||
binaryName: "paperclip-runnerd",
|
||||
packageName: "@paperclipai/paperclip-runner",
|
||||
binaryContractVersion: 2,
|
||||
durableSessionCapabilities: ["unlimited_runtime", "connection_lease_renewal"],
|
||||
prpTransportModes: ["listen_ws"],
|
||||
});
|
||||
} else if (command.args?.[0] === "--version") {
|
||||
if (command.command.endsWith("/.paperclip-runtime/paperclip-runner/bin/codex")) {
|
||||
throw new Error("reached-preinstalled-codex-verification");
|
||||
}
|
||||
stdout = `codex-cli ${version}`;
|
||||
} else if (script === "uname -s; uname -m") {
|
||||
stdout = `${process.platform === "darwin" ? "Darwin" : "Linux"}\n${process.arch === "arm64" ? "arm64" : "x86_64"}\n`;
|
||||
} else if (script.includes("command -v paperclip-runnerd")) {
|
||||
stdout = "/usr/local/bin/paperclip-runnerd\n";
|
||||
} else if (script.includes("command -v codex")) {
|
||||
stdout = "/opt/paperclip-runner/bin/codex\n";
|
||||
} else if (!script.includes("ln -sfn") && !script.includes("paperclip_codex_launcher_tmp")) {
|
||||
throw new Error(`unexpected command: ${command.command}`);
|
||||
}
|
||||
return { exitCode: 0, signal: null, timedOut: false, stderr: "", stdout };
|
||||
},
|
||||
);
|
||||
await createRunnerdBackend({
|
||||
db: leaseDb(gatedExecution),
|
||||
execution: gatedExecution,
|
||||
runnerInstanceId: "runner-image-runtime",
|
||||
onLog,
|
||||
runnerIngressAuthorized: true,
|
||||
runnerExecutionTarget: {
|
||||
kind: "remote",
|
||||
transport: "sandbox",
|
||||
remoteCwd: "/workspace",
|
||||
environmentId: "environment",
|
||||
leaseId: "lease",
|
||||
providerKey: "daytona",
|
||||
effectiveCapabilities: { runnerWebSocketIngress: true },
|
||||
runner: { execute: remoteExecute, syncIn },
|
||||
} as never,
|
||||
});
|
||||
state.createTransport.mockClear();
|
||||
state.createBackend.mock.calls.at(-1)![1].codexTransportFactory!();
|
||||
const transport = state.createTransport.mock
|
||||
.calls[0]![0] as RunnerTransportOptions & {
|
||||
controlPlaneRegistration: (authority: unknown) => Promise<unknown>;
|
||||
};
|
||||
await expect(transport.controlPlaneRegistration({})).rejects.toThrow(
|
||||
floor
|
||||
? `runner_remote_provider_artifact_incompatible: ${model} requires Codex ${floor} or newer with ChatGPT sign-in, received ${version} from the sandbox image; promote a sandbox image with Codex 0.160.0 or configure PAPERCLIP_RUNNER_REMOTE_CODEX_NPM_SPEC=@openai/codex@0.160.0`
|
||||
: "reached-preinstalled-codex-verification",
|
||||
);
|
||||
if (floor) {
|
||||
// No npm spec is configured, so there is no fallback install; the run
|
||||
// fails before launch instead of running against the stale CLI.
|
||||
expect(remoteExecute.mock.calls.some(([call]) => call.command === "npm")).toBe(false);
|
||||
expect(remoteExecute.mock.calls.some(([call]) => call.args?.[1]?.includes("paperclip_codex_launcher_tmp"))).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
it("binds a remote launch to the configured controller-owned runner artifact", async () => {
|
||||
const remoteCwd = "/home/daytona/paperclip-workspace";
|
||||
const controllerArtifact = "/controller/artifacts/paperclip-runnerd";
|
||||
|
||||
Reference in new issue
Block a user