Files
PaperClipAI/packages/adapters/codex-local/src/server/runtime-config.test.ts
T
Jannes StubbemannandClaude Opus 4.8 9e750d3e92 feat(codex-local): env-driven gateway routing via PAPERCLIP_CODEX_PROVIDERS config.toml (#7919)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The `codex-local` adapter runs the OpenAI Codex CLI; Paperclip
already maintains a managed `CODEX_HOME` per company and ships it to
remote/sandboxed execution targets
> - Deployments increasingly put an OpenAI-compatible LLM gateway
between the harness and the model for cost, governance, or
data-residency reasons: LiteLLM, OpenRouter, Portkey, Kong, a corporate
proxy, self-hosted models (vLLM/Ollama), or region-pinned/sovereign
endpoints. But Codex has no CLI flag or env var for a custom endpoint:
its only mechanism is `[model_providers.<id>]` tables (with `base_url`,
`env_key`, `wire_api`) in `$CODEX_HOME/config.toml`, selected by a
root-level `model_provider` key
> - Today there is no supported way to get such provider config into the
managed `CODEX_HOME`, so gateway routing requires hand-editing files the
adapter owns and regenerates
> - This pull request adds the codex analogue of #7837's opencode
mechanism: a `PAPERCLIP_CODEX_PROVIDERS` JSON env var whose shape maps
1:1 onto codex's TOML schema, merged into the managed `config.toml` so
the existing asset-shipping + `env.CODEX_HOME` mechanics deliver it to
local and sandboxed runs alike; nothing here is specific to one hosting
setup
> - The benefit is Codex works behind any OpenAI-compatible gateway with
config only; with no env set, behavior is unchanged

## Linked Issues or Issue Description

No existing issue; describing in-PR (feature / adapter enhancement).

- **Gap:** there is no supported way to register a custom/gateway
`[model_providers.*]` endpoint for `codex-local`. Codex's only
custom-endpoint mechanism is `config.toml` (`base_url` + `env_key` +
`wire_api`, selected via the root `model_provider` key), and the adapter
owns/regenerates the managed `CODEX_HOME`, so operators cannot durably
hand-edit it.
- Related: #7837 (the opencode-local analogue of this change, same
env-driven gateway-routing pattern). Searched for duplicate/related PRs:
no existing codex-local gateway/provider-routing PR found.

> Note on ROADMAP: this is adapter-level, opt-in config (defaults
unchanged) that *enables* gateway routing for one harness; it is not the
core "Cloud / Sandbox agents" platform work itself.

## What Changed

- New `prepareCodexRuntimeConfig()`
(`packages/adapters/codex-local/src/server/runtime-config.ts`): reads
`PAPERCLIP_CODEX_PROVIDERS` (run env first, then `process.env`), shaped
as `{"providers": {"<id>": {base_url, env_key, wire_api, ...}},
"model_provider": "<id>"}`, and merges it into the managed
`CODEX_HOME`'s `config.toml`. No-op when unset or empty.
- A malformed value (invalid JSON, not a JSON object, no `providers`
object, no usable provider entries, or individual entries with empty
names or non-object values, which are skipped by name) is never silently
dropped: each case surfaces a distinct, user-visible note (via the
prepare notes, which flow into command notes + `onLog`) and unusable
input leaves `config.toml` untouched.
- Merge is marker-delimited and TOML-correct: existing `config.toml`
content is preserved between two managed blocks. Root keys (e.g.
`model_provider`) are prepended **before the first table header** (TOML
root-region rule), `[model_providers.*]` tables are appended.
Pre-existing same-name provider sections and root `model_provider` keys
are excised so the managed definitions win without duplicate-table parse
errors.
- `{env:VAR}` placeholders are expanded server-side for
literal-credential fields; `env_key` indirection remains the preferred
path.
- Crash-safe restore: prepare writes a pre-run backup
(`config.toml.paperclip-backup`) before the merged file; `cleanup()`
restores the original in the execute `finally` and removes the backup.
If a run never reaches `cleanup()` (a throw during the setup between
prepare and execution, or SIGKILL), the next prepare restores the
original from the backup with full fidelity, including user
`[model_providers.*]` sections the merge excised (review feedback, P2);
plain block-stripping remains the fallback for pre-backup state.
- An explicit adapter-config `env.CODEX_HOME` override is treated as
user-managed: no merge, surfaced as a command note.
- Dependency-free hand-emitted TOML (strings/numbers/booleans, arrays of
scalars, plain objects as inline tables); basic strings escape
U+0000-U+001F and U+007F per TOML 1.0 (review feedback, P2). Merged
output was additionally validated locally with python tomllib during
development; the committed tests assert the structural invariants.
- `execute.ts` wiring: `prepareCodexRuntimeConfig` runs after
`prepareManagedCodexHome` (before the home ships to the remote target),
notes surface via `onLog` + command notes, and the `finally` calls
`cleanup()`.

**Note for reviewers:** current codex removed `wire_api = "chat"`
(openai/codex#10157, Feb 2026), so gateway provider configs must use
`wire_api = "responses"`, i.e. the gateway must speak `/v1/responses`.
The adapter passes the value through verbatim; this is a codex-side
constraint worth knowing when configuring it.

## Verification

- `pnpm --filter @paperclipai/adapter-codex-local build` and
`typecheck`: tsc clean against current `master`
- `pnpm exec vitest run packages/adapters/codex-local`: 45 passing
(incl. 17 `runtime-config` tests: fresh-merge + cleanup restore,
root-region placement, same-name provider override, inline
tables/arrays, DEL escaping, `{env:}` expansion from run env +
`process.env`, per-case malformed-input notes with `config.toml`
untouched, skipped-entry notes alongside a successful merge, silent
no-op when unset/empty, explicit-`CODEX_HOME` skip note, backup restore
of excised user sections after an interrupted run, backup removal on
cleanup, stale-block self-heal, re-run replacement)
- Verified end-to-end: a codex agent in a hardened Kubernetes (gVisor)
sandbox completed a real task routed through an OpenAI-compatible
gateway's `/v1/responses`, with a billed usage row recorded on the
gateway. That deployment supplies the verification evidence; the
mechanism is gateway-agnostic.

## Risks

Low. Entirely env-driven and opt-in; with `PAPERCLIP_CODEX_PROVIDERS`
unset the adapter never touches `config.toml` and behavior is
byte-identical to before. The merge preserves user content, restores the
original file on cleanup, and survives interrupted runs via the pre-run
backup; malformed input surfaces a visible note and is ignored without
touching `config.toml`. No migration/UI impact.

## Model Used

Claude Opus 4.8 (`claude-opus-4-8`, 1M context), extended thinking +
tool use, via Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md (adapter-level opt-in config enabling
gateway routing; not the core sandbox-platform work, noted above)
- [x] I have searched GitHub for duplicate or related PRs and linked
them above (#7837 is the opencode analogue; no codex-local duplicate
found)
- [x] I have either (a) linked existing issues OR (b) described the
issue in-PR following the relevant issue template
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots (n/a, no UI)
- [ ] I have updated relevant documentation to reflect my changes (env
var documented inline; no central doc references the adapter env yet)
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green (green on the previous head;
re-running on the final note-copy polish commit)
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(both review P2s are fixed at head: the interrupted-run restore via the
pre-run backup and the U+007F escaping; a re-review is requested for the
note-copy polish)
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 21:13:31 -07:00

417 lines
16 KiB
TypeScript

import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { prepareCodexRuntimeConfig } from "./runtime-config.js";
const cleanupPaths = new Set<string>();
afterEach(async () => {
await Promise.all(
[...cleanupPaths].map(async (filepath) => {
await fs.rm(filepath, { recursive: true, force: true });
cleanupPaths.delete(filepath);
}),
);
});
async function makeCodexHome(configToml?: string): Promise<string> {
const home = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-home-"));
cleanupPaths.add(home);
if (configToml !== undefined) {
await fs.writeFile(path.join(home, "config.toml"), configToml, "utf8");
}
return home;
}
async function readConfigToml(home: string): Promise<string> {
return fs.readFile(path.join(home, "config.toml"), "utf8");
}
const BIFROST_PROVIDERS = {
providers: {
bifrost: {
name: "bifrost",
base_url: "http://gateway.example.svc.cluster.local:8080/v1",
env_key: "OPENAI_API_KEY",
wire_api: "responses",
},
},
model_provider: "bifrost",
};
describe("prepareCodexRuntimeConfig", () => {
it("is a no-op when PAPERCLIP_CODEX_PROVIDERS is unset", async () => {
const home = await makeCodexHome("model = \"gpt-5.1-codex\"\n");
const prepared = await prepareCodexRuntimeConfig({ env: { FOO: "bar" }, codexHome: home });
expect(prepared.notes).toEqual([]);
expect(await readConfigToml(home)).toBe("model = \"gpt-5.1-codex\"\n");
await prepared.cleanup();
});
it("is a no-op when the home has no config.toml and the env is unset", async () => {
const home = await makeCodexHome();
const prepared = await prepareCodexRuntimeConfig({ env: {}, codexHome: home });
expect(prepared.notes).toEqual([]);
await expect(fs.access(path.join(home, "config.toml"))).rejects.toThrow();
await prepared.cleanup();
});
it("merges providers + model_provider into a fresh config.toml and cleans it up", async () => {
const home = await makeCodexHome();
const prepared = await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
codexHome: home,
});
const content = await readConfigToml(home);
expect(content).toContain('model_provider = "bifrost"');
expect(content).toContain("[model_providers.bifrost]");
expect(content).toContain('base_url = "http://gateway.example.svc.cluster.local:8080/v1"');
expect(content).toContain('env_key = "OPENAI_API_KEY"');
expect(content).toContain('wire_api = "responses"');
expect(prepared.notes.some((n) => n.includes("bifrost"))).toBe(true);
await prepared.cleanup();
await expect(fs.access(path.join(home, "config.toml"))).rejects.toThrow();
});
it("preserves existing config.toml content, keeps model_provider in the root region, and restores on cleanup", async () => {
const original = [
'model = "gpt-5.1-codex"',
"",
"[profiles.dev]",
'model = "gpt-5.1-codex-mini"',
"",
].join("\n");
const home = await makeCodexHome(original);
const prepared = await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
codexHome: home,
});
const content = await readConfigToml(home);
// Existing content survives.
expect(content).toContain('model = "gpt-5.1-codex"');
expect(content).toContain("[profiles.dev]");
// model_provider must precede the first table header (TOML root region),
// and the provider tables must come after the user's tables.
expect(content.indexOf('model_provider = "bifrost"')).toBeLessThan(
content.indexOf("[profiles.dev]"),
);
expect(content.indexOf("[model_providers.bifrost]")).toBeGreaterThan(
content.indexOf("[profiles.dev]"),
);
await prepared.cleanup();
expect(await readConfigToml(home)).toBe(original);
});
it("wins over a pre-existing same-name [model_providers.*] section and root model_provider key", async () => {
const original = [
'model_provider = "stale"',
"",
"[model_providers.bifrost]",
'base_url = "http://old.example/v1"',
'env_key = "OLD_KEY"',
"",
"[model_providers.bifrost.http_headers]",
'"X-Old" = "1"',
"",
"[model_providers.other]",
'base_url = "http://other.example/v1"',
"",
].join("\n");
const home = await makeCodexHome(original);
const prepared = await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
codexHome: home,
});
const content = await readConfigToml(home);
expect(content).not.toContain('model_provider = "stale"');
expect(content).not.toContain("http://old.example/v1");
expect(content).not.toContain("X-Old");
// Unrelated provider sections survive.
expect(content).toContain("[model_providers.other]");
expect(content).toContain("http://other.example/v1");
// Exactly one bifrost section: ours.
expect(content.split("[model_providers.bifrost]").length).toBe(2);
expect(content).toContain('base_url = "http://gateway.example.svc.cluster.local:8080/v1"');
await prepared.cleanup();
expect(await readConfigToml(home)).toBe(original);
});
it("emits plain-object fields as inline tables and arrays as TOML arrays", async () => {
const home = await makeCodexHome();
const prepared = await prepareCodexRuntimeConfig({
env: {
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({
providers: {
gw: {
base_url: "http://gw.example/v1",
query_params: { "api-version": "2026-01-01" },
http_headers: { "X Team": "agents" },
request_max_retries: 4,
},
},
}),
},
codexHome: home,
});
const content = await readConfigToml(home);
expect(content).toContain("[model_providers.gw]");
// Bare keys (incl. hyphens) stay bare; keys with other characters get quoted.
expect(content).toContain('query_params = { api-version = "2026-01-01" }');
expect(content).toContain('http_headers = { "X Team" = "agents" }');
expect(content).toContain("request_max_retries = 4");
// No model_provider was requested, so none is emitted.
expect(content).not.toContain("model_provider =");
await prepared.cleanup();
});
it("expands {env:VAR} placeholders from the run env and process.env, leaving unresolvable ones intact", async () => {
const home = await makeCodexHome();
process.env.PAPERCLIP_CODEX_TEST_PROCESS_KEY = "from-process-env";
try {
const prepared = await prepareCodexRuntimeConfig({
env: {
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({
providers: {
gw: {
base_url: "http://gw.example/v1",
http_headers: {
"X-Run": "{env:PAPERCLIP_CODEX_TEST_RUN_KEY}",
"X-Process": "{env:PAPERCLIP_CODEX_TEST_PROCESS_KEY}",
"X-Missing": "{env:DEFINITELY_UNSET_VAR_XYZ}",
},
},
},
model_provider: "gw",
}),
PAPERCLIP_CODEX_TEST_RUN_KEY: "from-run-env",
},
codexHome: home,
});
const content = await readConfigToml(home);
expect(content).toContain('X-Run = "from-run-env"');
expect(content).toContain('X-Process = "from-process-env"');
expect(content).toContain('X-Missing = "{env:DEFINITELY_UNSET_VAR_XYZ}"');
await prepared.cleanup();
} finally {
delete process.env.PAPERCLIP_CODEX_TEST_PROCESS_KEY;
}
});
it("reads PAPERCLIP_CODEX_PROVIDERS from process.env when absent from the run env", async () => {
const home = await makeCodexHome();
process.env.PAPERCLIP_CODEX_PROVIDERS = JSON.stringify(BIFROST_PROVIDERS);
try {
const prepared = await prepareCodexRuntimeConfig({ env: {}, codexHome: home });
const content = await readConfigToml(home);
expect(content).toContain("[model_providers.bifrost]");
await prepared.cleanup();
} finally {
delete process.env.PAPERCLIP_CODEX_PROVIDERS;
}
});
it("surfaces a note for malformed PAPERCLIP_CODEX_PROVIDERS without touching config.toml", async () => {
const home = await makeCodexHome("model = \"gpt-5.1-codex\"\n");
const cases: Array<[raw: string, noteFragment: string]> = [
["not json", "contains invalid JSON"],
[JSON.stringify(["providers"]), "is not a JSON object"],
[JSON.stringify({ no_providers: true }), 'has no "providers" object'],
[JSON.stringify({ providers: {} }), "contains no usable entries"],
[JSON.stringify({ providers: { gw: "nope" } }), "skipped provider(s) with empty names or non-object values: gw"],
];
for (const [raw, noteFragment] of cases) {
const prepared = await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: raw },
codexHome: home,
});
expect(prepared.notes).toHaveLength(1);
expect(prepared.notes[0]).toContain(noteFragment);
expect(prepared.notes[0]).toContain("custom providers ignored");
expect(await readConfigToml(home)).toBe("model = \"gpt-5.1-codex\"\n");
await prepared.cleanup();
}
});
it("stays silent when PAPERCLIP_CODEX_PROVIDERS is unset or empty", async () => {
const home = await makeCodexHome("model = \"gpt-5.1-codex\"\n");
const envs: Array<Record<string, string>> = [
{},
{ PAPERCLIP_CODEX_PROVIDERS: "" },
{ PAPERCLIP_CODEX_PROVIDERS: " " },
];
for (const env of envs) {
const prepared = await prepareCodexRuntimeConfig({ env, codexHome: home });
expect(prepared.notes).toEqual([]);
expect(await readConfigToml(home)).toBe("model = \"gpt-5.1-codex\"\n");
await prepared.cleanup();
}
});
it("surfaces skipped non-object provider entries while merging the usable ones", async () => {
const home = await makeCodexHome();
const prepared = await prepareCodexRuntimeConfig({
env: {
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({
providers: {
bad: "http://gateway.example/v1",
good: { base_url: "http://gateway.example/v1", env_key: "OPENAI_API_KEY" },
},
model_provider: "good",
}),
},
codexHome: home,
});
const content = await readConfigToml(home);
expect(content).toContain("[model_providers.good]");
expect(content).not.toContain("[model_providers.bad]");
expect(
prepared.notes.some((n) => n.includes("skipped provider(s) with empty names or non-object values: bad")),
).toBe(true);
expect(prepared.notes.some((n) => n.includes("Merged 1 custom Codex model provider(s)"))).toBe(true);
await prepared.cleanup();
});
it("escapes DEL (U+007F) in emitted TOML strings", async () => {
const del = String.fromCharCode(0x7f);
const home = await makeCodexHome();
const prepared = await prepareCodexRuntimeConfig({
env: {
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({
providers: { gw: { base_url: "http://gw.example/v1", name: `del${del}name` } },
}),
},
codexHome: home,
});
const content = await readConfigToml(home);
expect(content).not.toContain(del);
expect(content).toContain("u007fname");
await prepared.cleanup();
});
it("restores user provider sections from the pre-run backup after an interrupted run", async () => {
const userConfig = [
'model = "gpt-5.1-codex"',
"",
"[model_providers.bifrost]",
'base_url = "http://user.example/v1"',
"",
].join("\n");
const home = await makeCodexHome(userConfig);
// Simulate a crash: the merge excises the user's same-name provider
// section (the managed definition must win), and cleanup() never runs.
await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
codexHome: home,
});
expect(await readConfigToml(home)).toContain('env_key = "OPENAI_API_KEY"');
const prepared = await prepareCodexRuntimeConfig({ env: {}, codexHome: home });
expect(prepared.notes.some((n) => n.includes("backup"))).toBe(true);
expect(await readConfigToml(home)).toBe(userConfig);
await expect(fs.access(path.join(home, "config.toml.paperclip-backup"))).rejects.toThrow();
await prepared.cleanup();
});
it("removes the pre-run backup on cleanup", async () => {
const home = await makeCodexHome('approval_policy = "never"\n');
const prepared = await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
codexHome: home,
});
const backupPath = path.join(home, "config.toml.paperclip-backup");
expect(await fs.readFile(backupPath, "utf8")).toBe('approval_policy = "never"\n');
await prepared.cleanup();
expect(await readConfigToml(home)).toBe('approval_policy = "never"\n');
await expect(fs.access(backupPath)).rejects.toThrow();
});
it("skips the merge and surfaces a note when CODEX_HOME is explicitly configured", async () => {
const prepared = await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
codexHome: null,
});
expect(prepared.notes).toHaveLength(1);
expect(prepared.notes[0]).toContain("CODEX_HOME");
await prepared.cleanup();
});
it("self-heals stale managed blocks when the env is no longer set", async () => {
const home = await makeCodexHome();
const crashed = await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
codexHome: home,
});
// Simulate a crash: cleanup never runs, the managed blocks persist.
void crashed;
expect(await readConfigToml(home)).toContain("[model_providers.bifrost]");
const prepared = await prepareCodexRuntimeConfig({ env: {}, codexHome: home });
expect(prepared.notes.some((n) => n.includes("stale"))).toBe(true);
const content = await readConfigToml(home);
expect(content).not.toContain("model_provider");
expect(content).not.toContain("bifrost");
await prepared.cleanup();
});
it("re-running with changed providers replaces the previous managed blocks", async () => {
const home = await makeCodexHome("approval_policy = \"never\"\n");
const first = await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
codexHome: home,
});
void first; // simulate crash: no cleanup
const second = await prepareCodexRuntimeConfig({
env: {
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({
providers: { gw2: { base_url: "http://gw2.example/v1", env_key: "OPENAI_API_KEY" } },
model_provider: "gw2",
}),
},
codexHome: home,
});
const content = await readConfigToml(home);
expect(content).toContain('approval_policy = "never"');
expect(content).toContain("[model_providers.gw2]");
expect(content).toContain('model_provider = "gw2"');
expect(content).not.toContain("bifrost");
expect(content.split("model_provider =").length).toBe(2);
await second.cleanup();
});
it("rejects the block when model_provider names a filtered or missing provider", async () => {
const home = await makeCodexHome("model = \"gpt-5.1-codex\"\n");
const prepared = await prepareCodexRuntimeConfig({
env: {
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({
providers: { good: { base_url: "https://gateway.example/v1" }, bad: "not-an-object" },
model_provider: "bad",
}),
},
codexHome: home,
});
expect(prepared.notes).toContain(
'PAPERCLIP_CODEX_PROVIDERS: model_provider "bad" does not match any usable provider entry; custom providers ignored.',
);
const content = await readConfigToml(home);
expect(content).toBe("model = \"gpt-5.1-codex\"\n");
});
});