Files
PaperClipAI/packages/adapters/codex-local/src/server/runtime-config.test.ts
T
DottaandPaperclip 9179d8dcc3 fix: credit Paperclip for managed OpenRouter requests (#16850)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - AI connections choose credentials and routes independently of the
agent harness.
> - OpenRouter assigns app credit from the final request headers.
> - Harness defaults can assign Paperclip work to the underlying
harness.
> - This pull request applies one public Paperclip identity to managed
OpenRouter requests.
> - The benefit is consistent app credit across supported harnesses and
internal model calls.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

App attribution for existing OpenRouter connections and explicit
provider configurations.

**Current behavior**

Requests can omit attribution or carry the harness identity. Paperclip
does not receive consistent credit in OpenRouter rankings.

**Proposed behavior**

Use `https://paperclip.ing`, the title `Paperclip`, and the coding
categories `cloud-agent,programming-app` for managed requests to
OpenRouter. Keep credentials, routing, and unrelated providers intact.

**Reason and benefit**

The provider URL identifies the app across personal and shared API keys.
A search of public issues and pull requests found no duplicate
attribution work. This improves existing AI connections and does not add
a separate roadmap feature.

## What Changed

- Add a shared attribution policy and a parity-checked copy for the
standalone Runner.
- Apply it to managed Codex, Claude, OpenCode, and Hermes connections,
plus native Pi and explicit legacy provider configs.
- Preserve attribution across Codex config filtering, Claude environment
filtering, OpenCode proxy requests, and model-level overrides.
- Attribute direct fast-response and decision-model requests, including
custom connections to the OpenRouter endpoint.
- Add positive and negative regression tests and provider attribution
requirements to the connector playbook.
- Advance native Pi to profile 24, preserving historical identities and
binding the attribution source in its declaration and release checks.

## Verification

- Final head `6df282d2a58c2698e41127443903d6fc85b4b72f`: all CI/status
checks passed or were intentionally skipped; Greptile is **5/5**, with
zero unresolved review threads. [Final CI
run](https://github.com/paperclipai/paperclip/actions/runs/39274938069).
- Focused server/shared/config suites: 132 tests passed before the final
edge-case additions. Expanded adapter suites pass 48 tests; the final
OpenCode merge passes 15 tests; global/EU/US endpoint and policy-parity
suites pass 26 tests.
- Focused Runner suites: 85 tests passed, including model overrides and
recovery/config boundaries. Profile installation/environment/config
tests subsequently passed 72 tests (one optional distribution test
skipped); historical decoding passed 11 tests and updated recovery
coverage passed 27 tests. The Rust profile admission test and all five
native Pi launch/recovery tests passed.
- All 114 managed AI connection tests pass together. They include
custom-header preservation for legacy and native Claude runs, both
OpenRouter and other gateways, inherited headers, and unresolved secret
references.
- Actual requests from installed Codex 0.160.0, Claude Code 2.1.245,
OpenCode 1.18.34, Pi 0.74.0 and 1.0.0, and the managed Hermes runtime
carried all four Paperclip headers. A local endpoint used dummy
credentials and returned an intentional 401 after capture. Codex and
Claude retry requests retained attribution. This proves outgoing header
precedence; it is not a successful model-inference test. The first
Claude capture had an argument-separation error and sent no request;
correcting the command produced the captured requests.
- A live request through `fastResponseModel` completed with `attribution
verified`. OpenRouter recorded generation
`gen-1792420273-Ez1GUTCc7YuZ1FJMCGob` under app ID `3118063`, at a cost
of $0.00000375. The public app record maps that ID to
`https://paperclip.ing/`.
- Full `pnpm -r typecheck` and `pnpm build` passed, including a repeat
after the review fixes. The final server build stamp records commit
`6df282d2a58c2698e41127443903d6fc85b4b72f`. `pnpm check:token-gates`
passed.
- CI first caught the stale Pi source attestation; it is now a new
immutable profile. Greptile caught custom-header loss, now fixed and
covered by the complete managed-connection suite. The next CI run caught
the missing Docker allowlist entry for profile 24; the allowlist and
release smoke pins are now synchronized. Local Docker verification
cannot run because its daemon socket is absent; The exact Docker context
check passed in CI on the final head.
- The initial local `pnpm test:run` did **not** pass. It ran across
source/build edits, reported chat/tool-access collection failures, three
workspace-runtime failures, two new managed-header failures, one Dot
Runner readiness failure, and two native-session-resume failures, and
was interrupted after the final-head CI server coverage passed. Its
secrets suite also reported 91 cases skipped. The isolated tool-access
suite passed all 428 tests, and the complete managed-connection suite
passed all 114 tests. These retries establish recovery, not the cause of
the original local failures; no claim of a successful full local run is
made. Final acceptance uses immutable final-head CI plus the focused
local and live checks above.
- Final-head CI encountered one unchanged UI assertion in
`FileViewerSheet.copy.test.tsx` (missing “Copied link” feedback after
clipboard completion). That shard passed on the earlier head with
identical UI code, and all six clipboard tests passed locally. A timing
race around the asynchronous React update is suspected but not
conclusively established. The single CI retry passed, with no
product/test code changed for it. The earlier failure remains part of
this verification record.

## Risks

- Native Pi profile 24 requires a fresh provider session when
encountering a profile 23 identity; recovery tests cover that boundary.
- Harness upgrades can change header precedence. The playbook requires
actual request capture when changing provider paths.
- Externally managed agents, unmanaged harness settings, and gateways
that replace headers need separate qualification. This change does not
intercept arbitrary model calls made by task scripts.
- The existing OpenRouter entry for the Paperclip URL still displays an
old title, “PrettyFly Opus Testing,” and no categories on the final
public-page recheck. The live request received credit under that URL's
app ID. Public title propagation and leaderboard placement remain
controlled by OpenRouter.
- Global, EU, and US OpenRouter hosts are recognized explicitly; other
subdomains are not treated as OpenRouter. No database migration,
credential ownership change, or Paperclip telemetry event is involved.
Attribution contains public product constants only.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository editing, shell
execution, web research, and live API verification. The exact deployment
identifier and context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass — the focused suites listed
above; the unsuccessful full local attempt is explicitly recorded, with
full final-head coverage supplied by CI
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-19 10:28:53 -05:00

440 lines
17 KiB
TypeScript

import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { prepareCodexRuntimeConfig } from "./runtime-config.js";
const cleanupPaths = new Set<string>();
afterEach(async () => {
await Promise.all(
[...cleanupPaths].map(async (filepath) => {
await fs.rm(filepath, { recursive: true, force: true });
cleanupPaths.delete(filepath);
}),
);
});
async function makeCodexHome(configToml?: string): Promise<string> {
const home = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-home-"));
cleanupPaths.add(home);
if (configToml !== undefined) {
await fs.writeFile(path.join(home, "config.toml"), configToml, "utf8");
}
return home;
}
async function readConfigToml(home: string): Promise<string> {
return fs.readFile(path.join(home, "config.toml"), "utf8");
}
const BIFROST_PROVIDERS = {
providers: {
bifrost: {
name: "bifrost",
base_url: "http://gateway.example.svc.cluster.local:8080/v1",
env_key: "OPENAI_API_KEY",
wire_api: "responses",
},
},
model_provider: "bifrost",
};
describe("prepareCodexRuntimeConfig", () => {
it("attributes OpenRouter aliases without replacing credentials or unrelated providers", async () => {
const home = await makeCodexHome();
const prepared = await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({ providers: {
router: {
base_url: "https://openrouter.ai/api/v1", env_key: "ROUTER_KEY",
http_headers: { "http-referer": "https://harness.example", "X-Route": "selected" },
env_http_headers: { "x-title": "HARNESS_TITLE", "X-Tenant": "TENANT" },
},
other: { base_url: "https://other.example/v1", http_headers: { "X-Title": "Other" } },
} }) }, codexHome: home,
});
const content = await readConfigToml(home);
expect(content).toContain('HTTP-Referer = "https://paperclip.ing"');
expect(content).toContain('env_key = "ROUTER_KEY"');
expect(content).toContain('X-Route = "selected"');
expect(content).toContain('X-Tenant = "TENANT"');
expect(content).toContain('X-Title = "Other"');
expect(content).not.toContain("HARNESS_TITLE");
expect(content).not.toContain("harness.example");
await prepared.cleanup();
});
it("is a no-op when PAPERCLIP_CODEX_PROVIDERS is unset", async () => {
const home = await makeCodexHome("model = \"gpt-5.1-codex\"\n");
const prepared = await prepareCodexRuntimeConfig({ env: { FOO: "bar" }, codexHome: home });
expect(prepared.notes).toEqual([]);
expect(await readConfigToml(home)).toBe("model = \"gpt-5.1-codex\"\n");
await prepared.cleanup();
});
it("is a no-op when the home has no config.toml and the env is unset", async () => {
const home = await makeCodexHome();
const prepared = await prepareCodexRuntimeConfig({ env: {}, codexHome: home });
expect(prepared.notes).toEqual([]);
await expect(fs.access(path.join(home, "config.toml"))).rejects.toThrow();
await prepared.cleanup();
});
it("merges providers + model_provider into a fresh config.toml and cleans it up", async () => {
const home = await makeCodexHome();
const prepared = await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
codexHome: home,
});
const content = await readConfigToml(home);
expect(content).toContain('model_provider = "bifrost"');
expect(content).toContain("[model_providers.bifrost]");
expect(content).toContain('base_url = "http://gateway.example.svc.cluster.local:8080/v1"');
expect(content).toContain('env_key = "OPENAI_API_KEY"');
expect(content).toContain('wire_api = "responses"');
expect(prepared.notes.some((n) => n.includes("bifrost"))).toBe(true);
await prepared.cleanup();
await expect(fs.access(path.join(home, "config.toml"))).rejects.toThrow();
});
it("preserves existing config.toml content, keeps model_provider in the root region, and restores on cleanup", async () => {
const original = [
'model = "gpt-5.1-codex"',
"",
"[profiles.dev]",
'model = "gpt-5.1-codex-mini"',
"",
].join("\n");
const home = await makeCodexHome(original);
const prepared = await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
codexHome: home,
});
const content = await readConfigToml(home);
// Existing content survives.
expect(content).toContain('model = "gpt-5.1-codex"');
expect(content).toContain("[profiles.dev]");
// model_provider must precede the first table header (TOML root region),
// and the provider tables must come after the user's tables.
expect(content.indexOf('model_provider = "bifrost"')).toBeLessThan(
content.indexOf("[profiles.dev]"),
);
expect(content.indexOf("[model_providers.bifrost]")).toBeGreaterThan(
content.indexOf("[profiles.dev]"),
);
await prepared.cleanup();
expect(await readConfigToml(home)).toBe(original);
});
it("wins over a pre-existing same-name [model_providers.*] section and root model_provider key", async () => {
const original = [
'model_provider = "stale"',
"",
"[model_providers.bifrost]",
'base_url = "http://old.example/v1"',
'env_key = "OLD_KEY"',
"",
"[model_providers.bifrost.http_headers]",
'"X-Old" = "1"',
"",
"[model_providers.other]",
'base_url = "http://other.example/v1"',
"",
].join("\n");
const home = await makeCodexHome(original);
const prepared = await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
codexHome: home,
});
const content = await readConfigToml(home);
expect(content).not.toContain('model_provider = "stale"');
expect(content).not.toContain("http://old.example/v1");
expect(content).not.toContain("X-Old");
// Unrelated provider sections survive.
expect(content).toContain("[model_providers.other]");
expect(content).toContain("http://other.example/v1");
// Exactly one bifrost section: ours.
expect(content.split("[model_providers.bifrost]").length).toBe(2);
expect(content).toContain('base_url = "http://gateway.example.svc.cluster.local:8080/v1"');
await prepared.cleanup();
expect(await readConfigToml(home)).toBe(original);
});
it("emits plain-object fields as inline tables and arrays as TOML arrays", async () => {
const home = await makeCodexHome();
const prepared = await prepareCodexRuntimeConfig({
env: {
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({
providers: {
gw: {
base_url: "http://gw.example/v1",
query_params: { "api-version": "2026-01-01" },
http_headers: { "X Team": "agents" },
request_max_retries: 4,
},
},
}),
},
codexHome: home,
});
const content = await readConfigToml(home);
expect(content).toContain("[model_providers.gw]");
// Bare keys (incl. hyphens) stay bare; keys with other characters get quoted.
expect(content).toContain('query_params = { api-version = "2026-01-01" }');
expect(content).toContain('http_headers = { "X Team" = "agents" }');
expect(content).toContain("request_max_retries = 4");
// No model_provider was requested, so none is emitted.
expect(content).not.toContain("model_provider =");
await prepared.cleanup();
});
it("expands {env:VAR} placeholders from the run env and process.env, leaving unresolvable ones intact", async () => {
const home = await makeCodexHome();
process.env.PAPERCLIP_CODEX_TEST_PROCESS_KEY = "from-process-env";
try {
const prepared = await prepareCodexRuntimeConfig({
env: {
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({
providers: {
gw: {
base_url: "http://gw.example/v1",
http_headers: {
"X-Run": "{env:PAPERCLIP_CODEX_TEST_RUN_KEY}",
"X-Process": "{env:PAPERCLIP_CODEX_TEST_PROCESS_KEY}",
"X-Missing": "{env:DEFINITELY_UNSET_VAR_XYZ}",
},
},
},
model_provider: "gw",
}),
PAPERCLIP_CODEX_TEST_RUN_KEY: "from-run-env",
},
codexHome: home,
});
const content = await readConfigToml(home);
expect(content).toContain('X-Run = "from-run-env"');
expect(content).toContain('X-Process = "from-process-env"');
expect(content).toContain('X-Missing = "{env:DEFINITELY_UNSET_VAR_XYZ}"');
await prepared.cleanup();
} finally {
delete process.env.PAPERCLIP_CODEX_TEST_PROCESS_KEY;
}
});
it("reads PAPERCLIP_CODEX_PROVIDERS from process.env when absent from the run env", async () => {
const home = await makeCodexHome();
process.env.PAPERCLIP_CODEX_PROVIDERS = JSON.stringify(BIFROST_PROVIDERS);
try {
const prepared = await prepareCodexRuntimeConfig({ env: {}, codexHome: home });
const content = await readConfigToml(home);
expect(content).toContain("[model_providers.bifrost]");
await prepared.cleanup();
} finally {
delete process.env.PAPERCLIP_CODEX_PROVIDERS;
}
});
it("surfaces a note for malformed PAPERCLIP_CODEX_PROVIDERS without touching config.toml", async () => {
const home = await makeCodexHome("model = \"gpt-5.1-codex\"\n");
const cases: Array<[raw: string, noteFragment: string]> = [
["not json", "contains invalid JSON"],
[JSON.stringify(["providers"]), "is not a JSON object"],
[JSON.stringify({ no_providers: true }), 'has no "providers" object'],
[JSON.stringify({ providers: {} }), "contains no usable entries"],
[JSON.stringify({ providers: { gw: "nope" } }), "skipped provider(s) with empty names or non-object values: gw"],
];
for (const [raw, noteFragment] of cases) {
const prepared = await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: raw },
codexHome: home,
});
expect(prepared.notes).toHaveLength(1);
expect(prepared.notes[0]).toContain(noteFragment);
expect(prepared.notes[0]).toContain("custom providers ignored");
expect(await readConfigToml(home)).toBe("model = \"gpt-5.1-codex\"\n");
await prepared.cleanup();
}
});
it("stays silent when PAPERCLIP_CODEX_PROVIDERS is unset or empty", async () => {
const home = await makeCodexHome("model = \"gpt-5.1-codex\"\n");
const envs: Array<Record<string, string>> = [
{},
{ PAPERCLIP_CODEX_PROVIDERS: "" },
{ PAPERCLIP_CODEX_PROVIDERS: " " },
];
for (const env of envs) {
const prepared = await prepareCodexRuntimeConfig({ env, codexHome: home });
expect(prepared.notes).toEqual([]);
expect(await readConfigToml(home)).toBe("model = \"gpt-5.1-codex\"\n");
await prepared.cleanup();
}
});
it("surfaces skipped non-object provider entries while merging the usable ones", async () => {
const home = await makeCodexHome();
const prepared = await prepareCodexRuntimeConfig({
env: {
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({
providers: {
bad: "http://gateway.example/v1",
good: { base_url: "http://gateway.example/v1", env_key: "OPENAI_API_KEY" },
},
model_provider: "good",
}),
},
codexHome: home,
});
const content = await readConfigToml(home);
expect(content).toContain("[model_providers.good]");
expect(content).not.toContain("[model_providers.bad]");
expect(
prepared.notes.some((n) => n.includes("skipped provider(s) with empty names or non-object values: bad")),
).toBe(true);
expect(prepared.notes.some((n) => n.includes("Merged 1 custom Codex model provider(s)"))).toBe(true);
await prepared.cleanup();
});
it("escapes DEL (U+007F) in emitted TOML strings", async () => {
const del = String.fromCharCode(0x7f);
const home = await makeCodexHome();
const prepared = await prepareCodexRuntimeConfig({
env: {
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({
providers: { gw: { base_url: "http://gw.example/v1", name: `del${del}name` } },
}),
},
codexHome: home,
});
const content = await readConfigToml(home);
expect(content).not.toContain(del);
expect(content).toContain("u007fname");
await prepared.cleanup();
});
it("restores user provider sections from the pre-run backup after an interrupted run", async () => {
const userConfig = [
'model = "gpt-5.1-codex"',
"",
"[model_providers.bifrost]",
'base_url = "http://user.example/v1"',
"",
].join("\n");
const home = await makeCodexHome(userConfig);
// Simulate a crash: the merge excises the user's same-name provider
// section (the managed definition must win), and cleanup() never runs.
await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
codexHome: home,
});
expect(await readConfigToml(home)).toContain('env_key = "OPENAI_API_KEY"');
const prepared = await prepareCodexRuntimeConfig({ env: {}, codexHome: home });
expect(prepared.notes.some((n) => n.includes("backup"))).toBe(true);
expect(await readConfigToml(home)).toBe(userConfig);
await expect(fs.access(path.join(home, "config.toml.paperclip-backup"))).rejects.toThrow();
await prepared.cleanup();
});
it("removes the pre-run backup on cleanup", async () => {
const home = await makeCodexHome('approval_policy = "never"\n');
const prepared = await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
codexHome: home,
});
const backupPath = path.join(home, "config.toml.paperclip-backup");
expect(await fs.readFile(backupPath, "utf8")).toBe('approval_policy = "never"\n');
await prepared.cleanup();
expect(await readConfigToml(home)).toBe('approval_policy = "never"\n');
await expect(fs.access(backupPath)).rejects.toThrow();
});
it("skips the merge and surfaces a note when CODEX_HOME is explicitly configured", async () => {
const prepared = await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
codexHome: null,
});
expect(prepared.notes).toHaveLength(1);
expect(prepared.notes[0]).toContain("CODEX_HOME");
await prepared.cleanup();
});
it("self-heals stale managed blocks when the env is no longer set", async () => {
const home = await makeCodexHome();
const crashed = await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
codexHome: home,
});
// Simulate a crash: cleanup never runs, the managed blocks persist.
void crashed;
expect(await readConfigToml(home)).toContain("[model_providers.bifrost]");
const prepared = await prepareCodexRuntimeConfig({ env: {}, codexHome: home });
expect(prepared.notes.some((n) => n.includes("stale"))).toBe(true);
const content = await readConfigToml(home);
expect(content).not.toContain("model_provider");
expect(content).not.toContain("bifrost");
await prepared.cleanup();
});
it("re-running with changed providers replaces the previous managed blocks", async () => {
const home = await makeCodexHome("approval_policy = \"never\"\n");
const first = await prepareCodexRuntimeConfig({
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
codexHome: home,
});
void first; // simulate crash: no cleanup
const second = await prepareCodexRuntimeConfig({
env: {
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({
providers: { gw2: { base_url: "http://gw2.example/v1", env_key: "OPENAI_API_KEY" } },
model_provider: "gw2",
}),
},
codexHome: home,
});
const content = await readConfigToml(home);
expect(content).toContain('approval_policy = "never"');
expect(content).toContain("[model_providers.gw2]");
expect(content).toContain('model_provider = "gw2"');
expect(content).not.toContain("bifrost");
expect(content.split("model_provider =").length).toBe(2);
await second.cleanup();
});
it("rejects the block when model_provider names a filtered or missing provider", async () => {
const home = await makeCodexHome("model = \"gpt-5.1-codex\"\n");
const prepared = await prepareCodexRuntimeConfig({
env: {
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({
providers: { good: { base_url: "https://gateway.example/v1" }, bad: "not-an-object" },
model_provider: "bad",
}),
},
codexHome: home,
});
expect(prepared.notes).toContain(
'PAPERCLIP_CODEX_PROVIDERS: model_provider "bad" does not match any usable provider entry; custom providers ignored.',
);
const content = await readConfigToml(home);
expect(content).toBe("model = \"gpt-5.1-codex\"\n");
});
});