From 6f9d0a56ba3c2f0059a723b7de434320eed7eb22 Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Fri, 9 Oct 2026 08:19:51 -0500 Subject: [PATCH] fix: resolve installed Codex and preserve npm host dependencies (#15555) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Installed agents need their own runtime dependencies. > - Native Codex startup and browser login could depend on a global CLI. > - npm bundles do not inherit workspace dependency overrides or patches. > - Codex native packages must come from npm for the consumer host. > - This PR fixes executable resolution and the required npm dependency layout. > - Usable installed CLI versions can run without a numeric version gate. ## Linked Issues or Issue Description Refs: #15422. This is a prerequisite for the later Codex-default change. **What happened?** Packaged native Codex startup and browser login could require a global Codex CLI. The server's vendored runner lacked its own declared Codex bridge. A bundled wrapper could retain producer-host binaries or select the legacy adapter's separate platform version. **Expected behavior** Prefer the installed Codex executable. Accept usable older or newer versions. Use the selected host's PATH when the dependency is absent. Keep the patched JavaScript graph. Let npm install official native packages for the consumer platform. Grant sandbox reads only to the selected vendor resources. **Steps to reproduce** 1. Prepare a clean npm installation without a global Codex command. 2. Start native Codex or its browser login. 3. Inspect the selected executable, installed host package, and sandbox resource paths. **Paperclip version or commit** Frozen master base: `1881894973a2b25838d8abed9bd8aeebc3af4441`. **Deployment mode** Source and packaged self-hosted installation. ## What Changed - Share installed Codex command resolution across native startup, direct evals, and browser login. Accept usable version differences. Preserve explicit commands, recorded sessions, remote host boundaries, and the Linux ARM64 legacy login path. Return safe errors for missing executables and failed login terminals. - Declare the server's Codex bridge. Retain the patched JavaScript dependency graph in npm packages. Strip Codex native payloads. Declare official optional host packages on the published server manifest so native and legacy versions remain separate. Preserve consumer esbuild platform dependencies. - Adjust resource lookup for npm's separate platform packages. Retain package identity, path containment, resolver ownership, and narrow sandbox reads. Keep exact version and digest checks for explicit ACPX artifact qualification. - Extend existing packaging, installed-consumer, login, selection, recovery, and integrity tests. Document the retained behavior. The diff is now 23 files, 1,709 additions, and 53 deletions. The prior diff had 35 files and about 3,700 changed lines. Removed work is preserved on `codex/runner-packaging-full-snapshot` at `6f3060beaa842ffcee21af058370d3cab5f571e9`. Removed from this PR: release workflows and assembly, provider-pack changes, Docker materialization, Git installer changes, extra login HOME/working-directory isolation, and unrelated CI fixture repairs. This PR does not change agent defaults, stored runner choices, UI, schema, or provider qualification. ## Verification - Final candidate: `dde37d7ed0121c10b60b8801eda80f8dc17909ad`. [All 47 ordinary CI jobs passed](https://github.com/paperclipai/paperclip/actions/runs/37842288835) on attempt 1, including typecheck, tests, build, E2E, runner checks, and the installed-consumer canary. [Fresh Greptile review](https://github.com/paperclipai/paperclip/pull/15555#issuecomment-6059581546) is 5/5 on this head; no unresolved threads remain. Human approval is still outstanding. - Reused focused controls passed with Node 24: 71 initial narrowed checks, then 56 affected Codex/selection/eval checks after the relative PATH correction. Runner TypeScript no-emit, syntax, and diff checks passed. The existing fixture reproduces the original relative PATH failure and verifies working-directory selection, empty entries, ordering, and absolute launch. One CLI entrypoint test was initially blocked by sandbox IPC and passed with its existing local socket allowed. Login HOME, config-directory assignments, and working directory match master. - [The actual clean Linux npm consumer](https://github.com/paperclipai/paperclip/actions/runs/37842288835/job/113535523044) passed on the final candidate's CI integration. All 17 Paperclip tarballs omit native Codex payloads. Official npm host packages retain their own integrity and `inBundle=false`. Native Codex selects 0.160.0; the legacy closure retains 0.156.1. Package admission, command leases, narrow native sandbox resources, consumer hooks, preserved lock, and offline lifecycle controls passed. Provider calls were zero. No new workflow is added. - Actual official Codex 0.156.1 passed on the final committed source on macOS ARM64: installed dependency preference, absolute and relative PATH fallback, narrow resource lookup, and app-server initialize/initialized. Executed module hashes match the candidate. One scripts-disabled install, three version probes, and one handshake completed in 26 seconds; owned files and process group were removed. No login, account/model request, or provider task ran. - CI checked out `dfda8e708e87306d22ed735d78bdfcbe770e99b7` on base `65b558180533039a891ee0cd1ccab9988aa79adc`. Its 13 upstream paths do not overlap this PR's 23 paths or alter packaging/Codex inputs. The consumer report records producer `7b8e94c08657b5ddc265946459762340f125726c`, after the existing canary staged a generated-lock-only commit (one file, three insertions). These identities are kept separate; raw generated-lock bytes were not retained. - No local Docker or Rust build, paid provider turn, merge, or deployment. Later PRs must prove live onboarding and production cloud packaging before changing defaults. ## Risks - npm must install optional host dependencies. Missing dependencies still return errors. Paperclip tarballs do not pre-bundle Codex executables. - The repository requires CI-owned lockfile updates. PR CI resolves the changed manifest and stages its own producer lockfile. A raw source Docker build with `--frozen-lockfile` must wait for the existing master lockfile bot to merge its refresh, or use a disposable resolved checkout. No Docker build or deployment is qualified by this PR. - Ordinary Codex startup accepts version differences. Actual protocol or login failures remain errors. Explicit ACPX artifact checks retain their release pins. - The published server delegates platform installation outside its bundled JavaScript graph. Focused negative controls reject unsafe package metadata and paths. The hosted consumer test passed on this candidate’s CI integration. - Existing agents retain their stored runner choices. There is no data migration or automatic upgrade. Release pipeline and platform qualification work remain separate prerequisites for later defaults. ## Model Used OpenAI Codex, GPT-6, with reasoning, tool use, code execution, and parallel agents. The exact serving snapshot and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- doc/plans/2026-10-06-default-agent-runner.md | 96 +++++++ packages/paperclip-runner/README.md | 27 ++ .../src/cli/eval-provider-runtime.test.ts | 2 +- .../src/cli/eval-provider-runtime.ts | 17 +- .../paperclip-runner/src/cli/eval-session.ts | 2 +- .../acpx/installation-integrity.test.ts | 98 ++++++++ .../drivers/acpx/installation-integrity.ts | 95 ++++++- .../src/drivers/codex/codex-command.ts | 78 ++++++ .../codex/codex-security-config.test.ts | 232 +++++++++++++++++ .../drivers/codex/codex-security-config.ts | 86 ++++++- packages/paperclip-runner/src/index.ts | 1 + .../src/live/runnerd-codex-transport.test.ts | 103 ++++++++ .../src/live/runnerd-codex-transport.ts | 14 +- .../grok-public-install-sandbox.test.mjs | 156 +++++++++++- scripts/acpx-patch-packaging.test.mjs | 238 +++++++++++++++++- scripts/grok-public-install-sandbox.mjs | 144 +++++++++++ scripts/prepare-bundled-package.mjs | 232 ++++++++++++++++- scripts/verify-grok-npm-install.mjs | 16 +- server/package.json | 2 + .../services/local-ai-browser-login.test.ts | 87 ++++++- server/src/services/local-ai-browser-login.ts | 32 ++- server/src/services/local-ai-login.ts | 2 +- server/src/vendor/paperclip-runner/index.ts | 2 + 23 files changed, 1709 insertions(+), 53 deletions(-) create mode 100644 doc/plans/2026-10-06-default-agent-runner.md create mode 100644 packages/paperclip-runner/src/drivers/codex/codex-command.ts diff --git a/doc/plans/2026-10-06-default-agent-runner.md b/doc/plans/2026-10-06-default-agent-runner.md new file mode 100644 index 00000000000..d28e7960eaf --- /dev/null +++ b/doc/plans/2026-10-06-default-agent-runner.md @@ -0,0 +1,96 @@ +# Runner defaults: staged delivery + +Updated 2026-10-08. + +## Outcome and ownership + +Users choose a harness. New agents use Paperclip Runner where the harness and +execution target are qualified. Existing agents keep their recorded runner. +The task owns implementation, verification, review fixes, and reviewable PRs. +Merging and deployment require separate authorization. + +The original implementation is preserved on `codex/default-agent-runner` at +`171d841295808ca185a258aaeb10b3dee68bb2b9`. Its public review is +[PR #15422](https://github.com/paperclipai/paperclip/pull/15422). +The user approved splitting that work into four useful steps: + +1. Packaged runtime reliability and Codex prerequisites, without changing defaults. +2. Codex defaults across creation, onboarding, hiring, imports, and the UI. +3. Claude defaults and authentication. +4. The remaining supported harness defaults and provider-specific fixes. + +## Current slice + +Canonical branch: `codex/runner-packaging-prerequisites`. +Frozen master base: `1881894973a2b25838d8abed9bd8aeebc3af4441`. +Canonical PR: [#15555](https://github.com/paperclipai/paperclip/pull/15555). +Its [checks](https://github.com/paperclipai/paperclip/pull/15555/checks) and +[commits](https://github.com/paperclipai/paperclip/pull/15555/commits) record the +final candidate. Hosted results and review disposition belong to that exact +candidate; earlier green checks do not qualify a later revision. + +The user requested a smaller PR on 2026-10-08. This slice now covers Codex +executable resolution, native-free npm packaging, the installed JavaScript +dependency graph, and the sandbox resource lookup required by that npm layout. +It retains focused tests and extends the existing public npm consumer verifier. +It does not add a workflow, change Docker or Git installation, change shared +login HOME or working-directory behavior, or transfer runner release assets. +Agent defaults, saved runner choices, UI, schema, and provider qualification stay +unchanged. + +The broader prerequisite implementation is preserved on the pushed branch +`codex/runner-packaging-full-snapshot` at +`6f3060beaa842ffcee21af058370d3cab5f571e9`. Its Git staging, extra login isolation, +release pipeline, Docker materialization, and unrelated test-fixture changes +remain outside this PR. Release and platform packaging must be qualified in a +separate slice before enabling new-agent defaults. + +### Retained behavior + +Ordinary native Codex startup, direct evals, and browser login prefer the +installed dependency. They accept usable older or newer CLI versions. If the +dependency is absent, use the selected host's PATH. Explicit commands and recorded +sessions retain precedence. Real protocol/login failures remain actionable; +there is no silent runner fallback. Release pins and the separate ACPX artifact +qualification stay unchanged. Linux ARM64 keeps its existing legacy login path. + +Final review found a relative PATH regression in the retained resolver. Resolve +relative and empty PATH entries against the selected provider working directory; +retain installed-dependency preference. Extend the existing fresh-process fixture +and qualify the resulting commit separately from the green `cb5948e` candidate. + +Paperclip npm tarballs retain the patched JavaScript graph and strip Codex native +payloads. The published server declares official optional host packages. npm +installs them for the consumer's OS and architecture. This avoids collisions with +the legacy adapter's separate Codex version. The server owns the bridge dependency +used by its vendored runner. Sandbox reads remain confined to the selected native +vendor resources; package identities and path containment still apply. + +Browser login keeps master's provider-specific credential directories, ambient +HOME, and working directory. Only executable selection and safe errors change. + +## Evidence and gates + +| Gate | Status | Evidence | +| --- | --- | --- | +| Preserve original and broader work | Passed | Pushed branches and revisions above | +| Narrow scope against current master | Passed | Installed Codex/npm scope only; no workflow, Docker, Git installer, or unrelated fixture diff | +| Focused packaging and installed-consumer controls | Passed | 24 packaging tests with actual offline npm tarballs; 9 existing consumer assertion tests | +| Login controls after narrowing | Passed | 13 existing tests; HOME/config-directory/spawn semantics match frozen master | +| Codex selection, protocol, sandbox, and ACPX integrity | Passed locally | 21 selection/security/transport/eval tests; 4 npm layout integrity controls; runner TypeScript no-emit | +| Full checks and fresh review | Pending for relative PATH fix | `cb5948e` passed all 47 CI jobs and its clean Linux npm consumer; qualify the final fix with new CI and fresh 5/5 review | +| Previous broad candidate | Historical | 47 ordinary CI jobs passed at `6f3060b`; not proof for the narrowed candidate | +| Existing old CLI protocol proof | Historical | Official Codex 0.156.1 completed startup handshake; scope must be matched to retained sources | +| All-harness live onboarding and cloud qualification | Deferred | Later defaults slices; authentication probes and CI cannot close live journeys | + +Use Node 24 and one worker for focused local checks. Keep heavy builds in hosted +CI. Do not build Docker images or Rust locally. Do not author a lockfile change. +No provider-backed runs or new disposable environments are needed for this +prerequisite slice. Incremental provider spend for narrowing is $0. The original +$250 ceiling and cleanup obligations remain for later live qualification. +Existing credentials and user previews remain untouched. + +Next action: commit and push the narrowed candidate, rerun its checks and fresh +review, then update the PR and existing evidence report. Start the Codex-default +slice after the user chooses to proceed. Merging and deployment remain outside +this task. diff --git a/packages/paperclip-runner/README.md b/packages/paperclip-runner/README.md index 735cfddeb1c..e7d6f284d9b 100644 --- a/packages/paperclip-runner/README.md +++ b/packages/paperclip-runner/README.md @@ -58,6 +58,33 @@ checks installed dependency pins and agreement with Cursor's distribution manife and immutable release attestation (`cursor-contract.json`). Cursor's per-platform closure pins are generated from `cursor-distributions.json`. +Codex startup and browser login prefer the CLI from the installed dependency +graph. If that dependency is absent, they use an executable `codex` from the +selected execution host's `PATH`. An explicit execution command takes precedence, +and resumed sessions retain their recorded command. Older or newer CLI versions +are allowed. Relative PATH entries use the selected process working directory. +Compatibility is established by the actual protocol or login attempt. +Missing executables and real protocol failures still return actionable errors. +Package identity and executable containment remain checked. Browser login keeps +its existing provider-specific credential home. Linux ARM64 retains its existing +legacy login path because native execution is not qualified there. + +Exact dependency pins remain release and ACPX artifact-qualification checks. +They make published builds reproducible and verify sandbox artifacts; they do not +add a version-number gate to ordinary native Codex startup or browser login. + +Release packages retain the pinned Codex JavaScript dependency graph, but do not +bundle Codex native binaries. The published manifest declares the official, +exact-version platform packages as optional dependencies. npm installs the +package for the consumer's operating system and architecture. Those declarations +belong to the published server manifest; the bundled JavaScript wrapper delegates +platform installation there so npm can keep native and legacy versions separate. +The wrapper code and patched ACP bridge stay unchanged. +Codex sandbox read roots include the separately installed native vendor resources +without granting access to enclosing npm directories or credential homes. +This packaging does not change agent defaults or +the selected runner of an existing agent. + Every ACPX harness accepts an explicit caller-selected model without a Paperclip model allowlist. The adapter sends that ID unchanged and verifies the provider's effective model before prompting. An incomplete discovery catalog does not block diff --git a/packages/paperclip-runner/src/cli/eval-provider-runtime.test.ts b/packages/paperclip-runner/src/cli/eval-provider-runtime.test.ts index 5116264a92c..f6cf321bc6e 100644 --- a/packages/paperclip-runner/src/cli/eval-provider-runtime.test.ts +++ b/packages/paperclip-runner/src/cli/eval-provider-runtime.test.ts @@ -5,7 +5,7 @@ import { describe, expect, it } from "vitest"; import { evalProviderTransportOptions } from "./eval-provider-runtime.js"; describe("direct eval provider runtime", () => { - it("uses the installed pinned Codex dependency without requiring a global CLI", () => { + it("uses the installed Codex dependency without requiring a global CLI", () => { const options = evalProviderTransportOptions("codex"); expect(isAbsolute(options.codexCommand!)).toBe(true); expect(existsSync(options.codexCommand!)).toBe(true); diff --git a/packages/paperclip-runner/src/cli/eval-provider-runtime.ts b/packages/paperclip-runner/src/cli/eval-provider-runtime.ts index ff2c2f96d13..424c6d779a2 100644 --- a/packages/paperclip-runner/src/cli/eval-provider-runtime.ts +++ b/packages/paperclip-runner/src/cli/eval-provider-runtime.ts @@ -1,6 +1,4 @@ -import { readFileSync } from "node:fs"; -import { createRequire } from "node:module"; -import { dirname, resolve } from "node:path"; +import { resolveCodexCommand } from "../drivers/codex/codex-command.js"; import type { CapabilityRunnerdCodexTransportOptions } from "../live/runnerd-codex-transport.js"; @@ -8,6 +6,7 @@ import type { CapabilityRunnerdCodexTransportOptions } from "../live/runnerd-cod export function evalProviderTransportOptions( provider: "codex" | "opencode" | "acpx" | "claude_managed" | "aws_agentcore", turnTimeoutMs = 120_000, + workingDirectory = process.cwd(), ): Pick { @@ -22,15 +21,5 @@ export function evalProviderTransportOptions( return { acpxPermissionMode: "approve-all", acpxPermissionModePinned: true }; } if (provider !== "codex") return {}; - const runnerRequire = createRequire(import.meta.url); - const codexRequire = createRequire( - runnerRequire.resolve("@agentclientprotocol/codex-acp/package.json"), - ); - const manifestPath = codexRequire.resolve("@openai/codex/package.json"); - const manifest = JSON.parse(readFileSync(manifestPath, "utf8")) as { - bin?: string | Record; - }; - const executable = typeof manifest.bin === "string" ? manifest.bin : manifest.bin?.codex; - if (!executable) throw new Error("Pinned Codex dependency does not expose its executable"); - return { codexCommand: resolve(dirname(manifestPath), executable) }; + return { codexCommand: resolveCodexCommand(undefined, undefined, workingDirectory) }; } diff --git a/packages/paperclip-runner/src/cli/eval-session.ts b/packages/paperclip-runner/src/cli/eval-session.ts index 237f018dc51..a07a19b229c 100644 --- a/packages/paperclip-runner/src/cli/eval-session.ts +++ b/packages/paperclip-runner/src/cli/eval-session.ts @@ -379,7 +379,7 @@ export async function runEvalSessionCli( const service = options.serviceFactory?.(runnerdPath) ?? new CapabilityLiveSessionService({ transportOptions: { - ...evalProviderTransportOptions(requestedProvider, request.limits.turnTimeoutMs), + ...evalProviderTransportOptions(requestedProvider, request.limits.turnTimeoutMs, request.session.workingDirectory), runnerBinary: runnerdPath, ...(cli.candidateProfile === undefined ? {} : { acpxCandidateProfile: cli.candidateProfile }), runtimeContext, diff --git a/packages/paperclip-runner/src/drivers/acpx/installation-integrity.test.ts b/packages/paperclip-runner/src/drivers/acpx/installation-integrity.test.ts index 547b12a1d78..ed02700f757 100644 --- a/packages/paperclip-runner/src/drivers/acpx/installation-integrity.test.ts +++ b/packages/paperclip-runner/src/drivers/acpx/installation-integrity.test.ts @@ -54,6 +54,104 @@ afterEach(async () => { }); describe("ACPX installation integrity", () => { + it.each([false, true])("admits only the declared qualified Codex platform in its npm-hoisted slot (published platform declaration: %s)", async (publishedDeclaration) => { + const parent = await realpath(await mkdtemp(join(tmpdir(), "paperclip-codex-npm-hoist-"))); + temporaryDirectories.push(parent); + const root = join(parent, "node_modules/@paperclipai/server"); + const runtime = join(root, "node_modules/@openai/codex/package.json"); + const platformRoot = join(parent, "node_modules/@openai/codex-linux-x64"); + const platform = join(platformRoot, "package.json"); + const alias = "npm:@openai/codex@0.160.0-linux-x64"; + const selected = { name: "@paperclipai/server", optionalDependencies: { "@openai/codex-linux-x64": alias } }; + const runtimeMetadata = { name: "@openai/codex", version: "0.160.0", ...(!publishedDeclaration && { optionalDependencies: selected.optionalDependencies }) }; + const platformMetadata = { name: "@openai/codex", version: "0.160.0-linux-x64", os: ["linux"], cpu: ["x64"] }; + await mkdir(dirname(runtime), { recursive: true }); + await mkdir(platformRoot, { recursive: true }); + const reset = async () => Promise.all([ + writeFile(join(root, "package.json"), JSON.stringify(selected)), + writeFile(runtime, JSON.stringify(runtimeMetadata)), + writeFile(platform, JSON.stringify(platformMetadata)), + ]); + await reset(); + const resolver = createAcpxPackageJsonResolver(root); + expect(resolver("@openai/codex-linux-x64", runtime)).toBe(platform); + // Direct provider discovery remains closed; only the selected runtime can + // hand off its explicitly declared, qualified executable dependency. + expect(() => resolver("@openai/codex-linux-x64")).toThrow("outside the selected provider root"); + for (const [file, metadata] of [ + [join(root, "package.json"), { ...selected, optionalDependencies: {} }], + [join(root, "package.json"), { ...selected, optionalDependencies: { "@openai/codex-linux-x64": "latest" } }], + [runtime, { ...runtimeMetadata, version: "0.159.0" }], + [runtime, { ...runtimeMetadata, optionalDependencies: {} }], + [runtime, { ...runtimeMetadata, optionalDependencies: { "@openai/codex-linux-x64": "latest" } }], + [platform, { ...platformMetadata, version: "0.159.0-linux-x64" }], + [platform, { ...platformMetadata, name: "unqualified-provider" }], + [platform, { ...platformMetadata, cpu: ["arm64"] }], + ] as const) { + await writeFile(file, JSON.stringify(metadata)); + expect(() => resolver("@openai/codex-linux-x64", runtime)).toThrow("outside the selected provider root"); + await reset(); + } + const outside = join(parent, "outside-platform"); + await mkdir(outside); + await writeFile(join(outside, "package.json"), JSON.stringify(platformMetadata)); + await rm(platformRoot, { recursive: true }); + await symlink(outside, platformRoot); + expect(() => resolver("@openai/codex-linux-x64", runtime)).toThrow("outside the selected provider root"); + }); + + it.each([false, true])("retains Codex executable version and byte admission for the hoisted platform (stale bundled alias: %s)", async (staleBundledAlias) => { + const parent = await realpath(await mkdtemp(join(tmpdir(), "paperclip-codex-npm-admission-"))); + temporaryDirectories.push(parent); + const root = join(parent, "node_modules/@paperclipai/server"); + const bridge = join(root, "node_modules/@agentclientprotocol/codex-acp"); + const runtime = join(bridge, "node_modules/@openai/codex"); + const platformRoot = join(parent, "node_modules/@openai/codex-linux-x64"); + const native = join(platformRoot, "vendor/x86_64-unknown-linux-musl/bin/codex"); + const optionalDependencies = { "@openai/codex-linux-x64": "npm:@openai/codex@0.160.0-linux-x64" }; + await Promise.all([mkdir(runtime, { recursive: true }), mkdir(dirname(native), { recursive: true })]); + const command = 'process.stdout.write("unexecuted fixture");'; + await Promise.all([ + writeFile(join(root, "package.json"), JSON.stringify({ name: "@paperclipai/server", optionalDependencies })), + writeFile(join(bridge, "package.json"), JSON.stringify({ name: "@agentclientprotocol/codex-acp", version: "1.6.2", bin: "server.js" })), + writeFile(join(bridge, "server.js"), command), + writeFile(join(runtime, "package.json"), JSON.stringify({ name: "@openai/codex", version: "0.160.0" })), + writeFile(native, "unqualified native bytes", { mode: 0o755 }), + ]); + const profile = { ...resolveQualifiedAcpxProfile("codex", "gpt-5.6-sol"), commandDigest: `sha256:${createHash("sha256").update(command).digest("hex")}` }; + const resolver = createAcpxPackageJsonResolver(root); + const platformSpy = vi.spyOn(process, "platform", "get").mockReturnValue("linux"); + const archSpy = vi.spyOn(process, "arch", "get").mockReturnValue("x64"); + try { + const metadata = { name: "@openai/codex", version: "0.160.0-linux-x64", os: ["linux"], cpu: ["x64"] }; + await writeFile(join(platformRoot, "package.json"), JSON.stringify(metadata)); + // A stale in-bundle package shadows the correct declared npm slot and + // must fail rather than falling through to the hoisted installation. + if (staleBundledAlias) { + const stale = join(runtime, "node_modules/@openai/codex-linux-x64"); + await mkdir(stale, { recursive: true }); + await writeFile(join(stale, "package.json"), JSON.stringify({ ...metadata, version: "0.159.0-linux-x64" })); + } + // A callback without an explicit selected package authority cannot + // adopt the published server's platform declaration implicitly. + await expect(verifyQualifiedAcpxInstallation(profile, (...args) => resolver(...args))).rejects.toThrow( + "runtime omitted its verified platform executable package", + ); + await expect(verifyQualifiedAcpxInstallation(profile, resolver)).rejects.toThrow(staleBundledAlias + ? /runtime executable package version mismatch/ + : /digest mismatch/); + vi.stubEnv("PAPERCLIP_ACPX_PROVIDER_PACKAGE_ROOT", root); + vi.stubEnv("PAPERCLIP_ACPX_PROVIDER_PACKAGE_MANIFEST", undefined); + await expect(verifyQualifiedAcpxInstallation(profile)).rejects.toThrow(staleBundledAlias + ? /runtime executable package version mismatch/ + : /digest mismatch/); + } finally { + vi.unstubAllEnvs(); + platformSpy.mockRestore(); + archSpy.mockRestore(); + } + }); + it.each([["linux", "arm64"], ["darwin", "ia32"], ["freebsd", "x64"]] as const)( "rejects the actual Claude runtime probe on unsupported %s %s", async (platform, arch) => { diff --git a/packages/paperclip-runner/src/drivers/acpx/installation-integrity.ts b/packages/paperclip-runner/src/drivers/acpx/installation-integrity.ts index f015d60799d..6b7f7cc5607 100644 --- a/packages/paperclip-runner/src/drivers/acpx/installation-integrity.ts +++ b/packages/paperclip-runner/src/drivers/acpx/installation-integrity.ts @@ -7,7 +7,7 @@ import { type ChildProcess, type SpawnOptionsWithoutStdio, } from "node:child_process"; -import { constants, existsSync, realpathSync } from "node:fs"; +import { closeSync, constants, existsSync, fstatSync, openSync, readFileSync, realpathSync } from "node:fs"; import { lstat, open, @@ -289,6 +289,11 @@ export type AcpxPackageJsonResolver = ( issuerPackageJsonPath?: string, ) => string; +const rootedPackageResolvers = new WeakMap(); + export function createAcpxPackageJsonResolver( providerPackageRoot: string | undefined, providerPackageManifest?: string, @@ -332,7 +337,7 @@ export function createAcpxPackageJsonResolver( "ACPX provider node_modules resolves outside the selected provider root", ); } - return (packageName, issuerPackageJsonPath) => { + const resolver: AcpxPackageJsonResolver = (packageName, issuerPackageJsonPath) => { const canonicalIssuer = issuerPackageJsonPath === undefined ? canonicalManifest @@ -345,13 +350,75 @@ export function createAcpxPackageJsonResolver( const packageJsonPath = realpathSync( resolvePackageJsonFromIssuer(packageName, canonicalIssuer), ); - if (!pathIsInside(canonicalNodeModules, packageJsonPath)) { + if (!pathIsInside(canonicalNodeModules, packageJsonPath) + && !isDeclaredHoistedCodexPlatform(canonicalRoot, canonicalManifest, canonicalIssuer, packageName, packageJsonPath)) { throw new Error( `ACPX provider package ${packageName} resolves outside the selected provider root`, ); } return packageJsonPath; }; + rootedPackageResolvers.set(resolver, { root: canonicalRoot, manifest: canonicalManifest }); + return resolver; +} + +function hasPublishedCodexPlatformDeclaration(resolver: AcpxPackageJsonResolver, runtimeManifest: string): boolean { + const authority = rootedPackageResolvers.get(resolver); + if (!authority || !pathIsInside(authority.root, runtimeManifest)) return false; + const selected = readResolverPackageJson(authority.manifest); + const runtime = readResolverPackageJson(runtimeManifest); + const declarations = selected.optionalDependencies as Record | undefined; + return selected.name === "@paperclipai/server" + && runtime.name === QUALIFIED_CODEX_LINUX_X64_RUNTIME.runtimePackageName + && runtime.version === QUALIFIED_CODEX_LINUX_X64_RUNTIME.runtimePackageVersion + && runtime.optionalDependencies === undefined + && declarations?.[QUALIFIED_CODEX_LINUX_X64_RUNTIME.packageName] === QUALIFIED_CODEX_LINUX_X64_RUNTIME.dependencyDeclaration; +} + +// npm installs an optional platform alias outside a bundled server graph. +// Admit only the qualified executable package through its declared npm slot; +// all executable bytes still pass the existing digest/no-follow admission. +function isDeclaredHoistedCodexPlatform(root: string, manifest: string, issuer: string, packageName: string, packageJsonPath: string): boolean { + const qualified = QUALIFIED_CODEX_LINUX_X64_RUNTIME; + if (packageName !== qualified.packageName || issuer === manifest) return false; + try { + const selected = readResolverPackageJson(manifest); + const runtime = readResolverPackageJson(issuer); + const optional = (value: Record) => value.optionalDependencies as Record | undefined; + if (selected.name !== "@paperclipai/server" || runtime.name !== qualified.runtimePackageName + || runtime.version !== qualified.runtimePackageVersion + || optional(selected)?.[packageName] !== qualified.dependencyDeclaration + || (runtime.optionalDependencies !== undefined && optional(runtime)?.[packageName] !== qualified.dependencyDeclaration)) return false; + if (realpathSync(resolvePackageJsonFromIssuer(packageName, manifest)) !== packageJsonPath) return false; + const declaredSlot = (createRequire(manifest).resolve.paths(packageName) ?? []).some((directory) => { + if (!pathIsInside(dirname(directory), root)) return false; + try { + return realpathSync(directory) === directory + && pathIsInside(directory, packageJsonPath) + && realpathSync(resolve(directory, packageName, "package.json")) === packageJsonPath + && realpathSync(resolve(directory, packageName)) === resolve(directory, packageName); + } catch { return false; } + }); + if (!declaredSlot) return false; + const platform = readResolverPackageJson(packageJsonPath); + return (platform.name === qualified.packageName || platform.name === qualified.runtimePackageName) + && platform.version === qualified.packageVersion + && Array.isArray(platform.os) && platform.os.includes("linux") + && Array.isArray(platform.cpu) && platform.cpu.includes("x64"); + } catch { return false; } +} + +function readResolverPackageJson(path: string): Record { + const fd = openSync(path, constants.O_RDONLY | constants.O_NOFOLLOW); + try { + const before = fstatSync(fd, { bigint: true }); + if (!before.isFile() || before.size < 1n || before.size > BigInt(MAX_PACKAGE_JSON_BYTES)) throw new Error("Invalid provider package manifest"); + const bytes = readFileSync(fd), after = fstatSync(fd, { bigint: true }); + if (bytes.length !== Number(before.size) || !sameIdentity(fileIdentity(before), fileIdentity(after))) throw new Error("Provider package manifest changed during resolution"); + const value: unknown = JSON.parse(bytes.toString("utf8")); + if (!value || typeof value !== "object" || Array.isArray(value)) throw new Error("Invalid provider package metadata"); + return value as Record; + } finally { closeSync(fd); } } function resolvePackageJsonFromIssuer( @@ -563,6 +630,14 @@ export async function verifyQualifiedAcpxInstallation( profile: QualifiedAcpxProfile, resolvePackageJson: AcpxPackageJsonResolver = defaultPackageJsonResolver, ): Promise { + // Keep the selected package authority available when its bundled Codex + // wrapper delegates platform installation to the published server manifest. + if (profile.agent === "codex" && resolvePackageJson === defaultPackageJsonResolver && process.env.PAPERCLIP_ACPX_PROVIDER_PACKAGE_ROOT !== undefined) { + resolvePackageJson = createAcpxPackageJsonResolver( + process.env.PAPERCLIP_ACPX_PROVIDER_PACKAGE_ROOT, + process.env.PAPERCLIP_ACPX_PROVIDER_PACKAGE_MANIFEST, + ); + } const builtin = profile.agent === "grok"; if (builtin && (profile.agentServerPackage !== "builtin:grok-acp" || profile.agentServerVersion !== "1" || profile.agentRuntimePackage !== "native:grok" || profile.agentRuntimeVersion !== "1.0.13")) { throw new Error("Grok builtin profile identity mismatch"); @@ -938,14 +1013,12 @@ async function verifyQualifiedRuntimeExecutable(input: { } const optionalDependencies = input.runtimePackage.optionalDependencies; - if ( - typeof optionalDependencies !== "object" || - optionalDependencies === null || - Array.isArray(optionalDependencies) || - (optionalDependencies as Record)[ - qualification.packageName - ] !== qualification.dependencyDeclaration - ) { + const ownDeclaration = typeof optionalDependencies === "object" + && optionalDependencies !== null && !Array.isArray(optionalDependencies) + && (optionalDependencies as Record)[qualification.packageName] === qualification.dependencyDeclaration; + const publishedDeclaration = input.profile.agent === "codex" && optionalDependencies === undefined + && hasPublishedCodexPlatformDeclaration(input.resolvePackageJson, input.runtimePackageJsonPath); + if (!ownDeclaration && !publishedDeclaration) { throw new Error( `ACPX ${input.profile.agent} runtime omitted its verified platform executable package`, ); diff --git a/packages/paperclip-runner/src/drivers/codex/codex-command.ts b/packages/paperclip-runner/src/drivers/codex/codex-command.ts new file mode 100644 index 00000000000..37d53bf9fc2 --- /dev/null +++ b/packages/paperclip-runner/src/drivers/codex/codex-command.ts @@ -0,0 +1,78 @@ +import { accessSync, constants, readFileSync, realpathSync, statSync } from "node:fs"; +import { createRequire } from "node:module"; +import { delimiter, dirname, isAbsolute, relative, resolve, sep } from "node:path"; + +/** Prefer the installed dependency; compatibility is established by the CLI protocol. */ +export function resolveCodexCommand( + issuer: string | URL = import.meta.url, + environment: NodeJS.ProcessEnv = process.env, + workingDirectory = process.cwd(), +): string { + try { + const runnerRequire = createRequire(issuer); + let manifestPath: string; + try { + let bridgeManifest: string; + try { + bridgeManifest = runnerRequire.resolve("@agentclientprotocol/codex-acp/package.json"); + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "MODULE_NOT_FOUND") throw error; + // pnpm can retain the bridge inside its declared adapter dependency. + const adapter = runnerRequire.resolve("@paperclipai/adapter-codex-local/server"); + bridgeManifest = createRequire(adapter).resolve("@agentclientprotocol/codex-acp/package.json"); + } + manifestPath = createRequire(bridgeManifest).resolve("@openai/codex/package.json"); + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "MODULE_NOT_FOUND") throw error; + try { + manifestPath = runnerRequire.resolve("@openai/codex/package.json"); + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "MODULE_NOT_FOUND") throw error; + return resolveCodexOnPath(environment, workingDirectory); + } + } + manifestPath = realpathSync(manifestPath); + const manifest = JSON.parse(readFileSync(manifestPath, "utf8")) as { + name?: unknown; version?: unknown; bin?: string | Record; + }; + if (manifest.name !== "@openai/codex") throw new Error("Installed Codex dependency has an unexpected package identity"); + const bin = typeof manifest.bin === "string" ? manifest.bin : manifest.bin?.codex; + if (typeof bin !== "string" || !bin || bin.includes("\0") || isAbsolute(bin)) { + throw new Error("Installed Codex dependency does not expose a contained executable"); + } + const root = dirname(manifestPath), candidate = resolve(root, bin); + const inside = (path: string) => { + const value = relative(root, path); + return value !== "" && value !== ".." && !value.startsWith(`..${sep}`) && !isAbsolute(value); + }; + if (!inside(candidate)) throw new Error("Installed Codex executable escapes its package"); + const executable = realpathSync(candidate); + if (!inside(executable)) throw new Error("Installed Codex executable escapes its package"); + if (!statSync(executable).isFile()) throw new Error("Installed Codex executable is not a regular file"); + accessSync(executable, constants.X_OK); + return executable; + } catch (error) { + throw new Error(`Codex runtime unavailable: ${(error as Error).message}. Install Codex or restore Paperclip's runtime dependencies.`, { cause: error }); + } +} + +function resolveCodexOnPath(environment: NodeJS.ProcessEnv, workingDirectory: string): string { + const names = process.platform === "win32" + ? ["codex", ...(environment.PATHEXT ?? ".COM;.EXE;.BAT;.CMD").split(";").filter(Boolean).map(extension => `codex${extension.toLowerCase()}`)] + : ["codex"]; + // Empty PATH components mean cwd; an omitted PATH has no search entries. + for (const directory of environment.PATH === undefined ? [] : environment.PATH.split(delimiter)) { + for (const name of names) { + try { + const executable = realpathSync(resolve(workingDirectory, directory, name)); + if (!statSync(executable).isFile()) continue; + accessSync(executable, constants.X_OK); + return executable; + } catch { /* Continue looking for an executable in the selected host PATH. */ } + } + } + throw new Error("No installed Codex dependency or executable on PATH"); +} + +// Compatibility export: the release pin is not a runtime version gate. +export const resolvePinnedCodexCommand = resolveCodexCommand; diff --git a/packages/paperclip-runner/src/drivers/codex/codex-security-config.test.ts b/packages/paperclip-runner/src/drivers/codex/codex-security-config.test.ts index acecaf2c838..635afadad44 100644 --- a/packages/paperclip-runner/src/drivers/codex/codex-security-config.test.ts +++ b/packages/paperclip-runner/src/drivers/codex/codex-security-config.test.ts @@ -1,5 +1,7 @@ import { createRequire } from "node:module"; +import { execFileSync } from "node:child_process"; import { dirname, resolve } from "node:path"; +import { resolvePinnedCodexCommand } from "./codex-command.js"; import { evalProviderTransportOptions } from "../../cli/eval-provider-runtime.js"; import { describe, expect, it } from "vitest"; @@ -12,6 +14,236 @@ import { } from "./codex-security-config.js"; describe("Codex security configuration", () => { + it("uses the same actual pinned executable for native defaults and direct evals without a global Codex PATH", () => { + const command = resolvePinnedCodexCommand(); + expect(evalProviderTransportOptions("codex").codexCommand).toBe(command); + expect(execFileSync(command, ["--version"], { env: { PATH: dirname(process.execPath) }, + encoding: "utf8", timeout: 30_000, maxBuffer: 16 * 1024 }).trim()).toBe("codex-cli 0.160.0"); + }); + + it("resolves usable installed Codex commands and an absent-package PATH fallback while rejecting malformed or escaped dependencies", () => { + // Vitest adds its dependency directories to global module lookup paths. + // Use the existing Node/tsx boundary so an absent fixture dependency cannot + // be supplied by the test runner's installed bridge instead. + const source = ` + import assert from "node:assert/strict"; + import { execFileSync } from "node:child_process"; + import { createRequire } from "node:module"; + import { chmodSync, mkdirSync, mkdtempSync, realpathSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; + import { tmpdir } from "node:os"; + import { delimiter, dirname, isAbsolute, join } from "node:path"; + import { resolveCodexCommand, resolvePinnedCodexCommand } from ${JSON.stringify(new URL("./codex-command.ts", import.meta.url).href)}; + import { codexExecutableReadOnlyRoots, createIsolatedCodexAppServerArgs } from ${JSON.stringify(new URL("./codex-security-config.ts", import.meta.url).href)}; + const root = realpathSync(mkdtempSync(join(tmpdir(), "paperclip-pinned-codex-test-"))); + const pathOnlyRoot = realpathSync(mkdtempSync(join(tmpdir(), "paperclip-path-codex-test-"))); + const pathDirectory = join(pathOnlyRoot, "bin"); + const pathCommand = join(pathDirectory, "codex"); + const commandEnvironment = { PATH: pathDirectory }; + const issuer = join(root, "node_modules/@paperclipai/server/dist/vendor/paperclip-runner/drivers/codex/codex-command.js"); + const adapter = join(root, "node_modules/@paperclipai/adapter-codex-local"); + const bridge = join(adapter, "node_modules/@agentclientprotocol/codex-acp"); + const codex = join(bridge, "node_modules/@openai/codex"); + const executable = join(codex, "bin/codex.js"); + const platformName = "@openai/codex-" + process.platform + "-" + process.arch; + const platform = join(root, "node_modules", platformName), vendor = join(platform, "vendor"); + const metadata = { name: "@openai/codex", version: "0.160.0", bin: { codex: "bin/codex.js" }, + optionalDependencies: { [platformName]: "npm:@openai/codex@0.160.0-" + process.platform + "-" + process.arch } }; + try { + mkdirSync(pathDirectory); + writeFileSync(pathCommand, "#!" + process.execPath + "\\nif (process.argv.slice(2).join(' ') !== '--version') process.exit(9); console.log('codex-cli 0.141.0');\\n", { mode: 0o755 }); + const pathOnlyIssuer = join(pathOnlyRoot, "standalone-command.js"); + assert.equal(resolveCodexCommand(pathOnlyIssuer, commandEnvironment), pathCommand, "Only absent dependency graphs use the selected PATH command"); + assert.equal(execFileSync(resolveCodexCommand(pathOnlyIssuer, commandEnvironment), ["--version"], { + env: commandEnvironment, encoding: "utf8", timeout: 5_000 }).trim(), "codex-cli 0.141.0"); + const relativeCommand = resolveCodexCommand(pathOnlyIssuer, { PATH: "./bin" }, pathOnlyRoot); + assert.equal(relativeCommand, pathCommand, "Relative PATH entries use the provider working directory"); + assert.ok(isAbsolute(relativeCommand), "Provider launch receives an absolute executable"); + assert.equal(execFileSync(relativeCommand, ["--version"], { + cwd: root, env: { PATH: "/missing-codex-command" }, encoding: "utf8", timeout: 5_000 }).trim(), "codex-cli 0.141.0"); + const cwdCommand = join(pathOnlyRoot, "codex"); + writeFileSync(cwdCommand, "#!" + process.execPath + "\\n", { mode: 0o755 }); + const secondDirectory = join(pathOnlyRoot, "second-bin"); + mkdirSync(secondDirectory); + const secondCommand = join(secondDirectory, "codex"); + writeFileSync(secondCommand, "#!" + process.execPath + "\\n", { mode: 0o755 }); + for (const path of [".", "", "./missing" + delimiter, delimiter + "./bin"]) { + assert.equal(resolveCodexCommand(pathOnlyIssuer, { PATH: path }, pathOnlyRoot), cwdCommand, + "Dot and empty PATH entries retain their ordered current-directory meaning"); + } + for (const [path, expected] of [ + ["./bin" + delimiter + ".", pathCommand], + ["./bin" + delimiter + "./second-bin", pathCommand], + ["./second-bin" + delimiter + "./bin", secondCommand], + ["./missing" + delimiter + "./bin", pathCommand], + ]) assert.equal(resolveCodexCommand(pathOnlyIssuer, { PATH: path }, pathOnlyRoot), expected, + "Relative PATH resolution preserves search ordering and skips missing entries"); + for (const environment of [{}, { PATH: "/missing-codex-command" }, { PATH: "./missing" }]) { + assert.throws(() => resolveCodexCommand(pathOnlyIssuer, environment, pathOnlyRoot), + /runtime unavailable.*No installed Codex dependency or executable on PATH/, + "An absent PATH must not implicitly authorize the provider working directory"); + } + assert.throws(() => resolveCodexCommand(pathOnlyIssuer, { PATH: "./bin" }, root), /runtime unavailable/, + "Relative PATH must not use a different provider working directory"); + const previousCwd = process.cwd(); + try { + process.chdir(pathOnlyRoot); + assert.equal(resolveCodexCommand(pathOnlyIssuer, { PATH: "./bin" }), pathCommand, + "Omitted working directory follows the current process directory"); + } finally { process.chdir(previousCwd); } + const directCodex = join(pathOnlyRoot, "node_modules/@openai/codex"); + const directCommand = join(directCodex, "bin/codex.js"); + mkdirSync(dirname(directCommand), { recursive: true }); + writeFileSync(join(directCodex, "package.json"), JSON.stringify({ ...metadata, version: "0.155.0" })); + writeFileSync(directCommand, "#!" + process.execPath + "\\n", { mode: 0o755 }); + assert.equal(resolveCodexCommand(pathOnlyIssuer, commandEnvironment), directCommand, "A directly installed older wrapper takes precedence over PATH"); + writeFileSync(join(directCodex, "package.json"), JSON.stringify({ ...metadata, name: "unowned-wrapper" })); + assert.throws(() => resolveCodexCommand(pathOnlyIssuer, commandEnvironment), /unexpected package identity/); + writeFileSync(join(directCodex, "package.json"), "{"); + assert.throws(() => resolveCodexCommand(pathOnlyIssuer, commandEnvironment), /runtime unavailable/); + mkdirSync(dirname(issuer), { recursive: true }); + mkdirSync(join(codex, "bin"), { recursive: true }); + writeFileSync(join(adapter, "package.json"), JSON.stringify({ name: "@paperclipai/adapter-codex-local", exports: { "./server": "./server.js" } })); + writeFileSync(join(adapter, "server.js"), ""); + writeFileSync(join(bridge, "package.json"), JSON.stringify({ name: "@agentclientprotocol/codex-acp", version: "1.6.2" })); + writeFileSync(join(codex, "package.json"), JSON.stringify(metadata)); + mkdirSync(vendor, { recursive: true }); + const platformMetadata = { name: "@openai/codex", version: "0.160.0-" + process.platform + "-" + process.arch, + os: [process.platform], cpu: [process.arch] }; + writeFileSync(join(platform, "package.json"), JSON.stringify(platformMetadata)); + writeFileSync(executable, "#!" + process.execPath + "\\nif (process.argv.slice(2).join(' ') !== '--version') process.exit(9); console.log('codex-cli 0.160.0');\\n", { mode: 0o755 }); + assert.equal(resolveCodexCommand(issuer, commandEnvironment), executable, "The selected installed bridge closure takes precedence over PATH"); + assert.equal(resolvePinnedCodexCommand(issuer, commandEnvironment), executable, "The published compatibility alias keeps ordinary command semantics"); + const roots = codexExecutableReadOnlyRoots({ PATH: "/missing-codex-command" }, executable); + assert.ok(roots.includes(vendor), "The npm-hoisted native vendor directory remains readable"); + assert.ok(!roots.includes(platform) && !roots.includes(root), "Hoisting must not expose npm ancestry"); + assert.equal(execFileSync(resolveCodexCommand(issuer, commandEnvironment), ["--version"], { env: { PATH: "/missing-codex-command" }, + encoding: "utf8", timeout: 5_000 }).trim(), "codex-cli 0.160.0"); + + for (const bin of ["../../escaped-codex", "/usr/bin/codex", {}, ""]) { + writeFileSync(join(codex, "package.json"), JSON.stringify({ ...metadata, bin })); + assert.throws(() => resolveCodexCommand(issuer, commandEnvironment), /contained executable|escapes its package/); + } + for (const version of ["0.159.0", "0.161.0"]) { + writeFileSync(join(codex, "package.json"), JSON.stringify({ ...metadata, version })); + assert.equal(resolveCodexCommand(issuer, commandEnvironment), executable, "Independently usable older and newer installed wrappers remain selectable"); + } + writeFileSync(join(codex, "package.json"), JSON.stringify({ ...metadata, name: "unowned-wrapper" })); + assert.throws(() => resolveCodexCommand(issuer, commandEnvironment), /unexpected package identity/); + writeFileSync(join(codex, "package.json"), "{"); + assert.throws(() => resolveCodexCommand(issuer, commandEnvironment), /runtime unavailable/); + writeFileSync(join(codex, "package.json"), JSON.stringify(metadata)); + chmodSync(executable, 0o600); + assert.throws(() => resolveCodexCommand(issuer, commandEnvironment), /runtime unavailable/); + rmSync(executable); + writeFileSync(join(root, "external-codex"), "external executable", { mode: 0o755 }); + symlinkSync(join(root, "external-codex"), executable); + assert.throws(() => resolveCodexCommand(issuer, commandEnvironment), /escapes its package/); + rmSync(executable); + assert.throws(() => resolveCodexCommand(issuer, commandEnvironment), /runtime unavailable/); + rmSync(codex, { recursive: true }); + // Use the existing fresh-process boundary: Node caches earlier module + // paths, whereas a genuinely absent installed graph must use PATH. + const resolverModule = ${JSON.stringify(new URL("./codex-command.ts", import.meta.url).href)}; + const freshAbsence = [ + 'import assert from "node:assert/strict";', + 'import { resolveCodexCommand } from ' + JSON.stringify(resolverModule) + ';', + 'assert.equal(resolveCodexCommand(' + JSON.stringify(issuer) + ',' + JSON.stringify(commandEnvironment) + '),' + JSON.stringify(pathCommand) + ');', + 'assert.throws(() => resolveCodexCommand(' + JSON.stringify(issuer) + ', { PATH: "/missing-codex-command" }), /runtime unavailable.*No installed Codex dependency or executable on PATH/);', + ].join("\\n"); + execFileSync(process.execPath, ['--no-global-search-paths', '--import', createRequire(resolverModule).resolve('tsx'), + '--input-type=module', '--eval', freshAbsence], { env: commandEnvironment, encoding: 'utf8', timeout: 5_000 }); + + // npm installs the host alias declared by the published server while + // its bundled JS wrapper deliberately omits optionalDependencies. + const server = join(root, "node_modules/@paperclipai/server"); + const serverManifest = join(server, "package.json"); + const normalized = join(server, "node_modules/@openai/codex"); + const normalizedManifest = join(normalized, "package.json"); + const normalizedCommand = join(normalized, "bin/codex.js"); + const publishedBridgeManifest = join(server, "node_modules/@agentclientprotocol/codex-acp/package.json"); + const publishedBridgeMetadata = { name: "@agentclientprotocol/codex-acp", version: "1.6.2" }; + const serverMetadata = { name: "@paperclipai/server", dependencies: { "@agentclientprotocol/codex-acp": "1.6.2" }, + optionalDependencies: metadata.optionalDependencies }; + const normalizedMetadata = { ...metadata, optionalDependencies: undefined }; + mkdirSync(dirname(normalizedCommand), { recursive: true }); + mkdirSync(dirname(publishedBridgeManifest), { recursive: true }); + writeFileSync(publishedBridgeManifest, JSON.stringify(publishedBridgeMetadata)); + writeFileSync(serverManifest, JSON.stringify(serverMetadata)); + writeFileSync(normalizedManifest, JSON.stringify(normalizedMetadata)); + writeFileSync(normalizedCommand, "#!" + process.execPath + "\\n", { mode: 0o755 }); + const normalizedRoots = () => codexExecutableReadOnlyRoots({ PATH: "/missing-codex-command" }, normalizedCommand); + assert.ok(normalizedRoots().includes(vendor), "The normalized published wrapper must retain its selected native sandbox resources"); + const sandbox = createIsolatedCodexAppServerArgs({ HOME: "/private-provider-home" }, normalizedRoots()).join("\\n"); + assert.ok(sandbox.includes(JSON.stringify(vendor) + '=\"read\"'), "Native sandbox resources must reach the actual read-only permission configuration"); + assert.ok(sandbox.includes('\"/private-provider-home\"=\"none\"'), "The provider home remains denied"); + for (const ancestor of [platform, dirname(platform), join(root, "node_modules"), server, root]) { + assert.ok(!normalizedRoots().includes(ancestor), "Platform discovery must not grant enclosing npm or server roots"); + } + const restoreMetadata = () => { + writeFileSync(serverManifest, JSON.stringify(serverMetadata)); + writeFileSync(publishedBridgeManifest, JSON.stringify(publishedBridgeMetadata)); + writeFileSync(normalizedManifest, JSON.stringify(normalizedMetadata)); + writeFileSync(join(platform, "package.json"), JSON.stringify(platformMetadata)); + }; + // Resource lookup protects package ownership and paths. Version + // qualification belongs to ACPX, not ordinary Codex shell access. + for (const version of ["0.159.0", "0.161.0"]) { + for (const [path, changed] of [ + [serverManifest, { ...serverMetadata, optionalDependencies: { [platformName]: "npm:@openai/codex@" + version + "-" + process.platform + "-" + process.arch } }], + [serverManifest, { ...serverMetadata, dependencies: { "@agentclientprotocol/codex-acp": version } }], + [publishedBridgeManifest, { ...publishedBridgeMetadata, version }], + [normalizedManifest, { ...normalizedMetadata, version }], + [join(platform, "package.json"), { ...platformMetadata, version: version + "-" + process.platform + "-" + process.arch }], + ]) { + writeFileSync(path, JSON.stringify(changed)); + assert.ok(normalizedRoots().includes(vendor), "Older or newer usable metadata must retain native resources independently of other version numbers"); + for (const ancestor of [platform, dirname(platform), server, root]) assert.ok(!normalizedRoots().includes(ancestor)); + restoreMetadata(); + } + } + for (const [path, changed] of [ + [serverManifest, { ...serverMetadata, optionalDependencies: {} }], + [serverManifest, { ...serverMetadata, name: "unowned-server" }], + [serverManifest, { ...serverMetadata, optionalDependencies: { [platformName]: "npm:unowned/codex@0.160.0-" + process.platform + "-" + process.arch } }], + [serverManifest, { ...serverMetadata, optionalDependencies: { [platformName]: "file:/unowned/native" } }], + [serverManifest, { ...serverMetadata, optionalDependencies: { [platformName]: "npm:@openai/codex@0.160.0-unsupported-os-" + process.arch } }], + [serverManifest, { ...serverMetadata, dependencies: {} }], + [serverManifest, { ...serverMetadata, dependencies: { "@agentclientprotocol/codex-acp": "" } }], + [publishedBridgeManifest, { ...publishedBridgeMetadata, name: "unqualified-bridge" }], + [normalizedManifest, { ...normalizedMetadata, name: "unowned-wrapper" }], + [normalizedManifest, { ...normalizedMetadata, optionalDependencies: {} }], + [join(platform, "package.json"), { ...platformMetadata, name: "unqualified-native" }], + [join(platform, "package.json"), { ...platformMetadata, os: ["unsupported-os"] }], + [join(platform, "package.json"), { ...platformMetadata, cpu: ["unsupported-cpu"] }], + ]) { + writeFileSync(path, JSON.stringify(changed)); + assert.ok(!normalizedRoots().includes(vendor), "Unqualified metadata must not grant native resources"); + restoreMetadata(); + } + const outsideVendor = join(root, "outside-vendor"); + mkdirSync(outsideVendor); + rmSync(vendor, { recursive: true }); + symlinkSync(outsideVendor, vendor); + assert.ok(!normalizedRoots().includes(outsideVendor), "A vendor symlink must not escape the selected platform package"); + rmSync(vendor); mkdirSync(vendor); + const outsidePlatform = join(root, "outside-platform"); + mkdirSync(join(outsidePlatform, "vendor"), { recursive: true }); + writeFileSync(join(outsidePlatform, "package.json"), JSON.stringify(platformMetadata)); + rmSync(platform, { recursive: true }); symlinkSync(outsidePlatform, platform); + assert.ok(!normalizedRoots().includes(join(outsidePlatform, "vendor")), "An alias symlink must not grant a foreign package"); + rmSync(platform); mkdirSync(vendor, { recursive: true }); + writeFileSync(join(platform, "package.json"), JSON.stringify(platformMetadata)); + + } finally { rmSync(root, { recursive: true, force: true }); rmSync(pathOnlyRoot, { recursive: true, force: true }); } + process.stdout.write("PINNED_CODEX_ISOLATION_VERIFIED"); + `; + expect(execFileSync(process.execPath, [ + "--no-global-search-paths", "--import", createRequire(import.meta.url).resolve("tsx"), + "--input-type=module", "--eval", source, + ], { env: { ...process.env, NODE_PATH: "", NODE_OPTIONS: "" }, encoding: "utf8", timeout: 10_000, + maxBuffer: 32 * 1024 })).toBe("PINNED_CODEX_ISOLATION_VERIFIED"); + }); + it("allows only the registered private instruction directory while keeping shared context read-only", () => { const args = createIsolatedCodexAppServerArgs({ HOME: "/host/home" }, ["/runtime/immutable-context"], "/runtime/instruction-edits/run-1").join("\n"); expect(args).toContain('"/runtime/instruction-edits/run-1"="write"'); diff --git a/packages/paperclip-runner/src/drivers/codex/codex-security-config.ts b/packages/paperclip-runner/src/drivers/codex/codex-security-config.ts index f363bd8163b..d02e0cfb4b0 100644 --- a/packages/paperclip-runner/src/drivers/codex/codex-security-config.ts +++ b/packages/paperclip-runner/src/drivers/codex/codex-security-config.ts @@ -1,7 +1,8 @@ import { configuredEnvironmentKeys } from "../../configured-environment.js"; -import { resolve, isAbsolute, join, dirname, delimiter } from "node:path"; -import { existsSync, realpathSync, readFileSync, statSync } from "node:fs"; +import { resolve, isAbsolute, join, dirname, delimiter, relative, sep } from "node:path"; +import { closeSync, constants, existsSync, fstatSync, openSync, realpathSync, readFileSync, statSync } from "node:fs"; import { createRequire } from "node:module"; +import { QUALIFIED_ACPX_PROFILES } from "../acpx/qualified-profiles.js"; import { githubCredentialEnvironmentKeys, @@ -54,8 +55,12 @@ export function codexExecutableReadOnlyRoots(source: NodeJS.ProcessEnv, command // The npm entrypoint launches the platform package's native executable, // which Codex invokes again inside bwrap when starting each shell command. const platformPackage = `@openai/codex-${process.platform}-${process.arch}`; - if (manifest.optionalDependencies?.[platformPackage]) { - const platformManifest = createRequire(manifestPath).resolve(`${platformPackage}/package.json`); + const normalizedPlatformManifest = manifest.optionalDependencies === undefined + ? normalizedCodexPlatformManifest(manifestPath, platformPackage) + : undefined; + if (manifest.optionalDependencies?.[platformPackage] || normalizedPlatformManifest) { + const platformManifest = normalizedPlatformManifest + ?? createRequire(manifestPath).resolve(`${platformPackage}/package.json`); const packageRoot = realpathSync(dirname(platformManifest)); const vendor = realpathSync(resolve(packageRoot, "vendor")); if (vendor.startsWith(`${packageRoot}/`)) add(vendor); @@ -68,6 +73,79 @@ export function codexExecutableReadOnlyRoots(source: NodeJS.ProcessEnv, command return [...roots]; } +/** The published server, rather than its bundled JS wrapper, declares npm's host package. */ +function normalizedCodexPlatformManifest(runtimeManifest: string, platformPackage: string): string | undefined { + const profile = QUALIFIED_ACPX_PROFILES.codex; + const runtime = readCodexPackageManifest(runtimeManifest); + if (runtime.name !== profile.agentRuntimePackage || runtime.optionalDependencies !== undefined) return undefined; + for (let root = dirname(dirname(runtimeManifest)), depth = 0; depth < 24; depth += 1) { + const serverManifest = resolve(root, "package.json"); + if (existsSync(serverManifest)) { + const server = readCodexPackageManifest(serverManifest); + if (server.name === "@paperclipai/server") { + const bridgeDeclaration = server.dependencies?.[profile.agentServerPackage]; + const platformDeclaration = server.optionalDependencies?.[platformPackage]; + const declarationPrefix = `npm:${profile.agentRuntimePackage}@`; + const declarationSuffix = `-${process.platform}-${process.arch}`; + if (typeof bridgeDeclaration !== "string" || !bridgeDeclaration.trim() + || typeof platformDeclaration !== "string" || !platformDeclaration.startsWith(declarationPrefix) + || !platformDeclaration.endsWith(declarationSuffix) + || !/^[0-9A-Za-z][0-9A-Za-z.+-]*$/.test(platformDeclaration.slice(declarationPrefix.length, -declarationSuffix.length)) + || !codexPathInside(resolve(root, "node_modules"), runtimeManifest)) return undefined; + const bridgeManifest = createRequire(serverManifest).resolve(`${profile.agentServerPackage}/package.json`); + if (!codexPathInside(resolve(root, "node_modules"), bridgeManifest) + || realpathSync(bridgeManifest) !== bridgeManifest) return undefined; + const bridge = readCodexPackageManifest(bridgeManifest); + if (bridge.name !== profile.agentServerPackage + || createRequire(bridgeManifest).resolve(`${profile.agentRuntimePackage}/package.json`) !== runtimeManifest) return undefined; + const runtimeSelection = createRequire(runtimeManifest).resolve(`${platformPackage}/package.json`); + const selected = createRequire(serverManifest).resolve(`${platformPackage}/package.json`); + if (runtimeSelection !== selected || realpathSync(selected) !== selected) return undefined; + const slot = (createRequire(serverManifest).resolve.paths(platformPackage) ?? []).some(directory => { + try { + return (dirname(directory) === root || codexPathInside(dirname(directory), root)) + && realpathSync(directory) === directory + && selected === resolve(directory, platformPackage, "package.json") + && realpathSync(dirname(selected)) === dirname(selected); + } catch { return false; } + }); + if (!slot) return undefined; + const native = readCodexPackageManifest(selected); + if ((native.name !== platformPackage && native.name !== profile.agentRuntimePackage) + || !Array.isArray(native.os) || !native.os.includes(process.platform) + || !Array.isArray(native.cpu) || !native.cpu.includes(process.arch)) return undefined; + return selected; + } + } + const parent = dirname(root); + if (parent === root) break; + root = parent; + } + return undefined; +} + +function codexPathInside(root: string, path: string): boolean { + const value = relative(root, path); + return value !== "" && value !== ".." && !value.startsWith(`..${sep}`) && !isAbsolute(value); +} + +function readCodexPackageManifest(path: string): Record & { + dependencies?: Record; + optionalDependencies?: Record; +} { + const descriptor = openSync(path, constants.O_RDONLY | constants.O_NOFOLLOW); + try { + const before = fstatSync(descriptor, { bigint: true }); + if (!before.isFile() || before.nlink !== 1n || before.size < 1n || before.size > 256n * 1024n) throw new Error("Unsafe Codex package manifest"); + const bytes = readFileSync(descriptor), after = fstatSync(descriptor, { bigint: true }); + if (bytes.length !== Number(before.size) || before.dev !== after.dev || before.ino !== after.ino + || before.size !== after.size || before.mtimeNs !== after.mtimeNs || before.ctimeNs !== after.ctimeNs) throw new Error("Codex package manifest changed during lookup"); + const value: unknown = JSON.parse(bytes.toString("utf8")); + if (!value || typeof value !== "object" || Array.isArray(value)) throw new Error("Invalid Codex package manifest"); + return value as ReturnType; + } finally { closeSync(descriptor); } +} + export const CODEX_SKILLLESS_PERMISSION_PROFILE = "paperclip-runner-workspace-only"; export const CODEX_PLANNING_PERMISSION_PROFILE = diff --git a/packages/paperclip-runner/src/index.ts b/packages/paperclip-runner/src/index.ts index 2641cf334c6..8b498a1093e 100644 --- a/packages/paperclip-runner/src/index.ts +++ b/packages/paperclip-runner/src/index.ts @@ -45,6 +45,7 @@ export { export type { DurableRecoveryIdentity } from "./control-plane/prp-transport-types.js"; export * from "./drivers/codex/app-server-transport.js"; export * from "./drivers/codex/codex-app-server-driver.js"; +export { resolveCodexCommand, resolvePinnedCodexCommand } from "./drivers/codex/codex-command.js"; export * from "./drivers/opencode/opencode-server-driver.js"; export * from "./drivers/opencode/mcp-bridge.js"; export * from "./drivers/acpx/qualified-profiles.js"; diff --git a/packages/paperclip-runner/src/live/runnerd-codex-transport.test.ts b/packages/paperclip-runner/src/live/runnerd-codex-transport.test.ts index 612991674ac..486deb680b5 100644 --- a/packages/paperclip-runner/src/live/runnerd-codex-transport.test.ts +++ b/packages/paperclip-runner/src/live/runnerd-codex-transport.test.ts @@ -42,6 +42,7 @@ import { parsePaperclipQuestionSet } from "../contracts/question-set.js"; import { ACPX_CREDENTIAL_BINDING_ENV, createAcpxCredentialBinding, createAcpxSidecarHostEnvironment } from "../drivers/acpx/environment.js"; import { DurablePrpControlPlane } from "../control-plane/durable-prp-control-plane.js"; import * as durableControlPlane from "../control-plane/durable-prp-control-plane.js"; +import * as codexCommandRuntime from "../drivers/codex/codex-command.js"; import { NATIVE_RUNTIME_ASSET_SCHEMA, @@ -1940,6 +1941,108 @@ it("denies the isolated Codex home without denying a remote execution workspace" expect(serialized).toContain('\":workspace_roots\"={\".\"=\"write\"}'); }); +it.each(["installed", "explicit"] as const)("records the selected %s Codex command before launch", async (selection) => { + const root = await mkdtemp(join(tmpdir(), "paperclip-codex-command-selection-")); + const command = join(root, "codex"); + const runnerBinary = join(root, "runnerd"); + await writeFile(command, "#!/bin/sh\nexit 99\n", { mode: 0o700 }); + await writeFile(runnerBinary, "unexecuted fixture artifact", { mode: 0o700 }); + const resolver = vi.spyOn(codexCommandRuntime, "resolveCodexCommand").mockReturnValue(command); + let template: Record | null = null; + const launch = vi.fn(() => { throw new Error("A provider process must not start in this fixture"); }); + const { transport } = createCapabilityRunnerdCodexTransport({ + provider: "codex", runnerBinary, stateDirectory: root, sourceCodexHome: "", + environment: { PATH: "/missing-ambient-codex" }, + ...(selection === "explicit" ? { codexCommand: command } : {}), + controlPlaneRegistration: async (authority) => { + template = structuredClone(authority.store.state.runAttachTemplate!); + throw new Error("fixture_stop_before_runner_launch"); + }, + runnerProcessLauncher: launch, + }); + try { + await expect(transport.request("thread/start", { cwd: root, model: "gpt-6.1-sol", dynamicTools: [] })) + .rejects.toThrow("fixture_stop_before_runner_launch"); + expect(template).toMatchObject({ provider: { command, model: "gpt-6.1-sol" } }); + expect(JSON.stringify(template)).toContain(`${JSON.stringify(command).replaceAll('"', '\\"')}=\\"read\\"`); + if (selection === "explicit") expect(resolver).not.toHaveBeenCalled(); + else expect(resolver).toHaveBeenCalledExactlyOnceWith(undefined, { PATH: "/missing-ambient-codex" }, root); + expect(launch).not.toHaveBeenCalled(); + } finally { + resolver.mockRestore(); + await transport.close(); + await rm(root, { recursive: true, force: true }); + } +}); + +it("rejects remote Codex without a guest executable before resolving controller dependencies", async () => { + const root = await mkdtemp(join(tmpdir(), "paperclip-runner-remote-codex-command-")); + const runnerBinary = join(root, "runnerd"); + await writeFile(runnerBinary, "unexecuted fixture artifact", { mode: 0o700 }); + const resolver = vi.spyOn(codexCommandRuntime, "resolveCodexCommand").mockImplementation(() => { + throw new Error("Controller package resolution must not authorize a guest executable"); + }); + const { transport } = createCapabilityRunnerdCodexTransport({ + provider: "codex", runnerBinary, stateDirectory: root, sourceCodexHome: "", + runnerFilesystemRoot: "/workspaces/task/.paperclip-runtime/session", + }); + try { + await expect(transport.request("thread/start", { cwd: "/workspaces/task", model: "gpt-6.1-sol", dynamicTools: [] })) + .rejects.toThrow("runner_remote_provider_artifact_incompatible: remote Codex omitted its qualified guest executable"); + expect(resolver).not.toHaveBeenCalled(); + } finally { + resolver.mockRestore(); + await transport.close(); + await rm(root, { recursive: true, force: true }); + } +}); + +it.each(["missing", "changed"] as const)("reuses the recorded Codex command when dependency discovery is %s during resume", async (discovery) => { + const root = await mkdtemp(join(tmpdir(), "paperclip-codex-recorded-command-")); + const command = join(root, "recorded-codex"); + const replacement = join(root, "replacement-codex"); + const runnerBinary = join(root, "runnerd"); + await writeFile(command, "#!/bin/sh\nexit 99\n", { mode: 0o700 }); + await writeFile(runnerBinary, "unexecuted fixture artifact", { mode: 0o700 }); + const priorIdentity = { runnerInstanceId: "runner-command", environmentLeaseId: "lease-command", normalizedSessionId: "session-command", + runId: "run-prior", turnId: "turn-prior", itemId: "item-prior" }; + const nextIdentity = { ...priorIdentity, runId: "run-next", turnId: "turn-next", itemId: "item-next" }; + const core = new DurablePrpControlPlane({ stateDirectory: join(root, "control-plane"), identity: priorIdentity, + expectedRunnerVersion: "fixture", expectedRunnerDigest: "sha256:" + createHash("sha256").update(await readFile(runnerBinary)).digest("hex") }); + core.persistRunAttachTemplate({ provider: { kind: "codex", command, args: [], model: "gpt-6.1-sol" }, workspace: { cwd: root } }); + await core.stop(); + await mkdir(join(root, "runner"), { mode: 0o700 }); + await writeFile(join(root, "runner", "runner-state.json"), JSON.stringify({ schema: "paperclip.runner.durable.state.v1", ...priorIdentity, lifecycle: "suspended" }), { mode: 0o600 }); + const resolver = vi.spyOn(codexCommandRuntime, "resolveCodexCommand").mockImplementation(() => { + if (discovery === "missing") throw new Error("The current dependency is missing"); + return replacement; + }); + let template: Record | null = null; + const launch = vi.fn(() => { throw new Error("A provider process must not start in this fixture"); }); + const { transport } = createCapabilityRunnerdCodexTransport({ + provider: "codex", runnerBinary, stateDirectory: root, sourceCodexHome: "", prpIdentity: nextIdentity, + environment: { PATH: "/missing-ambient-codex" }, + controlPlaneRegistration: async (authority) => { + template = structuredClone(authority.store.state.commands.find(command => command.type === "run.attach")?.payload ?? null); + throw new Error("fixture_stop_before_resumed_runner_launch"); + }, + runnerProcessLauncher: launch, + }); + try { + await expect(transport.request("thread/read", { threadId: "provider-prior" })) + .rejects.toThrow("fixture_stop_before_resumed_runner_launch"); + expect(template).toMatchObject({ provider: { command, model: "gpt-6.1-sol" } }); + expect(JSON.stringify(template)).toContain(`${JSON.stringify(command).replaceAll('"', '\\"')}=\\"read\\"`); + expect(JSON.stringify(template)).not.toContain(replacement); + expect(resolver).not.toHaveBeenCalled(); + expect(launch).not.toHaveBeenCalled(); + } finally { + resolver.mockRestore(); + await transport.close(); + await rm(root, { recursive: true, force: true }); + } +}); + it("rejects remote OpenCode before spawn when provider-pack paths are absent", async () => { const root = await mkdtemp(join(tmpdir(), "paperclip-runner-remote-pack-")); const { transport } = createCapabilityRunnerdCodexTransport({ diff --git a/packages/paperclip-runner/src/live/runnerd-codex-transport.ts b/packages/paperclip-runner/src/live/runnerd-codex-transport.ts index ba6f3f32487..222c884056e 100644 --- a/packages/paperclip-runner/src/live/runnerd-codex-transport.ts +++ b/packages/paperclip-runner/src/live/runnerd-codex-transport.ts @@ -4,6 +4,7 @@ import { isAcpxCanonicalInputMethod } from "../drivers/acpx/profile-extensions.j import { RunnerdTraceFrameIndex } from "./runnerd-trace-frame-index.js"; import { waitForWarmAttachmentReadiness } from "./warm-attachment-readiness.js"; import { codexExecutableReadOnlyRoots } from "../drivers/codex/codex-security-config.js"; +import { resolveCodexCommand } from "../drivers/codex/codex-command.js"; import { isCanonicalProviderEventType } from "../provider-events.js"; import { execFileSync } from "node:child_process"; import { createHash, randomUUID } from "node:crypto"; @@ -4558,6 +4559,9 @@ class DurablePrpCodexTransport implements CodexAppServerTransport { } const provider = this.options.provider ?? "codex"; const sourceRuntimeContext = this.options.runtimeContext ?? null; + if (provider === "codex" && this.options.runnerFilesystemRoot && !this.options.codexCommand) { + throw new Error("runner_remote_provider_artifact_incompatible: remote Codex omitted its qualified guest executable; verify the selected environment's Codex artifacts"); + } const runtimeContext = this.options.runnerRuntimeContext ?? sourceRuntimeContext; const localRuntimeContextPath = resolve(this.#root, "runtime-context.json"); @@ -4607,6 +4611,9 @@ class DurablePrpCodexTransport implements CodexAppServerTransport { )); const providerNodeCommand = this.options.providerNodeCommand ?? process.execPath; + const effectiveCodexCommand = provider === "codex" + ? this.options.codexCommand ?? resolveCodexCommand(undefined, this.options.environment, String(params.cwd ?? tmpdir())) + : undefined; const opencodeExecutable = provider === "opencode" ? (this.options.opencodeCommand ?? @@ -4811,7 +4818,7 @@ class DurablePrpCodexTransport implements CodexAppServerTransport { command: provider === "opencode" ? providerNodeCommand - : (this.options.codexCommand ?? "codex"), + : effectiveCodexCommand!, args: provider === "opencode" ? [opencodeProxyPath] @@ -4819,7 +4826,7 @@ class DurablePrpCodexTransport implements CodexAppServerTransport { createRunnerdCodexAppServerArgs({ environment: this.options.environment, codexHome, - codexCommand: this.options.codexCommand, + codexCommand: effectiveCodexCommand, instructionWorkingCopyRoot: runtimeContext?.instructions.workingCopy?.rootPath, readOnlyRoots: [ ...trustedRuntimeReadOnlyRoots( @@ -5297,12 +5304,13 @@ class DurablePrpCodexTransport implements CodexAppServerTransport { if (provider === "codex") { // These controller-owned, token-free paths belong to the new run. // Keep the durable provider profile and thread identity unchanged. + const recordedCommand = record(runAttachTemplate.provider).command; runAttachTemplate.runtimeLaunchArgs = this.options.codexArgs ?? createRunnerdCodexAppServerArgs({ environment: this.options.environment, codexHome, - codexCommand: this.options.codexCommand, + codexCommand: typeof recordedCommand === "string" ? recordedCommand : this.options.codexCommand, instructionWorkingCopyRoot: runtimeContext?.instructions.workingCopy?.rootPath, readOnlyRoots: [ ...trustedRuntimeReadOnlyRoots(this.options.environment), diff --git a/scripts/__tests__/grok-public-install-sandbox.test.mjs b/scripts/__tests__/grok-public-install-sandbox.test.mjs index 316b417d316..d3acce38a3a 100644 --- a/scripts/__tests__/grok-public-install-sandbox.test.mjs +++ b/scripts/__tests__/grok-public-install-sandbox.test.mjs @@ -1,7 +1,10 @@ import assert from 'node:assert/strict'; -import { readFileSync } from 'node:fs'; +import { mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, writeFileSync } from 'node:fs'; +import { spawnSync } from 'node:child_process'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; import test from 'node:test'; -import { GROK_PUBLIC_INSTALL_IMAGE, GROK_PUBLIC_INSTALL_LIFECYCLE, grokConsumerDockerArgs } from '../grok-public-install-sandbox.mjs'; +import { GROK_PUBLIC_INSTALL_IMAGE, GROK_PUBLIC_INSTALL_LIFECYCLE, assertInstalledCodexPlatformProvenance, assertNoBundledCodexPayloads, grokConsumerDockerArgs, installedCodexProbeSource } from '../grok-public-install-sandbox.mjs'; const paths = { assets: '/private/staging/assets', consumer: '/private/staging/consumer', cache: '/private/staging/cache', uid: 1001, gid: 1001 }; const values = (args, flag) => args.flatMap((value, index) => value === flag ? [args[index + 1]] : []); @@ -50,3 +53,152 @@ test('verification never elevates PR-controlled provisioning or cleanup on the h assert.doesNotMatch(source, /\bsudo\b/); assert.ok(source.includes("const prerequisite = join(root, 'native/grok')")); }); + +test('the installed Codex probe exercises the public export and rejects incomplete or mismatched packages', () => { + const root = realpathSync(mkdtempSync(join(tmpdir(), 'installed-codex-probe-'))); + try { + const consumer = join(root, 'consumer'); mkdirSync(consumer); + const target = `${process.platform}-${process.arch}`; + const platformPath = `node_modules/@openai/codex-${target}`; + const platform = join(consumer, platformPath); mkdirSync(platform, { recursive: true }); + const nativeVendor = join(platform, 'vendor'); mkdirSync(nativeVendor); + const manifest = { name: '@openai/codex', version: `0.160.0-${target}`, os: [process.platform], cpu: [process.arch] }; + writeFileSync(join(platform, 'package.json'), JSON.stringify(manifest)); + const entry = { name: manifest.name, version: manifest.version, + resolved: `https://registry.npmjs.org/@openai/codex/-/codex-${manifest.version}.tgz`, integrity: `sha512-${Buffer.alloc(64, 1).toString('base64')}` }; + const lock = { lockfileVersion: 3, packages: { [platformPath]: entry } }; + writeFileSync(join(consumer, 'package-lock.json'), JSON.stringify(lock)); + const server = join(consumer, 'node_modules/@paperclipai/server'); + const vendor = join(server, 'dist/vendor/paperclip-runner'), drivers = join(vendor, 'drivers/acpx'); mkdirSync(drivers, { recursive: true }); + writeFileSync(join(server, 'package.json'), JSON.stringify({ name: '@paperclipai/server', type: 'module' })); + const integrity = join(drivers, 'installation-integrity.js'); + const security = join(vendor, 'drivers/codex/codex-security-config.js'); mkdirSync(join(vendor, 'drivers/codex')); + const securitySource = `export const codexExecutableReadOnlyRoots = () => [${JSON.stringify(nativeVendor)}]; + export const createIsolatedCodexAppServerArgs = (source, roots) => roots.map(root => JSON.stringify(root) + '=\"read\"');`; + writeFileSync(security, securitySource); + const moduleSource = `import assert from 'node:assert/strict'; import { writeFileSync } from 'node:fs'; + export function createAcpxPackageJsonResolver(root, manifest) { assert.equal(root, ${JSON.stringify(server)}); assert.equal(manifest, root + '/package.json'); return () => 'selected-authority'; } + export async function verifyQualifiedAcpxInstallation(profile, resolver) { assert.deepEqual(profile, { agent: 'codex', model: 'gpt-5.4' }); assert.equal(resolver(), 'selected-authority'); + return { openCommand: async () => ({ close: async () => { writeFileSync(${JSON.stringify(join(server, 'lease-closed'))}, 'ok'); } }) }; }`; + writeFileSync(integrity, moduleSource); + writeFileSync(join(drivers, 'qualified-profiles.js'), "export const resolveQualifiedAcpxProfile = (agent, model) => ({ agent, model });"); + const wrapper = join(consumer, 'node_modules/@openai/codex/bin'); mkdirSync(wrapper, { recursive: true }); + writeFileSync(join(wrapper, '../package.json'), JSON.stringify({ name: '@openai/codex', version: '0.160.0' })); + const index = join(vendor, 'index.mjs'), command = join(wrapper, 'codex.js'), probe = join(root, 'probe.mjs'); + const executable = version => writeFileSync(command, `#!/bin/sh\n[ "$1" = --version ] || exit 65\nprintf '%s\\n' 'codex-cli ${version}'\n`, { mode: 0o755 }); + writeFileSync(probe, installedCodexProbeSource(index, consumer, '0.160.0')); + const run = () => spawnSync(process.execPath, [probe], { encoding: 'utf8', timeout: 10_000, env: { PATH: '/usr/bin:/bin', NODE_PATH: '' } }); + writeFileSync(index, `export const resolvePinnedCodexCommand = () => ${JSON.stringify(command)};`); + executable('0.160.0'); + let result = run(); assert.equal(result.status, 0, result.stderr); + assert.equal(JSON.parse(result.stdout).pinnedCodexCommandVerified, true); + assert.equal(JSON.parse(result.stdout).codexPlatformPackageSource, 'official npm'); + assert.equal(JSON.parse(result.stdout).codexConsumerPlatform, target); + assert.equal(JSON.parse(result.stdout).codexQualifiedInstallationVerified, true); + assert.equal(JSON.parse(result.stdout).codexCommandLeaseVerified, true); + assert.equal(JSON.parse(result.stdout).codexNativeSandboxResourcesVerified, true); + assert.equal(readFileSync(join(server, 'lease-closed'), 'utf8'), 'ok'); + assert.equal(JSON.parse(result.stdout).providerCalls, 0); + const legacyPath = 'node_modules/legacy-codex-adapter/node_modules/@openai/codex-' + target; + const legacy = join(consumer, legacyPath); mkdirSync(legacy, { recursive: true }); + const legacyManifest = { ...manifest, version: '0.156.1-' + target }; + writeFileSync(join(legacy, 'package.json'), JSON.stringify(legacyManifest)); + lock.packages[legacyPath] = { ...entry, version: legacyManifest.version, + resolved: 'https://registry.npmjs.org/@openai/codex/-/codex-' + legacyManifest.version + '.tgz' }; + writeFileSync(join(consumer, 'package-lock.json'), JSON.stringify(lock)); + result = run(); assert.equal(result.status, 0, result.stderr); + assert.equal(JSON.parse(result.stdout).codexPlatformPackages.length, 2); + assert.equal(JSON.parse(result.stdout).codexSelectedPlatformPath, platformPath); + assert.equal(JSON.parse(result.stdout).codexSelectedPlatformVersion, '0.160.0-' + target); + const legacyCommand = join(legacy, 'codex'); + writeFileSync(legacyCommand, '#!/bin/sh\nexit 0\n', { mode: 0o755 }); + writeFileSync(index, `export const resolvePinnedCodexCommand = () => ${JSON.stringify(legacyCommand)};`); + result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /Selected Codex platform version must match the qualified pin/); + writeFileSync(index, `import { writeFileSync } from 'node:fs'; export const resolvePinnedCodexCommand = () => { + writeFileSync(${JSON.stringify(join(platform, 'package.json'))}, ${JSON.stringify(JSON.stringify({ ...manifest, altered: true }))}); + return ${JSON.stringify(command)}; };`); + result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /metadata must match its scanned provenance/); + writeFileSync(join(platform, 'package.json'), JSON.stringify(manifest)); + writeFileSync(index, "export const resolvePinnedCodexCommand = () => { throw new Error('Bundled Codex runtime version mismatch'); };"); + result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /Bundled Codex runtime version mismatch/); + writeFileSync(index, `export const resolvePinnedCodexCommand = () => ${JSON.stringify(command)};`); + writeFileSync(security, securitySource.replace(JSON.stringify(nativeVendor), '"/missing-vendor-root"')); + result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /vendor directory must be readable/); + writeFileSync(security, securitySource.replace(JSON.stringify(nativeVendor), `${JSON.stringify(nativeVendor)}, ${JSON.stringify(consumer)}`)); + result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /must not grant enclosing npm/); + writeFileSync(security, securitySource.replace("'=\"read\"'", "'=\"write\"'")); + result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /retain the selected native executable resources/); + writeFileSync(security, securitySource); + const emptySlot = join(server, 'node_modules/@openai/codex-' + target); + mkdirSync(emptySlot, { recursive: true }); + result = run(); assert.equal(result.status, 0, result.stderr); + assert.equal(JSON.parse(result.stdout).codexPlatformPackages.length, 2, + 'An empty nested npm optional slot must not obscure the qualified hoisted host package'); + writeFileSync(join(emptySlot, 'undeclared-native-payload'), 'native fixture; never executed'); + result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /empty regular directory; found.*undeclared-native-payload/); + rmSync(join(emptySlot, 'undeclared-native-payload')); + writeFileSync(join(emptySlot, 'package.json'), '{ malformed'); + result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /SyntaxError/); + rmSync(join(emptySlot, 'package.json')); + writeFileSync(integrity, moduleSource.replace('assert.deepEqual(profile,', "throw new Error('qualified executable digest mismatch'); assert.deepEqual(profile,")); + result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /qualified executable digest mismatch/); + writeFileSync(integrity, moduleSource.replace("writeFileSync(", "throw new Error('command lease close failed'); writeFileSync(")); + result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /command lease close failed/); + writeFileSync(integrity, moduleSource); + lock.packages[platformPath].inBundle = true; + writeFileSync(join(consumer, 'package-lock.json'), JSON.stringify(lock)); + result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /downloaded, not bundled/); + delete lock.packages[platformPath].inBundle; + writeFileSync(join(consumer, 'package-lock.json'), JSON.stringify(lock)); + const foreign = join(consumer, 'node_modules/@openai/codex-win32-arm64'); mkdirSync(foreign); + writeFileSync(join(foreign, 'package.json'), JSON.stringify({ name: '@openai/codex', version: '0.160.0-win32-arm64' })); + result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /only the host/); + rmSync(foreign, { recursive: true }); + executable('9.9.9'); result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /qualified pin/); + rmSync(command); result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /ENOENT/); + const outside = join(root, 'outside'); writeFileSync(outside, '#!/bin/sh\nexit 0\n', { mode: 0o755 }); + writeFileSync(index, `export const resolvePinnedCodexCommand = () => ${JSON.stringify(outside)};`); + result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /inside the installed consumer/); + writeFileSync(index, 'export const unrelated = true;'); result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /must export the pinned Codex resolver/); + } finally { rmSync(root, { recursive: true, force: true }); } +}); + +test('actual tar member checks allow Codex JavaScript but reject nested platform packages and native vendor payloads', () => { + const javascript = ['package/package.json', 'package/node_modules/@openai/codex/package.json', + 'package/node_modules/@openai/codex/bin/codex.js', 'package/dist/vendor/paperclip-runner/index.js']; + assert.doesNotThrow(() => assertNoBundledCodexPayloads(javascript)); + for (const payload of [ + 'package/node_modules/@openai/codex-linux-x64/package.json', + 'package/node_modules/bridge/node_modules/@openai/codex-darwin-arm64/package.json', + 'package/node_modules/@openai/codex-win32-x64/package.json', + 'package/node_modules/@openai/codex/vendor/x86_64-unknown-linux-musl/codex/codex', + 'package/dist/vendor/aarch64-apple-darwin/bin/codex', + 'package/dist/vendor/x86_64-pc-windows-msvc/bin/codex.exe', + ]) assert.throws(() => assertNoBundledCodexPayloads([...javascript, payload]), /must not bundle/); + assert.throws(() => assertNoBundledCodexPayloads(['package/../outside']), /inside the package/); + const source = readFileSync(new URL('../verify-grok-npm-install.mjs', import.meta.url), 'utf8'); + assert.match(source, /assertNoBundledCodexPayloads\(run\('tar', \['-tzf', tarball\]\)/); +}); + +test('consumer provenance rejects bundled, foreign, inconsistent and non-official Codex payloads', () => { + const path = 'node_modules/@openai/codex-linux-x64'; + const manifest = { name: '@openai/codex', version: '0.160.0-linux-x64', os: ['linux'], cpu: ['x64'] }; + const entry = { version: manifest.version, resolved: `https://registry.npmjs.org/@openai/codex/-/codex-${manifest.version}.tgz`, + integrity: `sha512-${Buffer.alloc(64, 1).toString('base64')}` }; + const check = (value = entry, installed = [{ path, manifest }]) => assertInstalledCodexPlatformProvenance({ packages: { [path]: value } }, installed, 'linux-x64'); + assert.equal(check().codexConsumerHostOnly, true); + const legacyManifest = { ...manifest, version: '0.156.1-linux-x64' }; + assert.equal(check({ ...entry, version: legacyManifest.version, + resolved: 'https://registry.npmjs.org/@openai/codex/-/codex-0.156.1-linux-x64.tgz' }, [{ path, manifest: legacyManifest }]).codexConsumerHostOnly, true); + assert.throws(() => check(entry, [{ path, manifest: { ...manifest, version: '^0.160.0-linux-x64' } }]), /exact host version/); + for (const change of [ + { inBundle: true }, { link: true }, { version: '0.156.1-linux-x64' }, + { resolved: 'https://example.com/@openai/codex/-/codex-0.160.0-linux-x64.tgz' }, + { resolved: 'https://registry.npmjs.org/@openai/codex/-/codex-0.156.1-linux-x64.tgz' }, + { resolved: 'https://user@registry.npmjs.org/@openai/codex/-/codex-0.160.0-linux-x64.tgz' }, + { integrity: undefined }, { integrity: 'sha512-AA==' }, + ]) assert.throws(() => check({ ...entry, ...change })); + assert.throws(() => check(entry, []), /must install its host/); + assert.throws(() => check(entry, [{ path: 'node_modules/@openai/codex-darwin-arm64', manifest }]), /only the host/); + assert.throws(() => check(entry, [{ path, manifest: { ...manifest, name: 'untrusted' } }]), /identity/); +}); diff --git a/scripts/acpx-patch-packaging.test.mjs b/scripts/acpx-patch-packaging.test.mjs index 7502350e168..fb7086946ef 100644 --- a/scripts/acpx-patch-packaging.test.mjs +++ b/scripts/acpx-patch-packaging.test.mjs @@ -6,7 +6,9 @@ import { mkdirSync, mkdtempSync, readFileSync, + realpathSync, rmSync, + symlinkSync, writeFileSync, } from "node:fs"; import { readFile } from "node:fs/promises"; @@ -20,11 +22,60 @@ import cliEsbuildConfig from "../cli/esbuild.config.mjs"; import { bundledCliNpmDependencies } from "./cli-bundled-npm-dependencies.mjs"; import { createBundledInstallManifest, + configureBundledProviderOverrides, materializePublishManifest, selectBundledDependencyPatches, + stageBundledEsbuildOptionalDependencies, + stageBundledProviderOptionalDependencies, } from "./prepare-bundled-package.mjs"; const rootPackage = JSON.parse(await readFile(new URL("../package.json", import.meta.url), "utf8")); +const profileData = JSON.parse(await readFile(new URL("../packages/paperclip-runner/acpx-profiles.json", import.meta.url), "utf8")); +const nativeTargets = ["linux-x64", "darwin-arm64", "darwin-x64"]; +function esbuildFixture(t) { + const directory = realpathSync(mkdtempSync(join(tmpdir(), "paperclip-bundled-esbuild-"))); + t.after(() => rmSync(directory, { recursive: true, force: true })); + const graph = join(directory, "node_modules"); + const writePackage = (location, metadata) => { + mkdirSync(location, { recursive: true }); + writeFileSync(join(location, "package.json"), JSON.stringify(metadata)); + return location; + }; + const acpx = writePackage(join(graph, "acpx"), { name: "acpx", version: "0.13.1" }); + const esbuild = writePackage(join(acpx, "node_modules/esbuild"), { + name: "esbuild", version: "0.28.2", scripts: { postinstall: "node install.js" }, + optionalDependencies: Object.fromEntries([...nativeTargets, "win32-x64"].map(target => [`@esbuild/${target}`, "0.28.2"])), + }); + const producerBinary = writePackage(join(graph, "@esbuild/linux-x64"), { + name: "@esbuild/linux-x64", version: "0.28.2", os: ["linux"], cpu: ["x64"], + }); + writePackage(join(graph, "unrelated-linux-x64"), { name: "unrelated-linux-x64", version: "1.0.0" }); + const publish = { name: "@paperclipai/adapter-utils", version: "0.3.1", bundleDependencies: ["acpx"], + dependencies: { acpx: "0.13.1" }, optionalDependencies: { unrelated: "1.0.0" } }; + return { directory, graph, esbuild, producerBinary, publish, writePackage }; +} +const codexTargets = ["linux-x64", "linux-arm64", "darwin-x64", "darwin-arm64", "win32-x64", "win32-arm64"]; +const codexOptional = Object.fromEntries(codexTargets.map(target => [`@openai/codex-${target}`, `npm:@openai/codex@0.160.0-${target}`])); +function codexFixture(t) { + const directory = realpathSync(mkdtempSync(join(tmpdir(), "paperclip-bundled-codex-"))); + t.after(() => rmSync(directory, { recursive: true, force: true })); + const graph = join(directory, "node_modules"); + const writePackage = (location, metadata) => { + mkdirSync(location, { recursive: true }); writeFileSync(join(location, "package.json"), JSON.stringify(metadata)); return location; + }; + const bridge = writePackage(join(graph, "@agentclientprotocol/codex-acp"), { name: "@agentclientprotocol/codex-acp", version: "1.6.2", dependencies: { "@openai/codex": "0.160.0" } }); + writeFileSync(join(bridge, "index.js"), "export {};\n"); + const runtime = writePackage(join(graph, "@openai/codex"), { name: "@openai/codex", version: "0.160.0", bin: { codex: "bin/codex.js" }, optionalDependencies: codexOptional }); + mkdirSync(join(runtime, "bin")); writeFileSync(join(runtime, "bin/codex.js"), "// official JavaScript fixture, never executed\n"); + mkdirSync(join(runtime, "vendor")); writeFileSync(join(runtime, "vendor/codex"), "producer fallback binary fixture, never executed"); + const producer = writePackage(join(graph, "@openai/codex-linux-x64"), { name: "@openai/codex", version: "0.160.0-linux-x64", os: ["linux"], cpu: ["x64"] }); + mkdirSync(join(producer, "vendor")); writeFileSync(join(producer, "vendor/codex"), "producer optional binary fixture, never executed"); + const nested = writePackage(join(bridge, "node_modules/@openai/codex-darwin-arm64"), { name: "@openai/codex", version: "0.160.0-darwin-arm64", os: ["darwin"], cpu: ["arm64"] }); + writePackage(join(graph, "unrelated-linux-x64"), { name: "unrelated-linux-x64", version: "1.0.0" }); + const profiles = [{ agent: "codex", ...profileData.profiles.codex }]; + const publish = { name: "paperclip-codex-pack-fixture", version: "1.0.0", dependencies: { "@agentclientprotocol/codex-acp": "1.6.2" }, bundleDependencies: ["@agentclientprotocol/codex-acp"] }; + return { directory, graph, bridge, runtime, producer, nested, profiles, publish, writePackage }; +} const adapterUtilsPackage = JSON.parse( await readFile(new URL("../packages/adapter-utils/package.json", import.meta.url), "utf8"), ); @@ -40,6 +91,7 @@ const dbPackage = JSON.parse( const releaseScript = await readFile(new URL("./release.sh", import.meta.url), "utf8"); const releaseLib = await readFile(new URL("./release-lib.sh", import.meta.url), "utf8"); const buildNpmScript = await readFile(new URL("./build-npm.sh", import.meta.url), "utf8"); + const acpxRuntimePatch = await readFile( new URL("../patches/acpx@0.13.1.patch", import.meta.url), "utf8", @@ -124,6 +176,149 @@ test("published packages preserve the patched ACPX runtime", () => { assert.equal(cliEsbuildConfig.external.includes("acpx"), false); }); +test("published Codex bridge uses its qualified producer closure without consumer overrides", () => { + const published = materializePublishManifest(serverPackage); + const { installManifest, profiles } = configureBundledProviderOverrides( + createBundledInstallManifest(published, serverPackage.bundleDependencies), + serverPackage.bundleDependencies, rootPackage, profileData, + ); + assert.equal(profiles.length, 1); + for (const profile of profiles) { + assert.equal(published.dependencies[profile.agentServerPackage], profile.agentServerVersion); + assert.ok(published.bundleDependencies.includes(profile.agentServerPackage)); + assert.equal(installManifest.overrides[`${profile.agentServerPackage}@${profile.agentServerVersion}`][profile.agentRuntimePackage], profile.agentRuntimeVersion); + } + assert.equal(published.overrides, undefined); + assert.equal(installManifest.overrides.rollup, undefined); + const bad = structuredClone(rootPackage); + bad.pnpm.overrides["@agentclientprotocol/codex-acp@1.6.2>@openai/codex"] = "0.156.1"; + assert.throws(() => configureBundledProviderOverrides(installManifest, serverPackage.bundleDependencies, bad, profileData), /runtime override must match/); + const unpinned = createBundledInstallManifest(published, serverPackage.bundleDependencies); + unpinned.dependencies["@agentclientprotocol/codex-acp"] = "^1.6.2"; + assert.throws(() => configureBundledProviderOverrides(unpinned, serverPackage.bundleDependencies, rootPackage, profileData), /exact qualified profile/); + const conflict = createBundledInstallManifest(published, serverPackage.bundleDependencies); + conflict.overrides = { "@agentclientprotocol/codex-acp@1.6.2": { "@openai/codex": "0.156.1" } }; + assert.throws(() => configureBundledProviderOverrides(conflict, serverPackage.bundleDependencies, rootPackage, profileData), /conflicting producer override/); +}); + +test("Codex bundles retain pinned JavaScript and delegate official consumer platform aliases without native payloads", (t) => { + const { directory, graph, bridge, runtime, producer, nested, profiles, publish } = codexFixture(t); + const before = [bridge, runtime].map(path => readFileSync(join(path, "package.json"), "utf8")); + const staged = stageBundledProviderOptionalDependencies(directory, publish, profiles); + assert.deepEqual(staged.optionalDependencies, codexOptional); + assert.deepEqual(staged.bundleDependencies, publish.bundleDependencies); + assert.equal(staged.paperclipProviderArtifacts, undefined); + assert.equal(existsSync(producer), false); + assert.equal(existsSync(nested), false, "Transitive producer platform packages must also be removed"); + assert.equal(existsSync(join(runtime, "vendor")), false, "Wrapper fallback native payload must be removed"); + assert.equal(existsSync(join(graph, "unrelated-linux-x64")), true); + assert.equal(readFileSync(join(bridge, "package.json"), "utf8"), before[0], "The patched bridge metadata is unchanged"); + const originalRuntime = JSON.parse(before[1]); delete originalRuntime.optionalDependencies; + assert.deepEqual(JSON.parse(readFileSync(join(runtime, "package.json"), "utf8")), originalRuntime, + "Only the wrapper platform declarations move to the published authority root"); + assert.equal(readFileSync(join(runtime, "bin/codex.js"), "utf8"), "// official JavaScript fixture, never executed\n"); + assert.deepEqual(stageBundledProviderOptionalDependencies(directory, staged, profiles), staged); + writeFileSync(join(directory, "package.json"), JSON.stringify(staged)); + const packs = JSON.parse(execFileSync("npm", ["pack", "--ignore-scripts", "--offline", "--json"], { + cwd: directory, encoding: "utf8", timeout: 30_000, + env: { ...process.env, HOME: directory, npm_config_cache: join(directory, "cache"), npm_config_update_notifier: "false" }, + })); + const paths = packs[0].files.map(file => file.path); + assert.ok(paths.some(path => path.endsWith("@openai/codex/bin/codex.js")), "The actual npm tarball retains its pinned JS runtime"); + assert.equal(paths.some(path => /@openai\/codex-(?:linux|darwin|win32)-|@openai\/codex\/vendor\//.test(path)), false, + "Actual npm pack must contain no native Codex payload on any platform"); +}); + +test("Codex bundle validation fails before deleting producer binaries for altered pins or optional declarations", (t) => { + const { directory, runtime, producer, profiles, publish } = codexFixture(t); + const file = join(runtime, "package.json"), original = JSON.parse(readFileSync(file, "utf8")); + writeFileSync(file, JSON.stringify({ ...original, version: "0.156.1" })); + assert.throws(() => stageBundledProviderOptionalDependencies(directory, publish, profiles), /runtime version mismatch/); + writeFileSync(file, JSON.stringify({ ...original, optionalDependencies: { ...codexOptional, "@openai/codex-untrusted": "0.160.0" } })); + assert.throws(() => stageBundledProviderOptionalDependencies(directory, publish, profiles), /not qualified/); + writeFileSync(file, JSON.stringify(original)); + const missing = { ...original }; delete missing.optionalDependencies; + writeFileSync(file, JSON.stringify(missing)); + assert.throws(() => stageBundledProviderOptionalDependencies(directory, publish, profiles), /not qualified/, + "Missing wrapper declarations require the exact published-root delegation mapping"); + writeFileSync(file, JSON.stringify(original)); + assert.throws(() => stageBundledProviderOptionalDependencies(directory, { ...publish, + optionalDependencies: { "@openai/codex-darwin-arm64": "npm:@openai/codex@0.156.1-darwin-arm64" } }, profiles), /conflicts with published/); + assert.equal(existsSync(producer), true); + assert.equal(readFileSync(file, "utf8"), JSON.stringify(original), "Conflicting root declarations must not normalize metadata"); +}); + +test("Codex stripping rejects linked native payloads and wrapper fallback directories without touching external files", (t) => { + const { directory, runtime, producer, profiles, publish, writePackage } = codexFixture(t); + const outside = writePackage(join(directory, "outside"), { name: "@openai/codex", version: "0.160.0-linux-x64", os: ["linux"], cpu: ["x64"] }); + rmSync(producer, { recursive: true }); symlinkSync(outside, producer, "dir"); + assert.throws(() => stageBundledProviderOptionalDependencies(directory, publish, profiles), /escapes its producer graph/); + rmSync(producer); rmSync(join(runtime, "vendor"), { recursive: true }); symlinkSync(outside, join(runtime, "vendor"), "dir"); + assert.throws(() => stageBundledProviderOptionalDependencies(directory, publish, profiles), /escapes its producer graph/); + assert.equal(existsSync(join(outside, "package.json")), true); +}); + +test("Linux-produced esbuild bundles publish exact consumer platform dependencies and retain their hooks", (t) => { + const { directory, graph, esbuild, producerBinary, publish, writePackage } = esbuildFixture(t); + const original = readFileSync(join(esbuild, "package.json"), "utf8"); + const issuer = createRequire(join(esbuild, "package.json")); + // The original producer-only topology cannot resolve a Mac binary. The + // public optional declaration lets npm obtain it before offline hooks run. + assert.throws(() => issuer.resolve("@esbuild/darwin-arm64/bin/esbuild"), /Cannot find module/); + const staged = stageBundledEsbuildOptionalDependencies(directory, publish); + for (const target of [...nativeTargets, "win32-x64"]) assert.equal(staged.optionalDependencies[`@esbuild/${target}`], "0.28.2"); + assert.equal(staged.optionalDependencies.unrelated, "1.0.0"); + assert.deepEqual(staged.bundleDependencies, ["acpx"]); + assert.deepEqual(staged.dependencies, publish.dependencies); + assert.equal(publish.optionalDependencies["@esbuild/darwin-arm64"], undefined); + assert.equal(existsSync(producerBinary), false); + assert.equal(existsSync(join(graph, "unrelated-linux-x64")), true); + assert.equal(readFileSync(join(esbuild, "package.json"), "utf8"), original); + assert.deepEqual(stageBundledEsbuildOptionalDependencies(directory, staged), staged); + for (const target of ["darwin-arm64", "darwin-x64"]) { + const artifact = writePackage(join(graph, `@esbuild/${target}`), { + name: `@esbuild/${target}`, version: staged.optionalDependencies[`@esbuild/${target}`], os: ["darwin"], cpu: [target.slice(7)], + }); + mkdirSync(join(artifact, "bin")); writeFileSync(join(artifact, "bin/esbuild"), "consumer-selected fixture; never executed"); + assert.equal(issuer.resolve(`@esbuild/${target}/bin/esbuild`), join(artifact, "bin/esbuild")); + } +}); + +test("bundled esbuild rejects conflicting published or nested platform versions without removing the producer", (t) => { + const { directory, graph, producerBinary, publish, writePackage } = esbuildFixture(t); + assert.throws(() => stageBundledEsbuildOptionalDependencies(directory, { + ...publish, optionalDependencies: { "@esbuild/darwin-arm64": "0.27.0" }, + }), /platform version conflicts/); + assert.equal(existsSync(producerBinary), true); + writePackage(join(graph, "esbuild"), { name: "esbuild", version: "0.27.0", + optionalDependencies: Object.fromEntries(nativeTargets.map(target => [`@esbuild/${target}`, "0.27.0"])) }); + assert.throws(() => stageBundledEsbuildOptionalDependencies(directory, publish), /platform version conflicts|installed platform identity mismatch/); + assert.equal(existsSync(producerBinary), true); +}); + +test("bundled esbuild rejects missing targets, altered declarations and incorrect platform identity", (t) => { + const { directory, esbuild, producerBinary, publish } = esbuildFixture(t); + const path = join(esbuild, "package.json"), metadata = JSON.parse(readFileSync(path, "utf8")); + writeFileSync(path, JSON.stringify({ ...metadata, optionalDependencies: { "@esbuild/linux-x64": "0.28.2" } })); + assert.throws(() => stageBundledEsbuildOptionalDependencies(directory, publish), /omitted a supported platform/); + writeFileSync(path, JSON.stringify({ ...metadata, optionalDependencies: { ...metadata.optionalDependencies, "@esbuild/win32-x64": "^0.28.2" } })); + assert.throws(() => stageBundledEsbuildOptionalDependencies(directory, publish), /exact package version/); + writeFileSync(path, JSON.stringify(metadata)); + writeFileSync(join(producerBinary, "package.json"), JSON.stringify({ name: "@esbuild/linux-x64", version: "0.27.0" })); + assert.throws(() => stageBundledEsbuildOptionalDependencies(directory, publish), /installed platform identity mismatch/); + assert.equal(existsSync(producerBinary), true); +}); + +test("bundled esbuild cannot follow a platform payload or package namespace outside its producer graph", (t) => { + const { directory, graph, producerBinary, publish, writePackage } = esbuildFixture(t); + const external = writePackage(join(directory, "outside"), { name: "@esbuild/linux-x64", version: "0.28.2" }); + rmSync(producerBinary, { recursive: true }); symlinkSync(external, producerBinary, "dir"); + assert.throws(() => stageBundledEsbuildOptionalDependencies(directory, publish), /escapes its producer graph/); + assert.equal(readFileSync(join(external, "package.json"), "utf8").includes("0.28.2"), true); + rmSync(join(graph, "@esbuild"), { recursive: true }); symlinkSync(external, join(graph, "@esbuild"), "dir"); + assert.throws(() => stageBundledEsbuildOptionalDependencies(directory, publish), /escapes its producer graph/); +}); + test("Paperclip Runner pins the qualified ACPX host callbacks", () => { assert.equal(rootPackage.pnpm.patchedDependencies["acpx@0.13.1"], "patches/acpx@0.13.1.patch"); assert.equal( @@ -285,11 +480,13 @@ test("bundled package patch selection rejects an unpatched installed version", ( }); test("server package staging applies every bundled runtime patch and preserves the vendored runner", (t) => { - const fixtureDir = mkdtempSync(join(tmpdir(), "paperclip-bundled-stage-")); + const fixtureDir = realpathSync(mkdtempSync(join(tmpdir(), "paperclip-bundled-stage-"))); const sourceDir = join(fixtureDir, "source"); const destinationDir = join(fixtureDir, "destination"); const binDir = join(fixtureDir, "bin"); const callLog = join(fixtureDir, "calls.log"); + const fixtureSourceRoot = join(fixtureDir, "repo"); + const fixtureArtifacts = join(fixtureDir, "native-artifacts"); mkdirSync(sourceDir); mkdirSync(join(sourceDir, "dist")); writeFileSync(join(sourceDir, "dist", "index.js"), "export {};\n"); @@ -300,6 +497,18 @@ test("server package staging applies every bundled runtime patch and preserves t JSON.stringify({ ...serverPackage, files: ["dist"] }), ); writeFileSync(callLog, ""); + mkdirSync(fixtureArtifacts); + const nativePackage = join(fixtureArtifacts, "@openai/codex-linux-x64"); + mkdirSync(join(nativePackage, "vendor"), { recursive: true }); + writeFileSync(join(nativePackage, "package.json"), JSON.stringify({ name: "@openai/codex", version: "0.160.0-linux-x64", os: ["linux"], cpu: ["x64"] })); + writeFileSync(join(nativePackage, "vendor/codex"), "producer host fixture; never executed"); + mkdirSync(join(fixtureSourceRoot, "packages/paperclip-runner/src/drivers/acpx"), { recursive: true }); + writeFileSync(join(fixtureSourceRoot, "package.json"), JSON.stringify(rootPackage)); + writeFileSync(join(fixtureSourceRoot, "packages/paperclip-runner/acpx-profiles.json"), JSON.stringify(profileData)); + mkdirSync(join(fixtureSourceRoot, "patches")); + for (const patchPath of Object.values(rootPackage.pnpm.patchedDependencies)) { + writeFileSync(join(fixtureSourceRoot, patchPath), readFileSync(new URL(`../${patchPath}`, import.meta.url))); + } t.after(() => rmSync(fixtureDir, { recursive: true, force: true })); const writeExecutable = (name, body) => { @@ -321,8 +530,10 @@ mkdir -p "$destination/node_modules/.pnpm" set -euo pipefail printf 'npm %s\\n' "$*" >> "$FAKE_CALL_LOG" [ "$*" = "install --omit=dev --ignore-scripts --no-audit --no-fund" ] -node -e 'const fs = require("node:fs"); const pkg = require("./package.json"); if ("devDependencies" in pkg) process.exit(1); for (const [name, version] of Object.entries(pkg.dependencies)) { const dir = "node_modules/" + name; fs.mkdirSync(dir + "/dist", { recursive: true }); fs.writeFileSync(dir + "/package.json", JSON.stringify({ name, version })); }' +node -e 'const fs = require("node:fs"); const pkg = require("./package.json"); if ("devDependencies" in pkg) process.exit(1); for (const [name, version] of Object.entries(pkg.dependencies)) { const dir = "node_modules/" + name; fs.mkdirSync(dir + "/dist", { recursive: true }); fs.writeFileSync(dir + "/package.json", JSON.stringify({ name, version })); } for (const [bridge, runtimes] of Object.entries(pkg.overrides ?? {})) { for (const [name, version] of Object.entries(runtimes)) { const dir = "node_modules/" + name; fs.mkdirSync(dir, { recursive: true }); const native = name === "@openai/codex"; const optionalDependencies = native ? Object.fromEntries(["linux-x64","linux-arm64","darwin-x64","darwin-arm64","win32-x64","win32-arm64"].map(target => [name+"-"+target,"npm:"+name+"@"+version+"-"+target])) : undefined; fs.writeFileSync(dir + "/package.json", JSON.stringify({ name, version, optionalDependencies })); } }' +node -e 'const fs=require("node:fs");fs.cpSync(process.env.FAKE_NATIVE_ARTIFACTS+"/@openai/codex-linux-x64","node_modules/@openai/codex-linux-x64",{recursive:true});fs.mkdirSync("node_modules/@openai/codex/vendor",{recursive:true});fs.writeFileSync("node_modules/@openai/codex/vendor/codex","producer fallback fixture");' mkdir -p node_modules/acpx/dist +node -e 'const fs=require("node:fs");fs.mkdirSync("node_modules/esbuild",{recursive:true});fs.writeFileSync("node_modules/esbuild/package.json",JSON.stringify({name:"esbuild",version:"0.28.2",scripts:{postinstall:"node install.js"},optionalDependencies:Object.fromEntries(["linux-x64","darwin-arm64","darwin-x64"].map(target=>["@esbuild/"+target,"0.28.2"]))}));fs.mkdirSync("node_modules/@esbuild/linux-x64",{recursive:true});fs.writeFileSync("node_modules/@esbuild/linux-x64/package.json",JSON.stringify({name:"@esbuild/linux-x64",version:"0.28.2",os:["linux"],cpu:["x64"]}));' printf 'unpatched runtime\\n' > node_modules/acpx/dist/runtime.js printf '{"name":"acpx","version":"0.13.1"}\\n' > node_modules/acpx/package.json `, @@ -356,9 +567,11 @@ printf 'patched spawnEnvironment runtime\\n' > "$target/dist/runtime.js" execFileSync( process.execPath, [ - new URL("./prepare-bundled-package.mjs", import.meta.url).pathname, + "--input-type=module", "-e", + `import { prepareBundledPackage } from ${JSON.stringify(new URL("./prepare-bundled-package.mjs", import.meta.url).href)}; prepareBundledPackage(process.argv[1], process.argv[2], { sourceRoot: process.argv[3] });`, sourceDir, destinationDir, + fixtureSourceRoot, ], { env: { @@ -366,6 +579,7 @@ printf 'patched spawnEnvironment runtime\\n' > "$target/dist/runtime.js" PATH: `${binDir}:${process.env.PATH}`, FAKE_CALL_LOG: callLog, FAKE_SOURCE_PACKAGE: join(sourceDir, "package.json"), + FAKE_NATIVE_ARTIFACTS: fixtureArtifacts, }, stdio: "pipe", }, @@ -375,6 +589,19 @@ printf 'patched spawnEnvironment runtime\\n' > "$target/dist/runtime.js" assert.equal(lstatSync(stagedAcpxDir).isDirectory(), true); assert.equal(lstatSync(stagedAcpxDir).isSymbolicLink(), false); assert.equal(existsSync(join(destinationDir, "node_modules/.pnpm")), false); + const stagedManifest = JSON.parse(readFileSync(join(destinationDir, "package.json"), "utf8")); + for (const [name, version] of Object.entries(codexOptional)) { + assert.equal(stagedManifest.optionalDependencies[name], version); + assert.equal(existsSync(join(destinationDir, "node_modules", name)), false); + assert.equal(stagedManifest.bundleDependencies.includes(name), false); + } + assert.equal(existsSync(join(destinationDir, "node_modules/@openai/codex/vendor")), false); + assert.equal(stagedManifest.paperclipProviderArtifacts, undefined); + assert.equal(JSON.parse(readFileSync(join(destinationDir, "node_modules/@openai/codex/package.json"), "utf8")).optionalDependencies, undefined); + assert.equal(stagedManifest.optionalDependencies["@esbuild/darwin-arm64"], "0.28.2"); + assert.equal(stagedManifest.optionalDependencies["@esbuild/darwin-x64"], "0.28.2"); + assert.equal(existsSync(join(destinationDir, "node_modules/@esbuild/linux-x64")), false); + assert.equal(JSON.parse(readFileSync(join(destinationDir, "node_modules/esbuild/package.json"))).scripts.postinstall, "node install.js"); assert.match( readFileSync(join(stagedAcpxDir, "dist/runtime.js"), "utf8"), /spawnEnvironment/, @@ -383,13 +610,14 @@ printf 'patched spawnEnvironment runtime\\n' > "$target/dist/runtime.js" readFileSync(callLog, "utf8"), /patch -p1 --forward -d .*node_modules\/acpx/, ); + const patchedBundledDependencies = serverPackage.bundleDependencies.filter(name => rootPackage.pnpm.patchedDependencies[`${name}@${serverPackage.dependencies[name]}`]); assert.equal( readFileSync(callLog, "utf8") .split("\n") .filter((line) => line.startsWith("patch ")).length, - serverPackage.bundleDependencies.length, + patchedBundledDependencies.length, ); - for (const name of serverPackage.bundleDependencies) { + for (const name of patchedBundledDependencies) { const specifier = `${name}@${serverPackage.dependencies[name]}`; const patchPath = rootPackage.pnpm.patchedDependencies[specifier]; assert.equal( diff --git a/scripts/grok-public-install-sandbox.mjs b/scripts/grok-public-install-sandbox.mjs index 07dc9207841..c53fc2c1902 100644 --- a/scripts/grok-public-install-sandbox.mjs +++ b/scripts/grok-public-install-sandbox.mjs @@ -1,3 +1,5 @@ +import assert from 'node:assert/strict'; + // Keep public-package lifecycle code off the verification host. Resolve and // cache the public npm graph without scripts, then execute it offline. export const GROK_PUBLIC_INSTALL_IMAGE = @@ -7,6 +9,148 @@ export const GROK_PUBLIC_INSTALL_LIFECYCLE = [ 'npm', 'rebuild', '--offline', '--ignore-scripts=false', '--dangerously-allow-all-scripts', ]; +// Inspect the actual npm archive, independently of the staging helper. +export function assertNoBundledCodexPayloads(entries) { + assert.ok(Array.isArray(entries) && entries.length > 0, 'Packed package must have tar members'); + for (const entry of entries) { + assert.equal(typeof entry, 'string'); + assert.ok(!entry.startsWith('/') && !entry.split('/').includes('..'), 'Tar member must remain inside the package'); + assert.doesNotMatch(entry, /(?:^|\/)node_modules\/@openai\/codex-(?:linux|darwin|win32)-(?:x64|arm64)(?:\/|$)/, + 'Paperclip tarballs must not bundle Codex platform packages'); + assert.doesNotMatch(entry, /(?:^|\/)node_modules\/@openai\/codex\/vendor(?:\/|$)/, + 'Paperclip tarballs must not bundle the Codex wrapper vendor payload'); + if (!entry.endsWith('/')) assert.doesNotMatch(entry, /(?:^|\/)codex(?:\.exe)?$/, + 'Paperclip tarballs must not bundle native Codex executables'); + } +} + +// Verify every host payload against its own official npm provenance, including +// independent legacy closures. The selected native runtime is pinned below. +export function assertInstalledCodexPlatformProvenance(lock, installed, target) { + assert.match(target, /^(linux|darwin|win32)-(x64|arm64)$/); + assert.ok(Array.isArray(installed) && installed.length > 0, 'Consumer must install its host Codex platform package'); + const alias = `@openai/codex-${target}`; + for (const { path, manifest } of installed) { + assert.ok(path.endsWith(`/node_modules/${alias}`) || path === `node_modules/${alias}`, + 'Consumer must contain only the host Codex platform payload'); + const version = manifest.version; + assert.ok(typeof version === 'string' && version.endsWith('-' + target) + && /^\d+\.\d+\.\d+$/.test(version.slice(0, -target.length - 1)), 'Installed Codex platform must declare an exact host version'); + assert.ok([alias, '@openai/codex'].includes(manifest.name), 'Installed Codex platform package identity must match its alias'); + const [os, cpu] = target.split('-'); + assert.deepEqual(manifest.os, [os]); + assert.deepEqual(manifest.cpu, [cpu]); + const entry = lock.packages?.[path]; + assert.ok(entry, 'Installed Codex platform must have a consumer lock entry'); + assert.notEqual(entry.inBundle, true, 'Codex platform package must be downloaded, not bundled'); + assert.notEqual(entry.link, true, 'Codex platform package must not be a local link'); + assert.equal(entry.version, version, 'Consumer lock must pin the host Codex platform version'); + const url = new URL(entry.resolved); + assert.equal(url.protocol, 'https:'); + assert.equal(url.host, 'registry.npmjs.org', 'Codex platform must resolve from official npm'); + assert.equal(url.username + url.password + url.search + url.hash, '', 'Codex npm URL must not have credentials or modifiers'); + assert.equal(decodeURIComponent(url.pathname), `/@openai/codex/-/codex-${version}.tgz`, 'Codex platform must use its exact official npm tarball'); + assert.match(entry.integrity ?? '', /^sha512-[A-Za-z0-9+/]+={0,2}$/, 'Codex platform must have npm integrity'); + assert.equal(Buffer.from(entry.integrity.slice(7), 'base64').length, 64, 'Codex npm integrity must be a SHA-512 digest'); + } + return { codexConsumerHostOnly: true, codexConsumerPlatform: target, codexPlatformPackageSource: 'official npm', + codexPlatformPackages: installed.map(({ path, manifest }) => ({ path, name: manifest.name, version: manifest.version, + resolved: lock.packages[path].resolved, integrity: lock.packages[path].integrity, inBundle: false })) }; +} + +// Execute this in the existing offline public consumer, never in the checkout. +// The public index and returned command must both come from the installed graph. +export function installedCodexProbeSource(indexPath, consumerRoot, expectedVersion) { + return ` + import assert from 'node:assert/strict'; + import { execFileSync } from 'node:child_process'; + import { createRequire } from 'node:module'; + import { accessSync, constants, existsSync, lstatSync, readFileSync, readdirSync, realpathSync, statSync } from 'node:fs'; + import { dirname, isAbsolute, join, relative, resolve, sep } from 'node:path'; + import { pathToFileURL } from 'node:url'; + const root = realpathSync(${JSON.stringify(consumerRoot)}); + const contained = path => { const value = relative(root, realpathSync(path)); return value !== '' && value !== '..' && !value.startsWith('..' + sep) && !isAbsolute(value); }; + const lock = JSON.parse(readFileSync(join(root, 'package-lock.json'), 'utf8')); + const installed = [], pending = [join(root, 'node_modules')]; + let visited = 0; + while (pending.length) { + const directory = pending.pop(); + assert.ok(contained(directory), 'Installed dependency directory must remain in the consumer'); + for (const entry of readdirSync(directory, { withFileTypes: true })) { + if (entry.name.startsWith('.') || !(entry.isDirectory() || entry.isSymbolicLink())) continue; + const namespace = join(directory, entry.name); + const packages = entry.name.startsWith('@') ? readdirSync(namespace, { withFileTypes: true }) + .filter(value => value.isDirectory() || value.isSymbolicLink()).map(value => join(namespace, value.name)) : [namespace]; + for (const packageRoot of packages) { + assert.ok(++visited <= 10000 && contained(packageRoot), 'Installed dependency graph must be bounded and contained'); + const path = relative(root, packageRoot).split(sep).join('/'); + if (${/(?:^|\/)node_modules\/@openai\/codex-(?:linux|darwin|win32)-(?:x64|arm64)$/.toString()}.test(path)) { + let manifest; + try { manifest = JSON.parse(readFileSync(join(packageRoot, 'package.json'), 'utf8')); } + catch (error) { + if (error.code !== 'ENOENT') throw error; + const contents = readdirSync(packageRoot), slot = lstatSync(packageRoot); + assert.ok(slot.isDirectory() && !slot.isSymbolicLink() && contents.length === 0, + 'Codex optional slot without a manifest must be an empty regular directory; found ' + + JSON.stringify(contents.sort().slice(0, 16).map(name => name.slice(0, 256)))); + // npm can leave an empty nested optional slot while hoisting the + // actual host payload. Its real lock and manifest still qualify below. + continue; + } + installed.push({ path, manifest }); + } + const nested = join(packageRoot, 'node_modules'); + if (existsSync(nested)) pending.push(nested); + } + } + } + const provenance = (${assertInstalledCodexPlatformProvenance.toString()})(lock, installed, process.platform + '-' + process.arch); + const index = ${JSON.stringify(indexPath)}; + assert.ok(contained(index), 'Codex public index must be installed in the consumer'); + const { resolvePinnedCodexCommand } = await import(pathToFileURL(index).href); + assert.equal(typeof resolvePinnedCodexCommand, 'function', 'Installed public index must export the pinned Codex resolver'); + const command = resolvePinnedCodexCommand(); + assert.ok(isAbsolute(command) && contained(command), 'Pinned Codex must resolve inside the installed consumer'); + assert.ok(statSync(command).isFile(), 'Pinned Codex must be a regular executable'); + accessSync(command, constants.X_OK); + const selectedPlatformManifest = realpathSync(createRequire(command).resolve('@openai/codex-' + process.platform + '-' + process.arch + '/package.json')); + const selectedPlatform = installed.find(({ path }) => realpathSync(join(root, path, 'package.json')) === selectedPlatformManifest); + assert.ok(selectedPlatform, 'Codex wrapper must resolve an official host platform package from its installed graph'); + const selectedMetadata = JSON.parse(readFileSync(selectedPlatformManifest, 'utf8')); + assert.deepEqual(selectedMetadata, selectedPlatform.manifest, 'Selected Codex platform metadata must match its scanned provenance'); + assert.equal(selectedMetadata.version, ${JSON.stringify(expectedVersion)} + '-' + process.platform + '-' + process.arch, + 'Selected Codex platform version must match the qualified pin'); + const version = execFileSync(command, ['--version'], { timeout: 30_000, maxBuffer: 128 * 1024, encoding: 'utf8', + env: { PATH: '/usr/local/bin:/usr/bin:/bin', HOME: '/tmp', NODE_PATH: '' } }).trim(); + assert.equal(version, 'codex-cli ' + ${JSON.stringify(expectedVersion)}, 'Installed Codex version must match the qualified pin'); + const vendor = dirname(index); + const integrityPath = join(vendor, 'drivers/acpx/installation-integrity.js'); + const profilesPath = join(vendor, 'drivers/acpx/qualified-profiles.js'); + const securityPath = join(vendor, 'drivers/codex/codex-security-config.js'); + assert.ok(contained(integrityPath) && contained(profilesPath) && contained(securityPath), 'Codex verification modules must come from the installed consumer'); + const { codexExecutableReadOnlyRoots, createIsolatedCodexAppServerArgs } = await import(pathToFileURL(securityPath).href); + const platformRoot = realpathSync(dirname(selectedPlatformManifest)); + const nativeVendor = realpathSync(join(platformRoot, 'vendor')); + assert.ok(contained(nativeVendor) && nativeVendor.startsWith(platformRoot + sep), 'Codex vendor resources must remain inside their installed platform package'); + const sandboxSource = { HOME: '/private-provider-home', PATH: '/usr/local/bin:/usr/bin:/bin' }; + const readRoots = codexExecutableReadOnlyRoots(sandboxSource, command); + assert.ok(readRoots.includes(nativeVendor), 'Selected native Codex vendor directory must be readable inside its sandbox'); + assert.ok(!readRoots.includes(platformRoot) && !readRoots.includes(root) && !readRoots.includes(sandboxSource.HOME), 'Codex sandbox must not grant enclosing npm or credential-home access'); + const sandboxArgs = createIsolatedCodexAppServerArgs(sandboxSource, readRoots).join('\\n'); + assert.ok(sandboxArgs.includes(JSON.stringify(nativeVendor) + '="read"'), 'Codex sandbox configuration must retain the selected native executable resources'); + const { createAcpxPackageJsonResolver, verifyQualifiedAcpxInstallation } = await import(pathToFileURL(integrityPath).href); + const { resolveQualifiedAcpxProfile } = await import(pathToFileURL(profilesPath).href); + const serverRoot = resolve(vendor, '../../..'); + assert.ok(contained(serverRoot), 'Codex package authority must be the installed server'); + const installation = await verifyQualifiedAcpxInstallation(resolveQualifiedAcpxProfile('codex', 'gpt-5.4'), + createAcpxPackageJsonResolver(serverRoot, join(serverRoot, 'package.json'))); + const lease = await installation.openCommand(); + await lease.close(); + console.log(JSON.stringify({ pinnedCodexCommandVerified: true, pinnedCodexVersion: ${JSON.stringify(expectedVersion)}, + ...provenance, codexSelectedPlatformPath: selectedPlatform.path, codexSelectedPlatformVersion: selectedMetadata.version, codexNativeSandboxResourcesVerified: true, codexQualifiedInstallationVerified: true, codexCommandLeaseVerified: true, providerCalls: 0 })); + `; +} + export function grokConsumerDockerArgs({ assets, consumer, cache, command, uid, gid, download = false, prerequisite, temporarySizeMb = 256 }) { if (!Number.isSafeInteger(uid) || uid <= 0 || !Number.isSafeInteger(gid) || gid <= 0) { throw new Error('Public-install verification requires an unprivileged host user'); diff --git a/scripts/prepare-bundled-package.mjs b/scripts/prepare-bundled-package.mjs index aeb2bef0c5d..cdc03387905 100644 --- a/scripts/prepare-bundled-package.mjs +++ b/scripts/prepare-bundled-package.mjs @@ -1,11 +1,13 @@ #!/usr/bin/env node import { execFileSync } from "node:child_process"; -import { cpSync, existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"; -import { resolve } from "node:path"; +import { cpSync, existsSync, lstatSync, mkdirSync, readFileSync, readdirSync, realpathSync, rmSync, writeFileSync } from "node:fs"; +import { createRequire } from "node:module"; +import { basename, dirname, isAbsolute, relative, resolve, sep } from "node:path"; import { fileURLToPath } from "node:url"; const repoRoot = resolve(fileURLToPath(new URL("..", import.meta.url))); +const nativeTargets = ["linux-x64", "darwin-arm64", "darwin-x64"]; export function materializePublishManifest(pkg) { const publishConfig = pkg.publishConfig ?? {}; @@ -48,6 +50,222 @@ export function createBundledInstallManifest(publishManifest, bundledDependencie return installManifest; } +// npm consumers cannot inherit the workspace's pnpm overrides or patches. +// Materialize the already-qualified Codex bridge closure in the tarball, +// with overrides confined to the temporary, scripts-disabled producer graph. +export function configureBundledProviderOverrides(installManifest, bundledDependencies, rootPackage, profileData) { + const result = structuredClone(installManifest); + const selected = []; + for (const [agent, serverPackage, runtimePackage] of [ + ["codex", "@agentclientprotocol/codex-acp", "@openai/codex"], + ]) { + if (!bundledDependencies.includes(serverPackage)) continue; + const profile = profileData?.profiles?.[agent]; + if (profileData?.schema !== "paperclip.acpx-profiles.v1" || profile?.agentServerPackage !== serverPackage + || profile.agentRuntimePackage !== runtimePackage || !/^\d+\.\d+\.\d+$/.test(profile.agentServerVersion ?? "") + || !/^\d+\.\d+\.\d+$/.test(profile.agentRuntimeVersion ?? "") + || result.dependencies?.[serverPackage] !== profile.agentServerVersion) { + throw new Error(`Bundled ${agent} bridge must match its exact qualified profile`); + } + const selector = `${serverPackage}@${profile.agentServerVersion}>${runtimePackage}`; + if (rootPackage.pnpm?.overrides?.[selector] !== profile.agentRuntimeVersion) { + throw new Error(`Bundled ${agent} runtime override must match its qualified profile`); + } + const npmSelector = `${serverPackage}@${profile.agentServerVersion}`; + if (result.overrides?.[npmSelector] !== undefined) { + throw new Error(`Bundled ${agent} runtime has a conflicting producer override`); + } + const overrides = { [runtimePackage]: profile.agentRuntimeVersion }; + result.overrides = { ...result.overrides, [npmSelector]: overrides }; + selected.push({ agent, ...profile }); + } + return { installManifest: result, profiles: selected }; +} + +const inside = (root, candidate) => { + const value = relative(root, candidate); + return value !== "" && value !== ".." && !value.startsWith(`..${sep}`) && !isAbsolute(value); +}; + +// Bundle the pinned JavaScript closure, never the producer's Codex executable. +// npm installs the official optional package for the consumer's own platform. +export function stageBundledProviderOptionalDependencies(destinationDir, publishManifest, profiles) { + const graphRoot = resolve(destinationDir, "node_modules"); + if (realpathSync(graphRoot) !== graphRoot) throw new Error("Bundled Codex graph must be a canonical owned directory"); + const result = structuredClone(publishManifest), remove = new Set(), optional = {}, normalize = new Map(); + const profile = profiles.find(value => value.agent === "codex"); + if (profiles.length !== 1 || !profile) throw new Error("Bundled provider graph must contain its qualified Codex profile"); + const targets = ["linux-x64", "linux-arm64", "darwin-x64", "darwin-arm64", "win32-x64", "win32-arm64"]; + const declarations = Object.fromEntries(targets.map(target => [`${profile.agentRuntimePackage}-${target}`, + `npm:${profile.agentRuntimePackage}@${profile.agentRuntimeVersion}-${target}`])); + const ownedDirectory = directory => { + const stat = lstatSync(directory); + if (!stat.isDirectory() || stat.isSymbolicLink() || !inside(graphRoot, realpathSync(directory))) { + throw new Error("Bundled Codex dependency escapes its producer graph"); + } + }; + const manifestAt = directory => { + const path = resolve(directory, "package.json"), stat = lstatSync(path); + if (!stat.isFile() || stat.isSymbolicLink() || stat.nlink !== 1 || stat.size > 256 * 1024) { + throw new Error("Bundled Codex dependency manifest must be a bounded regular file"); + } + return JSON.parse(readFileSync(path, "utf8")); + }; + const bridgeDirectory = resolve(graphRoot, profile.agentServerPackage); + ownedDirectory(bridgeDirectory); + const bridge = manifestAt(bridgeDirectory); + if (bridge.name !== profile.agentServerPackage || bridge.version !== profile.agentServerVersion) throw new Error("Bundled Codex bridge version mismatch"); + const issuer = createRequire(resolve(bridgeDirectory, "package.json")); + const runtimeManifest = realpathSync(issuer.resolve(`${profile.agentRuntimePackage}/package.json`)); + if (!inside(graphRoot, runtimeManifest)) throw new Error("Bundled Codex runtime escapes its producer graph"); + let runtimeCount = 0, packageCount = 0; + const pending = [{ directory: graphRoot, depth: 0 }]; + while (pending.length) { + const { directory, depth } = pending.pop(); + if (depth > 64) throw new Error("Bundled Codex graph exceeds its depth limit"); + const packages = []; + for (const entry of readdirSync(directory)) { + if (entry.startsWith(".")) continue; + const candidate = resolve(directory, entry); ownedDirectory(candidate); + if (entry.startsWith("@")) { + for (const child of readdirSync(candidate)) { + const scoped = resolve(candidate, child); ownedDirectory(scoped); packages.push(scoped); + } + } else packages.push(candidate); + } + for (const packageDirectory of packages) { + if (++packageCount > 20_000) throw new Error("Bundled Codex graph exceeds its package limit"); + const metadata = manifestAt(packageDirectory); + const binding = basename(dirname(packageDirectory)) === "@openai" ? `@openai/${basename(packageDirectory)}` : null; + if (binding?.startsWith("@openai/codex-")) { + const target = binding.slice("@openai/codex-".length), [os, cpu] = target.split("-"); + if (!Object.hasOwn(declarations, binding) || ![binding, profile.agentRuntimePackage].includes(metadata.name) + || metadata.version !== `${profile.agentRuntimeVersion}-${target}` || !metadata.os?.includes(os) || !metadata.cpu?.includes(cpu)) { + throw new Error("Bundled Codex installed platform identity mismatch"); + } + remove.add(packageDirectory); + continue; + } + const nested = resolve(packageDirectory, "node_modules"); + if (lstatExists(nested)) { ownedDirectory(nested); pending.push({ directory: nested, depth: depth + 1 }); } + if (binding !== profile.agentRuntimePackage) continue; + if (metadata.name !== profile.agentRuntimePackage || metadata.version !== profile.agentRuntimeVersion) throw new Error("Bundled Codex runtime version mismatch"); + runtimeCount++; + const delegated = metadata.optionalDependencies === undefined + && Object.entries(declarations).every(([name, specifier]) => result.optionalDependencies?.[name] === specifier); + if (!delegated && (Object.keys(metadata.optionalDependencies ?? {}).length !== targets.length + || targets.some(target => metadata.optionalDependencies?.[`${profile.agentRuntimePackage}-${target}`] !== declarations[`${profile.agentRuntimePackage}-${target}`]))) { + throw new Error("Bundled Codex platform declaration is not qualified"); + } + if (!delegated) { + const normalized = { ...metadata }; delete normalized.optionalDependencies; + normalize.set(resolve(packageDirectory, "package.json"), normalized); + } + const vendor = resolve(packageDirectory, "vendor"); + if (lstatExists(vendor)) { ownedDirectory(vendor); remove.add(vendor); } + } + } + if (!runtimeCount) throw new Error("Bundled Codex runtime omitted its platform declarations"); + for (const [name, specifier] of Object.entries(declarations)) { + if (result.optionalDependencies?.[name] !== undefined && result.optionalDependencies[name] !== specifier) throw new Error(`Bundled Codex platform conflicts with published dependency: ${name}`); + optional[name] = specifier; + } + // A bundled wrapper's optional edges make npm treat missing platform slots + // as bundled too. Delegate those exact declarations to the unbundled server + // root so normal npm can install the correct host version beside legacy deps. + // Validate the complete graph before changing metadata or deleting payloads. + for (const [path, metadata] of normalize) writeFileSync(path, `${JSON.stringify(metadata, null, 2)}\n`); + for (const directory of remove) rmSync(directory, { recursive: true, force: true }); + result.optionalDependencies = { ...result.optionalDependencies, ...optional }; + for (const field of ["bundleDependencies", "bundledDependencies"]) { + if (Array.isArray(result[field])) result[field] = result[field].filter(name => !Object.hasOwn(declarations, name)); + } + delete result.paperclipProviderArtifacts; + return result; +} + +function lstatExists(path) { + try { lstatSync(path); return true; } + catch (error) { if (error.code === "ENOENT") return false; throw error; } +} + +// A Linux-produced bundle contains esbuild's JavaScript but only the producer's +// optional executable. Expose its exact platform declarations to the consumer, +// just as embedded-postgres does below, before retaining the original hooks. +export function stageBundledEsbuildOptionalDependencies(destinationDir, publishManifest) { + const graphRoot = resolve(destinationDir, "node_modules"); + if (realpathSync(graphRoot) !== graphRoot) throw new Error("Bundled esbuild graph must be a canonical owned directory"); + const result = structuredClone(publishManifest), pending = [{ directory: graphRoot, depth: 0 }]; + const optional = {}, remove = new Set(); + let packageCount = 0; + const ownedDirectory = directory => { + const stat = lstatSync(directory); + if (!stat.isDirectory() || stat.isSymbolicLink() || !inside(graphRoot, realpathSync(directory))) { + throw new Error("Bundled esbuild dependency escapes its producer graph"); + } + }; + const manifestAt = directory => { + const file = resolve(directory, "package.json"), stat = lstatSync(file); + if (!stat.isFile() || stat.isSymbolicLink() || stat.nlink !== 1 || stat.size > 256 * 1024) { + throw new Error("Bundled esbuild dependency manifest must be a bounded regular file"); + } + return JSON.parse(readFileSync(file, "utf8")); + }; + while (pending.length) { + const { directory, depth } = pending.pop(); + if (depth > 64) throw new Error("Bundled esbuild dependency graph exceeds its depth limit"); + const packages = []; + for (const entry of readdirSync(directory)) { + if (entry.startsWith(".")) continue; + const candidate = resolve(directory, entry); + ownedDirectory(candidate); + if (entry.startsWith("@")) { + for (const child of readdirSync(candidate)) { + const scoped = resolve(candidate, child); ownedDirectory(scoped); packages.push(scoped); + } + } else packages.push(candidate); + } + for (const packageDirectory of packages) { + if (++packageCount > 20_000) throw new Error("Bundled esbuild dependency graph exceeds its package limit"); + const metadata = manifestAt(packageDirectory); + const nested = resolve(packageDirectory, "node_modules"); + if (lstatExists(nested)) { ownedDirectory(nested); pending.push({ directory: nested, depth: depth + 1 }); } + if (basename(packageDirectory) !== "esbuild") continue; + if (metadata.name !== "esbuild" || !/^\d+\.\d+\.\d+$/.test(metadata.version ?? "")) { + throw new Error("Bundled esbuild package identity mismatch"); + } + const declarations = metadata.optionalDependencies ?? {}; + if (nativeTargets.some(target => declarations[`@esbuild/${target}`] !== metadata.version)) { + throw new Error("Bundled esbuild omitted a supported platform declaration"); + } + const issuer = createRequire(resolve(packageDirectory, "package.json")); + for (const [name, version] of Object.entries(declarations)) { + if (!/^@esbuild\/[a-z0-9]+-[a-z0-9]+$/.test(name) || version !== metadata.version) { + throw new Error("Bundled esbuild platform declaration must match its exact package version"); + } + if ((optional[name] !== undefined && optional[name] !== version) + || (result.optionalDependencies?.[name] !== undefined && result.optionalDependencies[name] !== version)) { + throw new Error(`Bundled esbuild platform version conflicts with published dependency: ${name}`); + } + optional[name] = version; + for (const lookup of issuer.resolve.paths(name) ?? []) { + const candidate = resolve(lookup, name); + if (!inside(graphRoot, candidate) || !lstatExists(candidate)) continue; + ownedDirectory(candidate); + const installed = manifestAt(candidate); + if (installed.name !== name || installed.version !== version) { + throw new Error(`Bundled esbuild installed platform identity mismatch: ${name}`); + } + remove.add(candidate); + } + } + } + } + for (const directory of remove) rmSync(directory, { recursive: true, force: true }); + if (Object.keys(optional).length) result.optionalDependencies = { ...result.optionalDependencies, ...optional }; + return result; +} + function patchedDependencyPackageName(specifier) { const versionSeparator = specifier.lastIndexOf("@"); const packageNameEnd = specifier.startsWith("@") ? specifier.indexOf("/") : 0; @@ -157,7 +375,12 @@ export function prepareBundledPackage(sourceDir, destinationDir, { sourceRoot = const deployedPackagePath = resolve(destinationDir, "package.json"); const publishManifest = materializePublishManifest(sourcePackage); - const installManifest = createBundledInstallManifest(publishManifest, bundledDependencies); + const rootPackage = JSON.parse(readFileSync(resolve(sourceRoot, "package.json"), "utf8")); + const profileData = bundledDependencies.includes("@agentclientprotocol/codex-acp") + ? JSON.parse(readFileSync(resolve(sourceRoot, "packages/paperclip-runner/acpx-profiles.json"), "utf8")) : undefined; + const { installManifest, profiles } = configureBundledProviderOverrides( + createBundledInstallManifest(publishManifest, bundledDependencies), bundledDependencies, rootPackage, profileData, + ); writeFileSync(deployedPackagePath, `${JSON.stringify(installManifest, null, 2)}\n`); execFileSync( @@ -167,6 +390,7 @@ export function prepareBundledPackage(sourceDir, destinationDir, { sourceRoot = ); writeFileSync(deployedPackagePath, `${JSON.stringify(publishManifest, null, 2)}\n`); applyBundledDependencyPatches(destinationDir, bundledDependencies, sourceRoot); + if (profiles.length) writeFileSync(deployedPackagePath, `${JSON.stringify(stageBundledProviderOptionalDependencies(destinationDir, publishManifest, profiles), null, 2)}\n`); if (bundledDependencies.includes("acpx")) { const acpxPackage = JSON.parse( @@ -210,6 +434,8 @@ export function prepareBundledPackage(sourceDir, destinationDir, { sourceRoot = writeFileSync(deployedPackagePath, `${JSON.stringify(stagedPackage, null, 2)}\n`); rmSync(resolve(destinationDir, "node_modules/@embedded-postgres"), { recursive: true, force: true }); } + writeFileSync(deployedPackagePath, `${JSON.stringify(stageBundledEsbuildOptionalDependencies(destinationDir, + JSON.parse(readFileSync(deployedPackagePath, "utf8"))), null, 2)}\n`); } if (process.argv[1] === fileURLToPath(import.meta.url)) { diff --git a/scripts/verify-grok-npm-install.mjs b/scripts/verify-grok-npm-install.mjs index ee1788f9df5..e71085b64af 100644 --- a/scripts/verify-grok-npm-install.mjs +++ b/scripts/verify-grok-npm-install.mjs @@ -9,7 +9,7 @@ import { tmpdir } from 'node:os'; import { basename, dirname, join, resolve } from 'node:path'; import { fileURLToPath } from 'node:url'; import { materializePublishManifest, prepareBundledPackage } from './prepare-bundled-package.mjs'; -import { GROK_PUBLIC_INSTALL_IMAGE, GROK_PUBLIC_INSTALL_LIFECYCLE, grokConsumerDockerArgs } from './grok-public-install-sandbox.mjs'; +import { GROK_PUBLIC_INSTALL_IMAGE, GROK_PUBLIC_INSTALL_LIFECYCLE, assertNoBundledCodexPayloads, grokConsumerDockerArgs, installedCodexProbeSource } from './grok-public-install-sandbox.mjs'; const repo = resolve(dirname(fileURLToPath(import.meta.url)), '..'); assert.equal(process.platform, 'linux', 'Run this verification on disposable EC2 Linux, not a developer host'); const root = mkdtempSync(join(tmpdir(), 'paperclip-grok-public-install-')); @@ -57,7 +57,10 @@ try { } run('npm', ['pack', '--ignore-scripts', '--pack-destination', root], target); const packed = readdirSync(root).filter(f => f.endsWith('.tgz') && !tarballs.includes(join(root, f))); - assert.equal(packed.length, 1); tarballs.push(join(root, packed[0])); + assert.equal(packed.length, 1); + const tarball = join(root, packed[0]); + assertNoBundledCodexPayloads(run('tar', ['-tzf', tarball]).toString().trim().split('\n')); + tarballs.push(tarball); } const assets = join(root, 'assets'); mkdirSync(assets, { mode: 0o755 }); const consumer = join(root, 'consumer'); mkdirSync(consumer); @@ -94,6 +97,13 @@ try { assert.equal(existsSync(prerequisite), false, 'npm must not provision Grok'); const server = join(consumer, 'node_modules/@paperclipai/server'); const installed = join(server, 'dist/vendor/paperclip-runner'); + const codexVersion = JSON.parse(readFileSync(join(repo, 'packages/paperclip-runner/acpx-profiles.json'), 'utf8')).profiles.codex.agentRuntimeVersion; + assert.match(codexVersion, /^\d+\.\d+\.\d+$/); + writeFileSync(join(assets, 'codex-probe.mjs'), installedCodexProbeSource( + '/consumer/node_modules/@paperclipai/server/dist/vendor/paperclip-runner/index.js', '/consumer', codexVersion), { mode: 0o644 }); + const codex = JSON.parse(isolated(['node', '/packages/codex-probe.mjs']).toString().trim()); + assert.equal(codex.codexConsumerHostOnly, true); + assert.equal(codex.codexPlatformPackageSource, 'official npm'); assert.ok(existsSync(join(installed, 'providers/grok/launcher.cjs'))); assert.equal(existsSync(join(consumer, 'node_modules/@paperclipai/grok-acp')), false); assert.equal(existsSync(join(installed, 'providers/grok/bin')), false); @@ -114,7 +124,7 @@ try { // Only the positive probe sees this file at the canonical sandbox path. run(process.execPath, [join(repo, 'packages/paperclip-runner/scripts/provision-grok.mjs'), prerequisite]); isolated(['node', '/packages/probe.mjs', 'present'], { prerequisite }); - console.log(JSON.stringify({ schema: 'paperclip.grok.public-npm-install.v1', sourceRevision, releaseVersion, lifecycleScriptsEnabled: true, lifecycleSentinelVerified: true, lifecycleNetwork: 'none', consumerImage: GROK_PUBLIC_INSTALL_IMAGE, consumerUid, consumerLockPreserved: true, cleanNpmInstall: true, packageCount: needed.size, builtinLauncherPresent: true, separateGrokPackage: false, npmProvisionedBinary: false, missingPrerequisiteRejected: true, provisionedBinaryVerified: true, commandLeaseVerified: true, providerCalls: 0 })); + console.log(JSON.stringify({ schema: 'paperclip.grok.public-npm-install.v1', sourceRevision, releaseVersion, lifecycleScriptsEnabled: true, lifecycleSentinelVerified: true, lifecycleNetwork: 'none', consumerImage: GROK_PUBLIC_INSTALL_IMAGE, consumerUid, consumerLockPreserved: true, cleanNpmInstall: true, packageCount: needed.size, codexPackedPayloadsOmitted: true, codexPackedPackagesChecked: tarballs.length, ...codex, builtinLauncherPresent: true, separateGrokPackage: false, npmProvisionedBinary: false, missingPrerequisiteRejected: true, provisionedBinaryVerified: true, commandLeaseVerified: true, providerCalls: 0 })); } finally { rmSync(root, { recursive: true, force: true }); } diff --git a/server/package.json b/server/package.json index c96aabd922f..81cd24e5f11 100644 --- a/server/package.json +++ b/server/package.json @@ -44,6 +44,7 @@ "typecheck": "pnpm run prepare:runner-vendor && pnpm --filter @paperclipai/plugin-sdk ensure-build-deps && tsc --noEmit" }, "dependencies": { + "@agentclientprotocol/codex-acp": "1.6.2", "ai": "7.0.130", "@ai-sdk/openai": "4.0.86", "@openrouter/ai-sdk-provider": "3.1.0", @@ -107,6 +108,7 @@ "zod": "^4.4.3" }, "bundleDependencies": [ + "@agentclientprotocol/codex-acp", "@chat-adapter/discord", "@chat-adapter/github", "@chat-adapter/slack", diff --git a/server/src/services/local-ai-browser-login.test.ts b/server/src/services/local-ai-browser-login.test.ts index 410cc02d0ee..6bfc0d36365 100644 --- a/server/src/services/local-ai-browser-login.test.ts +++ b/server/src/services/local-ai-browser-login.test.ts @@ -1,13 +1,24 @@ import { afterEach, describe, expect, it, vi } from "vitest"; -import { mkdtemp, mkdir, readFile, rm, stat, writeFile } from "node:fs/promises"; +import { mkdtemp, mkdir, readFile, realpath, rm, stat, writeFile } from "node:fs/promises"; import os from "node:os"; import path from "node:path"; import { startLocalBrowserLogin } from "./local-ai-browser-login.js"; +import { resolveCodexCommand } from "../vendor/paperclip-runner/index.js"; + +vi.mock("../vendor/paperclip-runner/index.js", () => ({ + resolveCodexCommand: vi.fn(), +})); const initialPath = process.env.PATH; +const initialPlatform = Object.getOwnPropertyDescriptor(process, "platform")!; +const initialArchitecture = Object.getOwnPropertyDescriptor(process, "arch")!; let root: string | undefined; afterEach(async () => { + Object.defineProperty(process, "platform", initialPlatform); + Object.defineProperty(process, "arch", initialArchitecture); process.env.PATH = initialPath; + vi.unstubAllEnvs(); + vi.clearAllMocks(); if (root) await rm(root, { recursive: true, force: true }); root = undefined; }); @@ -17,13 +28,85 @@ async function fakeCli(name: string, source: string) { const bin = path.join(root, "bin"); await mkdir(bin, { recursive: true }); await writeFile(path.join(bin, name), `#!/bin/sh\n${source}\n`, { mode: 0o700 }); + if (name === "codex") vi.mocked(resolveCodexCommand).mockReturnValue(path.join(bin, name)); process.env.PATH = `${bin}${path.delimiter}${initialPath}`; const home = path.join(root, "credential-home"); await mkdir(home, { mode: 0o700 }); - return home; + return realpath(home); +} + +function targetPlatform(platform: string, architecture: string) { + Object.defineProperty(process, "platform", { ...initialPlatform, value: platform }); + Object.defineProperty(process, "arch", { ...initialArchitecture, value: architecture }); } describe.skipIf(process.platform === "win32")("local browser subscription login", () => { + it.each([false, true])("uses the selected Codex command with its credential home and ambient HOME/cwd (legacy Linux ARM64: %s)", async (legacy) => { + if (legacy) targetPlatform("linux", "arm64"); + const home = await fakeCli("codex", [ + '[ "$1 $2" = "login --device-auth" ] || exit 11', + '[ "$HOME" = "$CLAUDE_CONFIG_DIR" ] && [ "$HOME" != "$CODEX_HOME" ] && [ "$CODEX_HOME" = "$PAPERCLIP_TEST_LOGIN_CODEX_HOME" ] || exit 12', + '[ "$PWD" = "$PAPERCLIP_TEST_LOGIN_CWD" ] || exit 13', + '[ -z "$OPENAI_API_KEY$CODEX_API_KEY$ANTHROPIC_API_KEY$ANTHROPIC_AUTH_TOKEN$CLAUDE_CODE_OAUTH_TOKEN" ] || exit 14', + 'printf "1. Open this link in your browser and sign in to your account\\nhttps://auth.openai.com/codex/device\\n2. Enter this one-time code (expires in 15 minutes)\\nABCD-EFGHJ\\n"', + ].join("\n")); + if (legacy) vi.mocked(resolveCodexCommand).mockImplementation(() => { throw new Error("native artifact is unavailable"); }); + else process.env.PATH = "/usr/bin:/bin"; + const ambientHome = path.join(root!, "unrelated-host-home"); + await mkdir(ambientHome, { mode: 0o700 }); + for (const key of ["HOME", "CODEX_HOME", "CLAUDE_CONFIG_DIR"]) vi.stubEnv(key, ambientHome); + vi.stubEnv("PAPERCLIP_TEST_LOGIN_CODEX_HOME", home); + vi.stubEnv("PAPERCLIP_TEST_LOGIN_CWD", process.cwd()); + for (const key of ["OPENAI_API_KEY", "CODEX_API_KEY", "ANTHROPIC_API_KEY", "ANTHROPIC_AUTH_TOKEN", "CLAUDE_CODE_OAUTH_TOKEN"]) + vi.stubEnv(key, "unrelated-host-value"); + const login = startLocalBrowserLogin("openai", home); + await vi.waitFor(() => expect(login.outcome).toBe("success"), { timeout: 5000 }); + if (legacy) expect(resolveCodexCommand).not.toHaveBeenCalled(); + else expect(resolveCodexCommand).toHaveBeenCalledOnce(); + expect(login.code).toBe("ABCD-EFGHJ"); + }); + + it.each([["linux", "x64"], ["darwin", "arm64"], ["darwin", "x64"]] as const)("reports an unavailable Codex CLI on %s/%s without revealing private details", async (platform, architecture) => { + targetPlatform(platform, architecture); + const home = await fakeCli("codex", 'echo invoked > "$CODEX_HOME/unreviewed-cli"'); + const failure = new Error("private-installation-path-or-provider-output"); + vi.mocked(resolveCodexCommand).mockImplementation(() => { throw failure; }); + const login = startLocalBrowserLogin("openai", home); + await vi.waitFor(() => expect(login.outcome).toBe("failure"), { timeout: 5000 }); + expect(login.error).toMatch(/requires an installed Codex CLI.*Install Codex/); + expect(JSON.stringify(login)).not.toContain(failure.message); + await expect(readFile(path.join(home, "unreviewed-cli"))).rejects.toMatchObject({ code: "ENOENT" }); + expect(login.authorizationUrl).toBeUndefined(); + }); + + it("reports a missing Linux ARM64 Codex CLI without revealing the host PATH", async () => { + targetPlatform("linux", "arm64"); + const home = await fakeCli("codex", "exit 0"); + process.env.PATH = "/usr/bin:/bin"; + const login = startLocalBrowserLogin("openai", home); + await vi.waitFor(() => expect(login.outcome).toBe("failure"), { timeout: 5000 }); + expect(login.error).toMatch(/requires the Codex CLI.*Install Codex.*Paperclip's PATH/); + expect(JSON.stringify(login)).not.toContain(process.env.PATH); + expect(resolveCodexCommand).not.toHaveBeenCalled(); + }); + + it("reports a missing terminal prerequisite without exposing spawn details", async () => { + const home = await fakeCli("codex", "exit 0"); + process.env.PATH = path.join(root!, "no-python"); + const login = startLocalBrowserLogin("openai", home); + await vi.waitFor(() => expect(login.outcome).toBe("failure"), { timeout: 5000 }); + expect(login.error).toMatch(/requires Python 3.*Install Python 3/); + expect(JSON.stringify(login)).not.toContain(process.env.PATH); + }); + + it("cancels before launching the resolved Codex command", async () => { + const home = await fakeCli("codex", 'echo invoked > "$CODEX_HOME/cancelled-cli"'); + const login = startLocalBrowserLogin("openai", home); + login.abort(); + await vi.waitFor(() => expect(login.outcome).toBe("failure"), { timeout: 5000 }); + await expect(readFile(path.join(home, "cancelled-cli"))).rejects.toMatchObject({ code: "ENOENT" }); + }); + it("surfaces a Codex device link and code from a local PTY without a user shell command", async () => { const home = await fakeCli("codex", [ 'printf "1. Open this link in your browser and sign in to your account\\n"', diff --git a/server/src/services/local-ai-browser-login.ts b/server/src/services/local-ai-browser-login.ts index ad3ee6dfe7a..ad176b8618f 100644 --- a/server/src/services/local-ai-browser-login.ts +++ b/server/src/services/local-ai-browser-login.ts @@ -4,11 +4,14 @@ import { constants } from "node:fs"; import path from "node:path"; import { runSetupTokenLogin } from "@paperclipai/adapter-claude-local/server"; import { runDeviceLogin } from "@paperclipai/adapter-codex-local/server"; +import { ensureCommandResolvable, resolveCommandForLogs } from "@paperclipai/adapter-utils/server-utils"; +import { resolveCodexCommand } from "../vendor/paperclip-runner/index.js"; export type LocalBrowserLoginState = { authorizationUrl?: string; code?: string; outcome?: "success" | "failure"; + error?: string; submitCode?: (code: string) => void; abort: () => void; }; @@ -47,11 +50,31 @@ sys.exit(os.waitstatus_to_exitcode(status)) export function startLocalBrowserLogin(provider: "anthropic" | "openai", home: string): LocalBrowserLoginState { const controller = new AbortController(); const state: LocalBrowserLoginState = { abort: () => controller.abort() }; - const executable = provider === "anthropic" ? "claude" : "codex"; const args = provider === "anthropic" ? ["setup-token"] : ["login", "--device-auth"]; let child: ChildProcessWithoutNullStreams | null = null; const driver = { - start(_command: string, onData: (chunk: string) => void): Promise<{ exitCode: number | null }> { + async start(_command: string, onData: (chunk: string) => void): Promise<{ exitCode: number | null }> { + let executable = "claude"; + if (provider === "openai") { + // Linux ARM64 retains its supported legacy CLI path; it is not a + // qualified native runner target. Other hosts prefer the installed + // dependency, then an existing CLI when the dependency is absent. + const legacyPlatform = process.platform === "linux" && process.arch === "arm64"; + try { + if (legacyPlatform) { + await ensureCommandResolvable("codex", home, process.env); + executable = path.resolve(home, await resolveCommandForLogs("codex", home, process.env)); + } else { + executable = resolveCodexCommand(); + } + } catch { + state.error = legacyPlatform + ? "Codex browser sign-in requires the Codex CLI on this platform. Install Codex on this execution host and ensure codex is on Paperclip's PATH, then start sign-in again." + : "Codex browser sign-in requires an installed Codex CLI. Install Codex on this execution host or restore Paperclip's runtime dependencies, then start sign-in again."; + throw new Error("Browser login runtime unavailable"); + } + } + if (controller.signal.aborted) throw new Error("Local sign-in cancelled before launch"); const env = { ...process.env, ANTHROPIC_API_KEY: "", ANTHROPIC_AUTH_TOKEN: "", CLAUDE_CODE_OAUTH_TOKEN: "", OPENAI_API_KEY: "", CODEX_API_KEY: "", @@ -64,7 +87,10 @@ export function startLocalBrowserLogin(provider: "anthropic" | "openai", home: s child.stdout.on("data", (bytes: Buffer) => onData(bytes.toString("utf8"))); // Provider output may contain secrets. Never log or retain stderr. child.stderr.resume(); - child.once("error", reject); + child.once("error", () => { + state.error = "Browser sign-in requires Python 3 and a working local terminal. Install Python 3 on this execution host, then start sign-in again."; + reject(new Error("Browser login terminal unavailable")); + }); child.once("close", (exitCode) => resolve({ exitCode })); }); }, diff --git a/server/src/services/local-ai-login.ts b/server/src/services/local-ai-login.ts index 3492d47896a..71482abc4cb 100644 --- a/server/src/services/local-ai-login.ts +++ b/server/src/services/local-ai-login.ts @@ -171,7 +171,7 @@ export function localAiLoginService(db: Db) { ...(!login && intent.provider !== "xai" ? { error: "The server restarted during sign-in. Start sign-in again." } : login?.outcome === "failure" - ? { error: "Sign-in ended before the account was connected. Start sign-in again." } + ? { error: login.error ?? "Sign-in ended before the account was connected. Start sign-in again." } : {}) }; } } diff --git a/server/src/vendor/paperclip-runner/index.ts b/server/src/vendor/paperclip-runner/index.ts index bfab10f1e21..735f73a0a4b 100644 --- a/server/src/vendor/paperclip-runner/index.ts +++ b/server/src/vendor/paperclip-runner/index.ts @@ -133,6 +133,8 @@ export const QUALIFIED_ACPX_PROFILES = runner.QUALIFIED_ACPX_PROFILES; export const QUALIFIED_ACPX_VERSION = runner.QUALIFIED_ACPX_VERSION; export const CURSOR_DISTRIBUTION_PINS = runner.CURSOR_DISTRIBUTION_PINS; export const resolveSourceCodexHome = runner.resolveSourceCodexHome; +export const resolveCodexCommand = runner.resolveCodexCommand; +export const resolvePinnedCodexCommand = runner.resolvePinnedCodexCommand; export const validatePrpEvent = runner.validatePrpEvent; export const validatePrpStructuredRunResult = runner.validatePrpStructuredRunResult;