Files
PaperClipAI/packages/adapters/codex-local/src/server/codex-auth-copyback.ts
T
DottaandPaperclip 4e00818574 fix(runtime): support in-place workspace realization (#10230)
## Thinking Path

> - Paperclip is the control plane people use to coordinate AI agents
and their execution environments.
> - Environment realization decides where an agent runs and which
filesystem and toolchain are authoritative.
> - Copy-based realization is unsafe for container-anchored tasks
because absolute paths such as `/app` can point outside the synchronized
tree and task-specific binaries may be absent.
> - That mismatch can let an agent successfully verify work in a phantom
writable path while sync-back silently discards the result.
> - Existing task environments already provide the authoritative
filesystem and toolchain, so they should be executed in place rather
than copied.
> - Copy mode still needs explicit confinement rules so aliases target
the synchronized workspace and unsynchronized writable paths fail
visibly.
> - This pull request adds typed realization metadata, propagates the
authoritative root through orchestration, and teaches Codex to honor it.
> - The benefit is that container-anchored tasks operate on
verifier-visible state with the intended tools, while copy mode remains
safe and backward compatible.

## Linked Issues or Issue Description

No public GitHub issue exists for this defect.

GitHub duplicate searches for in-place execution, workspace realization,
and authoritative workspace roots found no related pull request to link.

### What happened?

Environment-backed agent runs were always realized through a copied
workspace. Tasks anchored to absolute container paths could therefore
write outside the synchronized tree, and task-provided toolchains were
unavailable in the copy. A run could report success even though
sync-back discarded its output.

### Expected behavior

Existing task environments should run against their real authoritative
root and toolchain. Copy-mode runs should map declared absolute aliases
into the synchronized tree and reject writable paths that cannot be
restored.

### Steps to reproduce

1. Run a Codex task environment whose required files live under `/app`
or `/workspace` and whose required binary exists only in the task
container.
2. Observe that copy realization changes the effective
filesystem/toolchain or permits writes outside the synchronized root.
3. Complete and verify the task inside the agent sandbox.
4. Observe that the verifier cannot see out-of-tree artifacts or that
task-specific commands were unavailable.

### Reproduction context

- Paperclip commit: `3a16b91217483d2c233926de5b7f7bc3a1077924`
- Deployment: built from source in a task-container execution
environment
- Adapter: Codex local
- Database: not database-related
- Access context: agent execution

## What Changed

- Added typed `copy | in_place` workspace-realization metadata,
authoritative roots, confined aliases, and outbound restore paths to
shared execution-target contracts.
- Selected in-place realization for existing task environments and
skipped archive prepare/restore when the authoritative environment is
used directly.
- Propagated the authoritative root into adapter context so Codex uses
it for cwd and `PAPERCLIP_WORKSPACE_*` semantics, including ACP
execution.
- Bound copy-mode aliases such as `/app` to the synchronized workspace
and rejected writable out-of-tree paths without explicit restore
mappings.
- Added focused regression coverage while preserving existing copy-mode
archive restore behavior.

## Verification

- `pnpm --filter @paperclipai/shared typecheck` — passed.
- `pnpm exec vitest run
packages/adapter-utils/src/local-process-sandbox.test.ts
packages/adapters/codex-local/src/server/acp.test.ts
packages/adapters/codex-local/src/server/execute.remote.test.ts
server/src/__tests__/environment-run-orchestrator.test.ts` — 48 passed,
4 skipped.
- `pnpm -r typecheck` — passed.
- `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY -u
AWS_SESSION_TOKEN pnpm test:run` — passed across all general and
serialized Vitest shards.
- `pnpm build` — passed.
- Codex `k=1` acceptance run completed July 24, 2026 at 23:54:30 UTC
with 4 completed, 0 exceptions, and mean reward 1.0: `build-cython-ext`,
`openssl-selfsigned-cert`, `prove-plus-comm`, and `sqlite-db-truncate`
each received terminal grade 1.0 against real task-environment paths and
toolchains.

## Risks

- In-place mode deliberately exposes the authoritative task root to the
adapter; incorrect environment metadata could point execution at the
wrong root. Typed metadata and focused orchestration tests cover
selection and propagation.
- Copy-mode writable-path validation is stricter and may reject
previously accepted unsafe configurations. The rejection is intentional
and produces a visible error instead of silently losing output.
- The acceptance run is focused on four Codex task-environment
workloads, not a broad cross-adapter benchmark. Existing copy-mode
archive tests and the full repository suite remain green.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex CLI coding agent; exact model ID and context-window size
were not exposed to this runtime. Capabilities used: extended reasoning,
repository editing, shell execution, test/build execution, Git, GitHub
CLI, and Paperclip API tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-25 08:29:26 -05:00

155 lines
7.3 KiB
TypeScript

import { execFile as execFileCallback } from "node:child_process";
import { mkdir, open, rename, rm } from "node:fs/promises";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { randomUUID } from "node:crypto";
import { withDirectoryMergeLock } from "@paperclipai/adapter-utils/workspace-restore-merge";
const execFile = promisify(execFileCallback);
// The outbound copy-back reuses the exact same direction-agnostic decision
// predicate the inbound restore runs (`codex-auth-merge-decision.cjs`). The
// predicate answers one question — "should the caller replace `destination`
// with `source`?" — purely by argument order (first = source, second =
// destination). For the copy-back the sandbox credential is the `source` and
// the shared host credential is the `destination`, so exit 10 (use source)
// means "install the sandbox copy onto the host" and exit 20 (keep destination)
// means "leave the host copy untouched". The predicate only ever reads the two
// files and exits with a code; it never prints token bytes.
const DECISION_SCRIPT_PATH = fileURLToPath(
new URL("./codex-auth-merge-decision.cjs", import.meta.url),
);
const USE_SOURCE_EXIT = 10;
const KEEP_DESTINATION_EXIT = 20;
/** Outcome of a copy-back attempt. No token material is ever surfaced. */
export type CopyBackCodexAuthOutcome = "copied" | "kept-host";
export interface CopyBackCodexAuthInput {
/**
* Reads the sandbox `auth.json` bytes back from the (about-to-be-destroyed)
* sandbox. In production this is bound to the managed-runtime restore
* context's `readFile` for `${assetDir}/auth.json`.
*/
readSandboxAuth: () => Promise<Buffer>;
/**
* Absolute path of the shared host credential to (maybe) overwrite — the
* symlink *source* the managed Codex homes point their `auth.json` at, never
* an in-sandbox or per-agent symlink.
*/
hostAuthPath: string;
/** Non-leaking progress sink: receives decision/outcome lines only. */
log: (line: string) => void | Promise<void>;
}
async function decideExitCode(sourcePath: string, destinationPath: string): Promise<number> {
try {
await execFile("node", [DECISION_SCRIPT_PATH, sourcePath, destinationPath]);
} catch (error) {
const code = (error as { code?: unknown }).code;
if (code === USE_SOURCE_EXIT || code === KEEP_DESTINATION_EXIT) {
return code;
}
// A non-numeric `code` (e.g. "ENOENT" when node is not on PATH) or any exit
// code other than 10/20 is a hard failure — fail loud so a broken predicate
// is never mistaken for a "keep host" decision.
const detail =
typeof code === "string"
? `node could not be executed (${code})`
: typeof code === "number"
? `unexpected predicate exit code ${code}`
: error instanceof Error
? error.message
: String(error);
throw new Error(`codex auth copy-back decision predicate failed: ${detail}`);
}
// Reached only when `execFile` resolved — i.e. the predicate exited 0. The
// predicate always exits 10 or 20, so a clean exit 0 is unexpected; throw
// directly here, outside the try/catch, so this already self-explanatory
// message is not re-wrapped by the catch's "...failed:" prefix.
throw new Error("codex auth copy-back decision predicate exited 0 (expected 10 or 20)");
}
/**
* Guards, locks, and atomically installs a strictly-newer sandbox Codex
* `auth.json` onto the shared host credential at teardown.
*
* Sequence, all under `withDirectoryMergeLock` on the host target's directory
* so a concurrent inbound restore or another copy-back can't interleave:
* 1. Read the sandbox credential bytes. A genuinely absent sandbox
* `auth.json` (ENOENT) means there is simply nothing to copy back, so it
* resolves to `kept-host` (benign no-op, host untouched); every other read
* error stays fail-loud.
* 2. Stage them to a `0600` temp file on the **same filesystem** as the host
* target (its directory), which doubles as the predicate `source`.
* 3. Run the Phase-3 decision predicate (`source` = sandbox temp, `destination`
* = host). Exit 10 → adopt the sandbox copy; exit 20 → keep the host copy.
* 4. On exit 10, `rename` the staged temp over the host target — an atomic
* same-directory swap that preserves mode `0600`. On exit 20, discard it.
* The staged temp is always removed (rename consumes it on the copy path; the
* finally cleans it up otherwise), so a failure never leaves a partial file.
* Never logs token bytes — only the decision outcome.
*/
export async function copyBackCodexAuth(input: CopyBackCodexAuthInput): Promise<CopyBackCodexAuthOutcome> {
const { readSandboxAuth, hostAuthPath, log } = input;
// Read first (outside the lock) — a read never mutates the host, so there is
// nothing to serialize yet. A genuinely absent sandbox `auth.json` (ENOENT —
// e.g. Codex removed it mid-run, or a non-provisioned edge) is a "nothing to
// copy back" no-op, not a teardown failure: return `kept-host` and log the
// benign outcome. Every other read error stays fail-loud so a real read fault
// is never silently mistaken for "nothing to copy back".
let sandboxAuthBytes: Buffer;
try {
sandboxAuthBytes = await readSandboxAuth();
} catch (error) {
if ((error as NodeJS.ErrnoException | null)?.code === "ENOENT") {
await log(
"[paperclip] Codex auth copy-back: no sandbox credential to copy back (absent auth.json); host credential kept.",
);
return "kept-host";
}
throw error;
}
const hostDir = path.dirname(hostAuthPath);
await mkdir(hostDir, { recursive: true });
return withDirectoryMergeLock(hostDir, async () => {
// Stage on the same filesystem as the host target so both the predicate read
// and the final rename stay device-local (rename across devices is not
// atomic and would fail with EXDEV).
const stagedTempPath = path.join(hostDir, `.auth.json.copyback-${process.pid}-${randomUUID()}.tmp`);
// `wx` + explicit mode create the temp private (0600) and fail if it somehow
// already exists, so we never write through a pre-existing symlink.
const handle = await open(stagedTempPath, "wx", 0o600);
try {
await handle.writeFile(sandboxAuthBytes);
await handle.close();
const decision = await decideExitCode(stagedTempPath, hostAuthPath);
if (decision === USE_SOURCE_EXIT) {
// Atomic same-directory swap; rename preserves the temp's 0600 mode.
await rename(stagedTempPath, hostAuthPath);
await log(
"[paperclip] Codex auth copy-back: sandbox credential is strictly newer for the same subscription identity; installed to the host at mode 0600.",
);
return "copied";
}
await log(
"[paperclip] Codex auth copy-back: host credential kept (sandbox copy is not a strictly-newer same-identity subscription credential).",
);
return "kept-host";
} finally {
// Close is idempotent-safe to skip after an explicit close; the temp is the
// thing that must never linger. On the copy path rename already consumed it
// (force makes the removal a no-op); on every other path this deletes the
// staged credential bytes.
await handle.close().catch(() => undefined);
await rm(stagedTempPath, { force: true }).catch(() => undefined);
}
});
}