mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-24 18:19:05 +02:00
## Thinking Path > - Paperclip is the control plane people use to coordinate AI agents and their execution environments. > - Environment realization decides where an agent runs and which filesystem and toolchain are authoritative. > - Copy-based realization is unsafe for container-anchored tasks because absolute paths such as `/app` can point outside the synchronized tree and task-specific binaries may be absent. > - That mismatch can let an agent successfully verify work in a phantom writable path while sync-back silently discards the result. > - Existing task environments already provide the authoritative filesystem and toolchain, so they should be executed in place rather than copied. > - Copy mode still needs explicit confinement rules so aliases target the synchronized workspace and unsynchronized writable paths fail visibly. > - This pull request adds typed realization metadata, propagates the authoritative root through orchestration, and teaches Codex to honor it. > - The benefit is that container-anchored tasks operate on verifier-visible state with the intended tools, while copy mode remains safe and backward compatible. ## Linked Issues or Issue Description No public GitHub issue exists for this defect. GitHub duplicate searches for in-place execution, workspace realization, and authoritative workspace roots found no related pull request to link. ### What happened? Environment-backed agent runs were always realized through a copied workspace. Tasks anchored to absolute container paths could therefore write outside the synchronized tree, and task-provided toolchains were unavailable in the copy. A run could report success even though sync-back discarded its output. ### Expected behavior Existing task environments should run against their real authoritative root and toolchain. Copy-mode runs should map declared absolute aliases into the synchronized tree and reject writable paths that cannot be restored. ### Steps to reproduce 1. Run a Codex task environment whose required files live under `/app` or `/workspace` and whose required binary exists only in the task container. 2. Observe that copy realization changes the effective filesystem/toolchain or permits writes outside the synchronized root. 3. Complete and verify the task inside the agent sandbox. 4. Observe that the verifier cannot see out-of-tree artifacts or that task-specific commands were unavailable. ### Reproduction context - Paperclip commit: `3a16b91217483d2c233926de5b7f7bc3a1077924` - Deployment: built from source in a task-container execution environment - Adapter: Codex local - Database: not database-related - Access context: agent execution ## What Changed - Added typed `copy | in_place` workspace-realization metadata, authoritative roots, confined aliases, and outbound restore paths to shared execution-target contracts. - Selected in-place realization for existing task environments and skipped archive prepare/restore when the authoritative environment is used directly. - Propagated the authoritative root into adapter context so Codex uses it for cwd and `PAPERCLIP_WORKSPACE_*` semantics, including ACP execution. - Bound copy-mode aliases such as `/app` to the synchronized workspace and rejected writable out-of-tree paths without explicit restore mappings. - Added focused regression coverage while preserving existing copy-mode archive restore behavior. ## Verification - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm exec vitest run packages/adapter-utils/src/local-process-sandbox.test.ts packages/adapters/codex-local/src/server/acp.test.ts packages/adapters/codex-local/src/server/execute.remote.test.ts server/src/__tests__/environment-run-orchestrator.test.ts` — 48 passed, 4 skipped. - `pnpm -r typecheck` — passed. - `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY -u AWS_SESSION_TOKEN pnpm test:run` — passed across all general and serialized Vitest shards. - `pnpm build` — passed. - Codex `k=1` acceptance run completed July 24, 2026 at 23:54:30 UTC with 4 completed, 0 exceptions, and mean reward 1.0: `build-cython-ext`, `openssl-selfsigned-cert`, `prove-plus-comm`, and `sqlite-db-truncate` each received terminal grade 1.0 against real task-environment paths and toolchains. ## Risks - In-place mode deliberately exposes the authoritative task root to the adapter; incorrect environment metadata could point execution at the wrong root. Typed metadata and focused orchestration tests cover selection and propagation. - Copy-mode writable-path validation is stricter and may reject previously accepted unsafe configurations. The rejection is intentional and produces a visible error instead of silently losing output. - The acceptance run is focused on four Codex task-environment workloads, not a broad cross-adapter benchmark. Existing copy-mode archive tests and the full repository suite remain green. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex CLI coding agent; exact model ID and context-window size were not exposed to this runtime. Capabilities used: extended reasoning, repository editing, shell execution, test/build execution, Git, GitHub CLI, and Paperclip API tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
155 lines
7.3 KiB
TypeScript
155 lines
7.3 KiB
TypeScript
import { execFile as execFileCallback } from "node:child_process";
|
|
import { mkdir, open, rename, rm } from "node:fs/promises";
|
|
import path from "node:path";
|
|
import { fileURLToPath } from "node:url";
|
|
import { promisify } from "node:util";
|
|
import { randomUUID } from "node:crypto";
|
|
import { withDirectoryMergeLock } from "@paperclipai/adapter-utils/workspace-restore-merge";
|
|
|
|
const execFile = promisify(execFileCallback);
|
|
|
|
// The outbound copy-back reuses the exact same direction-agnostic decision
|
|
// predicate the inbound restore runs (`codex-auth-merge-decision.cjs`). The
|
|
// predicate answers one question — "should the caller replace `destination`
|
|
// with `source`?" — purely by argument order (first = source, second =
|
|
// destination). For the copy-back the sandbox credential is the `source` and
|
|
// the shared host credential is the `destination`, so exit 10 (use source)
|
|
// means "install the sandbox copy onto the host" and exit 20 (keep destination)
|
|
// means "leave the host copy untouched". The predicate only ever reads the two
|
|
// files and exits with a code; it never prints token bytes.
|
|
const DECISION_SCRIPT_PATH = fileURLToPath(
|
|
new URL("./codex-auth-merge-decision.cjs", import.meta.url),
|
|
);
|
|
const USE_SOURCE_EXIT = 10;
|
|
const KEEP_DESTINATION_EXIT = 20;
|
|
|
|
/** Outcome of a copy-back attempt. No token material is ever surfaced. */
|
|
export type CopyBackCodexAuthOutcome = "copied" | "kept-host";
|
|
|
|
export interface CopyBackCodexAuthInput {
|
|
/**
|
|
* Reads the sandbox `auth.json` bytes back from the (about-to-be-destroyed)
|
|
* sandbox. In production this is bound to the managed-runtime restore
|
|
* context's `readFile` for `${assetDir}/auth.json`.
|
|
*/
|
|
readSandboxAuth: () => Promise<Buffer>;
|
|
/**
|
|
* Absolute path of the shared host credential to (maybe) overwrite — the
|
|
* symlink *source* the managed Codex homes point their `auth.json` at, never
|
|
* an in-sandbox or per-agent symlink.
|
|
*/
|
|
hostAuthPath: string;
|
|
/** Non-leaking progress sink: receives decision/outcome lines only. */
|
|
log: (line: string) => void | Promise<void>;
|
|
}
|
|
|
|
async function decideExitCode(sourcePath: string, destinationPath: string): Promise<number> {
|
|
try {
|
|
await execFile("node", [DECISION_SCRIPT_PATH, sourcePath, destinationPath]);
|
|
} catch (error) {
|
|
const code = (error as { code?: unknown }).code;
|
|
if (code === USE_SOURCE_EXIT || code === KEEP_DESTINATION_EXIT) {
|
|
return code;
|
|
}
|
|
// A non-numeric `code` (e.g. "ENOENT" when node is not on PATH) or any exit
|
|
// code other than 10/20 is a hard failure — fail loud so a broken predicate
|
|
// is never mistaken for a "keep host" decision.
|
|
const detail =
|
|
typeof code === "string"
|
|
? `node could not be executed (${code})`
|
|
: typeof code === "number"
|
|
? `unexpected predicate exit code ${code}`
|
|
: error instanceof Error
|
|
? error.message
|
|
: String(error);
|
|
throw new Error(`codex auth copy-back decision predicate failed: ${detail}`);
|
|
}
|
|
|
|
// Reached only when `execFile` resolved — i.e. the predicate exited 0. The
|
|
// predicate always exits 10 or 20, so a clean exit 0 is unexpected; throw
|
|
// directly here, outside the try/catch, so this already self-explanatory
|
|
// message is not re-wrapped by the catch's "...failed:" prefix.
|
|
throw new Error("codex auth copy-back decision predicate exited 0 (expected 10 or 20)");
|
|
}
|
|
|
|
/**
|
|
* Guards, locks, and atomically installs a strictly-newer sandbox Codex
|
|
* `auth.json` onto the shared host credential at teardown.
|
|
*
|
|
* Sequence, all under `withDirectoryMergeLock` on the host target's directory
|
|
* so a concurrent inbound restore or another copy-back can't interleave:
|
|
* 1. Read the sandbox credential bytes. A genuinely absent sandbox
|
|
* `auth.json` (ENOENT) means there is simply nothing to copy back, so it
|
|
* resolves to `kept-host` (benign no-op, host untouched); every other read
|
|
* error stays fail-loud.
|
|
* 2. Stage them to a `0600` temp file on the **same filesystem** as the host
|
|
* target (its directory), which doubles as the predicate `source`.
|
|
* 3. Run the Phase-3 decision predicate (`source` = sandbox temp, `destination`
|
|
* = host). Exit 10 → adopt the sandbox copy; exit 20 → keep the host copy.
|
|
* 4. On exit 10, `rename` the staged temp over the host target — an atomic
|
|
* same-directory swap that preserves mode `0600`. On exit 20, discard it.
|
|
* The staged temp is always removed (rename consumes it on the copy path; the
|
|
* finally cleans it up otherwise), so a failure never leaves a partial file.
|
|
* Never logs token bytes — only the decision outcome.
|
|
*/
|
|
export async function copyBackCodexAuth(input: CopyBackCodexAuthInput): Promise<CopyBackCodexAuthOutcome> {
|
|
const { readSandboxAuth, hostAuthPath, log } = input;
|
|
|
|
// Read first (outside the lock) — a read never mutates the host, so there is
|
|
// nothing to serialize yet. A genuinely absent sandbox `auth.json` (ENOENT —
|
|
// e.g. Codex removed it mid-run, or a non-provisioned edge) is a "nothing to
|
|
// copy back" no-op, not a teardown failure: return `kept-host` and log the
|
|
// benign outcome. Every other read error stays fail-loud so a real read fault
|
|
// is never silently mistaken for "nothing to copy back".
|
|
let sandboxAuthBytes: Buffer;
|
|
try {
|
|
sandboxAuthBytes = await readSandboxAuth();
|
|
} catch (error) {
|
|
if ((error as NodeJS.ErrnoException | null)?.code === "ENOENT") {
|
|
await log(
|
|
"[paperclip] Codex auth copy-back: no sandbox credential to copy back (absent auth.json); host credential kept.",
|
|
);
|
|
return "kept-host";
|
|
}
|
|
throw error;
|
|
}
|
|
|
|
const hostDir = path.dirname(hostAuthPath);
|
|
await mkdir(hostDir, { recursive: true });
|
|
return withDirectoryMergeLock(hostDir, async () => {
|
|
// Stage on the same filesystem as the host target so both the predicate read
|
|
// and the final rename stay device-local (rename across devices is not
|
|
// atomic and would fail with EXDEV).
|
|
const stagedTempPath = path.join(hostDir, `.auth.json.copyback-${process.pid}-${randomUUID()}.tmp`);
|
|
// `wx` + explicit mode create the temp private (0600) and fail if it somehow
|
|
// already exists, so we never write through a pre-existing symlink.
|
|
const handle = await open(stagedTempPath, "wx", 0o600);
|
|
try {
|
|
await handle.writeFile(sandboxAuthBytes);
|
|
await handle.close();
|
|
|
|
const decision = await decideExitCode(stagedTempPath, hostAuthPath);
|
|
if (decision === USE_SOURCE_EXIT) {
|
|
// Atomic same-directory swap; rename preserves the temp's 0600 mode.
|
|
await rename(stagedTempPath, hostAuthPath);
|
|
await log(
|
|
"[paperclip] Codex auth copy-back: sandbox credential is strictly newer for the same subscription identity; installed to the host at mode 0600.",
|
|
);
|
|
return "copied";
|
|
}
|
|
|
|
await log(
|
|
"[paperclip] Codex auth copy-back: host credential kept (sandbox copy is not a strictly-newer same-identity subscription credential).",
|
|
);
|
|
return "kept-host";
|
|
} finally {
|
|
// Close is idempotent-safe to skip after an explicit close; the temp is the
|
|
// thing that must never linger. On the copy path rename already consumed it
|
|
// (force makes the removal a no-op); on every other path this deletes the
|
|
// staged credential bytes.
|
|
await handle.close().catch(() => undefined);
|
|
await rm(stagedTempPath, { force: true }).catch(() => undefined);
|
|
}
|
|
});
|
|
}
|