mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-29 12:59:13 +01:00
fix(native): deliver selected skills to remote Codex homes (#18080)
## Thinking Path > - Paperclip coordinates company agents and preserves their assigned instructions and approval rules. > - Native runs pin an immutable skill inventory for each execution. > - Built-in remote Codex launches prepare that inventory on the controller, but transfer only authentication, configuration, and the separate context assets. > - Codex needs the selected skills under its isolated home. An explicit skill path does not load a skill outside its registered discovery roots. > - This pull request transfers the sealed selected inventory to the path already used by explicit skill inputs. > - Each launch replaces stale assignments inside the authorized workspace while preserving provider history. ## Linked Issues or Issue Description Refs #18025. The corrected first-task instructions exist in the selected bundle, but remote Codex must receive those bytes before it can follow them. The legacy skill resolver work in #16646 and #9323 uses different paths and does not repair native remote launch staging. **What happened?** A fresh remote native Codex run could not read its assigned first-task skill. It produced an approval question instead of the required confirmation. The Product E2E journey correctly rejected that question before child creation. **Expected behavior** Codex receives the selected immutable skill at the same provider-local discovery path used by explicit skill input. Removed or unassigned skills do not remain available after a new launch. **Steps to reproduce** 1. Prepare an isolated Codex home with one assigned skill. 2. Run the built-in remote launch staging path against a fresh remote filesystem. 3. Resolve an explicit invocation of that skill. Before the repair, its provider-local file is missing. ## What Changed - Transfer only the prepared selected skill inventory into the isolated remote Codex home before other launch writes. - Reject writable files, symlinks, missing entries, and extra unassigned packages in the controller projection. - Validate each provider-home path component below the authorized workspace. Reject redirected retained paths before deleting stale skills. - Create the private discovery root before native sandbox transfers, which omit the source-root metadata and can ignore the requested directory mode. - Replace stale or empty inventories without changing provider conversation history, preconfigured homes, other providers, or plugin-provided launch preparation. - Exercise the real home materializer and turn resolver through the existing test-only Runner surface. Document the remote discovery contract. - Synchronize an existing Telegram attachment-failure test with the durable provider-effect commit, preserving its required final states. This CI repair is a separate test-only commit. ## Verification - The new production-staging regression failed before the repair because the resolved skill file was absent. - All 23 focused staging checks pass. They cover delivery, file modes, empty/revoked inventories, history retention, extra packages, writable files, source symlinks, target symlinks, redirected ancestors on sandbox/SSH, and an authorized workspace alias. The filesystem fixture packs and extracts actual tar bytes without applying the requested root mode; it preserves exact projected-file digests and executable bits. - The unchanged materializer and prepared-launch contracts passed 26 tests. Eight selected native caller checks passed; other cases in that filtered run were not selected. - Independent probes used official Codex 0.160.0, the real materializer, production staging, the real resolver, and a local model endpoint. The original staging supplied no skill message. The final staging supplied the complete selected skill body with its exact SHA-256. A final probe also ran production Daytona `performSyncIn`, backing only its SDK exec/upload/chmod edge with the local filesystem under umask `022`: the pre-review source reproduced root `0755`, and the final source retained root `0700`, package `0555`, file `0444`, and the exact selected digest. All model requests returned local 403; there were no upstream requests or credentials. - Full local `pnpm -r typecheck` and `pnpm build` passed on verification commit `e7d4e7a9`, based on `60b8089a`. The full local `pnpm test:run` on that frozen tree was not completed: later embedded PostgreSQL startup failed because the host could not allocate semaphore sets (`semget`, SEMMNI/SEMMNS), and the owned attempt was stopped with its log preserved. Its earlier four caller-fixture failures and one workspace-conflict failure are recorded separately below; the later resource error is not an explanation for them. Delivery commit `567d838d` replayed the initial six-file patch onto `ee67d2ec`; production was last changed in `23456f51`, followed by test-only commits `002fa4a7` and `6e8fa595`. All 24 staging checks, all 26 materializer/prepared-launch checks, and the complete native executor file (968 tests, zero skipped) pass locally. Server typecheck also passed after these test edits. The complete CI matrix passed on `002fa4a7`; current-head CI is tracked separately. - Initial-head CI exposed four caller fixtures that manually wrote auth/config without preparing the skills inventory. The same four failures reproduced locally. Those fixtures now use the real home preparer for their existing empty inventory and assert its transfer; all 34 retained/replacement scenarios and the complete 966-test caller file pass. A sandboxed full-file attempt could not bind local PostgreSQL (`EPERM`); it was stopped with its log preserved, then the identical command passed with local fixture access. Initial-head CI also recorded timeouts in workspace recovery, approval-queue backoff, and heartbeat attribution. Their three complete files pass unchanged locally (102 tests); the next CI head also passed those cases. The original slowdown remains unexplained. The consequent approval cleanup foreign-key failure is a timeout teardown race already recorded in the owning worker documentation. - The frozen full local run also observed an existing workspace-conflict test fail. The test and owning workspace code are unchanged between the verification and delivery trees, and the isolated delivery case passes. The original failure remains unresolved: stopping the later resource-blocked run prevented the final assertion detail from being emitted. That isolated pass does not replace the earlier result. - CI on `23456f51` caught a Telegram test reading `provider_effect=processing` after observing the provider mock call. Production commits `processed` later, so the test now waits for the unchanged complete durable-state assertion. All 18 selected Telegram attachment/media cases pass (the other 1,670 cases were not selected). The same CI attempt also recorded an E2E runner shutdown followed by cancellation, without a failed Playwright assertion; the necessary new test-only head reruns the matrix. Earlier failures remain preserved and are not reported as passes. - Review requested an executor-level preconfigured-home negative test. Retained and replacement workspace cases now drive the actual launcher, assert successful context staging, and exclude the isolated home's skills/auth/config transfers. A counterfactual that removes only the executor exclusion makes both tests fail; the exact production bytes were restored before the complete 968-test pass. Flat and nested selected package delivery is also covered. Optional arbitrary remote-stderr expansion is deferred; the repair keeps the existing closed error code and report surface. - Final head `6e8fa595` passes the complete CI matrix, including typecheck, build, Runner, browser, and aggregate gates. Two first-attempt jobs were interrupted by runner shutdown/lost communication; one unchanged-head rerun of those jobs and their aggregates passed. The underlying host interruption cause remains unknown. Both current-head automated reviews are 5/5 with the implemented finding resolved. - The original checked-in lockfile fails a fresh frozen install on the baseline. The repository's maintained resolution-then-frozen sequence passes. No lockfile change is included. - Independent source review found and then verified the retained-ancestor guard. This PR does not claim that the full live Cloud onboarding campaign has passed. It changes no journey assertion or approval gate. ## Risks Launch preparation adds an asynchronous inventory walk, one bounded remote cleanup command, and one transfer of the selected skill tree. Those costs grow with selected files and bytes and occur only at provider launch. A failed validation, cleanup, or transfer prevents launch. Remote Node is already required by the shared SSH/sandbox launcher. The selected packages remain read-only; the private discovery root retains Codex's normal writable mode. No database, API, credential-routing, or deployment change is included. ## Model Used OpenAI Codex based on GPT-6, with reasoning, source inspection, tool use, and code execution. The host does not expose the exact serving model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed
1 parent
efd65c1125
commit
f8484a9f8d
8 files changed
+312
-24
No files matched your search
@@ -1624,6 +1624,15 @@ reads lightweight thread state, and fetches paginated turn metadata or specific
|
||||
turn items only when execution reconciliation needs them. Unsupported
|
||||
or incomplete history is an explicit error, not evidence of idle execution.
|
||||
|
||||
Built-in remote Codex launches install the current sealed skill inventory into
|
||||
the isolated provider home's `skills` directory before starting Codex. Explicit
|
||||
skill inputs use that same discovery path; a separate immutable context copy does
|
||||
not make a skill discoverable to Codex. Launch preparation replaces prior skills,
|
||||
including an empty or revoked inventory, while preserving provider conversation
|
||||
history. It rejects source or destination symlinks and does not copy ambient host
|
||||
skills. Preconfigured homes and plugin-provided launch preparation keep their
|
||||
existing contracts.
|
||||
|
||||
The root-thread usage snapshot sent during resume belongs to its reported
|
||||
completed turn. Retain a bounded local diagnostic and use cumulative totals as
|
||||
a baseline; do not emit a warning or charge its historical `last` usage to the
|
||||
|
||||
@@ -10,6 +10,8 @@ export * from "./conformance/control-plane-port.js";
|
||||
export * from "./conformance/capability-semantic-conformance.js";
|
||||
export * from "./conformance/harness-driver.js";
|
||||
export { startOpenCodeMcpBridge } from "./drivers/opencode/mcp-bridge.js";
|
||||
export { prepareIsolatedCodexHome, releaseMaterializedNativeRuntimeSkills } from "./drivers/runtime-context-materializer.js";
|
||||
export { resolveRunnerdCodexSkillInputs } from "./live/runnerd-codex-transport.js";
|
||||
export * from "./conformance/semantic-conformance.js";
|
||||
export * from "./mock-core/deterministic-harness-driver.js";
|
||||
export * from "./mock-core/mock-control-plane-adapter.js";
|
||||
|
||||
@@ -72348,17 +72348,19 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
await vi.waitFor(() =>
|
||||
expect(dm.post).toHaveBeenCalledWith(visibleFailure),
|
||||
);
|
||||
await expect(
|
||||
db
|
||||
.select({ kind: chatActions.kind, status: chatActions.status })
|
||||
.from(chatActions)
|
||||
.where(eq(chatActions.deliveryId, delivery.id)),
|
||||
).resolves.toEqual(
|
||||
expect.arrayContaining([
|
||||
{ kind: "inbound_wakeup", status: "failed" },
|
||||
{ kind: "provider_effect", status: "processed" },
|
||||
]),
|
||||
);
|
||||
await vi.waitFor(async () => {
|
||||
await expect(
|
||||
db
|
||||
.select({ kind: chatActions.kind, status: chatActions.status })
|
||||
.from(chatActions)
|
||||
.where(eq(chatActions.deliveryId, delivery.id)),
|
||||
).resolves.toEqual(
|
||||
expect.arrayContaining([
|
||||
{ kind: "inbound_wakeup", status: "failed" },
|
||||
{ kind: "provider_effect", status: "processed" },
|
||||
]),
|
||||
);
|
||||
});
|
||||
} finally {
|
||||
await retirePublicationFixture(service, endpoint.id);
|
||||
}
|
||||
|
||||
@@ -84,6 +84,7 @@ import {
|
||||
verifyNativeHarnessBackupStamp,
|
||||
} from "./native-harness-backup-stamp.js";
|
||||
import { nativeRuntimeContextFixture } from "./runtime-context.test-fixture.js";
|
||||
import { prepareIsolatedCodexHome } from "../../vendor/paperclip-runner/testing.js";
|
||||
import { nativeToolContractFingerprintForTarget } from "./native-session-resume.js";
|
||||
import * as nativeSessionResume from "./native-session-resume.js";
|
||||
import * as nativeLeaseRenewal from "./native-execution-lease.js";
|
||||
@@ -14801,10 +14802,13 @@ describe("runnerd provider runtime wiring", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it.each(["resumed", "replacement"].flatMap((outcome) =>
|
||||
it.each([
|
||||
...["resumed", "replacement"].flatMap((outcome) =>
|
||||
["fresh", "external_checkpoint", "external_pending_turn", "external_wrong_identity", "external_unregistered", "existing_state", "symlink_parent", "wrong_identity", "wrong_runner", "wrong_lease", "missing_control_plane", "connected", "committed_event", "committed_command", "pending_turn", "remote_probe_failed", "backup_present"]
|
||||
.map((scenario) => ({ outcome, scenario }))))(
|
||||
"bootstraps only an untouched provider session in a $outcome workspace lease: $scenario", async ({ outcome, scenario }) => {
|
||||
.map((scenario) => ({ outcome, scenario, homeMode: "isolated" }))),
|
||||
...["resumed", "replacement"].map(outcome => ({ outcome, scenario: "fresh", homeMode: "preconfigured" })),
|
||||
])(
|
||||
"bootstraps only an untouched provider session in a $outcome workspace lease: $scenario ($homeMode)", async ({ outcome, scenario, homeMode }) => {
|
||||
const remoteCwd = join(isolatedStateDirectory, "remote");
|
||||
const runtimeRoot = join(remoteCwd, ".paperclip-runtime", "paperclip-runner");
|
||||
await mkdir(runtimeRoot, { recursive: true });
|
||||
@@ -14831,6 +14835,9 @@ describe("runnerd provider runtime wiring", () => {
|
||||
if (command.args?.[0] === "--version") return {
|
||||
exitCode: 0, timedOut: false, stdout: "codex-cli 0.160.0", stderr: "",
|
||||
};
|
||||
if (command.args?.[1]?.includes("getent passwd")) return {
|
||||
exitCode: 0, timedOut: false, stdout: "/home/executor\n", stderr: "",
|
||||
};
|
||||
if (command.args?.[2] === "paperclip-runner-launch") {
|
||||
throw new Error("fixture_stop_after_launch_staging");
|
||||
}
|
||||
@@ -14840,12 +14847,16 @@ describe("runnerd provider runtime wiring", () => {
|
||||
return { exitCode: 0, timedOut: false, stdout: "", stderr: "" };
|
||||
});
|
||||
const runtimeContext = nativeRuntimeContextFixture();
|
||||
const executionWithContext = { ...execution, runtimeContext };
|
||||
const executionWithContext = homeMode === "preconfigured"
|
||||
? { ...execution, schema: "paperclip.native-execution-input.v7" as const,
|
||||
homeMode: "preconfigured" as const, executionMode: "default" as const, planningContext: null,
|
||||
provider: { kind: "codex" as const, model: null, approvalPolicy: "never" as const }, runtimeContext }
|
||||
: { ...execution, runtimeContext };
|
||||
const onTraceEvent = vi.fn(async (_event: AdapterRuntimeEvent) => undefined);
|
||||
const controllerArtifact = join(isolatedStateDirectory, "bootstrap-runnerd");
|
||||
if (scenario.startsWith("external_")) await writeFile(controllerArtifact, "fixture bootstrap artifact");
|
||||
const backend = await createRunnerdBackend({
|
||||
db: leaseDb(execution), execution: executionWithContext, runnerInstanceId: "runner-new-in-retained-workspace",
|
||||
db: leaseDb(executionWithContext), execution: executionWithContext, runnerInstanceId: "runner-new-in-retained-workspace",
|
||||
...(scenario.startsWith("external_") ? { runnerRemoteBinaryPath: controllerArtifact, runnerRemoteCodexPath: controllerArtifact } : {}),
|
||||
trace: createNativeRunTrace({ runId: execution.binding.runId, onEvent: onTraceEvent }),
|
||||
runnerIngressAuthorized: true,
|
||||
@@ -14934,9 +14945,11 @@ describe("runnerd provider runtime wiring", () => {
|
||||
// the current invocation's launch files. The actual launch must stage
|
||||
// those files without trying to claim/restore the session a second time.
|
||||
const home = join(options.stateDirectory!, "codex-home");
|
||||
await mkdir(home, { recursive: true });
|
||||
await writeFile(join(home, "auth.json"), "fixture-current-credential");
|
||||
await writeFile(join(home, "config.toml"), "fixture-current-config");
|
||||
if (homeMode === "isolated") {
|
||||
await prepareIsolatedCodexHome({ codexHome: home, context: runtimeContext });
|
||||
await writeFile(join(home, "auth.json"), "fixture-current-credential");
|
||||
await writeFile(join(home, "config.toml"), "fixture-current-config");
|
||||
}
|
||||
await writeFile(join(options.stateDirectory!, "runtime-context.json"), JSON.stringify(runtimeContext));
|
||||
syncIn.mockClear();
|
||||
await expect(options.prepareExternalRunnerState()).resolves.toBeUndefined();
|
||||
@@ -14945,10 +14958,20 @@ describe("runnerd provider runtime wiring", () => {
|
||||
command: "/controller/paperclip-runnerd", args: [], cwd: "/controller", environment: {},
|
||||
});
|
||||
await expect(launched.completion).rejects.toThrow("fixture_stop_after_launch_staging");
|
||||
for (const name of ["auth.json", "config.toml"]) {
|
||||
if (homeMode === "isolated") {
|
||||
expect(syncIn).toHaveBeenCalledWith([expect.objectContaining({
|
||||
files: [expect.objectContaining({ sourcePath: join(home, name), kind: "file", mode: 0o600 })],
|
||||
files: [expect.objectContaining({ sourcePath: join(home, "skills"), kind: "directory", mode: 0o700 })],
|
||||
})]);
|
||||
for (const name of ["auth.json", "config.toml"]) {
|
||||
expect(syncIn).toHaveBeenCalledWith([expect.objectContaining({
|
||||
files: [expect.objectContaining({ sourcePath: join(home, name), kind: "file", mode: 0o600 })],
|
||||
})]);
|
||||
}
|
||||
} else {
|
||||
const files = syncIn.mock.calls.flat(2).flatMap((entry: { files: Array<{ sourcePath: string }> }) => entry.files);
|
||||
for (const name of ["skills", "auth.json", "config.toml"]) {
|
||||
expect(files.some(file => file.sourcePath === join(home, name))).toBe(false);
|
||||
}
|
||||
}
|
||||
expect(syncIn.mock.calls.flat(2).flatMap((entry: { files: Array<{ sourcePath: string }> }) => entry.files)
|
||||
.filter((file: { sourcePath: string }) => file.sourcePath === runtimeContext.instructions.bundle.rootPath)).toHaveLength(1);
|
||||
|
||||
@@ -13186,6 +13186,9 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
caBundleMapping: remoteCaBundleMapping,
|
||||
stageFile: asset => stageRemoteRunnerFile({ target: remoteTarget, runner: remoteCommandRunner, ...asset }),
|
||||
stageDirectory: asset => stageRemoteRunnerDirectory({ target: remoteTarget, runner: remoteCommandRunner, ...asset }),
|
||||
isolatedCodexSkills: input.execution.provider.kind === "codex"
|
||||
&& !("homeMode" in input.execution && input.execution.homeMode === "preconfigured")
|
||||
? { workspaceRoot: remoteTarget.remoteCwd } : undefined,
|
||||
trace: input.trace,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1,9 +1,13 @@
|
||||
import { mkdtempSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { chmodSync, cpSync, existsSync, lstatSync, mkdtempSync, mkdirSync, readFileSync, readdirSync, rmSync, statSync, symlinkSync, writeFileSync } from "node:fs";
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { dirname, join } from "node:path";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { createTarballFromDirectory } from "@paperclipai/adapter-utils/sandbox-managed-runtime";
|
||||
import { nativeRuntimeContextFixture } from "./runtime-context.test-fixture.js";
|
||||
import { stageRemoteLaunchAssets } from "./remote-launch-assets.js";
|
||||
import { prepareIsolatedCodexHome, releaseMaterializedNativeRuntimeSkills, resolveRunnerdCodexSkillInputs } from "../../vendor/paperclip-runner/testing.js";
|
||||
|
||||
function deferred() {
|
||||
let resolve!: () => void;
|
||||
@@ -13,7 +17,15 @@ function deferred() {
|
||||
|
||||
const roots: string[] = [];
|
||||
afterEach(() => {
|
||||
for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true });
|
||||
function writable(path: string) {
|
||||
const stat = lstatSync(path);
|
||||
if (stat.isSymbolicLink()) return;
|
||||
if (stat.isDirectory()) {
|
||||
chmodSync(path, 0o700);
|
||||
for (const entry of readdirSync(path)) writable(join(path, entry));
|
||||
}
|
||||
}
|
||||
for (const root of roots.splice(0)) { writable(root); rmSync(root, { recursive: true, force: true }); }
|
||||
});
|
||||
|
||||
function fixture(options: { transport?: "sandbox" | "ssh"; concurrent?: boolean } = {}) {
|
||||
@@ -52,7 +64,156 @@ function fixture(options: { transport?: "sandbox" | "ssh"; concurrent?: boolean
|
||||
return { input, root, calls, stageFile, stageDirectory, execute };
|
||||
}
|
||||
|
||||
async function filesystemFixture() {
|
||||
const f = fixture();
|
||||
const source = f.input.runtimeContext!.source;
|
||||
f.input.isolatedCodexSkills = { workspaceRoot: f.root };
|
||||
mkdirSync(source.instructions.bundle.rootPath);
|
||||
writeFileSync(join(source.instructions.bundle.rootPath, "AGENTS.md"), "Agent instructions\n");
|
||||
mkdirSync(source.skills[0]!.bundle.rootPath);
|
||||
writeFileSync(join(source.skills[0]!.bundle.rootPath, "SKILL.md"), "---\nname: test\ndescription: Selected skill\n---\nUse a native confirmation.\n");
|
||||
mkdirSync(join(source.skills[0]!.bundle.rootPath, "scripts"));
|
||||
writeFileSync(join(source.skills[0]!.bundle.rootPath, "scripts", "selected.sh"), "#!/bin/sh\nprintf selected\\n\n", { mode: 0o755 });
|
||||
await prepareIsolatedCodexHome({ context: source, codexHome: join(f.root, "codex-home") });
|
||||
const remote = join(f.root, "remote", "codex-home");
|
||||
f.input.persistenceDirectories = [{ name: "codex-home", location: "filesystem", targetPath: remote }];
|
||||
f.input.stageFile = async ({ sourcePath, targetPath }) => {
|
||||
if (!targetPath.startsWith(f.root)) return;
|
||||
mkdirSync(dirname(targetPath), { recursive: true });
|
||||
cpSync(sourcePath, targetPath);
|
||||
};
|
||||
f.input.stageDirectory = vi.fn(async ({ sourcePath, targetPath }) => {
|
||||
if (!targetPath.startsWith(f.root)) return;
|
||||
// Native sandbox transfer archives omit the source root and mkdir the
|
||||
// destination without applying the mapping mode. Exercise those actual
|
||||
// tar/filesystem semantics rather than forcing the requested mode here.
|
||||
const archivePath = join(f.root, "native-transfer.tar");
|
||||
await createTarballFromDirectory({ localDir: sourcePath, archivePath });
|
||||
execFileSync("sh", ["-c", 'umask 022; mkdir -p "$1" && tar -xf "$2" -C "$1"', "native-directory-transfer", targetPath, archivePath]);
|
||||
});
|
||||
f.input.runner.execute = async ({ command, args }) => {
|
||||
try {
|
||||
const stdout = execFileSync(command, args, { encoding: "utf8", stdio: "pipe" });
|
||||
return { exitCode: 0, signal: null, pid: null, startedAt: null, timedOut: false, stdout: stdout.toString(), stderr: "" };
|
||||
} catch {
|
||||
return { exitCode: 1, signal: null, pid: null, startedAt: null, timedOut: false, stdout: "", stderr: "fixture command rejected" };
|
||||
}
|
||||
};
|
||||
return { ...f, source, remote };
|
||||
}
|
||||
|
||||
describe("remote launch asset staging", () => {
|
||||
it("delivers the selected Codex skills to the same discoverable path used by explicit turn input", async () => {
|
||||
const f = await filesystemFixture();
|
||||
await stageRemoteLaunchAssets(f.input);
|
||||
const [skill] = resolveRunnerdCodexSkillInputs([
|
||||
{ type: "skill", name: "test", path: join(f.source.skills[0]!.bundle.rootPath, "SKILL.md") },
|
||||
], f.source, f.remote);
|
||||
expect(existsSync(skill!.path)).toBe(true);
|
||||
expect(readFileSync(skill!.path, "utf8")).toContain("Use a native confirmation.");
|
||||
const digest = (path: string) => createHash("sha256").update(readFileSync(path)).digest("hex");
|
||||
for (const relative of ["SKILL.md", "scripts/selected.sh"]) {
|
||||
expect(digest(join(f.remote, "skills", "test", relative))).toBe(digest(join(f.root, "codex-home", "skills", "test", relative)));
|
||||
}
|
||||
expect(statSync(skill!.path).mode & 0o777).toBe(0o444);
|
||||
expect(statSync(join(f.remote, "skills", "test")).mode & 0o222).toBe(0);
|
||||
expect(statSync(join(f.remote, "skills", "test", "scripts", "selected.sh")).mode & 0o777).toBe(0o555);
|
||||
expect(statSync(join(f.remote, "skills")).mode & 0o777).toBe(0o700);
|
||||
});
|
||||
|
||||
it("replaces stale and revoked skills on resume while retaining provider history", async () => {
|
||||
const f = await filesystemFixture();
|
||||
await stageRemoteLaunchAssets(f.input);
|
||||
writeFileSync(join(f.remote, "conversation.json"), "saved provider conversation");
|
||||
mkdirSync(join(f.remote, "skills", "ambient"));
|
||||
writeFileSync(join(f.remote, "skills", "ambient", "SKILL.md"), "unassigned");
|
||||
await releaseMaterializedNativeRuntimeSkills(join(f.root, "codex-home", "skills"));
|
||||
f.source.skills = [];
|
||||
f.input.runtimeContext!.target.skills = [];
|
||||
await prepareIsolatedCodexHome({ context: f.source, codexHome: join(f.root, "codex-home") });
|
||||
await stageRemoteLaunchAssets(f.input);
|
||||
expect(readdirSync(join(f.remote, "skills"))).toEqual([]);
|
||||
expect(readFileSync(join(f.remote, "conversation.json"), "utf8")).toBe("saved provider conversation");
|
||||
});
|
||||
|
||||
it("delivers flat and nested selected packages without flattening their paths", async () => {
|
||||
const f = await filesystemFixture();
|
||||
const nestedRoot = join(f.root, "nested-skill");
|
||||
mkdirSync(nestedRoot);
|
||||
const body = "---\nname: nested\ndescription: Nested selected skill\n---\nKeep the nested package.\n";
|
||||
writeFileSync(join(nestedRoot, "SKILL.md"), body);
|
||||
f.source.skills.push({ ...f.source.skills[0]!, key: "nested", runtimeName: "group/nested",
|
||||
bundle: { ...f.source.skills[0]!.bundle, rootPath: nestedRoot } });
|
||||
f.input.runtimeContext!.target.skills = f.source.skills.map((skill, index) => ({ ...skill,
|
||||
bundle: { ...skill.bundle, rootPath: `/session/filesystem/context/skills/${index}-digest` } }));
|
||||
await releaseMaterializedNativeRuntimeSkills(join(f.root, "codex-home", "skills"));
|
||||
await prepareIsolatedCodexHome({ context: f.source, codexHome: join(f.root, "codex-home") });
|
||||
await stageRemoteLaunchAssets(f.input);
|
||||
expect(readFileSync(join(f.remote, "skills", "test", "SKILL.md"), "utf8")).toContain("Use a native confirmation.");
|
||||
expect(readFileSync(join(f.remote, "skills", "group", "nested", "SKILL.md"), "utf8")).toBe(body);
|
||||
expect(statSync(join(f.remote, "skills", "group", "nested", "SKILL.md")).mode & 0o222).toBe(0);
|
||||
});
|
||||
|
||||
it.each(["symlink", "ambient", "writable"] as const)("rejects an unsafe selected source projection before clearing retained skills: %s", async kind => {
|
||||
const f = await filesystemFixture();
|
||||
mkdirSync(join(f.remote, "skills"), { recursive: true });
|
||||
writeFileSync(join(f.remote, "skills", "retained"), "not touched before valid admission");
|
||||
const home = join(f.root, "codex-home", "skills");
|
||||
if (kind === "ambient") {
|
||||
mkdirSync(join(home, "unassigned"));
|
||||
writeFileSync(join(home, "unassigned", "SKILL.md"), "unassigned", { mode: 0o444 });
|
||||
} else if (kind === "writable") chmodSync(join(home, "test", "SKILL.md"), 0o644);
|
||||
else {
|
||||
chmodSync(join(home, "test"), 0o700);
|
||||
symlinkSync(join(f.root, "outside"), join(home, "test", "link"));
|
||||
}
|
||||
await expect(stageRemoteLaunchAssets(f.input)).rejects.toThrow("runner_remote_codex_skills_invalid");
|
||||
expect(readFileSync(join(f.remote, "skills", "retained"), "utf8")).toBe("not touched before valid admission");
|
||||
});
|
||||
|
||||
it.each([false, true])("rejects a retained skills-root symlink without touching its target (dangling: %s)", async dangling => {
|
||||
const f = await filesystemFixture();
|
||||
mkdirSync(f.remote, { recursive: true });
|
||||
const outside = join(f.root, "outside");
|
||||
if (!dangling) { mkdirSync(outside); writeFileSync(join(outside, "keep"), "untouched"); }
|
||||
symlinkSync(outside, join(f.remote, "skills"));
|
||||
await expect(stageRemoteLaunchAssets(f.input)).rejects.toThrow("runner_remote_codex_skills_reset_failed");
|
||||
expect(lstatSync(join(f.remote, "skills")).isSymbolicLink()).toBe(true);
|
||||
if (!dangling) expect(readFileSync(join(outside, "keep"), "utf8")).toBe("untouched");
|
||||
});
|
||||
|
||||
it.each(["sandbox", "ssh"] as const)("rejects a redirected retained ancestor before writing launch assets: %s", async transport => {
|
||||
const f = await filesystemFixture();
|
||||
f.input.transport = transport;
|
||||
const outside = join(f.root, "outside");
|
||||
mkdirSync(join(outside, "codex-home", "skills"), { recursive: true });
|
||||
writeFileSync(join(outside, "codex-home", "skills", "keep"), "untouched");
|
||||
symlinkSync(outside, join(f.root, "retained-filesystem"));
|
||||
f.input.persistenceDirectories = [{ name: "codex-home", location: "filesystem", targetPath: join(f.root, "retained-filesystem", "codex-home") }];
|
||||
await expect(stageRemoteLaunchAssets(f.input)).rejects.toThrow("runner_remote_codex_skills_reset_failed");
|
||||
expect(readFileSync(join(outside, "codex-home", "skills", "keep"), "utf8")).toBe("untouched");
|
||||
expect(existsSync(join(outside, "codex-home", "config.toml"))).toBe(false);
|
||||
});
|
||||
|
||||
it("allows an explicitly authorized workspace alias while checking its owned descendants", async () => {
|
||||
const f = await filesystemFixture();
|
||||
const actual = join(f.root, "authorized-workspace");
|
||||
mkdirSync(actual);
|
||||
const alias = join(f.root, "authorized-alias");
|
||||
symlinkSync(actual, alias);
|
||||
f.input.isolatedCodexSkills = { workspaceRoot: alias };
|
||||
f.input.persistenceDirectories = [{ name: "codex-home", location: "filesystem", targetPath: join(alias, "codex-home") }];
|
||||
await stageRemoteLaunchAssets(f.input);
|
||||
expect(readFileSync(join(actual, "codex-home", "skills", "test", "SKILL.md"), "utf8")).toContain("Use a native confirmation.");
|
||||
});
|
||||
|
||||
it("does not project skills for a preconfigured home or another provider", async () => {
|
||||
const f = await filesystemFixture();
|
||||
f.input.isolatedCodexSkills = undefined;
|
||||
await stageRemoteLaunchAssets(f.input);
|
||||
expect(existsSync(join(f.remote, "skills"))).toBe(false);
|
||||
});
|
||||
|
||||
it("retains serial paths, private launch modes and immutable context modes without an opt-in", async () => {
|
||||
const f = fixture();
|
||||
await stageRemoteLaunchAssets(f.input);
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { existsSync } from "node:fs";
|
||||
import { lstat, readdir } from "node:fs/promises";
|
||||
import { posix, resolve } from "node:path";
|
||||
import type { CommandManagedRuntimeRunner } from "@paperclipai/adapter-utils/command-managed-runtime";
|
||||
import type { NativeRuntimeContextSnapshot } from "../../vendor/paperclip-runner/index.js";
|
||||
@@ -27,9 +28,67 @@ type LaunchAssetsInput = {
|
||||
caBundleMapping: { sourcePath: string; targetPath: string } | null;
|
||||
stageFile: (input: StageAsset) => Promise<void>;
|
||||
stageDirectory: (input: StageAsset) => Promise<void>;
|
||||
isolatedCodexSkills?: { workspaceRoot: string };
|
||||
trace?: NativeRunTrace;
|
||||
};
|
||||
|
||||
async function assertSelectedSkillsTree(root: string, names: string[]): Promise<void> {
|
||||
const found = new Set<string>();
|
||||
const reject = () => { throw new Error("runner_remote_codex_skills_invalid"); };
|
||||
async function visit(relativePath: string) {
|
||||
const path = relativePath ? resolve(root, relativePath) : root;
|
||||
const stat = await lstat(path);
|
||||
if (stat.isSymbolicLink()) reject();
|
||||
if (stat.isDirectory()) {
|
||||
if (relativePath && !names.some(name => relativePath === name || relativePath.startsWith(`${name}/`) || name.startsWith(`${relativePath}/`))) reject();
|
||||
for (const entry of await readdir(path)) await visit(relativePath ? `${relativePath}/${entry}` : entry);
|
||||
} else {
|
||||
if (!stat.isFile() || (stat.mode & 0o222) !== 0 || !names.some(name => relativePath.startsWith(`${name}/`))) reject();
|
||||
for (const name of names) if (relativePath === `${name}/SKILL.md`) found.add(name);
|
||||
}
|
||||
}
|
||||
await visit("");
|
||||
if (found.size !== names.length) reject();
|
||||
}
|
||||
|
||||
// Launch preparation owns this isolated directory and runs only after the old
|
||||
// provider has stopped. Never follow a retained symlink while removing stale
|
||||
// assignments; auth, config and provider conversation history are not touched.
|
||||
const clearRemoteCodexSkills = `
|
||||
const fs = require("node:fs"), path = require("node:path");
|
||||
const home = path.resolve(process.argv[1]), workspace = path.resolve(process.argv[2]);
|
||||
const relative = path.relative(workspace, home);
|
||||
if (!relative || relative === ".." || relative.startsWith(".." + path.sep) || path.isAbsolute(relative)) throw Error("unsafe_codex_home");
|
||||
// The execution target authorizes the workspace anchor. Resolve only that
|
||||
// anchor (which can have an OS alias), then refuse links below it.
|
||||
let current = fs.realpathSync(workspace);
|
||||
if (!fs.lstatSync(current).isDirectory()) throw Error("unsafe_workspace_root");
|
||||
for (const segment of relative.split(path.sep)) {
|
||||
current = path.join(current, segment);
|
||||
if (!fs.lstatSync(current, { throwIfNoEntry: false })) fs.mkdirSync(current, { mode: 0o700 });
|
||||
const stat = fs.lstatSync(current);
|
||||
if (!stat.isDirectory() || stat.isSymbolicLink()) throw Error("unsafe_codex_home");
|
||||
}
|
||||
const skills = path.join(current, "skills");
|
||||
function unseal(directory) {
|
||||
const stat = fs.lstatSync(directory);
|
||||
if (stat.isSymbolicLink() || !stat.isDirectory()) throw Error("unsafe_skills_root");
|
||||
fs.chmodSync(directory, 0o700);
|
||||
for (const name of fs.readdirSync(directory)) {
|
||||
const child = path.join(directory, name), stat = fs.lstatSync(child);
|
||||
if (stat.isDirectory() && !stat.isSymbolicLink()) unseal(child);
|
||||
}
|
||||
}
|
||||
if (fs.lstatSync(skills, { throwIfNoEntry: false })) {
|
||||
unseal(skills);
|
||||
fs.rmSync(skills, { recursive: true });
|
||||
}
|
||||
// Native directory transfers omit the source root and may ignore its mapping
|
||||
// mode. Create the private discovery root before their mkdir/extract step.
|
||||
fs.mkdirSync(skills, { mode: 0o700 });
|
||||
fs.chmodSync(skills, 0o700);
|
||||
`;
|
||||
|
||||
export async function stageRemoteLaunchAssets(input: LaunchAssetsInput): Promise<void> {
|
||||
const measure = async (name: string, operation: () => Promise<void>) =>
|
||||
input.trace ? input.trace.measure(name, operation) : operation();
|
||||
@@ -80,6 +139,28 @@ export async function stageRemoteLaunchAssets(input: LaunchAssetsInput): Promise
|
||||
});
|
||||
});
|
||||
};
|
||||
const stageCodexSkills = async () => {
|
||||
if (!input.isolatedCodexSkills) return;
|
||||
const directory = input.persistenceDirectories.find(entry => entry.name === "codex-home" && entry.location === "filesystem");
|
||||
if (!directory?.targetPath) throw new Error("runner_harness_state_mismatch: bootstrap_target_unavailable");
|
||||
const sourcePath = resolve(input.root, "codex-home", "skills");
|
||||
await assertSelectedSkillsTree(sourcePath, input.runtimeContext?.source.skills.map(skill => skill.runtimeName) ?? []);
|
||||
await measure("stage.asset.codex_skills", async () => {
|
||||
const cleared = await input.runner.execute({
|
||||
command: "node",
|
||||
args: ["-e", clearRemoteCodexSkills, directory.targetPath!, input.isolatedCodexSkills!.workspaceRoot],
|
||||
bypassSession: true,
|
||||
timeoutMs: 10_000,
|
||||
});
|
||||
if (cleared.exitCode !== 0 || cleared.timedOut) throw new Error("runner_remote_codex_skills_reset_failed");
|
||||
// Match prepareIsolatedCodexHome: the discovery root is private/writable
|
||||
// for Codex's own built-ins; each selected package remains sealed.
|
||||
await input.stageDirectory({ sourcePath, targetPath: posix.join(directory.targetPath!, "skills"), mode: 0o700 });
|
||||
});
|
||||
};
|
||||
// Resolve the owned home and remove obsolete assignments before any launch
|
||||
// file can be written through a retained, redirected path.
|
||||
await stageCodexSkills();
|
||||
if (input.transport === "sandbox" && input.runner.allowConcurrentSyncOperations === true) {
|
||||
const outcomes = await Promise.allSettled([materializeLaunchState(), stageRuntimeContext()]);
|
||||
for (const outcome of outcomes) {
|
||||
|
||||
@@ -26,6 +26,13 @@ const sourceUrl = new URL(
|
||||
);
|
||||
const runnerTesting = await import(sourceUrl.href) as RunnerTestingModule;
|
||||
|
||||
export const prepareIsolatedCodexHome: RunnerTestingModule["prepareIsolatedCodexHome"] =
|
||||
runnerTesting.prepareIsolatedCodexHome;
|
||||
export const releaseMaterializedNativeRuntimeSkills: RunnerTestingModule["releaseMaterializedNativeRuntimeSkills"] =
|
||||
runnerTesting.releaseMaterializedNativeRuntimeSkills;
|
||||
export const resolveRunnerdCodexSkillInputs: RunnerTestingModule["resolveRunnerdCodexSkillInputs"] =
|
||||
runnerTesting.resolveRunnerdCodexSkillInputs;
|
||||
|
||||
export const startOpenCodeMcpBridge:
|
||||
RunnerTestingModule["startOpenCodeMcpBridge"] =
|
||||
runnerTesting.startOpenCodeMcpBridge;
|
||||
|
||||
Reference in new issue
Block a user