mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 07:10:34 +01:00
fix(native): export OpenCode launch artifacts through a confined workspace copy (#19743)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native recovery saves the qualified OpenCode launch files so a cold allocation can resume the same session (#19622) > - Core reads those files from the sandbox with the provider's bulk `syncOut` > - The launch root is under the sandbox account home, not under the workspace remote dir > - Daytona, Kubernetes, and CreateOS confine every export source to the workspace remote dir, so the export always fails > - Every remote OpenCode run on a durable-storage instance now fails 40-90 s after start with `native_session_interrupted` > - This pull request exports a confined scratch copy of the three launch files from inside the workspace runtime dir > - The benefit is that remote OpenCode runs work again, and the provider confinement guard stays unchanged ## Linked Issues or Issue Description Refs #19622 **What happened?** After #19622, every remote OpenCode run on Daytona fails at provider admission. The run error is `Daytona sync source path escapes the workspace remote dir: /home/daytona/.paperclip-runtime/paperclip-runner/provider-artifacts/<digest>`. The error code is `native_session_interrupted`. The last successful run of the affected agent was immediately before #19622 reached the deployment. **Expected behavior** Core saves the qualified launch files and the run continues. The provider confinement guard still refuses any export source outside the workspace remote dir. **Steps to reproduce** 1. Configure an OpenCode agent on a Daytona environment with durable harness storage. 2. Start a run. 3. The run fails when Core captures the provider artifacts, before the first turn. ## What Changed - `captureRemoteNativeProviderArtifacts` now takes the workspace `remoteCwd`. - Before the export, a sandbox command copies the three launch files into a new scratch directory at `<remoteCwd>/.paperclip-runtime/paperclip-runner/provider-artifact-export/<uuid>/`. The command refuses a symlink or a non-regular file. It uses `cp -P`, so it never follows a link. - `syncOut` reads from that scratch copy. A `finally` step removes the scratch copy after success or failure. A failed removal is logged and does not replace the transfer result. - The launch root does not move. OpenCode still binds to the same absolute paths. Core still verifies each downloaded byte against the manifest SHA-256 values. - `.paperclip-runtime` is already excluded from workspace sync, so the scratch copy never enters the user workspace. - The Daytona `assertConfinedSandboxPath` guard is not changed. ## Verification - New regression `server/src/services/native-runtime/native-provider-artifact-capture-confinement.test.ts`. It runs the real sandbox commands on a local directory tree that has a home-rooted launch root and a separate workspace. Its `syncOut` applies the same workspace confinement rule as the Daytona provider. - Before the fix: it fails with `Daytona sync source path escapes the workspace remote dir: .../home/daytona/.paperclip-runtime/paperclip-runner/provider-artifacts/dfe9e4b1d660/bin/nodeCommand`. This is the staging error. - After the fix: it passes. The published bytes match, every export source is under the workspace runtime dir, and the scratch directory is empty after the capture. - Safety cases: a failed transfer removes the scratch copy and does not publish. A launch file replaced by a symlink is refused before any export, and nothing is published. - `native-session-executor.test.ts`: the durable OpenCode admission case now asserts that the capture receives `remoteCwd` and that every `syncOut` source is under `/workspace/`. - `pnpm vitest run` for the capture, confinement, remote OpenCode artifact, and session root entries tests: 14 passed. - `vitest run native-session-executor.test.ts -t "opencode|provider artifact|durable"`: 62 passed. - `pnpm --filter @paperclipai/server typecheck`: passed. - Not run: a live Daytona run with this revision. The fix is not deployed to staging. ## Risks - Low risk. The capture now runs two short `sh` commands in the sandbox and holds a second copy of the launch files during the transfer. The files are bounded by the existing 512 MiB limit and are deleted afterwards. - The scratch copy uses `cp -P`, `mkdir`, and `rm -rf` from POSIX `sh`. The Daytona image already uses these tools for the workspace sync. - If the sandbox stops between the copy and the removal, a scratch directory can remain under `.paperclip-runtime`. Workspace sync excludes that directory. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edit, and read-only browser access to staging). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
9f0ebf6f70
commit
24cbea136f
5 files changed
+234
-10
No files matched your search
+148
@@ -0,0 +1,148 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { access, copyFile, mkdir, mkdtemp, readFile, readdir, rm, symlink, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join, posix } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
import { afterEach, expect, it, vi } from "vitest";
|
||||
import type { CommandManagedRuntimeRunner } from "@paperclipai/adapter-utils/command-managed-runtime";
|
||||
import type { NativeProviderArtifactSource } from "./native-provider-artifacts.js";
|
||||
import { captureRemoteNativeProviderArtifacts } from "./native-provider-artifact-capture.js";
|
||||
|
||||
const run = promisify(execFile);
|
||||
|
||||
// Mirrors the Daytona provider's outbound source guard (`assertConfinedSandboxPath`
|
||||
// in packages/plugins/sandbox-providers/daytona/src/file-sync.ts). Kubernetes and
|
||||
// CreateOS apply the same workspace-root confinement to export sources.
|
||||
function assertConfinedSandboxPath(remoteDir: string, candidate: string, label: string) {
|
||||
const root = posix.normalize(remoteDir);
|
||||
const normalized = posix.normalize(candidate);
|
||||
if (normalized !== root && !normalized.startsWith(`${root}/`)) {
|
||||
throw new Error(`Daytona sync ${label} path escapes the workspace remote dir: ${candidate}`);
|
||||
}
|
||||
}
|
||||
const names = ["nodeCommand", "opencodeProxy", "opencodeExecutable"] as const;
|
||||
const roots: string[] = [];
|
||||
afterEach(async () => {
|
||||
await Promise.all(roots.splice(0).map((root) => rm(root, { recursive: true, force: true })));
|
||||
});
|
||||
|
||||
/**
|
||||
* A sandbox whose bulk export is confined to the workspace remote dir, as the
|
||||
* Daytona, Kubernetes, and CreateOS providers are. The qualified launch root
|
||||
* lives under the sandbox account's home, outside that workspace.
|
||||
*/
|
||||
async function confinedSandbox() {
|
||||
const sandbox = await mkdtemp(join(tmpdir(), "pc-confined-sandbox-"));
|
||||
roots.push(sandbox);
|
||||
const home = join(sandbox, "home", "daytona");
|
||||
const remoteCwd = join(home, "workspace");
|
||||
await mkdir(remoteCwd, { recursive: true });
|
||||
const launchRoot = join(home, ".paperclip-runtime", "paperclip-runner", "provider-artifacts", "dfe9e4b1d660");
|
||||
const artifacts: Record<string, { path: string; sha256: string }> = {};
|
||||
const paths = {} as Record<(typeof names)[number], string>;
|
||||
for (const name of names) {
|
||||
const relative = `bin/${name}`;
|
||||
const bytes = `qualified ${name} bytes`;
|
||||
await mkdir(join(launchRoot, "bin"), { recursive: true });
|
||||
await writeFile(join(launchRoot, relative), bytes, { mode: 0o500 });
|
||||
artifacts[name] = { path: relative, sha256: `sha256:${createHash("sha256").update(bytes).digest("hex")}` };
|
||||
paths[name] = join(launchRoot, relative);
|
||||
}
|
||||
const syncOut = vi.fn<NonNullable<CommandManagedRuntimeRunner["syncOut"]>>(async (operations) => {
|
||||
for (const operation of operations) {
|
||||
for (const file of operation.files) {
|
||||
assertConfinedSandboxPath(remoteCwd, file.sourcePath, "source");
|
||||
await copyFile(file.sourcePath, file.targetPath);
|
||||
}
|
||||
}
|
||||
return { operations: [] };
|
||||
});
|
||||
const runner = {
|
||||
execute: vi.fn(async (input: { command: string; args?: string[] }) => {
|
||||
try {
|
||||
const result = await run(input.command, input.args ?? []);
|
||||
return { exitCode: 0, signal: null, timedOut: false, stdout: result.stdout, stderr: result.stderr };
|
||||
} catch (error) {
|
||||
const failed = error as { code?: number; stdout?: string; stderr?: string };
|
||||
return { exitCode: failed.code ?? 1, signal: null, timedOut: false, stdout: failed.stdout ?? "", stderr: failed.stderr ?? "" };
|
||||
}
|
||||
}),
|
||||
syncOut,
|
||||
} as unknown as CommandManagedRuntimeRunner;
|
||||
const manifest = { digest: `sha256:${"d".repeat(64)}`, payload: { artifacts } } as unknown as NativeProviderArtifactSource["manifest"];
|
||||
return { remoteCwd, launchRoot, paths, runner, syncOut, manifest };
|
||||
}
|
||||
|
||||
it("exports qualified launch files that live outside the workspace through a confined staging copy", async () => {
|
||||
const sandbox = await confinedSandbox();
|
||||
const published: Record<string, string> = {};
|
||||
const result = await captureRemoteNativeProviderArtifacts(
|
||||
{
|
||||
runner: sandbox.runner,
|
||||
remoteCwd: sandbox.remoteCwd,
|
||||
manifest: sandbox.manifest,
|
||||
signal: new AbortController().signal,
|
||||
launch: { root: sandbox.launchRoot, accountUid: 1000, paths: sandbox.paths },
|
||||
},
|
||||
async (source) => {
|
||||
for (const name of names) {
|
||||
published[name] = await readFile(join(source.sourceRoot, `bin/${name}`), "utf8");
|
||||
}
|
||||
expect(source.launchRoot).toBe(sandbox.launchRoot);
|
||||
return "published";
|
||||
},
|
||||
);
|
||||
|
||||
expect(result).toBe("published");
|
||||
expect(published).toEqual(Object.fromEntries(names.map((name) => [name, `qualified ${name} bytes`])));
|
||||
for (const file of sandbox.syncOut.mock.calls.flatMap(([operations]) => operations.flatMap((op) => op.files))) {
|
||||
expect(file.sourcePath.startsWith(`${sandbox.remoteCwd}/.paperclip-runtime/`)).toBe(true);
|
||||
}
|
||||
// The export copy is scratch: it is removed and the qualified launch root is untouched.
|
||||
const exportParent = posix.join(sandbox.remoteCwd, ".paperclip-runtime", "paperclip-runner", "provider-artifact-export");
|
||||
expect(await readdir(exportParent)).toEqual([]);
|
||||
for (const name of names) await access(sandbox.paths[name]);
|
||||
});
|
||||
|
||||
it("removes the confined staging copy when the export fails", async () => {
|
||||
const sandbox = await confinedSandbox();
|
||||
sandbox.syncOut.mockRejectedValueOnce(new Error("transfer failed"));
|
||||
const publish = vi.fn(async () => "must not publish");
|
||||
await expect(captureRemoteNativeProviderArtifacts(
|
||||
{
|
||||
runner: sandbox.runner,
|
||||
remoteCwd: sandbox.remoteCwd,
|
||||
manifest: sandbox.manifest,
|
||||
signal: new AbortController().signal,
|
||||
launch: { root: sandbox.launchRoot, accountUid: 1000, paths: sandbox.paths },
|
||||
},
|
||||
publish,
|
||||
)).rejects.toThrow("transfer failed");
|
||||
expect(publish).not.toHaveBeenCalled();
|
||||
const exportParent = posix.join(sandbox.remoteCwd, ".paperclip-runtime", "paperclip-runner", "provider-artifact-export");
|
||||
expect(await readdir(exportParent)).toEqual([]);
|
||||
});
|
||||
|
||||
it("refuses a launch file replaced by a symlink instead of exporting its target", async () => {
|
||||
const sandbox = await confinedSandbox();
|
||||
const secret = join(sandbox.remoteCwd, "..", "secret");
|
||||
await writeFile(secret, "not a qualified artifact");
|
||||
await rm(sandbox.paths.opencodeProxy);
|
||||
await symlink(secret, sandbox.paths.opencodeProxy);
|
||||
const publish = vi.fn(async () => "must not publish");
|
||||
await expect(captureRemoteNativeProviderArtifacts(
|
||||
{
|
||||
runner: sandbox.runner,
|
||||
remoteCwd: sandbox.remoteCwd,
|
||||
manifest: sandbox.manifest,
|
||||
signal: new AbortController().signal,
|
||||
launch: { root: sandbox.launchRoot, accountUid: 1000, paths: sandbox.paths },
|
||||
},
|
||||
publish,
|
||||
)).rejects.toThrow("native_provider_artifacts_remote_unavailable");
|
||||
expect(sandbox.syncOut).not.toHaveBeenCalled();
|
||||
expect(publish).not.toHaveBeenCalled();
|
||||
const exportParent = posix.join(sandbox.remoteCwd, ".paperclip-runtime", "paperclip-runner", "provider-artifact-export");
|
||||
expect(await readdir(exportParent)).toEqual([]);
|
||||
});
|
||||
@@ -34,9 +34,16 @@ it.each(["stop", "deadline"])(
|
||||
entered = done;
|
||||
});
|
||||
const runner = {
|
||||
execute: vi.fn(async () => {
|
||||
throw new Error("unexpected provider command");
|
||||
}),
|
||||
// The confined export staging and its cleanup run as sandbox commands.
|
||||
execute: vi.fn(async () => ({
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
stdout: "",
|
||||
stderr: "",
|
||||
pid: null,
|
||||
startedAt: null,
|
||||
})),
|
||||
syncOut: async (operations) => {
|
||||
const destination = operations[0].files[0].targetPath;
|
||||
root = dirname(destination);
|
||||
@@ -61,6 +68,7 @@ it.each(["stop", "deadline"])(
|
||||
const result = captureRemoteNativeProviderArtifacts(
|
||||
{
|
||||
runner,
|
||||
remoteCwd: "/workspace",
|
||||
manifest,
|
||||
context,
|
||||
signal: context.signal,
|
||||
|
||||
@@ -2,16 +2,53 @@ import { logger } from "../../middleware/logger.js";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { mkdtemp, mkdir } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join, dirname } from "node:path";
|
||||
import { join, dirname, posix } from "node:path";
|
||||
import type { CommandManagedRuntimeRunner } from "@paperclipai/adapter-utils/command-managed-runtime";
|
||||
import type { NativeProviderArtifactSource } from "./native-provider-artifacts.js";
|
||||
import type { OpenCodeLaunchArtifacts } from "./remote-opencode-artifacts.js";
|
||||
import type { NativeExecutionContext } from "./native-execution-context.js";
|
||||
import { removeNativeHarnessBackup } from "./native-harness-backup-cleanup.js";
|
||||
|
||||
const names = ["nodeCommand", "opencodeProxy", "opencodeExecutable"] as const;
|
||||
|
||||
// Copies the qualified launch files into a fresh export directory. The launch
|
||||
// root sits under the sandbox account's home, outside the workspace remote dir,
|
||||
// and sandbox providers confine bulk-export sources to that workspace. The copy
|
||||
// refuses symlinks, so the confined export can only carry these regular files;
|
||||
// Core still verifies every byte against the manifest digests after download.
|
||||
const stageExportScript = [
|
||||
"set -eu",
|
||||
"umask 077",
|
||||
'parent=$1; target=$2; shift 2',
|
||||
'mkdir -p -- "$parent"',
|
||||
'{ [ -d "$parent" ] && [ ! -L "$parent" ]; } || exit 41',
|
||||
'mkdir -- "$target"',
|
||||
"for name in nodeCommand opencodeProxy opencodeExecutable; do",
|
||||
' source=$1; shift',
|
||||
' { [ -f "$source" ] && [ ! -L "$source" ]; } || exit 42',
|
||||
' cp -P -- "$source" "$target/$name"',
|
||||
' { [ -f "$target/$name" ] && [ ! -L "$target/$name" ]; } || exit 43',
|
||||
"done",
|
||||
].join("\n");
|
||||
const removeExportScript = 'set -eu\nrm -rf -- "$1"';
|
||||
|
||||
async function runExportCommand(runner: CommandManagedRuntimeRunner, script: string, args: string[]) {
|
||||
const result = await runner.execute({
|
||||
command: "sh",
|
||||
args: ["-c", script, "sh", ...args],
|
||||
bypassSession: true,
|
||||
timeoutMs: 120_000,
|
||||
});
|
||||
if (result.exitCode !== 0 || result.timedOut) {
|
||||
throw new Error("native_provider_artifacts_remote_unavailable");
|
||||
}
|
||||
}
|
||||
|
||||
export async function captureRemoteNativeProviderArtifacts<T>(
|
||||
input: {
|
||||
runner: CommandManagedRuntimeRunner;
|
||||
/** Workspace remote dir: the only root a sandbox bulk export may read. */
|
||||
remoteCwd: string;
|
||||
launch: OpenCodeLaunchArtifacts;
|
||||
manifest: NativeProviderArtifactSource["manifest"];
|
||||
signal: AbortSignal;
|
||||
@@ -22,8 +59,15 @@ export async function captureRemoteNativeProviderArtifacts<T>(
|
||||
const root = await mkdtemp(join(tmpdir(), "paperclip-provider-launch-"));
|
||||
let ownedByOperation = false;
|
||||
try {
|
||||
const exportParent = posix.join(
|
||||
input.remoteCwd,
|
||||
".paperclip-runtime",
|
||||
"paperclip-runner",
|
||||
"provider-artifact-export",
|
||||
);
|
||||
const exportRoot = posix.join(exportParent, randomUUID());
|
||||
const files = await Promise.all(
|
||||
(["nodeCommand", "opencodeProxy", "opencodeExecutable"] as const).map(
|
||||
names.map(
|
||||
async (name) => {
|
||||
const targetPath = join(
|
||||
root,
|
||||
@@ -31,7 +75,7 @@ export async function captureRemoteNativeProviderArtifacts<T>(
|
||||
);
|
||||
await mkdir(dirname(targetPath), { recursive: true, mode: 0o700 });
|
||||
return {
|
||||
sourcePath: input.launch.paths[name],
|
||||
sourcePath: posix.join(exportRoot, name),
|
||||
targetPath,
|
||||
kind: "file" as const,
|
||||
mode: 0o500,
|
||||
@@ -46,9 +90,24 @@ export async function captureRemoteNativeProviderArtifacts<T>(
|
||||
.then(async () => {
|
||||
input.signal.throwIfAborted();
|
||||
input.context?.assertOpen();
|
||||
await input.runner.syncOut!([
|
||||
{ operationId: `provider-artifacts-${randomUUID()}`, files },
|
||||
]);
|
||||
try {
|
||||
await runExportCommand(input.runner, stageExportScript, [
|
||||
exportParent,
|
||||
exportRoot,
|
||||
...names.map((name) => input.launch.paths[name]),
|
||||
]);
|
||||
input.signal.throwIfAborted();
|
||||
input.context?.assertOpen();
|
||||
await input.runner.syncOut!([
|
||||
{ operationId: `provider-artifacts-${randomUUID()}`, files },
|
||||
]);
|
||||
} finally {
|
||||
// The export copy is scratch under the excluded runtime dir. A failed
|
||||
// removal must not replace the transfer outcome; it is logged instead.
|
||||
await runExportCommand(input.runner, removeExportScript, [exportRoot]).catch((err) => {
|
||||
logger.warn({ err }, "native provider artifact export cleanup failed");
|
||||
});
|
||||
}
|
||||
input.signal.throwIfAborted();
|
||||
input.context?.assertOpen();
|
||||
return publish({
|
||||
|
||||
@@ -2274,6 +2274,14 @@ describe("remote provider pack manifest", () => {
|
||||
expect(managedBackend.providerAdmissionRequired?.()).toBe(true);
|
||||
await managedBackend.providerAdmissionPersisted!();
|
||||
expect(providerArtifactCapture.captureRemoteNativeProviderArtifacts).toHaveBeenCalledTimes(1);
|
||||
expect(providerArtifactCapture.captureRemoteNativeProviderArtifacts).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ remoteCwd: "/workspace" }), expect.any(Function));
|
||||
// Sandbox providers confine export sources to the workspace remote dir;
|
||||
// the home-rooted launch files leave through a workspace staging copy.
|
||||
const exportSources = (syncOut.mock.calls as unknown as Array<[Array<{ files: Array<{ kind: string; sourcePath: string }> }>]>)
|
||||
.flatMap(([operations]) => operations.flatMap(operation => operation.files.map(file => file.sourcePath)));
|
||||
expect(exportSources.length).toBeGreaterThan(0);
|
||||
expect(exportSources.every(source => source.startsWith("/workspace/"))).toBe(true);
|
||||
expect(providerArtifactStorage.retainNativeProviderArtifacts).toHaveBeenCalledWith(expect.objectContaining({
|
||||
owner: expect.objectContaining({ runId: remoteExecution.binding.runId }), source: expect.objectContaining({ manifest: installed, launchRoot, accountUid: 1000 }) }));
|
||||
await driver.mock.calls.at(-1)![0].checkpoint!("settled");
|
||||
|
||||
@@ -16212,8 +16212,9 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
}
|
||||
if (!openCodeLaunchArtifacts) throw new Error("native_provider_artifacts_admission_missing");
|
||||
const signal = providerArtifactsSignal();
|
||||
if (!remoteTarget) throw new Error("native_provider_artifacts_admission_missing");
|
||||
providerArtifacts = await captureRemoteNativeProviderArtifacts({ runner: remoteCommandRunner,
|
||||
launch: openCodeLaunchArtifacts, manifest: expectedProviderPackManifest, signal, context: input.executionContext },
|
||||
remoteCwd: remoteTarget.remoteCwd, launch: openCodeLaunchArtifacts, manifest: expectedProviderPackManifest, signal, context: input.executionContext },
|
||||
source => retainNativeProviderArtifacts({ db: input.db, storage: artifactStorage, owner: backupOwner, source, signal }));
|
||||
} : undefined,
|
||||
bindDiagnosticSession: session => {
|
||||
|
||||
Reference in new issue
Block a user