Files
PaperClipAI/packages/adapter-utils/src/command-managed-runtime.ts
T
DottaandPaperclip 57e977be72 feat: integrate Pi 1.0 into the experimental Runner (#14921)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - The experimental Runner owns provider processes and durable
sessions.
> - Pi needs working task execution and human controls.
> - The five-PR stack must preserve changes already on master.
> - Each layer now carries the complete integrated source for a safe
sequential fallback.
> - This PR belongs to native GitHub stack #15602, ending at #14956.

## Linked Issues or Issue Description

Refs #14436, #14631, #14743 and #14956.

Ship Pi 1.0 through the experimental Paperclip Runner. The five PRs are
#14921, #14922, #14923, #14924 and #14956. The user authorized the
complete merge after checks pass. Existing `pi_local` execution is
unchanged. Accounting and wider provider/platform qualification remain
deferred.

## What Changed

- Recover missing final replies after workspace finalization changes
owners, using accepted-turn evidence without rerunning work or granting
external-chat publication.
- Preserve the admitted Pi instruction root across warm runs, while
retaining changed-root rejection.
- Give Pi a bounded 15-second default shutdown grace so stop, drain
acknowledgement and durable suspension can complete. Explicit deadlines
and other providers retain their existing behavior.
- Integrate the Pi 1.0 runtime and master contracts.
- Use Pi profile 22. Preserve explicit caller-selected models and exact
native thinking levels. Keep Pi's wrapper, helper, extension and
question/control behavior unchanged from the qualified profile-19
runtime.
- Preserve master's Dot lifecycle and consent fields, configured task
environment, status guards and current Codex/Claude dependency versions.
Cursor stays qualified. Copilot stays pending; profile 17 binds the
changed shared protocol validation sources.
- Exclude general AWS IAM credentials from Pi static/custom provider
bindings and selected task projections; preserve the provider-scoped
Bedrock bearer key. Profile 21 is retained as historical provenance.
Rust and cloud install probes use the current declaration.
- Patch bundled brace-expansion 5.0.9 to the exact official 5.0.12
payload. Pin the patch and complete runtime closures. Include the patch
in normal installed setup tooling. Keep the upstream Pi shrinkwrap as
provenance and permit only this exact security correction.
- Include current attestation files in the Docker build context. Keep
the repository lockfile unchanged from master. CI and private image
builds resolve manifest changes before their frozen installation.

## Verification

- Full local `pnpm -r typecheck` passes, including Runner Rust, server
and UI. Focused integration checks pass: 194 Runner
admission/environment tests, 63 profile/credential tests with one
expected skip, 152 Dot/UI configuration tests, and Pi transcript/notice
tests.
- Full local `pnpm build` passes on the final source.
- Fresh final-source checks pass: all 698 Rust workspace tests (32
binaries), 156 credential/profile/controller tests with one expected
skip, Runner TypeScript typecheck, and 20 package/setup/sandbox tests.
- The profile-21 Pi materializer passes on the native host with the
official pinned Node 24.21.0 and its npm. It verifies all 150 locked
packages, the patched dependency and the exact closure. Setup/package
bundle tests and UI token gates pass.
- The old hashes were reproduced for all three supported targets before
calculating the patched graph. New closure hashes are darwin-arm64
`282022db10150c6632b3444df421342e7d534bdf5d5fb1097a2e79d0625a2bcf`,
darwin-x64
`64e251e19009f755c0b04f73ce2138246faab71a961b0f13d75ebfcc34bef12e`, and
linux-x64
`713b1fdff42fb56a1518bdc084f181d70bee8ebadc3e4b1d76321ed9108c8410`.
Independent native platform execution is separate from graph identity
reproduction.
- Historical cloud qualification remains unchanged: all seven core cases
pass on shipping source `10dc43c9ec65d88c2f782d62afb296d09494f215`,
harness `1a4408a48cfb5a1f094a311141c257c92cd7a893`, image
`sha256:5b3a775b383591bda1b0c1889e509acc70ce7f37c53f09733c81d59037f02280`,
and accepted Sonnet 4.6/low fixture. All 215 canonical files and all
seven cleanup checks pass independent verification. These are profile-19
results and are not relabeled as fresh profile-22 runs.
- Current Pi digest:
`sha256:e92078bee3c23bec4100aa589013a44613d054cd686826534025d8019e9f39a9`.
[The readiness
plan](https://github.com/paperclipai/paperclip/blob/codex/pi-production-readiness/doc/plans/2026-10-02-pi-production-readiness.md)
preserves campaign and failed-attempt provenance.
- Merge only after every PR's current-head CI and fresh review pass.
Linux CI covers the full suites, build and browser tests. The local
embedded Postgres API-authority suite cannot start on this macOS/Node 26
host, so Linux CI must confirm that suite.

### Fresh profile-22 core qualification — 2026-10-08

All seven accepted core cases pass canonically on Pi profile 22, with
`openrouter/anthropic/claude-sonnet-4.6` and native-confirmed low
thinking. This model is a fixture; production accepts the caller's
explicit Pi provider/model.

Runtime/install source: `3241a992f2a7703e59e97ed0fd3e5d6405de4401`.
Frozen accepted harness: `1a4408a48cfb5a1f094a311141c257c92cd7a893`.
Immutable cloud image:
`ghcr.io/paperclipai/paperclip-daytona-runner@sha256:506f22db7edd78f37c0c40bec1cc084af1850455026dbf467194bfbb8fcef141`.
Pi digest:
`sha256:e92078bee3c23bec4100aa589013a44613d054cd686826534025d8019e9f39a9`.

[Hosted Linux image and clean-install
verification](https://github.com/paperclipai/paperclip/actions/runs/37868328023)
passes, including all 20 source-bound archives, normal CLI/Pi setup,
companion import and the production pack reader. This exact installation
source includes the latest master integration and the corrected Pi warm
instruction-root fence. Full local typecheck/build and current-head
hosted CI verify the final stack. All 13 focused real-root regressions
pass. The full local executor suite passed 662 tests; 15 database tests
could not start the Mac embedded PostgreSQL service. Hosted Linux CI
passes the full required verification and E2E checks. These fresh
results keep their own source identity; profile-19 results remain
historical.

| Core path | Canonical campaign | Retained archive SHA-256 |
| --- | --- | --- |
| File edit, validation, download and Done |
`pi-core22-replyfix-0-1791511228` | 23 files;
`a473e8603a3dd4737863291f8d3d1e392391f0b16d433c3e0e0e9d8baf7a97b0` |
| Pending question and controller restart |
`pi-core22-replyfix-1-1791511376` | 33 files;
`6b829c4eb74e1f32a89c692a4ae7130dbfc1c6d3cf13915effe2103d9e242c8e` |
| Three-turn session/process/workspace continuity |
`pi-core22-replyfix-2-1791511587` | 23 files;
`7a87021f8f9a3fdd3c58bb4467f8d82c635e3ea4795d6e75f144d9aa14818df8` |
| Four typed questions and browser reconnects |
`pi-core22-replyfix-3-1791511881` | 42 files;
`9e31755252be1f4f9cb0626c984c142d4d1ae5f5bee3a7af08444db8d12c280a` |
| Plan approval and completion | `pi-core22-replyfix-4-1791512031` | 22
files;
`a0383ce1aab38e7b5a25ce0e9dd3bebea5c037ebd96ae6b29dae19015da2ae2c` |
| Same-turn steering and permission denial |
`pi-core22-replyfix-5-1791512261` | 39 files;
`c929b8c7070f0b66aedc17e65ca46e6beab1e363926ac9f7e2a75fb250f05949` |
| Stop during pending permission | `pi-core22-replyfix-6-1791512390` |
33 files;
`7f0a58ae0f4d5bfc76149435f4e322537089c5bd16e7ffe9b5ad71f10a621a07` |

All 215 canonical files (28714587 bytes) are independently
hash-verified. All seven cleanup grades pass, with no owned runtime
process or temporary root after each case. Automatic retries are zero.
The owned cloud host stopped normally after retention. The prior
profile-22 warm attempt remains failed and separately retained: archive
SHA-256
`1e54eba5ec72b50cee1534b23d1d1d4f21a090006b8a64501ba70db972abfde5`. Its
original canonical classification is preserved. Diagnosis reproduced a
product bug comparing an agent-files root against an unset
checkpoint-only field. The fix stores the admitted physical root
separately from the adopted per-run collection capability. The real-root
regression fails before the fix and passes afterward, including
rejection of a changed physical root. Fixture, grader, model and all
seven accepted case IDs are unchanged; this fresh campaign tests
final-reply publication after file registration first. The intermediate
restart attempt also remains failed and retained: archive SHA-256
`5dcaefdf1d17cf4cd54fd4cf810f45e736667392339b8ce7caf08bb4e225277f`. Its
original canonical classification is preserved. Pi resumed, wrote the
verified answer and completed its task; exact runner suspension was
proven, but idle stop consumed about 5.2s and left under 3s for the
drain acknowledgement. The Pi-only default shutdown grace is now 15s,
preserving a full 5s drain round trip and a finite suspension reserve.
Explicit caller deadlines, other provider defaults, literal drain
receipts and exact suspension identity checks remain unchanged. The
timing regression fails before this correction and passes afterward; all
18 focused settlement tests and Runner typecheck pass. The final-source
file attempt is also preserved as failed (`candidate_failure`), archive
SHA-256
`db6767b6773ea618997927ac77bdb005a5ac81492c7b9c0ffbc900449f829bc9`.
Native edit, validation, exact downloadable artifact and Done/succeeded
all passed, and the exact final reply was durably recorded. A workspace
recovery owner completed before the live heartbeat reached presentation,
leaving that reply absent from task chat. Recovery now materializes only
a completed final reply from the accepted turn of an ordinary internal
Done task, preserving issue/run/contract binding, suppression,
external-chat authorization and same-run deduplication. The database
regression covers the generated file-preparation receipt, suppression,
unapproved external continuation and replay. Server typecheck and all 49
response-selection tests pass; hosted Linux verifies the database
regression because embedded PostgreSQL cannot start on this Mac.

The delayed-final-answer database regression passes on [the final
root-source Linux server
shard](https://github.com/paperclipai/paperclip/actions/runs/37868262553/job/113628594152),
alongside 1,108 passing tests. The first root Runner shard had one
unchanged durable-resume test exceed its 5-second timeout; the identical
top-source shard and the isolated exact test passed. One rerun of that
failed job and its required aggregate passed without source or test
changes. The original failed job log and the single-rerun receipt remain
retained.

### October 9 merge verification

Current merge head: `5a8fe63512a7166aaef5cf50065a25008aa8b44b`. All
current-head checks pass, including `ci / verify` and `ci / e2e`;
exact-head Greptile review is 5/5 with no unresolved threads. Current
master conflicts are resolved. The user authorized the maintainer
override of the code-owner review gate after these checks. The seven
retained live core cases remain bound to source
`3241a992f2a7703e59e97ed0fd3e5d6405de4401` and its recorded cloud image.

## Risks

- The security correction changes the dependency closure and profile
identity. Old sessions must reopen on the new profile. Exact identities
and credential bindings fail closed.
- The runner remains experimental and requires explicit selection.
Legacy Pi Local is unchanged. Caller model IDs pass through; the E2E
model is a fixture.
- Accounting and the broad platform/provider matrix remain deferred.
This merge does not publish a release or deploy a service.

## Model Used

OpenAI GPT-6 through Codex assisted with reasoning, repository
inspection, editing and tool use. The exact serving ID and context
window are not exposed in this session. Final live qualification uses Pi
1.0.0 with `openrouter/anthropic/claude-sonnet-4.6` and native-confirmed
low thinking.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-09 08:56:41 -05:00

670 lines
30 KiB
TypeScript

import { promises as fs } from "node:fs";
import { randomUUID } from "node:crypto";
import os from "node:os";
import path from "node:path";
import {
createTarballFromDirectory,
prepareSandboxManagedRuntime,
type PreparedSandboxManagedRuntime,
type SandboxAdditionalSource,
type SandboxManagedRuntimeAsset,
type SandboxManagedRuntimeClient,
type SandboxRemoteExecutionSpec,
type SandboxSyncOperation,
type SandboxSyncResult,
type WorkspaceDurableSeedPaths,
type WorkspaceInboundMode,
} from "./sandbox-managed-runtime.js";
import { preferredShellForSandbox, shellCommandArgs } from "./sandbox-shell.js";
import type { RunProcessResult } from "./server-utils.js";
import type { RuntimeProgressSink, RuntimeStatusSink } from "./runtime-progress.js";
import type { RuntimeSpanRunner } from "./acpx-engine/startup-timing.js";
import type { GitWorkspaceSnapshot } from "./git-workspace-sync.js";
import type { DirectorySnapshot } from "./workspace-restore-merge.js";
/**
* Input for a duplex channel open. The caller supplies only the command argument
* vector the sandbox runs as the channel child process. Element 0 is the program
* and the rest are its arguments. The runner adds the lease scope from its own
* closure. This type is separate from the worker manager's
* `DuplexChannelOpenInput`, which also carries the lease scope fields.
*/
export interface DuplexChannelOpenInput {
command: readonly string[];
}
/**
* A persistent bidirectional channel to one long-lived command in the sandbox.
* The caller writes raw input bytes, reads streamed output, and stops or closes
* the channel. This is the cross-layer channel type: the runner returns it, and
* the sandbox driver adapts the worker manager's host session to it.
*/
export interface CommandManagedDuplexChannel {
/** Writes raw input bytes to the channel. */
write(data: Uint8Array): void;
/** Registers the one data listener. The channel streams each raw byte chunk in order. */
onData(listener: (chunk: Uint8Array) => void): void;
/**
* Registers the one exit listener. The channel calls it one time with the exit.
* A numeric `exitCode` is a real process exit. `transportClosed` is true when the
* provider transport closed with no exit data, so a reader can tell a real
* process exit from a reason-less transport close.
*/
onExit(listener: (exit: { exitCode: number | null; transportClosed?: boolean }) => void): void;
/** Stops the child process. Safe to call more than one time. */
stop(): void;
/** Closes the channel and releases the route. Safe to call more than one time. */
close(): Promise<void>;
}
export interface CommandManagedRuntimeRunner {
/**
* True when the provider verified the concurrent-sync opt-in. A native runner
* carries the value from the effective capability snapshot
* (`concurrentSyncOperations`). The client copies it onto the prepared sync
* client only on the native path; the base64 fallback ignores it and always
* permits concurrency. The default is false, so an undeclared native provider
* never permits concurrent sync operations.
*/
allowConcurrentSyncOperations?: boolean;
/**
* True only when `execute({ stdin })` can surface useful in-flight progress
* for a single stdin-backed command. Provider-backed sandbox runners usually
* complete the entire RPC before returning, so they should leave this false
* and let the caller choose a chunked upload path when progress is requested.
*/
supportsSingleStreamStdinProgress?: boolean;
execute(input: {
command: string;
args?: string[];
cwd?: string;
env?: Record<string, string>;
stdin?: string;
timeoutMs?: number;
onLog?: (stream: "stdout" | "stderr", chunk: string) => Promise<void>;
onSpawn?: (meta: { pid: number; startedAt: string }) => Promise<void>;
/**
* Run this command through the lease's persistent session even when no run
* step is active. A sandbox provider opens the session on the first
* non-bypassed command; the ACP process session bridge sets this so the
* long-lived agent command streams its output through the session log
* stream. The default keeps the context-based session selection.
*/
useSession?: boolean;
/**
* Run this command outside the lease's persistent session even when a run
* step is active. The persistent session is a single serialized shell. In
* streamed mode the agent runs as one long-lived foreground command that
* holds the session for the whole run. The bridge control-plane execs
* (input delivery, output read, callback relay, and the queue/setup
* bookkeeping) must run concurrently with the agent, so they run as
* independent one-shot commands. On the session they queue behind the agent
* command that never returns — a permanent deadlock. An explicit bypass
* always wins over the context-based session selection and over
* `useSession`. The default keeps the context-based session selection.
*/
bypassSession?: boolean;
}): Promise<RunProcessResult>;
/**
* Optional native inbound file transfer. Present only when the sandbox
* provider advertises both `environmentSyncIn` and `environmentSyncOut`; the
* client exposes `syncIn`/`syncOut` only when BOTH are present, so the
* orchestrator either uses the native path for both directions or falls back
* to the base64 transport for both.
*/
syncIn?(operations: SandboxSyncOperation[]): Promise<SandboxSyncResult>;
/** Optional native outbound file transfer. See {@link syncIn}. */
syncOut?(operations: SandboxSyncOperation[]): Promise<SandboxSyncResult>;
/**
* Optional persistent duplex channel. Present only when the sandbox provider's
* effective capability grants `duplexCommandStream`. The runner opens one
* bidirectional channel to a long-lived command in the sandbox. The SSH runner
* and every provider without the capability omit the member, so a caller gates
* on its presence in the same style as {@link syncIn}/{@link syncOut}.
*
* HTTP/2 is the preferred transport. `queue_v1` is the soft-deprecated fallback.
*/
openDuplexChannel?(input: DuplexChannelOpenInput): Promise<CommandManagedDuplexChannel>;
}
export interface CommandManagedRuntimeSpec {
providerKey?: string | null;
shellCommand?: "bash" | "sh" | null;
leaseId?: string | null;
remoteCwd: string;
timeoutMs?: number | null;
}
export type CommandManagedRuntimeAsset = SandboxManagedRuntimeAsset;
function shellQuote(value: string) {
return `'${value.replace(/'/g, `'"'"'`)}'`;
}
function mergeRuntimeExcludes(entries: string[] | undefined): string[] {
return [...new Set([".paperclip-runtime", ...(entries ?? [])])];
}
// Largest base64 body we hand to the runner as a single stdin string. Normal
// multi-MB workspace/asset tarballs stay well under this and upload in one
// round-trip; anything larger uses the bounded chunked-append fallback so a
// runaway stdin string can't blow the runner/provider RPC limits.
const REMOTE_WRITE_SINGLE_STREAM_MAX_BASE64_BYTES = 96 * 1024 * 1024;
// Fallback chunk size (base64 bytes). Kept a multiple of 4 so each chunk is a
// self-contained base64 unit that decodes cleanly on its own.
const REMOTE_WRITE_FALLBACK_BASE64_CHUNK_SIZE = 4 * 1024 * 1024;
const REMOTE_WRITE_FALLBACK_DECODED_CHUNK_SIZE = (REMOTE_WRITE_FALLBACK_BASE64_CHUNK_SIZE / 4) * 3;
const REMOTE_READ_CHUNK_BYTES = REMOTE_WRITE_FALLBACK_DECODED_CHUNK_SIZE;
function base64EncodedLength(byteLength: number): number {
return Math.ceil(byteLength / 3) * 4;
}
function toBuffer(bytes: Buffer | Uint8Array | ArrayBuffer): Buffer {
if (Buffer.isBuffer(bytes)) return bytes;
if (bytes instanceof ArrayBuffer) return Buffer.from(bytes);
return Buffer.from(bytes.buffer, bytes.byteOffset, bytes.byteLength);
}
const FAILED_COMMAND_OUTPUT_TAIL_CHARS = 4_000;
function formatFailedCommandOutput(result: RunProcessResult): string {
const tail = (text: string): string => {
const trimmed = text.trim();
if (trimmed.length <= FAILED_COMMAND_OUTPUT_TAIL_CHARS) return trimmed;
return `...[truncated]\n${trimmed.slice(-FAILED_COMMAND_OUTPUT_TAIL_CHARS)}`;
};
const stderr = tail(result.stderr);
const stdout = tail(result.stdout);
const parts: string[] = [];
if (stderr.length > 0) parts.push(`stderr: ${stderr}`);
if (stdout.length > 0) parts.push(`stdout: ${stdout}`);
return parts.length > 0 ? `:\n${parts.join("\n")}` : "";
}
function requireSuccessfulResult(result: RunProcessResult, action: string): void {
if (result.exitCode === 0 && !result.timedOut) return;
const detail = formatFailedCommandOutput(result);
throw new Error(`${action} failed with exit code ${result.exitCode ?? "null"}${detail}`);
}
function bufferToArrayBuffer(buffer: Buffer): ArrayBuffer {
// Copy out of the (possibly pooled) Node Buffer so the ArrayBuffer we hand to
// the client transport owns exactly these bytes.
return buffer.buffer.slice(buffer.byteOffset, buffer.byteOffset + buffer.byteLength) as ArrayBuffer;
}
// Named builder (Security Condition C3): extract an uploaded tarball into its
// target directory as a clean destroy-then-replace, then remove the tarball.
// Every path is shell-quoted; the fallback NEVER concatenates untrusted asset
// keys / file names into the shell.
function buildSyncInExtractDirectoryCommand(input: { remoteTarPath: string; targetDir: string }): string {
return (
`rm -rf ${shellQuote(input.targetDir)} && ` +
`mkdir -p ${shellQuote(input.targetDir)} && ` +
`tar -xf ${shellQuote(input.remoteTarPath)} -C ${shellQuote(input.targetDir)} && ` +
`rm -f ${shellQuote(input.remoteTarPath)}`
);
}
// Named builder (C3): apply a POSIX mode to a placed file. Octal literal, quoted
// path; no interpolation of untrusted values.
function buildSyncInChmodCommand(input: { mode: number; targetPath: string }): string {
return `chmod ${(input.mode & 0o7777).toString(8)} ${shellQuote(input.targetPath)}`;
}
function buildUniqueStagingPath(input: { targetPath: string; suffix: string }): string {
return `${input.targetPath}${input.suffix}.${randomUUID()}`;
}
async function bestEffortRemoveRemotePath(client: SandboxManagedRuntimeClient, remotePath: string): Promise<void> {
await client.remove(remotePath).catch(() => undefined);
}
/**
* Host-side confinement guard for a sync operation's post-upload command `cwd`
* (Security Condition C2). Runs BEFORE any handoff — native delegation OR the
* generic fallback — so an out-of-root `cwd` is rejected fail-closed before a
* provider ever sees it. `cwd` (when present) MUST be an absolute POSIX path with
* no `..` segment, confined to (equal to or under) one of the operation's own
* file-mapping target paths. Commands with no `cwd` are unconstrained here and
* default to the runtime's stable command cwd at exec time.
*/
export function assertPostUploadCommandsConfined(operations: readonly SandboxSyncOperation[]): void {
for (const operation of operations) {
const commands = operation.postUploadCommands ?? [];
if (commands.length === 0) continue;
const targetRoots = operation.files.map((mapping) => path.posix.normalize(mapping.targetPath));
for (const command of commands) {
if (command.cwd == null) continue;
const raw = command.cwd;
if (!path.posix.isAbsolute(raw) || raw.split("/").includes("..")) {
throw new Error(`post-upload command cwd is not a confined absolute POSIX path: ${raw}`);
}
const normalized = path.posix.normalize(raw);
const within = targetRoots.some(
(root) => normalized === root || normalized.startsWith(`${root}/`),
);
if (!within) {
throw new Error(`post-upload command cwd escapes the operation's target root: ${raw}`);
}
}
}
}
export function createCommandManagedRuntimeClient(input: {
runner: CommandManagedRuntimeRunner;
commandCwd: string;
timeoutMs: number;
shellCommand?: "bash" | "sh" | null;
}): SandboxManagedRuntimeClient {
const shellCommand = preferredShellForSandbox(input.shellCommand);
const runShell = async (
script: string,
opts: {
stdin?: string;
timeoutMs?: number;
onLog?: (stream: "stdout" | "stderr", chunk: string) => Promise<void>;
} = {},
) => {
const result = await input.runner.execute({
command: shellCommand,
args: shellCommandArgs(script),
cwd: input.commandCwd,
stdin: opts.stdin,
timeoutMs: opts.timeoutMs ?? input.timeoutMs,
onLog: opts.onLog,
});
requireSuccessfulResult(result, script);
return result;
};
const client: SandboxManagedRuntimeClient = {
makeDir: async (remotePath) => {
await runShell(`mkdir -p ${shellQuote(remotePath)}`);
},
writeFile: async (remotePath, bytes, options) => {
const buffer = toBuffer(bytes);
const total = buffer.byteLength;
const encodedLength = base64EncodedLength(total);
const remoteDir = path.posix.dirname(remotePath);
const remoteTempPath = buildUniqueStagingPath({ targetPath: remotePath, suffix: ".paperclip-upload" });
const canUseSingleStreamProgressPath = input.runner.supportsSingleStreamStdinProgress === true;
try {
// Primary path: a single round-trip. Stream the entire base64 body to one
// `base64 -d` process via stdin, decode straight into a temp file, then
// atomically rename into place. This replaces the previous loop that did
// one `printf >> tmpfile` shell round-trip per 32 KB — thousands of serial
// processes for a large workspace — with exactly one process.
if (
encodedLength <= REMOTE_WRITE_SINGLE_STREAM_MAX_BASE64_BYTES &&
canUseSingleStreamProgressPath
) {
const body = buffer.toString("base64");
await options?.onProgress?.(0, total);
await runShell(
`cleanup() { rm -f ${shellQuote(remoteTempPath)}; }; trap cleanup EXIT INT TERM; ` +
`mkdir -p ${shellQuote(remoteDir)} && ` +
`base64 -d > ${shellQuote(remoteTempPath)} && ` +
`mv -f ${shellQuote(remoteTempPath)} ${shellQuote(remotePath)}`,
{ stdin: body },
);
await options?.onProgress?.(total, total);
return;
}
// Bounded fallback for payloads too large to hand the runner as one stdin
// string: append the base64 body to a remote temp file in large chunks
// (orders of magnitude fewer round-trips than the old 32 KB loop), decoding
// each self-contained chunk on arrival and emitting progress per write,
// then atomically rename into place.
await runShell(
`mkdir -p ${shellQuote(remoteDir)} && ` +
`rm -f ${shellQuote(remoteTempPath)} && : > ${shellQuote(remoteTempPath)}`,
);
for (let offset = 0; offset < total; offset += REMOTE_WRITE_FALLBACK_DECODED_CHUNK_SIZE) {
const end = Math.min(total, offset + REMOTE_WRITE_FALLBACK_DECODED_CHUNK_SIZE);
const chunk = buffer.subarray(offset, end).toString("base64");
await runShell(`base64 -d >> ${shellQuote(remoteTempPath)}`, { stdin: chunk });
await options?.onProgress?.(end, total);
}
await runShell(`mv -f ${shellQuote(remoteTempPath)} ${shellQuote(remotePath)}`);
await options?.onProgress?.(total, total);
} finally {
await bestEffortRemoveRemotePath(client, remoteTempPath);
}
},
readFile: async (remotePath, options) => {
// Chunked reads intentionally query the remote size first, even without
// a progress sink, so each sandbox RPC stays bounded and truncation is
// detected without materializing the whole file as one stdout string.
let sizeResult;
try {
sizeResult = await runShell(`wc -c < ${shellQuote(remotePath)}`);
} catch (error) {
// Shell-backed sandbox reads need the same absent-file contract as fs.
// Confirm the parent is searchable so permission/transport failures are
// never silently converted into a missing optional credential file.
const parent = shellQuote(path.posix.dirname(remotePath));
const missing = await runShell(
`if [ -d ${parent} ] && [ -x ${parent} ] && [ ! -e ${shellQuote(remotePath)} ]; ` +
`then printf 'missing'; fi`,
).catch(() => null);
if (missing?.stdout === "missing") {
throw Object.assign(new Error(`No such file: ${remotePath}`), {
code: "ENOENT",
path: remotePath,
});
}
throw error;
}
const totalBytes = Number.parseInt(sizeResult.stdout.trim(), 10);
if (!Number.isFinite(totalBytes) || totalBytes < 0) {
throw new Error(`Could not determine remote file size for ${remotePath}`);
}
// Read in bounded remote chunks so the runner never has to materialize a
// single base64 stdout string for the whole archive. The client API still
// returns the decoded file as a Buffer, but every command result stays
// small enough for provider-backed sandbox RPCs.
const decodedChunks: Buffer[] = [];
let decodedSoFar = 0;
if (totalBytes === 0) {
await options?.onProgress?.(0, 0);
return Buffer.alloc(0);
}
for (let chunkIndex = 0; decodedSoFar < totalBytes; chunkIndex++) {
const result = await runShell(
`dd if=${shellQuote(remotePath)} bs=${REMOTE_READ_CHUNK_BYTES} skip=${chunkIndex} count=1 2>/dev/null | base64`,
);
const chunk = Buffer.from(result.stdout.replace(/\s+/g, ""), "base64");
if (chunk.byteLength === 0) break;
decodedChunks.push(chunk);
decodedSoFar += chunk.byteLength;
await options?.onProgress?.(Math.min(decodedSoFar, totalBytes), totalBytes);
}
const out = Buffer.concat(decodedChunks);
if (out.byteLength !== totalBytes) {
throw new Error(`Remote file read was truncated for ${remotePath}: ${out.byteLength}/${totalBytes} bytes`);
}
await options?.onProgress?.(out.byteLength, totalBytes);
return out;
},
listFiles: async (remotePath) => {
const result = await runShell(
`if [ -d ${shellQuote(remotePath)} ]; then ` +
`for entry in ${shellQuote(remotePath)}/*; do ` +
`[ -f "$entry" ] || continue; ` +
`basename "$entry"; ` +
`done; ` +
`fi`,
);
return result.stdout
.split(/\r?\n/)
.map((entry) => entry.trim())
.filter((entry) => entry.length > 0)
.sort((left, right) => left.localeCompare(right));
},
remove: async (remotePath) => {
const result = await input.runner.execute({
command: shellCommand,
args: shellCommandArgs(`rm -rf ${shellQuote(remotePath)}`),
cwd: input.commandCwd,
timeoutMs: input.timeoutMs,
});
requireSuccessfulResult(result, `remove ${remotePath}`);
},
run: async (command, options) => {
const result = await input.runner.execute({
command: shellCommand,
args: shellCommandArgs(command),
cwd: input.commandCwd,
timeoutMs: options.timeoutMs,
});
requireSuccessfulResult(result, command);
},
};
// Generic base64-tar fallback for `syncIn` on runners without native sync:
// place each operation's files (host-side tarball → `writeFile` → destroy-then-
// replace untar for directories, direct `writeFile` for single files), then run
// the operation's ordered `postUploadCommands` fail-fast. Byte-for-byte
// behavior-equivalent to the caller-inlined tar path it will replace. All exec
// rides the shared `execute` seam.
const fallbackSyncIn = async (operations: SandboxSyncOperation[]): Promise<SandboxSyncResult> => {
const resultOperations: SandboxSyncResult["operations"] = [];
const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-syncin-fallback-"));
try {
for (const operation of operations) {
let filesTransferred = 0;
let bytesTransferred = 0;
for (const [index, mapping] of operation.files.entries()) {
const cleanupPaths: string[] = [];
try {
if (mapping.kind === "directory") {
const archivePath = path.join(tempDir, `syncin-${index}.tar`);
await createTarballFromDirectory({
localDir: mapping.sourcePath,
archivePath,
exclude: mapping.exclude,
followSymlinks: mapping.followSymlinks,
});
const tarBytes = await fs.readFile(archivePath);
const remoteTarPath = buildUniqueStagingPath({
targetPath: mapping.targetPath,
suffix: ".paperclip-syncin.tar",
});
cleanupPaths.push(remoteTarPath);
await client.writeFile(remoteTarPath, bufferToArrayBuffer(tarBytes));
await client.run(
buildSyncInExtractDirectoryCommand({ remoteTarPath, targetDir: mapping.targetPath }),
{ timeoutMs: input.timeoutMs },
);
bytesTransferred += tarBytes.byteLength;
} else {
const fileBytes = await fs.readFile(mapping.sourcePath);
await client.writeFile(mapping.targetPath, bufferToArrayBuffer(fileBytes));
if (mapping.mode != null) {
await client.run(
buildSyncInChmodCommand({ mode: mapping.mode, targetPath: mapping.targetPath }),
{ timeoutMs: input.timeoutMs },
);
}
bytesTransferred += fileBytes.byteLength;
}
} finally {
for (const cleanupPath of cleanupPaths.reverse()) {
await bestEffortRemoveRemotePath(client, cleanupPath);
}
}
filesTransferred += 1;
}
// Ordered, fail-fast post-upload commands. Each command string is
// executed VERBATIM — never rewritten, concatenated, or appended to.
// The first non-zero exit or timeout throws and stops the rest.
for (const command of operation.postUploadCommands ?? []) {
const result = await input.runner.execute({
command: shellCommand,
args: shellCommandArgs(command.command),
cwd: command.cwd ?? input.commandCwd,
timeoutMs: command.timeoutMs ?? input.timeoutMs,
});
requireSuccessfulResult(result, command.command);
}
resultOperations.push({
operationId: operation.operationId,
filesTransferred,
bytesTransferred,
});
}
} finally {
await fs.rm(tempDir, { recursive: true, force: true }).catch(() => undefined);
}
return { operations: resultOperations };
};
// `client.syncIn` is ALWAYS present: it delegates to the runner's native
// transport when the provider advertises BOTH sync verbs, otherwise it runs the
// generic fallback above. Either way, post-upload command `cwd` confinement (C2)
// is validated on the host BEFORE any handoff. `syncOut` stays native-only —
// there is no generic outbound fallback in this seam.
const nativeSyncIn = input.runner.syncIn;
const nativeSyncOut = input.runner.syncOut;
const hasNativeBoth = Boolean(nativeSyncIn && nativeSyncOut);
// The base64 fallback always permits concurrent sync operations. A native
// runner permits them only when the provider verified the opt-in; an
// undeclared native provider keeps concurrency off. One flag serves both sync
// directions.
client.allowConcurrentSyncOperations = hasNativeBoth
? input.runner.allowConcurrentSyncOperations === true
: true;
client.syncIn = async (operations) => {
assertPostUploadCommandsConfined(operations);
if (hasNativeBoth) {
return await nativeSyncIn!(operations);
}
return await fallbackSyncIn(operations);
};
if (hasNativeBoth) {
client.syncOut = (operations) => nativeSyncOut!(operations);
}
return client;
}
export async function prepareCommandManagedRuntime(input: {
runner: CommandManagedRuntimeRunner;
spec: CommandManagedRuntimeSpec;
adapterKey: string;
workspaceLocalDir: string;
workspaceRemoteDir?: string;
runtimeRootDir?: string;
syncWorkspace?: boolean;
workspaceInboundMode?: WorkspaceInboundMode;
workspaceDurableSeed?: WorkspaceDurableSeedPaths;
workspaceBaseline?: DirectorySnapshot;
workspaceGitSnapshot?: GitWorkspaceSnapshot | null;
workspaceExclude?: string[];
/** Plain persistent directories include all files, independent of Git and task cache exclusions. */
workspaceFileMode?: "all";
preserveAbsentOnRestore?: string[];
assets?: CommandManagedRuntimeAsset[];
/** Referenced (additional) projects to stage into the sandbox as plain, read-only trees. */
additionalSources?: SandboxAdditionalSource[];
installCommand?: string | null;
/** When provided alongside `installCommand`, skip the install if `command -v <detectCommand>` succeeds. */
detectCommand?: string | null;
// Upload progress sink. Forwarded to prepareSandboxManagedRuntime; the child
// task wires it into the byte-counting writeFile/readFile transport.
onProgress?: RuntimeProgressSink;
onRuntimeProgress?: RuntimeStatusSink;
// Optional host span runner for the workspace tarball build. Forwarded to
// prepareSandboxManagedRuntime so the host pack time rides one `pack` span
// under the `stage.sync` step. The default is a no-op.
runtimeSpan?: RuntimeSpanRunner;
}): Promise<PreparedSandboxManagedRuntime> {
const timeoutMs = input.spec.timeoutMs && input.spec.timeoutMs > 0 ? input.spec.timeoutMs : 300_000;
const workspaceRemoteDir = input.workspaceRemoteDir ?? input.spec.remoteCwd;
// Managed-runtime sync/restore scripts use absolute paths throughout, so
// run them from a stable cwd. The target workspace itself may be removed or
// recreated during a run, which breaks shell startup if we chdir into it.
const commandCwd = "/";
const runtimeSpec: SandboxRemoteExecutionSpec = {
transport: "sandbox",
provider: input.spec.providerKey ?? "sandbox",
sandboxId: input.spec.leaseId ?? "managed",
remoteCwd: input.spec.remoteCwd,
timeoutMs,
apiKey: null,
};
const client = createCommandManagedRuntimeClient({
runner: input.runner,
commandCwd,
timeoutMs,
shellCommand: input.spec.shellCommand,
});
const shellCommand = preferredShellForSandbox(input.spec.shellCommand);
if (input.installCommand?.trim()) {
const installCommand = input.installCommand.trim();
const detectCommand = input.detectCommand?.trim();
// Skip the install when the binary is already on PATH. Without this
// probe the install runs unconditionally on every execute() call (and
// also runs a second time after `ensureAdapterExecutionTargetCommandResolvable`
// has already installed it during the resolvability gate).
if (detectCommand) {
const probe = await input.runner.execute({
command: shellCommand,
args: shellCommandArgs(`command -v ${shellQuote(detectCommand)} >/dev/null 2>&1`),
cwd: commandCwd,
timeoutMs,
});
if (!probe.timedOut && (probe.exitCode ?? 1) === 0) {
return await prepareSandboxManagedRuntime({
spec: runtimeSpec,
client,
adapterKey: input.adapterKey,
workspaceLocalDir: input.workspaceLocalDir,
workspaceRemoteDir,
runtimeRootDir: input.runtimeRootDir,
syncWorkspace: input.syncWorkspace,
workspaceInboundMode: input.workspaceInboundMode,
workspaceDurableSeed: input.workspaceDurableSeed,
workspaceBaseline: input.workspaceBaseline,
workspaceGitSnapshot: input.workspaceGitSnapshot,
workspaceExclude: mergeRuntimeExcludes(input.workspaceExclude),
workspaceFileMode: input.workspaceFileMode,
preserveAbsentOnRestore: input.preserveAbsentOnRestore,
assets: input.assets,
additionalSources: input.additionalSources,
onProgress: input.onProgress,
onRuntimeProgress: input.onRuntimeProgress,
runtimeSpan: input.runtimeSpan,
});
}
}
const result = await input.runner.execute({
command: shellCommand,
args: shellCommandArgs(installCommand),
cwd: commandCwd,
timeoutMs,
});
// A failed install is not always fatal: the CLI may already be on PATH
// from a previous lease, the template image, or another path entry. Log
// and continue rather than aborting the agent run; downstream code that
// exec's the CLI will surface a clear "command not found" if it is in
// fact missing. The test path's `maybeRunSandboxInstallCommand` already
// honors this contract — keep them consistent.
if (result.timedOut || (result.exitCode ?? 0) !== 0) {
const tail = (text: string) =>
text.split(/\r?\n/).filter((line) => line.trim().length > 0).slice(-3).join(" | ").slice(0, 480);
const reason = result.timedOut ? "timed out" : `exited ${result.exitCode ?? "?"}`;
console.warn(
`[paperclip] managed-runtime install command ${reason}: ${installCommand} :: ${tail(result.stderr || result.stdout)}`,
);
}
}
return await prepareSandboxManagedRuntime({
spec: runtimeSpec,
client,
adapterKey: input.adapterKey,
workspaceLocalDir: input.workspaceLocalDir,
workspaceRemoteDir,
runtimeRootDir: input.runtimeRootDir,
syncWorkspace: input.syncWorkspace,
workspaceInboundMode: input.workspaceInboundMode,
workspaceDurableSeed: input.workspaceDurableSeed,
workspaceBaseline: input.workspaceBaseline,
workspaceGitSnapshot: input.workspaceGitSnapshot,
workspaceExclude: mergeRuntimeExcludes(input.workspaceExclude),
workspaceFileMode: input.workspaceFileMode,
preserveAbsentOnRestore: input.preserveAbsentOnRestore,
assets: input.assets,
additionalSources: input.additionalSources,
onProgress: input.onProgress,
onRuntimeProgress: input.onRuntimeProgress,
runtimeSpan: input.runtimeSpan,
});
}