Files
PaperClipAI/packages/adapter-utils/src/remote-managed-runtime.ts
T
DottaandPaperclip 4e00818574 fix(runtime): support in-place workspace realization (#10230)
## Thinking Path

> - Paperclip is the control plane people use to coordinate AI agents
and their execution environments.
> - Environment realization decides where an agent runs and which
filesystem and toolchain are authoritative.
> - Copy-based realization is unsafe for container-anchored tasks
because absolute paths such as `/app` can point outside the synchronized
tree and task-specific binaries may be absent.
> - That mismatch can let an agent successfully verify work in a phantom
writable path while sync-back silently discards the result.
> - Existing task environments already provide the authoritative
filesystem and toolchain, so they should be executed in place rather
than copied.
> - Copy mode still needs explicit confinement rules so aliases target
the synchronized workspace and unsynchronized writable paths fail
visibly.
> - This pull request adds typed realization metadata, propagates the
authoritative root through orchestration, and teaches Codex to honor it.
> - The benefit is that container-anchored tasks operate on
verifier-visible state with the intended tools, while copy mode remains
safe and backward compatible.

## Linked Issues or Issue Description

No public GitHub issue exists for this defect.

GitHub duplicate searches for in-place execution, workspace realization,
and authoritative workspace roots found no related pull request to link.

### What happened?

Environment-backed agent runs were always realized through a copied
workspace. Tasks anchored to absolute container paths could therefore
write outside the synchronized tree, and task-provided toolchains were
unavailable in the copy. A run could report success even though
sync-back discarded its output.

### Expected behavior

Existing task environments should run against their real authoritative
root and toolchain. Copy-mode runs should map declared absolute aliases
into the synchronized tree and reject writable paths that cannot be
restored.

### Steps to reproduce

1. Run a Codex task environment whose required files live under `/app`
or `/workspace` and whose required binary exists only in the task
container.
2. Observe that copy realization changes the effective
filesystem/toolchain or permits writes outside the synchronized root.
3. Complete and verify the task inside the agent sandbox.
4. Observe that the verifier cannot see out-of-tree artifacts or that
task-specific commands were unavailable.

### Reproduction context

- Paperclip commit: `3a16b91217483d2c233926de5b7f7bc3a1077924`
- Deployment: built from source in a task-container execution
environment
- Adapter: Codex local
- Database: not database-related
- Access context: agent execution

## What Changed

- Added typed `copy | in_place` workspace-realization metadata,
authoritative roots, confined aliases, and outbound restore paths to
shared execution-target contracts.
- Selected in-place realization for existing task environments and
skipped archive prepare/restore when the authoritative environment is
used directly.
- Propagated the authoritative root into adapter context so Codex uses
it for cwd and `PAPERCLIP_WORKSPACE_*` semantics, including ACP
execution.
- Bound copy-mode aliases such as `/app` to the synchronized workspace
and rejected writable out-of-tree paths without explicit restore
mappings.
- Added focused regression coverage while preserving existing copy-mode
archive restore behavior.

## Verification

- `pnpm --filter @paperclipai/shared typecheck` — passed.
- `pnpm exec vitest run
packages/adapter-utils/src/local-process-sandbox.test.ts
packages/adapters/codex-local/src/server/acp.test.ts
packages/adapters/codex-local/src/server/execute.remote.test.ts
server/src/__tests__/environment-run-orchestrator.test.ts` — 48 passed,
4 skipped.
- `pnpm -r typecheck` — passed.
- `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY -u
AWS_SESSION_TOKEN pnpm test:run` — passed across all general and
serialized Vitest shards.
- `pnpm build` — passed.
- Codex `k=1` acceptance run completed July 24, 2026 at 23:54:30 UTC
with 4 completed, 0 exceptions, and mean reward 1.0: `build-cython-ext`,
`openssl-selfsigned-cert`, `prove-plus-comm`, and `sqlite-db-truncate`
each received terminal grade 1.0 against real task-environment paths and
toolchains.

## Risks

- In-place mode deliberately exposes the authoritative task root to the
adapter; incorrect environment metadata could point execution at the
wrong root. Typed metadata and focused orchestration tests cover
selection and propagation.
- Copy-mode writable-path validation is stricter and may reject
previously accepted unsafe configurations. The rejection is intentional
and produces a visible error instead of silently losing output.
- The acceptance run is focused on four Codex task-environment
workloads, not a broad cross-adapter benchmark. Existing copy-mode
archive tests and the full repository suite remain green.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex CLI coding agent; exact model ID and context-window size
were not exposed to this runtime. Capabilities used: extended reasoning,
repository editing, shell execution, test/build execution, Git, GitHub
CLI, and Paperclip API tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-25 08:29:26 -05:00

178 lines
5.7 KiB
TypeScript

import path from "node:path";
import { GIT_ARCHIVE_EXCLUDES } from "./git-workspace-sync.js";
import {
type SshRemoteExecutionSpec,
prepareWorkspaceForSshExecution,
runSshCommand,
restoreWorkspaceFromSshExecution,
syncDirectoryToSsh,
} from "./ssh.js";
import type { SandboxManagedRuntimeAssetRestoreContext } from "./sandbox-managed-runtime.js";
import { captureDirectorySnapshot } from "./workspace-restore-merge.js";
import type { RuntimeProgressSink } from "./runtime-progress.js";
export interface RemoteManagedRuntimeAsset {
key: string;
localDir: string;
followSymlinks?: boolean;
exclude?: string[];
restore?: (ctx: SandboxManagedRuntimeAssetRestoreContext) => Promise<void>;
}
export interface PreparedRemoteManagedRuntime {
spec: SshRemoteExecutionSpec;
workspaceLocalDir: string;
workspaceRemoteDir: string;
runtimeRootDir: string;
assetDirs: Record<string, string>;
restoreWorkspace(onProgress?: RuntimeProgressSink): Promise<void>;
}
function asObject(value: unknown): Record<string, unknown> {
return value && typeof value === "object" && !Array.isArray(value)
? (value as Record<string, unknown>)
: {};
}
function asString(value: unknown): string {
return typeof value === "string" ? value : "";
}
function asNumber(value: unknown): number {
return typeof value === "number" ? value : Number(value);
}
function shellQuote(value: string): string {
return `'${value.replace(/'/g, `'"'"'`)}'`;
}
async function readRemoteFile(spec: SshRemoteExecutionSpec, remotePath: string): Promise<Buffer> {
const result = await runSshCommand(spec, `base64 < ${shellQuote(remotePath)}`, {
maxBuffer: 1024 * 1024,
});
return Buffer.from(result.stdout.replace(/\s+/g, ""), "base64");
}
export function buildRemoteExecutionSessionIdentity(spec: SshRemoteExecutionSpec | null) {
if (!spec) return null;
return {
transport: "ssh",
host: spec.host,
port: spec.port,
username: spec.username,
remoteCwd: spec.remoteCwd,
} as const;
}
export function remoteExecutionSessionMatches(saved: unknown, current: SshRemoteExecutionSpec | null): boolean {
const currentIdentity = buildRemoteExecutionSessionIdentity(current);
if (!currentIdentity) return false;
const parsedSaved = asObject(saved);
return (
asString(parsedSaved.transport) === currentIdentity.transport &&
asString(parsedSaved.host) === currentIdentity.host &&
asNumber(parsedSaved.port) === currentIdentity.port &&
asString(parsedSaved.username) === currentIdentity.username &&
asString(parsedSaved.remoteCwd) === currentIdentity.remoteCwd
);
}
export async function prepareRemoteManagedRuntime(input: {
spec: SshRemoteExecutionSpec;
runId: string;
adapterKey: string;
workspaceLocalDir: string;
workspaceRemoteDir?: string;
syncWorkspace?: boolean;
assets?: RemoteManagedRuntimeAsset[];
// Upload progress sink. Threaded for the byte-counting transport rewrite; the
// child task wires it into the workspace/asset transfers.
onProgress?: RuntimeProgressSink;
}): Promise<PreparedRemoteManagedRuntime> {
const baseWorkspaceRemoteDir = input.workspaceRemoteDir ?? input.spec.remoteCwd;
const syncWorkspace = input.syncWorkspace !== false;
const workspaceRemoteDir = syncWorkspace
? path.posix.join(
baseWorkspaceRemoteDir,
".paperclip-runtime",
"runs",
input.runId,
"workspace",
)
: baseWorkspaceRemoteDir;
const runtimeRootDir = path.posix.join(workspaceRemoteDir, ".paperclip-runtime", input.adapterKey);
const preparedWorkspace = syncWorkspace
? await prepareWorkspaceForSshExecution({
spec: input.spec,
localDir: input.workspaceLocalDir,
remoteDir: workspaceRemoteDir,
onProgress: input.onProgress,
})
: null;
const baselineSnapshot = preparedWorkspace
? await captureDirectorySnapshot(input.workspaceLocalDir, {
exclude: preparedWorkspace.gitBacked
? [...GIT_ARCHIVE_EXCLUDES, ".paperclip-runtime"]
: [".paperclip-runtime"],
})
: null;
const assetDirs: Record<string, string> = {};
try {
for (const asset of input.assets ?? []) {
const remoteDir = path.posix.join(runtimeRootDir, asset.key);
assetDirs[asset.key] = remoteDir;
await syncDirectoryToSsh({
spec: input.spec,
localDir: asset.localDir,
remoteDir,
followSymlinks: asset.followSymlinks,
exclude: asset.exclude,
onProgress: input.onProgress,
progressLabel: asset.key,
});
}
} catch (error) {
if (preparedWorkspace && baselineSnapshot) {
await restoreWorkspaceFromSshExecution({
spec: input.spec,
localDir: input.workspaceLocalDir,
remoteDir: workspaceRemoteDir,
baselineSnapshot,
restoreGitHistory: preparedWorkspace.gitBacked,
onProgress: input.onProgress,
});
}
throw error;
}
return {
spec: input.spec,
workspaceLocalDir: input.workspaceLocalDir,
workspaceRemoteDir,
runtimeRootDir,
assetDirs,
restoreWorkspace: async (onProgress?: RuntimeProgressSink) => {
if (preparedWorkspace && baselineSnapshot) {
await restoreWorkspaceFromSshExecution({
spec: input.spec,
localDir: input.workspaceLocalDir,
remoteDir: workspaceRemoteDir,
baselineSnapshot,
restoreGitHistory: preparedWorkspace.gitBacked,
onProgress,
});
}
for (const asset of input.assets ?? []) {
if (!asset.restore) continue;
await asset.restore({
assetDir: path.posix.join(runtimeRootDir, asset.key),
readFile: (remotePath) => readRemoteFile(input.spec, remotePath),
});
}
},
};
}