From fdf3e5baae9229235c308105f19f587d72e867a2 Mon Sep 17 00:00:00 2001 From: Dotta Date: Tue, 6 Oct 2026 08:44:01 -0500 Subject: [PATCH] Keep recovered agent-file transfer scratch outside native memory --- doc/agent-files.md | 8 +++ .../2026-10-02-pi-production-readiness.md | 59 ++++++++++++++++--- .../src/command-managed-runtime.ts | 5 +- .../adapter-utils/src/execution-target.ts | 3 + .../src/sandbox-managed-runtime.ts | 12 +++- .../__tests__/agent-directory-probe.test.ts | 46 +++++++++++++++ .../agent-directory-working-copies.test.ts | 7 ++- .../agent-directory-working-copies.ts | 22 ++++--- 8 files changed, 143 insertions(+), 19 deletions(-) diff --git a/doc/agent-files.md b/doc/agent-files.md index 0cfa6f58962..2dfec1d5ade 100644 --- a/doc/agent-files.md +++ b/doc/agent-files.md @@ -37,6 +37,14 @@ workspace: their independent agent copy therefore lives under the excluded `.paperclip-runtime/agent-files///` area. It is not included in task workspace sync, Git staging, or task deliverables. +Sandbox transfer scratch lives separately under +`.paperclip-runtime/paperclip-runner/agent-file-transfers///`. +Both native sync and the command fallback keep transfer files outside +`AGENT_HOME`, including recovery after controller restart. Cleanup removes +only the original materialization's agent directory and transfer scratch +through its owned lease. A later warm turn keeps that original materialization +identity; it does not grant collection or cleanup of another run's files. + Regular files (including binary bytes) and directories are supported, up to 100,000 entries (files and folders), 256 MiB per file and 2 GiB total. Symlinks, hardlinks, and special diff --git a/doc/plans/2026-10-02-pi-production-readiness.md b/doc/plans/2026-10-02-pi-production-readiness.md index 63df439ba5e..bcdea590a2d 100644 --- a/doc/plans/2026-10-02-pi-production-readiness.md +++ b/doc/plans/2026-10-02-pi-production-readiness.md @@ -5,6 +5,48 @@ set of release gates. Pi remains a candidate until every required gate passes. Do not expand this work to additional models, widgets, images, Cursor or Copilot qualification. The existing draft stack must be reviewed in dependency order. +## Restart transfer correction — 2026-10-06 + +The current source changes shipping inputs. The `f5024863e` packages and image +remain historical evidence; rebuild and freeze a new set before any live +qualification. No new paid attempt has run. Current qualified counts remain +Product 0/26 and Runner 0/7. + +A credential-free regression reproduces the fallback restore defect after +controller recovery: file bytes survive, but transfer creates a reserved +`.paperclip-runtime/agent-files` directory inside native `AGENT_HOME`. The +product's complete agent-directory probe rejects that directory. Transfer +scratch now lives in the original materialization's host runtime area, outside +native memory. Both staging and recovery use the same lease-confined path. +Owned cleanup removes that path with the corresponding agent copy. The strict +probe, byte checks and qualification graders are unchanged. The actual fallback +regression fails before this correction and passes afterward. This identifies +a product defect and matches the prior Linux watch calibration; it does not +regrade or identify the missing reason in the original v13 restart receipt. + +A stronger free byte calibration exercises the installed OpenRouter stream +parser, Pi agent loop, real RPC serializer, permission extension/bridge and +native write/read. A synthetic response split at every argument character +preserves 33 bytes and LF through each boundary. Its 32-byte negative control +stays 32 bytes and fails the exact grade. It makes no network requests or model +calls and does not qualify the model. The retained Mac and Linux run logs have +no native argument fields, so neither failed live input's byte count is known. +The earlier statement below about 33-byte native input is corrected to describe +the task specification. A live newline correction is still required. + +The initial database tests could not start because the owned dependency copy +omitted its pinned Postgres package's library-symlink postinstall. Restoring its +17 declared links fixes `initdb --version`; no binary, host setting, unrelated +service or shipping dependency changes. Preserve the failed test logs. Keep +rollout held until the new package/image set, all 33 cases, platform installs, +full checks and prerequisite reviews pass. No merge or release is authorized. + +All 92 memory/probe/cleanup tests and 146 sandbox transport controls pass. +Adapter-utils and server typechecks pass. Broad transport discovery also ran +stale compiled tests from `dist/`; its failures remain retained. The canonical +stable runner excludes `**/dist/**`; current source-only validation is pending. +New-head CI, full build/typecheck/tests and new distribution proof remain open. + ## Case corrections — 2026-10-05 The historical profile-14 proofs are **28/33: Product 21/26 and @@ -94,8 +136,9 @@ restart terminal snapshot's exact cause. A corrected live result is still needed Pi native definition v13 removes the competing bare nonce from the memory instructions. Its sole JSON content specification still decodes to 33 bytes with one final LF. Frozen native parser/validator/write/read preserve those bytes -without provider calls. The failed run's native input contained 33 bytes while -its native read returned 32; original managed content was not captured. No byte +without provider calls. The failed run's task specification contained 33 bytes; +its native input was not retained. Its native read returned 32; original managed +content was not captured. No byte conversion was found in the wrapper or public file API. The prompt correction addresses a copying ambiguity; it does not establish a product byte-loss cause. The missing-LF negative control and exact saved/readback assertions stay in place. @@ -329,13 +372,13 @@ local tests and release review are still required. | Gate | Acceptance evidence | Current result | | --- | --- | --- | -| Runtime identity and admission | Exact runtime/profile/model; verified effective thinking; startup below the 60-second admission limit; unsupported configuration fails before a prompt | Shipping runtime `f5024863e` passes normal profile-15 admission on ARM Mac and native Linux. Completed live runs confirm the frozen model and native low. Fresh Intel admission for these shipping bytes remains required. Profile-14 and earlier profile-15 results below are historical. | -| Live lifecycle | Pending native question survives controller restart in the original process; one creation and resolution; stale answer after provider death rejected; Stop retires owned processes; three warm turns keep process identity | Current Linux restart retains the same native question/run/turn/session/producer and the browser answer; the run succeeds, but its incomplete independent terminal evidence and failed canonical cleanup keep the case failed. Earlier `b012b3aeb` provider-death and native-question passes do not qualify `f5024863e`. | -| Product workflows | All 26 explicit Pi Product E2E cells (13 local, 13 Daytona); screenshots, public state, independent artifacts, terminal and cleanup evidence | Not qualified at `f5024863e`. Mac and latest Linux memory attempts fail exact saved bytes. Latest Linux restart fails independent terminal evidence. Both v13 Linux attempts fail canonical cleanup; owned children and controller are independently absent. V14 fixture calibration and 693 tests pass, but do not regrade those cases. Historical `0bd040093` has 21/26 passes, kept only as history. Fresh complete current-runtime coverage is required. | -| Runner protocol | Pi roster passes through the native packaged runner and authenticated mock control plane; lifecycle/denial/control cases remain distinct from Product tests | Current-runtime Runner qualification remains required. Historical `0bd040093` has 7/7 passes at private definitions `7b112ffe3`, with unchanged graders and zero automatic retries. Those passes retain their original source labels. | -| Installed distribution | Public CLI/server tars on ARM Mac, Intel Mac and Linux; normal Pi setup; exact Linux companion and immutable Daytona image imported without binary override | Normal ARM public install verifies 722 packages and 79,373 files at `f5024863e`. Normal native Linux install, companion import, Pi setup and admission pass at immutable candidate image `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:20c7fffddeee4298830ead1ea3f0b70819f61e549279588b605440d1155f7260`. No binary override is used. This image is not release-qualified. Fresh Intel proof remains required; earlier three-platform proofs stay historical. | +| Runtime identity and admission | Exact runtime/profile/model; verified effective thinking; startup below the 60-second admission limit; unsupported configuration fails before a prompt | New shipping source requires a rebuilt frozen package/image set and fresh profile-15 admission on ARM Mac, Intel Mac and Linux. Earlier `f5024863e` admission and live model/low proofs remain historical. | +| Live lifecycle | Pending native question survives controller restart in the original process; one creation and resolution; stale answer after provider death rejected; Stop retires owned processes; three warm turns keep process identity | The new fallback-transfer regression passes without changing strict probes or graders. The original v13 restart remains failed: provider success, incomplete independent terminal evidence and failed canonical cleanup. Fresh lifecycle qualification on the new artifacts is required. | +| Product workflows | All 26 explicit Pi Product E2E cells (13 local, 13 Daytona); screenshots, public state, independent artifacts, terminal and cleanup evidence | Current qualified count is 0/26. Missing LF remains unresolved. Original memory and restart failures, manifests and independently absent owned resources remain retained. V14 fixture controls and new source regressions do not qualify any live case. Run every explicit cell on the new frozen artifacts. | +| Runner protocol | Pi roster passes through the native packaged runner and authenticated mock control plane; lifecycle/denial/control cases remain distinct from Product tests | Current qualified count is 0/7. The seven historical passes at private definitions `7b112ffe3` retain their original runtime labels. Fresh packaged-runner qualification is required. | +| Installed distribution | Public CLI/server tars on ARM Mac, Intel Mac and Linux; normal Pi setup; exact Linux companion and immutable Daytona image imported without binary override | Rebuild the public CLI/server set, Linux companion and immutable image for the shipping correction. Verify normal ARM Mac, Intel Mac and Linux installs of those bytes. Earlier `f5024863e` ARM/Linux proofs and image remain historical; they cannot admit the new code. | | Governance and spend | Company isolation, human-only permission, duplicate/stale answers, Stop and budget hard stop; pricing estimates never become claimed bills | Historical 58 focused invariants and live pending-permission Stop keep their source labels. Fresh shipping qualification remains required. The dedicated key retains a $5 lifetime cap, zero BYOK usage and the $100 campaign ceiling. Final v14 read-only accounting observes $0.324966594 used and $4.675033406 remaining, with delayed settlement still possible. The key cap is not proof of Paperclip budget enforcement. | -| Integration and rollout | Review each prerequisite; final-head typecheck, tests, build and CI pass; no unresolved review; exact artifacts; rollback recorded | Full build and recursive typecheck pass at shipping runtime `f5024863e`. V14 fixture changes pass 693 tests and harness typecheck; latest-head CI and review must be recorded separately. The full local workspace test command remains failed. Prerequisite #14921's valid production-admission finding remains open. Complete current-runtime qualification and prerequisite disposition remain required. No merge or release is authorized. | +| Integration and rollout | Review each prerequisite; final-head typecheck, tests, build and CI pass; no unresolved review; exact artifacts; rollback recorded | 92 scoped memory/probe/cleanup tests, 146 sandbox transport tests and affected adapter-utils/server typechecks pass. Broad source-only tests and full current-source build/typecheck/tests remain pending. Previous 53 CI passes and Greptile 5/5 belong to `1eb327cd6`. New-head checks/review and the valid #14921 admission finding remain open. No merge or release is authorized. | ## Bounded execution diff --git a/packages/adapter-utils/src/command-managed-runtime.ts b/packages/adapter-utils/src/command-managed-runtime.ts index 9abfd933c9f..b46b2476f1c 100644 --- a/packages/adapter-utils/src/command-managed-runtime.ts +++ b/packages/adapter-utils/src/command-managed-runtime.ts @@ -538,6 +538,7 @@ export async function prepareCommandManagedRuntime(input: { adapterKey: string; workspaceLocalDir: string; workspaceRemoteDir?: string; + runtimeRootDir?: string; syncWorkspace?: boolean; workspaceInboundMode?: WorkspaceInboundMode; workspaceDurableSeed?: WorkspaceDurableSeedPaths; @@ -572,7 +573,7 @@ export async function prepareCommandManagedRuntime(input: { transport: "sandbox", provider: input.spec.providerKey ?? "sandbox", sandboxId: input.spec.leaseId ?? "managed", - remoteCwd: workspaceRemoteDir, + remoteCwd: input.spec.remoteCwd, timeoutMs, apiKey: null, }; @@ -605,6 +606,7 @@ export async function prepareCommandManagedRuntime(input: { adapterKey: input.adapterKey, workspaceLocalDir: input.workspaceLocalDir, workspaceRemoteDir, + runtimeRootDir: input.runtimeRootDir, syncWorkspace: input.syncWorkspace, workspaceInboundMode: input.workspaceInboundMode, workspaceDurableSeed: input.workspaceDurableSeed, @@ -649,6 +651,7 @@ export async function prepareCommandManagedRuntime(input: { adapterKey: input.adapterKey, workspaceLocalDir: input.workspaceLocalDir, workspaceRemoteDir, + runtimeRootDir: input.runtimeRootDir, syncWorkspace: input.syncWorkspace, workspaceInboundMode: input.workspaceInboundMode, workspaceDurableSeed: input.workspaceDurableSeed, diff --git a/packages/adapter-utils/src/execution-target.ts b/packages/adapter-utils/src/execution-target.ts index 6b118011d43..07c122bde7b 100644 --- a/packages/adapter-utils/src/execution-target.ts +++ b/packages/adapter-utils/src/execution-target.ts @@ -1431,6 +1431,8 @@ export async function prepareAdapterExecutionTargetRuntime(input: { workspaceLocalDir: string; timeoutSec?: number; workspaceRemoteDir?: string; + /** Sandbox transfer scratch, confined to the lease's reserved runtime tree. */ + runtimeRootDir?: string; syncWorkspace?: boolean; workspaceInboundMode?: WorkspaceInboundMode; workspaceDurableSeed?: WorkspaceDurableSeedPaths; @@ -1515,6 +1517,7 @@ export async function prepareAdapterExecutionTargetRuntime(input: { adapterKey: input.adapterKey, workspaceLocalDir: input.workspaceLocalDir, workspaceRemoteDir: input.workspaceRemoteDir, + runtimeRootDir: input.runtimeRootDir, syncWorkspace: input.syncWorkspace, workspaceInboundMode: input.workspaceInboundMode, workspaceDurableSeed: input.workspaceDurableSeed, diff --git a/packages/adapter-utils/src/sandbox-managed-runtime.ts b/packages/adapter-utils/src/sandbox-managed-runtime.ts index 10eeab606e4..bd5dba718ad 100644 --- a/packages/adapter-utils/src/sandbox-managed-runtime.ts +++ b/packages/adapter-utils/src/sandbox-managed-runtime.ts @@ -1094,6 +1094,9 @@ export async function prepareSandboxManagedRuntime(input: { client: SandboxManagedRuntimeClient; workspaceLocalDir: string; workspaceRemoteDir?: string; + /** Host-owned transfer scratch within the lease's reserved runtime tree. + * Persistent subdirectories can keep scratch outside their native file root. */ + runtimeRootDir?: string; syncWorkspace?: boolean; /** Selects authoritative host staging, exact durable-seed replay, or no-overwrite adoption. */ workspaceInboundMode?: WorkspaceInboundMode; @@ -1124,7 +1127,14 @@ export async function prepareSandboxManagedRuntime(input: { runtimeSpan?: RuntimeSpanRunner; }): Promise { const workspaceRemoteDir = input.workspaceRemoteDir ?? input.spec.remoteCwd; - const runtimeRootDir = path.posix.join(workspaceRemoteDir, ".paperclip-runtime", input.adapterKey); + const runtimeRootDir = input.runtimeRootDir ?? path.posix.join(workspaceRemoteDir, ".paperclip-runtime", input.adapterKey); + if (input.runtimeRootDir !== undefined) { + const reservedRoot = path.posix.join(input.spec.remoteCwd, ".paperclip-runtime"); + if (!path.posix.isAbsolute(runtimeRootDir) || path.posix.normalize(runtimeRootDir) !== runtimeRootDir + || runtimeRootDir.includes("\0") || runtimeRootDir.endsWith("/") || !runtimeRootDir.startsWith(`${reservedRoot}/`)) { + throw new Error("Transfer scratch must remain within the lease runtime tree"); + } + } // A workspace directory that does not exist on this host has nothing to // stage, no files for ignore rules to govern, and nothing to restore into — // callers that only stage credential assets (the adapter env tests) hand diff --git a/server/src/__tests__/agent-directory-probe.test.ts b/server/src/__tests__/agent-directory-probe.test.ts index 643f67144f2..01a7b4b0236 100644 --- a/server/src/__tests__/agent-directory-probe.test.ts +++ b/server/src/__tests__/agent-directory-probe.test.ts @@ -6,6 +6,7 @@ import { createRequire } from "node:module"; import childProcess, { execFileSync } from "node:child_process"; import { prepareAdapterExecutionTargetRuntime } from "@paperclipai/adapter-utils/execution-target"; import { captureDirectorySnapshot } from "@paperclipai/adapter-utils/workspace-restore-merge"; +import { prepareSandboxManagedRuntime } from "@paperclipai/adapter-utils/sandbox-managed-runtime"; import { agentDirectoryBaselineDigest, agentDirectoryProbeProgram, observeLocalAgentDirectory, probeAgentDirectory, retireAgentDirectoryTransferScratch, agentDirectoryTransferCleanupProgram } from "../services/agent-directory-probe.js"; describe("stable live agent directory observation", () => { @@ -98,6 +99,51 @@ describe("stable live agent directory observation", () => { expect(probeAgentDirectory(remote).digest).toBe(agentDirectoryBaselineDigest(baseline)); } finally { await runtime?.cleanupWorkspaceSnapshot?.(); fs.rmSync(remote, { recursive: true, force: true }); } }); + it("keeps restart fallback transfer scratch outside the native memory directory", async () => { + const remoteCwd = `${root}-remote`; + const agentHome = join(remoteCwd, ".paperclip-runtime", "agent-files", "agent", "original-run"); + const scratch = join(remoteCwd, ".paperclip-runtime", "paperclip-runner", "agent-file-transfers", "agent", "original-run"); + fs.mkdirSync(agentHome, { recursive: true }); + fs.cpSync(root, agentHome, { recursive: true }); + const target = { kind: "remote" as const, transport: "sandbox" as const, remoteCwd, + // A newly bound controller target may have no native sync hooks. Exercise + // the production command/base64 fallback rather than a transport mock. + runner: { execute: async (input: Parameters[0]) => ({ + stdout: execFileSync(input.command, input.args ?? [], { cwd: input.cwd, input: input.stdin, encoding: "utf8", + env: { PATH: process.env.PATH, ...input.env }, timeout: input.timeoutMs, maxBuffer: 32 * 1024 * 1024 }), + stderr: "", exitCode: 0, signal: null, timedOut: false, pid: null, startedAt: new Date().toISOString(), + }) } }; + let runtime: Awaited> | undefined; + try { + const baseline = await captureDirectorySnapshot(root); + runtime = await prepareAdapterExecutionTargetRuntime({ target, runId: "restart-transfer", adapterKey: "agent-files", + workspaceLocalDir: root, workspaceRemoteDir: agentHome, runtimeRootDir: scratch, + syncWorkspace: true, workspaceInboundMode: "adopt_remote", workspaceBaseline: baseline, + workspaceGitSnapshot: null, workspaceFileMode: "all", workspaceExclude: [".paperclip-runtime", ".paperclip-runtime/**"] }); + const memory = "0123456789abcdef0123456789abcdef\n"; + fs.writeFileSync(join(agentHome, "notes", "memory.txt"), memory); + await runtime.restoreWorkspace(); + expect(fs.readFileSync(join(root, "notes", "memory.txt"), "utf8")).toBe(memory); + // A transfer must not introduce a reserved path that the product's own + // full agent-directory probe rejects, including after controller recovery. + expect(() => probeAgentDirectory(agentHome)).not.toThrow(); + expect(fs.existsSync(join(agentHome, ".paperclip-runtime"))).toBe(false); + expect(runtime.runtimeRootDir).toBe(scratch); + expect(fs.readdirSync(scratch)).toEqual([]); + } finally { await runtime?.cleanupWorkspaceSnapshot?.(); fs.rmSync(remoteCwd, { recursive: true, force: true }); } + }); + it.each(["outside", "reserved-root", "relative", "traversal", "trailing-slash", "nul"])("rejects %s transfer scratch before filesystem work", async kind => { + const reserved = join(root, ".paperclip-runtime"); + const invalid = { + outside: `${root}-foreign/scratch`, "reserved-root": reserved, relative: ".paperclip-runtime/scratch", + traversal: `${reserved}/../../scratch`, "trailing-slash": `${reserved}/scratch/`, nul: `${reserved}/scratch\0`, + }[kind]!; + await expect(prepareSandboxManagedRuntime({ + spec: { transport: "sandbox", provider: "fixture", sandboxId: "owned", remoteCwd: root, apiKey: null }, + adapterKey: "agent-files", client: {} as never, workspaceLocalDir: root, runtimeRootDir: invalid, + })).rejects.toThrow("Transfer scratch must remain within the lease runtime tree"); + expect(fs.existsSync(reserved)).toBe(false); + }); it.each(["file", "extra-directory", "symlink", "race"])("fails closed on %s in transfer-owned scratch without deleting contents", kind => { const parent = join(root, ".paperclip-runtime"), scratch = join(parent, "agent-files"); fs.mkdirSync(scratch, { recursive: true }); diff --git a/server/src/__tests__/agent-directory-working-copies.test.ts b/server/src/__tests__/agent-directory-working-copies.test.ts index e7d2b32ee0d..693c37e023a 100644 --- a/server/src/__tests__/agent-directory-working-copies.test.ts +++ b/server/src/__tests__/agent-directory-working-copies.test.ts @@ -554,7 +554,8 @@ describe("persistent agent directories", () => { } expect(execute).toHaveBeenCalledTimes(2); expect(execute).toHaveBeenLastCalledWith(expect.objectContaining({ lease: expect.objectContaining({ id: leaseId, providerLeaseId: "original-sandbox" }), - command: "rm", args: ["-rf", "--", executionRoot], bypassSession: true })); + command: "rm", args: ["-rf", "--", executionRoot, + path.posix.join(remoteCwd, ".paperclip-runtime", "paperclip-runner", "agent-file-transfers", agentId, copy.runId)], bypassSession: true })); await copies.recoverCaptured(); expect(execute).toHaveBeenCalledTimes(2); await copies.release(companyId, copy.runId); @@ -854,6 +855,9 @@ describe("persistent agent directories", () => { await fs.mkdir(path.join(root, "build")); await fs.writeFile(path.join(root, "build", "personal.txt"), "cache-like names are still agent files"); const first = await prepare(); + const firstScratch = path.join(remoteCwd, ".paperclip-runtime", "paperclip-runner", "agent-file-transfers", agentId, first.runId); + expect(await fs.readdir(firstScratch)).toEqual([]); + await expect(fs.stat(path.join(first.executionRoot, ".paperclip-runtime"))).rejects.toMatchObject({ code: "ENOENT" }); expect(await fs.readFile(path.join(first.executionRoot, "build", "personal.txt"), "utf8")).toBe("cache-like names are still agent files"); await fs.mkdir(path.join(first.executionRoot, "notes")); await fs.writeFile(path.join(first.executionRoot, "notes", "bytes.bin"), Buffer.from([0, 128, 255])); @@ -863,6 +867,7 @@ describe("persistent agent directories", () => { expect((await execFile("git", ["-C", remoteCwd, "status", "--porcelain", "--untracked-files=all"])).stdout).toBe("?? task-only.txt\n"); expect((await copies.collectStopped({ companyId, runId: first.runId, target: executionTarget }))?.state).toBe("saved"); await copies.release(companyId, first.runId); + await expect(fs.stat(firstScratch)).rejects.toMatchObject({ code: "ENOENT" }); expect((await copies.get(companyId, first.runId))?.receipt?.baseline).toBeUndefined(); await expect(fs.stat(first.localRoot)).rejects.toMatchObject({ code: "ENOENT" }); await fs.rm(remoteCwd, { recursive: true }); diff --git a/server/src/services/agent-directory-working-copies.ts b/server/src/services/agent-directory-working-copies.ts index 249f88e01a4..3aea9266618 100644 --- a/server/src/services/agent-directory-working-copies.ts +++ b/server/src/services/agent-directory-working-copies.ts @@ -13,7 +13,7 @@ import { HttpError, conflict, notFound } from "../errors.js"; import type { AuthorizationActor } from "./authorization.js"; import type { EnvironmentRuntimeService } from "./environment-runtime.js"; import type { Environment, EnvironmentLease } from "@paperclipai/shared"; -import { agentDirectoryBaselineDigest, agentDirectoryProbeProgram, agentDirectoryTransferCleanupProgram, observeLocalAgentDirectory } from "./agent-directory-probe.js"; +import { agentDirectoryBaselineDigest, agentDirectoryProbeProgram, observeLocalAgentDirectory } from "./agent-directory-probe.js"; import { hasRemoteTerminationReceipt } from "./remote-execution-termination.js"; import { cachedAgentFileManifest, captureAgentFileCheckpoint, checkpointBaseline, checkpointSnapshot, type AgentFileManifest } from "./agent-file-checkpoints.js"; import { logger } from "../middleware/logger.js"; @@ -23,6 +23,10 @@ export class AgentDirectoryReuseInvalidatedError extends Error {} const completed = new Set(["saved", "unchanged", "resolved", "unavailable"]); const transports = new Map(); const key = (row: Pick) => `${row.companyId}:${row.runId}`; +function sandboxTransferRoot(row: Copy, remoteCwd: string): string { + const origin = typeof row.receipt?.materializationRunId === "string" ? row.receipt.materializationRunId : String(row.receipt?.directoryRunId ?? row.runId); + return path.posix.join(remoteCwd, ".paperclip-runtime", "paperclip-runner", "agent-file-transfers", row.agentId, origin); +} export function isAgentDirectoryCopy(row: Pick | null): boolean { return row?.receipt?.schema === AGENT_FILES_CONTRACT; } @@ -62,6 +66,10 @@ export function agentDirectoryWorkingCopyService(db: Db, get: (companyId: string } return prepareAdapterExecutionTargetRuntime({ target, runId: row.runId, adapterKey: "agent-files", workspaceLocalDir: row.localRoot, workspaceRemoteDir: row.executionRoot, + // Agent files are an independently observed native directory. Transfer + // scratch belongs to the host runtime, including the fallback selected + // while plugin capability discovery is cold after controller restart. + runtimeRootDir: sandboxTransferRoot(row, target.remoteCwd), syncWorkspace: true, workspaceInboundMode: recovering ? "adopt_remote" : undefined, workspaceBaseline: baseline(row), workspaceGitSnapshot: null, workspaceFileMode: "all", workspaceExclude: [".paperclip-runtime", ".paperclip-runtime/**"] }); @@ -198,11 +206,7 @@ export function agentDirectoryWorkingCopyService(db: Db, get: (companyId: string const runtime = await transport(row, input.target, false); transports.set(key(row), runtime); if (input.target.transport === "sandbox") { - if (runtime.runtimeRootDir !== path.posix.join(row.executionRoot, ".paperclip-runtime", "agent-files")) throw new Error("Agent directory transfer scratch path changed"); - const retired = await runAdapterExecutionTargetShellCommand(input.runId, input.target, - `node -e ${quote(agentDirectoryTransferCleanupProgram)} ${quote(row.executionRoot)}`, - { cwd: input.target.remoteCwd, env: {}, timeoutSec: 10 }); - if (retired.exitCode !== 0 || retired.timedOut) throw new Error("Agent directory transfer scratch could not be safely retired"); + if (runtime.runtimeRootDir !== sandboxTransferRoot(row, input.target.remoteCwd)) throw new Error("Agent directory transfer scratch path changed"); } } const observed = await observe(row, input.target).catch(() => null); @@ -416,7 +420,8 @@ export function agentDirectoryWorkingCopyService(db: Db, get: (companyId: string else if (!destroyed && row.processStoppedAt && completed.has(row.state) && cleanupTarget?.kind === "remote") { const expected = path.posix.join(cleanupTarget.remoteCwd, ".paperclip-runtime", "agent-files", row.agentId, typeof row.receipt?.materializationRunId === "string" ? row.receipt.materializationRunId : String(row.receipt?.directoryRunId ?? row.runId)); if (row.executionRoot !== expected) throw new Error("Agent directory cleanup path changed"); - const quoted = `'${expected.replaceAll("'", `'"'"'`)}'`; + const paths = cleanupTarget.transport === "sandbox" ? [expected, sandboxTransferRoot(row, cleanupTarget.remoteCwd)] : [expected]; + const quoted = paths.map(p => `'${p.replaceAll("'", `'"'"'`)}'`).join(" "); const remoteCleanupFailed = await runAdapterExecutionTargetShellCommand(row.runId, cleanupTarget, `rm -rf -- ${quoted}`, { cwd: cleanupTarget.remoteCwd, env: {}, timeoutSec: 15 }).then(result => result.exitCode !== 0 || result.timedOut, () => true); cleanupPending ||= remoteCleanupFailed; @@ -480,7 +485,8 @@ export function agentDirectoryWorkingCopyService(db: Db, get: (companyId: string if (!environment) return false; const remoteCwd = cleanup?.remoteCwd ?? lease.metadata?.remoteCwd; if (typeof remoteCwd !== "string" || row.executionRoot !== path.posix.join(remoteCwd, ".paperclip-runtime", "agent-files", row.agentId, typeof row.receipt?.materializationRunId === "string" ? row.receipt.materializationRunId : String(row.receipt?.directoryRunId ?? row.runId))) return false; - const result = await environmentRuntime.execute({ environment: environment as Environment, lease: lease as EnvironmentLease, command: "rm", args: ["-rf", "--", row.executionRoot], + const paths = environment.driver === "sandbox" ? [row.executionRoot, sandboxTransferRoot(row, remoteCwd)] : [row.executionRoot]; + const result = await environmentRuntime.execute({ environment: environment as Environment, lease: lease as EnvironmentLease, command: "rm", args: ["-rf", "--", ...paths], cwd: remoteCwd, env: {}, timeoutMs: 15_000, bypassSession: true }); return result.exitCode === 0 && !result.timedOut; }