Preserve immutable context during remote staging and resume

Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip committed 2026-10-15 09:27:02 -05:00
1 parent 698fc2f7de
commit 0670b85392
3 files changed
+131 -8

No files matched your search

@@ -132,6 +132,19 @@ The first new cloud campaign [38734513491](https://github.com/paperclipai/paperc
Budget remains $50 total with historical holds intact. The fresh fixture cap is now $2.30: $0.80 from the original safe reallocation, then $1.50 from unused allowance in held fixture `16cc3ada` while retaining its $1 pending reservation and $1 setup reserve. Donor remains hard-stopped and its agents paused. Receipts: `/private/tmp/claude-master-local-allocation-20261015.json` and `/private/tmp/claude-master-platform-allocation-20261015.json`. Fresh fixture observed usage is $0.688181200, pending/unpriced zero, leaving $1.611818800. Historical known rate-card estimate plus these runs is $3.742819800, excluding unknown charges. Per-run reservation remains $0.50; automatic paid retries and timer heartbeats remain disabled.
## Packaged acceptance and remote staging repair — 2026-10-15
Cloud producer [38738177651](https://github.com/paperclipai/paperclip/actions/runs/38738177651) passed all four native daemon builds and clean package consumers on source `bd74bf50596d099e138bc6824cf0230f1026bfe1`. Public package artifact 11976635617 has ZIP SHA256 `079a0e5277ce978ccbe8464e4413ddb7f9a537a76fd71da931e3ee355bbe4198`; all 18 package archives were independently verified and installed into `/private/tmp/claude-installed-bd74bf50-preview/consumer`. The [cloud image consumer](https://github.com/paperclipai/paperclip/actions/runs/38744991246) also passed. Image `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:41cde6894c649ff76a06ddb925f1be1945d5efc901b13a7e43d371427a205235` contains the exact producer's Linux x64 daemon and provider pack. No Docker or Rust build ran locally.
The installed Mac task/revision passed in ordinary New Agent UI with saved API key, Default model, high effort, and no runner override. Agent `766dddf5-1178-4cdf-81ee-ec542cfe0673` saved native ACPX Claude/Opus 5.5 with the experimental gate disabled. Preview http://127.0.0.1:3105/CLA/issues/CLA-3. Runs `ae8386b6-6a41-493c-a35c-a34bcd5feb7c` and `f79a151e-2267-4acd-ad81-6a34be25e1ff` succeeded across a controller restart with the same task/provider/backend session and high-effort acknowledgment. Independently downloaded v1 (114 bytes) SHA256 `f1da81caf11f90e80c3fe008adf3c1d6e180d38188f929dc8244edd64e4ff41a`; v2 (157 bytes) SHA256 `cae638f5f239806842b9ef1fa9694097e142035f803a89ff743696e558f5b3b2`. V1 remained unchanged. Settled API estimate $0.347345000. Report `/private/tmp/claude-v7-installed-mac-proof-20261015/acceptance.json` retains the state at acceptance; later company holds do not invalidate those settled runs.
The two subsequent ARM attempts failed before provider startup and remain pending accounting: `c2d8f460-381c-4764-ae04-4974b30739eb` (test-drive stripped inherited callback configuration) and `a6979058-b3d0-49ed-8c6d-159f507e3443` (read-only directory staging). Each retains a $0.50 hold. Callback configuration is now persisted through the instance's normal environment file. ARM uses a dedicated unprivileged account and private SSH listener, with the verified standard provider-pack installation; the system CLI and existing root listener are untouched. Root execution was rejected by Claude's ordinary permission check, reproduced without credentials or prompts.
Credential-free reproduction proved that BSD tar emits non-contiguous directory entries. GNU tar restored read-only directory permissions before later children arrived. The staging fix delays GNU directory restoration, omits macOS metadata, and verifies existing immutable context bytes and modes before reuse on follow-up. It keeps native provider sync behavior and durable checkpoint handling. Corrupt or symlinked targets do not become accepted cache hits. The regression failed before the fix; all 15 focused controls now pass with both GNU and BSD tar, including repeat staging and corrupt/symlink rejection. The actual patched function also passes twice on the unprivileged ARM host with zero provider calls; receipt `/private/tmp/claude-v7-arm-staging-fixed-20261015.json`. Remote live acceptance remains incomplete; no paid retry runs automatically.
Current allocation receipt `/private/tmp/claude-v7-remote-allocation-20261015.json` preserves the $50 aggregate ceiling and all historical holds. Known rate-card estimates total $4.090164800, excluding pending/unpriced charges and setup reservations. The current remote fixture has a $1.22 cap with its new $0.50 pending hold. Live subscription acceptance remains explicitly unverified under the user's API-key direction.
---
# Historical plan and evidence (retained)
@@ -22,7 +22,7 @@ import {
truncate,
writeFile,
} from "node:fs/promises";
import { execFileSync } from "node:child_process";
import { execFileSync, spawnSync } from "node:child_process";
import { tmpdir } from "node:os";
import { inspect } from "node:util";
import { join } from "node:path";
@@ -2518,6 +2518,98 @@ describe("remote provider checkpoint snapshots", () => {
});
describe("remote provider checkpoint restores", () => {
it("stages and verifies read-only context with non-contiguous archive entries", async () => {
const root = await mkdtemp(join(tmpdir(), "paperclip-readonly-restore-"));
const sourcePath = join(root, "source");
const targetPath = join(root, "target with 'quotes'");
try {
await mkdir(join(sourcePath, "references", "agents"), { recursive: true });
await writeFile(join(sourcePath, "SKILL.md"), "skill");
await writeFile(join(sourcePath, "references", "guide.md"), "guide");
await writeFile(join(sourcePath, "references", "agents", "coder.md"), "coder");
for (const path of [sourcePath, join(sourcePath, "references"), join(sourcePath, "references", "agents")]) {
await chmod(path, 0o555);
}
for (const path of [join(sourcePath, "SKILL.md"), join(sourcePath, "references", "guide.md"), join(sourcePath, "references", "agents", "coder.md")]) {
await chmod(path, 0o444);
}
const archive = execFileSync("tar", [
"--no-recursion", "--no-xattrs", "-czf", "-", "-C", sourcePath,
".", "./references", "./SKILL.md", "./references/agents",
"./references/guide.md", "./references/agents/coder.md",
], { env: { ...process.env, COPYFILE_DISABLE: "1" } });
const execute = vi.fn(async (command: { command: string; args: string[] }) => {
const result = spawnSync(command.command, command.args, {
input: archive.toString("base64"),
encoding: "utf8",
env: { ...process.env },
});
return { exitCode: result.status, timedOut: false, stdout: result.stdout, stderr: result.stderr };
});
for (let pass = 0; pass < 2; pass++) {
await stageRemoteRunnerDirectory({
target: { kind: "remote", transport: "ssh", remoteCwd: root } as never,
runner: { execute } as never,
sourcePath,
targetPath,
mode: 0o555,
});
}
expect(await readFile(join(targetPath, "references", "agents", "coder.md"), "utf8")).toBe("coder");
expect(await readFile(join(targetPath, "references", "guide.md"), "utf8")).toBe("guide");
expect(await readFile(join(targetPath, "SKILL.md"), "utf8")).toBe("skill");
expect((await lstat(join(targetPath, "references"))).mode & 0o777).toBe(0o500);
expect((await lstat(targetPath)).mode & 0o777).toBe(0o555);
const corruptedFile = join(targetPath, "references", "agents", "coder.md");
await chmod(corruptedFile, 0o600);
await writeFile(corruptedFile, "other");
await chmod(corruptedFile, 0o400);
await expect(stageRemoteRunnerDirectory({
target: { kind: "remote", transport: "ssh", remoteCwd: root } as never,
runner: { execute } as never,
sourcePath,
targetPath,
mode: 0o555,
})).rejects.toThrow("runner_remote_directory_staging_failed");
expect(await readFile(corruptedFile, "utf8")).toBe("other");
} finally {
for (const directory of [sourcePath, targetPath]) {
for (const path of [directory, join(directory, "references"), join(directory, "references", "agents")]) {
await chmod(path, 0o700).catch(() => undefined);
}
}
await rm(root, { recursive: true, force: true });
}
});
it("rejects context staging through a symlink", async () => {
const root = await mkdtemp(join(tmpdir(), "paperclip-context-symlink-"));
const sourcePath = join(root, "source");
const preservedPath = join(root, "preserved");
const targetPath = join(root, "target");
try {
await mkdir(sourcePath);
await mkdir(preservedPath);
await writeFile(join(preservedPath, "sentinel"), "preserved");
await symlink(preservedPath, targetPath);
const execute = vi.fn(async (command: { command: string; args: string[]; stdin?: string }) => {
const result = spawnSync(command.command, command.args, { input: command.stdin, encoding: "utf8" });
return { exitCode: result.status, timedOut: false, stdout: result.stdout, stderr: result.stderr };
});
await expect(stageRemoteRunnerDirectory({
target: { kind: "remote", transport: "ssh", remoteCwd: root } as never,
runner: { execute } as never,
sourcePath,
targetPath,
mode: 0o555,
})).rejects.toThrow("runner_remote_directory_staging_failed");
expect(await readFile(join(preservedPath, "sentinel"), "utf8")).toBe("preserved");
expect((await lstat(targetPath)).isSymbolicLink()).toBe(true);
} finally {
await rm(root, { recursive: true, force: true });
}
});
it("does not upload excluded Codex scratch trees or credentials", async () => {
const sourcePath = await mkdtemp(
join(tmpdir(), "paperclip-codex-restore-source-"),
@@ -10288,8 +10288,12 @@ export async function stageRemoteRunnerDirectory(input: {
stagingRoot = mkdtempSync(join(tmpdir(), "paperclip-runner-restore-"));
const archive = execFileSync(
"tar",
[...excludeArgs, "-czf", "-", "-C", input.sourcePath, "."],
{ encoding: "buffer", maxBuffer: 64 * 1024 * 1024 },
[...excludeArgs, "--no-xattrs", "-czf", "-", "-C", input.sourcePath, "."],
{
encoding: "buffer",
maxBuffer: 64 * 1024 * 1024,
env: { ...process.env, COPYFILE_DISABLE: "1" },
},
);
execFileSync("tar", ["-xzf", "-", "-C", stagingRoot], {
input: archive,
@@ -10317,14 +10321,28 @@ export async function stageRemoteRunnerDirectory(input: {
}
const archive = execFileSync(
"tar",
[...excludeArgs, "-czf", "-", "-C", input.sourcePath, "."],
{ encoding: "buffer", maxBuffer: 64 * 1024 * 1024 },
[...excludeArgs, "--no-xattrs", "-czf", "-", "-C", input.sourcePath, "."],
{
encoding: "buffer",
maxBuffer: 64 * 1024 * 1024,
env: { ...process.env, COPYFILE_DISABLE: "1" },
},
);
const escapedTarget = input.targetPath.replaceAll("'", "'\\''");
const compareImmutable = `const fs=require("node:fs"),p=require("node:path");function same(a,b){const x=fs.lstatSync(a),y=fs.lstatSync(b);if((x.mode&511)!==(y.mode&511))return false;if(x.isDirectory())return y.isDirectory()&&fs.readdirSync(a).every(n=>same(p.join(a,n),p.join(b,n)));return x.isFile()&&y.isFile()&&fs.readFileSync(a).equals(fs.readFileSync(b));}try{process.exit(same(process.argv[1],process.argv[2])?0:1)}catch{process.exit(1)}`;
const immutableStage = (input.mode & 0o222) === 0
? `staging=$(mktemp -d) && mkdir "$staging/compare" || exit 1; ` +
`trap 'find "$staging" -type d -exec chmod u+w {} +; rm -rf -- "$staging"' EXIT; ` +
`base64 -d > "$staging/archive" && ` +
`tar "$@" -xzf "$staging/archive" -C "$staging/compare" && ` +
`chmod ${input.mode.toString(8)} "$staging/compare" || exit 1; ` +
`if node -e '${compareImmutable}' "$staging/compare" '${escapedTarget}'; then exit 0; fi; ` +
`tar "$@" -xzf "$staging/archive" -C '${escapedTarget}'`
: `base64 -d | tar "$@" -xzf - -C '${escapedTarget}'`;
const script =
`umask 077; mkdir -p '${escapedTarget}' && ` +
`base64 -d | tar -xzf - -C '${escapedTarget}' && ` +
`chmod ${input.mode.toString(8)} '${escapedTarget}'`;
`case "$(tar --version 2>/dev/null)" in *"GNU tar"*) set -- --delay-directory-restore ;; *) set -- ;; esac; ` +
`umask 077; test ! -L '${escapedTarget}' && mkdir -p '${escapedTarget}' && ` +
`{ ${immutableStage}; } && chmod ${input.mode.toString(8)} '${escapedTarget}'`;
const result = await input.runner.execute({
command: "sh",
args: ["-c", script],