mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-04 19:28:46 +01:00
Preserve immutable context during remote staging and resume
Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
698fc2f7de
commit
0670b85392
3 files changed
+131
-8
No files matched your search
@@ -132,6 +132,19 @@ The first new cloud campaign [38734513491](https://github.com/paperclipai/paperc
|
||||
|
||||
Budget remains $50 total with historical holds intact. The fresh fixture cap is now $2.30: $0.80 from the original safe reallocation, then $1.50 from unused allowance in held fixture `16cc3ada` while retaining its $1 pending reservation and $1 setup reserve. Donor remains hard-stopped and its agents paused. Receipts: `/private/tmp/claude-master-local-allocation-20261015.json` and `/private/tmp/claude-master-platform-allocation-20261015.json`. Fresh fixture observed usage is $0.688181200, pending/unpriced zero, leaving $1.611818800. Historical known rate-card estimate plus these runs is $3.742819800, excluding unknown charges. Per-run reservation remains $0.50; automatic paid retries and timer heartbeats remain disabled.
|
||||
|
||||
|
||||
## Packaged acceptance and remote staging repair — 2026-10-15
|
||||
|
||||
Cloud producer [38738177651](https://github.com/paperclipai/paperclip/actions/runs/38738177651) passed all four native daemon builds and clean package consumers on source `bd74bf50596d099e138bc6824cf0230f1026bfe1`. Public package artifact 11976635617 has ZIP SHA256 `079a0e5277ce978ccbe8464e4413ddb7f9a537a76fd71da931e3ee355bbe4198`; all 18 package archives were independently verified and installed into `/private/tmp/claude-installed-bd74bf50-preview/consumer`. The [cloud image consumer](https://github.com/paperclipai/paperclip/actions/runs/38744991246) also passed. Image `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:41cde6894c649ff76a06ddb925f1be1945d5efc901b13a7e43d371427a205235` contains the exact producer's Linux x64 daemon and provider pack. No Docker or Rust build ran locally.
|
||||
|
||||
The installed Mac task/revision passed in ordinary New Agent UI with saved API key, Default model, high effort, and no runner override. Agent `766dddf5-1178-4cdf-81ee-ec542cfe0673` saved native ACPX Claude/Opus 5.5 with the experimental gate disabled. Preview http://127.0.0.1:3105/CLA/issues/CLA-3. Runs `ae8386b6-6a41-493c-a35c-a34bcd5feb7c` and `f79a151e-2267-4acd-ad81-6a34be25e1ff` succeeded across a controller restart with the same task/provider/backend session and high-effort acknowledgment. Independently downloaded v1 (114 bytes) SHA256 `f1da81caf11f90e80c3fe008adf3c1d6e180d38188f929dc8244edd64e4ff41a`; v2 (157 bytes) SHA256 `cae638f5f239806842b9ef1fa9694097e142035f803a89ff743696e558f5b3b2`. V1 remained unchanged. Settled API estimate $0.347345000. Report `/private/tmp/claude-v7-installed-mac-proof-20261015/acceptance.json` retains the state at acceptance; later company holds do not invalidate those settled runs.
|
||||
|
||||
The two subsequent ARM attempts failed before provider startup and remain pending accounting: `c2d8f460-381c-4764-ae04-4974b30739eb` (test-drive stripped inherited callback configuration) and `a6979058-b3d0-49ed-8c6d-159f507e3443` (read-only directory staging). Each retains a $0.50 hold. Callback configuration is now persisted through the instance's normal environment file. ARM uses a dedicated unprivileged account and private SSH listener, with the verified standard provider-pack installation; the system CLI and existing root listener are untouched. Root execution was rejected by Claude's ordinary permission check, reproduced without credentials or prompts.
|
||||
|
||||
Credential-free reproduction proved that BSD tar emits non-contiguous directory entries. GNU tar restored read-only directory permissions before later children arrived. The staging fix delays GNU directory restoration, omits macOS metadata, and verifies existing immutable context bytes and modes before reuse on follow-up. It keeps native provider sync behavior and durable checkpoint handling. Corrupt or symlinked targets do not become accepted cache hits. The regression failed before the fix; all 15 focused controls now pass with both GNU and BSD tar, including repeat staging and corrupt/symlink rejection. The actual patched function also passes twice on the unprivileged ARM host with zero provider calls; receipt `/private/tmp/claude-v7-arm-staging-fixed-20261015.json`. Remote live acceptance remains incomplete; no paid retry runs automatically.
|
||||
|
||||
Current allocation receipt `/private/tmp/claude-v7-remote-allocation-20261015.json` preserves the $50 aggregate ceiling and all historical holds. Known rate-card estimates total $4.090164800, excluding pending/unpriced charges and setup reservations. The current remote fixture has a $1.22 cap with its new $0.50 pending hold. Live subscription acceptance remains explicitly unverified under the user's API-key direction.
|
||||
|
||||
---
|
||||
|
||||
# Historical plan and evidence (retained)
|
||||
|
||||
@@ -22,7 +22,7 @@ import {
|
||||
truncate,
|
||||
writeFile,
|
||||
} from "node:fs/promises";
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { execFileSync, spawnSync } from "node:child_process";
|
||||
import { tmpdir } from "node:os";
|
||||
import { inspect } from "node:util";
|
||||
import { join } from "node:path";
|
||||
@@ -2518,6 +2518,98 @@ describe("remote provider checkpoint snapshots", () => {
|
||||
});
|
||||
|
||||
describe("remote provider checkpoint restores", () => {
|
||||
it("stages and verifies read-only context with non-contiguous archive entries", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "paperclip-readonly-restore-"));
|
||||
const sourcePath = join(root, "source");
|
||||
const targetPath = join(root, "target with 'quotes'");
|
||||
try {
|
||||
await mkdir(join(sourcePath, "references", "agents"), { recursive: true });
|
||||
await writeFile(join(sourcePath, "SKILL.md"), "skill");
|
||||
await writeFile(join(sourcePath, "references", "guide.md"), "guide");
|
||||
await writeFile(join(sourcePath, "references", "agents", "coder.md"), "coder");
|
||||
for (const path of [sourcePath, join(sourcePath, "references"), join(sourcePath, "references", "agents")]) {
|
||||
await chmod(path, 0o555);
|
||||
}
|
||||
for (const path of [join(sourcePath, "SKILL.md"), join(sourcePath, "references", "guide.md"), join(sourcePath, "references", "agents", "coder.md")]) {
|
||||
await chmod(path, 0o444);
|
||||
}
|
||||
const archive = execFileSync("tar", [
|
||||
"--no-recursion", "--no-xattrs", "-czf", "-", "-C", sourcePath,
|
||||
".", "./references", "./SKILL.md", "./references/agents",
|
||||
"./references/guide.md", "./references/agents/coder.md",
|
||||
], { env: { ...process.env, COPYFILE_DISABLE: "1" } });
|
||||
const execute = vi.fn(async (command: { command: string; args: string[] }) => {
|
||||
const result = spawnSync(command.command, command.args, {
|
||||
input: archive.toString("base64"),
|
||||
encoding: "utf8",
|
||||
env: { ...process.env },
|
||||
});
|
||||
return { exitCode: result.status, timedOut: false, stdout: result.stdout, stderr: result.stderr };
|
||||
});
|
||||
for (let pass = 0; pass < 2; pass++) {
|
||||
await stageRemoteRunnerDirectory({
|
||||
target: { kind: "remote", transport: "ssh", remoteCwd: root } as never,
|
||||
runner: { execute } as never,
|
||||
sourcePath,
|
||||
targetPath,
|
||||
mode: 0o555,
|
||||
});
|
||||
}
|
||||
expect(await readFile(join(targetPath, "references", "agents", "coder.md"), "utf8")).toBe("coder");
|
||||
expect(await readFile(join(targetPath, "references", "guide.md"), "utf8")).toBe("guide");
|
||||
expect(await readFile(join(targetPath, "SKILL.md"), "utf8")).toBe("skill");
|
||||
expect((await lstat(join(targetPath, "references"))).mode & 0o777).toBe(0o500);
|
||||
expect((await lstat(targetPath)).mode & 0o777).toBe(0o555);
|
||||
const corruptedFile = join(targetPath, "references", "agents", "coder.md");
|
||||
await chmod(corruptedFile, 0o600);
|
||||
await writeFile(corruptedFile, "other");
|
||||
await chmod(corruptedFile, 0o400);
|
||||
await expect(stageRemoteRunnerDirectory({
|
||||
target: { kind: "remote", transport: "ssh", remoteCwd: root } as never,
|
||||
runner: { execute } as never,
|
||||
sourcePath,
|
||||
targetPath,
|
||||
mode: 0o555,
|
||||
})).rejects.toThrow("runner_remote_directory_staging_failed");
|
||||
expect(await readFile(corruptedFile, "utf8")).toBe("other");
|
||||
} finally {
|
||||
for (const directory of [sourcePath, targetPath]) {
|
||||
for (const path of [directory, join(directory, "references"), join(directory, "references", "agents")]) {
|
||||
await chmod(path, 0o700).catch(() => undefined);
|
||||
}
|
||||
}
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("rejects context staging through a symlink", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "paperclip-context-symlink-"));
|
||||
const sourcePath = join(root, "source");
|
||||
const preservedPath = join(root, "preserved");
|
||||
const targetPath = join(root, "target");
|
||||
try {
|
||||
await mkdir(sourcePath);
|
||||
await mkdir(preservedPath);
|
||||
await writeFile(join(preservedPath, "sentinel"), "preserved");
|
||||
await symlink(preservedPath, targetPath);
|
||||
const execute = vi.fn(async (command: { command: string; args: string[]; stdin?: string }) => {
|
||||
const result = spawnSync(command.command, command.args, { input: command.stdin, encoding: "utf8" });
|
||||
return { exitCode: result.status, timedOut: false, stdout: result.stdout, stderr: result.stderr };
|
||||
});
|
||||
await expect(stageRemoteRunnerDirectory({
|
||||
target: { kind: "remote", transport: "ssh", remoteCwd: root } as never,
|
||||
runner: { execute } as never,
|
||||
sourcePath,
|
||||
targetPath,
|
||||
mode: 0o555,
|
||||
})).rejects.toThrow("runner_remote_directory_staging_failed");
|
||||
expect(await readFile(join(preservedPath, "sentinel"), "utf8")).toBe("preserved");
|
||||
expect((await lstat(targetPath)).isSymbolicLink()).toBe(true);
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("does not upload excluded Codex scratch trees or credentials", async () => {
|
||||
const sourcePath = await mkdtemp(
|
||||
join(tmpdir(), "paperclip-codex-restore-source-"),
|
||||
|
||||
@@ -10288,8 +10288,12 @@ export async function stageRemoteRunnerDirectory(input: {
|
||||
stagingRoot = mkdtempSync(join(tmpdir(), "paperclip-runner-restore-"));
|
||||
const archive = execFileSync(
|
||||
"tar",
|
||||
[...excludeArgs, "-czf", "-", "-C", input.sourcePath, "."],
|
||||
{ encoding: "buffer", maxBuffer: 64 * 1024 * 1024 },
|
||||
[...excludeArgs, "--no-xattrs", "-czf", "-", "-C", input.sourcePath, "."],
|
||||
{
|
||||
encoding: "buffer",
|
||||
maxBuffer: 64 * 1024 * 1024,
|
||||
env: { ...process.env, COPYFILE_DISABLE: "1" },
|
||||
},
|
||||
);
|
||||
execFileSync("tar", ["-xzf", "-", "-C", stagingRoot], {
|
||||
input: archive,
|
||||
@@ -10317,14 +10321,28 @@ export async function stageRemoteRunnerDirectory(input: {
|
||||
}
|
||||
const archive = execFileSync(
|
||||
"tar",
|
||||
[...excludeArgs, "-czf", "-", "-C", input.sourcePath, "."],
|
||||
{ encoding: "buffer", maxBuffer: 64 * 1024 * 1024 },
|
||||
[...excludeArgs, "--no-xattrs", "-czf", "-", "-C", input.sourcePath, "."],
|
||||
{
|
||||
encoding: "buffer",
|
||||
maxBuffer: 64 * 1024 * 1024,
|
||||
env: { ...process.env, COPYFILE_DISABLE: "1" },
|
||||
},
|
||||
);
|
||||
const escapedTarget = input.targetPath.replaceAll("'", "'\\''");
|
||||
const compareImmutable = `const fs=require("node:fs"),p=require("node:path");function same(a,b){const x=fs.lstatSync(a),y=fs.lstatSync(b);if((x.mode&511)!==(y.mode&511))return false;if(x.isDirectory())return y.isDirectory()&&fs.readdirSync(a).every(n=>same(p.join(a,n),p.join(b,n)));return x.isFile()&&y.isFile()&&fs.readFileSync(a).equals(fs.readFileSync(b));}try{process.exit(same(process.argv[1],process.argv[2])?0:1)}catch{process.exit(1)}`;
|
||||
const immutableStage = (input.mode & 0o222) === 0
|
||||
? `staging=$(mktemp -d) && mkdir "$staging/compare" || exit 1; ` +
|
||||
`trap 'find "$staging" -type d -exec chmod u+w {} +; rm -rf -- "$staging"' EXIT; ` +
|
||||
`base64 -d > "$staging/archive" && ` +
|
||||
`tar "$@" -xzf "$staging/archive" -C "$staging/compare" && ` +
|
||||
`chmod ${input.mode.toString(8)} "$staging/compare" || exit 1; ` +
|
||||
`if node -e '${compareImmutable}' "$staging/compare" '${escapedTarget}'; then exit 0; fi; ` +
|
||||
`tar "$@" -xzf "$staging/archive" -C '${escapedTarget}'`
|
||||
: `base64 -d | tar "$@" -xzf - -C '${escapedTarget}'`;
|
||||
const script =
|
||||
`umask 077; mkdir -p '${escapedTarget}' && ` +
|
||||
`base64 -d | tar -xzf - -C '${escapedTarget}' && ` +
|
||||
`chmod ${input.mode.toString(8)} '${escapedTarget}'`;
|
||||
`case "$(tar --version 2>/dev/null)" in *"GNU tar"*) set -- --delay-directory-restore ;; *) set -- ;; esac; ` +
|
||||
`umask 077; test ! -L '${escapedTarget}' && mkdir -p '${escapedTarget}' && ` +
|
||||
`{ ${immutableStage}; } && chmod ${input.mode.toString(8)} '${escapedTarget}'`;
|
||||
const result = await input.runner.execute({
|
||||
command: "sh",
|
||||
args: ["-c", script],
|
||||
|
||||
Reference in new issue
Block a user