From afbc5a885b52e96df10c1c6e643bea31b51caac8 Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Wed, 4 Nov 2026 10:52:39 -0600 Subject: [PATCH] fix(workspaces): save partial and shallow Git checkouts offline (#18990) ## Thinking Path > - Paperclip manages AI agents and their workspaces. > - A completed remote run must save its files before finalization can release its source. > - A shallow Git mirror uses upload-pack and can omit objects that exist only in the index. > - A filtered source can also fail when that mirror requests blobs the source never downloaded. > - Fetching missing history is not required to preserve an existing partial checkout. > - This change saves the available object stores and their existing shallow, sparse, and promisor state. > - The benefit is an offline save that preserves accepted work and remains readable by the existing archive reader. ## Linked Issues or Issue Description Refs #16727, #18705, and #18935. These changes cover managed clones, streaming capture, and safe diagnostics. This change repairs Git snapshot contents. **What happened?** Remote workspace capture invokes a mirror clone before it saves the archive. A shallow filtered sparse checkout with a missing promised blob fails that clone and surfaces `remote_workspace_operation_failed`. An ordinary shallow checkout can also lose an index-only object during the mirror transfer. **Expected behavior** Capture preserves the objects already available in the checkout, including staged work. It preserves partial, shallow, and sparse semantics without remote hydration. Corrupt or unsupported state must keep its source and receive no successful save receipt. **Steps to reproduce** 1. Create a real Git repository with two paths and commit history. 2. Clone it with `--filter=blob:none --depth=1 --sparse`, then select one path. 3. Stage a change, edit it again, and add an untracked file. 4. Remove access to the original repository and capture the workspace. 5. The base capture fails before transfer. The changed capture restores the available work after the original source is removed. **Paperclip version or commit** Reproduced on `86814c655`. The same transport/archive sources are present in `76e43852a`. ## What Changed - Preserve existing object files for shallow and partial repositories, including their original promisor pack markers. - Preserve shallow boundaries and the selected sparse settings and patterns. - Validate available object contents with full Git fsck. Reject unknown corruption instead of creating new promises. - Deny Git network protocols during capture. Keep the ordinary mirror's explicit local file transport. - Exclude ambient Git location, index, object-store, config, and template overrides. Disable templates for init and clone. - Keep the existing archive format, source fences, path guards, credential allowlist, and failure retention. - Add real command and reader regressions for full, shallow, filtered, sparse, corrupt, and unsupported repositories. ## Verification - Red: the compiled Node 24 transport completed helper installation, then failed capture with the same generic operation error and no transfer. Dirty source bytes remained intact. - Red: the new full/shallow/partial transport-to-reader cases exposed both filtered upload-pack failure and a missing staged-only object. Independent review then reproduced accepted corrupt object bytes and ambient template config/hook copying. - Green: all 110 transport/archive cases pass after those repairs. They cover staged, dirty, and untracked work; ordinary and forged-promisor corruption; malformed filters; alternate stores; unsafe links; credential redaction; and template isolation. An initial linked-worktree config regression was fixed before this complete passing run. - A compiled new capture restored through both the unchanged earlier reader and the new reader after source removal. HEAD, status, work bytes, sparse/shallow state, and the same missing promised object were preserved. Full Git fsck passed. No remote protocol was permitted. - Full repository `pnpm -r typecheck` and `pnpm build` pass at `91ca00a8327216def91303e19940e92ffe07c08f`. The build stamp names that exact head. - The [complete current-head CI run](https://github.com/paperclipai/paperclip/actions/runs/41865961403/attempts/2) is green: all 58 checks are terminal with no failure or pending check. Three expected skips and one neutral review remain those states. - Preserve the [first CI attempt](https://github.com/paperclipai/paperclip/actions/runs/41865961403/attempts/1): one browser case reached its 5-second heading deadline while the conversation pane was loading; a chat worker reported shutdown; another browser worker lost communication and its log blob was absent. A failed-job-only retry passed at the same source head. No test deadline, assertion, or source change was made for that retry. - The complete local `pnpm test:run` passed all 1,904 chat cases across six partitions. Its later general-server phase timed out two unchanged tool-access cases at 15 seconds. That known-red command was stopped before completion. Both cases then passed unchanged in a fresh isolated process. The original timeout cause remains unknown; this is not a full local-suite pass. - [Greptile reports 5/5](https://github.com/paperclipai/paperclip/pull/18990#issuecomment-6635931211) at the current head with no actionable finding. The product review's loose-object cost advisory was [answered with local measurements and resolved](https://github.com/paperclipai/paperclip/pull/18990#discussion_r4470261833). There are no unresolved review threads. - Original service acceptance remains unverified. Recorded checkout text is mechanism-compatible narrative evidence, not proof of its physical Git state or original failure cause. This PR includes no deployment or additional live recovery. ## Risks - Full object validation and copying available metadata use CPU and temporary space proportional to the local object store. - The offline path rejects alternate object stores and unsupported partial filters. Such sources remain protected. - Promised objects remain missing. Later work still needs ordinary repository access to request them. - There is no archive format or schema change. Existing readers restore standard Git files, and existing full snapshots remain covered. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, shell execution, and local verification. The host does not expose a more specific serving model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- doc/execution-semantics.md | 12 ++ .../remote-workspace-snapshot-archive.ts | 100 +++++++++++++-- .../remote-workspace-transport.test.ts | 114 ++++++++++++++++++ 3 files changed, 215 insertions(+), 11 deletions(-) diff --git a/doc/execution-semantics.md b/doc/execution-semantics.md index 625129be469..70c07a31455 100644 --- a/doc/execution-semantics.md +++ b/doc/execution-semantics.md @@ -862,6 +862,18 @@ receipt. A changed source or cancelled capture removes its incomplete archive; an existing archive remains untouched. Temporary space must still hold the normalized Git metadata and the complete archive together. +Shallow and partial Git checkouts retain their available object files instead +of asking a mirror clone to transfer promised or index-only objects. Capture +does not fetch missing objects. It preserves the shallow boundary, promisor pack +markers, portable promisor settings, and the selected sparse-checkout settings +and patterns. Both source and saved Git state must pass connectivity checks. +Missing objects in an ordinary repository remain a failure. Partial or shallow +repositories with alternate object stores, unsupported filters, or unsafe object +links remain protected sources and do not receive a successful save receipt. +This uses the existing archive format and ordinary Git files, so earlier archive +readers restore the same state. It does not add remote history or change the +authority to resume work, publish a result, or remove a source. + ### Remote Codex model compatibility Fresh remote Codex Runner runs check the selected image CLI before saving the diff --git a/server/src/services/remote-workspace-snapshot-archive.ts b/server/src/services/remote-workspace-snapshot-archive.ts index 8d6254d84ac..f04f1b2558a 100644 --- a/server/src/services/remote-workspace-snapshot-archive.ts +++ b/server/src/services/remote-workspace-snapshot-archive.ts @@ -42,11 +42,13 @@ async function workspaceArchive(input: { }; const git = (cwd: string, args: string[], stdin?: string) => new Promise((resolve, reject) => { check(); - const command = child.execFile("git", ["-c", "core.fsmonitor=false", "-c", "pack.threads=1", "-c", `core.hooksPath=${os.devNull}`, "-C", cwd, ...args], { + const command = child.execFile("git", ["-c", "core.fsmonitor=false", "-c", "pack.threads=1", "-c", "protocol.allow=never", "-c", `core.hooksPath=${os.devNull}`, "-C", cwd, ...args], { signal: input.signal, timeout: 300_000, maxBuffer: 64 * 1024 * 1024, env: { ...process.env, GIT_CONFIG_NOSYSTEM: "1", GIT_CONFIG_GLOBAL: os.devNull, GIT_TERMINAL_PROMPT: "0", GIT_OPTIONAL_LOCKS: "0", + GIT_DIR: undefined, GIT_COMMON_DIR: undefined, GIT_WORK_TREE: undefined, GIT_INDEX_FILE: undefined, + GIT_OBJECT_DIRECTORY: undefined, GIT_ALTERNATE_OBJECT_DIRECTORIES: undefined, GIT_CONFIG_PARAMETERS: undefined, GIT_CONFIG_COUNT: "0", GIT_TEMPLATE_DIR: undefined, GIT_AUTHOR_NAME: "Paperclip", GIT_COMMITTER_NAME: "Paperclip", GIT_AUTHOR_EMAIL: "workspace@paperclip.invalid", GIT_COMMITTER_EMAIL: "workspace@paperclip.invalid", - GIT_AUTHOR_DATE: "@0 +0000", GIT_COMMITTER_DATE: "@0 +0000" }, + GIT_AUTHOR_DATE: "@0 +0000", GIT_COMMITTER_DATE: "@0 +0000", GIT_NO_LAZY_FETCH: "1" }, }, (error, stdout) => error ? reject(error) : resolve(stdout)); if (stdin !== undefined) command.stdin?.end(stdin); }); @@ -64,18 +66,30 @@ async function workspaceArchive(input: { const portableGitSettings = async (repo: string) => { // Read only the repository's own allowlisted values. Includes, execution // hooks, filters, credential helpers, and machine-specific paths stay out. - const raw = await git(repo, ["config", "--null", "--local", "--no-includes", "--get-regexp", - "^(remote\\..*\\.(url|pushurl|fetch)|branch\\..*\\.(remote|merge|rebase)|user\\.(name|email))$"]) + const read = (scope: string, pattern: string) => git(repo, ["config", "--null", scope, "--no-includes", "--get-regexp", pattern]) .catch((error: { code?: number }) => { if (error.code === 1) return ""; throw error; }); - const settings = raw.split("\0").filter(Boolean).map(row => { + const sparsePattern = "^(core\\.(sparsecheckout|sparsecheckoutcone)|index\\.sparse)$"; + const perWorktree = await git(repo, ["config", "--local", "--no-includes", "--bool", "--get", "extensions.worktreeConfig"]) + .catch((error: { code?: number }) => { if (error.code === 1) return "false"; throw error; }); + const raw = await read("--local", "^(remote\\..*\\.(url|pushurl|fetch|promisor|partialclonefilter)|branch\\..*\\.(remote|merge|rebase)|user\\.(name|email)|extensions\\.partialclone|core\\.(sparsecheckout|sparsecheckoutcone)|index\\.sparse)$") + + (perWorktree.trim() === "true" ? await read("--worktree", sparsePattern) : ""); + const boolean = (value: string) => /^(true|yes|on|1|)$/iu.test(value) ? "true" + : /^(false|no|off|0)$/iu.test(value) ? "false" : fail("unsupported_git_setting"); + const rows = raw.split("\0").filter(Boolean).map(row => { const split = row.indexOf("\n"), key = row.slice(0, split), value = row.slice(split + 1); if (split < 1 || /[\u0000-\u0020\u007f\\]/u.test(key) || /[\u0000-\u001f\u007f\\]/u.test(value)) fail("unsupported_git_setting"); - return { key, value, remote: /^remote\.(.+)\.(url|pushurl|fetch)$/.exec(key), branch: /^branch\.(.+)\.(remote|merge|rebase)$/.exec(key), identity: /^user\.(name|email)$/.test(key) }; + return { key, value, remote: /^remote\.(.+)\.(url|pushurl|fetch|promisor|partialclonefilter)$/.exec(key), + branch: /^branch\.(.+)\.(remote|merge|rebase)$/.exec(key), identity: /^user\.(name|email)$/.test(key), + sparse: /^(core\.(sparsecheckout|sparsecheckoutcone)|index\.sparse)$/.test(key) }; }); + const settings = rows.filter((row, index) => !row.sparse || rows.findLastIndex(other => other.key === row.key) === index); const remotes = new Set(settings.flatMap(item => item.remote?.[2] === "url" ? [item.remote[1]!] : [])); for (const setting of settings) if (setting.remote && ["url", "pushurl"].includes(setting.remote[2]!)) { if (!portableRemoteUrl(setting.value)) remotes.delete(setting.remote[1]!); } + const promisors = new Set(settings.filter(setting => setting.remote?.[2] === "promisor" && boolean(setting.value) === "true") + .map(setting => setting.remote![1]!)); + if ([...promisors].some(name => !remotes.has(name))) fail("unsupported_git_setting"); const branches = new Map(); for (const setting of settings) if (setting.branch?.[2] === "remote") { const name = setting.branch[1]!; @@ -85,6 +99,9 @@ async function workspaceArchive(input: { for (const setting of settings) { if (setting.remote) { if (!remotes.has(setting.remote[1]!)) continue; + if (setting.remote[2] === "promisor") setting.value = boolean(setting.value); + if (setting.remote[2] === "partialclonefilter" && (!promisors.has(setting.remote[1]!) + || !/^(blob:none|blob:limit=[0-9]+[kmg]?|tree:[0-9]+)$/iu.test(setting.value))) fail("unsupported_git_setting"); if (setting.remote[2] === "fetch") { const refs = setting.value.replace(/^[+^]/u, "").split(":"); if (refs.length > 2 || !refs.some(Boolean)) fail("unsupported_git_setting"); @@ -100,6 +117,9 @@ async function workspaceArchive(input: { await git(repo, ["check-ref-format", "--allow-onelevel", setting.value]).catch(() => fail("unsupported_git_setting")); } if (setting.branch[2] === "rebase" && !["true", "false", "merges", "interactive", "preserve", "yes", "no", "on", "off", "1", "0", ""].includes(setting.value)) fail("unsupported_git_setting"); + } else if (setting.sparse) setting.value = boolean(setting.value); + else if (setting.key === "extensions.partialclone") { + if (!promisors.has(setting.value)) fail("unsupported_git_setting"); } else if (!setting.identity) fail("unsupported_git_setting"); retained.push({ key: setting.key, value: setting.value }); } @@ -139,6 +159,46 @@ async function workspaceArchive(input: { return { sha256: digest.digest("hex"), size, mode: Number(before.mode & 0o777n) }; } finally { await handle.close(); } }; + const copyGitFile = async (source: string, destination: string) => { + const entry = await fs.lstat(source).catch((error: NodeJS.ErrnoException) => { if (error.code !== "ENOENT") throw error; return null; }); + if (!entry) return; + if (!entry.isFile() || entry.isSymbolicLink()) fail("unsafe_git_link"); + await fs.mkdir(path.dirname(destination), { recursive: true, mode: 0o700 }); + const output = await fs.open(destination, "w", 0o600); + try { await readStableFile(source, bytes => output.writeFile(bytes)); } finally { await output.close(); } + }; + const gitStatePath = async (repo: string, name: string) => (await git(repo, ["rev-parse", "--path-format=absolute", "--git-path", name])).trim(); + const gitStateHash = async (repo: string, name: string) => { + const file = await gitStatePath(repo, name); + return readStableFile(file).then(entry => entry.sha256).catch((error: NodeJS.ErrnoException) => { + if (error.code !== "ENOENT") throw error; return null; + }); + }; + const gitObjectFiles = async (repo: string) => { + const objects = await gitStatePath(repo, "objects"); + const files: Array<{ path: string; identity: string[] }> = []; + const visit = async (relative: string) => { + const directory = path.join(objects, relative), entry = await fs.lstat(directory); + if (!entry.isDirectory() || entry.isSymbolicLink() || await fs.realpath(directory) !== directory) fail("unsafe_git_link"); + for (const name of (await fs.readdir(directory)).sort()) { + const next = relative ? `${relative}/${name}` : name, stat = await fs.lstat(path.join(objects, next), { bigint: true }); + if (stat.isSymbolicLink()) fail("unsafe_git_link"); + if (!relative && (name === "pack" || name === "info" || /^[a-f0-9]{2}$/u.test(name))) { await visit(next); continue; } + // Alternates are a separate local object store. The ordinary mirror + // path still dissociates them; this offline path must not drop them. + if (relative === "info" && ["alternates", "http-alternates"].includes(name)) fail("unsupported_git_setting"); + if (relative === "info" && name === "commit-graphs" && stat.isDirectory()) continue; + const object = /^[a-f0-9]{2}\/[a-f0-9]{38}(?:[a-f0-9]{24})?$/u.test(next) + || /^pack\/pack-[a-f0-9]{40}(?:[a-f0-9]{24})?\.(pack|idx|promisor|rev|bitmap|keep)$/u.test(next); + if (object) { + if (!stat.isFile()) fail("unsafe_git_link"); + files.push({ path: next, identity: [stat.dev, stat.ino, stat.size, stat.mode, stat.mtimeNs, stat.ctimeNs].map(String) }); + } else if (!stat.isFile()) fail("unsupported_git_setting"); + } + }; + await visit(""); + return { objects, files }; + }; const directories: string[] = []; const excludedPaths = new Set(); const walk = async (root: string, skipGit = false) => { @@ -200,15 +260,31 @@ async function workspaceArchive(input: { await assertGitOperationComplete(repo); const index = (await git(repo, ["rev-parse", "--path-format=absolute", "--git-path", "index"])).trim(); const indexHash = await hashFile(index).catch((error: NodeJS.ErrnoException) => error.code === "ENOENT" ? null : Promise.reject(error)); - return JSON.stringify([await git(repo, ["status", "--porcelain=v2", "-z", "--untracked-files=no"]), await git(repo, ["for-each-ref", "--format=%(objectname) %(refname)"]), await git(repo, ["symbolic-ref", "-q", "HEAD"]).catch(() => git(repo, ["rev-parse", "HEAD"])), indexHash, settings ?? await portableGitSettings(repo)]); + const config = settings ?? await portableGitSettings(repo), shallow = await gitStateHash(repo, "shallow"); + const partial = config.some(setting => /^remote\..*\.promisor$/u.test(setting.key) && setting.value === "true"); + return JSON.stringify([await git(repo, ["status", "--porcelain=v2", "-z", "--untracked-files=no"]), await git(repo, ["for-each-ref", "--format=%(objectname) %(refname)"]), await git(repo, ["symbolic-ref", "-q", "HEAD"]).catch(() => git(repo, ["rev-parse", "HEAD"])), indexHash, config, + shallow, await gitStateHash(repo, "info/sparse-checkout"), shallow || partial ? (await gitObjectFiles(repo)).files : null]); }; for (let index = 0; index < repositories.length; index++) { const relative = repositories[index]!, repo = path.join(root, relative), normalized = path.join(scratch, `repo-${index}.git`); const settings = await portableGitSettings(repo); gitSignatures.set(relative, await signature(repo, settings)); - // Mirror copies all refs; dissociation brings borrowed objects into this - // private repository. The selected worktree's index and HEAD are separate. - await git(root, ["clone", "--mirror", "--no-hardlinks", "--dissociate", "--", repo, normalized]); + const offline = await gitStateHash(repo, "shallow") !== null + || settings.some(setting => /^remote\..*\.promisor$/u.test(setting.key) && setting.value === "true"); + if (offline) { + // A shallow local clone uses upload-pack. It drops index-only objects + // and asks a filtered source for promised blobs it never downloaded. + // Preserve its existing stores and promisor boundaries without fetch. + await git(repo, ["fsck", "--full", "--no-dangling"]); + const format = (await git(repo, ["rev-parse", "--show-object-format"])).trim(); + if (!["sha1", "sha256"].includes(format)) fail("unsupported_git_setting"); + await git(root, ["init", "--bare", "--template=", `--object-format=${format}`, normalized]); + const { objects, files } = await gitObjectFiles(repo); + for (const file of files) await copyGitFile(path.join(objects, file.path), path.join(normalized, "objects", file.path)); + } else { + // Dissociation brings borrowed objects into the private repository. + await git(root, ["-c", "protocol.file.allow=always", "clone", "--mirror", "--no-hardlinks", "--dissociate", "--template=", "--", repo, normalized]); + } const destination = path.join(tree, relative, ".git"); await fs.mkdir(path.dirname(destination), { recursive: true, mode: 0o700 }); await fs.rename(normalized, destination); @@ -224,6 +300,8 @@ async function workspaceArchive(input: { await fs.mkdir(path.join(destination, "info"), { recursive: true }); await fs.copyFile(path.join(sourceCommon, "info", name), path.join(destination, "info", name)).catch((error: NodeJS.ErrnoException) => { if (error.code !== "ENOENT") throw error; }); } + await copyGitFile(await gitStatePath(repo, "shallow"), path.join(destination, "shallow")); + await copyGitFile(await gitStatePath(repo, "info/sparse-checkout"), path.join(destination, "info", "sparse-checkout")); const restoredRepo = path.join(tree, relative); await git(restoredRepo, ["config", "core.bare", "false"]); await git(restoredRepo, ["config", "--unset-all", "remote.origin.mirror"]).catch(() => undefined); @@ -237,7 +315,7 @@ async function workspaceArchive(input: { }).join(""); if (updates) await git(restoredRepo, ["update-ref", "--stdin"], updates); await fs.rm(path.join(destination, "objects", "info", "alternates"), { force: true }); - await git(restoredRepo, ["fsck", "--connectivity-only", "--no-dangling"]); + await git(restoredRepo, ["fsck", "--full", "--no-dangling"]); if (await signature(repo) !== gitSignatures.get(relative)) fail("changed"); } const originals = new Map(before.map(entry => [entry.path, entry])); diff --git a/server/src/services/remote-workspace-transport.test.ts b/server/src/services/remote-workspace-transport.test.ts index 53b05138959..9e71a52946a 100644 --- a/server/src/services/remote-workspace-transport.test.ts +++ b/server/src/services/remote-workspace-transport.test.ts @@ -5,6 +5,7 @@ import os from "node:os"; import path from "node:path"; import { pathToFileURL } from "node:url"; import { promisify } from "node:util"; +import { deflateSync } from "node:zlib"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import type { CommandManagedRuntimeRunner } from "@paperclipai/adapter-utils/command-managed-runtime"; import { @@ -527,6 +528,119 @@ describe("remote workspace transport safety with real sandbox commands", () => { } finally { await captured.cleanup(); } }); + it.each(["full", "shallow", "partial", "partial-sparse"] as const)("restores offline %s Git state through the archive reader", async kind => { + const origin = await repository({ revisions: true }); + for (const name of ["selected", "omitted"]) { + await fs.mkdir(path.join(origin, name)); + await fs.writeFile(path.join(origin, name, "file.txt"), `${name} committed work\n`); + } + await git(origin, "add", "."); await git(origin, "commit", "-qm", "Add sparse paths"); + await git(origin, "config", "uploadpack.allowFilter", "true"); + const repo = path.join(workspace, "repository"); + const shallow = ["shallow", "partial-sparse"].includes(kind), partial = kind.startsWith("partial"); + await git(scratch, "clone", "-q", ...(shallow ? ["--depth=1"] : []), + ...(partial ? ["--filter=blob:none"] : []), ...(kind === "partial-sparse" ? ["--sparse"] : []), pathToFileURL(origin).href, repo); + if (kind === "partial-sparse") await git(repo, "sparse-checkout", "set", "selected"); + await git(repo, "remote", "set-url", "origin", "https://example.invalid/source.git"); + const missing = async (directory: string) => (await git(directory, "rev-list", "--objects", "--all", "--missing=print")) + .stdout.split("\n").filter(line => line.startsWith("?")).sort(); + const missingBefore = await missing(repo); + if (partial) expect(missingBefore.length).toBeGreaterThan(0); + else expect(missingBefore).toEqual([]); + await fs.writeFile(path.join(repo, "selected", "file.txt"), "staged accepted work\n"); + await git(repo, "add", "selected/file.txt"); + await fs.writeFile(path.join(repo, "selected", "file.txt"), "dirty accepted work\n"); + await fs.writeFile(path.join(repo, "untracked.txt"), "untracked accepted work\n"); + const beforeStatus = (await git(repo, "status", "--porcelain=v2")).stdout; + const beforeHead = (await git(repo, "rev-parse", "HEAD")).stdout; + await fs.rm(origin, { recursive: true }); + const template = path.join(scratch, "unrelated-template"); await fs.mkdir(path.join(template, "hooks"), { recursive: true }); + await fs.writeFile(path.join(template, "config"), "[credential]\n helper = SYNTHETIC_PRIVATE_TEMPLATE\n[http]\n extraHeader = SYNTHETIC_PRIVATE_TEMPLATE\n"); + await fs.writeFile(path.join(template, "hooks", "template-hook"), "SYNTHETIC_PRIVATE_TEMPLATE\n", { mode: 0o700 }); + // Only local file transport is permitted. Capture must not fetch promised + // blobs or replace the original partial checkout with extra remote history. + const { remote } = target({ GIT_ALLOW_PROTOCOL: "file", GIT_INDEX_FILE: path.join(scratch, "unrelated-index"), GIT_TEMPLATE_DIR: template }, "one-shot"); + const captured = await captureRemoteWorkspaceArchive({ target: remote, remoteCwd: workspace, operationId: randomUUID() }); + try { + expect(await missing(repo)).toEqual(missingBefore); + expect(await fs.readFile(path.join(repo, "selected", "file.txt"), "utf8")).toBe("dirty accepted work\n"); + await fs.rm(workspace, { recursive: true }); remote.remoteCwd = scratch; + const restoredRoot = path.join(scratch, "offline-restored"), restoredRepo = path.join(restoredRoot, "repository"); + await restoreRemoteWorkspaceDirectory({ target: remote, remoteCwd: restoredRoot, archivePath: captured.archivePath, + manifestDigest: captured.manifestDigest, restorationIdentity: "d".repeat(64) }); + expect(await fs.readFile(path.join(restoredRepo, "selected", "file.txt"), "utf8")).toBe("dirty accepted work\n"); + expect(await fs.readFile(path.join(restoredRepo, "untracked.txt"), "utf8")).toBe("untracked accepted work\n"); + expect((await git(restoredRepo, "show", ":selected/file.txt")).stdout).toBe("staged accepted work\n"); + expect((await git(restoredRepo, "rev-parse", "HEAD")).stdout).toBe(beforeHead); + expect((await git(restoredRepo, "status", "--porcelain=v2")).stdout).toBe(beforeStatus); + expect((await git(restoredRepo, "rev-parse", "--is-shallow-repository")).stdout.trim()).toBe(String(shallow)); + expect(await missing(restoredRepo)).toEqual(missingBefore); + expect(await fs.readFile(path.join(restoredRepo, ".git", "config"), "utf8")).not.toContain("SYNTHETIC_PRIVATE_TEMPLATE"); + await expect(fs.access(path.join(restoredRepo, ".git", "hooks", "template-hook"))).rejects.toMatchObject({ code: "ENOENT" }); + if (partial) { + expect((await git(restoredRepo, "config", "--get", "remote.origin.promisor")).stdout.trim()).toBe("true"); + expect((await git(restoredRepo, "config", "--get", "remote.origin.partialclonefilter")).stdout.trim()).toBe("blob:none"); + } + if (kind === "partial-sparse") { + expect((await git(restoredRepo, "sparse-checkout", "list")).stdout.trim()).toBe("selected"); + await expect(fs.access(path.join(restoredRepo, "omitted"))).rejects.toMatchObject({ code: "ENOENT" }); + } + await git(restoredRepo, "fsck", "--connectivity-only", "--no-dangling"); + } finally { await captured.cleanup(); } + }); + + it.each(["ordinary", "unmarked-promisor"])("rejects missing objects in an %s repository instead of allowing corruption", async kind => { + await git(workspace, "init", "-q", "-b", "main"); + await fs.writeFile(path.join(workspace, "saved.txt"), "accepted work remains available\n"); + await git(workspace, "add", "."); await git(workspace, "commit", "-qm", "Initial work"); + const blob = (await git(workspace, "rev-parse", "HEAD:saved.txt")).stdout.trim(); + await fs.rm(path.join(workspace, ".git", "objects", blob.slice(0, 2), blob.slice(2))); + if (kind === "unmarked-promisor") { + await git(workspace, "config", "remote.origin.url", "https://example.invalid/source.git"); + await git(workspace, "config", "remote.origin.promisor", "true"); + await git(workspace, "config", "remote.origin.partialclonefilter", "blob:none"); + } + const { remote } = target({ GIT_ALLOW_PROTOCOL: "file" }, "one-shot"); + await expect(captureRemoteWorkspaceArchive({ target: remote, remoteCwd: workspace, operationId: randomUUID() })).rejects.toThrow(); + expect(await fs.readFile(path.join(workspace, "saved.txt"), "utf8")).toBe("accepted work remains available\n"); + }); + + it.each(["full", "shallow", "partial"] as const)("rejects corrupt available objects in %s Git before publishing a receipt", async kind => { + const origin = await repository({ revisions: true }); await git(origin, "config", "uploadpack.allowFilter", "true"); + const repo = path.join(workspace, "repository"); + await git(scratch, "clone", "-q", ...(kind === "shallow" ? ["--depth=1"] : []), + ...(kind === "partial" ? ["--filter=blob:none"] : []), pathToFileURL(origin).href, repo); + await git(repo, "remote", "set-url", "origin", "https://example.invalid/source.git"); + await fs.writeFile(path.join(repo, "staged.txt"), "accepted staged work\n"); await git(repo, "add", "staged.txt"); + const oid = (await git(repo, "rev-parse", ":staged.txt")).stdout.trim(); + const file = path.join(repo, ".git", "objects", oid.slice(0, 2), oid.slice(2)), corrupt = Buffer.from("different bytes\n"); + await fs.chmod(file, 0o600); await fs.writeFile(file, deflateSync(Buffer.concat([Buffer.from(`blob ${corrupt.length}\0`), corrupt]))); + // Valid zlib with the wrong object hash passes connectivity-only fsck. + // A successful receipt must also establish available object integrity. + await git(repo, "fsck", "--connectivity-only", "--no-dangling"); + await expect(git(repo, "fsck", "--full", "--no-dangling")).rejects.toThrow(); + const { remote } = target({ GIT_ALLOW_PROTOCOL: "file" }, "one-shot"); + await expect(captureRemoteWorkspaceArchive({ target: remote, remoteCwd: workspace, operationId: randomUUID() })).rejects.toThrow(); + expect(await fs.readFile(path.join(repo, "staged.txt"), "utf8")).toBe("accepted staged work\n"); + }); + + it.each(["unsupported-filter", "alternate-store", "object-symlink"] as const)("retains a partial source with unsupported %s state", async fault => { + const origin = await repository(); await git(origin, "config", "uploadpack.allowFilter", "true"); + const repo = path.join(workspace, "repository"); + await git(scratch, "clone", "-q", "--filter=blob:none", "--depth=1", pathToFileURL(origin).href, repo); + await git(repo, "remote", "set-url", "origin", "https://example.invalid/source.git"); + await fs.writeFile(path.join(repo, "saved.txt"), "retained accepted work\n"); + const outside = path.join(scratch, "SYNTHETIC_PRIVATE_OBJECT_PATH"); await fs.writeFile(outside, "outside sentinel\n"); + if (fault === "unsupported-filter") await git(repo, "config", "remote.origin.partialclonefilter", "SYNTHETIC_PRIVATE_FILTER"); + if (fault === "alternate-store") await fs.writeFile(path.join(repo, ".git", "objects", "info", "alternates"), `${scratch}\n`); + if (fault === "object-symlink") await fs.symlink(outside, path.join(repo, ".git", "objects", "pack", "unrelated-link")); + const { remote } = target({ GIT_ALLOW_PROTOCOL: "file" }, "one-shot"); + await expect(captureRemoteWorkspaceArchive({ target: remote, remoteCwd: workspace, operationId: randomUUID() })) + .rejects.toThrow(fault === "object-symlink" ? /^remote_workspace_archive_unsafe_git_link$/ : /^remote_workspace_archive_unsupported_git_setting$/); + expect(await fs.readFile(path.join(repo, "saved.txt"), "utf8")).toBe("retained accepted work\n"); + expect(await fs.readFile(outside, "utf8")).toBe("outside sentinel\n"); + }); + it("identifies a timed-out preparation without exposing transport output or retrying it", async () => { const { remote, execute } = target(); const resolveGitAuth = vi.fn(async () => null);