Files
PaperClipAI/server/src/services/git-workspace-restoration.ts
T
Devin FoleyandPaperclip ac15d883f7 fix: preserve remote-tracking history when expiring nested worktrees (#17535)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Nested project repositories are now worktrees of one shared clone
per project repository, and existing standalone clones are converted in
place
> - Conversion keeps the clone's remote-tracking refs as task-owned
branches (`paperclip/<id>/remotes/...`) so commits only they hold
survive
> - Expiration then required every task-owned branch, including those,
to be on the remote
> - Remote-tracking refs for branches deleted or force-pushed upstream
never are, so converted checkouts were kept forever
> - This pull request treats those refs the way the standalone-clone
path already does: commits missing upstream go into the restore point
instead of blocking
> - The benefit is that converted workspaces can expire and free disk
without losing any history

## Linked Issues or Issue Description

Refs #17407, #17382, #17289. Bug report described in-PR (bug report
template).

**What happened?**
On one instance, 17 closed workspaces in a two-repository project were
kept on every expiration sweep with
`unpushed_managed_checkout_branches`. Each nested checkout is a
converted worktree whose task-owned refs include
`paperclip/<id>/remotes/origin/<branch>` for upstream branches since
deleted or force-pushed (dependabot, squashed feature branches). Their
commits are on no remote, so `provenNestedWorktreeHistory` refused. The
task's own branch was fully pushed.

**Expected behavior**
As in the standalone-clone path (#17289), remote-tracking commits
missing upstream are preserved in the restore point, and only genuinely
local branches block expiration.

**Steps to reproduce**
1. Fetch an upstream branch into a standalone nested clone, then delete
that branch upstream.
2. Convert the clone into a worktree (nested clone migration).
3. Close the issue and let expiration inspect it: kept with
`unpushed_managed_checkout_branches`.

**Paperclip version or commit:** `master` at cb47dbb2da.
**Deployment mode:** any.
**Agent adapter(s) involved:** not adapter-specific (core).

## What Changed

- `provenNestedWorktreeHistory` in
`server/src/services/git-workspace-restoration.ts`:
- splits task-owned refs into those under `<prefix>remotes/` and the
rest;
  - still requires the rest (and their reflogs) to be on the remote;
- returns commits under `remotes/` that are missing upstream, and their
reflogs, as `preserve`, which the restore point bundles. This matches
the standalone-clone path.
- `restorePointHistoryDigest` in
`server/src/services/git-restore-point.ts`: for a linked worktree, it
fingerprints the task's own refs and their reflogs instead of the shared
clone's `refs/remotes`. Other tasks change those whenever they fetch,
and they never covered the task-owned remote refs. It loads
`nested-repository-worktrees` lazily to avoid an import cycle.

No code comments were added.

## Verification

- `pnpm exec vitest run nested-clone-migration` (in `server/`). New
cases:
- A converted checkout whose only extra history is a remote-tracking
commit deleted upstream expires with a restore point. After its refs are
removed and the shared clone is garbage-collected, reopening restores
the commit.
- The digest ignores the shared clone's remote refs and changes when the
task's own refs move.
- **Mutation checks**, each failing its test:
  - reverting the prover split;
  - dropping the preserve list;
  - reverting the digest change.
- The existing case where a converted checkout with a genuinely unpushed
branch is kept still passes.
- `pnpm exec vitest run git-restore-point nested-repository-worktrees
nested-clone-migration git-workspace-expiration
git-reproducible-managed-checkouts isolated-task-directory
isolated-repository-rehome git-credentials execution-workspace` (in
`server/`): 591 passed.
- `pnpm run typecheck` (in `server/`): pass.
- Not run: server build and boot smoke.

## Risks

- **Misclassified branches:** a local branch literally named
`remotes/...` in a standalone clone becomes `<prefix>remotes/...` after
conversion and is now preserved in the restore point instead of blocking
expiration. Its commits are still saved, not lost.
- **Leftover refs:** after expiry, the task's other owned refs stay in
the shared clone, as they did before this change, so their commits stay
in two places.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code
with tool use (shell, file edits, tests, read-only diagnosis on a live
instance).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-24 19:26:27 -07:00

781 lines
49 KiB
TypeScript

import { restorePointSavesIgnoredFile } from "./git-restore-point.js";
import { validateGitWorkspaceContents, type GitWorkspaceContents } from "./git-workspace-contents.js";
import { recoverGitExpirationLocks, type GitExpirationLockLocation } from "./git-workspace-expiration-locks.js";
import fs from "node:fs/promises";
import { constants } from "node:fs";
import os from "node:os";
import path from "node:path";
import { execFile as execFileCallback } from "node:child_process";
import { createHash } from "node:crypto";
import { resolveManagedProjectWorkspaceDir, resolvePaperclipInstanceRoot } from "../home-paths.js";
import { materializeManagedProjectWorkspace } from "./managed-repository-checkout.js";
import { promisify } from "node:util";
import type { GitRemoteAuthProvider } from "./git-credentials.js";
import { isRuntimeOwnedGitBranch } from "./execution-workspace-branch-ownership.js";
import { recordedIsolatedRemoteSource, verifiedRecordedIsolatedRemoteSource, materializeIsolatedRemoteWorkspaceSource } from "./isolated-remote-workspace-source.js";
import { findRegenerableWorkspaceDirs } from "./workspace-compaction.js";
import { ensureGitRestorePointCommit, validateGitRestorePointRef, type GitRestorePointRef } from "./git-restore-point-storage.js";
import { getStorageService } from "../storage/index.js";
import type { StorageService } from "../storage/types.js";
import { PROJECT_REPOSITORIES_DIR } from "@paperclipai/adapter-utils/git-workspace-sync";
import { OWNED_NESTED_REF_LIMIT, forgetExpiredNestedRepositoryWorktree, nestedRepositoryBaseCandidates, readLinkedNestedCheckout, recordedNestedRepositoryBase, type LinkedNestedCheckout } from "./nested-repository-worktrees.js";
const execFile = promisify(execFileCallback);
const shaPattern = /^[a-f0-9]{40}$/;
export const GIT_CACHE_RESTORATION_KEY = "gitCacheRestoration";
export const GIT_CACHE_EXPIRATION_REASON = "reproducible_git_cache_expired";
export const GIT_CACHE_RESTORATION_SCHEMA_V2 = "paperclip.git-cache-restoration.v2";
export type GitCacheRestoration = {
schema: "paperclip.git-cache-restoration.v1" | typeof GIT_CACHE_RESTORATION_SCHEMA_V2;
companyId: string;
workspaceId: string;
sourceIssueId: string;
repoUrl: string;
commit: string;
branchName: string;
sourceKind: "isolated_remote" | "managed_project";
projectId: string | null;
sourcePath: string;
worktreePath: string;
gitBranchOwnershipVersion: 1;
verifiedAt: string;
restoredAt?: string;
contentsRestoredAt?: string | null;
contents?: GitWorkspaceContents;
restorePoint?: GitRestorePointRef;
};
export type GitCacheExpiration = {
schema: "paperclip.git-cache-expiration.v1"; generation: number; workspaceRemoved: boolean;
worktreeTrash: string; worktreeAdmin: string; baseTrash: string | null; completed: boolean; lockToken?: string;
};
type Workspace = {
id: string; companyId: string; projectId: string | null; sourceIssueId: string | null; repoUrl: string | null;
mode: string | null; strategyType: string | null; branchName: string | null;
cwd: string | null; providerRef: string | null; status?: string; metadata: Record<string, unknown> | null;
};
export function managedSourcePaths(workspace: Workspace) {
if (!workspace.projectId || !workspace.repoUrl) return [];
let repoName: string | null = null;
try { repoName = new URL(workspace.repoUrl).pathname.replace(/\/+$/, "").split("/").pop()?.replace(/\.git$/i, "") || null; } catch { return []; }
const root = resolveManagedProjectWorkspaceDir({ companyId: workspace.companyId, projectId: workspace.projectId, repoName });
return [root, `${root}-${createHash("sha256").update(workspace.repoUrl).digest("hex").slice(0, 12)}`];
}
export async function verifiedGitCacheSource(workspace: Workspace, sourcePath: string) {
const privateSource = recordedIsolatedRemoteSource(workspace);
if (privateSource) {
if (sourcePath !== privateSource) throw new GitCacheRestorationFailure();
return (await verifiedRecordedIsolatedRemoteSource(workspace))!;
}
if (!managedSourcePaths(workspace).includes(sourcePath)) throw new GitCacheRestorationFailure();
const root = resolvePaperclipInstanceRoot();
const actual = await fs.realpath(sourcePath);
if (actual !== path.join(await fs.realpath(root), path.relative(root, sourcePath))) throw new GitCacheRestorationFailure();
const storage = await fs.lstat(path.join(actual, ".git"));
if (!storage.isDirectory() || storage.isSymbolicLink()
|| (await gitCacheCommand(actual, ["rev-parse", "--path-format=absolute", "--git-common-dir"])).trim() !== path.join(actual, ".git")
|| (await gitCacheCommand(actual, ["config", "--local", "--null", "--get-all", "remote.origin.url"])) !== `${workspace.repoUrl}\0`) throw new GitCacheRestorationFailure();
return sourcePath;
}
export class GitCacheRestorationFailure extends Error {
constructor() { super("The saved workspace commit could not be restored. Check repository access and retry."); }
}
export function readGitCacheRestoration(workspace: Workspace): GitCacheRestoration | null {
const value = workspace.metadata?.[GIT_CACHE_RESTORATION_KEY];
if (value === undefined) return null;
const recipe = value as GitCacheRestoration | null;
if (!recipe || !restorationSchemaMatches(recipe)
|| !isRuntimeOwnedGitBranch(workspace.metadata)
|| recipe.gitBranchOwnershipVersion !== 1
|| recipe.companyId !== workspace.companyId || recipe.workspaceId !== workspace.id
|| recipe.sourceIssueId !== workspace.sourceIssueId || recipe.repoUrl !== workspace.repoUrl
|| recipe.branchName !== workspace.branchName || recipe.worktreePath !== workspace.cwd
|| (workspace.providerRef && recipe.worktreePath !== workspace.providerRef)
|| recipe.projectId !== workspace.projectId
|| (recipe.sourceKind === "isolated_remote" ? recipe.sourcePath !== recordedIsolatedRemoteSource(workspace)
: recipe.sourceKind !== "managed_project" || recordedIsolatedRemoteSource(workspace) !== null || !managedSourcePaths(workspace).includes(recipe.sourcePath))
|| !shaPattern.test(recipe.commit) || !Number.isFinite(Date.parse(recipe.verifiedAt))
|| (recipe.restoredAt !== undefined && !Number.isFinite(Date.parse(recipe.restoredAt)))
|| (recipe.contentsRestoredAt != null && !Number.isFinite(Date.parse(recipe.contentsRestoredAt)))) {
throw new GitCacheRestorationFailure();
}
validateGitWorkspaceContents(recipe.contents);
validateRestorePoints(recipe, workspace);
return recipe;
}
function hasRestorePoints(recipe: GitCacheRestoration) {
return recipe.restorePoint !== undefined || (recipe.contents?.checkouts ?? []).some(checkout => checkout?.restorePoint !== undefined);
}
function restorationSchemaMatches(recipe: GitCacheRestoration) {
return recipe.schema === "paperclip.git-cache-restoration.v1"
? !hasRestorePoints(recipe)
: recipe.schema === GIT_CACHE_RESTORATION_SCHEMA_V2 && hasRestorePoints(recipe);
}
function validateRestorePoints(recipe: GitCacheRestoration, workspace: { companyId: string; id: string }) {
try {
if (recipe.restorePoint) validateGitRestorePointRef(recipe.restorePoint, workspace);
for (const checkout of recipe.contents?.checkouts ?? []) {
if (checkout.restorePoint) validateGitRestorePointRef(checkout.restorePoint, workspace, checkout.name);
}
} catch { throw new GitCacheRestorationFailure(); }
}
export async function gitCacheCommand(cwd: string, args: string[], signal?: AbortSignal) {
const result = await execFile("git", ["-c", "core.fsmonitor=false", "-C", cwd, ...args], {
timeout: 60_000, signal, maxBuffer: 16 * 1024 * 1024,
env: { ...process.env, GIT_TERMINAL_PROMPT: "0", GIT_OPTIONAL_LOCKS: "0" },
});
return result.stdout;
}
/** A fresh object store proves the object came from upstream, not our cache. */
export async function verifyRemoteGitCommit(repoUrl: string, commit: string, resolveGitAuth: GitRemoteAuthProvider, signal?: AbortSignal) {
if (!shaPattern.test(commit)) throw new GitCacheRestorationFailure();
const temporary = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-git-proof-"));
try {
const auth = await resolveGitAuth(repoUrl);
const env = { ...process.env, ...auth?.env, GIT_TERMINAL_PROMPT: "0", GIT_CONFIG_NOSYSTEM: "1", GIT_CONFIG_GLOBAL: os.devNull };
const options = { timeout: 60_000, signal, maxBuffer: 1024 * 1024, env };
await execFile("git", ["init", "--bare", temporary], options);
await execFile("git", [...auth?.configArgs ?? [], "-C", temporary, "fetch", "--no-tags", "--no-recurse-submodules", "--depth=1", "--", repoUrl, commit], options);
const result = await execFile("git", ["-C", temporary, "rev-parse", "FETCH_HEAD^{commit}"], options);
if (result.stdout.trim() !== commit) throw new GitCacheRestorationFailure();
} catch { throw new GitCacheRestorationFailure(); }
finally { await fs.rm(temporary, { recursive: true, force: true }); }
}
let templateFiles: Promise<Map<string, Buffer>> | undefined;
async function stockGitFiles() {
templateFiles ??= (async () => {
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-git-template-"));
const files = new Map<string, Buffer>();
try {
await execFile("git", ["init", "--bare", root], { timeout: 10_000,
env: { ...process.env, GIT_CONFIG_NOSYSTEM: "1", GIT_CONFIG_GLOBAL: os.devNull } });
for (const name of ["description", "info/exclude", ...(await fs.readdir(path.join(root, "hooks"))).map(name => `hooks/${name}`)]) {
files.set(name, await fs.readFile(path.join(root, name)));
}
return files;
} finally { await fs.rm(root, { recursive: true, force: true }); }
})();
return templateFiles;
}
export async function assertReproducibleBaseGitStorage(cwd: string, signal?: AbortSignal, options: { savedCommitMessage?: boolean } = {}) {
const root = path.join(cwd, ".git");
const allowed = new Set(["HEAD", "ORIG_HEAD", "FETCH_HEAD", "index", "config", "description", "hooks", "info", "refs", "objects", "logs", "packed-refs", "worktrees", "branches",
...(options.savedCommitMessage ? ["COMMIT_EDITMSG"] : [])]);
if ((await fs.readdir(root)).some(name => !allowed.has(name))) throw new Error("unknown_base_git_files");
const branches = path.join(root, "branches");
const branchesStat = await fs.lstat(branches).catch((error: NodeJS.ErrnoException) => {
if (error.code === "ENOENT") return null;
throw error;
});
if (branchesStat && (!branchesStat.isDirectory() || branchesStat.isSymbolicLink()
|| (await fs.readdir(branches)).length > 0)) throw new Error("custom_git_metadata");
const templates = await stockGitFiles();
for (const directory of ["hooks", "info"]) {
for (const name of await fs.readdir(path.join(root, directory))) {
const file = `${directory}/${name}`;
const entry = await fs.lstat(path.join(root, file));
if (!entry.isFile()) throw new Error("custom_git_metadata");
const actual = await fs.readFile(path.join(root, file));
const template = templates.get(file);
const managedExclude = file === "info/exclude" && template
? Buffer.concat([template, Buffer.from(`\n/${PROJECT_REPOSITORIES_DIR}/\n`)]) : null;
if (!template?.equals(actual) && !managedExclude?.equals(actual)) throw new Error("custom_git_metadata");
}
}
if (!templates.get("description")?.equals(await fs.readFile(path.join(root, "description")))) throw new Error("custom_git_metadata");
// A custom hook or metadata file inside object/ref storage must not disappear
// merely because Git's object/ref enumeration does not mention it.
async function walk(relative: string): Promise<void> {
signal?.throwIfAborted();
for (const entry of await fs.readdir(path.join(root, relative), { withFileTypes: true })) {
const name = `${relative}/${entry.name}`;
if (entry.isSymbolicLink() || (!entry.isDirectory() && !entry.isFile())) throw new Error("unsupported_git_storage");
if (entry.isDirectory()) { await walk(name); continue; }
const valid = /^objects\/[0-9a-f]{2}\/[0-9a-f]{38}$/.test(name)
|| /^objects\/pack\/pack-[0-9a-f]{40}\.(pack|idx|rev|bitmap)$/.test(name)
|| /^(refs|logs\/refs)\/(heads|remotes\/origin)\//.test(name)
|| name === "logs/HEAD";
if (!valid) throw new Error("unknown_git_storage_file");
}
}
for (const directory of ["objects", "refs", "logs"]) await walk(directory);
}
const PAPERCLIP_CLAUDE_SETTINGS_FILE = ".claude/settings.local.json";
export type ManagedCheckoutProver = (checkoutPath: string, name: string, signal?: AbortSignal) => Promise<void>;
const COMMIT_MESSAGE_LIMIT = 64 * 1024;
async function readCommitMessage(gitDir: string): Promise<string | null> {
const target = path.join(gitDir, "COMMIT_EDITMSG");
const entry = await fs.lstat(target).catch((error: NodeJS.ErrnoException) => {
if (error.code === "ENOENT") return null;
throw error;
});
if (!entry) return null;
if (!entry.isFile() || entry.size > COMMIT_MESSAGE_LIMIT) throw new Error("unsupported_commit_message");
const handle = await fs.open(target, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK);
try {
const stat = await handle.stat();
if (!stat.isFile() || stat.size > COMMIT_MESSAGE_LIMIT) throw new Error("unsupported_commit_message");
const buffer = Buffer.alloc(COMMIT_MESSAGE_LIMIT + 1);
let length = 0;
while (length < buffer.length) {
const { bytesRead } = await handle.read(buffer, length, buffer.length - length, length);
if (bytesRead === 0) break;
length += bytesRead;
}
if (length > COMMIT_MESSAGE_LIMIT) throw new Error("unsupported_commit_message");
return buffer.subarray(0, length).toString("utf8");
} finally { await handle.close(); }
}
async function assertCommitMessageSaved(cwd: string, gitDir: string, signal?: AbortSignal) {
const message = await readCommitMessage(gitDir);
if (message === null) return;
const saved = [(await gitCacheCommand(cwd, ["log", "-1", "--format=%B"], signal)).trim(),
...(await gitCacheCommand(cwd, ["log", "-g", "--format=%B%x00", "HEAD"], signal).catch(() => "")).split("\0").map(entry => entry.trim())];
if (!saved.includes(message.trim())) throw new Error("uncommitted_git_message");
}
async function ownedNestedRefs(linked: LinkedNestedCheckout, signal?: AbortSignal) {
const refs = (await gitCacheCommand(linked.checkout, ["for-each-ref", `--count=${OWNED_NESTED_REF_LIMIT + 1}`,
"--format=%(objectname) %(refname)", `refs/heads/${linked.branchPrefix}`], signal))
.split("\n").map(line => line.trim().split(" ")).filter(([sha, ref]) => shaPattern.test(sha ?? "") && ref) as Array<[string, string]>;
if (refs.length > OWNED_NESTED_REF_LIMIT) throw new Error("managed_checkout_local_tip_budget");
return refs;
}
async function ownedNestedReflogs(linked: LinkedNestedCheckout, refs: Array<[string, string]>, signal?: AbortSignal) {
const commits = new Set<string>();
for (const [, ref] of refs) {
for (const line of (await gitCacheCommand(linked.checkout, ["reflog", "show", "--format=%H", ref], signal).catch(() => "")).split("\n")) {
if (shaPattern.test(line.trim())) commits.add(line.trim());
}
if (commits.size > MANAGED_CHECKOUT_LOCAL_TIP_LIMIT) throw new Error("managed_checkout_local_tip_budget");
}
return [...commits];
}
async function assertOwnedNestedHead(linked: LinkedNestedCheckout, signal?: AbortSignal) {
const head = (await gitCacheCommand(linked.checkout, ["symbolic-ref", "--quiet", "HEAD"], signal).catch(() => "")).trim();
if (head && !head.startsWith(`refs/heads/${linked.branchPrefix}`)) throw new Error("unowned_nested_worktree_branch");
}
export async function assertReproducibleManagedCheckout(checkout: string, input: {
repoUrl: string; resolveGitAuth: GitRemoteAuthProvider; signal?: AbortSignal; restorePoint?: boolean; baseCandidates?: string[];
}): Promise<{ headOnRemote: boolean; preserve: string[] }> {
const { signal } = input;
const linked = await readLinkedNestedCheckout(checkout, input.baseCandidates ?? []);
if (linked) {
if ((await gitCacheCommand(checkout, ["config", "--local", "--null", "--get-all", "remote.origin.url"], signal)) !== `${input.repoUrl}\0`) {
throw new Error("unproven_managed_checkout");
}
await assertOwnedNestedHead(linked, signal);
} else {
const gitDir = await fs.lstat(path.join(checkout, ".git")).catch(() => null);
if (!gitDir?.isDirectory()) throw new Error("unproven_managed_checkout");
if ((await gitCacheCommand(checkout, ["rev-parse", "--show-toplevel"], signal)).trim() !== await fs.realpath(checkout)) {
throw new Error("unproven_managed_checkout");
}
const worktrees = (await gitCacheCommand(checkout, ["worktree", "list", "--porcelain"], signal))
.split("\n").filter(line => line.startsWith("worktree "));
if (worktrees.length !== 1) throw new Error("managed_checkout_has_worktrees");
}
await assertReproducibleGitFiles(checkout, signal, { preserveSettings: true, restorePoint: input.restorePoint,
...(linked ? { verifyFetchedCommit: (commit: string, verificationSignal?: AbortSignal) =>
verifyRemoteGitCommit(input.repoUrl, commit, input.resolveGitAuth, verificationSignal) } : {}) });
if (!linked) {
await assertReproducibleBaseGitStorage(checkout, signal, { savedCommitMessage: true });
await assertCommitMessageSaved(checkout, path.join(checkout, ".git"), signal);
}
const refs = (await gitCacheCommand(checkout, ["for-each-ref", "--format=%(refname)"], signal)).trim().split("\n").filter(Boolean);
if (refs.some(ref => !/^refs\/(heads|tags|remotes\/origin)\//.test(ref))) throw new Error("unsupported_managed_checkout_refs");
if (input.restorePoint) {
const { headOnRemote, preserve } = await provenManagedCheckoutHistory(checkout, input.repoUrl, input.resolveGitAuth, signal);
if (!linked) {
const unreachable = await gitCacheCommand(checkout, ["fsck", "--connectivity-only", "--unreachable", "--no-progress"], signal);
if (unreachable.trim()) throw new Error("unreachable_managed_checkout_objects");
}
return { headOnRemote, preserve };
}
const owned = linked ? await ownedNestedRefs(linked, signal) : [];
const localTips = new Set([
...(await gitCacheCommand(checkout, ["rev-parse", "--verify", "HEAD^{commit}"], signal)).split("\n"),
...linked ? owned.map(([sha]) => sha) : (await gitCacheCommand(checkout, ["for-each-ref", "--format=%(objectname)"], signal)).split("\n"),
...(await gitCacheCommand(checkout, ["reflog", "show", ...linked ? [] : ["--all"], "--format=%H", ...linked ? ["HEAD"] : []], signal).catch(() => "")).split("\n"),
...linked ? await ownedNestedReflogs(linked, owned, signal) : [],
].map(line => line.trim()).filter(sha => shaPattern.test(sha)));
if (localTips.size > MANAGED_CHECKOUT_LOCAL_TIP_LIMIT) throw new Error("managed_checkout_local_tip_budget");
if (localTips.size && await hasCommitsMissingUpstream(checkout, [...localTips], input.repoUrl, input.resolveGitAuth, signal)) {
throw new Error("unpushed_managed_checkout");
}
if (!linked) {
const unreachable = await gitCacheCommand(checkout, ["fsck", "--connectivity-only", "--unreachable", "--no-progress"], signal);
if (unreachable.trim()) throw new Error("unreachable_managed_checkout_objects");
}
return { headOnRemote: true, preserve: [] };
}
const MANAGED_CHECKOUT_LOCAL_TIP_LIMIT = 5_000;
export async function provenManagedCheckoutHistory(checkout: string, repoUrl: string, resolveGitAuth: GitRemoteAuthProvider,
signal?: AbortSignal): Promise<{ headOnRemote: boolean; preserve: string[] }> {
const linked = await readLinkedNestedCheckout(checkout);
if (linked) return provenNestedWorktreeHistory(linked, repoUrl, resolveGitAuth, signal);
const head = (await gitCacheCommand(checkout, ["rev-parse", "--verify", "HEAD^{commit}"], signal)).trim();
const shas = (output: string) => output.split("\n").map(line => line.trim()).filter(sha => shaPattern.test(sha));
const covered = new Set([head,
...shas(await gitCacheCommand(checkout, ["reflog", "show", "--format=%H", "HEAD"], signal).catch(() => "")),
...shas(await gitCacheCommand(checkout, ["rev-parse", "--verify", "-q", "ORIG_HEAD^{commit}"], signal).catch(() => ""))]);
const reflogs = [...new Set((await gitCacheCommand(checkout, ["reflog", "show", "--all", "--format=%H %gD"], signal).catch(() => ""))
.split("\n").map(line => line.trim().split(" ")).filter(([sha]) => shaPattern.test(sha ?? ""))
.map(([sha, selector]) => `${sha} ${(selector ?? "").startsWith("refs/remotes/") ? "remote" : "local"}`))].map(entry => entry.split(" "));
if (reflogs.length > MANAGED_CHECKOUT_LOCAL_TIP_LIMIT) throw new Error("managed_checkout_local_tip_budget");
const remote = (kind: string | undefined) => kind === "remote";
const recorded = new Set([
...shas(await gitCacheCommand(checkout, ["for-each-ref", "--format=%(objectname)", "refs/heads", "refs/tags"], signal)),
...reflogs.filter(([, selector]) => !remote(selector)).map(([sha]) => sha!),
]);
const remoteTracked = [...new Set([
...shas(await gitCacheCommand(checkout, ["for-each-ref", "--format=%(objectname)", "refs/remotes"], signal)),
...reflogs.filter(([, selector]) => remote(selector)).map(([sha]) => sha!),
])];
const current = (await gitCacheCommand(checkout, ["symbolic-ref", "--quiet", "HEAD"], signal).catch(() => "")).trim();
const otherTips = (await gitCacheCommand(checkout, ["for-each-ref", "--format=%(objectname) %(refname)", "refs/heads", "refs/tags"], signal))
.split("\n").map(line => line.trim().split(" ")).filter(([sha, ref]) => shaPattern.test(sha ?? "") && ref !== current).map(([sha]) => sha!);
if (otherTips.length && await hasCommitsMissingUpstream(checkout, otherTips, repoUrl, resolveGitAuth, signal, [head])) {
throw new Error("unpushed_managed_checkout_branches");
}
const uncovered = [...recorded].filter(sha => !covered.has(sha));
if (uncovered.length + covered.size + remoteTracked.length > MANAGED_CHECKOUT_LOCAL_TIP_LIMIT) throw new Error("managed_checkout_local_tip_budget");
if (uncovered.length && await hasCommitsMissingUpstream(checkout, uncovered, repoUrl, resolveGitAuth, signal, [...covered])) {
throw new Error("unpushed_managed_checkout_branches");
}
const headOnRemote = !await hasCommitsMissingUpstream(checkout, [head], repoUrl, resolveGitAuth, signal);
const preserve = remoteTracked.length
? await tipsMissingUpstream(checkout, remoteTracked, repoUrl, resolveGitAuth, signal, [...covered, ...recorded]) : [];
return { headOnRemote, preserve };
}
async function provenNestedWorktreeHistory(linked: LinkedNestedCheckout, repoUrl: string, resolveGitAuth: GitRemoteAuthProvider,
signal?: AbortSignal): Promise<{ headOnRemote: boolean; preserve: string[] }> {
const checkout = linked.checkout;
await assertOwnedNestedHead(linked, signal);
const head = (await gitCacheCommand(checkout, ["rev-parse", "--verify", "HEAD^{commit}"], signal)).trim();
const shas = (output: string) => output.split("\n").map(line => line.trim()).filter(sha => shaPattern.test(sha));
const covered = new Set([head,
...shas(await gitCacheCommand(checkout, ["reflog", "show", "--format=%H", "HEAD"], signal).catch(() => "")),
...shas(await gitCacheCommand(checkout, ["rev-parse", "--verify", "-q", "ORIG_HEAD^{commit}"], signal).catch(() => ""))]);
const allOwned = await ownedNestedRefs(linked, signal);
const remotePrefix = `refs/heads/${linked.branchPrefix}remotes/`;
const owned = allOwned.filter(([, ref]) => !ref.startsWith(remotePrefix));
const remoteTracked = [...new Set([...allOwned.filter(([, ref]) => ref.startsWith(remotePrefix)).map(([sha]) => sha),
...await ownedNestedReflogs(linked, allOwned.filter(([, ref]) => ref.startsWith(remotePrefix)), signal)])];
const recorded = new Set([...owned.map(([sha]) => sha), ...await ownedNestedReflogs(linked, owned, signal)]);
if (recorded.size + covered.size + remoteTracked.length > MANAGED_CHECKOUT_LOCAL_TIP_LIMIT) throw new Error("managed_checkout_local_tip_budget");
const current = (await gitCacheCommand(checkout, ["symbolic-ref", "--quiet", "HEAD"], signal).catch(() => "")).trim();
const otherTips = owned.filter(([, ref]) => ref !== current).map(([sha]) => sha);
if (otherTips.length && await hasCommitsMissingUpstream(checkout, otherTips, repoUrl, resolveGitAuth, signal, [head])) {
throw new Error("unpushed_managed_checkout_branches");
}
const uncovered = [...recorded].filter(sha => !covered.has(sha));
if (uncovered.length && await hasCommitsMissingUpstream(checkout, uncovered, repoUrl, resolveGitAuth, signal, [...covered])) {
throw new Error("unpushed_managed_checkout_branches");
}
const headOnRemote = !await hasCommitsMissingUpstream(checkout, [head], repoUrl, resolveGitAuth, signal);
const preserve = remoteTracked.length
? await tipsMissingUpstream(checkout, remoteTracked, repoUrl, resolveGitAuth, signal, [...covered, ...recorded]) : [];
return { headOnRemote, preserve };
}
async function tipsMissingUpstream(checkout: string, tips: string[], repoUrl: string,
resolveGitAuth: GitRemoteAuthProvider, signal?: AbortSignal, covered: string[] = []) {
const missing = new Set(await commitsMissingUpstream(checkout, tips, repoUrl, resolveGitAuth, signal, covered));
return tips.filter(tip => missing.has(tip)).sort();
}
export async function hasCommitsMissingUpstream(checkout: string, localTips: string[], repoUrl: string,
resolveGitAuth: GitRemoteAuthProvider, signal?: AbortSignal, covered: string[] = []) {
return (await commitsMissingUpstream(checkout, localTips, repoUrl, resolveGitAuth, signal, covered, 1)).length > 0;
}
async function commitsMissingUpstream(checkout: string, localTips: string[], repoUrl: string,
resolveGitAuth: GitRemoteAuthProvider, signal?: AbortSignal, covered: string[] = [], limit?: number) {
const store = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-git-checkout-proof-"));
try {
const options = { timeout: 120_000, signal, maxBuffer: 16 * 1024 * 1024, cwd: os.tmpdir(),
env: { ...process.env, GIT_TERMINAL_PROMPT: "0", GIT_CONFIG_NOSYSTEM: "1", GIT_CONFIG_GLOBAL: os.devNull } };
await execFile("git", ["init", "--bare", "--quiet", store], options);
const common = (await gitCacheCommand(checkout, ["rev-parse", "--path-format=absolute", "--git-common-dir"], signal)).trim();
await fs.writeFile(path.join(store, "objects", "info", "alternates"), `${path.join(await fs.realpath(common), "objects")}\n`);
try {
const auth = await resolveGitAuth(repoUrl);
await execFile("git", [...auth?.configArgs ?? [], "-C", store, "fetch", "--quiet", "--no-tags", "--no-recurse-submodules",
"--", repoUrl, "+refs/heads/*:refs/upstream/heads/*", "+refs/tags/*:refs/upstream/tags/*"],
{ ...options, env: { ...options.env, ...auth?.env } });
} catch (error) {
if (signal?.aborted) throw error;
throw new Error("managed_checkout_upstream_unavailable");
}
const missing = await execFile("git", ["-C", store, "rev-list", ...(limit ? [`--max-count=${limit}`] : []), ...localTips,
"--not", "--glob=refs/upstream/*", ...covered], options);
return missing.stdout.split("\n").map(line => line.trim()).filter(Boolean);
} finally {
await fs.rm(store, { recursive: true, force: true });
}
}
async function assertReproducibleFetchHead(cwd: string, gitDir: string, signal?: AbortSignal,
verifyFetchedCommit?: (commit: string, signal?: AbortSignal) => Promise<void>) {
const target = path.join(gitDir, "FETCH_HEAD");
const entry = await fs.lstat(target).catch((error: NodeJS.ErrnoException) => {
if (error.code === "ENOENT") return null;
throw error;
});
if (!entry) return;
const byteLimit = 64 * 1024;
if (!entry.isFile() || entry.isSymbolicLink() || entry.size > byteLimit) throw new Error("unsupported_worktree_fetch_head");
const handle = await fs.open(target, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK);
let contents: Buffer;
try {
const stat = await handle.stat();
if (!stat.isFile() || stat.nlink !== 1 || stat.size > byteLimit) throw new Error("unsupported_worktree_fetch_head");
const buffer = Buffer.alloc(byteLimit + 1);
let length = 0;
while (length < buffer.length) {
signal?.throwIfAborted();
const { bytesRead } = await handle.read(buffer, length, buffer.length - length, length);
if (bytesRead === 0) break;
length += bytesRead;
}
if (length > byteLimit) throw new Error("unsupported_worktree_fetch_head");
contents = buffer.subarray(0, length);
} finally { await handle.close(); }
const text = contents.toString("utf8");
if (!Buffer.from(text, "utf8").equals(contents) || text && !text.endsWith("\n")) throw new Error("unsupported_worktree_fetch_head");
const commits = new Set<string>();
for (const line of text ? text.slice(0, -1).split("\n") : []) {
const match = /^([a-f0-9]{40})\t(?:not-for-merge)?\t[^\t\r\n\0]*$/.exec(line);
if (!match) throw new Error("unsupported_worktree_fetch_head");
commits.add(match[1]!);
if (commits.size > 32) throw new Error("worktree_fetch_head_budget");
}
for (const commit of commits) {
signal?.throwIfAborted();
const objectType = await gitCacheCommand(cwd, ["cat-file", "-t", commit], signal).catch(() => "");
const ancestor = objectType.trim() === "commit" && await gitCacheCommand(cwd, ["merge-base", "--is-ancestor", commit, "HEAD"], signal)
.then(() => true, () => false);
signal?.throwIfAborted();
if (ancestor) continue;
if (!verifyFetchedCommit) throw new Error("unverified_fetched_commit");
await verifyFetchedCommit(commit, signal);
}
}
export async function assertReproducibleGitFiles(cwd: string, signal?: AbortSignal, options: {
proveManagedCheckout?: ManagedCheckoutProver;
preserveSettings?: boolean;
verifyFetchedCommit?: (commit: string, signal?: AbortSignal) => Promise<void>;
restorePoint?: boolean;
} = {}) {
if (!options.restorePoint && (await gitCacheCommand(cwd, ["status", "--porcelain=v1", "--untracked-files=all", "-z"], signal)).length) throw new Error("dirty_or_untracked_files");
if ((await gitCacheCommand(cwd, ["ls-files", "-v"], signal)).split("\n").some(line => /^[a-zS] /.test(line))) throw new Error("unsupported_index_flags");
const localConfig = await gitCacheCommand(cwd, ["config", "--local", "--name-only", "--list"], signal);
if (localConfig.split("\n").filter(Boolean).some(key => !/^(core\.(repositoryformatversion|filemode|bare|logallrefupdates|ignorecase|precomposeunicode)|remote\.origin\.(url|fetch)|branch\..+\.(remote|merge))$/.test(key))) throw new Error("unsupported_local_git_config");
const regenerable = await findRegenerableWorkspaceDirs(cwd, { signal });
const ignored = (await gitCacheCommand(cwd, ["ls-files", "--others", "--ignored", "--exclude-standard", "-z"], signal)).split("\0").filter(Boolean);
const unexplained = ignored.filter(file => !(options.preserveSettings && file === PAPERCLIP_CLAUDE_SETTINGS_FILE) && !regenerable.some(dir => file.startsWith(`${dir}/`))
&& !(options.restorePoint && (/^\.env(\..*)?$/.test(path.posix.basename(file)) || restorePointSavesIgnoredFile(file))));
const checkouts = new Set<string>();
for (const file of unexplained) {
const [namespace, name, rest] = file.split("/", 3);
if (namespace !== PROJECT_REPOSITORIES_DIR || !name || rest === undefined || !options.proveManagedCheckout
|| name.startsWith("task-repo-") || name.includes(".clone-")) throw new Error("unique_ignored_files");
checkouts.add(name);
}
for (const name of checkouts) {
const checkout = path.join(cwd, PROJECT_REPOSITORIES_DIR, name);
const entry = await fs.lstat(checkout);
if (!entry.isDirectory() || entry.isSymbolicLink()) throw new Error("unique_ignored_files");
await options.proveManagedCheckout!(checkout, name, signal);
}
const entries = await gitCacheCommand(cwd, ["ls-tree", "-r", "HEAD"], signal);
if (/^160000 /m.test(entries)) throw new Error("unsupported_submodules");
const gitDir = (await gitCacheCommand(cwd, ["rev-parse", "--absolute-git-dir"], signal)).trim();
const common = (await gitCacheCommand(cwd, ["rev-parse", "--path-format=absolute", "--git-common-dir"], signal)).trim();
for (const root of new Set([gitDir, common])) {
for (const name of ["shallow", "shallow.lock", "MERGE_HEAD", "CHERRY_PICK_HEAD", "REVERT_HEAD", "BISECT_LOG", "rebase-apply", "rebase-merge", "sequencer", "info/sparse-checkout", "objects/info/alternates", "refs/replace", "refs/stash", "modules", "lfs"]) {
if (await fs.lstat(path.join(root, name)).then(() => true, (e: NodeJS.ErrnoException) => { if(e.code === "ENOENT") return false; throw e; })) throw new Error("unsupported_local_git_state");
}
}
const original = options.restorePoint ? null : await gitCacheCommand(cwd, ["rev-parse", "--verify", "ORIG_HEAD"], signal).catch(() => null);
if (original) await gitCacheCommand(cwd, ["merge-base", "--is-ancestor", original.trim(), "HEAD"], signal)
.catch(() => { throw new Error("unique_original_head"); });
if (gitDir !== common) {
const allowed = new Set(["HEAD", "ORIG_HEAD", "FETCH_HEAD", "index", "commondir", "gitdir", "logs", "refs", "COMMIT_EDITMSG", "index.lock", "HEAD.lock"]);
if ((await fs.readdir(gitDir)).some(name => !allowed.has(name))) throw new Error("unknown_worktree_git_files");
await assertReproducibleFetchHead(cwd, gitDir, signal, options.verifyFetchedCommit);
async function assertEmptyRefs(root: string): Promise<void> {
const entries = await fs.readdir(root, { withFileTypes: true }).catch((error: NodeJS.ErrnoException) => {
if (error.code === "ENOENT") return []; throw error;
});
for (const entry of entries) {
if (!entry.isDirectory()) throw new Error("unique_worktree_refs");
await assertEmptyRefs(path.join(root, entry.name));
}
}
await assertEmptyRefs(path.join(gitDir, "refs"));
const message = await readCommitMessage(gitDir);
if (message !== null && message.trim() !== (await gitCacheCommand(cwd, ["log", "-1", "--format=%B"], signal)).trim()) {
const saved = options.restorePoint
? (await gitCacheCommand(cwd, ["log", "-g", "--format=%B%x00", "HEAD"], signal).catch(() => "")).split("\0").map(entry => entry.trim())
: [];
if (!saved.includes(message.trim())) throw new Error("uncommitted_git_message");
}
const logs = await fs.readdir(path.join(gitDir, "logs")).catch((error: NodeJS.ErrnoException) => {
if (error.code === "ENOENT") return []; throw error;
});
if (logs.some(name => name !== "HEAD")) throw new Error("unknown_worktree_git_logs");
}
// Packed stashes/replacement refs are just as important as loose refs.
if ((await gitCacheCommand(cwd, ["for-each-ref", "--format=%(refname)", "refs/stash", "refs/replace"], signal)).trim()) throw new Error("unique_git_refs");
// Rewrites can leave commits reachable only from this worktree's reflog.
for (const commit of options.restorePoint ? [] : new Set((await gitCacheCommand(cwd, ["reflog", "show", "--format=%H", "HEAD"], signal)).split("\n").filter(Boolean))) {
await gitCacheCommand(cwd, ["merge-base", "--is-ancestor", commit, "HEAD"], signal).catch(() => { throw new Error("unique_reflog_commits"); });
}
return regenerable;
}
export async function inspectGitCacheRecipe(workspace: Workspace, resolveGitAuth: GitRemoteAuthProvider, signal?: AbortSignal,
options: { proveManagedCheckout?: ManagedCheckoutProver } = {}): Promise<GitCacheRestoration> {
return (await inspectRecipe(workspace, resolveGitAuth, signal, options, false)).recipe;
}
export async function inspectGitCacheRecipeForRestorePoint(workspace: Workspace, resolveGitAuth: GitRemoteAuthProvider, signal?: AbortSignal,
options: { proveManagedCheckout?: ManagedCheckoutProver } = {}) {
return inspectRecipe(workspace, resolveGitAuth, signal, options, true);
}
async function inspectRecipe(workspace: Workspace, resolveGitAuth: GitRemoteAuthProvider, signal: AbortSignal | undefined,
options: { proveManagedCheckout?: ManagedCheckoutProver }, restorePoint: boolean): Promise<{ recipe: GitCacheRestoration; headOnRemote: boolean }> {
if (workspace.mode !== "isolated_workspace" || workspace.strategyType !== "git_worktree"
|| !isRuntimeOwnedGitBranch(workspace.metadata) || !workspace.sourceIssueId || !workspace.repoUrl
|| !workspace.cwd || !workspace.branchName || (workspace.providerRef && workspace.providerRef !== workspace.cwd)) throw new Error("unproven_workspace_ownership");
let source = await verifiedRecordedIsolatedRemoteSource(workspace);
if (!source) {
const common = (await gitCacheCommand(workspace.cwd, ["rev-parse", "--path-format=absolute", "--git-common-dir"], signal)).trim();
for (const candidate of managedSourcePaths(workspace)) {
if (await fs.realpath(path.join(candidate, ".git")).catch(() => null) === common) {
source = await verifiedGitCacheSource(workspace, candidate); break;
}
}
}
if (!source) throw new Error("unproven_source_ownership");
const stat = await fs.lstat(workspace.cwd);
if (!stat.isDirectory() || stat.isSymbolicLink() || await fs.realpath(workspace.cwd) !== path.join(await fs.realpath(path.dirname(workspace.cwd)), path.basename(workspace.cwd))) throw new Error("unsafe_worktree_path");
if (!(await fs.lstat(path.join(workspace.cwd, ".git"))).isFile()) throw new Error("not_linked_worktree");
if ((await gitCacheCommand(workspace.cwd, ["rev-parse", "--path-format=absolute", "--git-common-dir"], signal)).trim() !== path.join(await fs.realpath(source), ".git")) throw new Error("source_identity_changed");
if ((await gitCacheCommand(workspace.cwd, ["symbolic-ref", "--short", "HEAD"], signal)).trim() !== workspace.branchName) throw new Error("branch_identity_changed");
await assertReproducibleGitFiles(workspace.cwd, signal, { ...options, preserveSettings: true, restorePoint,
verifyFetchedCommit: (commit, verificationSignal) => verifyRemoteGitCommit(workspace.repoUrl!, commit, resolveGitAuth, verificationSignal) });
const commit = (await gitCacheCommand(workspace.cwd, ["rev-parse", "HEAD"], signal)).trim();
let headOnRemote = true;
try { await verifyRemoteGitCommit(workspace.repoUrl, commit, resolveGitAuth, signal); }
catch (error) {
if (!restorePoint || signal?.aborted) throw error;
headOnRemote = false;
}
return { headOnRemote, recipe: { schema: "paperclip.git-cache-restoration.v1", companyId: workspace.companyId, workspaceId: workspace.id,
sourceIssueId: workspace.sourceIssueId, repoUrl: workspace.repoUrl, commit, branchName: workspace.branchName,
sourceKind: recordedIsolatedRemoteSource(workspace) ? "isolated_remote" : "managed_project", projectId: workspace.projectId,
sourcePath: source, worktreePath: workspace.cwd, gitBranchOwnershipVersion: 1, verifiedAt: new Date().toISOString() } };
}
export function readGitCacheExpiration(workspace: Workspace): GitCacheExpiration | null {
const state = workspace.metadata?.gitCacheExpiration as GitCacheExpiration | undefined;
if (!state) return null;
const recipe = readGitCacheRestoration(workspace);
if (!recipe || state.schema !== "paperclip.git-cache-expiration.v1" || !Number.isSafeInteger(state.generation)
|| state.generation !== workspace.metadata?.lifecycleGeneration
|| (state.lockToken !== undefined && !/^[a-f0-9]{64}$/.test(state.lockToken))
|| state.worktreeTrash !== path.join(path.dirname(recipe.worktreePath), `.expired-git-worktree-${workspace.id}-${state.generation}`)
|| path.dirname(state.worktreeAdmin) !== path.join(recipe.sourcePath, ".git", "worktrees")
|| (state.baseTrash !== null && state.baseTrash !== path.join(path.dirname(recipe.sourcePath), `.expired-git-base-${workspace.id}-${state.generation}`))) {
throw new GitCacheRestorationFailure();
}
return state;
}
export async function gitCacheExpirationLockLocations(workspace: Workspace): Promise<GitExpirationLockLocation[]> {
const state = readGitCacheExpiration(workspace);
const recipe = readGitCacheRestoration(workspace);
if (!state || !recipe || !state.lockToken || state.completed) return [];
const branchPath = (branch: string) => {
if (!branch || branch.split("/").some(part => !part || part === "." || part === "..") || branch.includes("\\")) throw new GitCacheRestorationFailure();
return `refs/heads/${branch}.lock`;
};
const locations: GitExpirationLockLocation[] = [
...["index.lock", "HEAD.lock"].map(name => ({ root: recipe.sourcePath,
relativePath: path.relative(recipe.sourcePath, path.join(state.worktreeAdmin, name)), checkout: null })),
{ root: recipe.sourcePath, relativePath: path.join(".git", branchPath(recipe.branchName)), checkout: null },
];
for (const checkout of recipe.contents?.checkouts ?? []) {
if (checkout.worktree) {
const base = await recordedNestedBase(recipe, checkout.source.repoUrl, checkout.worktree.base);
for (const name of ["index.lock", "HEAD.lock"]) {
locations.push({ root: base, relativePath: path.join(".git", "worktrees", checkout.worktree.admin, name), checkout: checkout.name });
}
if (checkout.branchName) locations.push({ root: base, relativePath: path.join(".git", branchPath(checkout.branchName)), checkout: checkout.name });
continue;
}
for (const name of ["index.lock", "HEAD.lock", ...(checkout.branchName ? [branchPath(checkout.branchName)] : [])]) {
locations.push({ root: recipe.worktreePath, relativePath: path.join(PROJECT_REPOSITORIES_DIR, checkout.name, ".git", name), checkout: checkout.name });
}
}
return locations;
}
async function recordedNestedBase(recipe: GitCacheRestoration, repoUrl: string, recorded: string) {
if (!recipe.projectId) throw new GitCacheRestorationFailure();
const base = await recordedNestedRepositoryBase(recorded,
nestedRepositoryBaseCandidates({ companyId: recipe.companyId, projectId: recipe.projectId, repoUrl }));
if (!base) throw new GitCacheRestorationFailure();
return base;
}
export async function recoverGitCacheExpirationLocks(workspace: Workspace) {
const state = readGitCacheExpiration(workspace);
if (!state?.lockToken || state.completed || !["archived", "cleanup_failed"].includes(workspace.status ?? "")) return;
const recipe = readGitCacheRestoration(workspace)!;
if (await fs.lstat(recipe.sourcePath).then(() => true, (error: NodeJS.ErrnoException) => {
if (error.code === "ENOENT") return false;
throw error;
})) await verifiedGitCacheSource(workspace, recipe.sourcePath);
await recoverGitExpirationLocks(await gitCacheExpirationLockLocations(workspace), state.lockToken);
}
// Both restart recovery and reopening finish only recorded, uniquely named trash.
// No directory scan or age heuristic can authorize deleting an arbitrary path.
export async function finishGitCacheTrash(workspace: Workspace) {
const state = readGitCacheExpiration(workspace);
if (!state) return;
const recipe = readGitCacheRestoration(workspace)!;
const exists = (target: string) => fs.lstat(target).then(() => true, (error: NodeJS.ErrnoException) => {
if (error.code === "ENOENT") return false; throw error;
});
for (const target of [state.worktreeTrash, state.baseTrash]) {
if (!target || !await exists(target)) continue;
const entry = await fs.lstat(target);
if (!entry.isDirectory() || entry.isSymbolicLink()
|| await fs.realpath(target) !== path.join(await fs.realpath(path.dirname(target)), path.basename(target))) throw new GitCacheRestorationFailure();
await fs.rm(target, { recursive: true, force: true });
}
if (!await exists(recipe.worktreePath) && await exists(state.worktreeAdmin)) {
await verifiedGitCacheSource(workspace, recipe.sourcePath);
const entry = await fs.lstat(state.worktreeAdmin);
if (!entry.isDirectory() || entry.isSymbolicLink()
|| await fs.realpath(state.worktreeAdmin) !== path.join(await fs.realpath(recipe.sourcePath), ".git", "worktrees", path.basename(state.worktreeAdmin))) {
throw new GitCacheRestorationFailure();
}
const pointer = (await fs.readFile(path.join(state.worktreeAdmin, "gitdir"), "utf8")).trim();
const expected = path.join(await fs.realpath(path.dirname(recipe.worktreePath)), path.basename(recipe.worktreePath), ".git");
if (pointer !== expected) throw new GitCacheRestorationFailure();
await fs.rm(state.worktreeAdmin, { recursive: true, force: true });
}
if (await exists(recipe.worktreePath)) return;
for (const checkout of recipe.contents?.checkouts ?? []) {
if (!checkout.worktree) continue;
const base = await recordedNestedBase(recipe, checkout.source.repoUrl, checkout.worktree.base);
if (!await exists(base)) continue;
await forgetExpiredNestedRepositoryWorktree({ base: await fs.realpath(base), admin: checkout.worktree.admin,
checkout: path.join(await fs.realpath(path.dirname(recipe.worktreePath)), path.basename(recipe.worktreePath), PROJECT_REPOSITORIES_DIR, checkout.name),
branchName: checkout.branchName, commit: checkout.commit });
}
}
/** Called only after authorization and while holding the source/lifecycle fences. */
export function gitRestorePointExpired(recipe: GitCacheRestoration) {
return Boolean(recipe.restorePoint?.expiredAt) || (recipe.contents?.checkouts ?? []).some(checkout => Boolean(checkout.restorePoint?.expiredAt));
}
export async function restoreGitCacheBase(workspace: Workspace, recipe: GitCacheRestoration, resolveGitAuth: GitRemoteAuthProvider,
storage?: StorageService, options: { freshStart?: boolean } = {}) {
if (readGitCacheRestoration(workspace)?.commit !== recipe.commit) throw new GitCacheRestorationFailure();
try {
await finishGitCacheTrash(workspace);
const present = await fs.lstat(recipe.sourcePath).then(() => true, (error: NodeJS.ErrnoException) => {
if (error.code === "ENOENT") return false;
throw error;
});
if (!present && recipe.sourceKind === "managed_project") {
const root = resolvePaperclipInstanceRoot();
let parent = await fs.realpath(root);
for (const segment of path.relative(root, path.dirname(recipe.sourcePath)).split(path.sep)) {
parent = path.join(parent, segment);
await fs.mkdir(parent, { mode: 0o700 }).catch((error: NodeJS.ErrnoException) => { if (error.code !== "EEXIST") throw error; });
const entry = await fs.lstat(parent);
if (!entry.isDirectory() || entry.isSymbolicLink() || await fs.realpath(parent) !== parent) throw new GitCacheRestorationFailure();
}
}
const cwd = present ? await verifiedGitCacheSource(workspace, recipe.sourcePath)
: recipe.sourceKind === "isolated_remote"
? await materializeIsolatedRemoteWorkspaceSource({ companyId: recipe.companyId, issueId: recipe.sourceIssueId,
repoUrl: recipe.repoUrl, resolveGitAuth })
: (await materializeManagedProjectWorkspace(recipe.sourcePath, { repoUrl: recipe.repoUrl, resolveGitAuth })).cwd;
await verifiedGitCacheSource(workspace, recipe.sourcePath);
if (!cwd) throw new GitCacheRestorationFailure();
if (options.freshStart) return cwd;
if (recipe.restorePoint) {
await ensureGitRestorePointCommit({ storage: storage ?? getStorageService(), workspace: { companyId: workspace.companyId, id: workspace.id },
ref: recipe.restorePoint, repositoryCwd: cwd });
}
await ensureSavedGitCommit(cwd, recipe.repoUrl, recipe.commit, resolveGitAuth);
return cwd;
} catch { throw new GitCacheRestorationFailure(); }
}
export async function ensureSavedGitCommit(cwd: string, repoUrl: string, commit: string, resolveGitAuth: (url: string) => Promise<{ configArgs: string[]; env: Record<string, string> } | null>) {
if (!shaPattern.test(commit)) throw new GitCacheRestorationFailure();
if (await gitCacheCommand(cwd, ["cat-file", "-e", `${commit}^{commit}`]).then(() => true, () => false)) return;
try {
const auth = await resolveGitAuth(repoUrl);
await execFile("git", [...auth?.configArgs ?? [], "-c", `core.hooksPath=${os.devNull}`, "-C", cwd, "fetch", "--no-tags", "--no-recurse-submodules", "--", repoUrl, commit], {
timeout: 60_000, maxBuffer: 1024 * 1024, env: { ...process.env, ...auth?.env, GIT_TERMINAL_PROMPT: "0" },
});
if ((await gitCacheCommand(cwd, ["rev-parse", `${commit}^{commit}`])).trim() !== commit) throw new GitCacheRestorationFailure();
} catch { throw new GitCacheRestorationFailure(); }
}
/** Runtime callers must bind the recipe to the selected workspace and base. */
export function savedGitWorkspaceRestoration(input: { metadata?: Record<string, unknown> | null; companyId: string;
workspaceId?: string | null; cwd: string; baseCwd: string; repoUrl: string | null; branchName: string | null }) {
const raw = input.metadata?.[GIT_CACHE_RESTORATION_KEY];
if (raw === undefined) return null;
const value = raw as GitCacheRestoration | null;
if (!value || !restorationSchemaMatches(value) || !shaPattern.test(value.commit)
|| value.companyId !== input.companyId || value.workspaceId !== input.workspaceId
|| value.worktreePath !== input.cwd || value.sourcePath !== input.baseCwd
|| value.repoUrl !== input.repoUrl || value.branchName !== input.branchName
|| !isRuntimeOwnedGitBranch(input.metadata)
|| (value.restoredAt !== undefined && !Number.isFinite(Date.parse(value.restoredAt)))
|| (value.contentsRestoredAt != null && !Number.isFinite(Date.parse(value.contentsRestoredAt)))) throw new GitCacheRestorationFailure();
validateGitWorkspaceContents(value.contents);
validateRestorePoints(value, { companyId: input.companyId, id: input.workspaceId ?? "" });
const restorePoint: GitRestorePointRef | null = value.restorePoint ?? null;
return { commit: value.commit, restored: value.restoredAt !== undefined, contents: value.contents, restorePoint,
contentsRestored: value.contentsRestoredAt === undefined ? value.restoredAt !== undefined : value.contentsRestoredAt !== null };
}