fix: fence nested Git repositories during workspace migration (#19805)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Online workspace migration must preserve local files before it
assigns remote authority.
> - A task directory can contain a Git repository below its root.
> - The source inspector treated that directory as ordinary, but the
archive owner correctly required a shared Git source fence.
> - This change includes each nested repository in the existing source
ownership and locking checks.
> - The archive still captures the full task directory, including
sibling files and staged and unstaged Git work.

## Linked Issues or Issue Description

Related: #19688 and #19409. Those changes cover physical source fences
and nested Git archive reconstruction. This change repairs source
classification for an ordinary wrapper that contains a repository. No
duplicate fix was found.

**What happened?**

Online migration admitted a managed task directory whose root had no
`.git` marker. A repository below `.review/repository` then reached the
archive owner without a shared Git source fence. Capture failed with
`remote_workspace_archive_online_source_git_requires_shared_fence`. Its
operation and pending snapshot remained available for canonical resume.

**Expected behavior**

Migration must include the nested Git stores and all their source owners
in the existing physical fence. It must preserve the wrapper directory
and sibling files. It must refuse foreign aliases, unowned stores, busy
shared sources, and source changes.

**Steps to reproduce**

1. Create a managed ordinary task directory with a sibling file.
2. Create a real Git repository below `.review/repository` and leave
staged and unstaged edits.
3. Run canonical online workspace migration.
4. Observe the shared-fence refusal, instead of a durable archive that
can restore all the work.

## What Changed

- Add a bounded metadata walk for nested Git roots. Reuse the existing
strict Git pointer, common-directory, and alternate-store reader.
- Add nested source identities to the existing alias closure, company
checks, source locks, and metadata fingerprints. Keep the wrapper as the
capture root.
- Validate bounded Git-maintained reverse references in selected and
borrowed stores. Require stable two-way pointers and containment by
included registered workspaces. Bind the listings and pointer receipts
into publication revalidation.
- Add seventeen integration cases for fresh migration, pending-journal
resume, shared linked and alternate stores, unrelated progress, foreign
and unowned stores, and late metadata changes.

## Verification

- Genuine baseline RED: both fresh and pending-journal cases failed at
the existing archive shared-fence guard before the source fix.
- The first treatment passed six of seven cases. The late-change case
correctly failed at the earlier recapture guard; its expected-error
assertion was corrected. The final test also verifies that the canonical
Git fingerprint changed.
- Head `8bb9f5a3ce`: real PostgreSQL, all **187 tests passed** in 59.22
seconds across `remote-workspace-migration.test.ts` and
`workspace-git-source-roots.test.ts`. The canonical wrapper creates
disposable databases and completed cleanup. Run from `server/`:
  ```sh
node ../cli/node_modules/tsx/dist/cli.mjs
../packages/db/src/test-postgres-command.ts -- node
../node_modules/vitest/vitest.mjs run
src/services/remote-workspace-migration.test.ts
src/services/workspace-git-source-roots.test.ts --hookTimeout 90000
--maxWorkers 1 --no-file-parallelism --fsModuleCache
  ```
- Required alias RED: both hidden ordinary-wrapper cases failed before
the reverse-reference fix. The shared case bypassed its held writer
lock; the foreign case was incorrectly eligible. Both now pass. A
further RED proved the same omission through a borrowed store; its
hidden writer is now included too.
- Review RED: a repository below `.review/.paperclip-runtime` reached
the archive guard. Discovery now excludes the runtime name only at the
capture root, matching the archive.
- The positive cases download the authenticated immutable archive,
remove the disposable source, and cold restore the sibling file, HEAD,
index content, and working content. Resume retains the exact operation
and pending snapshot IDs.
- Resume fidelity: migration, snapshot, and layout owners are
byte-identical to source revision `2d4f80504e`. Source kind comes from
the workspace row, not online classification. The unchanged serializer
writes the same empty source, source fingerprint, marker, fixed empty
child scope, and pending generation-1/epoch-1 snapshot at the tested
failure boundary. The test asserts that shape and retains the exact IDs
on resume. The fixture injects the failure at that unchanged boundary;
it does not execute the old binary.
- Shared-store cases hold a real PostgreSQL source lock. The related
import is refused before a journal exists, while an unrelated ordinary
import completes.
- Invalid-link controls reject foreign and unregistered roots, malformed
pointers, and symlinked metadata before a journal exists. A new detached
worktree after upload changes the listing fingerprint and blocks
authority publication.
- Final head `e525b6b1a5`: all 17 focused real PostgreSQL cases pass in
16.23 seconds after the enclosing-host-repository review fix. Its true
RED first showed the valid managed wrapper as ineligible. The cold
restore now excludes the ancestor Git directory and host-only file and
leaves the host HEAD unchanged. Server TypeScript passes again.
- Server TypeScript and module boundary checks pass. Canonical agent
preflight passes. Environment: macOS arm64, Node 24.22.2, real
PostgreSQL and Git.
- A first test launch lacked the built plugin SDK, and a later sandbox
launch could not open TSX IPC. Both were setup failures, not regression
results. The existing dependency build and permitted isolated test
runner resolved them.
- Final exact-head CI passes: [run
42363940320](https://github.com/paperclipai/paperclip/actions/runs/42363940320).
The final gate has 56 successful checks, four skipped checks, one
neutral check, and no pending or failed checks. Greptile reviewed this
head at 5/5, with zero open review threads. No hosted workspace was
changed or imported by this PR work.

## Risks

- Nested discovery and reverse-reference checks add bounded directory
metadata I/O. Reverse listings allow at most 256 worktrees per store,
and all links must resolve to registered physical workspace roots. Stale
or unsupported links fail closed.
- Nested discovery refuses more than 100,000 entries, 64 directory
levels, or 64 repositories. A large or unsupported tree remains
ineligible. This includes dependency or build trees that exceed those
limits. No broad directory skip or metadata cache bypasses final source
checks.
- The archive guard, source lock owner, runtime writer checks,
publication checks, and recovery predicates remain required. This change
grants no permission to retire source work or ignore a held journal.
- No schema, API, runtime setting, provider lifecycle, or background
worker changes.
- This PR documents the internal source-classification contract above.
It adds no operator command or configuration to document separately.

## Model Used

OpenAI `gpt-6.1-sol` through Codex, with reasoning, repository tools,
code execution, and regression tests. The context window size is not
exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-11-07 04:40:08 -06:00
1 parent 795ad16c1f
commit e801be7aa9
3 files changed
+338 -13

No files matched your search

+56 -13
View File
@@ -1,11 +1,12 @@
import fs from "node:fs/promises";
import path from "node:path";
import { createHash } from "node:crypto";
import { and, eq, gt, inArray, ne, sql } from "drizzle-orm";
import { executionWorkspaces, heartbeatRuns, issueRecoveryActions, type Db } from "@paperclipai/db";
import { resolvePaperclipInstanceRoot } from "../home-paths.js";
import { inspectImportedWorkspaceSourcePaths, inspectOnlineOrdinarySource, physicalPath, requiresOnlineOrdinarySource, type WorkspaceSourceScope } from "./remote-workspace-migration-tree.js";
import { workspaceSourceIdentities } from "./workspace-source-lock.js";
import { readWorkspaceGitSourceRoots } from "./workspace-git-source-roots.js";
import { readNestedWorkspaceGitSourceRoots, readWorkspaceGitLinkedRoots, readWorkspaceGitSourceRoots, type WorkspaceGitSourceRoots } from "./workspace-git-source-roots.js";
import { inspectOnlineLegacyWorkspaceRecoveryHold, inspectOnlineLegacyWorkspaceRecoverySource, type OnlineLegacyWorkspaceRecoveryHold } from "./online-legacy-workspace-recovery.js";
import type { LegacyWorkspaceRecoverySource } from "./legacy-workspace-recovery-migration.js";
import { hasPendingGitExpiration } from "./git-workspace-expiration.js";
@@ -65,36 +66,75 @@ export async function inspectOnlineWorkspaceSource(db: ReadDb, workspace: Worksp
const cwd = workspace.cwd ?? workspace.providerRef;
if (!cwd || !path.isAbsolute(cwd) || !["local_fs", "git_worktree"].includes(workspace.providerType)) return null;
const ordinary = await inspectOnlineOrdinarySource(db, workspace);
let nestedGit: WorkspaceGitSourceRoots[] = [];
if (ordinary) {
const marker = await fs.lstat(path.join(ordinary.root, ".git")).catch((error: NodeJS.ErrnoException) => {
if (error.code !== "ENOENT") throw error; return null;
});
if (!marker) return { ...ordinary, kind: "ordinary", projectIds: ordinary.projectId ? [ordinary.projectId] : [], companions: [] };
if (!marker) {
try { nestedGit = await readNestedWorkspaceGitSourceRoots(ordinary.root); } catch { return null; }
if (!nestedGit.length) return { ...ordinary, kind: "ordinary", projectIds: ordinary.projectId ? [ordinary.projectId] : [], companions: [] };
}
}
let selected: Identities;
try { selected = await workspaceSourceIdentities(workspace.companyId, cwd); } catch { return null; }
if (!selected.git || selected.git.root !== await physicalPath(cwd) || selected.unownedPaths.length || !workspace.sourceIssueId
try {
selected = nestedGit.length && ordinary ? { projectIds: ordinary.projectId ? [ordinary.projectId] : [],
issueIds: [...ordinary.issueIds], unownedPaths: [], git: null } : await workspaceSourceIdentities(workspace.companyId, cwd);
for (const source of nestedGit) {
const nested = await workspaceSourceIdentities(workspace.companyId, source.root);
if (!nested.git || nested.git.fingerprint !== source.fingerprint) return null;
selected.projectIds = [...new Set([...selected.projectIds, ...nested.projectIds])].sort();
selected.issueIds = [...new Set([...selected.issueIds, ...nested.issueIds])].sort();
selected.unownedPaths = [...new Set([...selected.unownedPaths, ...nested.unownedPaths])].sort();
}
} catch { return null; }
const captureRoot = await physicalPath(cwd), gitSources = selected.git ? [selected.git] : nestedGit;
if (!gitSources.length || selected.git && selected.git.root !== captureRoot || selected.unownedPaths.length || !workspace.sourceIssueId
|| workspace.cwd && workspace.providerRef && workspace.cwd !== workspace.providerRef) return null;
const head = await fs.lstat(path.join(selected.git.gitDirectory.path, "HEAD")).catch((error: NodeJS.ErrnoException) => {
if (error.code !== "ENOENT") throw error; return null;
});
if (!head || !head.isFile() && !head.isSymbolicLink()) return null;
for (const source of gitSources) {
const head = await fs.lstat(path.join(source.gitDirectory.path, "HEAD")).catch((error: NodeJS.ErrnoException) => {
if (error.code !== "ENOENT") throw error; return null;
});
if (!head || !head.isFile() && !head.isSymbolicLink()) return null;
}
const linkedSources = new Map<string, Awaited<ReturnType<typeof readWorkspaceGitLinkedRoots>>[number]>();
try {
for (const source of gitSources) {
for (const proof of await readWorkspaceGitLinkedRoots(source)) {
const existing = linkedSources.get(proof.commonDirectory.path);
if (existing && JSON.stringify(existing) !== JSON.stringify(proof)) return null;
linkedSources.set(proof.commonDirectory.path, proof);
}
}
for (const proof of linkedSources.values()) {
for (const link of proof.links) {
const linked = await workspaceSourceIdentities(workspace.companyId, link.root.path);
if (!linked.git || linked.git.gitDirectory.path !== link.gitDirectory.path) return null;
selected.projectIds = [...new Set([...selected.projectIds, ...linked.projectIds])].sort();
selected.issueIds = [...new Set([...selected.issueIds, ...linked.issueIds])].sort();
selected.unownedPaths = [...new Set([...selected.unownedPaths, ...linked.unownedPaths])].sort();
}
}
} catch { return null; }
if (selected.unownedPaths.length) return null;
const selectedFingerprint = createHash("sha256").update(JSON.stringify([gitSources, [...linkedSources.values()]])).digest("hex");
const entry = await fs.lstat(cwd);
const rows = await db.select().from(executionWorkspaces).where(and(eq(executionWorkspaces.companyId, workspace.companyId),
inArray(executionWorkspaces.providerType, ["local_fs", "git_worktree"]))).orderBy(executionWorkspaces.id).limit(1001);
if (rows.length > 1000) return null;
const nodes: Array<{ row: Workspace; identities: Identities }> = [];
const nodes: Array<{ row: Workspace; identities: Identities; fingerprint: string | null }> = [];
for (const row of rows) {
const paths = localWorkspaceSourcePaths(row);
if (!paths.length || paths.some(cwd => !path.isAbsolute(cwd))) return null;
try {
const identities = await workspaceSourceIdentities(workspace.companyId, paths[0]);
const identities = paths[0] === captureRoot ? { ...selected, projectIds: [...selected.projectIds], issueIds: [...selected.issueIds] }
: await workspaceSourceIdentities(workspace.companyId, paths[0]);
for (const cwd of paths.slice(1)) {
const alternate = await workspaceSourceIdentities(workspace.companyId, cwd);
identities.projectIds = [...new Set([...identities.projectIds, ...alternate.projectIds])].sort();
identities.issueIds = [...new Set([...identities.issueIds, ...alternate.issueIds])].sort();
}
nodes.push({ row, identities });
nodes.push({ row, identities, fingerprint: paths[0] === captureRoot ? selectedFingerprint : identities.git?.fingerprint ?? null });
} catch { return null; }
}
const projectIds = new Set(selected.projectIds), physicalIssueIds = new Set(selected.issueIds), aliases = new Set<string>();
@@ -128,6 +168,9 @@ export async function inspectOnlineWorkspaceSource(db: ReadDb, workspace: Worksp
}
if (!aliases.has(workspace.id) || aliases.size > 256) return null;
const included = nodes.filter(node => aliases.has(node.row.id));
const registeredRoots = await Promise.all(included.flatMap(node => localWorkspaceSourcePaths(node.row)).map(physicalPath));
if ([...linkedSources.values()].some(proof => proof.links.some(link => !registeredRoots.some(root =>
link.root.path === root || link.root.path.startsWith(root + path.sep))))) return null;
const issueIds = [...new Set([...physicalIssueIds, ...included.map(node => node.row.sourceIssueId!)])].sort();
if (issueIds.length + projectIds.size > 128) return null;
const instance = await physicalPath(resolvePaperclipInstanceRoot());
@@ -179,11 +222,11 @@ export async function inspectOnlineWorkspaceSource(db: ReadDb, workspace: Worksp
if (proof.sources.length) settledPreparedWorkspaceIds.push(proof.workspaceId);
for (const id of proof.recoveryIssueIds) settledRecoveryIssueIds.add(id);
}
return { kind: "git", root: selected.git.root, projectId: null, projectIds: [...projectIds].sort(), issueIds,
return { kind: "git", root: captureRoot, projectId: null, projectIds: [...projectIds].sort(), issueIds,
workspaceIds: [...aliases].sort(), device: entry.dev, inode: entry.ino, gitSourceRoots, companions,
physicalRoots: included.flatMap(node => localWorkspaceSourcePaths(node.row)).sort(),
settledPreparedWorkspaceIds: settledPreparedWorkspaceIds.sort(), settledRecoveryIssueIds: [...settledRecoveryIssueIds].sort(),
settledFinalizationRunIds: [...new Set(companions.flatMap(proof => proof.sources.map(source => source.runId)))].sort(),
gitSourceFingerprints: included.map(node => ({ workspaceId: node.row.id, fingerprint: node.identities.git?.fingerprint ?? null })), importedChildren,
gitSourceFingerprints: included.map(node => ({ workspaceId: node.row.id, fingerprint: node.fingerprint })), importedChildren,
};
}
@@ -19,6 +19,7 @@ import { migrateWorkspaceTrees } from "./remote-workspace-migration-batch.js";
import { inventoryRemoteWorkspaceMigration, migrateRemoteWorkspace } from "./remote-workspace-migration.js";
import { getWorkspaceSnapshot, loadWorkspaceSnapshotArchive } from "./remote-workspace-snapshot-storage.js";
import { restoreRemoteWorkspaceArchive } from "./remote-workspace-snapshot-archive.js";
import * as workspaceArchive from "./remote-workspace-snapshot-archive.js";
import { captureTaskDirectorySnapshot, removeTaskDirectoryTree } from "./isolated-task-directory-snapshot.js";
import { captureRestorePoint, sealRestorePoint } from "./git-restore-point.js";
import type { TaskDirectoryRestorePointRef } from "./isolated-task-directory-retention.js";
@@ -34,6 +35,8 @@ import { managedProjectCheckoutName } from "./managed-project-checkout-name.js";
import { captureGitWorkspaceContents, readManagedCheckoutContents } from "./git-workspace-contents.js";
import { isolatedRemoteSourcePath } from "./isolated-remote-workspace-source.js";
import { verifiedGitCacheSource } from "./git-workspace-restoration.js";
import { withWorkspaceSourceLocks } from "./workspace-source-lock.js";
import { inspectOnlineWorkspaceSource } from "./online-workspace-source.js";
const execute = promisify(execFile);
const git = async (cwd: string, ...args: string[]) => (await execute("git", ["-C", cwd, ...args], { env: { ...process.env,
@@ -1143,6 +1146,220 @@ describe.skipIf(!support.supported)("remote workspace migration", () => {
expect(readRemoteWorkspaceAuthority((await workspace(f.workspaceId)).metadata)).toBeNull();
expect(await fs.readFile(path.join(f.cwd, "saved.txt"), "utf8")).toBe("new local edits\n");
});
async function nestedGitWrapperFixture(relative = ".review/repository") {
const original = await seed();
vi.stubEnv("PAPERCLIP_HOME", await fs.realpath(root));
const cwd = path.join(resolvePaperclipInstanceRoot(), "isolated-workspaces", original.companyId, original.issueId);
await fs.mkdir(path.dirname(cwd), { recursive: true }); await fs.rename(original.cwd, cwd);
await db.update(executionWorkspaces).set({ cwd, providerRef: cwd }).where(eq(executionWorkspaces.id, original.workspaceId));
const f = { ...original, cwd }, nested = path.join(cwd, relative);
await fs.mkdir(nested, { recursive: true }); await fs.writeFile(path.join(nested, "tracked.txt"), "committed\n");
await git(nested, "init", "-q", "-b", "feature"); await git(nested, "add", "."); await git(nested, "commit", "-qm", "nested work");
const head = await git(nested, "rev-parse", "HEAD");
await fs.writeFile(path.join(nested, "tracked.txt"), "staged\n"); await git(nested, "add", "tracked.txt");
await fs.writeFile(path.join(nested, "tracked.txt"), "unstaged\n");
return { f, nested, head };
}
it.each(["fresh", "resume", "nested-runtime", "host-repository"])("imports a nested Git wrapper and retains its pending journal on resume (%s)", async mode => {
const relative = mode === "nested-runtime" ? ".review/.paperclip-runtime/repository" : ".review/repository";
const { f, head } = await nestedGitWrapperFixture(relative);
let hostHead: string | undefined;
if (mode === "host-repository") {
await git(root, "init", "-q", "-b", "host"); await fs.writeFile(path.join(root, "host-only.txt"), "host repository\n");
await git(root, "add", "host-only.txt"); await git(root, "commit", "-qm", "host work"); hostHead = await git(root, "rev-parse", "HEAD");
}
expect(await inventoryRemoteWorkspaceMigration(db, f)).toMatchObject({ eligible: true });
let operationId: string | undefined, snapshotId: string | undefined;
if (mode === "resume") {
const capture = vi.spyOn(workspaceArchive, "captureRemoteWorkspaceDirectory")
.mockRejectedValueOnce(new Error("remote_workspace_archive_online_source_git_requires_shared_fence"));
try {
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true }))
.rejects.toThrow("remote_workspace_archive_online_source_git_requires_shared_fence");
} finally { capture.mockRestore(); }
const [operation] = await db.select().from(workspaceOperations), [snapshot] = await db.select().from(executionWorkspaceSnapshots);
expect(operation).toMatchObject({ status: "running", metadata: { source: { kind: "empty" }, sourceIssueBinding: f.workspaceId,
nestedWorkspaceScope: { version: 1, state: "fixed", children: [] } } });
expect(snapshot).toMatchObject({ status: "pending", sourceOperationId: operation!.id, generation: 1, epoch: 1,
sourceRunId: null, parentSnapshotId: null, byteSize: null, sha256: null });
operationId = operation!.id; snapshotId = snapshot!.id;
}
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).resolves.toMatchObject({ applied: true });
const authority = readRemoteWorkspaceAuthority((await workspace(f.workspaceId)).metadata)!;
const operations = await db.select().from(workspaceOperations), snapshots = await db.select().from(executionWorkspaceSnapshots);
expect(operations).toHaveLength(1); expect(snapshots).toHaveLength(1);
expect(operations[0]).toMatchObject({ ...(operationId ? { id: operationId } : {}), status: "succeeded" });
expect(snapshots[0]).toMatchObject({ ...(snapshotId ? { id: snapshotId } : {}), status: "committed", sourceOperationId: operations[0]!.id });
expect(authority.snapshotId).toBe(snapshots[0]!.id);
const ref = await getWorkspaceSnapshot(db, { ...f, snapshotId: authority.snapshotId! });
const archivePath = path.join(root, "nested.pcws"), targetDir = path.join(root, "restored-nested");
await loadWorkspaceSnapshotArchive({ storage, reference: ref!, targetPath: archivePath });
await fs.rm(f.cwd, { recursive: true });
await restoreRemoteWorkspaceArchive({ archivePath, targetDir, expectedManifestDigest: ref!.manifestDigest });
const restored = path.join(targetDir, relative);
expect(await fs.readFile(path.join(targetDir, "saved.txt"), "utf8")).toBe("original saved work\n");
expect(await git(restored, "rev-parse", "HEAD")).toBe(head);
expect(await git(restored, "show", ":tracked.txt")).toBe("staged");
expect(await fs.readFile(path.join(restored, "tracked.txt"), "utf8")).toBe("unstaged\n");
if (hostHead) {
expect(await git(root, "rev-parse", "HEAD")).toBe(hostHead);
expect(await fs.lstat(path.join(targetDir, ".git")).catch(() => null)).toBeNull();
expect(await fs.lstat(path.join(targetDir, "host-only.txt")).catch(() => null)).toBeNull();
}
});
it.each(["linked", "alternate", "alternate-linked-wrapper"])("fences a nested Git wrapper's shared %s source before journal creation", async mode => {
const { f, nested } = await nestedGitWrapperFixture();
const [sourceIssue] = await db.insert(issues).values({ companyId: f.companyId, title: "Shared Git source" }).returning();
const source = path.join(resolvePaperclipInstanceRoot(), "isolated-repository-sources", f.companyId, sourceIssue!.id, "checkout");
await fs.mkdir(source, { recursive: true }); await fs.writeFile(path.join(source, "source.txt"), "shared source\n");
await git(source, "init", "-q", "-b", "main"); await git(source, "add", "."); await git(source, "commit", "-qm", "shared source");
const sourceWorkspaceId = randomUUID();
await db.insert(executionWorkspaces).values({ id: sourceWorkspaceId, companyId: f.companyId, sourceIssueId: sourceIssue!.id,
mode: "isolated_workspace", strategyType: "git_worktree", providerType: "git_worktree", name: "Shared source", cwd: source, providerRef: source });
await fs.rm(nested, { recursive: true });
if (mode === "linked") await git(source, "worktree", "add", "-b", "nested", nested);
else { await git(root, "clone", "--shared", source, nested); await git(nested, "remote", "remove", "origin"); }
const workspaceIds = [f.workspaceId, sourceWorkspaceId], issueIds = [f.issueId, sourceIssue!.id];
let writerIssueId = sourceIssue!.id;
if (mode === "alternate-linked-wrapper") {
const [writer] = await db.insert(issues).values({ companyId: f.companyId, title: "Linked source writer" }).returning();
writerIssueId = writer!.id;
const writerWorkspaceId = randomUUID(), writerCwd = path.join(resolvePaperclipInstanceRoot(), "isolated-workspaces", f.companyId, writerIssueId);
await fs.mkdir(writerCwd, { recursive: true }); await git(source, "worktree", "add", "--detach", path.join(writerCwd, "nested"));
await db.insert(executionWorkspaces).values({ id: writerWorkspaceId, companyId: f.companyId, sourceIssueId: writerIssueId,
name: "Linked writer wrapper", mode: "isolated_workspace", strategyType: "project_primary", providerType: "local_fs", cwd: writerCwd, providerRef: writerCwd });
workspaceIds.push(writerWorkspaceId); issueIds.push(writerIssueId);
}
const scope = await inspectOnlineWorkspaceSource(db, await workspace(f.workspaceId));
expect(scope).toMatchObject({ kind: "git", root: f.cwd });
expect(scope!.workspaceIds).toEqual(workspaceIds.sort());
expect(scope!.issueIds).toEqual(issueIds.sort());
let acquire!: () => void, release!: () => void;
const acquired = new Promise<void>(resolve => { acquire = resolve; });
const released = new Promise<void>(resolve => { release = resolve; });
const holder = withWorkspaceSourceLocks(db, { companyId: f.companyId, issueIds: [writerIssueId] }, async () => {
acquire(); await released;
});
await acquired;
try {
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("workspace_source_busy");
expect(await db.select().from(workspaceOperations)).toHaveLength(0);
const [other] = await db.insert(issues).values({ companyId: f.companyId, title: "Unrelated workspace" }).returning();
const otherId = randomUUID(), otherCwd = path.join(resolvePaperclipInstanceRoot(), "isolated-workspaces", f.companyId, other!.id);
await fs.mkdir(otherCwd, { recursive: true }); await fs.writeFile(path.join(otherCwd, "other.txt"), "unrelated work\n");
await db.insert(executionWorkspaces).values({ id: otherId, companyId: f.companyId, sourceIssueId: other!.id,
name: "Unrelated workspace", mode: "isolated_workspace", strategyType: "project_primary", providerType: "local_fs", cwd: otherCwd, providerRef: otherCwd });
await db.update(issues).set({ executionWorkspaceId: otherId }).where(eq(issues.id, other!.id));
await expect(migrateRemoteWorkspace({ db, storage, companyId: f.companyId, workspaceId: otherId, apply: true }))
.resolves.toMatchObject({ applied: true });
}
finally { release(); await holder; }
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).resolves.toMatchObject({ applied: true });
expect(await fs.readFile(path.join(nested, "source.txt"), "utf8")).toBe("shared source\n");
});
it.each(["foreign-alias", "unowned-alternate"])("keeps a nested Git wrapper blocked by %s", async mode => {
const { f, nested } = await nestedGitWrapperFixture();
if (mode === "foreign-alias") {
const foreignCompanyId = randomUUID(), foreignIssueId = randomUUID(), foreignCwd = path.join(root, "foreign-worktree");
await db.insert(companies).values({ id: foreignCompanyId, name: "Foreign company", issuePrefix: "OTHER" });
await db.insert(issues).values({ id: foreignIssueId, companyId: foreignCompanyId, title: "Foreign task" });
await git(nested, "worktree", "add", "-b", "foreign", foreignCwd);
await db.insert(executionWorkspaces).values({ companyId: foreignCompanyId, sourceIssueId: foreignIssueId,
name: "Foreign alias", mode: "isolated_workspace", strategyType: "git_worktree", providerType: "git_worktree", cwd: foreignCwd, providerRef: foreignCwd });
} else {
const objects = path.join(root, "unowned-objects"); await fs.mkdir(objects);
await fs.writeFile(path.join(nested, ".git", "objects", "info", "alternates"), objects + "\n");
}
expect(await inventoryRemoteWorkspaceMigration(db, f)).toMatchObject({ eligible: false, reasons: ["online_workspace_source_scope_unproven"] });
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("online_workspace_source_scope_unproven");
expect(await db.select().from(workspaceOperations)).toHaveLength(0);
expect(await db.select().from(executionWorkspaceSnapshots)).toHaveLength(0);
expect(await fs.readFile(path.join(nested, "tracked.txt"), "utf8")).toBe("unstaged\n");
});
it.each([false, true])("fences another ordinary wrapper's nested linked Git alias (foreign=%s)", async foreign => {
const { f, nested } = await nestedGitWrapperFixture();
const companyId = foreign ? randomUUID() : f.companyId, issueId = randomUUID(), workspaceId = randomUUID();
if (foreign) await db.insert(companies).values({ id: companyId, name: "Foreign company", issuePrefix: "OTHER" });
await db.insert(issues).values({ id: issueId, companyId, title: "Nested linked alias" });
const cwd = path.join(resolvePaperclipInstanceRoot(), "isolated-workspaces", companyId, issueId);
await fs.mkdir(cwd, { recursive: true });
await git(nested, "worktree", "add", "-b", "hidden-alias", path.join(cwd, "nested"));
await db.insert(executionWorkspaces).values({ id: workspaceId, companyId, sourceIssueId: issueId,
name: "Ordinary wrapper alias", mode: "isolated_workspace", strategyType: "project_primary", providerType: "local_fs", cwd, providerRef: cwd });
if (foreign) {
expect(await inventoryRemoteWorkspaceMigration(db, f)).toMatchObject({ eligible: false, reasons: ["online_workspace_source_scope_unproven"] });
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("online_workspace_source_scope_unproven");
} else {
const scope = await inspectOnlineWorkspaceSource(db, await workspace(f.workspaceId));
expect(scope!.workspaceIds).toEqual([f.workspaceId, workspaceId].sort());
expect(scope!.issueIds).toEqual([f.issueId, issueId].sort());
let acquire!: () => void, release!: () => void;
const acquired = new Promise<void>(resolve => { acquire = resolve; });
const released = new Promise<void>(resolve => { release = resolve; });
const holder = withWorkspaceSourceLocks(db, { companyId, issueIds: [issueId] }, async () => { acquire(); await released; });
await acquired;
try {
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("workspace_source_busy");
} finally { release(); await holder; }
}
expect(await db.select().from(workspaceOperations)).toHaveLength(0);
expect(await db.select().from(executionWorkspaceSnapshots)).toHaveLength(0);
expect(readRemoteWorkspaceAuthority((await workspace(f.workspaceId)).metadata)).toBeNull();
if (!foreign) await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).resolves.toMatchObject({ applied: true });
});
it.each(["unregistered", "backlink", "pointer", "symlink-backlink"])("refuses a nested Git wrapper with an unproven linked %s", async mode => {
const { f, nested } = await nestedGitWrapperFixture(), issueId = randomUUID();
await db.insert(issues).values({ id: issueId, companyId: f.companyId, title: "Linked wrapper" });
const cwd = path.join(resolvePaperclipInstanceRoot(), "isolated-workspaces", f.companyId, issueId), alias = path.join(cwd, "nested");
await fs.mkdir(cwd, { recursive: true }); await git(nested, "worktree", "add", "-b", "linked-proof", alias);
if (mode !== "unregistered") await db.insert(executionWorkspaces).values({ companyId: f.companyId, sourceIssueId: issueId,
name: "Linked wrapper", mode: "isolated_workspace", strategyType: "project_primary", providerType: "local_fs", cwd, providerRef: cwd });
const admin = await git(alias, "rev-parse", "--absolute-git-dir"), backlink = path.join(admin, "gitdir");
if (mode === "backlink") await fs.writeFile(backlink, path.join(alias, "wrong-marker") + "\n");
if (mode === "pointer") await fs.writeFile(path.join(alias, ".git"), `gitdir: ${path.join(admin, "wrong")}\n`);
if (mode === "symlink-backlink") { await fs.rename(backlink, backlink + ".real"); await fs.symlink(backlink + ".real", backlink); }
expect(await inventoryRemoteWorkspaceMigration(db, f)).toMatchObject({ eligible: false, reasons: ["online_workspace_source_scope_unproven"] });
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("online_workspace_source_scope_unproven");
expect(await db.select().from(workspaceOperations)).toHaveLength(0);
expect(await db.select().from(executionWorkspaceSnapshots)).toHaveLength(0);
});
it("revalidates the linked Git listing before publishing a nested wrapper", async () => {
const { f, nested } = await nestedGitWrapperFixture(), issueId = randomUUID();
await db.insert(issues).values({ id: issueId, companyId: f.companyId, title: "Linked wrapper" });
const cwd = path.join(resolvePaperclipInstanceRoot(), "isolated-workspaces", f.companyId, issueId);
await fs.mkdir(cwd, { recursive: true }); await git(nested, "worktree", "add", "--detach", path.join(cwd, "first"));
await db.insert(executionWorkspaces).values({ companyId: f.companyId, sourceIssueId: issueId, name: "Linked wrapper",
mode: "isolated_workspace", strategyType: "project_primary", providerType: "local_fs", cwd, providerRef: cwd });
const before = await inspectOnlineWorkspaceSource(db, await workspace(f.workspaceId)), putFile = storage.putFile.bind(storage);
vi.spyOn(storage, "putFile").mockImplementationOnce(async input => {
const result = await putFile(input);
await git(nested, "worktree", "add", "--detach", path.join(cwd, "second"));
return result;
});
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("online_workspace_source_scope_changed");
const after = await inspectOnlineWorkspaceSource(db, await workspace(f.workspaceId));
expect(after!.workspaceIds).toEqual(before!.workspaceIds);
expect(after!.gitSourceFingerprints).not.toEqual(before!.gitSourceFingerprints);
expect(readRemoteWorkspaceAuthority((await workspace(f.workspaceId)).metadata)).toBeNull();
});
it("rechecks a nested Git wrapper's alternate metadata before publication", async () => {
const { f, nested } = await nestedGitWrapperFixture(), objects = path.join(f.cwd, "alternate-objects");
await fs.mkdir(objects);
const before = await inspectOnlineWorkspaceSource(db, await workspace(f.workspaceId));
expect(before).toMatchObject({ kind: "git", root: f.cwd });
const putFile = storage.putFile.bind(storage);
vi.spyOn(storage, "putFile").mockImplementationOnce(async input => {
const result = await putFile(input);
await fs.writeFile(path.join(nested, ".git", "objects", "info", "alternates"), objects + "\n");
return result;
});
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("remote_workspace_migration_source_changed");
const after = await inspectOnlineWorkspaceSource(db, await workspace(f.workspaceId));
expect(after).toMatchObject({ kind: "git", root: f.cwd });
expect(after!.gitSourceFingerprints).not.toEqual(before!.gitSourceFingerprints);
expect(readRemoteWorkspaceAuthority((await workspace(f.workspaceId)).metadata)).toBeNull();
expect(await fs.readFile(path.join(nested, "tracked.txt"), "utf8")).toBe("unstaged\n");
});
it("imports dirty unpushed Git work and restores it after the original folder disappears", async () => {
let f = await seed();
vi.stubEnv("PAPERCLIP_HOME", await fs.realpath(root));
@@ -113,3 +113,68 @@ export async function readWorkspaceGitSourceRoots(cwd: string, options: { ancest
const result = { root, gitDirectory, commonDirectory, objectDirectories, metadata };
return { ...result, fingerprint: sha256(JSON.stringify(result)) };
}
export async function readNestedWorkspaceGitSourceRoots(cwd: string): Promise<WorkspaceGitSourceRoots[]> {
const roots: WorkspaceGitSourceRoots[] = [];
let entries = 0;
const visit = async (directory: string, depth: number, inRepository: boolean) => {
if (depth > 64) throw invalid();
await directoryIdentity(directory);
const children = (await fs.readdir(directory, { withFileTypes: true })).sort((a, b) => a.name.localeCompare(b.name));
entries += children.length;
if (entries > 100_000) throw invalid();
if (children.some(child => child.name === ".git")) {
const source = await readWorkspaceGitSourceRoots(directory);
if (!source || roots.length >= 64) throw invalid();
roots.push(source); inRepository = true;
} else if (children.some(child => child.name === "objects") && children.some(child => child.name === "HEAD")) throw invalid();
for (const child of children) {
if (!child.isDirectory() || child.name === ".git" || depth === 0 && child.name === ".paperclip-runtime") continue;
const next = path.join(directory, child.name);
if (!inRepository) {
const marker = await entry(child.name === "node_modules" ? path.join(directory, "package.json") : path.join(next, "pyvenv.cfg"));
if (marker?.isFile() && !marker.isSymbolicLink()) continue;
}
await visit(next, depth + 1, inRepository);
}
};
await visit(path.resolve(cwd), 0, false);
return roots;
}
export async function readWorkspaceGitLinkedRoots(source: WorkspaceGitSourceRoots) {
const commonDirectories = new Map([[source.commonDirectory.path, source.commonDirectory]]);
for (const objects of source.objectDirectories) {
if (path.basename(objects.path) !== "objects" || commonDirectories.has(path.dirname(objects.path))) continue;
const common = await directoryIdentity(path.dirname(objects.path)); commonDirectories.set(common.path, common);
}
const results: Array<{ commonDirectory: Identity; directory: Identity | null;
links: Array<{ root: Identity; gitDirectory: Identity }>; metadata: Metadata[] }> = [];
for (const commonDirectory of [...commonDirectories.values()].sort((a, b) => a.path.localeCompare(b.path))) {
const target = path.join(commonDirectory.path, "worktrees"), present = await entry(target);
if (!present) { results.push({ commonDirectory, directory: null, links: [], metadata: [] }); continue; }
const directory = await directoryIdentity(target), names = (await fs.readdir(directory.path)).sort();
if (names.length > 256) throw invalid();
const links: Array<{ root: Identity; gitDirectory: Identity }> = [], metadata: Metadata[] = [];
for (const name of names) {
const gitDirectory = await directoryIdentity(path.join(directory.path, name));
const backlink = await readMetadata(path.join(gitDirectory.path, "gitdir"), 4096);
const common = await readMetadata(path.join(gitDirectory.path, "commondir"), 4096);
if (!backlink || !common || path.resolve(gitDirectory.path, metadataPath(common.text)) !== commonDirectory.path) throw invalid();
const marker = path.resolve(gitDirectory.path, metadataPath(backlink.text));
if (path.basename(marker) !== ".git") throw invalid();
const root = await directoryIdentity(path.dirname(marker)), pointer = await readMetadata(marker, 4096);
if (marker !== path.join(root.path, ".git") || !pointer?.text.startsWith("gitdir: ")
|| path.resolve(root.path, metadataPath(pointer.text.slice(8))) !== gitDirectory.path) throw invalid();
links.push({ root, gitDirectory }); metadata.push(backlink.metadata, common.metadata, pointer.metadata);
}
if (JSON.stringify(names) !== JSON.stringify((await fs.readdir(directory.path)).sort())
|| JSON.stringify(directory) !== JSON.stringify(await directoryIdentity(target))) throw invalid();
for (const receipt of metadata) {
const current = await readMetadata(receipt.path, 4096);
if (JSON.stringify(current?.metadata) !== JSON.stringify(receipt)) throw invalid();
}
results.push({ commonDirectory, directory, links, metadata: metadata.sort((a, b) => a.path.localeCompare(b.path)) });
}
return results;
}