From f5f5184f233292fe2d7f525591cb8d400aa0487b Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Wed, 4 Nov 2026 22:35:55 -0600 Subject: [PATCH] fix(workspaces): preserve retained branches and private commit sources (#19147) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Ephemeral web servers must preserve existing task files before volume removal. > - An old private checkout can be absent while its recorded branch still exists. > - A deleted branch can also retain its exact commit in the original private object source. > - The importer currently rejects these surviving sources. > - This pull request saves the verified committed tree and records its exact source proof. > - It keeps task state, branch ownership and recovery obligations unchanged. ## Linked Issues or Issue Description Refs #19138, #19144, #18826. **What happened?** Existing Git workspace records can have missing checkout folders and retained branch objects. The importer returns `local_source_unsafe_or_missing`. A successful deletion receipt can also have a valid private object source that the importer overlooks in favor of a missing or changed project cache. **Expected behavior** Save the surviving committed tree from the exact recorded branch or deletion receipt. Reject competing owners, live worktree registrations, changed branches and invalid source identities. Do not use the current default branch as a substitute. **Steps to reproduce** 1. Retain an isolated workspace branch in its managed project cache. 2. Remove its clean checkout but keep the branch and original workspace record. 3. Attempt workspace migration. 4. Alternatively, retain a deleted commit in its recorded private source while its shared source is unavailable. ## What Changed - Pin a retained branch in its verified company and project Git cache. - Require the recorded private path, no competing workspace owner and no surviving Git worktree registration or index authority. - For a user-owned branch, require its exact successful unforced removal receipt after archival. Preserve the branch ownership metadata. - Prefer a verified recorded private object source for successful deletion receipts when that source still exists. - Recheck branch and receipt proof before publication. Preserve compatibility with existing migration journals. - Allow missing active records only through the existing offline writer fence. Keep their status unchanged. - Add byte-restoration and rejected-authority tests. Document the source limits. - Revalidate and honor interrupted journals' original Git source path. Reject detached registrations and retained orphan index data. Reuse the cleanup producer command formatter for paths containing spaces. ## Verification - 172 focused migration, concurrency and isolated-source tests pass with real PostgreSQL and Git repositories. - The previous importer fails the retained-branch and offline-active restoration cases. The previous source selection also fails the private-cache and clean user-owned receipt cases. - Tests remove the old home and restore the pinned commit and task file bytes. New shared default-branch work is excluded. - Tests reject competing owners, registered worktrees, missing branch ownership, deleted branches, changed branch tips, forced removal and revoked receipts. - Read-only re-inventory of 79 actual failed source records makes 78 eligible with committed owner overlays. One older local test repository remains blocked. This is source-overlay qualification, not full released-image acceptance. - A separate bounded archive replay completed six imports before its 25-minute deadline. That timing failure remains open; it does not establish complete migration. - Local server typecheck has no changed-file diagnostics. Stale linked declarations prevent a clean local whole-server result. Full clean CI remains required. - Serving enrollment and empty-home recovery remain separate acceptance gates. - Latest rebased head 9f42901683 passes 172 focused migration/archive/concurrency tests. Four review regressions fail on the previous source; the revoked journal source already rejected correctly. The real cleanup fixture uses an unforced removal under a home path containing spaces. The orphan-index fixture is locked to keep Git maintenance from deleting its synthetic retained state before inspection. - The prerebase layered server typecheck passed. Current-head full CI remains required after integrating master. The local environment reset removed earlier temporary logs; the current red/green logs and source now live in persistent worktree folders. - Current-head full CI passes on run 41965901512 after one bounded failed-shard retry. The first run failed an unrelated status-card fake-clock deadline assertion by 150 ms; the same source passed the retry. Its cause remains unestablished, and the first failure log remains in Actions and persistent local evidence. No migration-related test failed. ## Risks - A retained branch saves committed data that still exists. It cannot certify recovery of files that were already absent. - User-owned branches remain user-owned. A clean removal receipt grants preservation authority, not deletion authority. - Any retained registration, conflicting record or changed proof keeps migration blocked. - This PR does not resolve the remaining source, recovery or migration timing gates. ## Model Used OpenAI GPT-6.1 Codex, exact model ID `gpt-6.1-sol`. Used for reasoning, code editing, shell tools and test execution. The runtime does not expose its context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- ...-existing-workspace-ephemeral-migration.md | 16 +++ .../remote-workspace-migration-git.ts | 116 ++++++++++++++++-- .../remote-workspace-migration-tree.ts | 2 +- .../remote-workspace-migration.test.ts | 95 +++++++++++++- .../services/remote-workspace-migration.ts | 5 +- .../src/services/workspace-command-display.ts | 5 + server/src/services/workspace-runtime.ts | 7 +- 7 files changed, 219 insertions(+), 27 deletions(-) create mode 100644 server/src/services/workspace-command-display.ts diff --git a/doc/plans/2026-11-03-existing-workspace-ephemeral-migration.md b/doc/plans/2026-11-03-existing-workspace-ephemeral-migration.md index 8ba9c30ed72..d48c93e2b55 100644 --- a/doc/plans/2026-11-03-existing-workspace-ephemeral-migration.md +++ b/doc/plans/2026-11-03-existing-workspace-ephemeral-migration.md @@ -165,6 +165,22 @@ creation time. A queued follow-up can predate the prior turn completion. A remot stop can also predate a later logical failure caused by recovery timeout. These proofs use the recorded execution start and never rewrite lifecycle timestamps. +A missing managed task checkout can still have its runtime-owned branch in the +verified company and project Git cache. Preserve that exact commit when the +recorded private path matches, no other workspace owns the branch, and Git has +no surviving worktree registration or index for it. Pin and recheck the commit +before publication. This saves the surviving committed tree; it does not +certify recovery of files that were already absent. An active record requires +the existing offline provider fence and keeps its original status. Invalid +restore recipes, live registrations and competing branch owners stay blocked. + +For a user-owned retained branch, require its exact successful unforced worktree +removal receipt after archival. Forced removal or a revoked receipt does not +grant this authority. Preserve the branch and its ownership metadata unchanged. +For a deleted private branch, prefer its verified recorded private object source +while it still exists. A missing project cache or a later project-source change +must not hide that original retained commit. + ### Existing typed checkpoint authority When a retained source already has a valid typed database checkpoint, derive its diff --git a/server/src/services/remote-workspace-migration-git.ts b/server/src/services/remote-workspace-migration-git.ts index 5f905fae544..ea66217b354 100644 --- a/server/src/services/remote-workspace-migration-git.ts +++ b/server/src/services/remote-workspace-migration-git.ts @@ -3,7 +3,7 @@ import fs from "node:fs/promises"; import os from "node:os"; import path from "node:path"; import { promisify } from "node:util"; -import { and, desc, eq, gte, or, sql } from "drizzle-orm"; +import { and, desc, eq, gte, isNotNull, ne, or, sql } from "drizzle-orm"; import { executionWorkspaces, projectWorkspaces, workspaceOperations, type Db } from "@paperclipai/db"; import { PROJECT_REPOSITORIES_DIR } from "@paperclipai/adapter-utils/git-workspace-sync"; import type { StorageService } from "../storage/types.js"; @@ -14,6 +14,10 @@ import { ensureSavedGitCommit, gitRestorePointExpired, managedSourcePaths, readG verifiedGitCacheSource, verifiedManagedGitObjectSource, type GitCacheRestoration } from "./git-workspace-restoration.js"; import { localRepositoryPath } from "./local-repository-path.js"; import { materializeManagedProjectWorkspace } from "./managed-repository-checkout.js"; +import { isRuntimeOwnedGitBranch } from "./execution-workspace-branch-ownership.js"; +import { physicalPath } from "./remote-workspace-migration-tree.js"; +import { formatCommandForDisplay } from "./workspace-command-display.js"; +import { recordedIsolatedRemoteSource } from "./isolated-remote-workspace-source.js"; type Workspace = typeof executionWorkspaces.$inferSelect; export type ArchivedGitMigrationSource = { @@ -21,6 +25,7 @@ export type ArchivedGitMigrationSource = { sourcePath: string; cleanupOperationId: string | null; recipe: GitCacheRestoration | null; + retainedBranch?: string; }; const execute = promisify(execFile); const unavailable = () => new Error("remote_workspace_migration_archived_git_source_unavailable"); @@ -36,17 +41,69 @@ async function git(cwd: string, args: string[]) { })).stdout.trim(); } +async function assertNoRetainedWorktree(sourcePath: string, expected: string, branch: string) { + const target = await physicalPath(expected); + const common = path.join(sourcePath, ".git"); + const admin = path.join(common, "worktrees"); + const entries = await fs.readdir(admin, { withFileTypes: true }).catch((error: NodeJS.ErrnoException) => { + if (error.code === "ENOENT") return []; + throw error; + }); + for (const entry of entries) { + if (entry.isSymbolicLink()) throw unavailable(); + if (!entry.isDirectory()) continue; + const root = path.join(admin, entry.name); + const pointer = await fs.readFile(path.join(root, "gitdir"), "utf8").catch((error: NodeJS.ErrnoException) => { + if (error.code === "ENOENT") return null; + throw error; + }); + if (pointer && await physicalPath(pointer.trim()) === path.join(target, ".git")) throw unavailable(); + // An index without its registration can still own staged work. It has no + // reliable checkout identity, so a branch-only import cannot discard it. + if (!pointer && await fs.lstat(path.join(root, "index")).catch((error: NodeJS.ErrnoException) => { + if (error.code === "ENOENT") return null; + throw error; + })) throw unavailable(); + } + const records = (await git(sourcePath, ["worktree", "list", "--porcelain", "-z"])).split("\0\0"); + for (const record of records) { + const fields = record.split("\0"); + const worktree = fields.find(field => field.startsWith("worktree "))?.slice(9); + if (fields.includes(`branch refs/heads/${branch}`) || worktree && await physicalPath(worktree) === target) throw unavailable(); + } +} + /** A removed branch must be reconstructed from its saved commit or its exact * successful deletion receipt. Today's default branch is never a substitute. */ -export async function inspectArchivedGitMigrationSource(db: Pick, workspace: Workspace): Promise { - if (workspace.status !== "archived" || !workspace.repoUrl || !workspace.branchName || !workspace.sourceIssueId) return null; +export async function inspectArchivedGitMigrationSource(db: Pick, workspace: Workspace, + options: { allowActiveRetainedBranch?: boolean; operationId?: string } = {}): Promise { + const archived = workspace.status === "archived"; + if ((!archived && !options.allowActiveRetainedBranch) || !workspace.repoUrl || !workspace.branchName || !workspace.sourceIssueId) return null; const recipe = readGitCacheRestoration(workspace); if (recipe) { + if (!archived) return null; if (gitRestorePointExpired(recipe)) throw new Error("remote_workspace_migration_restore_point_expired"); return { commit: recipe.commit, sourcePath: recipe.sourcePath, recipe, cleanupOperationId: null }; } - if (workspace.providerType !== "git_worktree" || !workspace.closedAt || workspace.metadata?.gitBranchOwnershipVersion !== 1) return null; - const [operation] = await db.select().from(workspaceOperations).where(and( + if (workspace.providerType !== "git_worktree" || workspace.metadata?.gitBranchOwnershipVersion !== 1) return null; + const [projectWorkspace] = workspace.projectWorkspaceId ? await db.select({ cwd: projectWorkspaces.cwd, repoUrl: projectWorkspaces.repoUrl }) + .from(projectWorkspaces).where(and(eq(projectWorkspaces.id, workspace.projectWorkspaceId), + eq(projectWorkspaces.companyId, workspace.companyId), workspace.projectId ? eq(projectWorkspaces.projectId, workspace.projectId) : undefined)) : []; + const privateSource = recordedIsolatedRemoteSource(workspace); + const privatePresent = privateSource ? await fs.lstat(privateSource).catch((error: NodeJS.ErrnoException) => { + if (error.code === "ENOENT") return null; + throw error; + }) : null; + const [journal] = options.operationId ? await db.select({ metadata: workspaceOperations.metadata }).from(workspaceOperations).where(and( + eq(workspaceOperations.companyId, workspace.companyId), eq(workspaceOperations.executionWorkspaceId, workspace.id), + eq(workspaceOperations.id, options.operationId), eq(workspaceOperations.phase, "remote_workspace_migration"), + )) : []; + const saved = journal?.metadata?.archivedGitSource; + const savedPath = saved && typeof saved === "object" && !Array.isArray(saved) + && typeof (saved as { sourcePath?: unknown }).sourcePath === "string" ? (saved as { sourcePath: string }).sourcePath : null; + const sourcePath = savedPath ?? (privatePresent && privateSource ? privateSource : projectWorkspace?.cwd ?? managedSourcePaths(workspace)[0]); + if (!sourcePath || !privatePresent && projectWorkspace?.repoUrl && projectWorkspace.repoUrl !== workspace.repoUrl) throw unavailable(); + const [operation] = archived && workspace.closedAt ? await db.select().from(workspaceOperations).where(and( eq(workspaceOperations.companyId, workspace.companyId), eq(workspaceOperations.executionWorkspaceId, workspace.id), eq(workspaceOperations.phase, "worktree_cleanup"), eq(workspaceOperations.status, "succeeded"), or(eq(workspaceOperations.command, `git branch -d ${workspace.branchName}`), and( @@ -55,16 +112,50 @@ export async function inspectArchivedGitMigrationSource(db: Pick, sql`${workspaceOperations.metadata}->>'branchName' = ${workspace.branchName}`, sql`${workspaceOperations.metadata}->>'workspacePath' = ${workspace.providerRef ?? workspace.cwd}`, )), gte(workspaceOperations.startedAt, workspace.closedAt), - )).orderBy(desc(workspaceOperations.startedAt), desc(workspaceOperations.id)).limit(1); + )).orderBy(desc(workspaceOperations.startedAt), desc(workspaceOperations.id)).limit(1) : []; + if (!operation) { + // A branch that was never deleted still owns its committed tree. It does + // not certify lost untracked files. Any surviving worktree registration + // can retain additional index/files authority and must stay blocked. + if (workspace.mode !== "isolated_workspace" || !workspace.projectId || !managedSourcePaths(workspace).includes(sourcePath)) return null; + await assertNoRetainedWorktree(sourcePath, path.join(sourcePath, ".paperclip", "worktrees", workspace.branchName), workspace.branchName); + await git(sourcePath, ["check-ref-format", `refs/heads/${workspace.branchName}`]); + const expected = path.join(sourcePath, ".paperclip", "worktrees", workspace.branchName); + if (workspace.cwd !== expected || workspace.providerRef && workspace.providerRef !== expected) return null; + let removalId: string | null = null; + if (!isRuntimeOwnedGitBranch(workspace.metadata)) { + // User-owned branches must not be deleted. A successful unforced removal + // proves their old checkout was clean; preserve the branch without + // converting that receipt into branch-deletion authority. + if (workspace.metadata?.createdByRuntime !== false || !archived || !workspace.closedAt) return null; + const [removal] = await db.select({ id: workspaceOperations.id }).from(workspaceOperations).where(and( + eq(workspaceOperations.companyId, workspace.companyId), eq(workspaceOperations.executionWorkspaceId, workspace.id), + eq(workspaceOperations.phase, "worktree_cleanup"), eq(workspaceOperations.status, "succeeded"), + eq(workspaceOperations.command, formatCommandForDisplay("git", ["worktree", "remove", expected])), isNotNull(workspaceOperations.finishedAt), + gte(workspaceOperations.startedAt, workspace.closedAt), + sql`${workspaceOperations.metadata}->>'cleanupAction' = 'worktree_remove'`, + sql`${workspaceOperations.metadata}->>'workspaceId' = ${workspace.id}`, + sql`${workspaceOperations.metadata}->>'workspacePath' = ${expected}`, + sql`${workspaceOperations.metadata}->>'branchName' = ${workspace.branchName}`, + )).orderBy(desc(workspaceOperations.startedAt), desc(workspaceOperations.id)).limit(1); + if (!removal) return null; + removalId = removal.id; + } + await verifiedManagedGitObjectSource(workspace, sourcePath); + const [other] = await db.select({ id: executionWorkspaces.id }).from(executionWorkspaces).where(and( + eq(executionWorkspaces.companyId, workspace.companyId), eq(executionWorkspaces.projectId, workspace.projectId), + eq(executionWorkspaces.repoUrl, workspace.repoUrl), eq(executionWorkspaces.branchName, workspace.branchName), + ne(executionWorkspaces.id, workspace.id), + )).limit(1); + if (other) throw unavailable(); + const commit = await git(sourcePath, ["rev-parse", "--verify", `refs/heads/${workspace.branchName}^{commit}`]); + if (!/^[a-f0-9]{40}$/.test(commit)) throw unavailable(); + return { commit, sourcePath, cleanupOperationId: removalId, recipe: null, retainedBranch: workspace.branchName }; + } const receipt = operation?.stdoutExcerpt?.trim(), prefix = `Deleted branch ${workspace.branchName} (was `; if (!operation || !receipt?.startsWith(prefix) || !receipt.endsWith(").")) return null; const abbreviation = receipt.slice(prefix.length, -2); if (!/^[a-f0-9]{7,40}$/.test(abbreviation)) throw unavailable(); - const [projectWorkspace] = workspace.projectWorkspaceId ? await db.select({ cwd: projectWorkspaces.cwd, repoUrl: projectWorkspaces.repoUrl }) - .from(projectWorkspaces).where(and(eq(projectWorkspaces.id, workspace.projectWorkspaceId), - eq(projectWorkspaces.companyId, workspace.companyId), workspace.projectId ? eq(projectWorkspaces.projectId, workspace.projectId) : undefined)) : []; - const sourcePath = projectWorkspace?.cwd ?? managedSourcePaths(workspace)[0]; - if (!sourcePath || projectWorkspace?.repoUrl && projectWorkspace.repoUrl !== workspace.repoUrl) throw unavailable(); await verifiedReceiptObjectSource(workspace, sourcePath); const commit = await git(sourcePath, ["rev-parse", "--verify", `${abbreviation}^{commit}`]); if (!/^[a-f0-9]{40}$/.test(commit)) throw unavailable(); @@ -87,9 +178,10 @@ export async function materializeArchivedGitMigrationSource(input: { // A deletion receipt owns a specific committed tree. Its objects can come // from the verified project cache after the private checkout was removed. // Saved private edits still require their original restore-point source. - if (!source.recipe && !source.cleanupOperationId) throw unavailable(); + if (!source.recipe && !source.cleanupOperationId && source.retainedBranch !== workspace.branchName) throw unavailable(); const verified = source.recipe ? await verifiedGitCacheSource(workspace, source.sourcePath) : await verifiedReceiptObjectSource(workspace, source.sourcePath); + if (source.retainedBranch && (await git(verified, ["rev-parse", "--verify", `refs/heads/${source.retainedBranch}^{commit}`])) !== source.commit) throw unavailable(); await fs.mkdir(cwd, { mode: 0o700 }); await git(cwd, ["init", "--quiet"]); // Fetch objects, never the current working tree of a shared project root. const presentCommit = await git(verified, ["cat-file", "-e", `${source.commit}^{commit}`]).then(() => true, () => false); diff --git a/server/src/services/remote-workspace-migration-tree.ts b/server/src/services/remote-workspace-migration-tree.ts index 4685a231f62..c634e41b9fb 100644 --- a/server/src/services/remote-workspace-migration-tree.ts +++ b/server/src/services/remote-workspace-migration-tree.ts @@ -18,7 +18,7 @@ const rootOf = (workspace: Pick) => workspace. // Missing historical paths still resolve through their nearest surviving // parent, so a symlinked ancestor cannot hide an alias from the writer fence. -async function physicalPath(value: string): Promise { +export async function physicalPath(value: string): Promise { let current = path.resolve(value); const missing: string[] = []; for (;;) { try { return path.join(await fs.realpath(current), ...missing.reverse()); } diff --git a/server/src/services/remote-workspace-migration.test.ts b/server/src/services/remote-workspace-migration.test.ts index 79f5eb7a865..9f61f38566d 100644 --- a/server/src/services/remote-workspace-migration.test.ts +++ b/server/src/services/remote-workspace-migration.test.ts @@ -452,9 +452,9 @@ describe.skipIf(!support.supported)("remote workspace migration", () => { await expect(migrateRemoteWorkspace({ db, storage, ...parent, apply: true, writersStopped: true })).rejects.toThrow("nested_workspace_source_changed"); expect(readRemoteWorkspaceAuthority((await workspace(parent.workspaceId)).metadata)).toBeNull(); }); - async function gitArchiveFixture(repoUrl = "https://example.invalid/repository.git") { + async function gitArchiveFixture(repoUrl = "https://example.invalid/repository.git", home = root) { const f = await seed(), projectId = randomUUID(), projectWorkspaceId = randomUUID(), branchName = "saved-task"; - vi.stubEnv("PAPERCLIP_HOME", root); vi.stubEnv("PAPERCLIP_INSTANCE_ID", "default"); + vi.stubEnv("PAPERCLIP_HOME", home); vi.stubEnv("PAPERCLIP_INSTANCE_ID", "default"); const base = resolveManagedProjectWorkspaceDir({ companyId: f.companyId, projectId, repoName: "repository" }); await fs.mkdir(base, { recursive: true }); await git(base, "init", "-q", "-b", "main"); await fs.writeFile(path.join(base, "base.txt"), "base\n"); await git(base, "add", "."); await git(base, "commit", "-qm", "base"); @@ -504,7 +504,7 @@ describe.skipIf(!support.supported)("remote workspace migration", () => { expect(await fs.readFile(path.join(targetDir, "base.txt"), "utf8")).toBe("base\n"); expect(await fs.readFile(path.join(targetDir, "saved.txt"), "utf8")).toBe("original saved work\n"); }); - it.each(["valid", "foreign-cache", "wrong-origin", "missing-receipt", "revoked-receipt"])("recovers deleted isolated history only from an authorized receipt and project object cache (%s)", async state => { + it.each(["valid", "foreign-cache", "wrong-origin", "missing-receipt", "revoked-receipt", "private-cache", "journaled-shared", "journaled-source-changed"])("recovers deleted isolated history only from an authorized receipt and project object cache (%s)", async state => { const f = await gitArchiveFixture(), privateBase = isolatedRemoteSourcePath(f.companyId, f.issueId); const privateCwd = path.join(privateBase, ".paperclip", "worktrees", f.branchName); await fs.mkdir(path.dirname(privateBase), { recursive: true }); @@ -515,7 +515,8 @@ describe.skipIf(!support.supported)("remote workspace migration", () => { await git(privateBase, "worktree", "remove", privateCwd); const receipt = await git(privateBase, "branch", "-d", f.branchName); await git(f.base, "merge", "--ff-only", f.branchName); await git(f.base, "worktree", "remove", f.cwd); - await git(f.base, "branch", "-d", f.branchName); await fs.rm(privateBase, { recursive: true }); + await git(f.base, "branch", "-d", f.branchName); + if (!["private-cache", "journaled-shared", "journaled-source-changed"].includes(state)) await fs.rm(privateBase, { recursive: true }); const closedAt = new Date(); await db.update(executionWorkspaces).set({ cwd: privateCwd, providerRef: privateCwd, status: "archived", closedAt, metadata: { createdByRuntime: true, gitBranchOwnershipVersion: 1, isolatedRemoteSource: { @@ -532,7 +533,7 @@ describe.skipIf(!support.supported)("remote workspace migration", () => { await fs.writeFile(path.join(f.base, "base.txt"), "new shared checkout\n"); await fs.writeFile(path.join(f.base, "unrelated.txt"), "not task history\n"); await git(f.base, "add", "."); await git(f.base, "commit", "-qm", "new shared default"); - if (state === "wrong-origin") await git(f.base, "remote", "set-url", "origin", "https://example.invalid/foreign.git"); + if (state === "wrong-origin" || state === "private-cache") await git(f.base, "remote", "set-url", "origin", "https://example.invalid/foreign.git"); if (state === "foreign-cache") { const foreign = resolveManagedProjectWorkspaceDir({ companyId: randomUUID(), projectId: f.projectId, repoName: "repository" }); await fs.mkdir(path.dirname(foreign), { recursive: true }); await fs.rename(f.base, foreign); @@ -547,7 +548,16 @@ describe.skipIf(!support.supported)("remote workspace migration", () => { return result; }; } - if (state !== "valid") { + if (state.startsWith("journaled-")) { + const put = storage.putFile.bind(storage); let fail = true; + storage.putFile = async input => { const saved = await put(input); if (fail) { fail = false; throw new Error("upload reply lost"); } return saved; }; + await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("upload reply lost"); + const [operation] = await db.select().from(workspaceOperations).where(and(eq(workspaceOperations.executionWorkspaceId, f.workspaceId), eq(workspaceOperations.phase, "remote_workspace_migration"))); + await db.update(workspaceOperations).set({ metadata: { ...operation!.metadata, + archivedGitSource: { ...(operation!.metadata!.archivedGitSource as object), sourcePath: f.base } } }).where(eq(workspaceOperations.id, operation!.id)); + if (state === "journaled-source-changed") await git(f.base, "remote", "set-url", "origin", "https://example.invalid/foreign.git"); + } + if (!["valid", "private-cache", "journaled-shared"].includes(state)) { await expect(migrateRemoteWorkspace({ db, storage, companyId: f.companyId, workspaceId: f.workspaceId, apply: true })).rejects.toThrow(); expect(readRemoteWorkspaceAuthority((await workspace(f.workspaceId)).metadata)).toBeNull(); return; } @@ -555,6 +565,7 @@ describe.skipIf(!support.supported)("remote workspace migration", () => { const row = await workspace(f.workspaceId), authority = readRemoteWorkspaceAuthority(row.metadata)!; expect(row.status).toBe("archived"); expect(authority.source).toMatchObject({ kind: "git", ref: f.commit }); await fs.rm(f.base, { recursive: true }); + await fs.rm(privateBase, { recursive: true, force: true }); const ref = await getWorkspaceSnapshot(db, { ...f, snapshotId: authority.snapshotId! }); const archivePath = path.join(root, "isolated-history.pcws"), targetDir = path.join(root, "restored-isolated-history"); await loadWorkspaceSnapshotArchive({ storage, reference: ref!, targetPath: archivePath }); @@ -564,6 +575,78 @@ describe.skipIf(!support.supported)("remote workspace migration", () => { expect(await fs.readFile(path.join(targetDir, "base.txt"), "utf8")).toBe("base\n"); expect(await fs.stat(path.join(targetDir, "unrelated.txt")).catch(() => null)).toBeNull(); }); + it.each(["valid", "changed-branch", "competing-owner", "registered-branch", "unowned", "missing-branch", "active-online", "active-offline", "user-owned-receipt", "user-owned-force-receipt", "user-owned-revoked-receipt", "user-owned-quoted-receipt", "detached-registration", "orphan-index"])("preserves a missing checkout from its retained owned branch (%s)", async mode => { + const f = await gitArchiveFixture(undefined, mode === "user-owned-quoted-receipt" ? path.join(root, "home with spaces") : root), closedAt = new Date(); + const active = mode.startsWith("active-"); + await db.update(executionWorkspaces).set({ status: active ? "active" : "archived", closedAt: active ? null : closedAt }) + .where(eq(executionWorkspaces.id, f.workspaceId)); + if (mode === "user-owned-quoted-receipt") { + await db.update(executionWorkspaces).set({ metadata: { createdByRuntime: false, gitBranchOwnershipVersion: 1 } }).where(eq(executionWorkspaces.id, f.workspaceId)); + const cleaned = await cleanupExecutionWorkspaceArtifacts({ workspace: await workspace(f.workspaceId), + projectWorkspace: { cwd: f.base, cleanupCommand: null }, runCleanupCommands: false, forceWorktreeRemoval: false, + recorder: workspaceOperationService(db).createRecorder({ companyId: f.companyId, executionWorkspaceId: f.workspaceId, issueId: f.issueId }) }); + expect(cleaned.warnings).toEqual([]); + } else if (mode === "detached-registration") { + await git(f.cwd, "checkout", "--detach"); + await fs.writeFile(path.join(f.cwd, "newer.txt"), "recoverable detached work"); + await git(f.cwd, "add", "."); await git(f.cwd, "commit", "-qm", "newer detached work"); + await fs.rm(f.cwd, { recursive: true }); + } else await git(f.base, "worktree", "remove", f.cwd); + await fs.writeFile(path.join(f.base, "base.txt"), "new shared default branch\n"); + await git(f.base, "add", "."); await git(f.base, "commit", "-qm", "unrelated shared work"); + if (mode === "orphan-index") { + const admin = path.join(f.base, ".git", "worktrees", "orphan"); + await fs.mkdir(admin, { recursive: true }); await fs.copyFile(path.join(f.base, ".git", "index"), path.join(admin, "index")); + await fs.writeFile(path.join(admin, "locked"), "retained staged recovery work\n"); + } + if (mode === "changed-branch") { + const put = storage.putFile.bind(storage); + storage.putFile = async input => { const saved = await put(input); await git(f.base, "update-ref", `refs/heads/${f.branchName}`, await git(f.base, "rev-parse", "HEAD")); return saved; }; + } + if (mode === "competing-owner") { + const [issue] = await db.insert(issues).values({ companyId: f.companyId, title: "Different branch owner" }).returning(); + await db.insert(executionWorkspaces).values({ companyId: f.companyId, sourceIssueId: issue!.id, projectId: f.projectId, + name: "Other owner", mode: "isolated_workspace", strategyType: "git_worktree", providerType: "git_worktree", + repoUrl: f.repoUrl, branchName: f.branchName, cwd: path.join(root, "other-owner") }); + } + if (mode === "registered-branch") await git(f.base, "worktree", "add", path.join(root, "registered"), f.branchName); + if (mode.startsWith("user-owned-") && mode !== "user-owned-quoted-receipt") { + await db.update(executionWorkspaces).set({ metadata: { createdByRuntime: false, gitBranchOwnershipVersion: 1 } }).where(eq(executionWorkspaces.id, f.workspaceId)); + const [receipt] = await db.insert(workspaceOperations).values({ companyId: f.companyId, executionWorkspaceId: f.workspaceId, + phase: "worktree_cleanup", status: "succeeded", startedAt: new Date(), finishedAt: new Date(), + command: `git worktree remove ${mode === "user-owned-force-receipt" ? "--force " : ""}${f.cwd}`, + metadata: { cleanupAction: "worktree_remove", workspaceId: f.workspaceId, workspacePath: f.cwd, branchName: f.branchName } }).returning(); + if (mode === "user-owned-revoked-receipt") { + const put = storage.putFile.bind(storage); + storage.putFile = async input => { const saved = await put(input); await db.update(workspaceOperations).set({ status: "failed" }) + .where(eq(workspaceOperations.id, receipt!.id)); return saved; }; + } + } + if (mode === "unowned") await db.update(executionWorkspaces).set({ metadata: { gitBranchOwnershipVersion: 1 } }).where(eq(executionWorkspaces.id, f.workspaceId)); + if (mode === "missing-branch") await git(f.base, "branch", "-D", f.branchName); + const offline = mode === "active-offline"; + if (offline) vi.stubEnv("PAPERCLIP_EPHEMERAL_MIGRATION", "true"); + const migrate = () => migrateRemoteWorkspace({ db, storage, ...f, apply: true, writersStopped: offline }); + if (!["valid", "active-offline", "user-owned-receipt", "user-owned-quoted-receipt"].includes(mode)) { + await expect(migrate()).rejects.toThrow(); + expect(readRemoteWorkspaceAuthority((await workspace(f.workspaceId)).metadata)).toBeNull(); return; + } + await migrate(); + const row = await workspace(f.workspaceId), authority = readRemoteWorkspaceAuthority(row.metadata)!; + expect(row.status).toBe(active ? "active" : "archived"); + expect(authority.source).toMatchObject({ kind: "git", ref: f.commit }); + const [operation] = await db.select().from(workspaceOperations).where(and( + eq(workspaceOperations.executionWorkspaceId, f.workspaceId), eq(workspaceOperations.phase, "remote_workspace_migration"))); + expect(operation!.metadata?.archivedGitSource).toMatchObject({ commit: f.commit, retainedBranch: f.branchName }); + await fs.rm(f.base, { recursive: true }); + const ref = await getWorkspaceSnapshot(db, { ...f, snapshotId: authority.snapshotId! }); + const archivePath = path.join(root, "retained-branch.pcws"), targetDir = path.join(root, "restored-retained-branch"); + await loadWorkspaceSnapshotArchive({ storage, reference: ref!, targetPath: archivePath }); + await restoreRemoteWorkspaceArchive({ archivePath, targetDir, expectedManifestDigest: ref!.manifestDigest }); + expect(await git(targetDir, "rev-parse", "HEAD")).toBe(f.commit); + expect(await fs.readFile(path.join(targetDir, "base.txt"), "utf8")).toBe("base\n"); + expect(await fs.readFile(path.join(targetDir, "saved.txt"), "utf8")).toBe("original saved work\n"); + }); async function serveRepository() { const directory = path.join(root, "http"), bare = path.join(directory, "repository.git"); await fs.mkdir(bare, { recursive: true }); await git(bare, "init", "--bare", "--initial-branch=main"); diff --git a/server/src/services/remote-workspace-migration.ts b/server/src/services/remote-workspace-migration.ts index a2f579466c8..afa8455e195 100644 --- a/server/src/services/remote-workspace-migration.ts +++ b/server/src/services/remote-workspace-migration.ts @@ -49,6 +49,7 @@ function fingerprint(workspace: Workspace) { } const gitSourceProof = (source: ArchivedGitMigrationSource | null) => source ? { commit: source.commit, sourcePath: source.sourcePath, cleanupOperationId: source.cleanupOperationId, + ...(source.retainedBranch ? { retainedBranch: source.retainedBranch } : {}), } : null; async function inspect(db: ReadDb, input: { companyId: string; workspaceId: string; operationId?: string; writersStopped?: boolean }) { @@ -85,8 +86,8 @@ async function inspect(db: ReadDb, input: { companyId: string; workspaceId: stri } } } - if (!entry && workspace.status === "archived" && !retainedDirectory) { - try { archivedGitSource = await inspectArchivedGitMigrationSource(db, workspace); } + if (!entry && !retainedDirectory && (workspace.status === "archived" || input.writersStopped)) { + try { archivedGitSource = await inspectArchivedGitMigrationSource(db, workspace, { allowActiveRetainedBranch: input.writersStopped, operationId: allowedOperation }); } catch (error) { reasons.push(error instanceof Error && error.message === "remote_workspace_migration_restore_point_expired" ? "restore_point_expired" : "archived_git_source_unavailable"); } } diff --git a/server/src/services/workspace-command-display.ts b/server/src/services/workspace-command-display.ts new file mode 100644 index 00000000000..a609774744b --- /dev/null +++ b/server/src/services/workspace-command-display.ts @@ -0,0 +1,5 @@ +export function formatCommandForDisplay(command: string, args: string[]) { + return [command, ...args] + .map((part) => (/^[A-Za-z0-9_./:-]+$/.test(part) ? part : JSON.stringify(part))) + .join(" "); +} diff --git a/server/src/services/workspace-runtime.ts b/server/src/services/workspace-runtime.ts index 8a6211968a6..b9bdfcfdda3 100644 --- a/server/src/services/workspace-runtime.ts +++ b/server/src/services/workspace-runtime.ts @@ -1,3 +1,4 @@ +import { formatCommandForDisplay } from "./workspace-command-display.js"; import { notifyStatusCardIssueChange } from "./status-card-notifications.js"; import { insertIssueComment } from "./issue-comment-writes.js"; import { restoreGitWorkspaceContents } from "./git-workspace-contents.js"; @@ -834,12 +835,6 @@ function resolveConfiguredPath(value: string, baseDir: string): string { return path.resolve(baseDir, value); } -function formatCommandForDisplay(command: string, args: string[]) { - return [command, ...args] - .map((part) => (/^[A-Za-z0-9_./:-]+$/.test(part) ? part : JSON.stringify(part))) - .join(" "); -} - function trimToLastBytes(value: string, limit: number) { const byteLength = Buffer.byteLength(value, "utf8"); if (byteLength <= limit) return value;