diff --git a/doc/DEVELOPING.md b/doc/DEVELOPING.md index 182afe1d850..78621549793 100644 --- a/doc/DEVELOPING.md +++ b/doc/DEVELOPING.md @@ -1878,6 +1878,22 @@ or recovery, unsafe paths, and explicitly concurrent shared writers block import. An interrupted apply retains its journal and admission hold; retry the same command to reconcile it before starting another writer. +An import preserves the workspace's lifecycle state and the task's current +workspace binding. It can import an archived or superseded workspace without +reopening the task or selecting that older workspace for future turns. A changed +task binding during the import blocks publication. + +If an archived folder has already been removed, the importer can reconstruct it +in private scratch space from its encrypted task-directory restore point or +validated Git restoration recipe. For older Git workspaces without a recipe, +it requires a successful branch-deletion receipt and the exact saved commit in +the verified source repository. It never substitutes today's default branch. +Git restore points retain unpushed history and staged and unstaged edits, even +when the source repository has pruned the saved branch. Missing, expired, +corrupt, or unbound sources block the import. Publication still requires an +independent download and restore of the new encrypted snapshot. The old folder +remains absent, and the importer does not change the original restore point. + Supported sources include empty task folders, repository URLs, and managed mutable repositories. Authorized mentioned projects attach all their configured Git repositories as writable copies under `.paperclip-repositories/` in the task diff --git a/doc/plans/2026-11-03-existing-workspace-ephemeral-migration.md b/doc/plans/2026-11-03-existing-workspace-ephemeral-migration.md new file mode 100644 index 00000000000..65c7dd3d351 --- /dev/null +++ b/doc/plans/2026-11-03-existing-workspace-ephemeral-migration.md @@ -0,0 +1,45 @@ +# Preserve existing workspaces during ephemeral enrollment + +The deployment endpoint remains a volume-free Cloud tenant, followed by a real +rolling deployment while an agent works and new work is admitted. Enrollment must +preserve existing task files and historical recovery sources. It must not reset +task workspaces, manufacture successful copyback, or weaken migration guards. + +## Evidence and remaining work + +| Required outcome | Current evidence | Remaining gap | Owner / next action | +| --- | --- | --- | --- | +| Existing active task retains files | The current import encrypts, uploads, independently restores, compares the source again and publishes under the lifecycle lock. A live completed task imported successfully with its original directory retained. | The importer only accepts an active workspace with the current task binding. | Workspace migration owner: support historical records without changing the task binding or lifecycle state. | +| Archived task retains its saved files | Retention records include encrypted task-directory restore points, Git restore points and verified Git reconstruction recipes. | The importer rejects missing local folders even when an existing restore point can reconstruct them. | Restore-point owners: reconstruct into migration scratch space, reject expired/missing/corrupt sources, and feed the verified result into the normal import publication. | +| Shared project roots preserve all task files | Legacy roots contain nested task worktrees and multiple records can reference one physical folder. | A naive root capture recursively imports those other workspaces; the current per-record writer check does not fence aliases. | Workspace migration owner: inventory physical overlaps, fence all writers, and preserve nested workspace coverage before excluding any directory. | +| Failed runs retain unsaved files | Provider inspection distinguishes surviving archived sandboxes from missing allocations. Existing recovery validation accepts some sources and rejects others for missing provenance or ambiguous allocation ownership. | Recovery must settle or durably preserve each exact source. A successful old preservation action does not necessarily update the legacy sync descriptor. | Workspace recovery owner: verify existing preservation receipts, recover eligible sources, and retain explicit unresolved obligations for unavailable/ambiguous sources. Do not replay agent work. | +| Tenant is eligible without its volume | External DB and S3 are configured; the original volume remains attached. | Workspaces, local run logs and remaining owner-maintained data must pass the packaged maintenance receipt. | Cloud enrollment owner, after workspace migration is complete. | +| Ordinary upgrades remain available | The no-drain Cloud deployment path is merged and qualified on another enrolled tenant. | Exact final app image qualification and a live deployment on this tenant. | Cloud deployment owner: qualify, enroll, then prove reads, submissions and active-run adoption through an actual replacement. | + +## Acceptance boundaries + +The first implementation slice supports archived and historical records, +encrypted task-directory restoration, Git restoration recipes, and legacy +branch-deletion receipts. PostgreSQL and real-file tests prove cold restoration +of saved bytes, index state, and unpushed commits, including after pruning the +old Git object store. Invalid or unavailable retained data still blocks import. +Shared-root migration and recovery-obligation reconciliation remain separate +gaps; this slice does not make a whole tenant eligible for enrollment. + +- Import existing files or a verified retained source. A missing folder alone is + not permission to publish an empty workspace or choose a newer Git branch. +- Preserve archived status, task bindings, task history, unpushed Git history, + staged and unstaged edits, and saved recovery copies. +- Preserve original files and the detached original volume until verified + migration receipts and the retention interval permit removal. +- Parent roots may omit a nested workspace only with durable, inspectable + coverage of that exact nested directory. Arbitrary exclusion globs are not an + acceptable data migration. +- Resume interrupted imports with their original identity; cancellation may + release an empty reservation only after the producer is stopped and no upload + was journaled. Preserve an audit receipt. +- Verify restored file bytes independently, including after making the old + source unavailable in an isolated test. Exercise current and historical + bindings, archived sources, overlap/writer races, and unavailable sources. +- Unrelated tenants, production services, and agent task execution remain outside + the migration's mutation scope. diff --git a/server/src/services/isolated-task-directory-retention.ts b/server/src/services/isolated-task-directory-retention.ts index cb5a8dc8a37..b4c83c70e66 100644 --- a/server/src/services/isolated-task-directory-retention.ts +++ b/server/src/services/isolated-task-directory-retention.ts @@ -124,7 +124,7 @@ function restorePointObjectKey(companyId: string, workspaceId: string, generatio return `${companyId}/${NAMESPACE}/${workspaceId}/${generation}/task-directory/${id}.pcrp`; } -function validateRestorePointRef(ref: TaskDirectoryRestorePointRef | null | undefined, workspace: { companyId: string; id: string }) { +export function validateTaskDirectoryRestorePointRef(ref: TaskDirectoryRestorePointRef | null | undefined, workspace: { companyId: string; id: string }) { if (!ref || ref.schema !== TASK_DIRECTORY_RESTORE_POINT_SCHEMA || typeof ref.id !== "string" || !UUID.test(ref.id) || !Number.isSafeInteger(ref.generation) || ref.generation < 1 || ref.objectKey !== restorePointObjectKey(workspace.companyId, workspace.id, ref.generation, ref.id) @@ -136,6 +136,25 @@ function validateRestorePointRef(ref: TaskDirectoryRestorePointRef | null | unde return ref; } +/** Read a retained directory without reopening its workspace or replacing its + * original path. Administrative migration uses the same identity and byte checks + * as ordinary restore, and never treats an expired object as an empty folder. */ +export async function loadTaskDirectoryRestorePoint(input: { + storage: StorageService; + workspace: { companyId: string; id: string }; + ref: TaskDirectoryRestorePointRef; + signal?: AbortSignal; +}) { + const ref = validateTaskDirectoryRestorePointRef(input.ref, input.workspace); + if (ref.expiredAt) throw new TaskDirectoryRestoreFailure("restore_point_expired"); + const ciphertext = await readRestorePointObject(input.storage, input.workspace.companyId, ref.objectKey, input.signal).catch(() => { + throw new TaskDirectoryRestoreFailure("restore_point_unavailable"); + }); + if (ciphertext.length !== ref.byteSize) throw new TaskDirectoryRestoreFailure("ciphertext_size_mismatch"); + return openRestorePoint({ ciphertext, sha256: ref.sha256, key: ref.key, + identity: { companyId: input.workspace.companyId, workspaceId: input.workspace.id, restorePointId: ref.id, generation: ref.generation } }); +} + function skipReason(error: unknown) { if (error instanceof TaskDirectorySnapshotSkip) return error.reason; if (error instanceof IsolatedRepositorySourceBusy) return "source_busy"; @@ -341,7 +360,7 @@ export async function restoreIsolatedTaskDirectory(input: { const parent = path.dirname(cwd); await fs.mkdir(parent, { recursive: true, mode: 0o700 }); if (await fs.realpath(parent) !== parent) throw new TaskDirectoryRestoreFailure("task_directory_identity_changed"); - const ref = validateRestorePointRef(state.restorePoint, input.workspace); + const ref = validateTaskDirectoryRestorePointRef(state.restorePoint, input.workspace); if (await directoryState(cwd) === "present") { if (!state.removed) return { restored: false, expired: false }; if ((await fs.readdir(cwd)).length === 0) await fs.rmdir(cwd); @@ -355,12 +374,7 @@ export async function restoreIsolatedTaskDirectory(input: { await fs.mkdir(cwd, { mode: 0o700 }); return { restored: false, expired: true }; } - const ciphertext = await readRestorePointObject(input.storage, input.workspace.companyId, ref.objectKey, input.signal).catch(() => { - throw new TaskDirectoryRestoreFailure("restore_point_unavailable"); - }); - if (ciphertext.length !== ref.byteSize) throw new TaskDirectoryRestoreFailure("ciphertext_size_mismatch"); - const payload = await openRestorePoint({ ciphertext, sha256: ref.sha256, key: ref.key, - identity: { companyId: input.workspace.companyId, workspaceId: input.workspace.id, restorePointId: ref.id, generation: ref.generation } }); + const payload = await loadTaskDirectoryRestorePoint({ ...input, ref }); const staging = await materializeTaskDirectorySnapshot({ payload, target: cwd, expectedDigest: ref.digest, signal: input.signal }); try { await fs.rename(staging, cwd); diff --git a/server/src/services/remote-workspace-migration-git.ts b/server/src/services/remote-workspace-migration-git.ts new file mode 100644 index 00000000000..66d82639c38 --- /dev/null +++ b/server/src/services/remote-workspace-migration-git.ts @@ -0,0 +1,142 @@ +import { execFile } from "node:child_process"; +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { promisify } from "node:util"; +import { and, desc, eq, gte, or, sql } from "drizzle-orm"; +import { executionWorkspaces, projectWorkspaces, workspaceOperations, type Db } from "@paperclipai/db"; +import { PROJECT_REPOSITORIES_DIR } from "@paperclipai/adapter-utils/git-workspace-sync"; +import type { StorageService } from "../storage/types.js"; +import { createIssueGitRemoteAuthProvider } from "./git-credentials.js"; +import { applyGitRestorePointToWorktree, ensureGitRestorePointCommit, loadGitRestorePoint } from "./git-restore-point-storage.js"; +import { restoreGitWorkspaceContents } from "./git-workspace-contents.js"; +import { ensureSavedGitCommit, gitRestorePointExpired, managedSourcePaths, readGitCacheRestoration, + verifiedGitCacheSource, type GitCacheRestoration } from "./git-workspace-restoration.js"; +import { localRepositoryPath } from "./local-repository-path.js"; +import { materializeManagedProjectWorkspace } from "./managed-repository-checkout.js"; + +type Workspace = typeof executionWorkspaces.$inferSelect; +export type ArchivedGitMigrationSource = { + commit: string; + sourcePath: string; + cleanupOperationId: string | null; + recipe: GitCacheRestoration | null; +}; +const execute = promisify(execFile); +const unavailable = () => new Error("remote_workspace_migration_archived_git_source_unavailable"); +async function git(cwd: string, args: string[]) { + return (await execute("git", ["-c", "core.fsmonitor=false", "-c", `core.hooksPath=${os.devNull}`, "-C", cwd, ...args], { + timeout: 120_000, maxBuffer: 1024 * 1024, + env: { ...process.env, GIT_CONFIG_NOSYSTEM: "1", GIT_CONFIG_GLOBAL: os.devNull, + GIT_TERMINAL_PROMPT: "0", GIT_OPTIONAL_LOCKS: "0" }, + })).stdout.trim(); +} + +/** A removed branch must be reconstructed from its saved commit or its exact + * successful deletion receipt. Today's default branch is never a substitute. */ +export async function inspectArchivedGitMigrationSource(db: Pick, workspace: Workspace): Promise { + if (workspace.status !== "archived" || !workspace.repoUrl || !workspace.branchName || !workspace.sourceIssueId) return null; + const recipe = readGitCacheRestoration(workspace); + if (recipe) { + if (gitRestorePointExpired(recipe)) throw new Error("remote_workspace_migration_restore_point_expired"); + return { commit: recipe.commit, sourcePath: recipe.sourcePath, recipe, cleanupOperationId: null }; + } + if (workspace.providerType !== "git_worktree" || !workspace.closedAt || workspace.metadata?.gitBranchOwnershipVersion !== 1) return null; + const [operation] = await db.select().from(workspaceOperations).where(and( + eq(workspaceOperations.companyId, workspace.companyId), eq(workspaceOperations.executionWorkspaceId, workspace.id), + eq(workspaceOperations.phase, "worktree_cleanup"), eq(workspaceOperations.status, "succeeded"), + or(eq(workspaceOperations.command, `git branch -d ${workspace.branchName}`), and( + sql`${workspaceOperations.metadata}->>'cleanupAction' = 'branch_delete'`, + sql`${workspaceOperations.metadata}->>'workspaceId' = ${workspace.id}`, + sql`${workspaceOperations.metadata}->>'branchName' = ${workspace.branchName}`, + sql`${workspaceOperations.metadata}->>'workspacePath' = ${workspace.providerRef ?? workspace.cwd}`, + )), gte(workspaceOperations.startedAt, workspace.closedAt), + )).orderBy(desc(workspaceOperations.startedAt), desc(workspaceOperations.id)).limit(1); + const receipt = operation?.stdoutExcerpt?.trim(), prefix = `Deleted branch ${workspace.branchName} (was `; + if (!operation || !receipt?.startsWith(prefix) || !receipt.endsWith(").")) return null; + const abbreviation = receipt.slice(prefix.length, -2); + if (!/^[a-f0-9]{7,40}$/.test(abbreviation)) throw unavailable(); + const [projectWorkspace] = workspace.projectWorkspaceId ? await db.select({ cwd: projectWorkspaces.cwd, repoUrl: projectWorkspaces.repoUrl }) + .from(projectWorkspaces).where(and(eq(projectWorkspaces.id, workspace.projectWorkspaceId), + eq(projectWorkspaces.companyId, workspace.companyId), workspace.projectId ? eq(projectWorkspaces.projectId, workspace.projectId) : undefined)) : []; + const sourcePath = projectWorkspace?.cwd ?? managedSourcePaths(workspace)[0]; + if (!sourcePath || projectWorkspace?.repoUrl && projectWorkspace.repoUrl !== workspace.repoUrl) throw unavailable(); + await verifiedGitCacheSource(workspace, sourcePath); + const commit = await git(sourcePath, ["rev-parse", "--verify", `${abbreviation}^{commit}`]); + if (!/^[a-f0-9]{40}$/.test(commit)) throw unavailable(); + return { commit, sourcePath, cleanupOperationId: operation.id, recipe: null }; +} + +/** Restore only into private migration scratch. Original repositories, task + * folders, lifecycle state and restore-point metadata remain unchanged. */ +export async function materializeArchivedGitMigrationSource(input: { + db: Db; storage: StorageService; workspace: Workspace; source: ArchivedGitMigrationSource; scratch: string; +}) { + const { workspace, source } = input, cwd = path.join(input.scratch, "retained-git"); + if (!workspace.sourceIssueId || !workspace.repoUrl || !workspace.branchName) throw unavailable(); + const resolveGitAuth = createIssueGitRemoteAuthProvider(input.db, workspace.companyId, workspace.sourceIssueId); + const present = await fs.lstat(source.sourcePath).catch((error: NodeJS.ErrnoException) => { + if (error.code === "ENOENT") return null; + throw error; + }); + if (present) { + const verified = await verifiedGitCacheSource(workspace, source.sourcePath); + await fs.mkdir(cwd, { mode: 0o700 }); await git(cwd, ["init", "--quiet"]); + // Fetch objects, never the current working tree of a shared project root. + const presentCommit = await git(verified, ["cat-file", "-e", `${source.commit}^{commit}`]).then(() => true, () => false); + if (presentCommit) { + await git(cwd, ["fetch", "--no-tags", "--no-recurse-submodules", "--", verified, source.commit]); + } else if (source.recipe?.restorePoint && !source.recipe.restorePoint.headOnRemote) { + // The archived branch can have been pruned from the base while its + // encrypted bundle still owns the unpushed commits. Import only that + // bundle's explicit prerequisites, then let the restore-point owner + // verify and import the saved history below. + const { manifest } = await loadGitRestorePoint({ storage: input.storage, workspace, ref: source.recipe.restorePoint }); + for (const commit of manifest.prerequisites) { + const local = await git(verified, ["cat-file", "-e", `${commit}^{commit}`]).then(() => true, () => false); + if (local) await git(cwd, ["fetch", "--no-tags", "--no-recurse-submodules", "--", verified, commit]); + else await ensureSavedGitCommit(cwd, workspace.repoUrl, commit, resolveGitAuth); + } + } + } else { + if (!source.recipe) throw unavailable(); + await materializeManagedProjectWorkspace(cwd, { repoUrl: workspace.repoUrl, resolveGitAuth }); + } + if (source.recipe?.restorePoint) await ensureGitRestorePointCommit({ storage: input.storage, workspace, + ref: source.recipe.restorePoint, repositoryCwd: cwd }); + await ensureSavedGitCommit(cwd, workspace.repoUrl, source.commit, resolveGitAuth); + await git(cwd, ["check-ref-format", `refs/heads/${workspace.branchName}`]); + await git(cwd, ["checkout", "--detach", source.commit]); + await git(cwd, ["update-ref", `refs/heads/${workspace.branchName}`, source.commit]); + await git(cwd, ["symbolic-ref", "HEAD", `refs/heads/${workspace.branchName}`]); + await git(cwd, ["config", "remote.origin.url", workspace.repoUrl]); + if (source.recipe?.restorePoint) await applyGitRestorePointToWorktree({ storage: input.storage, workspace, + ref: source.recipe.restorePoint, repositoryCwd: cwd, worktreePath: cwd }); + const approvedLocalSources = new Set(); + await restoreGitWorkspaceContents({ cwd, companyId: workspace.companyId, workspaceId: workspace.id, + projectId: workspace.projectId, contents: source.recipe?.contents, storage: input.storage, resolveGitAuth, + authorizeSource: async saved => { + if (!workspace.projectId) throw unavailable(); + const [row] = await input.db.select().from(projectWorkspaces).where(and(eq(projectWorkspaces.id, saved.id), + eq(projectWorkspaces.companyId, workspace.companyId), eq(projectWorkspaces.projectId, workspace.projectId))); + if (!row || row.repoUrl !== saved.repoUrl) throw unavailable(); + const local = localRepositoryPath(saved.repoUrl); if (local) approvedLocalSources.add(local); + }, + prepareLocalSource: async local => { + if (!approvedLocalSources.has(local)) throw unavailable(); + const identity = await fs.realpath(local); + return async () => { if (await fs.realpath(local) !== identity) throw unavailable(); }; + }, + }); + // A checkout records scratch-file inode and time fields in its index. Those + // fields must not change the saved archive identity after an upload reply is + // lost. Rebuild only these reconstructed indexes from their staged trees; + // the original repository and unstaged files remain untouched. + for (const repository of [cwd, ...(source.recipe?.contents?.checkouts ?? []) + .map(checkout => path.join(cwd, PROJECT_REPOSITORIES_DIR, checkout.name))]) { + const tree = await git(repository, ["write-tree"]); + await git(repository, ["read-tree", "--empty"]); + await git(repository, ["read-tree", tree]); + } + return cwd; +} diff --git a/server/src/services/remote-workspace-migration.test.ts b/server/src/services/remote-workspace-migration.test.ts index 0e55b224eab..e2345e3cb88 100644 --- a/server/src/services/remote-workspace-migration.test.ts +++ b/server/src/services/remote-workspace-migration.test.ts @@ -4,6 +4,8 @@ import fs from "node:fs/promises"; import os from "node:os"; import path from "node:path"; import { promisify } from "node:util"; +import { createServer, type Server } from "node:http"; +import { pathToFileURL } from "node:url"; import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest"; import { eq } from "drizzle-orm"; import { activityLog, applyPendingMigrations, closeRegisteredClients, companies, createDb, executionWorkspaceSnapshots, @@ -16,7 +18,14 @@ import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } fro import { inventoryRemoteWorkspaceMigration, migrateRemoteWorkspace } from "./remote-workspace-migration.js"; import { getWorkspaceSnapshot, loadWorkspaceSnapshotArchive } from "./remote-workspace-snapshot-storage.js"; import { restoreRemoteWorkspaceArchive } from "./remote-workspace-snapshot-archive.js"; -import { removeTaskDirectoryTree } from "./isolated-task-directory-snapshot.js"; +import { captureTaskDirectorySnapshot, removeTaskDirectoryTree } from "./isolated-task-directory-snapshot.js"; +import { captureRestorePoint, sealRestorePoint } from "./git-restore-point.js"; +import type { TaskDirectoryRestorePointRef } from "./isolated-task-directory-retention.js"; +import { resolveManagedProjectWorkspaceDir } from "../home-paths.js"; +import { cleanupExecutionWorkspaceArtifacts } from "./workspace-runtime.js"; +import { workspaceOperationService } from "./workspace-operations.js"; +import { managedProjectCheckoutName } from "./managed-project-checkout-name.js"; +import { captureGitWorkspaceContents, readManagedCheckoutContents } from "./git-workspace-contents.js"; const execute = promisify(execFile); const git = async (cwd: string, ...args: string[]) => (await execute("git", ["-C", cwd, ...args], { env: { ...process.env, @@ -28,6 +37,7 @@ if (url && !/^remote_workspace_[a-z_]*test$/.test(new URL(url).pathname.slice(1) const support = url ? { supported: true } : await getEmbeddedPostgresTestSupport(); describe.skipIf(!support.supported)("remote workspace migration", () => { let db: ReturnType, root: string, storage: StorageService; + const servers: Server[] = []; let temporary: Awaited> | undefined; beforeAll(async () => { if (url) { await applyPendingMigrations(url); db = createDb(url); } @@ -37,6 +47,7 @@ describe.skipIf(!support.supported)("remote workspace migration", () => { beforeEach(async () => { vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY", "61".repeat(32)); root = await fs.mkdtemp(path.join(os.tmpdir(), "pc-migration-test-")); storage = createStorageService(createLocalDiskStorageProvider(path.join(root, "objects"))); }); afterEach(async () => { + await Promise.all(servers.splice(0).map(server => new Promise((resolve, reject) => server.close(error => error ? reject(error) : resolve())))); vi.unstubAllEnvs(); await db.delete(activityLog); await db.delete(workspaceOperations); await db.delete(executionWorkspaceSnapshots); await db.delete(workspaceRuntimeServices); await db.delete(executionWorkspaceRuntimeLeases); await db.delete(issues); await db.delete(executionWorkspaces); await db.delete(projectWorkspaces); await db.delete(projects); @@ -53,6 +64,256 @@ describe.skipIf(!support.supported)("remote workspace migration", () => { return { companyId, workspaceId, issueId: issue!.id, cwd }; } async function workspace(id: string) { return (await db.select().from(executionWorkspaces).where(eq(executionWorkspaces.id, id)))[0]!; } + async function gitArchiveFixture(repoUrl = "https://example.invalid/repository.git") { + const f = await seed(), projectId = randomUUID(), projectWorkspaceId = randomUUID(), branchName = "saved-task"; + vi.stubEnv("PAPERCLIP_HOME", root); vi.stubEnv("PAPERCLIP_INSTANCE_ID", "default"); + const base = resolveManagedProjectWorkspaceDir({ companyId: f.companyId, projectId, repoName: "repository" }); + await fs.mkdir(base, { recursive: true }); await git(base, "init", "-q", "-b", "main"); + await fs.writeFile(path.join(base, "base.txt"), "base\n"); await git(base, "add", "."); await git(base, "commit", "-qm", "base"); + await git(base, "remote", "add", "origin", repoUrl); const upstream = await git(base, "rev-parse", "HEAD"); + const cwd = path.join(base, ".paperclip", "worktrees", branchName); + await git(base, "worktree", "add", "-b", branchName, cwd); + await fs.copyFile(path.join(f.cwd, "saved.txt"), path.join(cwd, "saved.txt")); await fs.rm(f.cwd, { recursive: true }); + await git(cwd, "add", "."); await git(cwd, "commit", "-qm", "saved task commit"); const commit = await git(cwd, "rev-parse", "HEAD"); + await db.insert(projects).values({ id: projectId, companyId: f.companyId, name: "Archived project" }); + await db.insert(projectWorkspaces).values({ id: projectWorkspaceId, companyId: f.companyId, projectId, name: "Source", cwd: base, repoUrl }); + await db.update(executionWorkspaces).set({ cwd, providerRef: cwd, projectId, projectWorkspaceId, repoUrl, baseRef: "main", + branchName, strategyType: "git_worktree", providerType: "git_worktree", metadata: { createdByRuntime: true, gitBranchOwnershipVersion: 1 } }).where(eq(executionWorkspaces.id, f.workspaceId)); + return { ...f, cwd, base, upstream, commit, projectId, projectWorkspaceId, repoUrl, branchName }; + } + it.each(["legacy", "verified", "retry"])("recovers an archived Git workspace from its exact branch-deletion receipt (%s)", async receiptKind => { + const f = await gitArchiveFixture(), closedAt = new Date(); + await db.update(executionWorkspaces).set({ status: "archived", closedAt }).where(eq(executionWorkspaces.id, f.workspaceId)); + if (receiptKind !== "verified") { + await git(f.base, "merge", "--ff-only", f.branchName); + await git(f.base, "worktree", "remove", f.cwd); + const receipt = await git(f.base, "branch", "-d", f.branchName); + await db.insert(workspaceOperations).values({ companyId: f.companyId, executionWorkspaceId: f.workspaceId, + phase: "worktree_cleanup", status: "succeeded", command: `git branch -d ${f.branchName}`, + stdoutExcerpt: receipt, startedAt: new Date(), finishedAt: new Date() }); + } else { + const cleaned = await cleanupExecutionWorkspaceArtifacts({ workspace: await workspace(f.workspaceId), + projectWorkspace: { cwd: f.base, cleanupCommand: null }, expectedBranchHeadSha: f.commit, runCleanupCommands: false, + recorder: workspaceOperationService(db).createRecorder({ companyId: f.companyId, executionWorkspaceId: f.workspaceId, issueId: f.issueId }) }); + expect(cleaned.warnings).toEqual([]); + } + await fs.writeFile(path.join(f.base, "base.txt"), "newer default branch\n"); + await git(f.base, "add", "base.txt"); await git(f.base, "commit", "-qm", "new default branch work"); + if (receiptKind === "retry") { + const put = storage.putFile.bind(storage); let fail = true; + storage.putFile = async input => { const result = await put(input); if (fail) { fail = false; throw new Error("upload reply lost"); } return result; }; + await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("upload reply lost"); + } + await migrateRemoteWorkspace({ db, storage, ...f, apply: true }); + const row = await workspace(f.workspaceId), authority = readRemoteWorkspaceAuthority(row.metadata)!; + expect(row.status).toBe("archived"); expect(authority.source).toMatchObject({ kind: "git", ref: f.commit }); + await fs.rm(f.base, { recursive: true }); + const ref = await getWorkspaceSnapshot(db, { ...f, snapshotId: authority.snapshotId! }); + const archivePath = path.join(root, "legacy-git.pcws"), targetDir = path.join(root, "restored-legacy-git"); + await loadWorkspaceSnapshotArchive({ storage, reference: ref!, targetPath: archivePath }); + await restoreRemoteWorkspaceArchive({ archivePath, targetDir, expectedManifestDigest: ref!.manifestDigest }); + expect(await git(targetDir, "rev-parse", "HEAD")).toBe(f.commit); + expect(await fs.readFile(path.join(targetDir, "base.txt"), "utf8")).toBe("base\n"); + expect(await fs.readFile(path.join(targetDir, "saved.txt"), "utf8")).toBe("original saved work\n"); + }); + async function serveRepository() { + const directory = path.join(root, "http"), bare = path.join(directory, "repository.git"); + await fs.mkdir(bare, { recursive: true }); await git(bare, "init", "--bare", "--initial-branch=main"); + const server = createServer(async (request, response) => { + const relative = new URL(request.url!, "http://localhost").pathname.slice(1); + const target = path.resolve(directory, relative); + if (!target.startsWith(`${directory}/`)) { response.writeHead(403).end(); return; } + try { response.end(await fs.readFile(target)); } catch { response.writeHead(404).end(); } + }); + await new Promise(resolve => server.listen(0, "127.0.0.1", resolve)); servers.push(server); + const address = server.address(); if (!address || typeof address === "string") throw new Error("test server not bound"); + return { bare, repoUrl: `http://127.0.0.1:${address.port}/repository.git` }; + } + it.each(["present", "pruned", "retry", "missing-base", "nested", "unapproved-nested", "missing", "expired", "corrupt"])("preserves archived Git edits or refuses an unavailable restore point (%s)", async state => { + const remote = state === "missing-base" ? await serveRepository() : null; + const f = await gitArchiveFixture(remote?.repoUrl); + if (remote) { await git(remote.bare, "fetch", f.base, "main:refs/heads/main"); await git(remote.bare, "update-server-info"); } + await fs.writeFile(path.join(f.cwd, "saved.txt"), "staged edit\n"); await git(f.cwd, "add", "saved.txt"); + await fs.writeFile(path.join(f.cwd, "saved.txt"), "unstaged edit\n"); + const captured = (await captureRestorePoint({ cwd: f.cwd, upstreamCommits: [f.upstream] }))!; + const id = randomUUID(), generation = 1, sealed = await sealRestorePoint(captured.payload, + { companyId: f.companyId, workspaceId: f.workspaceId, restorePointId: id, generation }); + const objectKey = `${f.companyId}/git-restore-points/${f.workspaceId}/${generation}/${id}.pcrp`; + await storage.putFile({ companyId: f.companyId, namespace: "git-restore-points", objectKey, + originalFilename: "git.pcrp", contentType: "application/octet-stream", body: sealed.ciphertext }); + const restorePoint = { schema: "paperclip.git-restore-point.v1", id, generation, objectKey, byteSize: sealed.ciphertext.length, + sha256: sealed.sha256, key: sealed.key, head: captured.manifest.head, index: captured.manifest.index, + worktree: captured.manifest.worktree, ignored: captured.manifest.ignored, modes: captured.manifest.modes, headOnRemote: false }; + let contents; + if (state === "nested" || state === "unapproved-nested") { + const sourceCwd = path.join(root, "nested-source"); + await fs.mkdir(sourceCwd); await git(sourceCwd, "init", "-q", "-b", "main"); + await fs.writeFile(path.join(sourceCwd, "nested.txt"), "nested source work\n"); + await git(sourceCwd, "add", "."); await git(sourceCwd, "commit", "-qm", "nested source"); + const source = { id: randomUUID(), repoUrl: pathToFileURL(sourceCwd).href, repoRef: "main", cwd: sourceCwd }; + const name = managedProjectCheckoutName(source), nestedCwd = path.join(f.cwd, ".paperclip-repositories", name); + await git(f.cwd, "clone", source.repoUrl, nestedCwd); await git(nestedCwd, "checkout", "--detach"); + await fs.mkdir(path.join(nestedCwd, ".claude")); + await fs.writeFile(path.join(nestedCwd, ".claude", "settings.local.json"), '{"permissions":{"allow":["Read"]}}'); + contents = await captureGitWorkspaceContents(f.cwd, { companyId: f.companyId, workspaceId: f.workspaceId }, + [await readManagedCheckoutContents(nestedCwd, name, source)]); + if (state === "nested") await db.insert(projectWorkspaces).values({ ...source, companyId: f.companyId, projectId: f.projectId, name: "Nested source" }); + } + await db.update(executionWorkspaces).set({ status: "archived", closedAt: new Date(), metadata: { + createdByRuntime: true, gitBranchOwnershipVersion: 1, gitCacheRestoration: { schema: "paperclip.git-cache-restoration.v2", companyId: f.companyId, + workspaceId: f.workspaceId, sourceIssueId: f.issueId, repoUrl: f.repoUrl, commit: f.commit, branchName: f.branchName, + sourceKind: "managed_project", projectId: f.projectId, sourcePath: f.base, worktreePath: f.cwd, + gitBranchOwnershipVersion: 1, verifiedAt: new Date().toISOString(), restorePoint, contents }, + } }).where(eq(executionWorkspaces.id, f.workspaceId)); + await git(f.base, "worktree", "remove", "--force", f.cwd); await git(f.base, "branch", "-D", f.branchName); + if (state === "missing-base") await fs.rm(f.base, { recursive: true }); + if (state === "unapproved-nested") { + await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow(); + expect(readRemoteWorkspaceAuthority((await workspace(f.workspaceId)).metadata)).toBeNull(); + expect((await workspace(f.workspaceId)).status).toBe("archived"); + return; + } + if (state === "pruned") { + await git(f.base, "reflog", "expire", "--expire=now", "--all"); + await git(f.base, "gc", "--prune=now"); + await expect(git(f.base, "cat-file", "-e", `${f.commit}^{commit}`)).rejects.toThrow(); + } + if (["missing", "expired", "corrupt"].includes(state)) { + if (state === "missing") await storage.deleteObject(f.companyId, objectKey); + else { + const row = await workspace(f.workspaceId); + const recipe = row.metadata!.gitCacheRestoration as Record; + await db.update(executionWorkspaces).set({ metadata: { ...row.metadata, gitCacheRestoration: { ...recipe, + restorePoint: { ...restorePoint, ...(state === "expired" ? { expiredAt: new Date().toISOString() } : { sha256: "00".repeat(32) }) }, + } } }).where(eq(executionWorkspaces.id, f.workspaceId)); + } + await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow(); + const row = await workspace(f.workspaceId); + expect(row).toMatchObject({ status: "archived", providerType: "git_worktree", cwd: f.cwd }); + expect(readRemoteWorkspaceAuthority(row.metadata)).toBeNull(); + expect(await fs.stat(f.cwd).catch(() => null)).toBeNull(); + return; + } + if (state === "retry") { + const put = storage.putFile.bind(storage); let fail = true; + storage.putFile = async input => { const result = await put(input); if (fail) { fail = false; throw new Error("upload reply lost"); } return result; }; + await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("upload reply lost"); + } + await migrateRemoteWorkspace({ db, storage, ...f, apply: true }); + const authority = readRemoteWorkspaceAuthority((await workspace(f.workspaceId)).metadata)!; + await fs.rm(f.base, { recursive: true, force: true }); await fs.rm(path.join(root, "nested-source"), { recursive: true, force: true }); await storage.deleteObject(f.companyId, objectKey); + const ref = await getWorkspaceSnapshot(db, { ...f, snapshotId: authority.snapshotId! }); + const archivePath = path.join(root, "retained-git.pcws"), targetDir = path.join(root, "restored-retained-git"); + await loadWorkspaceSnapshotArchive({ storage, reference: ref!, targetPath: archivePath }); + await restoreRemoteWorkspaceArchive({ archivePath, targetDir, expectedManifestDigest: ref!.manifestDigest }); + expect(await git(targetDir, "rev-parse", "HEAD")).toBe(f.commit); + if (state === "nested") { + expect(await fs.readFile(path.join(targetDir, ".paperclip-repositories", contents!.checkouts[0]!.name, "nested.txt"), "utf8")).toBe("nested source work\n"); + expect(JSON.parse(await fs.readFile(path.join(targetDir, ".paperclip-repositories", contents!.checkouts[0]!.name, ".claude/settings.local.json"), "utf8"))) + .toEqual({ permissions: { allow: ["Read"] } }); + } + expect(await git(targetDir, "show", ":saved.txt")).toBe("staged edit"); + expect(await fs.readFile(path.join(targetDir, "saved.txt"), "utf8")).toBe("unstaged edit\n"); + }); + async function retainDirectory(f: Awaited>) { + const captured = await captureTaskDirectorySnapshot(f.cwd), id = randomUUID(), generation = 1; + const sealed = await sealRestorePoint(captured.payload, { companyId: f.companyId, workspaceId: f.workspaceId, restorePointId: id, generation }); + const objectKey = `${f.companyId}/git-restore-points/${f.workspaceId}/${generation}/task-directory/${id}.pcrp`; + await storage.putFile({ companyId: f.companyId, namespace: "git-restore-points", objectKey, + originalFilename: "retained.pcrp", contentType: "application/octet-stream", body: sealed.ciphertext }); + const ref: TaskDirectoryRestorePointRef = { schema: "paperclip.task-directory-restore-point.v1", id, generation, + objectKey, sha256: sealed.sha256, byteSize: sealed.ciphertext.length, key: sealed.key, + digest: captured.digest, files: captured.files, excluded: captured.excluded }; + await db.update(executionWorkspaces).set({ status: "archived", closedAt: new Date(), metadata: { + isolatedTaskDirectoryRetention: { schema: "paperclip.task-directory-retention.v1", generation, removed: true, + trash: path.join(root, "removed-trash"), restorePoint: ref }, + } }).where(eq(executionWorkspaces.id, f.workspaceId)); + await fs.rm(f.cwd, { recursive: true }); + return ref; + } + it("imports an archived directory from its encrypted restore point after the old folder is gone", async () => { + const f = await seed(), retained = await retainDirectory(f); + expect(await inventoryRemoteWorkspaceMigration(db, f)).toMatchObject({ eligible: true }); + await migrateRemoteWorkspace({ db, storage, ...f, apply: true }); + const saved = await workspace(f.workspaceId), authority = readRemoteWorkspaceAuthority(saved.metadata)!; + expect(saved.status).toBe("archived"); + expect(await fs.stat(f.cwd).catch(() => null)).toBeNull(); + await storage.deleteObject(f.companyId, retained.objectKey); + const ref = await getWorkspaceSnapshot(db, { ...f, snapshotId: authority.snapshotId! }); + const archivePath = path.join(root, "retained.pcws"), targetDir = path.join(root, "restored-retained"); + await loadWorkspaceSnapshotArchive({ storage, reference: ref!, targetPath: archivePath }); + await restoreRemoteWorkspaceArchive({ archivePath, targetDir, expectedManifestDigest: ref!.manifestDigest }); + expect(await fs.readFile(path.join(targetDir, "saved.txt"), "utf8")).toBe("original saved work\n"); + }); + it.each(["missing", "expired", "corrupt"])("does not replace an unavailable retained directory with an empty workspace (%s)", async failure => { + const f = await seed(), retained = await retainDirectory(f); + if (failure === "missing") await storage.deleteObject(f.companyId, retained.objectKey); + else { + const row = await workspace(f.workspaceId); + const retention = row.metadata!.isolatedTaskDirectoryRetention as Record; + await db.update(executionWorkspaces).set({ metadata: { ...row.metadata, isolatedTaskDirectoryRetention: { + ...retention, restorePoint: { ...retained, ...(failure === "expired" ? { expiredAt: new Date().toISOString() } : { sha256: "00".repeat(32) }) }, + } } }).where(eq(executionWorkspaces.id, f.workspaceId)); + } + await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow(); + const row = await workspace(f.workspaceId); + expect(row).toMatchObject({ status: "archived", providerType: "local_fs", cwd: f.cwd }); + expect(readRemoteWorkspaceAuthority(row.metadata)).toBeNull(); + expect(await fs.stat(f.cwd).catch(() => null)).toBeNull(); + }); + it("keeps newly restored local files when they appear during an archived import", async () => { + const f = await seed(); await retainDirectory(f); + const put = storage.putFile.bind(storage); + storage.putFile = async input => { + const result = await put(input); + await fs.mkdir(f.cwd); await fs.writeFile(path.join(f.cwd, "saved.txt"), "new local edits\n"); + return result; + }; + await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("remote_workspace_migration_source_changed"); + expect(readRemoteWorkspaceAuthority((await workspace(f.workspaceId)).metadata)).toBeNull(); + expect(await fs.readFile(path.join(f.cwd, "saved.txt"), "utf8")).toBe("new local edits\n"); + }); + it("preserves an archived workspace without reopening its task or losing its files", async () => { + const f = await seed(), closedAt = new Date("2026-10-20T12:00:00Z"); + await db.update(executionWorkspaces).set({ status: "archived", closedAt }).where(eq(executionWorkspaces.id, f.workspaceId)); + await db.update(issues).set({ status: "done" }).where(eq(issues.id, f.issueId)); + await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).resolves.toMatchObject({ applied: true }); + const saved = await workspace(f.workspaceId), authority = readRemoteWorkspaceAuthority(saved.metadata)!; + expect(saved.status).toBe("archived"); expect(saved.closedAt).toEqual(closedAt); + const [issue] = await db.select().from(issues).where(eq(issues.id, f.issueId)); + expect(issue).toMatchObject({ status: "done", executionWorkspaceId: f.workspaceId }); + const ref = await getWorkspaceSnapshot(db, { ...f, snapshotId: authority.snapshotId! }); + await fs.rm(f.cwd, { recursive: true }); + const archivePath = path.join(root, "archived.pcws"), targetDir = path.join(root, "restored-archived"); + await loadWorkspaceSnapshotArchive({ storage, reference: ref!, targetPath: archivePath }); + await restoreRemoteWorkspaceArchive({ archivePath, targetDir, expectedManifestDigest: ref!.manifestDigest }); + expect(await fs.readFile(path.join(targetDir, "saved.txt"), "utf8")).toBe("original saved work\n"); + }); + it("imports a historical workspace without replacing the task's current workspace", async () => { + const f = await seed(), currentId = randomUUID(), currentCwd = path.join(root, "current"); + await fs.mkdir(currentCwd); await fs.writeFile(path.join(currentCwd, "saved.txt"), "newer current work\n"); + await db.insert(executionWorkspaces).values({ id: currentId, companyId: f.companyId, sourceIssueId: f.issueId, + mode: "isolated_workspace", strategyType: "project_primary", providerType: "local_fs", name: "Current workspace", cwd: currentCwd }); + await db.update(issues).set({ executionWorkspaceId: currentId }).where(eq(issues.id, f.issueId)); + await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).resolves.toMatchObject({ applied: true }); + const [issue] = await db.select().from(issues).where(eq(issues.id, f.issueId)); + expect(issue?.executionWorkspaceId).toBe(currentId); + expect(await fs.readFile(path.join(currentCwd, "saved.txt"), "utf8")).toBe("newer current work\n"); + expect(readRemoteWorkspaceAuthority((await workspace(currentId)).metadata)).toBeNull(); + expect(readRemoteWorkspaceAuthority((await workspace(f.workspaceId)).metadata)?.snapshotId).toBeTruthy(); + }); + it("refuses publication when the task changes its workspace during import", async () => { + const f = await seed(), put = storage.putFile.bind(storage); + storage.putFile = async input => { + const result = await put(input); + await db.update(issues).set({ executionWorkspaceId: null }).where(eq(issues.id, f.issueId)); + return result; + }; + await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("remote_workspace_migration_source_changed"); + expect(readRemoteWorkspaceAuthority((await workspace(f.workspaceId)).metadata)).toBeNull(); + expect(await fs.readFile(path.join(f.cwd, "saved.txt"), "utf8")).toBe("original saved work\n"); + await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("remote_workspace_migration_source_changed"); + }); it("defaults to dry run and preserves both database authority and local files", async () => { const f = await seed(); expect(await migrateRemoteWorkspace({ db, storage, ...f })).toMatchObject({ eligible: true, applied: false, alreadyMigrated: false }); expect((await workspace(f.workspaceId)).providerType).toBe("local_fs"); diff --git a/server/src/services/remote-workspace-migration.ts b/server/src/services/remote-workspace-migration.ts index b26dcfcf1b7..47dd4c7435f 100644 --- a/server/src/services/remote-workspace-migration.ts +++ b/server/src/services/remote-workspace-migration.ts @@ -14,7 +14,9 @@ import { withExecutionWorkspaceLifecycleLock } from "./execution-workspace-lifec import { captureRemoteWorkspaceDirectory, restoreRemoteWorkspaceArchive } from "./remote-workspace-snapshot-archive.js"; import { beginWorkspaceSnapshot, loadWorkspaceSnapshotArchive, uploadWorkspaceSnapshot } from "./remote-workspace-snapshot-storage.js"; import { notifyRemoteWorkspaceSnapshots } from "./remote-workspace-snapshot-retention.js"; -import { removeTaskDirectoryTree } from "./isolated-task-directory-snapshot.js"; +import { materializeTaskDirectorySnapshot, removeTaskDirectoryTree } from "./isolated-task-directory-snapshot.js"; +import { loadTaskDirectoryRestorePoint, mayBeIsolatedTaskDirectoryWorkspace, readTaskDirectoryRetention, validateTaskDirectoryRestorePointRef } from "./isolated-task-directory-retention.js"; +import { inspectArchivedGitMigrationSource, materializeArchivedGitMigrationSource, type ArchivedGitMigrationSource } from "./remote-workspace-migration-git.js"; const SCHEMA = "paperclip.remote-workspace-migration.v1"; type Workspace = typeof executionWorkspaces.$inferSelect; @@ -32,6 +34,9 @@ function fingerprint(workspace: Workspace) { providerType: workspace.providerType, cwd: workspace.cwd, providerRef: workspace.providerRef, repoUrl: workspace.repoUrl, baseRef: workspace.baseRef, branchName: workspace.branchName, status: workspace.status, closedAt: workspace.closedAt?.toISOString(), metadata })).digest("hex"); } +const gitSourceProof = (source: ArchivedGitMigrationSource | null) => source ? { + commit: source.commit, sourcePath: source.sourcePath, cleanupOperationId: source.cleanupOperationId, +} : null; async function inspect(db: ReadDb, input: { companyId: string; workspaceId: string; operationId?: string }) { const [workspace] = await db.select().from(executionWorkspaces).where(and(eq(executionWorkspaces.companyId, input.companyId), eq(executionWorkspaces.id, input.workspaceId))); @@ -40,21 +45,43 @@ async function inspect(db: ReadDb, input: { companyId: string; workspaceId: stri const marker = object(workspace.metadata?.remoteWorkspaceMigration); const allowedOperation = input.operationId ?? (marker.schema === SCHEMA && typeof marker.operationId === "string" ? marker.operationId : undefined); const reasons: string[] = []; - if (authority) return { workspace, reasons, authority, marker, source: authority.source }; + if (authority) return { workspace, reasons, authority, marker, source: authority.source, issueBinding: null, retainedDirectory: null, archivedGitSource: null }; if (!["local_fs", "git_worktree"].includes(workspace.providerType)) reasons.push("workspace_provider_unsupported"); - if (!["active", "idle", "in_review"].includes(workspace.status)) reasons.push("workspace_not_active"); + if (!["active", "idle", "in_review", "archived"].includes(workspace.status)) reasons.push("workspace_not_active"); if (!workspace.sourceIssueId) reasons.push("task_binding_required"); const nestedMigration = workspace.metadata?.nestedCloneMigrationPending; if (nestedMigration != null && (typeof nestedMigration !== "object" || Array.isArray(nestedMigration) || Object.keys(nestedMigration).length > 0)) reasons.push("nested_clone_migration_unresolved"); const cwd = workspace.cwd ?? workspace.providerRef; + let retainedDirectory = null as ReturnType; + let archivedGitSource: ArchivedGitMigrationSource | null = null; if (!cwd || !path.isAbsolute(cwd)) reasons.push("local_source_missing"); else { - const entry = await fs.lstat(cwd).catch(() => null); - if (!entry?.isDirectory() || entry.isSymbolicLink()) reasons.push("local_source_unsafe_or_missing"); + const entry = await fs.lstat(cwd).catch((error: NodeJS.ErrnoException) => { + if (error.code === "ENOENT") return null; + throw error; + }); + if (!entry && workspace.status === "archived" && mayBeIsolatedTaskDirectoryWorkspace(workspace)) { + const retained = readTaskDirectoryRetention(workspace); + if (retained?.removed === true && retained.restorePoint && retained.generation === retained.restorePoint.generation) { + if (retained.restorePoint.expiredAt) reasons.push("restore_point_expired"); + else { + try { validateTaskDirectoryRestorePointRef(retained.restorePoint, workspace); retainedDirectory = retained; } + catch { reasons.push("restore_point_invalid"); } + } + } + } + if (!entry && workspace.status === "archived" && !retainedDirectory) { + try { archivedGitSource = await inspectArchivedGitMigrationSource(db, workspace); } + catch (error) { reasons.push(error instanceof Error && error.message === "remote_workspace_migration_restore_point_expired" + ? "restore_point_expired" : "archived_git_source_unavailable"); } + } + if (!retainedDirectory && !archivedGitSource && (!entry?.isDirectory() || entry.isSymbolicLink())) reasons.push("local_source_unsafe_or_missing"); } const [issue] = workspace.sourceIssueId ? await db.select().from(issues).where(and(eq(issues.companyId, workspace.companyId), eq(issues.id, workspace.sourceIssueId))) : []; - if (workspace.sourceIssueId && (!issue || issue.executionWorkspaceId !== workspace.id)) reasons.push("task_binding_changed"); + // Historical records still own their saved files after the task selects a + // different workspace. Importing them must not reopen or rebind that task. + if (workspace.sourceIssueId && !issue) reasons.push("task_binding_changed"); const [project] = workspace.projectId ? await db.select({ policy: projects.executionWorkspacePolicy }).from(projects) .where(and(eq(projects.companyId, workspace.companyId), eq(projects.id, workspace.projectId))) : []; const issuePolicy = object(issue?.executionWorkspaceSettings), projectPolicy = object(project?.policy); @@ -109,10 +136,10 @@ async function inspect(db: ReadDb, input: { companyId: string; workspaceId: stri eq(executionWorkspaceRepositories.executionWorkspaceId, workspace.id))); if (repositories.some(repository => repository.state !== "ready")) reasons.push("repository_preparation_unresolved"); const source: RemoteWorkspaceSource = workspace.repoUrl ? { kind: "git", repoUrl: workspace.repoUrl, - ref: workspace.baseRef, branchName: workspace.branchName } : { kind: "empty" }; + ref: archivedGitSource?.commit ?? workspace.baseRef, branchName: workspace.branchName } : { kind: "empty" }; if (repositories.length) source.repositories = repositories.map(repository => ({ id: repository.id, relativePath: repository.relativePath, repoUrl: repository.repoUrl, ref: repository.requestedRef, ...(repository.pinnedCommit ? { pinnedCommit: repository.pinnedCommit } : {}) })); - return { workspace, reasons: [...new Set(reasons)], authority, marker, source }; + return { workspace, reasons: [...new Set(reasons)], authority, marker, source, issueBinding: issue?.executionWorkspaceId ?? null, retainedDirectory, archivedGitSource }; } export async function inventoryRemoteWorkspaceMigration(db: Db, input: { companyId: string; workspaceId: string }) { @@ -134,7 +161,7 @@ export async function migrateRemoteWorkspace(input: { if (!workspace) throw new Error("remote_workspace_migration_workspace_missing"); if (workspace.sourceIssueId) await tx.select({ id: issues.id }).from(issues).where(and(eq(issues.companyId, workspace.companyId), eq(issues.id, workspace.sourceIssueId))).for("update"); const current = await inspect(tx as unknown as Db, input); - if (current.authority) return { workspace, authority: current.authority, operation: null, source: current.source, sourceFingerprint: "" }; + if (current.authority) return { workspace, authority: current.authority, operation: null, source: current.source, sourceFingerprint: "", issueBinding: current.issueBinding, retainedDirectory: null, archivedGitSource: null }; if (current.reasons.length) throw new Error(`remote_workspace_migration_blocked:${current.reasons.join(",")}`); const source = remoteWorkspaceSourceSchema.parse(input.source ?? current.source); const sourceFingerprint = fingerprint(workspace); @@ -144,16 +171,26 @@ export async function migrateRemoteWorkspace(input: { const [operation] = await tx.select().from(workspaceOperations).where(and(eq(workspaceOperations.companyId, workspace.companyId), eq(workspaceOperations.executionWorkspaceId, workspace.id), eq(workspaceOperations.id, marker.operationId), eq(workspaceOperations.phase, "remote_workspace_migration"))); if (!operation) throw new Error("remote_workspace_migration_operation_missing"); - return { workspace, authority: null, operation, source, sourceFingerprint }; + // Older operations only admitted the task's current workspace. Preserve + // that binding when resuming an import journaled before this field existed. + const issueBinding = Object.hasOwn(operation.metadata ?? {}, "sourceIssueBinding") + ? operation.metadata!.sourceIssueBinding : workspace.id; + if (issueBinding !== current.issueBinding) throw new Error("remote_workspace_migration_source_changed"); + if (Boolean(operation.metadata?.retainedDirectory) !== Boolean(current.retainedDirectory)) throw new Error("remote_workspace_migration_source_changed"); + if (canonical(operation.metadata?.archivedGitSource ?? null) !== canonical(gitSourceProof(current.archivedGitSource))) throw new Error("remote_workspace_migration_source_changed"); + return { workspace, authority: null, operation, source, sourceFingerprint, issueBinding, retainedDirectory: current.retainedDirectory, archivedGitSource: current.archivedGitSource }; } const [operation] = await tx.insert(workspaceOperations).values({ companyId: workspace.companyId, executionWorkspaceId: workspace.id, issueId: workspace.sourceIssueId, cwd: workspace.cwd ?? workspace.providerRef, phase: "remote_workspace_migration", status: "running", - metadata: { schema: SCHEMA, sourceFingerprint, source } }).returning(); + metadata: { schema: SCHEMA, sourceFingerprint, source, sourceIssueBinding: current.issueBinding, + ...(current.archivedGitSource ? { archivedGitSource: gitSourceProof(current.archivedGitSource) } : {}), + ...(current.retainedDirectory ? { retainedDirectory: { restorePointId: current.retainedDirectory.restorePoint.id, + generation: current.retainedDirectory.generation, digest: current.retainedDirectory.restorePoint.digest } } : {}) } }).returning(); await notifyRemoteWorkspaceSnapshots(tx, `workspace:${workspace.id}`); await tx.update(executionWorkspaces).set({ metadata: { ...workspace.metadata, remoteWorkspaceMigration: { schema: SCHEMA, operationId: operation!.id, state: "running", sourceFingerprint, source, } }, updatedAt: new Date() }).where(eq(executionWorkspaces.id, workspace.id)); - return { workspace, authority: null, operation: operation!, source, sourceFingerprint }; + return { workspace, authority: null, operation: operation!, source, sourceFingerprint, issueBinding: current.issueBinding, retainedDirectory: current.retainedDirectory, archivedGitSource: current.archivedGitSource }; }); if (reserved.authority) return { ...await inventoryRemoteWorkspaceMigration(input.db, input), applied: true, replayed: true }; const operation = reserved.operation!; @@ -161,15 +198,21 @@ export async function migrateRemoteWorkspace(input: { operationId: operation.id, generation: 1, epoch: 1 }); const scratch = await fs.mkdtemp(path.join(os.tmpdir(), "pc-workspace-migration-")); try { + const retained = reserved.retainedDirectory?.restorePoint; + const sourceRoot = retained ? await materializeTaskDirectorySnapshot({ + payload: await loadTaskDirectoryRestorePoint({ storage: input.storage, workspace: reserved.workspace, ref: retained }), + target: path.join(scratch, "retained-source"), expectedDigest: retained.digest, + }) : reserved.archivedGitSource ? await materializeArchivedGitMigrationSource({ db: input.db, storage: input.storage, + workspace: reserved.workspace, source: reserved.archivedGitSource, scratch }) : reserved.workspace.cwd ?? reserved.workspace.providerRef!; const archivePath = path.join(scratch, "workspace.pcws"); - const captured = await captureRemoteWorkspaceDirectory({ cwd: reserved.workspace.cwd ?? reserved.workspace.providerRef!, archivePath }); + const captured = await captureRemoteWorkspaceDirectory({ cwd: sourceRoot, archivePath }); const ref = await uploadWorkspaceSnapshot({ db: input.db, storage: input.storage, snapshotId: snapshot.id, companyId: input.companyId, workspaceId: input.workspaceId, archivePath, manifestDigest: captured.manifestDigest }); const restoredArchive = path.join(scratch, "verified.pcws"), restored = path.join(scratch, "restored"); await loadWorkspaceSnapshotArchive({ storage: input.storage, reference: ref, targetPath: restoredArchive }); await restoreRemoteWorkspaceArchive({ archivePath: restoredArchive, targetDir: restored, expectedManifestDigest: ref.manifestDigest }); const currentArchive = path.join(scratch, "current.pcws"); - const current = await captureRemoteWorkspaceDirectory({ cwd: reserved.workspace.cwd ?? reserved.workspace.providerRef!, archivePath: currentArchive }); + const current = await captureRemoteWorkspaceDirectory({ cwd: sourceRoot, archivePath: currentArchive }); if (current.manifestDigest !== captured.manifestDigest) throw new Error("remote_workspace_migration_source_changed"); await lock.execute(sql`select 1`); const publication = await input.db.transaction(async tx => { @@ -181,6 +224,9 @@ export async function migrateRemoteWorkspace(input: { || object(workspace.metadata?.remoteWorkspaceMigration).operationId !== operation.id) throw new Error("remote_workspace_migration_publication_changed"); if (workspace.sourceIssueId) await tx.select({ id: issues.id }).from(issues).where(eq(issues.id, workspace.sourceIssueId)).for("update"); const validation = await inspect(tx as unknown as Db, { ...input, operationId: operation.id }); + if (validation.issueBinding !== reserved.issueBinding) throw new Error("remote_workspace_migration_source_changed"); + if (Boolean(validation.retainedDirectory) !== Boolean(reserved.retainedDirectory)) throw new Error("remote_workspace_migration_source_changed"); + if (canonical(gitSourceProof(validation.archivedGitSource)) !== canonical(gitSourceProof(reserved.archivedGitSource))) throw new Error("remote_workspace_migration_source_changed"); if (validation.reasons.length) throw new Error(`remote_workspace_migration_blocked:${validation.reasons.join(",")}`); const [uploaded] = await tx.select().from(executionWorkspaceSnapshots).where(and(eq(executionWorkspaceSnapshots.companyId, workspace.companyId), eq(executionWorkspaceSnapshots.executionWorkspaceId, workspace.id), eq(executionWorkspaceSnapshots.id, ref.id))).for("update");