mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 04:21:24 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A project can use a storage provider plugin. That plugin prepares each task's git worktree > - A board Retry after a failed run can start a fresh attempt. The fresh attempt starts from the project source > - `preparePluginWorkspace` returns `null` when the task is still bound to a workspace that no storage provider owns. It does this also for a fresh attempt > - On a plugin task environment, the host then reserves an empty adapter-managed "Fresh task workspace" with no repository. The runner fails in it. The next Retry finds that empty workspace bound to the task, returns `null` again and reserves another empty workspace. The task can never start again > - This pull request lets a fresh attempt use the project storage provider even when the previous workspace is not provider-owned > - The benefit is that one Retry gives the task a normal provider worktree again ## Linked Issues or Issue Description **What happened?** Tasks moved off an older execution environment kept a binding to a workspace that no storage provider owned. A board Retry selected `fresh_isolated_workspace`. `preparePluginWorkspace` returned `null` because the bound workspace was not plugin-owned. The host fell back to `reserveFreshPluginTaskWorkspace`, which made an empty `adapter_managed` workspace (strategy `project_primary`, no repository, no recovery snapshots). The run failed in about one second with "Plugin-provided Runner execution operation unavailable; reconcile the same task before retrying". Each later Retry made another empty workspace, chained to the last one. **Steps to reproduce** 1. Use a project with a storage provider plugin that supports fresh workspaces. 2. Bind a task to a workspace that the provider does not own (for example an adapter-managed workspace). 3. Call `preparePluginWorkspace` for a fresh attempt with that workspace as the previous workspace. **Expected behavior** The storage provider prepares a new worktree for the fresh attempt. **Actual behavior** `preparePluginWorkspace` returns `null` and never calls the provider. ## What Changed - `server/src/services/plugin-workspaces.ts`: the early `return null` for a previous workspace that no provider owns no longer applies to a fresh attempt. The fresh attempt then uses the project's storage provider, as for a task with no bound workspace. - `server/src/__tests__/plugin-fresh-workspaces.integration.test.ts`: new test "prepares provider storage for a fresh retry that follows a workspace the provider does not own". ## Verification - The first commit adds only the test. CI on that commit (`d09ad235`, [failed job](https://github.com/paperclipai/paperclip/actions/runs/42976154239/job/130393285731)) shows only this test fail on the current code (`preparePluginWorkspace` returns `null`). - The second commit adds the fix. CI shows the test pass. - The other tests in the file keep the fresh-attempt guarantees: an older provider still fails closed, a changed binding revision still stops the call, and a lost reply still reconciles to the same operation. - I did not run the suite locally. The test needs embedded Postgres, and this environment does not install packages. CI runs it. ## Risks - Low to medium risk. A fresh attempt in a project with a storage provider now uses that provider even when the previous workspace was local or adapter-managed. Before, such an attempt used the fallback path. This matches the project's storage policy, and a fresh attempt does not reuse the previous workspace's files. - Projects without a storage provider are not affected: the function still returns `null` for them. - Non-fresh preparation is not affected. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), through Claude Code with tool use (shell, file edits). Standard reasoning mode. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: lucktastic <13757243+lucktastic@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1004 lines
43 KiB
TypeScript
1004 lines
43 KiB
TypeScript
import { isDeepStrictEqual } from "node:util";
|
|
import { notifyStatusCardIssueChange } from "./status-card-notifications.js";
|
|
import { assertProjectStorageEnvironment, effectiveProjectWorkspacePolicy, requireWorkspaceProviderWorker, type WorkspaceWorkers } from "./project-workspace-provider.js";
|
|
import { releaseProviderRecoveryTransfers } from "./provider-fresh-workspace.js";
|
|
import { retireAbandonedPluginWorkspaceReservation } from "./plugin-workspace-reservations.js";
|
|
import { randomUUID, createHash } from "node:crypto";
|
|
import { and, eq, sql } from "drizzle-orm";
|
|
import {
|
|
environmentLeases,
|
|
executionWorkspaces,
|
|
heartbeatRuns,
|
|
issues,
|
|
projects,
|
|
projectWorkspaces,
|
|
type Db,
|
|
} from "@paperclipai/db";
|
|
import {
|
|
readWorkspaceProviderBinding,
|
|
type WorkspaceProviderBinding,
|
|
type ExecutionWorkspace,
|
|
} from "@paperclipai/shared";
|
|
import {
|
|
pluginWorkspaceParamsSchema,
|
|
workspaceProviderOperationSchema,
|
|
type PluginWorkspaceOperation,
|
|
type PluginWorkspaceRecoveryPullRequest,
|
|
type PluginWorkspaceParams,
|
|
type PluginFreshWorkspace,
|
|
} from "@paperclipai/plugin-sdk";
|
|
import { callWorkspaceProvider } from "./plugin-workspace-preparation.js";
|
|
import { withPluginWorkspaceReplay } from "./plugin-workspace-replay-context.js";
|
|
import { pluginRegistryService } from "./plugin-registry.js";
|
|
import { notifyTerminalWorkspaceWork } from "./terminal-workspace-work.js";
|
|
import { withExecutionWorkspaceLifecycleLock } from "./execution-workspace-lifecycle-lock.js";
|
|
import {
|
|
assertTaskWorkspaceExecutionAccess,
|
|
assertTaskWorkspaceSourceProjectAccess,
|
|
} from "./task-workspace-access.js";
|
|
import { forbidden, conflict } from "../errors.js";
|
|
|
|
export { callWorkspaceProvider } from "./plugin-workspace-preparation.js";
|
|
export type { WorkspaceWorkers } from "./project-workspace-provider.js";
|
|
type Workspace = typeof executionWorkspaces.$inferSelect;
|
|
type TaskPlacement = NonNullable<PluginWorkspaceParams["taskLease"]>;
|
|
const preparationPlacementSchema = pluginWorkspaceParamsSchema.pick({
|
|
generation: true, runId: true, agentId: true, taskLease: true,
|
|
});
|
|
|
|
function readPreparationPlacement(workspace: Workspace) {
|
|
const saved = workspace.metadata?.workspaceProviderPreparationPlacement;
|
|
if (saved === undefined) return undefined;
|
|
const placement = preparationPlacementSchema.parse(saved);
|
|
if (!placement.runId || !placement.agentId ||
|
|
placement.generation !== (workspace.metadata?.lifecycleGeneration ?? 0))
|
|
throw conflict("Workspace preparation placement changed");
|
|
return placement;
|
|
}
|
|
type EnsureTaskWorkspace = (taskId: string, assertOwned: () => void) => Promise<void> | undefined;
|
|
|
|
export async function pluginWorkspaceSupportsMergeAncestry(db: Db, workers: WorkspaceWorkers | undefined, workspace: Workspace) {
|
|
const pluginId = readWorkspaceProviderBinding(workspace.metadata)?.pluginId;
|
|
if (!pluginId) return true;
|
|
const selected = await requireWorkspaceProviderWorker(db, workers, workspace.companyId, pluginId);
|
|
return selected.manifestJson.workspaceProvider?.supportsMergeAncestry !== false;
|
|
}
|
|
|
|
export async function preparePluginWorkspace(
|
|
db: Db,
|
|
workers: WorkspaceWorkers | undefined,
|
|
input: {
|
|
companyId: string;
|
|
projectId: string | null;
|
|
projectWorkspaceId: string | null;
|
|
issueId: string | null;
|
|
runId: string;
|
|
agentId: string;
|
|
responsibleUserId: string | null;
|
|
existingWorkspaceId: string | null;
|
|
mode: string;
|
|
strategy: string;
|
|
selectedProviderPluginId?: string | null;
|
|
environment?: Pick<import("@paperclipai/shared").Environment, "id" | "driver" | "config" | "name"> | null;
|
|
branchMode?: "create" | "existing";
|
|
branchName?: string | null;
|
|
baseRef?: string | null;
|
|
fresh?: PluginFreshWorkspace;
|
|
expectedBindingRevision?: number;
|
|
verifyExisting?: true;
|
|
assertOwned?: () => void;
|
|
acquireTaskLease?: (workspace: Pick<ExecutionWorkspace, "id" | "mode">) => Promise<{ leaseId: string; taskLease: TaskPlacement }>;
|
|
onTaskPrepared?: (ensure: EnsureTaskWorkspace) => void;
|
|
},
|
|
) {
|
|
const [previous] = input.existingWorkspaceId
|
|
? await db
|
|
.select()
|
|
.from(executionWorkspaces)
|
|
.where(
|
|
and(
|
|
eq(executionWorkspaces.id, input.existingWorkspaceId),
|
|
eq(executionWorkspaces.companyId, input.companyId),
|
|
),
|
|
)
|
|
: [];
|
|
const [reservation] = input.fresh ? await db.select().from(executionWorkspaces).where(and(
|
|
eq(executionWorkspaces.companyId, input.companyId),
|
|
eq(executionWorkspaces.sourceIssueId, input.issueId!),
|
|
sql`${executionWorkspaces.metadata}#>>'{workspaceProviderFresh,attemptId}' = ${input.fresh.attemptId}`,
|
|
)).limit(1) : [];
|
|
const existing = input.fresh ? reservation : previous;
|
|
if (input.verifyExisting && (!existing || existing.metadata?.workspaceProviderRemoved === true ||
|
|
existing.metadata?.workspaceProviderRestoration)) return null;
|
|
const prior = existing ?? previous;
|
|
if (
|
|
existing?.metadata?.workspaceProviderRetired === true ||
|
|
existing?.metadata?.workspaceProviderRemoval
|
|
)
|
|
throw conflict("Workspace storage is retired or awaiting cleanup");
|
|
if (!existing && (!input.projectId || !input.projectWorkspaceId)) return null;
|
|
const saved = readWorkspaceProviderBinding(existing?.metadata);
|
|
const priorBinding = readWorkspaceProviderBinding(prior?.metadata);
|
|
const mode = input.fresh ? "isolated_workspace" : existing?.mode ?? input.mode,
|
|
strategy = existing?.strategyType ?? input.strategy;
|
|
const projectId = priorBinding?.projectId ?? input.projectId;
|
|
const [project] = projectId
|
|
? await db
|
|
.select()
|
|
.from(projects)
|
|
.where(
|
|
and(
|
|
eq(projects.id, projectId),
|
|
eq(projects.companyId, input.companyId),
|
|
),
|
|
)
|
|
: [];
|
|
// A fresh attempt starts from the project source, so storage the provider does not own does not decide it.
|
|
if (!input.fresh && prior && prior.providerType !== "plugin" && !priorBinding && !prior.metadata?.workspaceProviderOwner) return null;
|
|
const owner = priorBinding?.pluginId ?? prior?.metadata?.workspaceProviderOwner ??
|
|
(!existing && input.selectedProviderPluginId !== undefined ? input.selectedProviderPluginId :
|
|
!existing && project ? (await effectiveProjectWorkspacePolicy(db, input.companyId, project.executionWorkspacePolicy, false))?.storageProviderPluginId : null);
|
|
const pluginId = typeof owner === "string" ? owner : null;
|
|
if (!pluginId) {
|
|
if (existing?.providerType === "plugin")
|
|
throw conflict("Workspace storage ownership is missing");
|
|
return null;
|
|
}
|
|
if (
|
|
!input.issueId ||
|
|
!project ||
|
|
project.archivedAt ||
|
|
!input.projectWorkspaceId
|
|
)
|
|
throw conflict(
|
|
"Managed storage requires a task and an available project source",
|
|
);
|
|
if (
|
|
!["shared_workspace", "isolated_workspace", "operator_branch"].includes(
|
|
input.mode,
|
|
) ||
|
|
!["project_primary", "git_worktree"].includes(strategy)
|
|
)
|
|
throw conflict("Unsupported managed workspace strategy");
|
|
if (existing && existing.providerType !== "plugin")
|
|
throw conflict(
|
|
"An existing local workspace cannot change storage providers",
|
|
);
|
|
const selectedProvider = await requireWorkspaceProviderWorker(db, workers, input.companyId, pluginId, true);
|
|
if (input.environment) assertProjectStorageEnvironment(selectedProvider, input.environment);
|
|
const declaration = selectedProvider.manifestJson.workspaceProvider!;
|
|
if (input.verifyExisting && declaration.workspaceVerificationVersion !== 1) return null;
|
|
if (input.fresh && declaration.freshWorkspaceVersion !== 1)
|
|
throw conflict("A fresh attempt is unavailable for this workspace storage provider. The existing workspace is retained.", { reason: "fresh_workspace_storage_unsupported" });
|
|
if (
|
|
!declaration.modes.includes(mode as "isolated_workspace") ||
|
|
!declaration.strategies.includes(strategy as "git_worktree")
|
|
) {
|
|
throw conflict(
|
|
"The storage plugin does not support this workspace mode or strategy",
|
|
);
|
|
}
|
|
const actor = {
|
|
type: "agent" as const,
|
|
source: "agent_jwt" as const,
|
|
companyId: input.companyId,
|
|
agentId: input.agentId,
|
|
runId: input.runId,
|
|
onBehalfOfUserId:
|
|
input.responsibleUserId === "local-board"
|
|
? null
|
|
: input.responsibleUserId,
|
|
};
|
|
const authorize = async (workspaceId = existing?.id) => {
|
|
await assertTaskWorkspaceExecutionAccess(db, actor, input.companyId, {
|
|
issueId: input.issueId!,
|
|
...(workspaceId
|
|
? { workspaceId }
|
|
: { projectWorkspaceId: input.projectWorkspaceId }),
|
|
});
|
|
await assertTaskWorkspaceSourceProjectAccess(
|
|
db,
|
|
actor,
|
|
input.companyId,
|
|
project.id,
|
|
);
|
|
const [run] = await db
|
|
.select()
|
|
.from(heartbeatRuns)
|
|
.where(
|
|
and(
|
|
eq(heartbeatRuns.id, input.runId),
|
|
eq(heartbeatRuns.companyId, input.companyId),
|
|
),
|
|
);
|
|
if (
|
|
!run ||
|
|
run.agentId !== input.agentId ||
|
|
![run.issueId, run.nativeIssueId].includes(input.issueId!) ||
|
|
run.status !== "running"
|
|
)
|
|
throw forbidden("Workspace preparation no longer has an active run");
|
|
};
|
|
await authorize();
|
|
const [source] = await db
|
|
.select()
|
|
.from(projectWorkspaces)
|
|
.where(
|
|
and(
|
|
eq(
|
|
projectWorkspaces.id,
|
|
saved?.projectWorkspaceId ?? input.projectWorkspaceId,
|
|
),
|
|
eq(projectWorkspaces.projectId, project.id),
|
|
eq(projectWorkspaces.companyId, input.companyId),
|
|
),
|
|
);
|
|
if (!source) throw conflict("Workspace repository source is unavailable");
|
|
const sourceInput = (prior?.metadata?.workspaceProviderSource as
|
|
| Extract<PluginWorkspaceOperation, { kind: "prepare" }>["source"]
|
|
| undefined) ?? {
|
|
id: source.id,
|
|
revision:
|
|
saved?.sourceRevision ??
|
|
createHash("sha256")
|
|
.update(
|
|
JSON.stringify([
|
|
source.id,
|
|
source.repoUrl,
|
|
source.repoRef,
|
|
source.defaultRef,
|
|
]),
|
|
)
|
|
.digest("hex"),
|
|
repoUrl: existing?.repoUrl ?? source.repoUrl,
|
|
repoRef: existing?.baseRef ?? source.repoRef,
|
|
defaultRef: source.defaultRef,
|
|
};
|
|
const branchName = input.fresh ? null : existing
|
|
? existing.branchName
|
|
: (input.branchName ??
|
|
(strategy === "git_worktree" ? `workspace-${randomUUID()}` : null));
|
|
// Reserve and bind before calling the provider. A crash must not orphan a created checkout.
|
|
const workspace =
|
|
existing ??
|
|
(await db.transaction(async (tx) => {
|
|
const [currentProject] = await tx
|
|
.select()
|
|
.from(projects)
|
|
.where(eq(projects.id, project.id))
|
|
.for("update");
|
|
const [currentSource] = await tx
|
|
.select()
|
|
.from(projectWorkspaces)
|
|
.where(eq(projectWorkspaces.id, source.id));
|
|
if (
|
|
!currentProject ||
|
|
currentProject.archivedAt ||
|
|
(await effectiveProjectWorkspacePolicy(tx, input.companyId, currentProject.executionWorkspacePolicy, false))?.storageProviderPluginId !==
|
|
pluginId ||
|
|
!currentSource ||
|
|
currentSource.repoUrl !== source.repoUrl ||
|
|
currentSource.repoRef !== source.repoRef ||
|
|
currentSource.defaultRef !== source.defaultRef
|
|
) {
|
|
throw conflict(
|
|
"Project storage configuration changed; retry preparation",
|
|
);
|
|
}
|
|
const [task] = await tx
|
|
.select()
|
|
.from(issues)
|
|
.where(
|
|
and(
|
|
eq(issues.id, input.issueId!),
|
|
eq(issues.companyId, input.companyId),
|
|
),
|
|
)
|
|
.for("update");
|
|
if (!task || (input.fresh ? task.executionWorkspaceId !== input.fresh.previousWorkspaceId || task.workspaceBindingRevision !== input.expectedBindingRevision : Boolean(task.executionWorkspaceId)))
|
|
throw conflict("Task workspace binding changed; retry preparation");
|
|
const id = randomUUID();
|
|
await notifyTerminalWorkspaceWork(tx, `workspace:${id}`);
|
|
const [row] = await tx
|
|
.insert(executionWorkspaces)
|
|
.values({
|
|
id,
|
|
companyId: input.companyId,
|
|
projectId: project.id,
|
|
projectWorkspaceId: source.id,
|
|
sourceIssueId: input.issueId,
|
|
mode,
|
|
strategyType: strategy,
|
|
name: branchName ?? source.name,
|
|
providerType: "plugin",
|
|
providerRef: null,
|
|
cwd: null,
|
|
branchName,
|
|
repoUrl: source.repoUrl,
|
|
baseRef: input.baseRef ?? source.repoRef,
|
|
status: "idle",
|
|
metadata: {
|
|
workspaceProviderOwner: pluginId,
|
|
workspaceProviderReservationRunId: input.runId,
|
|
workspaceProviderSource: sourceInput,
|
|
workspaceProviderBranchMode: input.fresh ? "create" : input.branchMode ?? "create",
|
|
...(input.fresh ? { workspaceProviderFresh: input.fresh } : {}),
|
|
},
|
|
})
|
|
.returning();
|
|
await notifyStatusCardIssueChange(tx, task.companyId);
|
|
await tx
|
|
.update(issues)
|
|
.set({
|
|
executionWorkspaceId: id,
|
|
executionWorkspacePreference: "reuse_existing",
|
|
workspaceBindingRevision: task.workspaceBindingRevision + 1,
|
|
workspacePendingSelection:
|
|
task.workspacePendingSelection?.expectedBindingRevision ===
|
|
task.workspaceBindingRevision
|
|
? {
|
|
...task.workspacePendingSelection,
|
|
expectedBindingRevision: task.workspaceBindingRevision + 1,
|
|
}
|
|
: task.workspacePendingSelection,
|
|
updatedAt: new Date(),
|
|
})
|
|
.where(eq(issues.id, task.id));
|
|
return row!;
|
|
}));
|
|
const driverKey = input.environment?.driver === "plugin" ? input.environment.config?.driverKey : undefined;
|
|
const placement = selectedProvider.manifestJson.environmentDrivers?.some(driver =>
|
|
driver.driverKey === driverKey && driver.sessionHandoffVersion === 1)
|
|
? await input.acquireTaskLease?.({ id: workspace.id, mode: mode as ExecutionWorkspace["mode"] }) : undefined;
|
|
const prepare = (workspace: Workspace, taskLease?: TaskPlacement, replacement?: { leaseId: string; assertOwned: () => void }) => withExecutionWorkspaceLifecycleLock(db, workspace.id, async (tx) => {
|
|
(replacement?.assertOwned ?? input.assertOwned)?.();
|
|
const [current] = await tx
|
|
.select()
|
|
.from(executionWorkspaces)
|
|
.where(eq(executionWorkspaces.id, workspace.id));
|
|
if (
|
|
!current ||
|
|
!["active", "idle", "in_review"].includes(current.status) ||
|
|
current.metadata?.workspaceProviderRetired === true ||
|
|
current.metadata?.workspaceProviderRemoval ||
|
|
current.providerRef !== workspace.providerRef ||
|
|
(current.metadata?.lifecycleGeneration ?? 0) !==
|
|
(workspace.metadata?.lifecycleGeneration ?? 0)
|
|
) {
|
|
throw conflict("Workspace lifecycle changed; reopen before preparing");
|
|
}
|
|
if (taskLease) await authorize(replacement ? workspace.id : undefined);
|
|
const replacementScope = replacement && taskLease ? and(
|
|
eq(environmentLeases.id, replacement.leaseId),
|
|
eq(environmentLeases.companyId, input.companyId),
|
|
eq(environmentLeases.heartbeatRunId, input.runId),
|
|
eq(environmentLeases.executionWorkspaceId, workspace.id),
|
|
eq(environmentLeases.environmentId, taskLease.environmentId),
|
|
eq(environmentLeases.providerLeaseId, taskLease.providerLeaseId),
|
|
eq(environmentLeases.status, "active"),
|
|
sql`${environmentLeases.metadata}->>'pluginId' = ${pluginId} and ${environmentLeases.metadata}->>'driverKey' = ${taskLease.driverKey}`,
|
|
sql`${environmentLeases.releasedAt} is null and (${environmentLeases.expiresAt} is null or ${environmentLeases.expiresAt} > clock_timestamp())`,
|
|
sql`coalesce(${environmentLeases.metadata}->>'runnerTaskRetiring', 'false') != 'true'`,
|
|
) : undefined;
|
|
if (replacementScope && !(await tx.select({ id: environmentLeases.id }).from(environmentLeases).where(replacementScope)).length)
|
|
throw conflict("Workspace Launcher placement changed");
|
|
if (!current.providerRef && !input.verifyExisting) {
|
|
const generation = current.metadata?.lifecycleGeneration ?? 0;
|
|
const intent = { generation, runId: input.runId, agentId: input.agentId, ...(taskLease ? { taskLease } : {}) };
|
|
const previous = readPreparationPlacement(current);
|
|
if (!isDeepStrictEqual(previous, intent)) {
|
|
// The request must survive a lost reply or rollback of the lifecycle transaction.
|
|
const [recorded] = await db.update(executionWorkspaces).set({
|
|
metadata: sql`coalesce(${executionWorkspaces.metadata}, '{}'::jsonb) || ${JSON.stringify({ workspaceProviderPreparationPlacement: intent })}::jsonb`,
|
|
}).where(and(eq(executionWorkspaces.id, current.id), eq(executionWorkspaces.companyId, current.companyId),
|
|
sql`${executionWorkspaces.providerRef} is null
|
|
and coalesce((${executionWorkspaces.metadata}->>'lifecycleGeneration')::int, 0) = ${generation}
|
|
and coalesce(${executionWorkspaces.metadata}->'workspaceProviderPreparationPlacement', 'null'::jsonb)
|
|
= ${JSON.stringify(previous ?? null)}::jsonb`)).returning({ id: executionWorkspaces.id });
|
|
if (!recorded) throw conflict("Workspace preparation placement changed");
|
|
}
|
|
}
|
|
const result = await callWorkspaceProvider(tx as unknown as Db, workers, {
|
|
workspace,
|
|
pluginId,
|
|
runId: input.runId,
|
|
agentId: input.agentId,
|
|
taskLease,
|
|
environment: input.environment,
|
|
assertOwned: replacement?.assertOwned ?? input.assertOwned,
|
|
operation: {
|
|
kind: "prepare",
|
|
source: sourceInput,
|
|
mode: workspace.mode as "isolated_workspace",
|
|
strategy: workspace.strategyType as "git_worktree",
|
|
branchName: workspace.metadata?.workspaceProviderFresh ? null : workspace.branchName,
|
|
branchMode:
|
|
(workspace.metadata?.workspaceProviderBranchMode as
|
|
| "create"
|
|
| "existing"
|
|
| undefined) ??
|
|
input.branchMode ??
|
|
"create",
|
|
baseRef: workspace.baseRef,
|
|
existing: readWorkspaceProviderBinding(workspace.metadata),
|
|
...(input.verifyExisting ? { verifyExisting: true } : {}),
|
|
...(workspace.metadata?.workspaceProviderFresh ? { fresh: workspace.metadata.workspaceProviderFresh as PluginFreshWorkspace } : {}),
|
|
},
|
|
});
|
|
if (result.kind !== "prepared")
|
|
throw conflict("Workspace preparation is incomplete");
|
|
if (replacement && result.branchName !== workspace.branchName)
|
|
throw conflict("Workspace Launcher prepared a different branch");
|
|
const savedBase = workspace.metadata?.workspaceProviderBaseSha;
|
|
if (input.verifyExisting && (result.created || !result.baseSha ||
|
|
(saved && result.binding.workspaceRef !== saved.workspaceRef) ||
|
|
(workspace.branchName !== null && result.branchName !== workspace.branchName) ||
|
|
(typeof savedBase === "string" && result.baseSha !== savedBase))) {
|
|
return null;
|
|
}
|
|
(replacement?.assertOwned ?? input.assertOwned)?.();
|
|
await authorize();
|
|
const [currentProject] = await tx
|
|
.select()
|
|
.from(projects)
|
|
.where(eq(projects.id, project.id));
|
|
if (!currentProject || currentProject.archivedAt)
|
|
throw conflict("Project was archived during workspace preparation");
|
|
const [task] = await tx
|
|
.select()
|
|
.from(issues)
|
|
.where(eq(issues.id, input.issueId!));
|
|
if (task?.executionWorkspaceId !== workspace.id)
|
|
throw conflict("Task workspace binding changed during preparation");
|
|
if (replacementScope) {
|
|
const [currentLease] = await tx.update(environmentLeases).set({
|
|
metadata: sql`coalesce(${environmentLeases.metadata}, '{}'::jsonb) || ${JSON.stringify({ workspaceProvider: result.binding })}::jsonb`,
|
|
}).where(replacementScope).returning({ id: environmentLeases.id });
|
|
if (!currentLease) throw conflict("Workspace Launcher placement changed during preparation");
|
|
}
|
|
await notifyTerminalWorkspaceWork(tx, `workspace:${workspace.id}`);
|
|
const { workspaceProviderPreparationPlacement: _placement, ...metadata } = current.metadata ?? {};
|
|
const [updated] = await tx
|
|
.update(executionWorkspaces)
|
|
.set({
|
|
providerRef: result.binding.workspaceRef,
|
|
branchName: result.branchName,
|
|
status: "active",
|
|
lastUsedAt: new Date(),
|
|
updatedAt: new Date(),
|
|
metadata: {
|
|
...metadata,
|
|
workspaceProvider: result.binding,
|
|
workspaceProviderBaseSha: result.baseSha,
|
|
...(result.destinationRef !== undefined ? { workspaceProviderDestinationRef: result.destinationRef } : {}),
|
|
workspaceProviderSource: sourceInput,
|
|
workspaceProviderRemoved: false,
|
|
createdByRuntime: result.branchCreatedByRuntime,
|
|
},
|
|
})
|
|
.where(
|
|
and(
|
|
eq(executionWorkspaces.id, workspace.id),
|
|
eq(executionWorkspaces.companyId, input.companyId),
|
|
sql`coalesce((${executionWorkspaces.metadata}->>'lifecycleGeneration')::int, 0) = ${workspace.metadata?.lifecycleGeneration ?? 0}`,
|
|
sql`${executionWorkspaces.providerRef} IS NOT DISTINCT FROM ${workspace.providerRef}`,
|
|
),
|
|
)
|
|
.returning();
|
|
if (!updated)
|
|
throw conflict("Workspace changed before preparation could be recorded");
|
|
return updated!;
|
|
});
|
|
const prepared = await prepare(workspace, placement?.taskLease);
|
|
if (prepared && placement && input.onTaskPrepared) {
|
|
let preparedTaskId = placement.taskLease.providerLeaseId;
|
|
let preparedWorkspace = prepared;
|
|
input.onTaskPrepared((taskId, assertOwned) => {
|
|
if (taskId === preparedTaskId) return;
|
|
return prepare(preparedWorkspace, { ...placement.taskLease, providerLeaseId: taskId }, { leaseId: placement.leaseId, assertOwned }).then(updated => {
|
|
if (!updated) throw conflict("Workspace preparation is incomplete");
|
|
preparedWorkspace = updated;
|
|
preparedTaskId = taskId;
|
|
});
|
|
});
|
|
}
|
|
return prepared;
|
|
}
|
|
|
|
/** Recover a lost prepare result even when the originating run has already stopped. */
|
|
async function reconcilePluginWorkspaceReservation(
|
|
db: Db,
|
|
workers: WorkspaceWorkers | undefined,
|
|
snapshot: Workspace,
|
|
): Promise<Workspace> {
|
|
const [workspace] = await db
|
|
.select()
|
|
.from(executionWorkspaces)
|
|
.where(
|
|
and(
|
|
eq(executionWorkspaces.id, snapshot.id),
|
|
eq(executionWorkspaces.companyId, snapshot.companyId),
|
|
),
|
|
);
|
|
if (!workspace) throw conflict("Workspace is unavailable");
|
|
if (readWorkspaceProviderBinding(workspace.metadata) || workspace.metadata?.workspaceProviderRemoval ||
|
|
workspace.metadata?.workspaceProviderRemoved === true) return workspace;
|
|
const owner = workspace.metadata?.workspaceProviderOwner;
|
|
const retired = typeof owner === "string" ? await retireAbandonedPluginWorkspaceReservation(db, workspace) : null;
|
|
if (retired) return retired;
|
|
if (typeof owner !== "string" || !workspace.projectId || !workspace.metadata?.workspaceProviderSource)
|
|
throw conflict("Workspace storage reservation is unavailable");
|
|
const operation = workspaceProviderOperationSchema.parse({
|
|
kind: "prepare",
|
|
source: workspace.metadata.workspaceProviderSource,
|
|
mode: workspace.mode,
|
|
strategy: workspace.strategyType,
|
|
branchName: workspace.metadata.workspaceProviderFresh ? null : workspace.branchName,
|
|
branchMode: workspace.metadata.workspaceProviderBranchMode ?? "create",
|
|
baseRef: workspace.baseRef,
|
|
existing: null,
|
|
...(workspace.metadata?.workspaceProviderFresh ? { fresh: workspace.metadata.workspaceProviderFresh as PluginFreshWorkspace } : {}),
|
|
});
|
|
if (operation.kind !== "prepare") throw conflict("Workspace preparation is unavailable");
|
|
const result = await withPluginWorkspaceReplay({
|
|
pluginId: owner, companyId: workspace.companyId, projectId: workspace.projectId,
|
|
projectWorkspaceId: operation.source.id, executionWorkspaceId: workspace.id,
|
|
}, () => callWorkspaceProvider(db, workers, {
|
|
workspace,
|
|
pluginId: owner,
|
|
...readPreparationPlacement(workspace),
|
|
operation,
|
|
}));
|
|
if (result.kind !== "prepared")
|
|
throw conflict("Workspace reconciliation is incomplete");
|
|
const { workspaceProviderPreparationPlacement: _placement, ...metadata } = workspace.metadata ?? {};
|
|
const [updated] = await db.transaction(async (tx) => {
|
|
await notifyTerminalWorkspaceWork(tx, `workspace:${workspace.id}`);
|
|
return tx
|
|
.update(executionWorkspaces)
|
|
.set({
|
|
providerRef: result.binding.workspaceRef,
|
|
branchName: result.branchName,
|
|
updatedAt: new Date(),
|
|
metadata: {
|
|
...metadata,
|
|
workspaceProvider: result.binding,
|
|
workspaceProviderBaseSha: result.baseSha,
|
|
...(result.destinationRef !== undefined ? { workspaceProviderDestinationRef: result.destinationRef } : {}),
|
|
createdByRuntime: result.branchCreatedByRuntime,
|
|
},
|
|
})
|
|
.where(
|
|
and(
|
|
eq(executionWorkspaces.id, workspace.id),
|
|
eq(executionWorkspaces.companyId, workspace.companyId),
|
|
sql`${executionWorkspaces.providerRef} IS NULL`,
|
|
sql`coalesce((${executionWorkspaces.metadata}->>'lifecycleGeneration')::int, 0) = ${workspace.metadata?.lifecycleGeneration ?? 0}`,
|
|
),
|
|
)
|
|
.returning();
|
|
});
|
|
if (!updated)
|
|
throw conflict("Workspace changed during reconciliation; retry inspection");
|
|
return updated;
|
|
}
|
|
|
|
async function inspectBinding(
|
|
db: Db,
|
|
workers: WorkspaceWorkers | undefined,
|
|
workspace: Workspace,
|
|
recoveryPullRequest?: PluginWorkspaceRecoveryPullRequest,
|
|
) {
|
|
const binding = readWorkspaceProviderBinding(workspace.metadata);
|
|
if (!binding) throw conflict("Workspace storage binding is unavailable");
|
|
const result = await callWorkspaceProvider(db, workers, {
|
|
workspace,
|
|
operation: { kind: "inspect", binding, ...(recoveryPullRequest ? { recoveryPullRequest } : {}) },
|
|
});
|
|
if (result.kind !== "inspection")
|
|
throw conflict("Workspace inspection is incomplete");
|
|
return result;
|
|
}
|
|
|
|
/** Reconcile durable lifecycle intent under the same lock for either read path. */
|
|
async function withRetainedWorkspace<T>(db: Db, workers: WorkspaceWorkers | undefined,
|
|
workspace: Workspace, operation: (workspace: Workspace) => Promise<T>) {
|
|
return withExecutionWorkspaceLifecycleLock(db, workspace.id, async () => {
|
|
workspace = await reconcilePluginWorkspaceReservation(db, workers, workspace);
|
|
if (workspace.metadata?.workspaceProviderRestoration)
|
|
workspace = await finishPendingRestoration(db, workers, workspace);
|
|
if (workspace.metadata?.workspaceProviderRemoval)
|
|
workspace = await finishPendingRemoval(db, workers, workspace);
|
|
return operation(workspace);
|
|
});
|
|
}
|
|
|
|
export async function inspectPluginWorkspace(db: Db, workers: WorkspaceWorkers | undefined, workspace: Workspace, recoveryPullRequest?: PluginWorkspaceRecoveryPullRequest) {
|
|
return withRetainedWorkspace(db, workers, workspace, (current) =>
|
|
current.metadata?.workspaceProviderRemoved === true
|
|
? Promise.resolve(removedInspection()) : inspectBinding(db, workers, current, recoveryPullRequest));
|
|
}
|
|
|
|
type PluginWorkspaceFinalizationPlan = { kind: "inspect" } | { kind: "finalize"; taskLease?: TaskPlacement };
|
|
|
|
export async function resolvePluginWorkspaceFinalization(db: Db, workspace: Workspace,
|
|
context: { runId: string; agentId: string; taskLease?: TaskPlacement | null }): Promise<PluginWorkspaceFinalizationPlan | undefined> {
|
|
if (workspace.providerType !== "plugin") return undefined;
|
|
if (context.taskLease) return { kind: "finalize", taskLease: context.taskLease };
|
|
const binding = readWorkspaceProviderBinding(workspace.metadata);
|
|
if (!binding) throw conflict("Workspace storage ownership is missing");
|
|
const selected = await pluginRegistryService(db).getById(binding.pluginId);
|
|
if (!selected) throw conflict("The workspace storage plugin is unavailable");
|
|
// Existing providers continue to use their established inspection contract.
|
|
if (selected.manifestJson.workspaceProvider?.finalizationVersion !== 1) return { kind: "inspect" };
|
|
const portableDrivers = new Set(selected.manifestJson.environmentDrivers?.filter(driver =>
|
|
driver.sessionHandoffVersion === 1).map(driver => driver.driverKey));
|
|
let taskLease: TaskPlacement | undefined;
|
|
if (context.taskLease === undefined && portableDrivers.size > 0) {
|
|
const leases = await db.select().from(environmentLeases).where(and(
|
|
eq(environmentLeases.companyId, workspace.companyId),
|
|
eq(environmentLeases.executionWorkspaceId, workspace.id),
|
|
eq(environmentLeases.heartbeatRunId, context.runId),
|
|
eq(environmentLeases.status, "active"),
|
|
));
|
|
const placements = leases.flatMap(lease => {
|
|
const driverKey = lease.metadata?.driverKey;
|
|
return lease.metadata?.pluginId === binding.pluginId && lease.metadata?.agentId === context.agentId &&
|
|
lease.metadata?.runnerTaskLease === true && lease.environmentId && lease.providerLeaseId &&
|
|
typeof driverKey === "string" && portableDrivers.has(driverKey)
|
|
? [{ driverKey, environmentId: lease.environmentId, providerLeaseId: lease.providerLeaseId }] : [];
|
|
});
|
|
if (placements.length > 1) throw conflict("Workspace Launcher placement is ambiguous");
|
|
taskLease = placements[0];
|
|
}
|
|
return { kind: "finalize", taskLease };
|
|
}
|
|
|
|
/** Completion does not need the deletion-safety proof supplied by inspect. */
|
|
export async function finalizePluginWorkspace(db: Db, workers: WorkspaceWorkers | undefined,
|
|
workspace: Workspace, context: { runId: string; agentId: string; taskLease?: TaskPlacement | null;
|
|
plan?: PluginWorkspaceFinalizationPlan }) {
|
|
return withRetainedWorkspace(db, workers, workspace, async (current) => {
|
|
if (current.metadata?.workspaceProviderRemoved === true)
|
|
throw conflict("Workspace was removed before finalization");
|
|
const binding = readWorkspaceProviderBinding(current.metadata);
|
|
if (!binding) throw conflict("Workspace storage ownership is missing");
|
|
if (context.plan && !isDeepStrictEqual(binding, readWorkspaceProviderBinding(workspace.metadata)))
|
|
throw conflict("Workspace storage ownership changed");
|
|
const plan = context.plan ?? await resolvePluginWorkspaceFinalization(db, current, context);
|
|
if (!plan) throw conflict("The workspace storage plugin is unavailable");
|
|
if (plan.kind === "inspect") return inspectBinding(db, workers, current);
|
|
const taskLease = plan.taskLease;
|
|
const result = await callWorkspaceProvider(db, workers, {
|
|
workspace: current, runId: context.runId, agentId: context.agentId, taskLease, operation: { kind: "finalize", binding },
|
|
});
|
|
if (result.kind !== "finalized") throw conflict("Workspace finalization is incomplete");
|
|
return result;
|
|
});
|
|
}
|
|
|
|
const removedInspection = () => ({
|
|
kind: "inspection" as const,
|
|
headSha: null,
|
|
branchName: null,
|
|
hasDirtyTrackedFiles: false,
|
|
hasUntrackedFiles: false,
|
|
hasLocalOnlyData: false,
|
|
isMergedIntoBase: null,
|
|
remotelyRecoverable: true,
|
|
});
|
|
|
|
async function finishPendingRestoration(
|
|
db: Db,
|
|
workers: WorkspaceWorkers | undefined,
|
|
workspace: Workspace,
|
|
): Promise<Workspace> {
|
|
const pending = workspace.metadata?.workspaceProviderRestoration as {
|
|
generation?: unknown;
|
|
operation?: unknown;
|
|
};
|
|
const operation = workspaceProviderOperationSchema.safeParse(
|
|
pending?.operation,
|
|
);
|
|
const generation =
|
|
(workspace.metadata?.lifecycleGeneration as number | undefined) ?? 0;
|
|
if (
|
|
!operation.success ||
|
|
operation.data.kind !== "prepare" ||
|
|
pending.generation !== generation + 1 ||
|
|
JSON.stringify(operation.data.existing) !==
|
|
JSON.stringify(readWorkspaceProviderBinding(workspace.metadata)) ||
|
|
!["archived", "cleanup_failed"].includes(workspace.status)
|
|
)
|
|
throw conflict("Workspace restoration intent is unavailable or stale");
|
|
const result = await callWorkspaceProvider(db, workers, {
|
|
workspace,
|
|
operation: operation.data,
|
|
advanceGeneration: true,
|
|
});
|
|
if (result.kind !== "prepared")
|
|
throw conflict("Workspace restoration is incomplete");
|
|
const [updated] = await db.transaction(async (tx) => {
|
|
await notifyTerminalWorkspaceWork(tx, `workspace:${workspace.id}`);
|
|
return tx
|
|
.update(executionWorkspaces)
|
|
.set({
|
|
providerRef: result.binding.workspaceRef,
|
|
branchName: result.branchName,
|
|
updatedAt: new Date(),
|
|
metadata: {
|
|
...workspace.metadata,
|
|
lifecycleGeneration: generation + 1,
|
|
workspaceProvider: result.binding,
|
|
workspaceProviderBaseSha: result.baseSha,
|
|
...(result.destinationRef !== undefined ? { workspaceProviderDestinationRef: result.destinationRef } : {}),
|
|
workspaceProviderRemoved: false,
|
|
workspaceProviderRestoration: null,
|
|
},
|
|
})
|
|
.where(
|
|
and(
|
|
eq(executionWorkspaces.id, workspace.id),
|
|
eq(executionWorkspaces.companyId, workspace.companyId),
|
|
eq(executionWorkspaces.status, workspace.status),
|
|
sql`coalesce((${executionWorkspaces.metadata}->>'lifecycleGeneration')::int, 0) = ${generation}`,
|
|
sql`${executionWorkspaces.metadata}->'workspaceProviderRestoration' = ${JSON.stringify(pending)}::jsonb`,
|
|
),
|
|
)
|
|
.returning();
|
|
});
|
|
if (!updated)
|
|
throw conflict("Workspace changed during restoration reconciliation");
|
|
return updated;
|
|
}
|
|
|
|
/** Replay the recorded delete before trying to inspect a checkout that may already be gone. */
|
|
async function finishPendingRemoval(
|
|
db: Db,
|
|
workers: WorkspaceWorkers | undefined,
|
|
workspace: Workspace,
|
|
): Promise<Workspace> {
|
|
if (workspace.metadata?.workspaceProviderRemoved === true) return workspace;
|
|
const pending = workspace.metadata?.workspaceProviderRemoval as
|
|
| { generation?: unknown; operation?: unknown; operationId?: unknown }
|
|
| undefined;
|
|
const parsed = workspaceProviderOperationSchema.safeParse(pending?.operation);
|
|
const binding = readWorkspaceProviderBinding(workspace.metadata);
|
|
if (
|
|
!parsed.success ||
|
|
parsed.data.kind !== "remove" ||
|
|
pending?.generation !== (workspace.metadata?.lifecycleGeneration ?? 0) ||
|
|
JSON.stringify(parsed.data.binding) !== JSON.stringify(binding)
|
|
)
|
|
throw conflict("Workspace removal intent is unavailable or stale");
|
|
if (pending.operationId === undefined && parsed.data.recoveryPullRequest)
|
|
throw conflict("Workspace removal identity is ambiguous; reconciliation requires the original request");
|
|
const operationId = createHash("sha256").update(JSON.stringify([workspace.id, pending.generation, parsed.data])).digest("hex");
|
|
if (pending.operationId !== undefined && pending.operationId !== operationId)
|
|
throw conflict("Workspace removal identity does not match its recorded payload");
|
|
const owner = workspace.metadata?.workspaceProviderOwner;
|
|
const result = await callWorkspaceProvider(db, workers, {
|
|
workspace,
|
|
pluginId: typeof owner === "string" ? owner : undefined,
|
|
operation: parsed.data,
|
|
removalOperationId: operationId,
|
|
});
|
|
if (result.kind !== "removed")
|
|
throw conflict("Workspace removal is incomplete");
|
|
if (workspace.metadata?.workspaceProviderFresh)
|
|
await releaseProviderRecoveryTransfers(workspace.companyId, workspace.metadata.workspaceProviderFresh as PluginFreshWorkspace);
|
|
const [updated] = await db.transaction(async (tx) => {
|
|
await notifyTerminalWorkspaceWork(tx, `workspace:${workspace.id}`);
|
|
return tx
|
|
.update(executionWorkspaces)
|
|
.set({
|
|
...(workspace.status === "cleanup_failed"
|
|
? { status: "archived", cleanupReason: null }
|
|
: {}),
|
|
metadata: {
|
|
...workspace.metadata,
|
|
workspaceProviderRemoval: null,
|
|
workspaceProviderRemoved: true,
|
|
},
|
|
updatedAt: new Date(),
|
|
})
|
|
.where(
|
|
and(
|
|
eq(executionWorkspaces.id, workspace.id),
|
|
eq(executionWorkspaces.companyId, workspace.companyId),
|
|
sql`coalesce((${executionWorkspaces.metadata}->>'lifecycleGeneration')::int, 0) = ${pending!.generation}`,
|
|
),
|
|
)
|
|
.returning();
|
|
});
|
|
if (!updated)
|
|
throw conflict("Workspace changed during removal reconciliation");
|
|
return updated;
|
|
}
|
|
|
|
export async function removePluginWorkspace(
|
|
db: Db,
|
|
workers: WorkspaceWorkers | undefined,
|
|
workspace: Workspace,
|
|
expectedHeadSha: string | null,
|
|
recoveryPullRequest?: PluginWorkspaceRecoveryPullRequest,
|
|
beforeRemove?: () => Promise<void>,
|
|
) {
|
|
workspace = await reconcilePluginWorkspaceReservation(db, workers, workspace);
|
|
if (workspace.metadata?.workspaceProviderRestoration)
|
|
throw conflict("Workspace restoration must be reconciled before removal");
|
|
if (workspace.metadata?.workspaceProviderRemoved === true) {
|
|
if (workspace.metadata?.workspaceProviderFresh)
|
|
await releaseProviderRecoveryTransfers(workspace.companyId, workspace.metadata.workspaceProviderFresh as PluginFreshWorkspace);
|
|
return;
|
|
}
|
|
if (workspace.metadata?.workspaceProviderRemoval) {
|
|
await finishPendingRemoval(db, workers, workspace);
|
|
return;
|
|
}
|
|
const binding = readWorkspaceProviderBinding(workspace.metadata);
|
|
if (!binding) throw conflict("Workspace storage binding is unavailable");
|
|
const inspection = await inspectBinding(db, workers, workspace, recoveryPullRequest);
|
|
if (
|
|
inspection.hasDirtyTrackedFiles ||
|
|
inspection.hasUntrackedFiles ||
|
|
inspection.hasLocalOnlyData ||
|
|
inspection.headSha !== expectedHeadSha ||
|
|
(inspection.remotelyRecoverable !== true &&
|
|
inspection.isMergedIntoBase !== true)
|
|
) {
|
|
throw conflict(
|
|
"Workspace has local changes or cannot be recovered remotely",
|
|
);
|
|
}
|
|
await beforeRemove?.();
|
|
const generation = workspace.metadata?.lifecycleGeneration ?? 0;
|
|
const selectedProvider = await requireWorkspaceProviderWorker(db, workers, workspace.companyId, binding.pluginId);
|
|
const operation = workspaceProviderOperationSchema.parse({ kind: "remove", binding, expectedHeadSha,
|
|
...(recoveryPullRequest && selectedProvider.manifestJson.workspaceProvider?.recoveryPullRequestVersion === 1 ? { recoveryPullRequest } : {}) });
|
|
const operationId = createHash("sha256").update(JSON.stringify([workspace.id, generation, operation])).digest("hex");
|
|
const [pending] = await db.transaction(async (tx) => {
|
|
await notifyTerminalWorkspaceWork(tx, `workspace:${workspace.id}`);
|
|
return tx
|
|
.update(executionWorkspaces)
|
|
.set({
|
|
metadata: {
|
|
...workspace.metadata,
|
|
workspaceProviderRemoval: { generation, operation, operationId },
|
|
},
|
|
updatedAt: new Date(),
|
|
})
|
|
.where(
|
|
and(
|
|
eq(executionWorkspaces.id, workspace.id),
|
|
eq(executionWorkspaces.companyId, workspace.companyId),
|
|
sql`coalesce((${executionWorkspaces.metadata}->>'lifecycleGeneration')::int, 0) = ${generation}`,
|
|
eq(executionWorkspaces.providerRef, binding.workspaceRef),
|
|
),
|
|
)
|
|
.returning();
|
|
});
|
|
if (!pending)
|
|
throw conflict("Workspace changed before removal could be recorded");
|
|
await finishPendingRemoval(db, workers, pending);
|
|
}
|
|
|
|
/** An authorized storage edit atomically retires closed, removed dependencies under the reopen fence. */
|
|
export async function retireRemovedPluginWorkspaceDependents(
|
|
db: Db,
|
|
projectId: string,
|
|
sourceId?: string,
|
|
) {
|
|
const rows = await db
|
|
.select({ id: executionWorkspaces.id })
|
|
.from(executionWorkspaces)
|
|
.where(
|
|
and(
|
|
eq(executionWorkspaces.projectId, projectId),
|
|
eq(executionWorkspaces.providerType, "plugin"),
|
|
sourceId
|
|
? eq(executionWorkspaces.projectWorkspaceId, sourceId)
|
|
: undefined,
|
|
),
|
|
);
|
|
for (const { id } of rows.sort((a, b) => a.id.localeCompare(b.id))) {
|
|
const acquired = await db.execute(
|
|
sql`select pg_try_advisory_xact_lock(hashtextextended(${`execution_workspace_lifecycle:${id}`}, 0)) as acquired`,
|
|
);
|
|
if (!acquired[0]?.acquired)
|
|
throw conflict("Workspace lifecycle is busy; retry the source edit");
|
|
const [row] = await db
|
|
.select()
|
|
.from(executionWorkspaces)
|
|
.where(eq(executionWorkspaces.id, id));
|
|
if (!row || row.metadata?.workspaceProviderRetired === true) continue;
|
|
if (
|
|
row.metadata?.workspaceProviderRemoved !== true ||
|
|
row.metadata?.workspaceProviderRestoration ||
|
|
!["archived", "cleanup_failed"].includes(row.status)
|
|
) {
|
|
throw conflict(
|
|
"Retire managed execution workspaces before changing their storage owner or repository source",
|
|
);
|
|
}
|
|
await db
|
|
.update(executionWorkspaces)
|
|
.set({
|
|
metadata: { ...row.metadata, workspaceProviderRetired: true },
|
|
updatedAt: new Date(),
|
|
})
|
|
.where(eq(executionWorkspaces.id, row.id));
|
|
}
|
|
}
|
|
|
|
/** Called inside Core's existing lifecycle fence after task authorization. */
|
|
export async function restorePluginWorkspace(
|
|
db: Db,
|
|
workers: WorkspaceWorkers | undefined,
|
|
workspace: Workspace,
|
|
) {
|
|
workspace = await reconcilePluginWorkspaceReservation(db, workers, workspace);
|
|
if (workspace.metadata?.workspaceProviderRetired === true)
|
|
throw conflict("Workspace storage was retired");
|
|
if (workspace.metadata?.workspaceProviderRemoval)
|
|
workspace = await finishPendingRemoval(db, workers, workspace);
|
|
const binding = readWorkspaceProviderBinding(workspace.metadata);
|
|
const source = workspace.metadata?.workspaceProviderSource;
|
|
if (!binding || !source)
|
|
throw conflict("Workspace restoration recipe is unavailable");
|
|
const operation = workspaceProviderOperationSchema.parse({
|
|
kind: "prepare",
|
|
source,
|
|
mode: workspace.mode,
|
|
strategy: workspace.strategyType,
|
|
branchName: workspace.metadata?.workspaceProviderFresh ? null : workspace.branchName,
|
|
branchMode: workspace.metadata?.workspaceProviderFresh ? "create" : "existing",
|
|
baseRef: workspace.baseRef,
|
|
existing: binding,
|
|
...(workspace.metadata?.workspaceProviderFresh ? { fresh: workspace.metadata.workspaceProviderFresh as PluginFreshWorkspace } : {}),
|
|
});
|
|
const [pending] = await db.transaction(async (tx) => {
|
|
await notifyTerminalWorkspaceWork(tx, `workspace:${workspace.id}`);
|
|
return tx
|
|
.update(executionWorkspaces)
|
|
.set({
|
|
metadata: {
|
|
...workspace.metadata,
|
|
workspaceProviderRemoved: false,
|
|
workspaceProviderRestoration: {
|
|
generation:
|
|
((workspace.metadata?.lifecycleGeneration as number) ?? 0) + 1,
|
|
operation,
|
|
},
|
|
},
|
|
updatedAt: new Date(),
|
|
})
|
|
.where(
|
|
and(
|
|
eq(executionWorkspaces.id, workspace.id),
|
|
eq(executionWorkspaces.companyId, workspace.companyId),
|
|
sql`coalesce((${executionWorkspaces.metadata}->>'lifecycleGeneration')::int, 0) = ${workspace.metadata?.lifecycleGeneration ?? 0}`,
|
|
),
|
|
)
|
|
.returning();
|
|
});
|
|
if (!pending)
|
|
throw conflict("Workspace changed before restoration could be recorded");
|
|
workspace = pending;
|
|
const result = await callWorkspaceProvider(db, workers, {
|
|
workspace,
|
|
operation,
|
|
advanceGeneration: true,
|
|
});
|
|
if (result.kind !== "prepared")
|
|
throw conflict("Workspace restoration is incomplete");
|
|
return {
|
|
...workspace.metadata,
|
|
workspaceProvider: result.binding,
|
|
workspaceProviderRemoved: false,
|
|
workspaceProviderRestoration: null,
|
|
workspaceProviderBaseSha: result.baseSha,
|
|
...(result.destinationRef !== undefined ? { workspaceProviderDestinationRef: result.destinationRef } : {}),
|
|
};
|
|
}
|