From a708a133363a73bd6ad0491cf38dbddfd54c180e Mon Sep 17 00:00:00 2001 From: Nicky Leach Date: Fri, 23 Oct 2026 09:07:35 -0700 Subject: [PATCH] Separate plugin workspace completion from maintenance inspection (#17329) ## Thinking Path > - Paperclip manages AI agents and their work. > - Storage plugins own remote task workspaces. > - Run completion currently requests the same inspection used for storage cleanup. > - That inspection can fetch remote refs and scan files after the agent has finished. > - Completion needs retained workspace identity and durable writes, not permission to delete files. > - This change adds an optional completion operation and keeps maintenance inspection separate. ## Linked Issues or Issue Description **What happened?** A successful plugin-backed run can fail during completion because the storage provider performs a slow remote-recoverability inspection. Dirty or unpublished output does not need a deletion-safety proof to remain valid task output. **Expected behavior** Completion verifies the assigned workspace and confirms durable writes through the provider. Cleanup still requires the existing full inspection. **Steps to reproduce** Run a task with plugin-owned storage whose inspection fetches remote refs. Let the agent finish, then observe completion waiting for the maintenance inspection. Related context: #16428 added finalization observability. The duplicate search found no existing PR for this provider contract. ## What Changed - Add optional `workspaceProvider.finalizationVersion: 1` and strict `finalize` / `finalized` schemas. - Pass original run and agent context through live completion and recovered completion. - Give the new operation a 30-second deadline and keep ownership checks and lifecycle serialization. - Retain the inspection fallback for providers without the capability. Keep maintenance inspection and removal unchanged. ## Verification - SDK build passed. - Targeted SDK and completion unit tests: 7 passed. - Added integration coverage for capability selection, legacy fallback, maintenance separation, and failure propagation. - Local integration setup timed out starting PostgreSQL; the integration suite passed in GitHub CI. - Full build/typecheck and broader tests passed in GitHub CI (55 successful checks, 2 skipped). Local checks remained targeted as requested. - Diff reviewed for secrets and operational identifiers. ## Risks The capability is opt-in. Providers must not advertise it until their completion implementation is available. An unavailable provider or failed durability barrier remains retryable. No schema or stored-binding migration is required. Test-file size exception: the provider integration suite grows from 973 to 1,002 lines. The 29-line case follows the same prepared binding through legacy completion, opted-in completion, and maintenance inspection using the existing single-connection database fixture. Keeping it there avoids a second embedded-Postgres lifecycle for one transition test. The completion helper unit tests are in a separate focused file. ## Model Used OpenAI GPT-6 Codex, with reasoning and code/test tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- doc/plugins/PLUGIN_RUNNER_EXECUTION.md | 20 +++++++ .../plugins/sdk/src/workspace-provider.ts | 9 ++- .../sdk/tests/workspace-provider.test.ts | 10 ++++ packages/shared/src/types/plugin.ts | 2 + packages/shared/src/validators/plugin.ts | 1 + .../plugin-workspaces.integration.test.ts | 29 ++++++++++ .../heartbeat/runtime-execution.test.ts | 4 +- .../services/heartbeat/runtime-execution.ts | 2 +- ...tive-plugin-workspace-finalization.test.ts | 20 +++++++ .../native-plugin-workspace-finalization.ts | 5 +- .../native-workspace-finalizer.test.ts | 6 +- .../native-workspace-finalizer.ts | 2 +- server/src/services/plugin-workspaces.ts | 55 +++++++++++++------ 13 files changed, 137 insertions(+), 28 deletions(-) create mode 100644 server/src/services/native-runtime/native-plugin-workspace-finalization.test.ts diff --git a/doc/plugins/PLUGIN_RUNNER_EXECUTION.md b/doc/plugins/PLUGIN_RUNNER_EXECUTION.md index 46dafc05e93..8b537369074 100644 --- a/doc/plugins/PLUGIN_RUNNER_EXECUTION.md +++ b/doc/plugins/PLUGIN_RUNNER_EXECUTION.md @@ -139,3 +139,23 @@ after it saves the agent. Core does not require a computer before this save. The Runner fixture uses a synthetic Codex provider and local sockets. It does not qualify a deployed plugin, live account, or production WSS proxy. + +## Workspace completion and maintenance + +A storage provider can declare `workspaceProvider.finalizationVersion: 1`. +Core then sends `workspaceOperation` with `operation.kind: "finalize"`, the retained +binding, lifecycle generation, and originating `runId` and `agentId`. The operation +must verify workspace identity and make the run's retained writes durable before +returning `{ kind: "finalized", headSha, branchName }`. Git workspaces must still +report the assigned branch. Non-Git workspaces return null Git fields. + +Completion permits dirty, untracked, ignored, and unpublished work. It does not +prove remote recoverability or authorize deletion. The provider chooses how to +meet this contract; Core does not prescribe a VM, filesystem, or Git layout. +Core gives completion a 30-second deadline. An unavailable execution host or an +unconfirmed durability barrier must fail retryably, never return success. +Recovery receives the original run and agent context. + +The existing agent-independent `inspect` and `remove` operations continue to +protect storage maintenance. Providers that do not declare finalization version 1 +retain the inspection-based completion path. No stored binding migration is required. diff --git a/packages/plugins/sdk/src/workspace-provider.ts b/packages/plugins/sdk/src/workspace-provider.ts index 456411cbf92..bd28c5610fd 100644 --- a/packages/plugins/sdk/src/workspace-provider.ts +++ b/packages/plugins/sdk/src/workspace-provider.ts @@ -44,6 +44,9 @@ export const workspaceProviderOperationSchema = z.discriminatedUnion("kind", [ fresh: freshWorkspaceSchema.optional(), }) .strict(), + // Verify the retained workspace and flush run writes. This does not certify + // remote recoverability or authorize removal; dirty work is valid output. + z.object({ kind: z.literal("finalize"), binding: workspaceProviderBindingSchema }).strict(), z .object({ kind: z.literal("inspect"), @@ -81,6 +84,8 @@ export const workspaceProviderResultSchema = z.discriminatedUnion("kind", [ remotelyRecoverable: z.boolean().nullable(), }) .strict(), + // Return only after retained files are durable and their identity is verified. + z.object({ kind: z.literal("finalized"), headSha: sha.nullable(), branchName: z.string().nullable() }).strict(), // Return only after removal is confirmed. Busy, failed and uncertain operations must throw. z.object({ kind: z.literal("removed") }).strict(), ]); @@ -105,7 +110,7 @@ export const pluginWorkspaceParamsSchema = z operation: workspaceProviderOperationSchema, }) .strict(); -/** Run and agent are optional context. Storage operations never require an execution lease. */ +/** Finalization receives the originating run/agent context; maintenance remains agent-independent. */ export type PluginWorkspaceParams = z.infer; export function parseWorkspaceProviderResult( @@ -118,7 +123,7 @@ export function parseWorkspaceProviderResult( ? "prepared" : params.operation.kind === "inspect" ? "inspection" - : "removed"; + : params.operation.kind === "finalize" ? "finalized" : "removed"; if (result.kind !== expected) throw new Error("Workspace provider returned the wrong operation result"); if ( diff --git a/packages/plugins/sdk/tests/workspace-provider.test.ts b/packages/plugins/sdk/tests/workspace-provider.test.ts index 5f79260f6dd..b21f7320857 100644 --- a/packages/plugins/sdk/tests/workspace-provider.test.ts +++ b/packages/plugins/sdk/tests/workspace-provider.test.ts @@ -50,6 +50,16 @@ const result = { branchCreatedByRuntime: true, }; describe("workspace provider SDK", () => { + it("separates completion acknowledgement from removal safety", () => { + const input: PluginWorkspaceParams = { ...params, agentId: id(6), runId: id(7), + operation: { kind: "finalize", binding: result.binding } }; + expect(workspaceProviderOperationSchema.parse(input.operation)).toEqual(input.operation); + expect(parseWorkspaceProviderResult(input, { kind: "finalized", headSha: "a".repeat(40), branchName: "task" })) + .toEqual({ kind: "finalized", headSha: "a".repeat(40), branchName: "task" }); + expect(() => parseWorkspaceProviderResult(input, { kind: "removed" })).toThrow(); + expect(() => parseWorkspaceProviderResult(input, { kind: "finalized", headSha: "bad", branchName: "task" })).toThrow(); + expect(() => workspaceProviderOperationSchema.parse({ ...input.operation, localPath: "/untrusted" })).toThrow(); + }); it("prepares without a VM, lease, or host path", async () => { const harness = createWorkspaceProviderHarness(async (input) => { expect(input.agentId).toBeNull(); diff --git a/packages/shared/src/types/plugin.ts b/packages/shared/src/types/plugin.ts index 89b70b6519b..acfebb4395a 100644 --- a/packages/shared/src/types/plugin.ts +++ b/packages/shared/src/types/plugin.ts @@ -697,6 +697,8 @@ export interface PaperclipPluginManifestV1 { agentLifecycle?: true; /** Own project storage and durable workspace operations as one contract. */ workspaceProvider?: { + /** Supports bounded run completion separately from removal-safety inspection. */ + finalizationVersion?: 1; freshWorkspaceVersion?: 1; modes: Array<"shared_workspace" | "isolated_workspace" | "operator_branch">; strategies: Array<"project_primary" | "git_worktree">; diff --git a/packages/shared/src/validators/plugin.ts b/packages/shared/src/validators/plugin.ts index 70a688978dc..16b8092f0c0 100644 --- a/packages/shared/src/validators/plugin.ts +++ b/packages/shared/src/validators/plugin.ts @@ -792,6 +792,7 @@ export const pluginManifestV1Schema = z.object({ capabilities: z.array(z.enum(PLUGIN_CAPABILITIES)).min(1), agentLifecycle: z.literal(true).optional(), workspaceProvider: z.object({ + finalizationVersion: z.literal(1).optional(), freshWorkspaceVersion: z.literal(1).optional(), modes: z.array(z.enum(["shared_workspace", "isolated_workspace", "operator_branch"])).min(1).max(3), strategies: z.array(z.enum(["project_primary", "git_worktree"])).min(1).max(2), diff --git a/server/src/__tests__/plugin-workspaces.integration.test.ts b/server/src/__tests__/plugin-workspaces.integration.test.ts index 7f3952b5f86..0d6a92f3c99 100644 --- a/server/src/__tests__/plugin-workspaces.integration.test.ts +++ b/server/src/__tests__/plugin-workspaces.integration.test.ts @@ -14,6 +14,7 @@ import { import { preparePluginWorkspace, inspectPluginWorkspace, + finalizePluginWorkspace, removePluginWorkspace, type WorkspaceWorkers, } from "../services/plugin-workspaces.js"; @@ -42,6 +43,34 @@ const support = await getEmbeddedPostgresTestSupport(); }); const fixture = (companyId?: string) => createPluginWorkspaceFixture(db, companyId); + it("uses bounded finalization only for opted-in providers, preserving maintenance inspection", async () => { + const f = await fixture(); + const workspace = (await preparePluginWorkspace(db, f.workers, f.input))!; + const context = { runId: f.runId, agentId: f.input.agentId }; + expect((await finalizePluginWorkspace(db, f.workers, workspace, context)).kind).toBe("inspection"); + const [plugin] = await db.select().from(plugins).where(eq(plugins.id, f.pluginId)); + await db.update(plugins).set({ manifestJson: { ...plugin.manifestJson, + workspaceProvider: { ...plugin.manifestJson.workspaceProvider!, finalizationVersion: 1 } } }).where(eq(plugins.id, f.pluginId)); + const completionIds: string[] = []; + const call = f.workers.call.bind(f.workers); + f.workers.call = async (...args) => { + const p = args[2] as any; + if (p.operation.kind !== "finalize") return call(...args); + expect(p.agentId).toBe(context.agentId); + completionIds.push(p.operationId); + expect(Date.parse(p.deadline) - Date.now()).toBeLessThanOrEqual(30_000); + expect(args[3]).toBe(40_000); + return { kind: "finalized", branchName: workspace.branchName, headSha: "a".repeat(40) }; + }; + expect((await finalizePluginWorkspace(db, f.workers, workspace, context)).kind).toBe("finalized"); + await finalizePluginWorkspace(db, f.workers, workspace, context); + await finalizePluginWorkspace(db, f.workers, workspace, { ...context, runId: randomUUID() }); + expect(completionIds[1]).toBe(completionIds[0]); + expect(completionIds[2]).not.toBe(completionIds[0]); + expect((await inspectPluginWorkspace(db, f.workers, workspace)).kind).toBe("inspection"); + f.workers.call = async () => { throw new Error("sync failed"); }; + await expect(finalizePluginWorkspace(db, f.workers, workspace, context)).rejects.toThrow("incomplete"); + }); it("keeps the single-connection pool responsive while provider preparation holds its lifecycle lock", async () => { const f = await fixture(); let entered!: () => void, release!: () => void; diff --git a/server/src/services/heartbeat/runtime-execution.test.ts b/server/src/services/heartbeat/runtime-execution.test.ts index 2b4e91c57cc..c080ba515ff 100644 --- a/server/src/services/heartbeat/runtime-execution.test.ts +++ b/server/src/services/heartbeat/runtime-execution.test.ts @@ -209,7 +209,7 @@ describe.skipIf(!support.supported)("heartbeat runtime execution boundary", () = name: "Plugin worktree", providerType: "plugin", providerRef: "provider:opaque-worktree", cwd: "/workspaces/plugin-only", mode: "isolated_workspace", strategyType: "git_worktree", branchName: "task-branch" }).returning(); input.workspace.persistedExecutionWorkspace = workspace; - const inspect = vi.spyOn(pluginWorkspaces, "inspectPluginWorkspace").mockResolvedValue({ kind: "inspection", + const inspect = vi.spyOn(pluginWorkspaces, "finalizePluginWorkspace").mockResolvedValue({ kind: "inspection", branchName: branch === "expected" ? "task-branch" : "other-branch", headSha: "a".repeat(40), hasDirtyTrackedFiles: true, hasUntrackedFiles: false, hasLocalOnlyData: true, isMergedIntoBase: false, remotelyRecoverable: false }); try { @@ -221,7 +221,7 @@ describe.skipIf(!support.supported)("heartbeat runtime execution boundary", () = await expect(result).rejects.toThrow("plugin_workspace_branch_mismatch"); expect((await finalizations()).some(row => row.status === "succeeded")).toBe(false); } - expect(inspect).toHaveBeenCalledWith(db, undefined, expect.objectContaining({ id: workspace.id, providerRef: workspace.providerRef })); + expect(inspect).toHaveBeenCalledWith(db, undefined, expect.objectContaining({ id: workspace.id, providerRef: workspace.providerRef }), { agentId: input.agent.id, runId: input.run.id }); } finally { inspect.mockRestore(); } }); diff --git a/server/src/services/heartbeat/runtime-execution.ts b/server/src/services/heartbeat/runtime-execution.ts index 039dbe69348..c080a24cae8 100644 --- a/server/src/services/heartbeat/runtime-execution.ts +++ b/server/src/services/heartbeat/runtime-execution.ts @@ -284,7 +284,7 @@ export async function executeHeartbeatRuntime(db: Db, input: HeartbeatRuntimeExe ) : persistedExecutionWorkspace; pluginWorkspaceFinalization = workspaceRecord - ? await inspectPluginWorkspaceForFinalization(db, options.pluginWorkerManager, workspaceRecord, agent.id) + ? await inspectPluginWorkspaceForFinalization(db, options.pluginWorkerManager, workspaceRecord, agent.id, run.id) : null; if (pluginWorkspaceFinalization || workspaceRecord?.strategyType !== "git_worktree") return null; diff --git a/server/src/services/native-runtime/native-plugin-workspace-finalization.test.ts b/server/src/services/native-runtime/native-plugin-workspace-finalization.test.ts new file mode 100644 index 00000000000..f8c461f6744 --- /dev/null +++ b/server/src/services/native-runtime/native-plugin-workspace-finalization.test.ts @@ -0,0 +1,20 @@ +import { describe, it, expect, vi } from "vitest"; +import { inspectPluginWorkspaceForFinalization } from "./native-plugin-workspace-finalization.js"; +import { finalizePluginWorkspace } from "../plugin-workspaces.js"; +vi.mock("../plugin-workspaces.js", () => ({ finalizePluginWorkspace: vi.fn() })); +const db = {} as Parameters[0]; +const workspace = { id: "workspace", providerType: "plugin", strategyType: "git_worktree", branchName: "task" } as Parameters[2]; +describe("provider run completion", () => { + it("passes original run ownership and accepts retained unpublished work", async () => { + vi.mocked(finalizePluginWorkspace).mockResolvedValue({kind:"finalized",headSha:"a".repeat(40),branchName:"task"}); + expect(await inspectPluginWorkspaceForFinalization(db,undefined,workspace,"agent","run")) + .toMatchObject({providerType:"plugin",inspection:{kind:"finalized"}}); + expect(finalizePluginWorkspace).toHaveBeenLastCalledWith(db,undefined,workspace,{agentId:"agent",runId:"run"}); + }); + it("fails closed on changed branch and durability failure", async () => { + vi.mocked(finalizePluginWorkspace).mockResolvedValue({kind:"finalized",headSha:"a".repeat(40),branchName:"wrong"}); + await expect(inspectPluginWorkspaceForFinalization(db,undefined,workspace,"agent","run")).rejects.toThrow("branch_mismatch"); + vi.mocked(finalizePluginWorkspace).mockRejectedValue(new Error("sync incomplete")); + await expect(inspectPluginWorkspaceForFinalization(db,undefined,workspace,"agent","run")).rejects.toThrow("sync incomplete"); + }); +}); diff --git a/server/src/services/native-runtime/native-plugin-workspace-finalization.ts b/server/src/services/native-runtime/native-plugin-workspace-finalization.ts index 3d94e756df0..5eec1afccb0 100644 --- a/server/src/services/native-runtime/native-plugin-workspace-finalization.ts +++ b/server/src/services/native-runtime/native-plugin-workspace-finalization.ts @@ -1,5 +1,5 @@ import type { Db, executionWorkspaces } from "@paperclipai/db"; -import { inspectPluginWorkspace, type WorkspaceWorkers } from "../plugin-workspaces.js"; +import { finalizePluginWorkspace, type WorkspaceWorkers } from "../plugin-workspaces.js"; /** Validate retained plugin files without interpreting a provider reference as a Core path. */ export async function inspectPluginWorkspaceForFinalization( @@ -7,9 +7,10 @@ export async function inspectPluginWorkspaceForFinalization( workers: WorkspaceWorkers | undefined, workspace: typeof executionWorkspaces.$inferSelect, agentId: string, + runId: string, ): Promise | null> { if (workspace.providerType === "plugin") { - const inspection = await inspectPluginWorkspace(db, workers, workspace); + const inspection = await finalizePluginWorkspace(db, workers, workspace, { agentId, runId }); if (workspace.strategyType === "git_worktree" && (!workspace.branchName || inspection.branchName !== workspace.branchName)) { throw new Error("plugin_workspace_branch_mismatch"); diff --git a/server/src/services/native-runtime/native-workspace-finalizer.test.ts b/server/src/services/native-runtime/native-workspace-finalizer.test.ts index c674bcfb4e2..f79698d5b9a 100644 --- a/server/src/services/native-runtime/native-workspace-finalizer.test.ts +++ b/server/src/services/native-runtime/native-workspace-finalizer.test.ts @@ -730,12 +730,12 @@ describe("native workspace finalization recovery", () => { mode: "isolated_workspace", strategyType: providerType === "plugin" ? "git_worktree" : "project_primary", branchName: "task-branch", metadata: providerType === "plugin" ? {} : { runnerTaskWorkspaceOwner: { agentId } } }).returning(); const seeded = await seedRun({ executionWorkspaceId: workspace.id, title: "Plugin result before controller restart" }); - const inspect = vi.spyOn(pluginWorkspaces, "inspectPluginWorkspace").mockResolvedValue({ kind: "inspection", + const inspect = vi.spyOn(pluginWorkspaces, "finalizePluginWorkspace").mockResolvedValue({ kind: "inspection", branchName: "task-branch", headSha: "a".repeat(40), hasDirtyTrackedFiles: true, hasUntrackedFiles: false, hasLocalOnlyData: true, isMergedIntoBase: false, remotelyRecoverable: false }); try { expect(await resumeNativeWorkspaceFinalization({ db, runId: seeded.runId })).toMatchObject({ status: "succeeded" }); - if (providerType === "plugin") expect(inspect).toHaveBeenCalledWith(db, undefined, expect.objectContaining({ id: workspace.id })); + if (providerType === "plugin") expect(inspect).toHaveBeenCalledWith(db, undefined, expect.objectContaining({ id: workspace.id }), { agentId, runId: seeded.runId }); else expect(inspect).not.toHaveBeenCalled(); } finally { inspect.mockRestore(); } }); @@ -749,7 +749,7 @@ describe("native workspace finalization recovery", () => { ]).returning(); const blocked = await seedRun({ executionWorkspaceId: unavailable.id, title: "Plugin is unavailable after restart" }); const ready = await seedRun({ executionWorkspaceId: healthy.id, title: "Other result can recover" }); - const inspect = vi.spyOn(pluginWorkspaces, "inspectPluginWorkspace").mockRejectedValue(new Error(message)); + const inspect = vi.spyOn(pluginWorkspaces, "finalizePluginWorkspace").mockRejectedValue(new Error(message)); const code = message.startsWith("plugin_workspace_") ? message : "plugin_workspace_inspection_failed"; try { await expect(reconcileNativeFinalizations(db, [blocked.runId, ready.runId])).resolves.toBeDefined(); diff --git a/server/src/services/native-runtime/native-workspace-finalizer.ts b/server/src/services/native-runtime/native-workspace-finalizer.ts index 9e0220004c0..0607284b972 100644 --- a/server/src/services/native-runtime/native-workspace-finalizer.ts +++ b/server/src/services/native-runtime/native-workspace-finalizer.ts @@ -244,7 +244,7 @@ export async function resumeNativeWorkspaceFinalization(input: { let pluginWorkspace: Record | null; try { pluginWorkspace = workspace - ? await inspectPluginWorkspaceForFinalization(input.db, input.pluginWorkerManager, workspace, bound.agentId) + ? await inspectPluginWorkspaceForFinalization(input.db, input.pluginWorkerManager, workspace, bound.agentId, input.runId) : null; } catch (error) { await ownership.assertHeld(); diff --git a/server/src/services/plugin-workspaces.ts b/server/src/services/plugin-workspaces.ts index 9e97724144c..e2383dcfc74 100644 --- a/server/src/services/plugin-workspaces.ts +++ b/server/src/services/plugin-workspaces.ts @@ -90,6 +90,9 @@ export async function callWorkspaceProvider( const selectedProvider = await provider(db, workers, workspace.companyId, pluginId); if (input.operation.kind === "prepare" && input.operation.fresh && selectedProvider.manifestJson.workspaceProvider?.freshWorkspaceVersion !== 1) throw conflict("Fresh workspace preparation is unavailable"); + if (input.operation.kind === "finalize" && + (selectedProvider.manifestJson.workspaceProvider?.finalizationVersion !== 1 || !input.runId || !input.agentId)) + throw conflict("Workspace finalization is unavailable"); const generation = typeof workspace.metadata?.lifecycleGeneration === "number" ? workspace.metadata.lifecycleGeneration @@ -111,11 +114,12 @@ export async function callWorkspaceProvider( ? randomUUID() : createHash("sha256") .update( - JSON.stringify([workspace.id, operationGeneration, operation]), + JSON.stringify([workspace.id, operationGeneration, operation, + ...(operation.kind === "finalize" ? [input.runId, input.agentId] : [])]), ) .digest("hex"), generation: operationGeneration, - deadline: new Date(Date.now() + 300_000).toISOString(), + deadline: new Date(Date.now() + (operation.kind === "finalize" ? 30_000 : 300_000)).toISOString(), runId: input.runId ?? null, agentId: input.agentId ?? null, operation, @@ -126,7 +130,7 @@ export async function callWorkspaceProvider( pluginId, "workspaceOperation", params, - 310_000, + operation.kind === "finalize" ? 40_000 : 310_000, ); } catch { throw conflict( @@ -604,25 +608,42 @@ async function inspectBinding( return result; } -export async function inspectPluginWorkspace( - db: Db, - workers: WorkspaceWorkers | undefined, - workspace: Workspace, -) { +/** Reconcile durable lifecycle intent under the same lock for either read path. */ +async function withRetainedWorkspace(db: Db, workers: WorkspaceWorkers | undefined, + workspace: Workspace, operation: (workspace: Workspace) => Promise) { return withExecutionWorkspaceLifecycleLock(db, workspace.id, async () => { - // Keep recovery intent/results durable across an ambiguous provider response. - workspace = await reconcilePluginWorkspaceReservation( - db, - workers, - workspace, - ); + workspace = await reconcilePluginWorkspaceReservation(db, workers, workspace); if (workspace.metadata?.workspaceProviderRestoration) workspace = await finishPendingRestoration(db, workers, workspace); if (workspace.metadata?.workspaceProviderRemoval) workspace = await finishPendingRemoval(db, workers, workspace); - if (workspace.metadata?.workspaceProviderRemoved === true) - return removedInspection(); - return inspectBinding(db, workers, workspace); + return operation(workspace); + }); +} + +export async function inspectPluginWorkspace(db: Db, workers: WorkspaceWorkers | undefined, workspace: Workspace) { + return withRetainedWorkspace(db, workers, workspace, (current) => + current.metadata?.workspaceProviderRemoved === true + ? Promise.resolve(removedInspection()) : inspectBinding(db, workers, current)); +} + +/** Completion does not need the deletion-safety proof supplied by inspect. */ +export async function finalizePluginWorkspace(db: Db, workers: WorkspaceWorkers | undefined, + workspace: Workspace, context: { runId: string; agentId: string }) { + return withRetainedWorkspace(db, workers, workspace, async (current) => { + if (current.metadata?.workspaceProviderRemoved === true) + throw conflict("Workspace was removed before finalization"); + const binding = readWorkspaceProviderBinding(current.metadata); + if (!binding) throw conflict("Workspace storage ownership is missing"); + const selected = await provider(db, workers, current.companyId, binding.pluginId); + // Existing providers continue to use their established inspection contract. + if (selected.manifestJson.workspaceProvider?.finalizationVersion !== 1) + return inspectBinding(db, workers, current); + const result = await callWorkspaceProvider(db, workers, { + workspace: current, ...context, operation: { kind: "finalize", binding }, + }); + if (result.kind !== "finalized") throw conflict("Workspace finalization is incomplete"); + return result; }); }