Support provider-owned fresh retry workspaces (#17252)

## Thinking Path

> - Paperclip coordinates AI agents and retains their work across
failures.
> - A retry can require a new workspace while preserving the failed
attempt.
> - Core rejects this recovery strategy for provider-owned storage.
> - Providers should implement the required outcome using their own
storage design.
> - This pull request adds a semantic fresh-preparation contract and
delegates recovery preparation.
> - The benefit is that a supported provider can run a fresh retry
without losing prior work.

## Linked Issues or Issue Description

**What happened?**
A retry that selects `fresh_isolated_workspace` fails with
`fresh_workspace_storage_unsupported` for provider-owned storage.

**Expected behavior**
Core asks a capable provider to prepare new storage and attach verified
recovery files. The failed workspace remains intact.

**Steps to reproduce**
Run a task in a task-capable plugin environment. Let the task fail
without a verified workspace preservation receipt. Retry the task so
Core selects a fresh attempt.

**Paperclip version or commit**
6204239cf (rebased on current master).

**Deployment mode**
Self-hosted server with a plugin task environment and managed workspace
provider.

Search found no duplicate implementation. This branch includes the
default-provider changes from #17247 and preserves their source
selection behavior.

## What Changed

- Define an opt-in fresh-preparation contract with a stable attempt
identity and verified recovery file descriptors.
- Describe retained-storage and recovery-attachment guarantees without
requiring a Git storage layout.
- Reserve replacement task bindings with revision checks and keep the
failed workspace intact. Projectless and project-backed task
environments both replay their stable attempt reservation.
- Reconcile an interrupted preparation against its reserved workspace
and stable operation identity.
- Stage verified recovery archives at stable transfer paths. Delegate
attachment materialization to the provider.
- Preserve projectless attempt directory identities through later turns
and pin the intent into each execution lease.
- Protect fresh intent and projectless ownership from caller metadata
replacement or injection.

## Verification

- SDK workspace-provider tests: 4 passed.
- Targeted reservation and task-dispatch tests: 31 passed.
- Recovery transfer staging and lifecycle focus: 7 passed. The separate
cache-cleanup failure reconciliation case also passed.
- After rebasing on current master, 7 focused preparation and ownership
cases passed. These cover inherited provider defaults, fresh
preparation, stable projectless identity, and authorization denial.
- The focused metadata authority regression passed, including forged
intent, replacement, and null metadata patches.
- Three assembled interrupted task-reservation cases passed:
projectless, project with a saved source, and project with a newly
selected default source.
- Targeted tests use worktree SDK sources. No full local build or
typecheck ran.
- Local checks remain targeted as requested. Latest-head GitHub CI
passed: 54 successful checks, including typecheck, build, server suites,
Runner validation, and E2E shards; 2 unrelated Storybook checks were
skipped.
- Greptile reviewed commit 73595b0c61 at
5/5 with zero unresolved threads.
- Live VM E2E has not run.

## Risks

- Existing providers retain their existing behavior. Fresh attempts
require explicit version support.
- The companion provider implementation must materialize recovery
snapshots safely and preserve the old workspace.
- Fresh retries require the companion provider implementation. Providers
without the version marker fail closed.

## Model Used

- OpenAI GPT-6.1 (`gpt-6.1-sol`), reasoning and coding tools. The
runtime does not expose the context-window limit.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

server/src/services/plugin-workspaces.ts

@@ -1,4 +1,5 @@
 import { effectiveProjectWorkspacePolicy, isWorkspaceProvider } from "./project-workspace-provider.js";
+import { releaseProviderRecoveryTransfers, stageProviderFreshWorkspace } from "./provider-fresh-workspace.js";
 import { randomUUID, createHash } from "node:crypto";
 import { and, eq, sql } from "drizzle-orm";
 import {
@@ -20,6 +21,7 @@ import {
   workspaceProviderOperationSchema,
   type PluginWorkspaceOperation,
   type PluginWorkspaceParams,
+  type PluginFreshWorkspace,
 } from "@paperclipai/plugin-sdk";
 import type { PluginWorkerManager } from "./plugin-worker-manager.js";
 import { notifyTerminalWorkspaceWork } from "./terminal-workspace-work.js";
@@ -85,13 +87,20 @@ export async function callWorkspaceProvider(
   const pluginId = binding?.pluginId ?? input.pluginId;
   if (!pluginId || !workspace.projectId)
     throw conflict("Workspace storage ownership is missing");
-  await provider(db, workers, workspace.companyId, pluginId);
+  const selectedProvider = await provider(db, workers, workspace.companyId, pluginId);
+  if (input.operation.kind === "prepare" && input.operation.fresh && selectedProvider.manifestJson.workspaceProvider?.freshWorkspaceVersion !== 1)
+    throw conflict("Fresh workspace preparation is unavailable");
   const generation =
     typeof workspace.metadata?.lifecycleGeneration === "number"
       ? workspace.metadata.lifecycleGeneration
       : 0;
   const operationGeneration = generation + (input.advanceGeneration ? 1 : 0);
   const operation = workspaceProviderOperationSchema.parse(input.operation);
+  if (operation.kind === "prepare" && operation.fresh) {
+    if (!workspace.sourceIssueId) throw conflict("Fresh workspace task ownership is missing");
+    operation.fresh = await stageProviderFreshWorkspace({ db, companyId: workspace.companyId,
+      issueId: workspace.sourceIssueId, ...operation.fresh, existing: operation.fresh });
+  }
   const params: PluginWorkspaceParams = {
     pluginId,
     companyId: workspace.companyId,
@@ -167,9 +176,11 @@ export async function preparePluginWorkspace(
     branchMode?: "create" | "existing";
     branchName?: string | null;
     baseRef?: string | null;
+    fresh?: PluginFreshWorkspace;
+    expectedBindingRevision?: number;
   },
 ) {
-  const [existing] = input.existingWorkspaceId
+  const [previous] = input.existingWorkspaceId
     ? await db
         .select()
         .from(executionWorkspaces)
@@ -180,6 +191,13 @@ export async function preparePluginWorkspace(
           ),
         )
     : [];
+  const [reservation] = input.fresh ? await db.select().from(executionWorkspaces).where(and(
+    eq(executionWorkspaces.companyId, input.companyId),
+    eq(executionWorkspaces.sourceIssueId, input.issueId!),
+    sql`${executionWorkspaces.metadata}#>>'{workspaceProviderFresh,attemptId}' = ${input.fresh.attemptId}`,
+  )).limit(1) : [];
+  const existing = input.fresh ? reservation : previous;
+  const prior = existing ?? previous;
   if (
     existing?.metadata?.workspaceProviderRetired === true ||
     existing?.metadata?.workspaceProviderRemoval
@@ -187,9 +205,10 @@ export async function preparePluginWorkspace(
     throw conflict("Workspace storage is retired or awaiting cleanup");
   if (!existing && (!input.projectId || !input.projectWorkspaceId)) return null;
   const saved = readWorkspaceProviderBinding(existing?.metadata);
-  const mode = existing?.mode ?? input.mode,
+  const priorBinding = readWorkspaceProviderBinding(prior?.metadata);
+  const mode = input.fresh ? "isolated_workspace" : existing?.mode ?? input.mode,
     strategy = existing?.strategyType ?? input.strategy;
-  const projectId = saved?.projectId ?? input.projectId;
+  const projectId = priorBinding?.projectId ?? input.projectId;
   const [project] = projectId
     ? await db
         .select()
@@ -201,8 +220,8 @@ export async function preparePluginWorkspace(
           ),
         )
     : [];
-  if (existing && existing.providerType !== "plugin" && !saved && !existing.metadata?.workspaceProviderOwner) return null;
-  const owner = saved?.pluginId ?? existing?.metadata?.workspaceProviderOwner ??
+  if (prior && prior.providerType !== "plugin" && !priorBinding && !prior.metadata?.workspaceProviderOwner) return null;
+  const owner = priorBinding?.pluginId ?? prior?.metadata?.workspaceProviderOwner ??
     (!existing && input.selectedProviderPluginId !== undefined ? input.selectedProviderPluginId :
       !existing && project ? (await effectiveProjectWorkspacePolicy(db, input.companyId, project.executionWorkspacePolicy, false))?.storageProviderPluginId : null);
   const pluginId = typeof owner === "string" ? owner : null;
@@ -238,6 +257,8 @@ export async function preparePluginWorkspace(
     pluginId,
   );
   const declaration = selectedProvider.manifestJson.workspaceProvider!;
+  if (input.fresh && declaration.freshWorkspaceVersion !== 1)
+    throw conflict("A fresh attempt is unavailable for this workspace storage provider. The existing workspace is retained.", { reason: "fresh_workspace_storage_unsupported" });
   if (
     !declaration.modes.includes(mode as "isolated_workspace") ||
     !declaration.strategies.includes(strategy as "git_worktree")
@@ -302,7 +323,7 @@ export async function preparePluginWorkspace(
       ),
     );
   if (!source) throw conflict("Workspace repository source is unavailable");
-  const sourceInput = (existing?.metadata?.workspaceProviderSource as
+  const sourceInput = (prior?.metadata?.workspaceProviderSource as
     | Extract<PluginWorkspaceOperation, { kind: "prepare" }>["source"]
     | undefined) ?? {
     id: source.id,
@@ -322,7 +343,7 @@ export async function preparePluginWorkspace(
     repoRef: existing?.baseRef ?? source.repoRef,
     defaultRef: source.defaultRef,
   };
-  const branchName = existing
+  const branchName = input.fresh ? null : existing
     ? existing.branchName
     : (input.branchName ??
       (strategy === "git_worktree" ? `workspace-${randomUUID()}` : null));
@@ -363,7 +384,7 @@ export async function preparePluginWorkspace(
           ),
         )
         .for("update");
-      if (!task || task.executionWorkspaceId)
+      if (!task || (input.fresh ? task.executionWorkspaceId !== input.fresh.previousWorkspaceId || task.workspaceBindingRevision !== input.expectedBindingRevision : Boolean(task.executionWorkspaceId)))
         throw conflict("Task workspace binding changed; retry preparation");
       const id = randomUUID();
       await notifyTerminalWorkspaceWork(tx, `workspace:${id}`);
@@ -375,8 +396,8 @@ export async function preparePluginWorkspace(
           projectId: project.id,
           projectWorkspaceId: source.id,
           sourceIssueId: input.issueId,
-          mode: input.mode,
-          strategyType: input.strategy,
+          mode,
+          strategyType: strategy,
           name: branchName ?? source.name,
           providerType: "plugin",
           providerRef: null,
@@ -388,7 +409,8 @@ export async function preparePluginWorkspace(
           metadata: {
             workspaceProviderOwner: pluginId,
             workspaceProviderSource: sourceInput,
-            workspaceProviderBranchMode: input.branchMode ?? "create",
+            workspaceProviderBranchMode: input.fresh ? "create" : input.branchMode ?? "create",
+            ...(input.fresh ? { workspaceProviderFresh: input.fresh } : {}),
           },
         })
         .returning();
@@ -437,7 +459,7 @@ export async function preparePluginWorkspace(
         source: sourceInput,
         mode: workspace.mode as "isolated_workspace",
         strategy: workspace.strategyType as "git_worktree",
-        branchName,
+        branchName: workspace.metadata?.workspaceProviderFresh ? null : branchName,
         branchMode:
           (workspace.metadata?.workspaceProviderBranchMode as
             | "create"
@@ -447,6 +469,7 @@ export async function preparePluginWorkspace(
           "create",
         baseRef: workspace.baseRef,
         existing: saved,
+        ...(workspace.metadata?.workspaceProviderFresh ? { fresh: workspace.metadata.workspaceProviderFresh as PluginFreshWorkspace } : {}),
       },
     });
     if (result.kind !== "prepared")
@@ -522,10 +545,11 @@ async function reconcilePluginWorkspaceReservation(
     source: workspace.metadata.workspaceProviderSource,
     mode: workspace.mode,
     strategy: workspace.strategyType,
-    branchName: workspace.branchName,
+    branchName: workspace.metadata.workspaceProviderFresh ? null : workspace.branchName,
     branchMode: workspace.metadata.workspaceProviderBranchMode ?? "create",
     baseRef: workspace.baseRef,
     existing: null,
+    ...(workspace.metadata?.workspaceProviderFresh ? { fresh: workspace.metadata.workspaceProviderFresh as PluginFreshWorkspace } : {}),
   });
   const result = await callWorkspaceProvider(db, workers, {
     workspace,
@@ -701,6 +725,8 @@ async function finishPendingRemoval(
   });
   if (result.kind !== "removed")
     throw conflict("Workspace removal is incomplete");
+  if (workspace.metadata?.workspaceProviderFresh)
+    await releaseProviderRecoveryTransfers(workspace.companyId, workspace.metadata.workspaceProviderFresh as PluginFreshWorkspace);
   const [updated] = await db.transaction(async (tx) => {
     await notifyTerminalWorkspaceWork(tx, `workspace:${workspace.id}`);
     return tx
@@ -739,7 +765,11 @@ export async function removePluginWorkspace(
   workspace = await reconcilePluginWorkspaceReservation(db, workers, workspace);
   if (workspace.metadata?.workspaceProviderRestoration)
     throw conflict("Workspace restoration must be reconciled before removal");
-  if (workspace.metadata?.workspaceProviderRemoved === true) return;
+  if (workspace.metadata?.workspaceProviderRemoved === true) {
+    if (workspace.metadata?.workspaceProviderFresh)
+      await releaseProviderRecoveryTransfers(workspace.companyId, workspace.metadata.workspaceProviderFresh as PluginFreshWorkspace);
+    return;
+  }
   if (workspace.metadata?.workspaceProviderRemoval) {
     await finishPendingRemoval(db, workers, workspace);
     return;
@@ -855,10 +885,11 @@ export async function restorePluginWorkspace(
     source,
     mode: workspace.mode,
     strategy: workspace.strategyType,
-    branchName: workspace.branchName,
-    branchMode: "existing",
+    branchName: workspace.metadata?.workspaceProviderFresh ? null : workspace.branchName,
+    branchMode: workspace.metadata?.workspaceProviderFresh ? "create" : "existing",
     baseRef: workspace.baseRef,
     existing: binding,
+    ...(workspace.metadata?.workspaceProviderFresh ? { fresh: workspace.metadata.workspaceProviderFresh as PluginFreshWorkspace } : {}),
   });
   const [pending] = await db.transaction(async (tx) => {
     await notifyTerminalWorkspaceWork(tx, `workspace:${workspace.id}`);