Preserve remote work when required workspace restore fails (#15479)

Persist exact source-retention obligations before run finalization and protect them across cancellation, restart and task changes. Require board-authorized repair evidence without replaying old work or changing current task ownership, state or locks. Hide an unavailable retry action and document operator recovery and retention costs.

Validated with 674 scoped regressions, 224 combined integration tests, full typecheck/build, independent safety reviews and green CI with Greptile 5/5. No historical file recovery is claimed.

Co-Authored-By: Paperclip <noreply@paperclip.ing>

doc/workspace-restore-locks.md

@@ -54,3 +54,46 @@ is safe to remove from its age, an absent PID, or a successful task response.
 
 This change prevents new orphaned ownership. It cannot recover file changes
 that an earlier failed collection discarded.
+
+## Required remote restore failures
+
+When a legacy sandbox run cannot copy required workspace files back, the server
+keeps the original allocation for repair. It writes the failed run, the exact
+lease retention intent, and a board-owned repair hold in one transaction. The
+hold survives reassignment and conversation resets and coexists with newer
+recovery incidents; it never supersedes them.
+Cleanup stops that allocation and verifies its provider receipt. It does not
+use ordinary ephemeral destruction or return the source to warm reuse. Stop
+failures use the existing cleanup retry and backoff policy.
+
+The retained source uses `retain_on_failure`. A stopped source can still incur
+provider storage charges. It stays available until an operator recovers or
+explicitly discards it; elapsed time never proves that its files were copied.
+The task's `GET /api/issues/:id/recovery-actions` response lists these repair holds
+alongside any newer active incident, including the original lease IDs. Inspect
+those exact leases through `GET /api/environment-leases/:leaseId`,
+confirm that cleanup stopped them, recover the missing files, and record
+`workspaceRepairEvidence` through the existing recovery-action resolution API.
+Send `POST /api/issues/:id/recovery-actions/resolve` with the exact recovery action
+ID, `outcome: "restored"`, the task's **unchanged**
+`sourceIssueStatus`, and `executionReconciliation` for the recorded source run.
+Only a board operator can record this repair. It preserves the current task
+owner, conversation generation, status, and execution locks. It does not queue a
+continuation or replay the stopped run; after repair, any new work still follows
+the normal admission rules.
+The board currently shows repair guidance and an Inspect run link, not a repair
+form. Recording the decision is an operator API/runbook workflow; the recovery
+API response contains the exact action and lease IDs. Provider access may require
+an instance administrator or Cloud support. This change does not add a repair UI.
+Then use the provider's operator console to remove that exact retained
+allocation when its files are no longer needed. Verify its immutable provider
+allocation ID against the recorded lease before removal. The ordinary reusable
+sandbox cleanup does not select `retain_on_failure` sources; do not use
+environment deletion as proof that this allocation was removed. Keep the
+environment and its provider credentials until recovery is complete. Recording a
+repair decision does not automatically destroy it.
+
+A new message, Retry, or `/new` cannot clear this workspace repair hold. Ordinary
+local lock-timeout retries and runs without a required remote copy-back failure
+keep their existing policy. This change preserves future failures; it cannot
+recover files from an allocation that was already destroyed.

packages/shared/src/types/execution-projection.ts

@@ -11,6 +11,8 @@ export interface ExecutionBlocker {
   canContinue?: boolean;
   /** Explicit Retry candidate; the server rechecks stop proof and execution gates. */
   canRetry?: boolean;
+  /** Required workspace files must be recovered before another provider turn. */
+  workspaceRepairRequired?: boolean;
   savedMessageCount?: number;
 }
 

packages/shared/src/validators/issue.test.ts

@@ -314,6 +314,20 @@ describe("issue validators", () => {
     ).toBe(false);
   });
 
+  it.each(["backlog", "in_progress", "blocked", "cancelled"])("requires workspace evidence to record a repair at unchanged %s status", sourceIssueStatus => {
+    const input = { outcome: "restored", sourceIssueStatus };
+    expect(resolveIssueRecoveryActionSchema.safeParse(input).success).toBe(false);
+    expect(resolveIssueRecoveryActionSchema.safeParse({ ...input, executionReconciliation: {
+      runId: "11111111-1111-4111-8111-111111111111", providerStopped: true, actionOutcome: "mixed",
+      outcomeEvidence: "Observed and reconciled the source run action outcomes.",
+    } }).success).toBe(false);
+    expect(resolveIssueRecoveryActionSchema.safeParse({ ...input, executionReconciliation: {
+      runId: "11111111-1111-4111-8111-111111111111", providerStopped: true, actionOutcome: "mixed",
+      outcomeEvidence: "Observed and reconciled the source run action outcomes.",
+      workspaceRepairEvidence: "Copied the retained source files and verified the restored files.",
+    } }).success).toBe(true);
+  });
+
   it("allows cancelled recovery resolutions to atomically restore the source issue status", () => {
     expect(
       resolveIssueRecoveryActionSchema.parse({

packages/shared/src/validators/issue.ts

@@ -576,12 +576,15 @@ export const resolveIssueRecoveryActionSchema = z
       .optional(),
     actionId: z.string().guid().optional(),
     outcome: z.enum(RESOLVE_ISSUE_RECOVERY_ACTION_OUTCOMES),
-    sourceIssueStatus: z.enum(["todo", "done", "in_review", "blocked"]),
+    sourceIssueStatus: z.enum(ISSUE_STATUSES),
     resolutionNote: multilineTextSchema.optional().nullable(),
   })
   .strict()
   .superRefine((value, ctx) => {
     if (value.outcome === "restored") {
+      // A retained-source repair records evidence without changing task state.
+      // The route verifies the exact server-owned source and unchanged status.
+      if (value.executionReconciliation?.workspaceRepairEvidence) return;
       if (
         value.sourceIssueStatus !== "todo" &&
         value.sourceIssueStatus !== "done" &&

server/src/__tests__/heartbeat-process-recovery.test.ts

@@ -8,7 +8,7 @@ import * as runEvents from "../services/heartbeat-run-events.js";
 import { waitForPendingRunFailureReports } from "../services/run-failure-report.js";
 import { instanceSettingsService } from "../services/instance-settings.js";
 import { createHash, randomUUID } from "node:crypto";
-import { terminalizeLegacyExecution } from "../services/legacy-execution-recovery.js";
+import { recordLegacyWorkspaceRestoreFailure, terminalizeLegacyExecution } from "../services/legacy-execution-recovery.js";
 import { issueService } from "../services/issues.js";
 import { getExecutionBlocker } from "../services/execution-blocker.js";
 import { adapterExecutionControls, createAdapterExecutionControl } from "../services/adapter-execution-control.js";
@@ -7977,9 +7977,11 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
       try {
         await vi.waitFor(async () => {
           const [row] = await db.execute<{ count: number }>(sql`
-          select count(*)::int as count from pg_stat_activity
-          where datname = current_database() and ${pid} = any(pg_blocking_pids(pid))
-            and query ilike '%update%heartbeat_runs%'
+          select count(*)::int as count from pg_stat_activity activity
+          where datname = current_database() and ${pid} = any(pg_blocking_pids(activity.pid))
+            and wait_event_type = 'Lock'
+            and exists (select 1 from pg_locks locks where locks.pid = activity.pid
+              and locks.relation = 'heartbeat_runs'::regclass)
         `);
           expect(row!.count).toBeGreaterThan(0);
         });
@@ -8054,6 +8056,63 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
     }
   });
 
+  it("preserves a copy-back obligation recorded after Stop reads the running adapter", async () => {
+    const { runId, companyId, issueId } = await seedRunFixture({ runtimeMode: "legacy", adapterType: "codex_local" });
+    const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId));
+    const [environment] = await db.insert(environments).values({ name: `Stop restore race ${runId}`, driver: "sandbox" }).returning();
+    const [lease] = await db.insert(environmentLeases).values({ companyId, issueId, heartbeatRunId: runId,
+      environmentId: environment.id, provider: "daytona", providerLeaseId: randomUUID(), status: "active",
+      leasePolicy: "ephemeral", metadata: { driver: "sandbox" },
+    }).returning();
+    const control = createAdapterExecutionControl();
+    adapterExecutionControls.set(runId, control);
+    let releaseWrite!: () => void;
+    const writeGate = new Promise<void>(resolve => { releaseWrite = resolve; });
+    let reportWrite!: () => void;
+    const writeCaptured = new Promise<void>(resolve => { reportWrite = resolve; });
+    const originalUpdate = db.update.bind(db);
+    const writeSpy = vi.spyOn(db, "update").mockImplementationOnce((table) => {
+      const update = originalUpdate(table);
+      const originalSet = update.set.bind(update);
+      vi.spyOn(update, "set").mockImplementationOnce((value) => {
+        const query = originalSet(value);
+        const originalWhere = query.where.bind(query);
+        vi.spyOn(query, "where").mockImplementationOnce((condition) => {
+          reportWrite();
+          return writeGate.then(() => originalWhere(condition)) as unknown as ReturnType<typeof query.where>;
+        });
+        return query;
+      });
+      return update;
+    });
+    const heartbeat = heartbeatService(db);
+    const stopping = heartbeat.cancelRun(runId);
+    try {
+      await writeCaptured;
+      // This commits after Stop captured its old result, before Stop's write.
+      await recordLegacyWorkspaceRestoreFailure(db, run, { workspaceRestoreFailure: "restore_failed" });
+      releaseWrite();
+      await vi.waitFor(() => expect(control.controller.signal.aborted).toBe(true));
+      const [requested] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId));
+      expect(requested.resultJson).toMatchObject({ workspaceRestoreFailure: "restore_failed",
+        workspaceRestoreRecovery: { leaseIds: [lease.id] }, executionCancellation: { state: "requested" } });
+      expect((await db.select().from(environmentLeases).where(eq(environmentLeases.id, lease.id)))[0])
+        .toMatchObject({ leasePolicy: "retain_on_failure", status: "pending_cleanup" });
+      const cancelled = await terminalizeLegacyExecution({ db, run: requested, status: "cancelled", reconcileIfNeeded: true,
+        patch: { resultJson: { executionCancellation: { state: "acknowledged" } } } });
+      control.finish();
+      await expect(stopping).resolves.toMatchObject({ status: "cancelled" });
+      expect(cancelled?.resultJson).toMatchObject({ workspaceRestoreFailure: "restore_failed", workspaceRestoreRecovery: { leaseIds: [lease.id] } });
+      expect(await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, issueId!))).toHaveLength(1);
+    } finally {
+      releaseWrite();
+      writeSpy.mockRestore();
+      control.finish();
+      adapterExecutionControls.delete(runId);
+      await stopping.catch(() => undefined);
+    }
+  });
+
   it("clears the detached warning when the run reports activity again", async () => {
     const { runId } = await seedRunFixture({
       includeIssue: false,
@@ -8254,9 +8313,9 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
         expect(duplicateSettled).toBe(false);
         if (failure === "write") {
           const error = new Error("owned cancellation write unavailable");
-          writeSpy = adapterType === "codex_local"
-            ? vi.spyOn(db, "update").mockImplementationOnce(() => { throw error; })
-            : vi.spyOn(db, "transaction").mockRejectedValueOnce(error);
+          // Both conversation and process cancellation now finalize under the
+          // task/run transaction. Fail that write before either owner resolves.
+          writeSpy = vi.spyOn(db, "transaction").mockRejectedValueOnce(error);
         }
       } finally {
         releaseTermination();
@@ -8297,6 +8356,86 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
     },
   );
 
+  it("preserves exact restore sources through late process Stop metadata with an oversized result", async () => {
+    const actualProcess = await vi.importActual<typeof import("../adapters/process/execute.js")>("../adapters/process/execute.js");
+    const actualSupervisor = await vi.importActual<typeof import("../services/local-service-supervisor.js")>("../services/local-service-supervisor.js");
+    let reportReady!: () => void;
+    const ready = new Promise<void>(resolve => { reportReady = resolve; });
+    let releaseTermination!: () => void;
+    const terminationGate = new Promise<void>(resolve => { releaseTermination = resolve; });
+    mockAdapterExecute.mockImplementationOnce((async (input: unknown) => {
+      const context = input as Parameters<typeof actualProcess.execute>[0];
+      const result = await actualProcess.execute({ ...context, onLog: async (stream, text) => {
+        await context.onLog(stream, text);
+        if (text.includes("restore stop ready")) reportReady();
+      } });
+      return { ...result, resultJson: { ...result.resultJson, lateAdapterReceipt: true } };
+    }) as typeof mockAdapterExecute);
+    mockTerminateLocalService.mockImplementationOnce(async (...args) => {
+      await actualSupervisor.terminateLocalService(...args);
+      await terminationGate;
+    });
+    const { runId, agentId, companyId, issueId } = await seedRunFixture({
+      adapterType: "process", runtimeMode: "legacy", agentStatus: "idle", runStatus: "queued",
+    });
+    await db.update(agents).set({ adapterConfig: { command: process.execPath,
+      args: ["-e", "console.log('restore stop ready'); setInterval(() => {}, 1000)"], graceSec: 1,
+    } }).where(eq(agents.id, agentId));
+    const originalFactory = instructionWorkingCopies.agentInstructionWorkingCopyService;
+    let completionReached = false;
+    const factory = vi.spyOn(instructionWorkingCopies, "agentInstructionWorkingCopyService").mockImplementation((...args) => {
+      const service = originalFactory(...args);
+      return { ...service, release: async (...releaseArgs) => {
+        await service.release(...releaseArgs);
+        if (releaseArgs[1] === runId && !completionReached) {
+          completionReached = true;
+          // Host completion reaches its owned-Stop await in microtasks after
+          // instruction cleanup. Settle Stop on the next turn, without a delay.
+          setImmediate(releaseTermination);
+        }
+      } };
+    });
+    const heartbeat = heartbeatService(db);
+    let stopping: ReturnType<typeof heartbeat.cancelRun> | undefined;
+    try {
+      await heartbeat.resumeQueuedRuns();
+      await ready;
+      expect(runningProcesses.get(runId)?.child.pid).toBeTruthy();
+      expect(adapterExecutionControls.has(runId)).toBe(false);
+      const [environment] = await db.insert(environments).values({ name: `Late restore Stop ${runId}`, driver: "sandbox" }).returning();
+      const [lease] = await db.insert(environmentLeases).values({ companyId, issueId, heartbeatRunId: runId,
+        environmentId: environment.id, provider: "daytona", providerLeaseId: randomUUID(), status: "active",
+        leasePolicy: "ephemeral", metadata: { driver: "sandbox" },
+      }).returning();
+      // Incompressible fixture data exercises the database-size projection,
+      // which keeps the recovery schema but omits its internal source IDs.
+      const payload = Array.from({ length: 8192 }, () => randomUUID()).join("");
+      const [run] = await db.update(heartbeatRuns).set({ resultJson: { payload } }).where(eq(heartbeatRuns.id, runId)).returning();
+      await recordLegacyWorkspaceRestoreFailure(db, run, { workspaceRestoreFailure: "restore_failed" });
+      expect((await heartbeat.getRun(runId))?.resultJson?.workspaceRestoreRecovery).toEqual({ schema: "paperclip.workspace-restore-recovery.v1" });
+      stopping = heartbeat.cancelRun(runId, "Stopped by test operator");
+      await expect(stopping).resolves.toMatchObject({ status: "cancelled" });
+      await heartbeat.waitForRunExecutionDrain(runId);
+      const [finished] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId));
+      expect(completionReached).toBe(true);
+      expect(finished).toMatchObject({ status: "cancelled", resultJson: {
+        lateAdapterReceipt: true,
+        workspaceRestoreFailure: "restore_failed",
+        workspaceRestoreRecovery: { schema: "paperclip.workspace-restore-recovery.v1", leaseIds: [lease.id] },
+        cancellation: { expected: true },
+      } });
+      expect((await db.select().from(environmentLeases).where(eq(environmentLeases.id, lease.id)))[0])
+        .toMatchObject({ leasePolicy: "retain_on_failure", status: "pending_cleanup" });
+      expect(await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, issueId))).toHaveLength(1);
+    } finally {
+      releaseTermination();
+      await stopping?.catch(() => undefined);
+      if (runningProcesses.has(runId)) await heartbeat.cancelRun(runId).catch(() => undefined);
+      await heartbeat.waitForRunExecutionDrain(runId);
+      factory.mockRestore();
+    }
+  });
+
   it.each([
     {
       mode: "clean exit",

server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts

@@ -359,6 +359,9 @@ function createRunContextDb(
       orderBy: vi.fn(async () => []),
       limit: vi.fn(() => ({
         then: async (resolve: (limitedRows: unknown[]) => unknown) => resolve(await rowsForSelection(selection, chatBindingQuery, settledRecoveryQuery)),
+        for: vi.fn(() => ({
+          then: async (resolve: (lockedRows: unknown[]) => unknown) => resolve(await rowsForSelection(selection, chatBindingQuery, settledRecoveryQuery)),
+        })),
       })),
       for: vi.fn(() => ({
         then: async (resolve: (selectedRows: unknown[]) => unknown) => resolve(await rowsForSelection(selection, chatBindingQuery, settledRecoveryQuery)),

server/src/__tests__/legacy-workspace-restore-recovery.test.ts

@@ -0,0 +1,229 @@
+import { randomUUID } from "node:crypto";
+import { and, eq, sql } from "drizzle-orm";
+import { afterAll, beforeAll, describe, expect, it } from "vitest";
+import { agents, companies, createDb, environmentLeases, environments, heartbeatRuns, issueRecoveryActions, issues } from "@paperclipai/db";
+import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
+import { recordLegacyWorkspaceRestoreFailure, legacyExecutionNeedsReconciliationWithEvidence, terminalizeLegacyExecution } from "../services/legacy-execution-recovery.js";
+import { hasConversationContinuationPolicy, conversationRecoveryActionPredicate } from "../services/conversation-continuation.js";
+import { preserveWorkspaceRestoreRecoveryMetadataSql } from "../services/legacy-workspace-restore-recovery.js";
+import { settleStopOnlyCleanup } from "../services/sandbox-stop-and-retain.js";
+import { hasRequiredWorkspaceRecovery } from "../services/workspace-restore-recovery-state.js";
+
+const externalTestDatabaseUrl = process.env.PAPERCLIP_TEST_DATABASE_URL?.trim();
+const support = externalTestDatabaseUrl ? { supported: true } : await getEmbeddedPostgresTestSupport();
+(support.supported ? describe : describe.skip)("legacy remote workspace recovery", () => {
+  let db: ReturnType<typeof createDb>;
+  let temporary: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
+  beforeAll(async () => { if (externalTestDatabaseUrl) db = createDb(externalTestDatabaseUrl);
+    else { temporary = await startEmbeddedPostgresTestDatabase("legacy-workspace-recovery-"); db = createDb(temporary.connectionString); } }, 30_000);
+  afterAll(async () => { await db?.$client.end(); await temporary?.cleanup(); });
+  async function seed(options: { local?: boolean; taskDone?: boolean; reusable?: boolean } = {}) {
+    const companyId = randomUUID(), agentId = randomUUID(), issueId = randomUUID(), runId = randomUUID();
+    await db.insert(companies).values({ id: companyId, name: "Workspace recovery", issuePrefix: companyId.slice(0, 8) });
+    await db.insert(agents).values({ id: agentId, companyId, name: "Engineer", role: "engineer", adapterType: "codex_local" });
+    await db.insert(issues).values({ id: issueId, companyId, title: "Restore files", status: options.taskDone ? "done" : "in_progress", assigneeAgentId: agentId });
+    const [run] = await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, status: "running", runtimeMode: "legacy", contextSnapshot: { issueId } }).returning();
+    if (options.local) await db.insert(environments).values({ name: "Local", driver: "local" }).onConflictDoNothing();
+    const [environment] = options.local ? await db.select().from(environments).where(eq(environments.driver, "local"))
+      : await db.insert(environments).values({ name: `Isolated test sandbox ${runId}`, driver: "sandbox" }).returning();
+    const [lease] = await db.insert(environmentLeases).values({ companyId, issueId, heartbeatRunId: runId, environmentId: environment.id,
+      status: "active", leasePolicy: options.reusable ? "reuse_by_environment" : "ephemeral",
+      provider: options.local ? "local" : "daytona", providerLeaseId: randomUUID(),
+      metadata: { driver: options.local ? "local" : "sandbox", pluginId: randomUUID(), sandboxProviderPlugin: !options.local },
+    }).returning();
+    return { companyId, runId, issueId, run, lease };
+  }
+  const patch = { errorCode: "workspace_restore_failed", finishedAt: new Date(), resultJson: {
+    workspaceRestoreFailure: "restore_failed", conversationContinuation: "continue_conversation_v1",
+  } };
+  const readLease = async (id: string) => (await db.select().from(environmentLeases).where(eq(environmentLeases.id, id)))[0];
+
+  it.each([false, true])("persists terminal status, stop-only source and board action atomically (task done=%s)", async taskDone => {
+    const f = await seed({ taskDone, reusable: true });
+    expect(await legacyExecutionNeedsReconciliationWithEvidence(db, { ...f.run, status: "failed", ...patch })).toBe(true);
+    const failed = await terminalizeLegacyExecution({ db, run: f.run, status: "failed", patch });
+    expect(failed?.status).toBe("failed");
+    expect(failed?.resultJson?.workspaceRestoreRecovery).toMatchObject({ leaseIds: [f.lease.id] });
+    expect(hasConversationContinuationPolicy(failed?.resultJson)).toBe(false);
+    const lease = await readLease(f.lease.id);
+    expect(lease).toMatchObject({ status: "pending_cleanup", leasePolicy: "retain_on_failure", metadata: {
+      sandboxStopAndRetain: { companyId: f.companyId, runId: f.runId, leaseId: f.lease.id, providerLeaseId: f.lease.providerLeaseId },
+      pendingCleanupInFlight: false,
+    } });
+    const [action] = await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, f.issueId));
+    expect(action).toMatchObject({ ownerType: "board", evidence: { workspaceRestoreRecovery: { leaseIds: [f.lease.id] } } });
+    expect(await db.select().from(issueRecoveryActions).where(and(eq(issueRecoveryActions.id, action.id), conversationRecoveryActionPredicate()))).toEqual([]);
+    // A new controller uses the durable stop-only intent. It cannot turn a
+    // successful stop into proof that the workspace files were restored.
+    const receipt = { schema: "paperclip.remote-termination.v1", runId: f.runId, companyId: f.companyId,
+      leaseId: lease.id, provider: lease.provider, providerLeaseId: lease.providerLeaseId, state: "stopped", confirmedAt: new Date().toISOString() };
+    await settleStopOnlyCleanup(db, lease, { attemptId: String(lease.metadata?.pendingCleanupAttemptId), receipt });
+    expect(await readLease(lease.id)).toMatchObject({ status: "released", leasePolicy: "retain_on_failure", metadata: { remoteExecutionTermination: { state: "stopped" } } });
+    expect(hasRequiredWorkspaceRecovery(failed?.resultJson)).toBe(true);
+    expect(await legacyExecutionNeedsReconciliationWithEvidence(db, failed!)).toBe(true);
+    // A late adapter metadata write cannot erase the source pin when its
+    // in-memory result predates the terminal transaction.
+    expect(await legacyExecutionNeedsReconciliationWithEvidence(db, { ...failed!, resultJson: patch.resultJson })).toBe(true);
+    const repeated = await terminalizeLegacyExecution({ db, run: failed!, status: "failed", patch });
+    expect(repeated?.resultJson?.workspaceRestoreRecovery).toMatchObject({ leaseIds: [f.lease.id] });
+    expect((await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, f.issueId)))).toHaveLength(1);
+  });
+
+  it("leaves local lock retry and successful remote cleanup policy unchanged", async () => {
+    const local = await seed({ local: true });
+    const failure = { ...patch, resultJson: { ...patch.resultJson, workspaceRestoreFailure: "restore_lock_timeout" } };
+    expect(await legacyExecutionNeedsReconciliationWithEvidence(db, { ...local.run, status: "failed", ...failure })).toBe(false);
+    expect(await readLease(local.lease.id)).toMatchObject({ status: "active", leasePolicy: "ephemeral" });
+    const remote = await seed();
+    expect(await legacyExecutionNeedsReconciliationWithEvidence(db, { ...remote.run, status: "succeeded", resultJson: {} })).toBe(false);
+    expect(await readLease(remote.lease.id)).toMatchObject({ status: "active", leasePolicy: "ephemeral" });
+  });
+
+  it("does not retain another run's lease or revive an already released source", async () => {
+    const f = await seed(), other = await seed();
+    await db.update(environmentLeases).set({ status: "released", releasedAt: new Date() }).where(eq(environmentLeases.id, f.lease.id));
+    expect(await legacyExecutionNeedsReconciliationWithEvidence(db, { ...f.run, status: "failed", ...patch })).toBe(false);
+    await terminalizeLegacyExecution({ db, run: f.run, status: "failed", patch });
+    expect(await readLease(f.lease.id)).toMatchObject({ status: "released", leasePolicy: "ephemeral" });
+    expect(await readLease(other.lease.id)).toMatchObject({ status: "active", leasePolicy: "ephemeral" });
+  });
+
+  it("keeps a competing physical owner out of the stop receipt", async () => {
+    const f = await seed(), other = await seed();
+    await db.update(environmentLeases).set({ providerLeaseId: f.lease.providerLeaseId }).where(eq(environmentLeases.id, other.lease.id));
+    await terminalizeLegacyExecution({ db, run: f.run, status: "failed", patch });
+    expect(await readLease(f.lease.id)).toMatchObject({ status: "pending_cleanup", metadata: { sandboxStopAndRetain: { runId: f.runId } } });
+    expect(await readLease(other.lease.id)).toMatchObject({ status: "active", heartbeatRunId: other.runId });
+  });
+
+
+  it("keeps the original source if host finalization fails after adapter settlement", async () => {
+    const f = await seed();
+    await recordLegacyWorkspaceRestoreFailure(db, f.run, patch.resultJson);
+    const [current] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.runId));
+    expect(current.status).toBe("running");
+    expect(await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, f.issueId))).toHaveLength(1);
+    expect(hasRequiredWorkspaceRecovery(current.resultJson)).toBe(true);
+    expect(await readLease(f.lease.id)).toMatchObject({ status: "pending_cleanup", leasePolicy: "retain_on_failure" });
+    // A restart's process-loss finalization sees the durable obligation, even
+    // when its own result predates the restore failure.
+    const stale = { stopReason: "process_lost" };
+    const failed = await terminalizeLegacyExecution({ db, run: f.run, status: "failed", reconcileIfNeeded: true, patch: { errorCode: "process_lost", resultJson: stale } });
+    expect(hasRequiredWorkspaceRecovery(failed?.resultJson)).toBe(true);
+    expect(await readLease(f.lease.id)).toMatchObject({ status: "pending_cleanup", leasePolicy: "retain_on_failure" });
+    expect(hasConversationContinuationPolicy(failed?.resultJson)).toBe(false);
+  });
+
+
+  it("records a repair action when cancellation wins before adapter copy-back reports failure", async () => {
+    const f = await seed();
+    await db.update(heartbeatRuns).set({ status: "cancelled", finishedAt: new Date(), resultJson: { executionCancellation: { state: "acknowledged", proof: "provider_termination_receipt" } } }).where(eq(heartbeatRuns.id, f.runId));
+    await recordLegacyWorkspaceRestoreFailure(db, f.run, patch.resultJson);
+    const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.runId));
+    expect(run.status).toBe("cancelled");
+    expect(run.resultJson?.executionCancellation).toMatchObject({ state: "acknowledged", proof: "provider_termination_receipt" });
+    expect(hasRequiredWorkspaceRecovery(run.resultJson)).toBe(true);
+    expect(await readLease(f.lease.id)).toMatchObject({ status: "pending_cleanup", leasePolicy: "retain_on_failure" });
+    expect(await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, f.issueId))).toHaveLength(1);
+  });
+
+  it.each(["write_condition", "native_takeover"])("preserves the existing terminal-write fence for %s", async scenario => {
+    const f = await seed();
+    if (scenario === "native_takeover") await db.update(heartbeatRuns).set({ runtimeMode: "native" }).where(eq(heartbeatRuns.id, f.runId));
+    const failed = await terminalizeLegacyExecution({ db, run: f.run, status: "failed", patch,
+      reconcileIfNeeded: true, ...(scenario === "write_condition" ? { writeConditions: [sql`false`] } : {}),
+    });
+    expect(failed).toBeNull();
+    expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.runId)))[0].status).toBe("running");
+    expect(await readLease(f.lease.id)).toMatchObject({ status: "active", leasePolicy: "ephemeral" });
+    expect(await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, f.issueId))).toEqual([]);
+  });
+
+  it.each([false, true])("keeps exact retained source IDs through a projected metadata update (merge=%s)", async mergeCurrent => {
+    const f = await seed();
+    await recordLegacyWorkspaceRestoreFailure(db, f.run, patch.resultJson);
+    const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.runId));
+    const projection = { workspaceRestoreFailure: "restore_failed", workspaceRestoreRecovery: { schema: "paperclip.workspace-restore-recovery.v1" }, presentationDecision: "updated" };
+    const [updated] = await db.update(heartbeatRuns).set({ resultJson: preserveWorkspaceRestoreRecoveryMetadataSql(projection, mergeCurrent) })
+      .where(eq(heartbeatRuns.id, f.runId)).returning();
+    expect(updated.resultJson).toMatchObject({ presentationDecision: "updated", workspaceRestoreRecovery: run.resultJson!.workspaceRestoreRecovery });
+    expect(updated.resultJson?.workspaceRestoreFailure).toBe("restore_failed");
+  });
+
+  it("keeps ordinary metadata replacement and null semantics without a repair marker", async () => {
+    const f = await seed();
+    await db.update(heartbeatRuns).set({ resultJson: { oldValue: true } }).where(eq(heartbeatRuns.id, f.runId));
+    const [replaced] = await db.update(heartbeatRuns).set({ resultJson: preserveWorkspaceRestoreRecoveryMetadataSql({ newValue: true }) })
+      .where(eq(heartbeatRuns.id, f.runId)).returning();
+    expect(replaced.resultJson).toEqual({ newValue: true });
+    const [cleared] = await db.update(heartbeatRuns).set({ resultJson: preserveWorkspaceRestoreRecoveryMetadataSql(null) })
+      .where(eq(heartbeatRuns.id, f.runId)).returning();
+    expect(cleared.resultJson).toBeNull();
+  });
+
+  it("rolls back terminal status and retention together on a failed transaction", async () => {
+    const f = await seed();
+    await expect(db.transaction(async tx => { await terminalizeLegacyExecution({ db: tx as unknown as typeof db, run: f.run, status: "failed", patch }); throw new Error("outer transaction failed"); })).rejects.toThrow("outer transaction failed");
+    expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.runId)))[0].status).toBe("running");
+    expect(await readLease(f.lease.id)).toMatchObject({ status: "active", leasePolicy: "ephemeral" });
+    expect(await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, f.issueId))).toEqual([]);
+  });  it.each(["conversation_reset", "new_owner"])("late restore remains repair-blocked after %s", async scenario => {
+    const f = await seed();
+    await db.update(heartbeatRuns).set({ status: "cancelled", finishedAt: new Date(),
+      contextSnapshot: { issueId: f.issueId, conversationSessionGeneration: 0 },
+      resultJson: { executionCancellation: { state: "acknowledged" }, conversationContinuation: "continue_conversation_v1" },
+    }).where(eq(heartbeatRuns.id, f.runId));
+    if (scenario === "conversation_reset") await db.update(issues).set({ conversationAgentId: f.run.agentId, conversationUserId: "board-review-fixture", conversationState: "active",
+      conversationSessionGeneration: scenario === "conversation_reset" ? 1 : 0,
+    }).where(eq(issues.id, f.issueId));
+    if (scenario === "new_owner") {
+      const id = randomUUID();
+      await db.insert(agents).values({ id, companyId: f.companyId, name: "New owner", role: "engineer", adapterType: "codex_local" });
+      await db.update(issues).set({ assigneeAgentId: id }).where(eq(issues.id, f.issueId));
+    }
+    await recordLegacyWorkspaceRestoreFailure(db, f.run, patch.resultJson);
+    const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.runId));
+    expect(hasRequiredWorkspaceRecovery(run.resultJson)).toBe(true);
+    const lease = await readLease(f.lease.id);
+    expect(lease).toMatchObject({ status: "pending_cleanup", leasePolicy: "retain_on_failure" });
+    await settleStopOnlyCleanup(db, lease, { attemptId: String(lease.metadata?.pendingCleanupAttemptId),
+      receipt: { providerLeaseId: lease.providerLeaseId, state: "stopped" } });
+    const { getExecutionBlocker } = await import("../services/execution-blocker.js");
+    expect(await getExecutionBlocker(db, f.companyId, f.issueId)).toMatchObject({
+      recoveryActionId: expect.any(String), canRetry: false, canContinue: false,
+    });
+    expect(await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, f.issueId)))
+      .toHaveLength(1);
+  });
+
+  it("keeps a late source hold alongside a newer incident without replacing its execution", async () => {
+    const f = await seed(), newerRunId = randomUUID();
+    await db.insert(heartbeatRuns).values({ id: newerRunId, companyId: f.companyId, agentId: f.run.agentId,
+      status: "running", runtimeMode: "legacy", contextSnapshot: { issueId: f.issueId } });
+    await db.update(issues).set({ executionRunId: newerRunId, checkoutRunId: newerRunId }).where(eq(issues.id, f.issueId));
+    const [newerAction] = await db.insert(issueRecoveryActions).values({ companyId: f.companyId, sourceIssueId: f.issueId,
+      kind: "active_run_watchdog", status: "active", cause: "another_incident", fingerprint: `newer:${newerRunId}`,
+      evidence: { runId: newerRunId }, nextAction: "Inspect the current incident." }).returning();
+    const failed = await terminalizeLegacyExecution({ db, run: f.run, status: "failed", patch });
+    await terminalizeLegacyExecution({ db, run: failed!, status: "failed", patch });
+    const [task] = await db.select().from(issues).where(eq(issues.id, f.issueId));
+    expect(task).toMatchObject({ status: "in_progress", executionRunId: newerRunId, checkoutRunId: newerRunId });
+    const actions = await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, f.issueId));
+    expect(actions).toHaveLength(2);
+    expect(actions.find(a => a.id === newerAction.id)).toEqual(newerAction);
+    const source = actions.find(a => a.evidence.runId === f.runId)!;
+    expect(source).toMatchObject({ status: "resolved", outcome: "blocked", ownerType: "board",
+      returnOwnerAgentId: null, attemptCount: 1, evidence: { automaticRecovery: { replay: "blocked" } } });
+    const { markExecutionReconciliation } = await import("../services/execution-recovery-resolution.js");
+    await markExecutionReconciliation(db, source, { runId: f.runId, providerStopped: true, actionOutcome: "mixed",
+      outcomeEvidence: "The original provider's actions were inspected.",
+      workspaceRepairEvidence: "The exact retained workspace was recovered and verified.",
+    }, "board-test", undefined, { workspaceRepairOnly: true });
+    await terminalizeLegacyExecution({ db, run: failed!, status: "failed", patch });
+    const [repaired] = await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, source.id));
+    expect(repaired.evidence.automaticRecovery).toBeUndefined();
+    expect(repaired.evidence.continuationDelivery).toBe("not_requested");
+    expect(repaired.evidence.executionReconciliation).toMatchObject({ runId: f.runId });
+  });
+
+});

server/src/__tests__/workspace-restore-repair-route.test.ts

@@ -0,0 +1,235 @@
+import { randomUUID } from "node:crypto";
+import express from "express";
+import request from "supertest";
+import { and, eq, inArray } from "drizzle-orm";
+import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
+import {
+  activityLog, agents, agentWakeupRequests, companies, createDb,
+  environmentLeases, heartbeatRuns, issueRecoveryActions, issues,
+} from "@paperclipai/db";
+import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
+import { errorHandler } from "../middleware/index.js";
+import { issueRoutes } from "../routes/issues.js";
+import { getExecutionBlocker } from "../services/execution-blocker.js";
+import { remoteTerminationReceipt } from "../services/remote-execution-termination.js";
+import { LEGACY_WORKSPACE_RECOVERY_SCHEMA } from "../services/workspace-restore-recovery-state.js";
+
+const external = process.env.PAPERCLIP_TEST_DATABASE_URL?.trim();
+const support = external ? { supported: true } : await getEmbeddedPostgresTestSupport();
+
+(support.supported ? describe : describe.skip)("retained workspace repair route", () => {
+  let db: ReturnType<typeof createDb>;
+  let temporary: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | undefined;
+  beforeAll(async () => {
+    if (external) db = createDb(external);
+    else {
+      temporary = await startEmbeddedPostgresTestDatabase("workspace-restore-repair-route-");
+      db = createDb(temporary.connectionString);
+    }
+  }, 30_000);
+  afterAll(async () => { await db?.$client.end(); await temporary?.cleanup(); });
+
+  async function seed(options: { newerAction?: boolean; conversationReset?: boolean; noCurrentOwner?: boolean } = {}) {
+    const companyId = randomUUID(), originalAgentId = randomUUID(), currentAgentId = randomUUID();
+    const issueId = randomUUID(), sourceRunId = randomUUID(), currentRunId = randomUUID(), leaseId = randomUUID();
+    const actionId = randomUUID();
+    await db.insert(companies).values({ id: companyId, name: "Workspace repair API", issuePrefix: `W${companyId.slice(0, 7)}` });
+    await db.insert(agents).values([
+      { id: originalAgentId, companyId, name: "Original worker", role: "engineer", adapterType: "codex_local" },
+      { id: currentAgentId, companyId, name: "Current worker", role: "engineer", adapterType: "codex_local" },
+    ]);
+    const ownerId = options.conversationReset ? originalAgentId : currentAgentId;
+    await db.insert(issues).values({ id: issueId, companyId, title: "Preserve changed task", status: "in_progress",
+      assigneeAgentId: options.noCurrentOwner ? null : ownerId,
+      ...(options.conversationReset ? { conversationAgentId: originalAgentId, conversationUserId: "fixture-board",
+        conversationState: "active", conversationSessionGeneration: 2 } : {}),
+    });
+    const workspaceRestoreRecovery = { schema: LEGACY_WORKSPACE_RECOVERY_SCHEMA, leaseIds: [leaseId] };
+    await db.insert(heartbeatRuns).values([
+      { id: sourceRunId, companyId, agentId: originalAgentId, runtimeMode: "legacy", status: "cancelled",
+        finishedAt: new Date("2000-01-01T00:00:00Z"), contextSnapshot: { issueId, conversationSessionGeneration: 1 },
+        runnerProfileJson: { adapterDispatch: { adapterType: "codex_local" } },
+        resultJson: { workspaceRestoreFailure: "restore_failed", workspaceRestoreRecovery,
+          conversationContinuation: "continue_conversation_v1" } },
+      { id: currentRunId, companyId, agentId: ownerId, runtimeMode: "legacy", status: "running",
+        contextSnapshot: { issueId, conversationSessionGeneration: 2 } },
+    ]);
+    await db.update(issues).set({ executionRunId: currentRunId, checkoutRunId: currentRunId,
+      executionLockedAt: new Date("2000-01-01T00:00:00Z"), executionAgentNameKey: "current-worker" })
+      .where(eq(issues.id, issueId));
+    const identity = { id: leaseId, companyId, heartbeatRunId: sourceRunId, provider: "daytona", providerLeaseId: `fixture-${leaseId}` };
+    await db.insert(environmentLeases).values({ ...identity, issueId, status: "released", leasePolicy: "retain_on_failure",
+      releasedAt: new Date(), cleanupStatus: "success", metadata: {
+        remoteExecutionTermination: remoteTerminationReceipt(identity, { providerLeaseId: identity.providerLeaseId, state: "stopped" }),
+      } });
+    await db.insert(issueRecoveryActions).values({ id: actionId, companyId, sourceIssueId: issueId,
+      kind: "active_run_watchdog", cause: "legacy_execution_requires_reconciliation", ownerType: "board",
+      returnOwnerAgentId: originalAgentId, fingerprint: `legacy-execution:${sourceRunId}`,
+      status: "resolved", outcome: "blocked", nextAction: "Repair the retained workspace before continuing.",
+      evidence: { runId: sourceRunId, workspaceRestoreFailure: "restore_failed", workspaceRestoreRecovery,
+        automaticRecovery: { replay: "blocked" } },
+    });
+    const newerId = options.newerAction ? randomUUID() : null;
+    if (newerId) await db.insert(issueRecoveryActions).values({ id: newerId, companyId, sourceIssueId: issueId,
+      kind: "configuration_validation", cause: "configuration_incomplete", ownerType: "board", status: "active",
+      fingerprint: `newer:${currentRunId}`, nextAction: "Review the current configuration.", evidence: { runId: currentRunId } });
+    const [issueBefore] = await db.select().from(issues).where(eq(issues.id, issueId));
+    const [leaseBefore] = await db.select().from(environmentLeases).where(eq(environmentLeases.id, leaseId));
+    const body = { actionId, outcome: "restored", sourceIssueStatus: issueBefore.status,
+      executionReconciliation: { runId: sourceRunId, providerStopped: true, actionOutcome: "mixed",
+        outcomeEvidence: "Reviewed the original action receipts without replaying any previous work.",
+        workspaceRepairEvidence: "Recovered and verified the required files from the exact retained sandbox." } };
+    return { companyId, issueId, sourceRunId, currentRunId, originalAgentId, currentAgentId, ownerId,
+      leaseId, actionId, newerId, issueBefore, leaseBefore, body };
+  }
+
+  function appFor(actor: Record<string, unknown> = { type: "board", source: "local_implicit" }) {
+    const wake = vi.fn(async () => null);
+    const app = express();
+    app.use(express.json());
+    app.use((req, _res, next) => { (req as unknown as { actor: unknown }).actor = actor; next(); });
+    app.use("/api", issueRoutes(db, {} as Parameters<typeof issueRoutes>[1], { recoveryActionEnqueueWakeup: wake }));
+    app.use(errorHandler);
+    return { app, wake };
+  }
+
+  async function assertSourceUnchanged(f: Awaited<ReturnType<typeof seed>>) {
+    expect((await db.select().from(issues).where(eq(issues.id, f.issueId)))[0]).toEqual(f.issueBefore);
+    expect((await db.select().from(environmentLeases).where(eq(environmentLeases.id, f.leaseId)))[0]).toEqual(f.leaseBefore);
+    expect(await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.companyId, f.companyId))).toEqual([]);
+    expect(await db.select().from(heartbeatRuns).where(inArray(heartbeatRuns.id, [f.sourceRunId, f.currentRunId]))).toHaveLength(2);
+  }
+
+  it.each(["reassigned", "conversation_reset", "unassigned"])("records board repair after %s without restarting or changing the current task", async scenario => {
+    const f = await seed({ conversationReset: scenario === "conversation_reset", noCurrentOwner: scenario === "unassigned" });
+    const { app, wake } = appFor();
+    expect(await getExecutionBlocker(db, f.companyId, f.issueId)).not.toBeNull();
+    await request(app).post(`/api/issues/${f.issueId}/recovery-actions/resolve`).send(f.body).expect(200);
+    await assertSourceUnchanged(f);
+    expect(wake).not.toHaveBeenCalled();
+    const [action] = await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, f.actionId));
+    expect(action.evidence).toMatchObject({ workspaceRestoreRecovery: { leaseIds: [f.leaseId] },
+      executionReconciliation: { runId: f.sourceRunId, workspaceRepairEvidence: f.body.executionReconciliation.workspaceRepairEvidence } });
+    expect(action.evidence.automaticRecovery).toBeUndefined();
+    expect(action.evidence.continuationDelivery).not.toBe("pending");
+    expect(await getExecutionBlocker(db, f.companyId, f.issueId)).toBeNull();
+    const auditBefore = await db.select().from(activityLog).where(eq(activityLog.companyId, f.companyId));
+    expect(auditBefore.length).toBeGreaterThan(0);
+    await request(app).post(`/api/issues/${f.issueId}/recovery-actions/resolve`).send(f.body).expect(200);
+    expect((await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, f.actionId)))[0]).toEqual(action);
+    expect(await db.select().from(activityLog).where(eq(activityLog.companyId, f.companyId))).toEqual(auditBefore);
+    await assertSourceUnchanged(f);
+    expect(wake).not.toHaveBeenCalled();
+  });
+
+  it.each(["blocked", "done", "cancelled", "backlog"])("records repair without changing the current %s task", async status => {
+    const f = await seed();
+    await db.update(heartbeatRuns).set({ status: "succeeded", finishedAt: new Date() }).where(eq(heartbeatRuns.id, f.currentRunId));
+    const [before] = await db.update(issues).set({ status, executionRunId: null, checkoutRunId: null,
+      executionLockedAt: null, executionAgentNameKey: null }).where(eq(issues.id, f.issueId)).returning();
+    const { app, wake } = appFor();
+    await request(app).post(`/api/issues/${f.issueId}/recovery-actions/resolve`)
+      .send({ ...f.body, sourceIssueStatus: status }).expect(200);
+    await assertSourceUnchanged({ ...f, issueBefore: before });
+    expect(wake).not.toHaveBeenCalled();
+  });
+
+  it("repairs only the recorded source while a newer active recovery action remains unchanged", async () => {
+    const f = await seed({ newerAction: true });
+    const [newerBefore] = await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, f.newerId!));
+    const { app, wake } = appFor();
+    const before = await request(app).get(`/api/issues/${f.issueId}/recovery-actions`).expect(200);
+    expect(before.body.active).toMatchObject({ id: f.newerId });
+    expect(before.body.actions).toEqual(expect.arrayContaining([
+      expect.objectContaining({ id: f.actionId, evidence: expect.objectContaining({
+        workspaceRestoreRecovery: { schema: LEGACY_WORKSPACE_RECOVERY_SCHEMA, leaseIds: [f.leaseId] },
+      }) }), expect.objectContaining({ id: f.newerId }),
+    ]));
+    const response = await request(app).post(`/api/issues/${f.issueId}/recovery-actions/resolve`).send(f.body).expect(200);
+    expect(response.body.issue.activeRecoveryAction).toMatchObject({ id: f.newerId });
+    expect((await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, f.newerId!)))[0]).toEqual(newerBefore);
+    const repeated = await request(app).post(`/api/issues/${f.issueId}/recovery-actions/resolve`).send(f.body).expect(200);
+    expect(repeated.body.issue.activeRecoveryAction).toMatchObject({ id: f.newerId });
+    const after = await request(app).get(`/api/issues/${f.issueId}/recovery-actions`).expect(200);
+    expect(after.body.active).toMatchObject({ id: f.newerId });
+    expect(after.body.actions.map((action: { id: string }) => action.id)).toEqual([f.newerId]);
+    await assertSourceUnchanged(f);
+    expect(wake).not.toHaveBeenCalled();
+  });
+
+  it("rejects an agent even when it owns the current task and run", async () => {
+    const f = await seed();
+    const { app, wake } = appFor({ type: "agent", source: "agent_jwt", companyId: f.companyId,
+      agentId: f.ownerId, runId: f.currentRunId });
+    await request(app).post(`/api/issues/${f.issueId}/recovery-actions/resolve`).send(f.body).expect(403);
+    await assertSourceUnchanged(f);
+    expect(wake).not.toHaveBeenCalled();
+    expect((await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, f.actionId)))[0].evidence)
+      .not.toHaveProperty("executionReconciliation");
+  });
+
+  it("does not expose retained-source evidence to another company", async () => {
+    const f = await seed(), other = await seed();
+    const { app } = appFor({ type: "agent", source: "agent_jwt", companyId: other.companyId,
+      agentId: other.ownerId, runId: other.currentRunId });
+    const response = await request(app).get(`/api/issues/${f.issueId}/recovery-actions`).expect(404);
+    expect(JSON.stringify(response.body)).not.toContain(f.leaseId);
+    expect(JSON.stringify(response.body)).not.toContain(f.actionId);
+  });
+
+  it.each(["other_task", "other_company", "wrong_decision_run"])("rejects repair evidence for %s", async scenario => {
+    const f = await seed();
+    const other = await seed();
+    const foreignTaskId = scenario === "other_task" ? randomUUID() : other.issueId;
+    if (scenario === "other_task") await db.insert(issues).values({ id: foreignTaskId, companyId: f.companyId,
+      title: "Other task", status: "todo", assigneeAgentId: f.originalAgentId });
+    if (scenario !== "wrong_decision_run") await db.update(heartbeatRuns).set({
+      ...(scenario === "other_company" ? { companyId: other.companyId, agentId: other.originalAgentId } : {}),
+      contextSnapshot: { issueId: foreignTaskId },
+    }).where(eq(heartbeatRuns.id, f.sourceRunId));
+    const { app, wake } = appFor();
+    const body = scenario === "wrong_decision_run" ? { ...f.body,
+      executionReconciliation: { ...f.body.executionReconciliation, runId: other.sourceRunId } } : f.body;
+    await request(app).post(`/api/issues/${f.issueId}/recovery-actions/resolve`).send(body).expect(409);
+    await assertSourceUnchanged(f);
+    expect(wake).not.toHaveBeenCalled();
+  });
+
+  it("does not authorize ordinary execution reconciliation through the repair-only request shape", async () => {
+    const f = await seed();
+    await db.update(heartbeatRuns).set({ resultJson: { conversationContinuation: "continue_conversation_v1" } })
+      .where(eq(heartbeatRuns.id, f.sourceRunId));
+    await db.update(issueRecoveryActions).set({ evidence: { runId: f.sourceRunId, automaticRecovery: { replay: "blocked" } } })
+      .where(eq(issueRecoveryActions.id, f.actionId));
+    const { app, wake } = appFor();
+    await request(app).post(`/api/issues/${f.issueId}/recovery-actions/resolve`).send(f.body).expect(409);
+    await assertSourceUnchanged(f);
+    expect(wake).not.toHaveBeenCalled();
+  });
+
+  it("rejects a client-supplied repair-only authority flag", async () => {
+    const f = await seed();
+    const { app, wake } = appFor();
+    await request(app).post(`/api/issues/${f.issueId}/recovery-actions/resolve`)
+      .send({ ...f.body, workspaceRepairOnly: true }).expect(400);
+    await assertSourceUnchanged(f);
+    expect(wake).not.toHaveBeenCalled();
+  });
+
+  it.each(["missing_repair_evidence", "unconfirmed_stop", "missing_source_marker", "status_change"])("rejects %s without resolving the hold", async scenario => {
+    const f = await seed();
+    if (scenario === "unconfirmed_stop") await db.update(environmentLeases).set({ metadata: {} }).where(eq(environmentLeases.id, f.leaseId));
+    if (scenario === "missing_source_marker") await db.update(heartbeatRuns).set({ resultJson: { workspaceRestoreFailure: "restore_failed" } })
+      .where(eq(heartbeatRuns.id, f.sourceRunId));
+    const { app, wake } = appFor();
+    const body = scenario === "missing_repair_evidence" ? { ...f.body,
+      executionReconciliation: { ...f.body.executionReconciliation, workspaceRepairEvidence: undefined } }
+      : scenario === "status_change" ? { ...f.body, sourceIssueStatus: "todo" } : f.body;
+    await request(app).post(`/api/issues/${f.issueId}/recovery-actions/resolve`).send(body)
+      .expect(scenario === "missing_repair_evidence" ? 400 : 409);
+    expect((await db.select().from(issues).where(eq(issues.id, f.issueId)))[0]).toEqual(f.issueBefore);
+    expect((await db.select().from(issueRecoveryActions).where(and(eq(issueRecoveryActions.id, f.actionId),
+      eq(issueRecoveryActions.companyId, f.companyId))))[0].evidence).not.toHaveProperty("executionReconciliation");
+    expect(wake).not.toHaveBeenCalled();
+  });
+});

server/src/routes/issues.ts

@@ -1,3 +1,4 @@
+import { hasRequiredWorkspaceRecovery, LEGACY_WORKSPACE_RECOVERY_SCHEMA } from "../services/workspace-restore-recovery-state.js";
 import { monitorPoliciesEqual, applyActorMonitorScheduledBy, assertCanManageIssueMonitor, summarizeIssueMonitor } from "../services/issue-monitors.js";
 import type { IssuePrivacyConstraints } from "@paperclipai/shared";
 import { canActorReadHeartbeatRun } from "../services/heartbeat-run-privacy.js";
@@ -9369,9 +9370,15 @@ export function issueRoutes(
       trigger: "read_projection",
       actor: getActorInfo(req),
     });
+    const retainedHolds = await db.select().from(issueRecoveryActions).where(and(
+      eq(issueRecoveryActions.companyId, issue.companyId), eq(issueRecoveryActions.sourceIssueId, issue.id),
+      sql`${issueRecoveryActions.evidence}->'workspaceRestoreRecovery'->>'schema' = ${LEGACY_WORKSPACE_RECOVERY_SCHEMA}`,
+      sql`${issueRecoveryActions.evidence}->'automaticRecovery'->>'replay' = 'blocked'`,
+    )).orderBy(desc(issueRecoveryActions.createdAt));
     res.json({
       active,
-      actions: active ? [active] : [],
+      actions: [...(active ? [active] : []), ...retainedHolds
+        .filter(action => action.id !== active?.id).map(issueRecoveryActionReadModel)],
     });
   });
 
@@ -9465,6 +9472,37 @@ export function issueRoutes(
           .then((rows) => rows[0] ?? null);
         if (!lockedIssue) throw notFound("Issue not found");
 
+        // Retained files are a source-scoped obligation, independent of the
+        // current task owner/generation and any newer active recovery incident.
+        // Repair records evidence only: it must not reopen or replay old work.
+        const [workspaceRepairAction] = await tx.select().from(issueRecoveryActions).where(and(
+          eq(issueRecoveryActions.companyId, lockedIssue.companyId),
+          eq(issueRecoveryActions.sourceIssueId, lockedIssue.id),
+          actionId ? eq(issueRecoveryActions.id, actionId) : inArray(issueRecoveryActions.status, ["active", "escalated"]),
+        )).limit(1).for("update");
+        if (workspaceRepairAction && hasRequiredWorkspaceRecovery(workspaceRepairAction.evidence)) {
+          assertBoard(req);
+          await requireRecoveryActionAuthority(req, lockedIssue, issueRecoveryActionReadModel(workspaceRepairAction),
+            { source: "recovery_action_resolution" });
+          if (outcome !== "restored" || sourceIssueStatus !== lockedIssue.status) {
+            throw conflict("Record workspace repair without changing the current task status or requesting a retry.");
+          }
+          await validateExecutionReconciliation({ db: tx as unknown as Db,
+            companyId: lockedIssue.companyId, issueId: lockedIssue.id, agentId: lockedIssue.assigneeAgentId,
+            sourceRunId: workspaceRepairAction.evidence.runId, decision: executionReconciliation,
+            workspaceRepairOnly: true,
+          });
+          if (workspaceRepairAction.evidence.executionReconciliation) {
+            return { issue: lockedIssue, recoveryAction: workspaceRepairAction, replayed: true, workspaceRepairOnly: true };
+          }
+          await markExecutionReconciliation(tx as unknown as Db, workspaceRepairAction,
+            executionReconciliation!, actor.actorId, undefined, { workspaceRepairOnly: true });
+          const [recoveryAction] = await tx.update(issueRecoveryActions).set({ status: "resolved", outcome: "restored",
+            resolutionNote: resolutionNote ?? null, resolvedAt: new Date(), updatedAt: new Date(),
+          }).where(eq(issueRecoveryActions.id, workspaceRepairAction.id)).returning();
+          return { issue: lockedIssue, recoveryAction, chatRetry: null, workspaceRepairOnly: true };
+        }
+
         let activeRecoveryAction = await recoveryActionsSvc.getActiveForIssue(
           lockedIssue.companyId,
           lockedIssue.id,
@@ -9845,7 +9883,9 @@ export function issueRoutes(
       });
       if (result.replayed) {
         res.json({
-          issue: result.issue,
+          issue: "workspaceRepairOnly" in result
+            ? { ...result.issue, activeRecoveryAction: await recoveryActionsSvc.getActiveForIssue(result.issue.companyId, result.issue.id) }
+            : result.issue,
           recoveryAction: result.recoveryAction,
         });
         return;
@@ -9954,7 +9994,9 @@ export function issueRoutes(
       res.json({
         issue: {
           ...result.issue,
-          activeRecoveryAction: null,
+          activeRecoveryAction: "workspaceRepairOnly" in result
+            ? await recoveryActionsSvc.getActiveForIssue(result.issue.companyId, result.issue.id)
+            : null,
         },
         recoveryAction: result.recoveryAction,
       });

server/src/services/conversation-continuation.ts

@@ -1,6 +1,8 @@
+import { hasRequiredWorkspaceRecovery } from "./workspace-restore-recovery-state.js";
 import { and, desc, eq, inArray, isNotNull, or, sql } from "drizzle-orm";
 import { environmentLeases, heartbeatRunEvents, heartbeatRuns, issueRecoveryActions, type Db } from "@paperclipai/db";
 import { readProcessStartedAt } from "./hot-restart.js";
+import { hasRemoteTerminationReceipt } from "./remote-execution-termination.js";
 
 // These adapters accept a conversation turn. Retrying a process or webhook can
 // replay the action itself, so those adapters retain their recovery contract.
@@ -16,7 +18,7 @@ export function isConversationAdapter(adapterType: string): boolean {
 export const CONVERSATION_CONTINUATION_POLICY = "continue_conversation_v1";
 
 export function hasConversationContinuationPolicy(result: Record<string, unknown> | null | undefined): boolean {
-  return result?.workspaceRestoreFailure !== "restore_unsafe_archive" && result?.conversationContinuation === CONVERSATION_CONTINUATION_POLICY;
+  return result?.workspaceRestoreFailure !== "restore_unsafe_archive" && !hasRequiredWorkspaceRecovery(result) && result?.conversationContinuation === CONVERSATION_CONTINUATION_POLICY;
 }
 
 /** Persisted by the server when it claims the run, before remote provisioning. */
@@ -52,7 +54,7 @@ export async function historicalAdapterType(db: Db, run: typeof heartbeatRuns.$i
 }
 
 export async function runUsedConversationAdapter(db: Db, run: typeof heartbeatRuns.$inferSelect): Promise<boolean> {
-  if (run.resultJson?.workspaceRestoreFailure === "restore_unsafe_archive") return false;
+  if (run.resultJson?.workspaceRestoreFailure === "restore_unsafe_archive" || hasRequiredWorkspaceRecovery(run.resultJson)) return false;
   if (hasConversationContinuationPolicy(run.resultJson)) return true;
   const adapterType = await historicalAdapterType(db, run);
   return adapterType !== null && isConversationAdapter(adapterType);
@@ -72,6 +74,7 @@ export function conversationRecoveryActionPredicate() {
         and coalesce(${heartbeatRuns.nativeIssueId}::text, ${heartbeatRuns.contextSnapshot}->>'issueId') = ${issueRecoveryActions.sourceIssueId}::text
         and ${heartbeatRuns.runtimeMode} = 'legacy'
         and coalesce(${heartbeatRuns.resultJson}->>'workspaceRestoreFailure', '') <> 'restore_unsafe_archive'
+        and coalesce(${heartbeatRuns.resultJson}->'workspaceRestoreRecovery'->>'schema', '') <> 'paperclip.workspace-restore-recovery.v1'
         and ${inArray(heartbeatRuns.status, ['failed', 'timed_out', 'interrupted', 'cancelled'])}
         and ${conversationRunPredicate()}
         and ${or(
@@ -110,9 +113,29 @@ export async function getConversationOwnershipBlocker(db: Db, companyId: string,
       conversationRunPredicate(),
       sql`coalesce(${heartbeatRuns.nativeIssueId}::text, ${heartbeatRuns.contextSnapshot}->>'issueId') = ${issueId}`,
       inArray(heartbeatRuns.status, ["failed", "timed_out", "interrupted", "cancelled"]),
-      or(isNotNull(heartbeatRuns.processPid), isNotNull(heartbeatRuns.processGroupId), activeLease),
+      or(isNotNull(heartbeatRuns.processPid), isNotNull(heartbeatRuns.processGroupId), activeLease,
+        sql`${heartbeatRuns.resultJson}->'workspaceRestoreRecovery'->>'schema' = 'paperclip.workspace-restore-recovery.v1'`),
     )).orderBy(desc(heartbeatRuns.createdAt), desc(heartbeatRuns.id));
   for (const { run, activeLease: leaseHeld } of candidates) {
+    if (hasRequiredWorkspaceRecovery(run.resultJson)) {
+      const leases = await db.select().from(environmentLeases).where(and(
+        eq(environmentLeases.companyId, companyId), eq(environmentLeases.heartbeatRunId, run.id),
+      ));
+      const retainedIds = (run.resultJson!.workspaceRestoreRecovery as { leaseIds?: unknown }).leaseIds;
+      const remoteLeases = leases.filter(lease => lease.provider !== "local");
+      const remoteStopped = Array.isArray(retainedIds) && retainedIds.length > 0
+        && retainedIds.every(id => typeof id === "string" && remoteLeases.some(lease =>
+          lease.id === id && hasRemoteTerminationReceipt(lease)))
+        && remoteLeases.every(hasRemoteTerminationReceipt);
+      if (!remoteStopped) return {
+        runId: run.id, agentId: run.agentId, cause: "execution_owner_active",
+        nextAction: "The retained workspace environment has not confirmed that it stopped. Wait for cleanup before continuing this task.",
+      };
+      // These PIDs came from the sandbox. A matching process on this host is
+      // unrelated and must not hide the repair action or block a repaired task.
+      // A separate local lease keeps the existing host process checks below.
+      if (leases.every(lease => lease.provider !== "local")) continue;
+    }
     let pidAlive = run.processPid !== null && processMayBeAlive(run.processPid);
     if (pidAlive && run.processStartedAt) {
       // A recycled PID cannot keep an old task blocked. An unreadable identity

server/src/services/execution-blocker.ts

@@ -7,6 +7,7 @@ import { canContinueCancelledRun, readRunCancellation } from "./run-cancellation
 import { canRetryStoppedRun } from "./cancelled-native-startup.js";
 import { queuedCommentIdsFromWakePayload } from "./issue-queued-comment-queue.js";
 import { EXECUTION_RECONCILIATION_CAUSES, type ExecutionBlocker } from "@paperclipai/shared";
+import { LEGACY_WORKSPACE_RECOVERY_SCHEMA } from "./workspace-restore-recovery-state.js";
 
 /** Resolved recovery bookkeeping can still carry an effective no-replay hold. */
 export function executionBlockerPredicate() {
@@ -24,13 +25,16 @@ export async function getExecutionBlocker(db: Db, companyId: string, issueId: st
     assigneeAgentId: issues.assigneeAgentId }).from(issues).where(and(
     eq(issues.companyId, companyId), eq(issues.id, issueId),
   )).limit(1);
-  // Resetting model context cannot make an unsafe workspace safe. This hold
+  // Resetting model context cannot repair or recover a workspace. This hold
   // survives conversation boundaries until the existing repair/reconciliation path clears it.
   const [restoreHold] = await db.select().from(issueRecoveryActions).where(and(
     eq(issueRecoveryActions.companyId, companyId),
     eq(issueRecoveryActions.sourceIssueId, issueId),
     executionBlockerPredicate(),
-    sql`${issueRecoveryActions.evidence}->>'workspaceRestoreFailure' = 'restore_unsafe_archive'`,
+    or(
+      sql`${issueRecoveryActions.evidence}->>'workspaceRestoreFailure' = 'restore_unsafe_archive'`,
+      sql`${issueRecoveryActions.evidence}->'workspaceRestoreRecovery'->>'schema' = ${LEGACY_WORKSPACE_RECOVERY_SCHEMA}`,
+    ),
   )).orderBy(desc(issueRecoveryActions.updatedAt)).limit(1);
   // A persisted user /new is an ordered context command, not a retry of uncertain work.
   // The normal issue execution lock still serializes it behind any active turn.
@@ -80,9 +84,9 @@ export async function getExecutionBlocker(db: Db, companyId: string, issueId: st
         or consumed.context_snapshot->'wakeCommentIds' @> jsonb_build_array(${issueComments.id}::text)))`,
   )) : [];
   const cancellation = run ? readRunCancellation(run.resultJson) : null;
-  const eligibleContinuation = Boolean(run && conversation?.assigneeAgentId === run.agentId &&
+  const eligibleContinuation = Boolean(!restoreHold && run && conversation?.assigneeAgentId === run.agentId &&
     !["done", "cancelled"].includes(conversation.status) && canContinueCancelledRun(run));
-  const canRetry = Boolean(run && conversation?.assigneeAgentId === run.agentId &&
+  const canRetry = Boolean(!restoreHold && run && conversation?.assigneeAgentId === run.agentId &&
     !["done", "cancelled"].includes(conversation.status) && await canRetryStoppedRun(db, run));
   const runError = run && canRetry && isPreDispatchReviewWait(run)
     ? "Waiting for review; this continuation never started."
@@ -110,7 +114,7 @@ export async function getExecutionBlocker(db: Db, companyId: string, issueId: st
     } else {
       nextAction = "Send a new chat message to continue this conversation.";
     }
-  } else if (run?.status === "cancelled" && !eligibleContinuation && !canRetry && action.cause === "legacy_execution_requires_reconciliation") {
+  } else if (!restoreHold && run?.status === "cancelled" && !eligibleContinuation && !canRetry && action.cause === "legacy_execution_requires_reconciliation") {
     nextAction += " Inspect the run before sending a new message to request continuation.";
   }
 
@@ -125,6 +129,7 @@ export async function getExecutionBlocker(db: Db, companyId: string, issueId: st
     runError: runError?.slice(0, 1024) ?? null,
     canContinue: eligibleContinuation && !chatBinding,
     canRetry: canRetry && !chatBinding,
+    ...(restoreHold ? { workspaceRepairRequired: true } : {}),
     savedMessageCount: saved.length,
   };
 }

server/src/services/execution-recovery-resolution.ts

@@ -22,6 +22,8 @@ import {
 } from "@paperclipai/shared";
 import { parseIssueExecutionState } from "./issue-execution-policy.js";
 import { isSupersededConversationRun } from "./agent-conversations.js";
+import { hasRequiredWorkspaceRecovery } from "./workspace-restore-recovery-state.js";
+import { hasRemoteTerminationReceipt } from "./remote-execution-termination.js";
 
 /** An operator records observed outcomes; this is not permission to blindly retry. */
 export async function validateExecutionReconciliation(input: {
@@ -31,9 +33,11 @@ export async function validateExecutionReconciliation(input: {
   agentId: string | null;
   sourceRunId: unknown;
   decision: ExecutionReconciliation | undefined;
+  /** Board-only source repair: never authorize a continuation of this run. */
+  workspaceRepairOnly?: boolean;
 }) {
   const { db, companyId, issueId, agentId, decision } = input;
-  if (!decision || decision.runId !== input.sourceRunId || !agentId) {
+  if (!decision || decision.runId !== input.sourceRunId || (!agentId && !input.workspaceRepairOnly)) {
     throw conflict(
       "Reconcile the recorded execution and its action outcomes before continuing this task.",
     );
@@ -59,11 +63,13 @@ export async function validateExecutionReconciliation(input: {
     review?.status === "pending" &&
     review.currentParticipant?.type === "agent" &&
     review.currentParticipant.agentId === run?.agentId;
+  const retainedWorkspace = hasRequiredWorkspaceRecovery(run?.resultJson);
+  const repairOnly = input.workspaceRepairOnly === true && retainedWorkspace;
   if (
+    (input.workspaceRepairOnly && !retainedWorkspace) ||
     !run ||
     !task ||
-    task.assigneeAgentId !== agentId ||
-    (run.agentId !== agentId && !isCurrentReviewer) ||
+    (!repairOnly && (task.assigneeAgentId !== agentId || (run.agentId !== agentId && !isCurrentReviewer))) ||
     (run.nativeIssueId ?? run.contextSnapshot?.issueId) !== issueId ||
     !["failed", "interrupted", "timed_out", "cancelled"].includes(run.status)
   ) {
@@ -75,10 +81,32 @@ export async function validateExecutionReconciliation(input: {
       (!decision.workspaceRepairEvidence || decision.workspaceRepairEvidence.trim().length < 20)) {
     throw conflict("Verify safe workspace staging or repair and record workspaceRepairEvidence before continuing this run.");
   }
-  for (const pid of [
+  let checkLocalProcesses = true;
+  if (retainedWorkspace) {
+    const leases = await db.select().from(environmentLeases).where(and(
+      eq(environmentLeases.companyId, companyId), eq(environmentLeases.heartbeatRunId, run.id),
+    ));
+    const recovery = run.resultJson!.workspaceRestoreRecovery as { leaseIds?: unknown };
+    const retainedIds = recovery.leaseIds;
+    // The terminal transaction records the exact source allocations. A release
+    // timestamp is written before stop dispatch, so only matching provider
+    // acknowledgements can establish that repair will not race active writes.
+    if (!Array.isArray(retainedIds) || !retainedIds.length ||
+        !retainedIds.every(id => typeof id === "string" && leases.some(lease =>
+          lease.id === id && hasRemoteTerminationReceipt(lease))) ||
+        leases.some(lease => lease.provider === "local"
+          ? !lease.releasedAt || lease.status === "pending_cleanup" || lease.cleanupStatus === "failed"
+          : !hasRemoteTerminationReceipt(lease))) {
+      throw conflict("The retained workspace environment has not confirmed that it stopped. Wait for cleanup before recording workspace repair.");
+    }
+    checkLocalProcesses = leases.some(lease => lease.provider === "local");
+  }
+  // A retained remote source's PIDs belong to its sandbox, not this server.
+  // Its exact lease receipts above replace host-local process probes.
+  for (const pid of checkLocalProcesses ? [
     run.processPid,
     run.processGroupId ? -run.processGroupId : null,
-  ]) {
+  ] : []) {
     if (!pid) continue;
     try {
       process.kill(pid, 0);
@@ -120,11 +148,11 @@ export async function validateExecutionReconciliation(input: {
     throw conflict(
       "The previous execution environment has not finished releasing its authority.",
     );
-  await buildExecutionContinuation({
+  if (!repairOnly) await buildExecutionContinuation({
     db,
     companyId,
     issueId,
-    agentId,
+    agentId: agentId!,
     context: { previousRunId: run.id },
     summary: null,
     exposeLowTrustRaw: false,
@@ -142,7 +170,11 @@ export async function markExecutionReconciliation(
   decision: ExecutionReconciliation,
   actorId: string,
   deliveryOwner?: { kind: "chat_failed_run_retry"; actionId: string },
+  options?: { workspaceRepairOnly?: boolean },
 ) {
+  if (options?.workspaceRepairOnly && (deliveryOwner || !hasRequiredWorkspaceRecovery(action.evidence))) {
+    throw conflict("Workspace repair must target the retained source without a continuation owner.");
+  }
   if (deliveryOwner) {
     const [retry] = await db
       .select()
@@ -187,7 +219,7 @@ export async function markExecutionReconciliation(
           actorId,
           recordedAt: new Date().toISOString(),
         },
-        continuationDelivery: deliveryOwner ? "delegated" : "pending",
+        continuationDelivery: options?.workspaceRepairOnly ? "not_requested" : deliveryOwner ? "delegated" : "pending",
         ...(deliveryOwner ? { continuationDeliveryOwner: deliveryOwner } : {}),
       },
     })

server/src/services/explicit-native-continuation.ts

@@ -1,3 +1,4 @@
+import { hasRequiredWorkspaceRecovery } from "./workspace-restore-recovery-state.js";
 import { createHash } from "node:crypto";
 import { appendHeartbeatRunEvent } from "./heartbeat-run-events.js";
 import { canContinueCancelledRun } from "./run-cancellation.js";
@@ -197,7 +198,7 @@ export async function admitExplicitNativeContinuation(input: {
     if (!lockedRun || lockedRun.status !== run.status || lockedRun.agentId !== run.agentId ||
         lockedRun.finishedAt?.getTime() !== run.finishedAt.getTime()) return null;
     run = lockedRun;
-    if (run.resultJson?.workspaceRestoreFailure === "restore_unsafe_archive") {
+    if (run.resultJson?.workspaceRestoreFailure === "restore_unsafe_archive" || hasRequiredWorkspaceRecovery(run.resultJson)) {
       return blocked("workspace_repair_required", "Verify safe workspace staging or repair before continuing. Your message is saved.");
     }
     const cancelledStartup = await isCancelledNativeStartup(db, run, coordinator);

server/src/services/heartbeat.ts

@@ -1,3 +1,6 @@
+import { preserveWorkspaceRestoreRecoveryMetadataSql } from "./legacy-workspace-restore-recovery.js";
+import { preserveWorkspaceRestoreRecoveryMetadata } from "./workspace-restore-recovery-state.js";
+import { recordLegacyWorkspaceRestoreFailure } from "./legacy-execution-recovery.js";
 import { CONFIGURED_ENVIRONMENT_KEYS, configuredEnvironmentProjection } from "../vendor/paperclip-runner/index.js";
 import { activeIssueInteractionCondition, TASK_QUESTION_GUIDANCE } from "./issue-question-context.js";
 import { createAgentIdentityRedactor } from "./agent-identity-redaction.js";
@@ -3507,6 +3510,8 @@ const heartbeatRunSafeResultJsonColumn = sql<Record<string, unknown> | null>`
             'errorMessage', left(${heartbeatRuns.resultJson} #>> '{instructionSave,errorMessage}', 1024),
             'storageWarning', left(${heartbeatRuns.resultJson} #>> '{instructionSave,storageWarning}', 1024)
           )) end,
+        'workspaceRestoreRecovery', case when ${heartbeatRuns.resultJson} #>> '{workspaceRestoreRecovery,schema}' = 'paperclip.workspace-restore-recovery.v1'
+          then jsonb_build_object('schema', 'paperclip.workspace-restore-recovery.v1') end,
         'workspaceRestoreFailure', case when ${heartbeatRuns.resultJson} ->> 'workspaceRestoreFailure'
           in ('restore_permission_denied', 'restore_lock_timeout', 'restore_unsafe_archive', 'restore_failed')
           then ${heartbeatRuns.resultJson} -> 'workspaceRestoreFailure' end,
@@ -3586,6 +3591,7 @@ const heartbeatRunExecutionEvidenceColumn = sql<Record<string, unknown> | null>`
     'stopReason', ${heartbeatRuns.resultJson} -> 'stopReason',
     'timeoutSource', ${heartbeatRuns.resultJson} -> 'timeoutSource',
     'workspaceRestoreFailure', ${heartbeatRuns.resultJson} -> 'workspaceRestoreFailure',
+    'workspaceRestoreRecovery', ${heartbeatRuns.resultJson} -> 'workspaceRestoreRecovery',
     'executionCancellation', ${heartbeatRuns.resultJson} -> 'executionCancellation',
     'nativeCancellation', ${heartbeatRuns.resultJson} -> 'nativeCancellation',
     'cancelledByActorType', ${heartbeatRuns.resultJson} -> 'cancelledByActorType',
@@ -13001,6 +13007,10 @@ export function heartbeatService(
       .where(eq(heartbeatRuns.id, runId))
       .then((rows) => rows[0] ?? null);
 
+    if (previousStatus && patch?.resultJson !== undefined) patch = { ...patch,
+      resultJson: preserveWorkspaceRestoreRecoveryMetadata(previousStatus.resultJson, patch.resultJson),
+    };
+
     // Preserve the receipt-source fence when finalization enriches usage. A
     // late spool replay must still be able to complete an unfinished receipt.
     if (patch?.usageJson && previousStatus?.usageJson) patch = { ...patch, usageJson: { ...previousStatus.usageJson, ...patch.usageJson } };
@@ -13027,17 +13037,13 @@ export function heartbeatService(
       };
     }
     const updated =
-      previousStatus &&
-      await legacyExecutionNeedsReconciliationWithEvidence(db, {
-        ...previousStatus,
-        status,
-        ...patch,
-      })
+      previousStatus && previousStatus.runtimeMode === "legacy" && isHeartbeatRunTerminalStatus(status)
         ? await terminalizeLegacyExecution({
             db,
             run: previousStatus,
             status,
             patch,
+            reconcileIfNeeded: true,
           })
         : await db
             .update(heartbeatRuns)
@@ -13099,6 +13105,10 @@ export function heartbeatService(
       patch = { ...patch, resultJson: cancellationResultJson(previousStatus, status, patch?.resultJson, patch?.errorCode, patch?.error) };
     }
 
+    if (previousStatus && patch?.resultJson !== undefined) patch = { ...patch,
+      resultJson: preserveWorkspaceRestoreRecoveryMetadata(previousStatus.resultJson, patch.resultJson),
+    };
+
     // Preserve the receipt-source fence when finalization enriches usage. A
     // late spool replay must still be able to complete an unfinished receipt.
     if (patch?.usageJson && previousStatus?.usageJson) patch = { ...patch, usageJson: { ...previousStatus.usageJson, ...patch.usageJson } };
@@ -13122,18 +13132,15 @@ export function heartbeatService(
       };
     }
     const updated =
-      previousStatus &&
-      await legacyExecutionNeedsReconciliationWithEvidence(db, {
-        ...previousStatus,
-        status,
-        ...patch,
-      })
+      previousStatus && previousStatus.runtimeMode === "legacy" && isHeartbeatRunTerminalStatus(status)
         ? await terminalizeLegacyExecution({
             db,
             run: previousStatus,
             status,
             patch,
             fromStatuses,
+            reconcileIfNeeded: true,
+            writeConditions: [nativeRunnerOwnershipNotHeldCondition(), ...(cancellationCondition ? [cancellationCondition] : [])],
           })
         : await db
             .update(heartbeatRuns)
@@ -13847,9 +13854,8 @@ export function heartbeatService(
         .update(heartbeatRuns)
         .set({
           livenessReason: UNMANAGED_BACKGROUND_TASK_LIVENESS_REASON,
-          resultJson: withUnmanagedBackgroundTaskStopReason(
-            parseObject(run.resultJson),
-          ),
+          resultJson: sql`coalesce(${heartbeatRuns.resultJson}, '{}'::jsonb) ||
+            ${JSON.stringify({ stopReason: UNMANAGED_BACKGROUND_TASK_STOP_REASON })}::jsonb`,
           updatedAt: new Date(),
         })
         .where(eq(heartbeatRuns.id, run.id));
@@ -15056,7 +15062,7 @@ export function heartbeatService(
       const updated = await db
         .update(heartbeatRuns)
         .set({
-          resultJson,
+          resultJson: preserveWorkspaceRestoreRecoveryMetadataSql(resultJson, true),
           error:
             run.errorCode === DETACHED_PROCESS_ERROR_CODE ? null : run.error,
           errorCode:
@@ -20589,6 +20595,7 @@ export function heartbeatService(
     let nativeWorkspaceSync: Awaited<
       ReturnType<typeof prepareNativeWorkspaceSync>
     > = null;
+    let requiredWorkspaceRestoreEvidence: Record<string, unknown> | null = null;
     let providerResourceDispositionForRun:
       ProviderResourceDisposition | undefined;
     let nativeLifecycleTelemetryForRun:
@@ -25276,6 +25283,16 @@ export function heartbeatService(
             adapterResult = await guardedDispatch.resultPromise;
           }
           adapterResult = identityRedactor.redact(adapterResult);
+          if (run.runtimeMode === "legacy" && hasWorkspaceRestoreFailure(adapterResult.resultJson)
+              && executionTarget?.kind === "remote" && executionTarget.transport === "sandbox") {
+            requiredWorkspaceRestoreEvidence = {
+              workspaceRestoreFailure: adapterResult.resultJson!.workspaceRestoreFailure,
+              ...(adapterResult.resultJson?.workspaceRestoreDiagnostic ? { workspaceRestoreDiagnostic: adapterResult.resultJson.workspaceRestoreDiagnostic } : {}),
+            };
+            // Retention is the fallback even if recording this receipt fails.
+            providerResourceDispositionForRun = "stop_and_retain";
+            await recordLegacyWorkspaceRestoreFailure(db, run, requiredWorkspaceRestoreEvidence);
+          }
           for (const stream of ["stdout", "stderr"] as const) {
             const tail = identityRedactor.finish(stream);
             if (tail) await appendIdentityRedactedLog(stream, tail);
@@ -25863,7 +25880,9 @@ export function heartbeatService(
               .update(heartbeatRuns)
               .set({
                 ...finalRunPatch,
-                resultJson: cancellationResultJson(persistedRunWrite.run, status, finalRunPatch.resultJson, runErrorCode, runErrorMessage),
+                resultJson: preserveWorkspaceRestoreRecoveryMetadataSql(
+                  cancellationResultJson(persistedRunWrite.run, status, finalRunPatch.resultJson, runErrorCode, runErrorMessage) ?? null,
+                ),
 
                 usageJson: { ...parseObject(persistedRunWrite.run.usageJson), ...parseObject(finalRunPatch.usageJson) },
                 finishedAt:
@@ -26071,10 +26090,7 @@ export function heartbeatService(
             await db
               .update(heartbeatRuns)
               .set({
-                resultJson: {
-                  ...persistedResultJson,
-                  presentationDecision,
-                },
+                resultJson: sql`coalesce(${heartbeatRuns.resultJson}, '{}'::jsonb) || ${JSON.stringify({ presentationDecision })}::jsonb`,
                 updatedAt: new Date(),
               })
               .where(eq(heartbeatRuns.id, livenessRun.id));
@@ -26474,6 +26490,7 @@ export function heartbeatService(
             errorMessage: message,
             resultJson: {
               ...parseObject(stopSnapshot?.resultJson),
+              ...requiredWorkspaceRestoreEvidence,
               ...(workspaceValidationFailure?.resultJson ??
                 configurationIncompleteFailure?.resultJson ??
                 {}),
@@ -29944,18 +29961,15 @@ export function heartbeatService(
               .set({
                 error: reason,
                 errorCode,
-                resultJson: {
-                  ...parseObject(run.resultJson),
+                // Stop may have read the row before adapter settlement saved
+                // a required-copyback receipt. Merge into current database JSON
+                // and leave those server-owned fields to the restore recorder.
+                resultJson: preserveWorkspaceRestoreRecoveryMetadataSql({
                   ...resultJson,
-                  ...(!running
-                    ? {
-                        executionCancellation: {
-                          state: "requested",
-                          requestedAt: new Date().toISOString(),
-                        },
-                      }
-                    : {}),
-                },
+                  ...(!running ? { executionCancellation: {
+                    state: "requested", requestedAt: new Date().toISOString(),
+                  } } : {}),
+                }, true),
                 updatedAt: new Date(),
               })
               .where(

server/src/services/legacy-execution-recovery.ts

@@ -1,9 +1,11 @@
+import { hasRequiredWorkspaceRecovery, preserveWorkspaceRestoreRecoveryMetadata, LEGACY_WORKSPACE_RECOVERY_SCHEMA } from "./workspace-restore-recovery-state.js";
+import { hasUnrestoredRemoteWorkspace, preserveLegacyWorkspaceRestoreSources } from "./legacy-workspace-restore-recovery.js";
 import { isPreDispatchReviewWaitVerified } from "./pre-dispatch-review-wait.js";
 import { hasWorkspaceRestoreFailure } from "@paperclipai/shared";
 import { normalizeMaxTurnStopReason } from "./heartbeat-stop-metadata.js";
 import { claimedAdapterType, hasConversationContinuationPolicy } from "./conversation-continuation.js";
 import { randomUUID } from "node:crypto";
-import { and, eq, inArray, isNull, or, sql } from "drizzle-orm";
+import { and, desc, eq, inArray, isNull, or, sql, type SQL } from "drizzle-orm";
 import { environmentLeases, heartbeatRuns, issueRecoveryActions, issues, nativeRunFinalizations, type Db } from "@paperclipai/db";
 import { issueRecoveryActionService } from "./issue-recovery-actions.js";
 import { parseIssueExecutionState } from "./issue-execution-policy.js";
@@ -25,7 +27,7 @@ export function legacyExecutionNeedsReconciliation(
   )
     return false;
   // A fresh model turn cannot repair or verify unrestored files.
-  if (run.resultJson?.workspaceRestoreFailure === "restore_unsafe_archive") return true;
+  if (run.resultJson?.workspaceRestoreFailure === "restore_unsafe_archive" || hasRequiredWorkspaceRecovery(run.resultJson)) return true;
   // A fresh conversation turn lets the agent decide what remains. The retry
   // scheduler, not an action-outcome hold, owns the automatic attempt limit.
   if (hasConversationContinuationPolicy(run.resultJson)) return false;
@@ -57,7 +59,8 @@ export function legacyExecutionNeedsReconciliation(
 /** Review-wait receipts are only exempt after retained execution evidence agrees.
  * The synchronous classifier stays conservative for callers without a DB proof. */
 export async function legacyExecutionNeedsReconciliationWithEvidence(db: Db, run: Run): Promise<boolean> {
-  return legacyExecutionNeedsReconciliation(run) && !(await isPreDispatchReviewWaitVerified(db, run));
+  return await hasUnrestoredRemoteWorkspace(db, run) ||
+    (legacyExecutionNeedsReconciliation(run) && !(await isPreDispatchReviewWaitVerified(db, run)));
 }
 
 /** Persist the failed legacy run, owned lock release and operator decision together. */
@@ -67,8 +70,13 @@ export async function terminalizeLegacyExecution(input: {
   status: string;
   patch?: Partial<typeof heartbeatRuns.$inferInsert>;
   fromStatuses?: string[];
+  reconcileIfNeeded?: boolean;
+  writeConditions?: SQL[];
+  /** Adapter settlement records a copy-back failure before host finalization. */
+  recordRestoreFailureOnly?: boolean;
 }) {
-  const { db, run, status, patch } = input;
+  const { db, run, status } = input;
+  let patch = input.patch;
   const issueId =
     run.nativeIssueId ??
     (typeof run.contextSnapshot?.issueId === "string"
@@ -87,10 +95,19 @@ export async function terminalizeLegacyExecution(input: {
           )
           .for("update")
       : [];
-    const [updated] = await tx
+    const [current] = await tx.select().from(heartbeatRuns).where(and(
+      eq(heartbeatRuns.id, run.id), eq(heartbeatRuns.companyId, run.companyId),
+    )).for("update");
+    if (!current || current.runtimeMode !== "legacy" ||
+        (!input.recordRestoreFailureOnly && !(input.fromStatuses ?? [run.status]).includes(current.status))) return null;
+    if (patch?.resultJson !== undefined) patch = { ...patch,
+      resultJson: preserveWorkspaceRestoreRecoveryMetadata(current.resultJson,
+        input.recordRestoreFailureOnly ? { ...current.resultJson, ...patch.resultJson } : patch.resultJson),
+    };
+    let [updated] = await tx
       .update(heartbeatRuns)
       .set({
-        status,
+        status: input.recordRestoreFailureOnly ? current.status : status,
         ...patch,
         executionStatusDeliveryId: randomUUID(),
         updatedAt: new Date(),
@@ -99,12 +116,24 @@ export async function terminalizeLegacyExecution(input: {
         and(
           eq(heartbeatRuns.id, run.id),
           eq(heartbeatRuns.companyId, run.companyId),
-          inArray(heartbeatRuns.status, input.fromStatuses ?? [run.status]),
+          eq(heartbeatRuns.status, current.status),
+          ...(input.writeConditions ?? []),
         ),
       )
       .returning();
     if (!updated) return null;
-    if (task?.executionRunId === run.id)
+    const retainedLeaseIds = await preserveLegacyWorkspaceRestoreSources(tx as unknown as Db, input.recordRestoreFailureOnly ? { ...updated, status: "failed" } : updated);
+    if (retainedLeaseIds.length) {
+      [updated] = await tx.update(heartbeatRuns).set({ resultJson: {
+        ...updated.resultJson,
+        workspaceRestoreRecovery: { schema: LEGACY_WORKSPACE_RECOVERY_SCHEMA, leaseIds: retainedLeaseIds },
+      } }).where(and(eq(heartbeatRuns.id, updated.id), eq(heartbeatRuns.companyId, updated.companyId))).returning();
+    }
+    const needsRecovery = input.recordRestoreFailureOnly
+      ? retainedLeaseIds.length > 0 || hasRequiredWorkspaceRecovery(updated.resultJson)
+      : !input.reconcileIfNeeded || await legacyExecutionNeedsReconciliationWithEvidence(tx as unknown as Db, updated);
+    if (!needsRecovery) return updated;
+    if (!input.recordRestoreFailureOnly && task?.executionRunId === run.id)
       await tx
         .update(issues)
         .set({
@@ -113,11 +142,45 @@ export async function terminalizeLegacyExecution(input: {
           executionLockedAt: null,
         })
         .where(eq(issues.id, task.id));
-    if (task?.checkoutRunId === run.id)
+    if (!input.recordRestoreFailureOnly && task?.checkoutRunId === run.id)
       await tx
         .update(issues)
         .set({ checkoutRunId: null })
         .where(eq(issues.id, task.id));
+    if (task && hasRequiredWorkspaceRecovery(updated.resultJson)) {
+      // File recovery belongs to this exact source, even if cancellation let a
+      // newer owner or conversation generation start before copy-back settled.
+      // A resolved no-replay hold can coexist with another active incident; the
+      // one-active-action index must never make us supersede either obligation.
+      const [existing] = await tx.select().from(issueRecoveryActions).where(and(
+        eq(issueRecoveryActions.companyId, run.companyId), eq(issueRecoveryActions.sourceIssueId, task.id),
+        eq(issueRecoveryActions.fingerprint, `legacy-execution:${run.id}`),
+      )).orderBy(desc(issueRecoveryActions.createdAt)).limit(1).for("update");
+      const decision = existing?.evidence.executionReconciliation as Record<string, unknown> | undefined;
+      if (hasRequiredWorkspaceRecovery(existing?.evidence) && decision?.runId === run.id) return updated;
+      const now = new Date();
+      const evidence = {
+        ...existing?.evidence,
+        runId: run.id,
+        originalFailureCode: updated.errorCode,
+        workspaceRestoreFailure: updated.resultJson!.workspaceRestoreFailure,
+        workspaceRestoreRecovery: updated.resultJson!.workspaceRestoreRecovery,
+        executionReconciliation: undefined,
+        continuationDelivery: "invalidated",
+        automaticRecovery: { policy: "preserve_without_replay_v1", runId: run.id, replay: "blocked",
+          actionOutcome: "unknown", recordedAt: now.toISOString() },
+      };
+      const values = { status: "resolved", outcome: "blocked", ownerType: "board", ownerAgentId: null,
+        ownerUserId: null, returnOwnerAgentId: null, evidence, resolvedAt: now, updatedAt: now,
+        wakePolicy: null, monitorPolicy: null,
+        nextAction: "The original sandbox is retained for workspace repair. Verify its exact stop receipt, recover the missing files, and record workspaceRepairEvidence without changing the current task. Repair does not replay the stopped run. Explicitly remove the retained allocation after recovery.",
+      };
+      if (existing) await tx.update(issueRecoveryActions).set(values).where(eq(issueRecoveryActions.id, existing.id));
+      else await tx.insert(issueRecoveryActions).values({ ...values, companyId: run.companyId,
+        sourceIssueId: task.id, kind: "active_run_watchdog", cause: LEGACY_RECOVERY_CAUSE,
+        fingerprint: `legacy-execution:${run.id}`, attemptCount: 1 });
+      return updated;
+    }
     const review = task?.status === "in_review" ? parseIssueExecutionState(task.executionState) : null;
     const isCurrentReviewer = review?.status === "pending" &&
       review.currentParticipant?.type === "agent" && review.currentParticipant.agentId === run.agentId;
@@ -125,7 +188,7 @@ export async function terminalizeLegacyExecution(input: {
       task &&
       !isSupersededConversationRun(task, updated) &&
       (task.assigneeAgentId === run.agentId || isCurrentReviewer) &&
-      !["done", "cancelled"].includes(task.status)
+      (!["done", "cancelled"].includes(task.status) || retainedLeaseIds.length > 0)
     ) {
       // Periodic stranded-work checks may revisit this terminal run before its
       // reconciled continuation is dispatched. Preserve the recorded decision
@@ -135,11 +198,15 @@ export async function terminalizeLegacyExecution(input: {
           eq(issueRecoveryActions.companyId, run.companyId),
           eq(issueRecoveryActions.sourceIssueId, task.id),
           eq(issueRecoveryActions.status, "resolved"),
+          ...(hasRequiredWorkspaceRecovery(updated.resultJson) ? [
+            sql`${issueRecoveryActions.evidence}->'workspaceRestoreRecovery'->>'schema' = ${LEGACY_WORKSPACE_RECOVERY_SCHEMA}`,
+          ] : []),
           or(
             sql`${issueRecoveryActions.evidence}->'executionReconciliation'->>'runId' = ${run.id}`,
             and(
               sql`${issueRecoveryActions.evidence}->>'runId' = ${run.id}`,
-              sql`${issueRecoveryActions.evidence}->>'workspaceRestoreFailure' = 'restore_unsafe_archive'`,
+              or(sql`${issueRecoveryActions.evidence}->>'workspaceRestoreFailure' = 'restore_unsafe_archive'`,
+                sql`${issueRecoveryActions.evidence}->'workspaceRestoreRecovery'->>'schema' = ${LEGACY_WORKSPACE_RECOVERY_SCHEMA}`),
               sql`${issueRecoveryActions.evidence}->'automaticRecovery'->>'replay' = 'blocked'`,
             ),
           ),
@@ -158,10 +225,13 @@ export async function terminalizeLegacyExecution(input: {
           ...(isCurrentReviewer ? { reviewParticipantAgentId: run.agentId } : {}),
           originalFailureCode: updated.errorCode,
           ...(hasWorkspaceRestoreFailure(updated.resultJson) ? { workspaceRestoreFailure: updated.resultJson!.workspaceRestoreFailure } : {}),
+          ...(hasRequiredWorkspaceRecovery(updated.resultJson) ? { workspaceRestoreRecovery: updated.resultJson!.workspaceRestoreRecovery } : {}),
           adapterRecovery: "unsupported_or_unknown",
           attempt: executionFailureRetryCount(run) + 1,
         },
-        nextAction: hasWorkspaceRestoreFailure(updated.resultJson)
+        nextAction: hasRequiredWorkspaceRecovery(updated.resultJson)
+          ? "The original sandbox is retained for workspace repair. Verify that it has stopped, recover the missing files, record workspaceRepairEvidence, and explicitly remove the retained allocation when recovery is complete. A new agent turn cannot prove file recovery."
+          : hasWorkspaceRestoreFailure(updated.resultJson)
           ? "Verify safe workspace staging or repair, then reconcile the stopped run before continuing. Saved work and approval decisions remain in force."
           : "Inspect the stopped provider and recorded actions, then reconcile their outcomes before continuing. This adapter has not established a safe resume checkpoint.",
         maxAttempts: 3,
@@ -237,3 +307,12 @@ export async function settleInterruptedNativeBootstrap(
     return settled;
   });
 }
+
+/** Capture the required-file obligation before unrelated host writes. Keeps
+ * execution ownership while the adapter's host finalization is still running. */
+export async function recordLegacyWorkspaceRestoreFailure(db: Db, run: Run, evidence: Record<string, unknown>) {
+  if (run.runtimeMode !== "legacy" || !hasWorkspaceRestoreFailure(evidence)) return;
+  await terminalizeLegacyExecution({ db, run, status: run.status,
+    recordRestoreFailureOnly: true, patch: { resultJson: evidence },
+  });
+}

server/src/services/legacy-workspace-restore-recovery.ts

@@ -0,0 +1,82 @@
+import { and, eq, inArray, isNotNull, ne, or, sql } from "drizzle-orm";
+import { environmentLeases, environments, heartbeatRuns, type Db } from "@paperclipai/db";
+import { hasWorkspaceRestoreFailure, WORKSPACE_RESTORE_FAILURE_CODES } from "@paperclipai/shared";
+import { prepareSandboxStopAndRetain } from "./sandbox-stop-and-retain.js";
+
+import { hasRequiredWorkspaceRecovery, LEGACY_WORKSPACE_RECOVERY_SCHEMA } from "./workspace-restore-recovery-state.js";
+
+type Run = typeof heartbeatRuns.$inferSelect;
+
+function needsRemoteRestore(run: Run): boolean {
+  return run.runtimeMode === "legacy" && ["failed", "timed_out", "interrupted", "cancelled"].includes(run.status)
+    && hasWorkspaceRestoreFailure(run.resultJson);
+}
+
+function leaseScope(run: Run) {
+  return and(eq(environmentLeases.companyId, run.companyId), eq(environmentLeases.heartbeatRunId, run.id),
+    or(eq(environmentLeases.status, "active"), and(
+      sql`${environmentLeases.metadata}->'workspaceRestoreRecovery'->>'schema' = ${LEGACY_WORKSPACE_RECOVERY_SCHEMA}`,
+      sql`${environmentLeases.metadata}->'workspaceRestoreRecovery'->>'runId' = ${run.id}`)),
+    isNotNull(environmentLeases.providerLeaseId),
+    isNotNull(environmentLeases.provider), ne(environmentLeases.provider, "local"),
+    or(eq(environments.driver, "sandbox"), sql`${environmentLeases.metadata}->>'driver' = 'sandbox'`,
+      sql`${environmentLeases.metadata}->>'sandboxProviderPlugin' = 'true'`));
+}
+
+/** A local lock timeout has no remote source to retain and keeps its old policy. */
+export async function hasUnrestoredRemoteWorkspace(db: Db, run: Run): Promise<boolean> {
+  if (!needsRemoteRestore(run)) return false;
+  if (hasRequiredWorkspaceRecovery(run.resultJson)) return true;
+  const [lease] = await db.select({ id: environmentLeases.id }).from(environmentLeases)
+    .leftJoin(environments, eq(environments.id, environmentLeases.environmentId)).where(leaseScope(run)).limit(1);
+  return !!lease;
+}
+
+/** Call in the terminal-status transaction. No cleanup worker may observe the
+ * failed run without its exact stop-only intent. The original allocation must
+ * not enter warm reuse; it belongs to the board's workspace repair action. */
+export async function preserveLegacyWorkspaceRestoreSources(db: Db, run: Run): Promise<string[]> {
+  if (!needsRemoteRestore(run)) return [];
+  const rows = await db.select({ lease: environmentLeases }).from(environmentLeases)
+    .leftJoin(environments, eq(environments.id, environmentLeases.environmentId))
+    .where(leaseScope(run)).for("update", { of: environmentLeases });
+  const retained: string[] = [];
+  for (const { lease } of rows) {
+    if (lease.status !== "active") { retained.push(lease.id); continue; }
+    // The stop-only worker rechecks exact allocation ownership, including
+    // competing leases, before dispatch. Persist the intent even when cleanup
+    // must wait, so no terminal-run sweep can fall back to destruction.
+    const prepared = await prepareSandboxStopAndRetain(db, lease);
+    if (!prepared) throw new Error("Workspace recovery lease ownership changed before terminal commit.");
+    await db.update(environmentLeases).set({ leasePolicy: "retain_on_failure",
+      metadata: sql`${environmentLeases.metadata} || ${JSON.stringify({ workspaceRestoreRecovery: {
+        schema: LEGACY_WORKSPACE_RECOVERY_SCHEMA, runId: run.id,
+      } })}::jsonb`,
+    }).where(and(
+      eq(environmentLeases.id, prepared.id), eq(environmentLeases.companyId, run.companyId),
+      eq(environmentLeases.heartbeatRunId, run.id), eq(environmentLeases.status, "pending_cleanup"),
+    ));
+    retained.push(lease.id);
+  }
+  return retained;
+}
+
+/** Metadata writers may use safe projections or snapshots from before copy-back
+ * settled. Only the restore recorder owns these fields; preserve their current
+ * database values rather than a stale or schema-only incoming projection. */
+export function preserveWorkspaceRestoreRecoveryMetadataSql(
+  incoming: Record<string, unknown> | null,
+  mergeCurrent = false,
+) {
+  const payload = sql`${JSON.stringify(incoming)}::jsonb`;
+  const result = mergeCurrent
+    ? sql`coalesce(${heartbeatRuns.resultJson}, '{}'::jsonb) || coalesce(nullif(${payload}, 'null'::jsonb), '{}'::jsonb)`
+    : payload;
+  return sql`case when
+    ${heartbeatRuns.resultJson}->'workspaceRestoreRecovery'->>'schema' = ${LEGACY_WORKSPACE_RECOVERY_SCHEMA}
+    and ${inArray(sql`${heartbeatRuns.resultJson}->>'workspaceRestoreFailure'`, [...WORKSPACE_RESTORE_FAILURE_CODES])}
+    then coalesce(nullif(${result}, 'null'::jsonb), '{}'::jsonb) || jsonb_build_object(
+      'workspaceRestoreFailure', ${heartbeatRuns.resultJson}->'workspaceRestoreFailure',
+      'workspaceRestoreRecovery', ${heartbeatRuns.resultJson}->'workspaceRestoreRecovery')
+    else ${result} end`;
+}

server/src/services/recovery/service.ts

@@ -4182,16 +4182,22 @@ export function recoveryService(
       classification,
     );
 
-    await db
+    const classificationMetadata = withAdapterFailureRecoveryClassification(
+      { ...latestRun, resultJson: {} }, classification,
+    ).resultJson;
+    const [updated] = await db
       .update(heartbeatRuns)
       .set({
         errorCode: classifiedRun.errorCode,
-        resultJson: parseObject(classifiedRun.resultJson),
+        // Classification owns retry metadata, not workspace repair receipts
+        // that may have committed after this run was selected.
+        resultJson: sql`coalesce(${heartbeatRuns.resultJson}, '{}'::jsonb) || ${JSON.stringify(classificationMetadata)}::jsonb`,
         updatedAt: new Date(),
       })
-      .where(eq(heartbeatRuns.id, latestRun.id));
+      .where(eq(heartbeatRuns.id, latestRun.id))
+      .returning({ resultJson: heartbeatRuns.resultJson, errorCode: heartbeatRuns.errorCode });
 
-    return classifiedRun;
+    return updated ? { ...classifiedRun, ...updated } : classifiedRun;
   }
 
   function withAdapterFailureRecoveryClassification(

server/src/services/workspace-restore-reconciliation.test.ts

@@ -0,0 +1,157 @@
+import { randomUUID } from "node:crypto";
+import { and, eq } from "drizzle-orm";
+import { afterAll, beforeAll, describe, expect, it } from "vitest";
+import {
+  agents, companies, createDb, environmentLeases, heartbeatRuns,
+  issueComments, issueRecoveryActions, issues,
+} from "@paperclipai/db";
+import type { ExecutionReconciliation } from "@paperclipai/shared";
+import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../__tests__/helpers/embedded-postgres.js";
+import { getExecutionBlocker } from "./execution-blocker.js";
+import { markExecutionReconciliation, validateExecutionReconciliation } from "./execution-recovery-resolution.js";
+import { remoteTerminationReceipt } from "./remote-execution-termination.js";
+import { LEGACY_WORKSPACE_RECOVERY_SCHEMA } from "./workspace-restore-recovery-state.js";
+
+const external = process.env.PAPERCLIP_TEST_DATABASE_URL;
+const support = external ? { supported: true } : await getEmbeddedPostgresTestSupport();
+(support.supported ? describe : describe.skip)("retained workspace repair admission", () => {
+  let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | undefined;
+  let db: ReturnType<typeof createDb>;
+  beforeAll(async () => {
+    if (external) db = createDb(external);
+    else {
+      database = await startEmbeddedPostgresTestDatabase("workspace-restore-reconciliation-");
+      db = createDb(database.connectionString);
+    }
+  }, 30_000);
+  afterAll(async () => { await db?.$client.end(); await database?.cleanup(); });
+
+  async function seed() {
+    const companyId = randomUUID(), agentId = randomUUID(), issueId = randomUUID();
+    const sourceRunId = randomUUID(), leaseId = randomUUID(), actionId = randomUUID();
+    await db.insert(companies).values({ id: companyId, name: "Workspace repair", issuePrefix: `W${companyId.slice(0, 6)}` });
+    await db.insert(agents).values({ id: agentId, companyId, name: "Worker", role: "engineer", adapterType: "claude_local" });
+    await db.insert(issues).values({ id: issueId, companyId, title: "Recover saved work", status: "blocked", assigneeAgentId: agentId });
+    const workspaceRestoreRecovery = { schema: LEGACY_WORKSPACE_RECOVERY_SCHEMA, leaseIds: [leaseId] };
+    await db.insert(heartbeatRuns).values({ id: sourceRunId, companyId, agentId, runtimeMode: "legacy",
+      status: "failed", errorCode: "workspace_restore_failed", contextSnapshot: { issueId },
+      finishedAt: new Date("2000-01-01T00:00:00Z"), runnerProfileJson: { adapterDispatch: { adapterType: "claude_local" } },
+      resultJson: { workspaceRestoreFailure: "restore_failed", conversationContinuation: "continue_conversation_v1", workspaceRestoreRecovery },
+    });
+    const identity = { id: leaseId, companyId, heartbeatRunId: sourceRunId, provider: "daytona", providerLeaseId: `sandbox-${leaseId}` };
+    await db.insert(environmentLeases).values({ ...identity, leasePolicy: "retain_on_failure", status: "released",
+      releasedAt: new Date(), cleanupStatus: "success", metadata: {
+        remoteExecutionTermination: remoteTerminationReceipt(identity, { providerLeaseId: identity.providerLeaseId, state: "stopped" }),
+      },
+    });
+    await db.insert(issueRecoveryActions).values({ id: actionId, companyId, sourceIssueId: issueId,
+      kind: "active_run_watchdog", cause: "legacy_execution_requires_reconciliation", fingerprint: sourceRunId,
+      status: "resolved", outcome: "blocked", nextAction: "Recover the missing workspace files and record repair evidence.",
+      evidence: { runId: sourceRunId, workspaceRestoreFailure: "restore_failed", workspaceRestoreRecovery, automaticRecovery: { replay: "blocked" } },
+    });
+    const decision: ExecutionReconciliation = { runId: sourceRunId, providerStopped: true, actionOutcome: "mixed",
+      outcomeEvidence: "Inspected the saved run and confirmed its external action outcomes.",
+      workspaceRepairEvidence: "Recovered the missing files from the retained sandbox and verified their contents.",
+    };
+    return { companyId, agentId, issueId, sourceRunId, leaseId, actionId, identity, decision };
+  }
+
+  it.each([
+    { scenario: "reset_command", status: "failed" },
+    { scenario: "existing_boundary", status: "failed" },
+    { scenario: "reset_command", status: "cancelled" },
+  ])("keeps a workspace hold through $scenario after $status without offering a task retry", async ({ scenario, status }) => {
+    const f = await seed(), commentId = randomUUID();
+    await db.update(heartbeatRuns).set({ status }).where(eq(heartbeatRuns.id, f.sourceRunId));
+    await db.insert(issueComments).values({ id: commentId, companyId: f.companyId, issueId: f.issueId,
+      authorType: "user", authorUserId: "board", body: "/new" });
+    await db.update(issues).set({ conversationAgentId: f.agentId, conversationUserId: "board", conversationState: "active",
+      ...(scenario === "existing_boundary" ? { conversationBoundaryCommentId: commentId } : {}),
+    }).where(eq(issues.id, f.issueId));
+    const blocker = await getExecutionBlocker(db, f.companyId, f.issueId,
+      scenario === "reset_command" ? { conversationResetCommentId: commentId } : undefined);
+    expect(blocker).toMatchObject({ recoveryActionId: f.actionId, runId: f.sourceRunId,
+      canRetry: false, canContinue: false, workspaceRepairRequired: true, nextAction: "Recover the missing workspace files and record repair evidence." });
+  });
+
+  it("accepts verified repair after exact remote stop without interpreting the sandbox PID on this server", async () => {
+    const f = await seed();
+    await db.update(heartbeatRuns).set({ processPid: process.pid }).where(eq(heartbeatRuns.id, f.sourceRunId));
+    await expect(validateExecutionReconciliation({ db, ...f })).resolves.toMatchObject({ id: f.sourceRunId });
+  });
+
+  it.each(["pending_stop", "wrong_receipt", "missing_source", "unrecorded_source", "empty_sources"])("rejects repair with %s even when releasedAt is populated", async scenario => {
+    const f = await seed();
+    if (scenario === "pending_stop") {
+      await db.update(environmentLeases).set({ status: "pending_cleanup", cleanupStatus: "failed" }).where(eq(environmentLeases.id, f.leaseId));
+    } else if (scenario === "wrong_receipt") {
+      await db.update(environmentLeases).set({ metadata: { remoteExecutionTermination: remoteTerminationReceipt({ ...f.identity, id: randomUUID() },
+        { providerLeaseId: f.identity.providerLeaseId, state: "stopped" }) } }).where(eq(environmentLeases.id, f.leaseId));
+    } else if (scenario === "missing_source") {
+      await db.delete(environmentLeases).where(eq(environmentLeases.id, f.leaseId));
+    } else {
+      const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.sourceRunId));
+      await db.update(heartbeatRuns).set({ resultJson: { ...run.resultJson,
+        workspaceRestoreRecovery: { schema: LEGACY_WORKSPACE_RECOVERY_SCHEMA, leaseIds: scenario === "empty_sources" ? [] : [randomUUID()] },
+      } }).where(eq(heartbeatRuns.id, f.sourceRunId));
+    }
+    await expect(validateExecutionReconciliation({ db, ...f })).rejects.toThrow("has not confirmed that it stopped");
+    expect(await db.select().from(issueRecoveryActions).where(and(eq(issueRecoveryActions.id, f.actionId),
+      eq(issueRecoveryActions.status, "resolved")))).toHaveLength(1);
+  });
+
+  it("requires workspace repair evidence even after stop succeeds", async () => {
+    const f = await seed();
+    await expect(validateExecutionReconciliation({ db, ...f, decision: { ...f.decision, workspaceRepairEvidence: undefined } }))
+      .rejects.toThrow("workspaceRepairEvidence");
+  });
+
+  it("clears the replay hold only after validated repair is recorded, preserving the source and recovery evidence", async () => {
+    const f = await seed();
+    await db.update(heartbeatRuns).set({ processPid: process.pid }).where(eq(heartbeatRuns.id, f.sourceRunId));
+    const repaired = await validateExecutionReconciliation({ db, ...f });
+    expect(await getExecutionBlocker(db, f.companyId, f.issueId)).toMatchObject({ recoveryActionId: f.actionId });
+    const [action] = await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, f.actionId));
+    await markExecutionReconciliation(db, action, f.decision, "board");
+    expect(await getExecutionBlocker(db, f.companyId, f.issueId)).toBeNull();
+    const [recorded] = await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, f.actionId));
+    expect(recorded.evidence).toMatchObject({ workspaceRestoreRecovery: repaired.resultJson!.workspaceRestoreRecovery,
+      executionReconciliation: { runId: f.sourceRunId, workspaceRepairEvidence: f.decision.workspaceRepairEvidence },
+      continuationDelivery: "pending" });
+    const [source] = await db.select().from(environmentLeases).where(eq(environmentLeases.id, f.leaseId));
+    expect(source).toMatchObject({ status: "released", leasePolicy: "retain_on_failure",
+      metadata: { remoteExecutionTermination: { state: "stopped" } } });
+  });
+
+  it("preserves the host process check for local restore failures", async () => {
+    const f = await seed();
+    await db.delete(environmentLeases).where(eq(environmentLeases.id, f.leaseId));
+    await db.update(heartbeatRuns).set({ processPid: process.pid, resultJson: { workspaceRestoreFailure: "restore_lock_timeout" } })
+      .where(eq(heartbeatRuns.id, f.sourceRunId));
+    await expect(validateExecutionReconciliation({ db, ...f })).rejects.toThrow("previous provider is still running");
+  });
+
+  it("does not let a remote stop receipt waive a separate local execution's process check", async () => {
+    const f = await seed();
+    await db.update(heartbeatRuns).set({ processPid: process.pid }).where(eq(heartbeatRuns.id, f.sourceRunId));
+    await db.insert(environmentLeases).values({ companyId: f.companyId, heartbeatRunId: f.sourceRunId,
+      provider: "local", leasePolicy: "ephemeral", status: "released", releasedAt: new Date(), cleanupStatus: "success" });
+    expect(await getExecutionBlocker(db, f.companyId, f.issueId)).toMatchObject({
+      recoveryActionId: null, cause: "execution_owner_active", nextAction: expect.stringContaining("provider process is still running"),
+    });
+    await expect(validateExecutionReconciliation({ db, ...f })).rejects.toThrow("previous provider is still running");
+  });
+
+  it.each(["pending_stop", "wrong_receipt", "missing_source"])("keeps remote ownership blocked with %s even without a host PID", async scenario => {
+    const f = await seed();
+    if (scenario === "pending_stop") {
+      await db.update(environmentLeases).set({ status: "pending_cleanup", cleanupStatus: "failed" }).where(eq(environmentLeases.id, f.leaseId));
+    } else if (scenario === "wrong_receipt") {
+      await db.update(environmentLeases).set({ metadata: { remoteExecutionTermination: remoteTerminationReceipt({ ...f.identity, id: randomUUID() },
+        { providerLeaseId: f.identity.providerLeaseId, state: "stopped" }) } }).where(eq(environmentLeases.id, f.leaseId));
+    } else await db.delete(environmentLeases).where(eq(environmentLeases.id, f.leaseId));
+    expect(await getExecutionBlocker(db, f.companyId, f.issueId)).toMatchObject({
+      recoveryActionId: null, cause: "execution_owner_active", nextAction: expect.stringContaining("has not confirmed that it stopped"),
+    });
+  });
+});

server/src/services/workspace-restore-recovery-state.ts

@@ -0,0 +1,23 @@
+import { hasWorkspaceRestoreFailure } from "@paperclipai/shared";
+
+export const LEGACY_WORKSPACE_RECOVERY_KEY = "workspaceRestoreRecovery";
+export const LEGACY_WORKSPACE_RECOVERY_SCHEMA = "paperclip.workspace-restore-recovery.v1";
+
+export function hasRequiredWorkspaceRecovery(result: Record<string, unknown> | null | undefined): boolean {
+  const value = result?.[LEGACY_WORKSPACE_RECOVERY_KEY];
+  return hasWorkspaceRestoreFailure(result) && !!value && typeof value === "object" && !Array.isArray(value)
+    && (value as Record<string, unknown>).schema === LEGACY_WORKSPACE_RECOVERY_SCHEMA;
+}
+
+
+
+/** Late adapter/projection writes cannot certify recovery or remove its pin. */
+export function preserveWorkspaceRestoreRecoveryMetadata(
+  current: Record<string, unknown> | null | undefined,
+  incoming: Record<string, unknown> | null | undefined,
+): Record<string, unknown> | null | undefined {
+  return hasRequiredWorkspaceRecovery(current) ? { ...incoming,
+    workspaceRestoreFailure: current!.workspaceRestoreFailure,
+    workspaceRestoreRecovery: current!.workspaceRestoreRecovery,
+  } : incoming;
+}

ui/src/components/ExecutionBlockerNotice.test.tsx

@@ -99,6 +99,20 @@ describe("stopped task recovery notice", () => {
     expect(link.getAttribute("href")).toBe("/agents/agent/runs/failed-run");
     expect(agentsApi.retryFailedRun).not.toHaveBeenCalled();
   });
+  it("keeps workspace repair guidance and inspection without offering the legacy Retry fallback", async () => {
+    await act(async () => root.render(<QueryClientProvider client={client}>
+      <ExecutionBlockerNotice companyId="company" issueId="task" onRetried={onRetried} blocker={{
+        recoveryActionId: "repair", runId: "failed-run", agentId: "agent", cause: "legacy_execution_requires_reconciliation",
+        workspaceRepairRequired: true, canRetry: false, canContinue: false,
+        nextAction: "The original sandbox is retained. Recover the missing files and record workspace repair evidence.",
+      }} />
+    </QueryClientProvider>));
+    expect(container.textContent).toContain("Recover the missing files");
+    expect(container.querySelector("button")).toBeNull();
+    expect(container.querySelector("a")?.textContent).toBe("Inspect run");
+    expect(agentsApi.retryFailedRun).not.toHaveBeenCalled();
+  });
+
   it("retries the exact failed run and refreshes the task", async () => {
     vi.mocked(agentsApi.retryFailedRun).mockResolvedValue({} as never);
     await act(async () => container.querySelector<HTMLButtonElement>("button")!.click());

ui/src/components/ExecutionBlockerNotice.tsx

@@ -47,7 +47,7 @@ export function ExecutionBlockerNotice({ companyId, issueId, blocker, onRetried
           <Link to={`/agents/${blocker.agentId}/runs/${blocker.runId}`}>Inspect run</Link>
         </Button>
       )}
-      {(!requiresInspection || blocker.canRetry) && blocker.agentId && blocker.runId &&
+      {!blocker.workspaceRepairRequired && (!requiresInspection || blocker.canRetry) && blocker.agentId && blocker.runId &&
         ((blocker.cause === "legacy_execution_requires_reconciliation" && failedRun) || blocker.canContinue || blocker.canRetry) && (
         <Button variant="outline" size="sm" disabled={retry.isPending} onClick={() => retry.mutate()}>
           {retry.isPending ? "Starting…" : blocker.canContinue ? "Continue" : "Retry"}