fix(lifecycle): align stop proof for pre-dispatch resource waits (#20384)

## Thinking Path > - Paperclip manages AI agents and controls when their work can run. > - An agent pause waits until every earlier run has stopped. > - Recovery already treats verified workspace and AI-capacity waits as pre-dispatch outcomes. > - The stop proof did not accept those same receipts. Historical waits could keep an agent pausing forever. > - This pull request shares the resource-wait classification and the existing execution-event check. > - The benefit is consistent stop evidence without a new lifecycle command, database backfill, or host-boot assumption. ## Linked Issues or Issue Description Refs #18147 and #17718. This fixes the verified resource-wait case of #18147. It does not certify ambiguous historical runs that have no positive non-execution evidence. An agent can have an old cancelled legacy run with `errorCode: workspace_busy`, `executionStage: preparing`, and `executionRecovery: { kind: workspace_wait, providerWorkStarted: false }`. No process, native coordinator, or retained resource exists. Recovery recognizes this as a resource wait, but pause previously required a stop acknowledgment from an executor that never started. ### Credit and related work Thanks to @br1an-c0des (#17692) and @crownlayerapp (#15383) for the focused preparation and shared-admission analysis, @lucasweismann-ai (#18072) for the durable dispatch-evidence and race analysis, and @Pdesengrini (#18905) for the cancellation-receipt and process-identity guard cases. We reviewed these alternatives before implementing this smaller change. No contributor commits were cherry-picked. The design uses receipts already written by core. It does not adopt a reboot-time bypass, a new adapter-entry journal, or a pause-abort route. #17236 and #20037 address the separate cost of history reads. ## What Changed - Add one shared classifier for matching workspace and AI-capacity wait receipts. Use it in legacy recovery and cancellation stop proof. - Reuse the existing dispatch-event predicate instead of maintaining another event list. - Retain checks for native execution references, unresolved workspace obligations, controller leases, and resource cleanup. A live controller or cleanup still in flight cannot pass the proof. - Add a focused PostgreSQL suite. It drives the real pause worker with historical wait records and checks Retry eligibility against the same evidence. - Document the contract. No schema, API, UI, dependency, or scheduling change is required. ## Verification Source checkout on macOS arm64, Node 26.12.0, isolated local PostgreSQL 18, synthetic data only. ```sh node node_modules/vitest/vitest.mjs run --root server \ src/modules/agent-lifecycle/__tests__/resource-wait-stop.test.ts \ src/services/legacy-execution-recovery.test.ts ``` - 61 tests passed across these two files. - All eight positive regressions failed on unchanged master. The fix restores them. - After rebasing onto current master, the 61 focused tests passed again. Another 53 selected tests passed in `explicit-native-continuation.test.ts` with `-t 'pre-dispatch review wait|unclaimed workspace retry|historical native preparation'`. The other 361 cases in that file were not run. - Guard cases cover wrong/missing receipts, dispatch events, process identity, native ownership, live controller leases, workspace recovery, cleanup in flight, unreceipted remote resources, and an unknown historical HTTP dispatch. - A remote lease passes only after its exact termination receipt is stored. - Secret/PII diff scan and `git diff --check` passed. - Standard Greptile reviewed commit `201a61d2bb63290ca9b9b9d524e6c6715a872310`: [5/5 with no actionable findings](https://github.com/paperclipai/paperclip/pull/20384#issuecomment-6753165178). No review threads are unresolved. - Local testing is limited to affected cases. GitHub CI owns broad typecheck, build, and test coverage. Verification setup failures remain accounted for: worktree preflight could not complete because Cargo is unavailable; the first bundled PostgreSQL run failed because its macOS binary lacks a shared library. The focused tests passed on a separate, temporary local PostgreSQL 18 cluster. An early local server typecheck could not resolve the unbuilt Runner package; it is not claimed as a pass. ## Risks - Stop proof now accepts existing positive resource-wait receipts. Missing PIDs, terminal status, elapsed time, and a different boot ID remain insufficient. - The receipt must match its error code. The runtime must remain unresolved, with a preparation or settled stage and no conflicting native or workspace references. - The shared cancellation proof is also used by explicit continuation and Retry. Those paths keep their existing user-intent, ownership, pause, budget, and authorization gates. A stop proof does not itself start another run. - Runs that lack usable evidence remain held. This PR does not claim to repair every run shape reported on #18147. - No migration or live data edit is needed. No hosted deployment is included. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, code execution, PostgreSQL regression tests, and GitHub tools. The exact serving revision, context window, and internal thinking configuration are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-11 05:11:49 +01:00 · 2026-11-09 14:40:14 -08:00
1 parent 9c634e42e4
commit 2550f399d9
@@ -3831,6 +3831,17 @@ provider event, or process-launch evidence may exist. Environment cleanup still
 requires the same receipts. This historical proof permits explicit Retry or a
 newer saved user message; it does not replay the cancelled input.

+Workspace and AI-capacity waits use the same pre-dispatch classification for
+recovery and stop proof. A cancelled legacy run must retain its matching
+`workspace_wait` or `ai_connection_wait` receipt with `providerWorkStarted: false`.
+The runtime must remain unresolved, and the stage must be `preparing` or
+`settled`. Native execution references, workspace recovery obligations, a live
+controller lease, retained resources, or dispatch events still reject the proof.
+Remote cleanup requires the exact termination receipt. This rule also covers
+historical waits without a process ID; neither an old timestamp nor a different
+server boot proves that a run stopped. Pause and terminate can then finish,
+while a subsequent run still requires ordinary admission and new authorization.
+
 The existing bounded saved-message worker rechecks this proof after restart.
 Admission atomically settles an unclaimed coordinator and admits one fresh turn,
 preserving history, unknown action outcomes, and attempt counts. Pauses, approvals,
@@ -0,0 +1,142 @@
+import { randomUUID } from "node:crypto";
+import { eq } from "drizzle-orm";
+import { agents, applyPendingMigrations, closeRegisteredClients, companies, createDb, environmentLeases, heartbeatRunEvents, heartbeatRuns, issues, nativeRunFinalizations, type Db } from "@paperclipai/db";
+import { afterAll, beforeAll, describe, expect, it } from "vitest";
+import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../../../__tests__/helpers/embedded-postgres.js";
+import { resolveExternalTestDatabaseUrl } from "../../../__tests__/helpers/external-test-database.js";
+import { createAgentLifecycle, configureAgentLifecycle } from "../../../services/agent-lifecycle.js";
+import { createLifecycleDriver } from "../../../services/agent-lifecycle-driver.js";
+import { agentExecutionsHaveStopped } from "../../../services/agent-execution-stop.js";
+import { canRetryStoppedRun } from "../../../services/cancelled-native-startup.js";
+import { legacyExecutionNeedsReconciliation } from "../../../services/legacy-execution-recovery.js";
+import { legacyControllerBootId } from "../../../services/legacy-controller-lease.js";
+import { PROCESS_START_REQUESTED } from "../../../services/native-local-process-stop.js";
+import { remoteTerminationReceipt } from "../../../services/remote-execution-termination.js";
+
+const external = resolveExternalTestDatabaseUrl("PAPERCLIP_AGENT_LIFECYCLE_TEST_DATABASE_URL");
+const support = external ? { supported: true } : await getEmbeddedPostgresTestSupport();
+(support.supported ? describe : describe.skip)("pre-dispatch resource wait stop proof", () => {
+  let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | undefined;
+  let db: Db;
+  beforeAll(async () => {
+    if (external) { await applyPendingMigrations(external); db = createDb(external); }
+    else { database = await startEmbeddedPostgresTestDatabase("paperclip-resource-wait-stop-"); db = createDb(database.connectionString); }
+  }, 90_000);
+  afterAll(async () => { if (external) await closeRegisteredClients(external); else await database?.cleanup(); });
+
+  async function fixture(errorCode = "workspace_busy", executionStage = "preparing") {
+    const id = randomUUID();
+    const [company] = await db.insert(companies).values({ name: "Resource wait", issuePrefix: `W${id.slice(0, 7)}` }).returning();
+    const [agent] = await db.insert(agents).values({ companyId: company.id, name: "Migrated agent",
+      adapterType: "paperclip_runner", status: "idle", lifecycleState: "ready" }).returning();
+    const [run] = await db.insert(heartbeatRuns).values({ companyId: company.id, agentId: agent.id,
+      status: "cancelled", invocationSource: "on_demand", errorCode, runtimeMode: "legacy", executionStage,
+      startedAt: new Date("2000-01-01T00:00:00Z"), finishedAt: new Date("2000-01-01T00:00:01Z"),
+      controllerBootId: legacyControllerBootId, controllerLeaseExpiresAt: new Date("2000-01-01T00:01:00Z"),
+      runnerProfileJson: { adapterDispatch: { adapterType: "claude_local" } },
+      resultJson: {
+        executionRecovery: { kind: errorCode === "workspace_busy" ? "workspace_wait" : "ai_connection_wait", providerWorkStarted: false },
+        cancellation: { source: "control_plane", expected: true, initiator: { type: "system" },
+          reason: "Waiting for a resource", recordedAt: "2000-01-01T00:00:01Z" },
+      },
+    }).returning();
+    return { agent, run };
+  }
+
+  it.each(["workspace_busy", "ai_connection_busy", "ai_connection_pool_exhausted", "provider_limit_wait"].flatMap(errorCode =>
+    ["preparing", "settled"].map(stage => [errorCode, stage])))("finishes a pause after a %s wait at %s", async (errorCode, stage) => {
+    const { agent, run } = await fixture(errorCode, stage);
+    const work = configureAgentLifecycle(db, {
+      requiredPluginIds: async () => [], runPlugin: async () => "complete",
+      runHost: createLifecycleDriver(db, {} as never).runHost,
+    });
+    try {
+      expect(legacyExecutionNeedsReconciliation(run)).toBe(false);
+      expect(await canRetryStoppedRun(db, run)).toBe(true);
+      await createAgentLifecycle(db).pauseAgent(agent.id);
+      await work.process(agent.id);
+      const [paused] = await db.select().from(agents).where(eq(agents.id, agent.id));
+      expect(paused).toMatchObject({ status: "paused", lifecycleState: "paused", lifecycleHolds: ["manual"] });
+      expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.agentId, agent.id)))).toHaveLength(1);
+      expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, run.id)))[0].status).toBe("cancelled");
+    } finally { await work.stop(); }
+  });
+
+  it.each([
+    { resultJson: {} },
+    { resultJson: { executionRecovery: { kind: "workspace_wait", providerWorkStarted: true } } },
+    { resultJson: { executionRecovery: { kind: "bootstrap", providerWorkStarted: false } } },
+    { errorCode: "cancelled" }, { executionStage: "dispatching" }, { executionStage: null },
+    { runtimeMode: "native" }, { runtimeModeResolvedAt: new Date("2000-01-01T00:00:00Z") },
+    { processPid: process.pid }, { processStartedAt: new Date("2000-01-01T00:00:00Z") },
+    { nativeSessionId: randomUUID() }, { nativeIssueId: randomUUID() }, { sessionIdAfter: "provider-session" },
+    { controllerLeaseExpiresAt: new Date("2099-01-01T00:00:00Z") },
+    { runnerProfileJson: { nativeExecutionInput: {} } }, { runnerProfileJson: { remoteWorkspace: {} } },
+    { runnerProfileJson: { nativeWorkspaceSync: {} } },
+  ])("rejects contradictory run evidence: %j", async patch => {
+    const { agent, run } = await fixture();
+    await db.update(heartbeatRuns).set(patch).where(eq(heartbeatRuns.id, run.id));
+    const [changed] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, run.id));
+    expect(await canRetryStoppedRun(db, changed)).toBe(false);
+    expect(await agentExecutionsHaveStopped(db, [agent.id])).toBe(false);
+  });
+
+  it.each(["restore_unsafe_archive", "restore_lock_timeout"])("preserves an outstanding %s workspace obligation", async workspaceRestoreFailure => {
+    const { agent, run } = await fixture();
+    const [changed] = await db.update(heartbeatRuns).set({ resultJson: { ...run.resultJson, workspaceRestoreFailure,
+      workspaceRestoreRecovery: { schema: "paperclip.workspace-restore-recovery.v1" } } }).where(eq(heartbeatRuns.id, run.id)).returning();
+    expect(await canRetryStoppedRun(db, changed)).toBe(false);
+    expect(await agentExecutionsHaveStopped(db, [agent.id])).toBe(false);
+  });
+
+  it.each(["adapter.invoke", PROCESS_START_REQUESTED, "session.started", "provider.event", "tool.execution.started", "source_event"])(
+    "retains a wait with a recorded %s event", async eventType => {
+      const { agent, run } = await fixture();
+      await db.insert(heartbeatRunEvents).values({ companyId: run.companyId, agentId: agent.id, runId: run.id,
+        seq: 1, eventType, ...(eventType === "source_event" ? { sourceEventId: "provider-event" } : {}) });
+      expect(await canRetryStoppedRun(db, run)).toBe(false);
+      expect(await agentExecutionsHaveStopped(db, [agent.id])).toBe(false);
+    },
+  );
+
+  it.each([
+    { provider: "local", status: "active" },
+    { provider: "local", status: "pending_cleanup", releasedAt: new Date(0) },
+    { provider: "local", status: "released", cleanupStatus: "failed", releasedAt: new Date(0) },
+    { provider: "local", status: "released", cleanupStatus: "success", releasedAt: new Date(0), metadata: { pendingCleanupInFlight: true } },
+    { provider: "ssh", status: "released", cleanupStatus: "success", releasedAt: new Date(0), providerLeaseId: "synthetic-lease" },
+  ])("requires resource cleanup proof: %j", async lease => {
+    const { agent, run } = await fixture();
+    await db.insert(environmentLeases).values({ companyId: run.companyId, heartbeatRunId: run.id, ...lease });
+    expect(await canRetryStoppedRun(db, run)).toBe(false);
+    expect(await agentExecutionsHaveStopped(db, [agent.id])).toBe(false);
+  });
+
+  it("rejects a native coordinator even without a current owner", async () => {
+    const { agent, run } = await fixture();
+    const [issue] = await db.insert(issues).values({ companyId: run.companyId, title: "Native ownership" }).returning();
+    const [changed] = await db.update(heartbeatRuns).set({ nativeIssueId: issue.id }).where(eq(heartbeatRuns.id, run.id)).returning();
+    await db.insert(nativeRunFinalizations).values({ companyId: run.companyId, runId: run.id, issueId: issue.id, phase: "observed" });
+    expect(await canRetryStoppedRun(db, changed)).toBe(false);
+    expect(await agentExecutionsHaveStopped(db, [agent.id])).toBe(false);
+  });
+
+  it("accepts a remote resource only after its exact termination receipt is saved", async () => {
+    const { agent, run } = await fixture();
+    const [lease] = await db.insert(environmentLeases).values({ companyId: run.companyId, heartbeatRunId: run.id,
+      provider: "ssh", providerLeaseId: "synthetic-remote", status: "released", releasedAt: new Date(0), cleanupStatus: "success" }).returning();
+    expect(await canRetryStoppedRun(db, run)).toBe(false);
+    await db.update(environmentLeases).set({ metadata: { remoteExecutionTermination:
+      remoteTerminationReceipt(lease, { providerLeaseId: lease.providerLeaseId, state: "stopped" }) } }).where(eq(environmentLeases.id, lease.id));
+    expect(await canRetryStoppedRun(db, run)).toBe(true);
+    expect(await agentExecutionsHaveStopped(db, [agent.id])).toBe(true);
+  });
+
+  it("does not mistake an unproven HTTP dispatch for a resource wait", async () => {
+    const { agent, run } = await fixture("cancelled", "dispatching");
+    const [changed] = await db.update(heartbeatRuns).set({ resultJson: null,
+      runnerProfileJson: { adapterDispatch: { adapterType: "http" } } }).where(eq(heartbeatRuns.id, run.id)).returning();
+    expect(await canRetryStoppedRun(db, changed)).toBe(false);
+    expect(await agentExecutionsHaveStopped(db, [agent.id])).toBe(false);
+  });
+});
@@ -1,11 +1,10 @@
 import { isRetainedWorkspaceWorkerStopProof } from "./native-runtime/native-workspace-platform-proof.js";
 import { isPreDispatchReviewWait } from "./pre-dispatch-review-wait.js";
 import { createHash } from "node:crypto";
-import { and, eq, gt, inArray, isNotNull, isNull, lte, or, sql } from "drizzle-orm";
+import { and, eq, gt, inArray, isNotNull, isNull, lte, sql } from "drizzle-orm";
 import { environmentLeases, heartbeatRunEvents, heartbeatRuns, issueTreeHolds, issueTreeHoldMembers, nativeRunFinalizations, nativeRunResults, type Db } from "@paperclipai/db";
 import { isUuidLike } from "@paperclipai/shared";
 import { claimedAdapterType } from "./conversation-continuation.js";
-import { PROCESS_IDENTITY_RECORDED, PROCESS_START_REQUESTED } from "./native-local-process-stop.js";
 import { hasRemoteTerminationReceipt } from "./remote-execution-termination.js";
 import { canContinueCancelledRun, readRunCancellation } from "./run-cancellation.js";
 import { legacyControllerBootId } from "./legacy-controller-lease.js";
@@ -13,7 +12,7 @@ import { hasRequiredWorkspaceRecovery } from "./workspace-restore-recovery-state
 import { hasVerifiedWorkspaceRepair } from "./workspace-recovery-receipt.js";
 import { readFailedPreparationSettlement } from "./failed-preparation-settlement.js";
 import { hasVerifiedPreparationLog } from "./verified-preparation-log.js";
-import { executionStartedEventPredicate, noRetainedExecutionPredicate } from "./pre-dispatch-execution-evidence.js";
+import { executionStartedEventPredicate, isPreDispatchResourceWait, noRetainedExecutionPredicate } from "./pre-dispatch-execution-evidence.js";

 type Run = Pick<typeof heartbeatRuns.$inferSelect,
  "id" | "companyId" | "status" | "startedAt" | "finishedAt" | "runtimeMode" |
@@ -261,9 +260,14 @@ export async function isCancelledNativeStartup(db: Db, run: Run, coordinator: Co
    (run.resultJson === null || cancellation?.beforeNativeSelection === true) &&
    Boolean(run.controllerBootId && run.controllerBootId !== legacyControllerBootId &&
      run.controllerLeaseExpiresAt && run.controllerLeaseExpiresAt <= new Date());
-  const beforeReviewDispatch = isPreDispatchReviewWait(run) && !coordinator;
+  const beforeDispatch = !coordinator && (isPreDispatchReviewWait(run) ||
+    isPreDispatchResourceWait(run) && !run.runtimeModeResolvedAt && !run.nativeIssueId && !run.nativeSessionId &&
+    ["preparing", "settled"].includes(run.executionStage ?? "") &&
+    run.runnerProfileJson?.nativeExecutionInput == null && run.runnerProfileJson?.nativeWorkspaceSync == null &&
+    run.runnerProfileJson?.remoteWorkspace == null && run.resultJson?.workspaceRestoreFailure == null &&
+    !hasRequiredWorkspaceRecovery(run.resultJson));
  const unclaimedWorkspaceRetry = !coordinator && await isCancelledUnclaimedWorkspaceWaitRetry(db, run);
-  const beforeSelection = beforeReviewDispatch || unclaimedWorkspaceRetry || historicalBeforeSelection || run.runtimeMode === "legacy" && !run.runtimeModeResolvedAt &&
+  const beforeSelection = beforeDispatch || unclaimedWorkspaceRetry || historicalBeforeSelection || run.runtimeMode === "legacy" && !run.runtimeModeResolvedAt &&
    !run.nativeSessionId && !coordinator && claimedAdapterType(run) === "paperclip_runner" &&
    cancellation?.beforeNativeSelection === true;
  const neverClaimed = run.runtimeMode === "native" && coordinator &&
@@ -272,16 +276,16 @@ export async function isCancelledNativeStartup(db: Db, run: Run, coordinator: Co
    !coordinator.controllerPid && !coordinator.leaseOwner && !coordinator.leaseExpiresAt &&
    !coordinator.resultId && !coordinator.failureDetail?.successorRunId;
  if (!beforeSelection && !neverClaimed) return false;
-  const settled = beforeReviewDispatch || unclaimedWorkspaceRetry || typeof run.resultJson?.startupPreparationSettledAt === "string";
+  const settled = beforeDispatch || unclaimedWorkspaceRetry || typeof run.resultJson?.startupPreparationSettledAt === "string";
  // The old preparer can still be unwinding even though the run is terminal.
-  if (!settled && run.controllerLeaseExpiresAt && run.controllerLeaseExpiresAt > new Date()) return false;
+  if (run.controllerLeaseExpiresAt && run.controllerLeaseExpiresAt > new Date()) return false;
  const leases = await db.select().from(environmentLeases).where(and(
    eq(environmentLeases.companyId, run.companyId), eq(environmentLeases.heartbeatRunId, run.id),
  ));
  if ((!settled && leases.length === 0 && !historicalBeforeSelection) || leases.some(lease =>
-    lease.provider === "local"
+    lease.metadata?.pendingCleanupInFlight === true || (lease.provider === "local"
      ? !lease.releasedAt || lease.status === "pending_cleanup" || lease.cleanupStatus === "failed"
-      : !hasRemoteTerminationReceipt(lease))) return false;
+      : !hasRemoteTerminationReceipt(lease)))) return false;
  // Reject contradictory retained evidence, including a crash after a launch
  // request but before the PID callback. Provider events never certify a stop.
  return !await nativeStartupHasExecutionEvidence(db, run);
@@ -290,10 +294,7 @@ export async function isCancelledNativeStartup(db: Db, run: Run, coordinator: Co
 async function nativeStartupHasExecutionEvidence(db: Db, run: Run) {
  const [execution] = await db.select({ id: heartbeatRunEvents.id }).from(heartbeatRunEvents).where(and(
    eq(heartbeatRunEvents.companyId, run.companyId), eq(heartbeatRunEvents.runId, run.id),
-    or(isNotNull(heartbeatRunEvents.sourceEventId),
-      inArray(heartbeatRunEvents.eventType, ["adapter.invoke", PROCESS_START_REQUESTED, PROCESS_IDENTITY_RECORDED,
-        "harness.ready", "session.started", "session.resumed", "session.updated", "turn.started",
-        "provider.event", "provider.rpc_result", "tool.execution.started"])),
+    executionStartedEventPredicate(),
  )).limit(1);
  return Boolean(execution);
 }
@@ -20,7 +20,7 @@ import { canonicalExecutionUuid } from "./execution-identity.js";
 import { randomUUID } from "node:crypto";
 import { and, desc, eq, getTableColumns, inArray, isNull, or, sql, type SQL } from "drizzle-orm";
 import { environmentLeases, heartbeatRunEvents, heartbeatRuns, issueRecoveryActions, issues, nativeRunFinalizations, type Db } from "@paperclipai/db";
-import { executionStartedEventPredicate } from "./pre-dispatch-execution-evidence.js";
+import { executionStartedEventPredicate, isPreDispatchResourceWait } from "./pre-dispatch-execution-evidence.js";
 import { readRunCancellation } from "./run-cancellation.js";
 import { hasRemoteTerminationReceipt } from "./remote-execution-termination.js";
 import { issueRecoveryActionService } from "./issue-recovery-actions.js";
@@ -77,10 +77,7 @@ export function legacyExecutionNeedsReconciliation(
      && (run.resultJson?.executionCancellation as Record<string, unknown> | undefined)?.state === "acknowledged") return false;
  // Waiting for a subscription or workspace precedes provider execution. It is
  // a resource wait, not a failed provider attempt or permission to replay work.
-  if (run.status === "cancelled" && (run.errorCode === "ai_connection_busy" || run.errorCode === "ai_connection_pool_exhausted" || run.errorCode === "provider_limit_wait") &&
-      evidence?.kind === "ai_connection_wait" && evidence.providerWorkStarted === false) return false;
-  if (run.status === "cancelled" && run.errorCode === "workspace_busy" &&
-      evidence?.kind === "workspace_wait" && evidence.providerWorkStarted === false) return false;
+  if (isPreDispatchResourceWait(run)) return false;
  // Setup owns the bounded retry budget for temporary workspace scans. Its
  // exhaustion needs workspace repair, not reconciliation of provider actions
  // that the bootstrap evidence proves never started. Keep unknown outcomes held.
@@ -3,6 +3,15 @@ import { environmentLeases, heartbeatRunEvents, heartbeatRuns, nativeRunFinaliza
 import { PROCESS_IDENTITY_RECORDED, PROCESS_START_REQUESTED } from "./native-local-process-stop.js";
 import { LEGACY_WORKSPACE_RECOVERY_SCHEMA } from "./workspace-restore-recovery-state.js";

+export function isPreDispatchResourceWait(run: Pick<typeof heartbeatRuns.$inferSelect,
+  "runtimeMode" | "status" | "errorCode" | "resultJson">): boolean {
+  const evidence = run.resultJson?.executionRecovery as Record<string, unknown> | undefined;
+  return run.runtimeMode === "legacy" && run.status === "cancelled" && evidence?.providerWorkStarted === false &&
+    (run.errorCode === "workspace_busy" && evidence.kind === "workspace_wait" ||
+      ["ai_connection_busy", "ai_connection_pool_exhausted", "provider_limit_wait"].includes(run.errorCode ?? "") &&
+      evidence.kind === "ai_connection_wait");
+}
+
 export function executionStartedEventPredicate() {
  return or(isNotNull(heartbeatRunEvents.sourceEventId), inArray(heartbeatRunEvents.eventType,
    ["adapter.invoke", PROCESS_START_REQUESTED, PROCESS_IDENTITY_RECORDED, "harness.ready", "session.started",