feat(server): adopt running remote agents through durable controller checkpoints (#17439)

## Thinking Path

> - Paperclip must keep admitted remote work alive while its main server
is replaced.
> - Runner owns recovery mechanics and exposes opaque checkpoints
through #17097.
> - The server owns company scope, encryption, Stop, tool authority and
execution ownership.
> - A successor must prove current ownership before it adopts the
original execution.
> - Checkpoint bursts must also leave capacity for normal requests and
lease renewal.
> - This PR persists and adopts that authority through the existing
runtime boundaries.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Remote agents retain the original execution during a bounded main-server
outage.

**Current behavior**

A server restart can lose the controller's local state and connection.
An unsafe replacement could repeat a turn or an uncertain mutation.
Checkpoint writes in the request pool can also delay ordinary requests
during an aligned burst.

**Proposed behavior**

Restore encrypted control records from Postgres and adopt the original
runner under a fenced lease. Recover known tool receipts through current
tool authority. Give control checkpoints a shared pool capped at two
connections per database owner.

Depends on #17097. Successors are #17455 and #17456. Qualification and
presentation remain separate review slices.

## What Changed

- Add company/run-bound encrypted checkpoint records and migration 0394.
Keep the SQL idempotent after renumbering against current master.
- Recover startup and expired ownership through native runtime and
heartbeat. Preserve the sandbox lease, admitted runner artifact, harness
configuration, native session and active turn.
- Keep Stop, finalization, preparation ownership and stale-writer
fencing transactional. Preserve graceful detach and natural crash
takeover.
- Recover committed tool receipts through the existing authority.
Reauthorize reads and hold uncertain mutations.
- Bound startup retention with the existing coordinator and preserve
zero idle scans.
- Share at most two checkpoint connections per database client. Keep
application and liveness capacity separate. Close the checkpoint pool
during shutdown; late writes cannot reopen it.
- Count SQL across both application and checkpoint clients. Prove
request/liveness progress, command recovery and failure after pool
shutdown with real Postgres.

## Verification

Project creation recovery now uses the existing project activity receipt
owner. It retains authenticated input normalization, fingerprint checks
and current run authority after the receipt lock. An absent receipt
returns no result before mutation. The new lost-HTTP-response and
no-effect regression uses real API routes and Postgres. The seven
transport cases pass locally. This Mac skips all 20 database cases. On
current source `5de803c2cd`, Linux [job
122593415890](https://github.com/paperclipai/paperclip/actions/runs/40620576702/job/122593415890)
executes all 20 project-tool cases successfully (9.480 seconds),
including the committed-project lost-response regression. Its shard
passes 1,627 cases with three unrelated skips. Server typecheck passes.
The transport regression initially has two failures; a subsequent
incorrect expectation about schema defaults was corrected before the
final pass. Ordinary execution adds no checkpoint writes or recurring
work.

Current candidate `5de803c2cd` includes the project-creation receipt
recovery fix and master `2cd747d1b6`. Migration 0394 retains the prior
idempotent checkpoint DDL. Every master SQL file and journal entry is
preserved. Snapshots 0390–0394, migration checks and server typecheck
pass.

The ordinary warm-turn regression fails on its second turn before
`a219ab51ad`. After correction, three completed warm turns select the
correct prior checkpoint and preserve the original session. All 288
provider-wiring cases pass. The control plane, external persistence
callback, identity rotation and filesystem are real; the backend and
database-load seam are scripted. The 13 warm admission/configuration
variants also pass after final master integration. This does not qualify
an underlying harness.

At current source `5de803c2cd`, Linux [job
122593415907](https://github.com/paperclipai/paperclip/actions/runs/40620576702/job/122593415907)
passes all 940 native-session-executor cases (61.431 seconds), including
the warm external-checkpoint behavior; the shard passes 2,358 cases with
one skip. Current [Runner job
122593415475](https://github.com/paperclipai/paperclip/actions/runs/40620576702/job/122593415475)
passes the native recovery lane: 115 passes and five container-only
skips, including the encrypted restore guard, fresh preparation (63.189
seconds), blocked claim lock (1.226 seconds) and recovery beyond 101
blocked owners (3.897 seconds). The first lane passes 2,215 cases with
one skip. The SQL cost test reports zero idle and streaming SQL for both
output volumes; this is component evidence, not a production capacity
claim. At prior candidate `1ff7bb44f4`, Linux job 122549307215 passes
all 938 native-executor cases, including the three-turn
external-checkpoint regression. Job 122549306469 passes the native
recovery lane with 115 passes and five container-only skips, including
the encrypted-restore session guard, fresh preparation (63.664 seconds),
claim-lock contention (3.961 seconds), and 101-owner recovery (3.824
seconds). This Mac skips the real-Postgres suite. The earlier full local
executor attempt has 921 passes and 16 failures during embedded-Postgres
initialization before assertions. The original fixture collisions and
their corrections remain recorded.

At earlier candidate `8fbe62b41d`, server shard job 122535673841 has
3,876 passes, one failure and one skip in an unchanged expiry-worker
retry/drain timing test. The same server/database tree on qualification
source `b64cac6951` passes all 20 expiry cases and all 3,877 shard
cases. The single diagnostic retry, job 122541891915, passes all 20
expiry cases and all 3,877 shard cases (one skip). No assertion was
weakened. The original failure remains recorded and its exact timing
race is not proven. All Actions checks and legacy status passed on
`8fbe62b41d`, with completed Greptile 5/5 and no open feedback. The next
candidate `1ff7bb44f4` also passed all Actions checks and Greptile 5/5.
Its Paperclip Review remained queued before the new master conflict
required integration. The new candidate needs fresh exact-head CI and
reviews; old checks do not qualify it. Migration generation is
checkpoint-only, all master SQL and journal entries remain unchanged,
and migration checks, server typecheck and all 13 warm-owner variants
pass after integration. The first migration check was blocked by TSX
sandbox IPC; its permitted repeat passes. The first warm-test command
used an incorrect path and ran no tests; the corrected command supplies
the 13-case receipt.

All 16 database pool-lifetime/options/teardown cases pass after granting
loopback permission; the first batch had two EPERM failures and 14
passes. Earlier semantic-tool/HTTP recovery, adoption, encryption-batch,
zero-streaming-write and idle-SQL receipts remain source-attributed in
the [qualification
record](https://github.com/paperclipai/paperclip/blob/2ff66f57da0da35bac1c19a2366ea4b84d87a986/doc/plans/2026-10-22-hot-upgrade-qualification.md).
That record retains earlier assertion, setup and cleanup failures and
their corrections. No failed gate is waived.

The historical 300-session component campaign passes four after phases:
steady read p99 is 107–108 ms at 500 reads/s and 100 writes/s;
aligned-burst read p99 is 103–105 ms after the bounded pool, versus
584–689 ms before. This comparison isolates the persistence corrections.
It does not measure the final stack's hosted active-agent capacity. The
hosted zero-agent API pair leaves SQL and CPU essentially unchanged, but
both arms fail absolute latency gates and candidate peak cgroup memory
fails its gate. The active-agent/viewer matrix, different-artifact
upgrade and rollback remain open.

Run `PAPERCLIP_TEST_DATABASE_URL=<owned-disposable-postgres> pnpm
--filter @paperclipai/server exec vitest run
src/services/native-runtime/native-runner-restart-recovery.integration.test.ts`.

Previous head `f2db8a6568` had one failed workspace setup gate: [job
122555672441](https://github.com/paperclipai/paperclip/actions/runs/40608984238/job/122555672441)
passes all 4,169 UI tests and 285 CLI cases (one skip), but the company
import/export suite reaches its 30-second health deadline during doctor
startup. Its output does not reach the server-start step. The CLI,
server, database, shared, adapter and plugin trees are identical on
qualification source `a866619d7b`, whose [job
122555886663](https://github.com/paperclipai/paperclip/actions/runs/40609057937/job/122555886663)
passes that fixture and all 286 CLI cases. The one diagnostic retry,
[job
122561057225](https://github.com/paperclipai/paperclip/actions/runs/40608984238/job/122561057225),
passes the formerly failing fixture and all 286 CLI cases. All Actions
checks and legacy status on `f2db8a6568` pass. Its exact-head Greptile
completed at 5/5 with no open feedback. The current integrated head
requires fresh checks and a completed Paperclip Review before merge. The
original failure remains recorded; its exact timing cause is unproven. A
repeated failure requires investigation and repair rather than another
retry.

All current-head CI, legacy status, exact-head completed Greptile 5/5,
resolved feedback and ordinary GitHub merge eligibility remain required
before merge.

The latest integration preserves master’s published Copilot v70 profiles
and every published migration. Checkpoint migration
`0394_conscious_whizzer.sql` was generated with `pnpm db:generate` and
retains the prior idempotent DDL. The Runner vendor rebuild and replay
parity checks pass. Server typecheck and all 13 warm-owner variants pass
after that rebuild. The imported report-lock patch is byte-identical to
master; its intentional patch-context whitespace is retained, and the
feature delta passes whitespace checks. No hosted experiment was run.

## Risks

The pool adds up to two active connections per instance, shared across
agents. Checkpoint delivery can wait behind that bound; it still
requires its commit. Steady component-test Postgres CPU rises by about
1.7 percentage points of a two-CPU allocation. These corrections do not
remove the stack's control and renewal writes. The migration and key
must survive replacement. Fresh-database tests and local load do not
establish different-artifact hosted upgrade or rollback continuity.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools and code
execution. The runtime does not expose a more specific model ID or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

server/src/__tests__/chat-project-tools.test.ts

@@ -1,8 +1,9 @@
+import { PaperclipRunnerToolAuthority } from "../services/native-runtime/paperclip-runner-tool-authority.js";
 import { projectService } from "../services/projects.js";
 import { callProjectTool } from "../services/project-tools.js";
 import { createLocalAgentJwt } from "../agent-auth-jwt.js";
 import { randomUUID } from "node:crypto";
-import { afterAll, beforeAll, describe, expect, it } from "vitest";
+import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
 import { eq } from "drizzle-orm";
 import { agents, issues, heartbeatRuns, projects, companyMemberships, executionWorkspaces } from "@paperclipai/db";
 import { startRunnerApiTestServer } from "./helpers/runner-api-server.js";
@@ -359,6 +360,57 @@ const support = await getEmbeddedPostgresTestSupport();
     await expect(call(f, "create_project", { ...input, name: "Changed" })).rejects.toThrow(/different inputs/);
   });
 
+  it("recovers a committed project after losing its HTTP response without repeating the effect", async () => {
+    const f = await server.fixture({ conversation: true, disableWakeOnDemand: true });
+    const request = { tool: "create_project", arguments: { name: "Surviving project", idempotencyKey: "lost-response" }, callId: randomUUID() };
+    const realFetch = globalThis.fetch;
+    const lostResponse = vi.spyOn(globalThis, "fetch").mockImplementation(async (url, options) => {
+      const response = await realFetch(url, options);
+      if (response.status === 201 && String(url).endsWith(`/companies/${f.companyId}/projects`)) {
+        await response.arrayBuffer();
+        throw new Error("Controller lost the committed response");
+      }
+      return response;
+    });
+    try {
+      await expect(f.authority.execute(request)).rejects.toThrow(/lost the committed response/);
+    } finally { lostResponse.mockRestore(); }
+    const committed = await f.snapshot();
+    const created = committed.projects.find(project => project.name === request.arguments.name)!;
+    expect(created).toBeDefined();
+    const recovered = await Promise.all(Array.from({ length: 4 }, () => new PaperclipRunnerToolAuthority(server.db, f).recover(request)));
+    expect(recovered).toEqual(Array.from({ length: 4 }, () => ({ result: expect.objectContaining({ id: created.id }) })));
+    await expect(new PaperclipRunnerToolAuthority(server.db, f).recover({ ...request, arguments: { ...request.arguments, name: "Changed" } })).rejects.toThrow(/different inputs/);
+    const after = await f.snapshot();
+    expect(after.projects.map(project => project.id).sort()).toEqual(committed.projects.map(project => project.id).sort());
+    expect(after.activity.filter(event => event.action === "project.created")).toHaveLength(1);
+    const foreign = await server.fixture({ disableWakeOnDemand: true });
+    expect(await foreign.authority.recover(request)).toBeNull();
+    await server.db.update(heartbeatRuns).set({ status: "cancelled" }).where(eq(heartbeatRuns.id, f.runId));
+    await expect(new PaperclipRunnerToolAuthority(server.db, f).recover(request)).rejects.toThrow();
+  });
+
+  it("holds an unknown project mutation without creating a project or audit event", async () => {
+    const f = await server.fixture({ disableWakeOnDemand: true });
+    const before = await f.snapshot();
+    const request = { tool: "create_project", arguments: { name: "Uncertain project", idempotencyKey: "unknown" }, callId: randomUUID() };
+    expect(await f.authority.recover(request)).toBeNull();
+    const token = createLocalAgentJwt(f.agentId, f.companyId, "paperclip_runner", f.runId, f.responsibleUserId)!;
+    const response = await fetch(`${server.apiUrl}/api/companies/${f.companyId}/projects`, {
+      method: "POST", headers: { Authorization: `Bearer ${token}`, "Content-Type": "application/json", "X-Paperclip-Receipt-Only": "true" },
+      body: JSON.stringify(request.arguments),
+    });
+    expect(response.status).toBe(204);
+    const missingKey = await fetch(`${server.apiUrl}/api/companies/${f.companyId}/projects`, {
+      method: "POST", headers: { Authorization: `Bearer ${token}`, "Content-Type": "application/json", "X-Paperclip-Receipt-Only": "true" },
+      body: JSON.stringify({ name: "Missing receipt key" }),
+    });
+    expect(missingKey.status).toBe(422);
+    const after = await f.snapshot();
+    expect(after.projects.map(project => project.id).sort()).toEqual(before.projects.map(project => project.id).sort());
+    expect(after.activity.filter(event => event.action === "project.created")).toEqual([]);
+  });
+
   it("includes an explicit workspace repository in the committed project card", async () => {
     const f = await server.fixture({ conversation: true });
     const project = await call(f, "create_project", { name: "Workspace repo", workspace: { repoUrl: "https://github.com/example/web" }, idempotencyKey: "workspace" }) as any;

server/src/__tests__/environment-run-orchestrator.test.ts

@@ -880,6 +880,22 @@ describe("live remote runner lease reattachment", () => {
     expect(result.leaseContext.executionWorkspaceId).toBe("ew-1");
     expect(result.environment.config.reuseLease).toBe(false);
   });
+  it.each(["active", "foreign-company", "revoked", "undispatched", "stopped"])("inspects the original plugin task lease (%s)", async state => {
+    mockGetEnvironment.mockResolvedValue(makeEnvironment("plugin"));
+    const lease = originalLease();
+    lease.metadata = { agentId: "agent-1", runnerTaskLease: true,
+      runnerTaskDispatched: state !== "undispatched", runnerTaskExecutionStopped: state === "stopped" };
+    if (state === "foreign-company") lease.companyId = "foreign";
+    if (state === "revoked") lease.status = "released";
+    mockGetLease.mockResolvedValue(lease);
+    const runtime = makeMockRuntime();
+    const acquired = environmentRunOrchestrator({} as never, { environmentRuntime: runtime }).acquireForRun({
+      ...input, reattachRemoteLease: { kind: "plugin_task", leaseId: lease.id },
+    });
+    if (state === "active") await expect(acquired).resolves.toMatchObject({ lease });
+    else await expect(acquired).rejects.toThrow("native_remote_recovery_lease_mismatch");
+    expect(runtime.acquireRunLease).not.toHaveBeenCalled();
+  });
   it.each([
     { companyId: "foreign" }, { environmentId: "foreign" }, { heartbeatRunId: "another-run" },
     { issueId: "another-task" }, { executionWorkspaceId: "another-workspace" },

server/src/__tests__/helpers/recovery-postgres.ts

@@ -0,0 +1,27 @@
+import { randomUUID } from "node:crypto";
+import { applyPendingMigrations, closeRegisteredClients, createDb } from "@paperclipai/db";
+import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./embedded-postgres.js";
+
+export const recoveryPostgresSupport = process.env.PAPERCLIP_TEST_DATABASE_URL
+  ? { supported: true } : await getEmbeddedPostgresTestSupport();
+
+/** Optional external host, always a fresh owned database; never reuse its data. */
+export async function startRecoveryPostgres(prefix: string) {
+  const external = process.env.PAPERCLIP_TEST_DATABASE_URL;
+  if (!external) return startEmbeddedPostgresTestDatabase(prefix);
+  const databaseName = `recovery_${randomUUID().replaceAll("-", "")}`;
+  const admin = createDb(external);
+  try { await admin.$client.unsafe(`create database "${databaseName}"`); }
+  finally { await admin.$client.end(); }
+  const url = new URL(external); url.pathname = `/${databaseName}`;
+  const connectionString = url.toString();
+  const cleanup = async () => {
+    await closeRegisteredClients(connectionString, { scope: "database" });
+    const owner = createDb(external);
+    try { await owner.$client.unsafe(`drop database "${databaseName}"`); }
+    finally { await owner.$client.end(); }
+  };
+  try { await applyPendingMigrations(connectionString); }
+  catch (error) { await cleanup().catch(() => undefined); throw error; }
+  return { connectionString, cleanup };
+}

server/src/__tests__/helpers/runner-api-server.ts

@@ -11,7 +11,7 @@ import { agents, authUsers, authAccounts, companies, companyMemberships, createD
 import { documentService } from "../../services/documents.js";
 import { connectionIntentService } from "../../services/connection-intents.js";
 import { initializeRunIdentity } from "../../services/run-identity.js";
-import { startEmbeddedPostgresTestDatabase } from "./embedded-postgres.js";
+import { startRecoveryPostgres } from "./recovery-postgres.js";
 import { createApp } from "../../app.js";
 import { createLocalDiskStorageProvider } from "../../storage/local-disk-provider.js";
 import { createStorageService } from "../../storage/service.js";
@@ -30,8 +30,8 @@ export async function startRunnerApiTestServer(options: {
 } = {}) {
   const root = await mkdtemp(join(tmpdir(), "paperclip-api-eval-"));
   const temporary = options.databaseUrl
-    ? { connectionString: options.databaseUrl, cleanup: () => closeRegisteredClients(options.databaseUrl!) }
-    : await startEmbeddedPostgresTestDatabase("paperclip-api-eval-db-");
+    ? { connectionString: options.databaseUrl, cleanup: () => closeRegisteredClients(options.databaseUrl!, { scope: "database" }) }
+    : await startRecoveryPostgres("paperclip-api-eval-db-");
   if (options.databaseUrl) await applyPendingMigrations(options.databaseUrl);
   const db = createDb(temporary.connectionString);
   const storage = createStorageService(createLocalDiskStorageProvider(join(root, "storage")));

server/src/__tests__/local-encrypted-cipher.test.ts

@@ -0,0 +1,63 @@
+import { randomBytes } from "node:crypto";
+import { mkdtempSync, rmSync, unlinkSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
+import { createLocalEncryptedCipher, localEncryptedProvider } from "../secrets/local-encrypted-provider.js";
+
+describe("operation-scoped local encryption", () => {
+  let root: string;
+  let keyFile: string;
+
+  beforeEach(() => {
+    root = mkdtempSync(join(tmpdir(), "paperclip-encryption-batch-"));
+    keyFile = join(root, "master.key");
+    vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY", "");
+    vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY_FILE", keyFile);
+  });
+
+  afterEach(() => {
+    vi.unstubAllEnvs();
+    rmSync(root, { recursive: true, force: true });
+  });
+
+  it("encrypts and recovers 4096 records without reading the key again within a batch", () => {
+    const writer = createLocalEncryptedCipher("write");
+    const reader = createLocalEncryptedCipher("read");
+    unlinkSync(keyFile);
+    const records = Array.from({ length: 4096 }, (_, i) => writer.encrypt(`control-record-${i}`));
+    expect(new Set(records.map(record => record.iv)).size).toBe(records.length);
+    expect(records.map(record => reader.decrypt(record))).toEqual(
+      Array.from({ length: 4096 }, (_, i) => `control-record-${i}`),
+    );
+    expect(() => createLocalEncryptedCipher("read")).toThrow("Secrets master key file is missing");
+  });
+
+  it("reloads a replaced key for the next operation and keeps decryption failures typed", () => {
+    const old = createLocalEncryptedCipher("write");
+    const material = old.encrypt("original control state");
+    writeFileSync(keyFile, randomBytes(32).toString("base64"), { mode: 0o600 });
+    const replacement = createLocalEncryptedCipher("read");
+    expect(old.decrypt(material)).toBe("original control state");
+    expect(() => replacement.decrypt(material)).toThrow(expect.objectContaining({
+      code: "provider_unavailable", provider: "local_encrypted", operation: "resolveVersion",
+    }));
+  });
+
+  it("reads existing provider material and writes material the existing provider can read", async () => {
+    const original = await localEncryptedProvider.createVersion({ value: "existing secret" });
+    const batch = createLocalEncryptedCipher("read");
+    expect(batch.decrypt(original.material)).toBe("existing secret");
+    await expect(localEncryptedProvider.resolveVersion({
+      material: batch.encrypt("checkpoint part"), externalRef: null,
+    })).resolves.toBe("checkpoint part");
+    await expect(localEncryptedProvider.resolveVersion({ material: {}, externalRef: null }))
+      .rejects.toMatchObject({ code: "provider_unavailable", operation: "resolveVersion" });
+  });
+
+  it("never creates a replacement key when recovering a missing key", () => {
+    expect(() => createLocalEncryptedCipher("read")).toThrow(expect.objectContaining({
+      code: "provider_unavailable", operation: "resolveVersion",
+    }));
+  });
+});

server/src/__tests__/native-harness-backup-expiration.test.ts

@@ -2,7 +2,7 @@ import { createHash, randomUUID } from "node:crypto";
 import os from "node:os";
 import path from "node:path";
 import { promises as fs } from "node:fs";
-import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest";
+import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
 import { eq } from "drizzle-orm";
 import {
   agents,
@@ -13,14 +13,13 @@ import {
   heartbeatRuns,
   issues,
   nativeRunFinalizations,
+  nativeRunnerCheckpoints,
 } from "@paperclipai/db";
-import {
-  getEmbeddedPostgresTestSupport,
-  startEmbeddedPostgresTestDatabase,
-} from "./helpers/embedded-postgres.js";
+import { recoveryPostgresSupport as embeddedPostgresSupport, startRecoveryPostgres } from "./helpers/recovery-postgres.js";
 import { nativeHarnessBackupExpirationService } from "../services/native-runtime/native-harness-backup-expiration.ts";
+import { createDeliveryWorkCoordinator } from "../services/delivery-work-coordinator.js";
+import { logger } from "../middleware/logger.js";
 
-const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
 const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
 
 if (!embeddedPostgresSupport.supported) {
@@ -37,12 +36,12 @@ async function exists(target: string) {
 
 describeEmbeddedPostgres("native harness backup expiration", () => {
   let db!: ReturnType<typeof createDb>;
-  let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
+  let tempDb: Awaited<ReturnType<typeof startRecoveryPostgres>> | null = null;
   let stateBase = "";
   const now = new Date();
 
   beforeAll(async () => {
-    tempDb = await startEmbeddedPostgresTestDatabase("paperclip-native-backup-expiration-");
+    tempDb = await startRecoveryPostgres("paperclip-native-backup-expiration-");
     db = createDb(tempDb.connectionString);
   }, 20_000);
 
@@ -134,7 +133,7 @@ describeEmbeddedPostgres("native harness backup expiration", () => {
         metadata: { nativeHarnessBackup: { sessionScopeSha256: scope } },
       });
     }
-    return { companyId, agentId: agent!.id, issueId: issue!.id, sessionId, scope, root, current };
+    return { companyId, agentId: agent!.id, issueId: issue!.id, runId: run!.id, sessionId, scope, root, current };
   }
 
   function service() {
@@ -150,6 +149,187 @@ describeEmbeddedPostgres("native harness backup expiration", () => {
     return (await db.select().from(agentTaskSessions).where(eq(agentTaskSessions.agentId, agentId))).length;
   }
 
+  async function seedCheckpoint(session: Awaited<ReturnType<typeof seedSession>>) {
+    await db.insert(nativeRunnerCheckpoints).values(["header", "cursor", "event:1"].map(part => ({
+      companyId: session.companyId, issueId: session.issueId, runId: session.runId, part, material: { encrypted: true },
+    })));
+  }
+
+  it.each([false, true])("adds no periodic database work after startup (retained history: %s)", async retained => {
+    if (retained) {
+      const session = await seedSession({ finalization: { phase: "committed" } });
+      await seedCheckpoint(session); // Saved resume pointer deliberately protects it.
+      await fs.rm(stateBase, { recursive: true, force: true });
+    } else {
+      stateBase = await fs.mkdtemp(path.join(os.tmpdir(), "empty-checkpoint-retention-"));
+    }
+    const expiration = service();
+    const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => { throw error; } });
+    const select = vi.spyOn(db, "select");
+    try {
+      await expiration.registerCheckpointExpiration(coordinator).ready;
+      expect(select).toHaveBeenCalled();
+      const reads = select.mock.calls.length;
+      expect(coordinator.nextWakeAt()).toBeNull();
+      vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout", "Date"] });
+      await vi.advanceTimersByTimeAsync(3 * 60 * 60 * 1000);
+      // The unchanged hourly filesystem sweep must not enumerate DB checkpoints.
+      for (let hour = 0; hour < 3; hour++) await expiration.tick();
+      expect(select).toHaveBeenCalledTimes(reads);
+      expect(coordinator.nextWakeAt()).toBeNull();
+    } finally {
+      await coordinator.stop();
+      select.mockRestore();
+      vi.useRealTimers();
+    }
+    if (retained) {
+      expect(await db.select().from(nativeRunnerCheckpoints)).toHaveLength(3);
+      await db.delete(agentTaskSessions);
+      const nextStartup = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: vi.fn() });
+      try {
+        await service().registerCheckpointExpiration(nextStartup).ready;
+        expect(await db.select().from(nativeRunnerCheckpoints)).toEqual([]);
+        expect(nextStartup.nextWakeAt()).toBeNull();
+      } finally { await nextStartup.stop(); }
+    }
+  });
+
+  it("preserves startup expiration across drain without querying while paused", async () => {
+    const session = await seedSession({ finalization: { phase: "committed" } });
+    await seedCheckpoint(session);
+    await db.delete(agentTaskSessions);
+    let draining = true;
+    vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout", "Date"] });
+    const select = vi.spyOn(db, "select");
+    const onError = vi.fn();
+    const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => !draining, canReconcile: () => true, onError });
+    try {
+      await service().registerCheckpointExpiration(coordinator).ready;
+      await vi.advanceTimersByTimeAsync(60_000);
+      expect(select).not.toHaveBeenCalled();
+      draining = false;
+      await vi.advanceTimersByTimeAsync(1000);
+      await vi.waitFor(() => expect(coordinator.nextWakeAt()).toBeNull());
+      await coordinator.stop(); // Await the DB transaction before asserting.
+      expect(onError).not.toHaveBeenCalled();
+      expect(await db.select().from(nativeRunnerCheckpoints)).toEqual([]);
+    } finally {
+      await coordinator.stop();
+      select.mockRestore();
+      vi.useRealTimers();
+    }
+  });
+
+  it("retries a failed checkpoint from an earlier page before startup cleanup goes quiet", async () => {
+    for (let index = 0; index < 2; index++) {
+      const session = await seedSession({ finalization: { phase: "committed" } });
+      await seedCheckpoint(session);
+    }
+    await db.delete(agentTaskSessions);
+    const transaction = vi.spyOn(db, "transaction").mockRejectedValueOnce(new Error("temporary retention failure"));
+    const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: vi.fn() });
+    try {
+      const expiration = nativeHarnessBackupExpirationService(db, { now, maxEntries: 1 });
+      await expiration.registerCheckpointExpiration(coordinator).ready;
+      await vi.waitFor(async () => {
+        expect(await db.select().from(nativeRunnerCheckpoints)).toEqual([]);
+        expect(coordinator.nextWakeAt()).toBeNull();
+      }, { timeout: 5000 });
+    } finally {
+      await coordinator.stop();
+      transaction.mockRestore();
+    }
+  });
+
+  it.each(["query", "transaction"] as const)("bounds persistent startup %s failures and becomes quiet until the next startup", async fault => {
+    const session = await seedSession({ finalization: { phase: "committed" } });
+    await seedCheckpoint(session);
+    await db.delete(agentTaskSessions);
+    const failure = fault === "query"
+      ? vi.spyOn(db, "select").mockImplementation(() => { throw new Error("retention candidate query unavailable"); })
+      : vi.spyOn(db, "transaction").mockRejectedValue(new Error("retention transaction unavailable"));
+    const warning = vi.spyOn(logger, "warn");
+    const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: vi.fn() });
+    try {
+      await service().registerCheckpointExpiration(coordinator).ready;
+      await vi.waitFor(() => {
+        // No scheduled deadline can also mean a retry is currently running.
+        // Wait for the owner's final deferral before asserting quiet time.
+        expect(warning).toHaveBeenCalledWith(expect.objectContaining({ failedPasses: 3 }),
+          "startup checkpoint expiration retries exhausted; retained checkpoints deferred to next startup");
+        expect(failure).toHaveBeenCalledTimes(3);
+        expect(coordinator.nextWakeAt()).toBeNull();
+      }, { timeout: 5000 });
+      const attempts = failure.mock.calls.length;
+      expect(attempts).toBe(3);
+      vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout", "Date"] });
+      await vi.advanceTimersByTimeAsync(3 * 60 * 60 * 1000);
+      expect(failure).toHaveBeenCalledTimes(attempts);
+    } finally {
+      await coordinator.stop(); failure.mockRestore(); warning.mockRestore(); vi.useRealTimers();
+    }
+    expect(await db.select().from(nativeRunnerCheckpoints)).toHaveLength(3);
+    const nextStartup = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: vi.fn() });
+    try {
+      await service().registerCheckpointExpiration(nextStartup).ready;
+      expect(await db.select().from(nativeRunnerCheckpoints)).toEqual([]);
+      expect(nextStartup.nextWakeAt()).toBeNull();
+    } finally { await nextStartup.stop(); }
+  });
+
+  it("expires unreferenced settled checkpoints even with an empty controller disk and fences old writers", async () => {
+    const session = await seedSession({ finalization: { phase: "committed" } });
+    await seedCheckpoint(session);
+    await db.delete(agentTaskSessions);
+    await fs.rm(stateBase, { recursive: true, force: true });
+    expect(await service().tickCheckpoints()).toMatchObject({ expiredCheckpoints: 1, checkpointCandidates: 1, failed: 0 });
+    expect(await db.select().from(nativeRunnerCheckpoints)).toEqual([]);
+    const [authority] = await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, session.runId));
+    expect(authority?.controllerGeneration).toBe(1);
+    expect(await db.select().from(heartbeatRuns)).toHaveLength(1);
+  });
+
+  it.each(["resume", "lease", "lease_stamp", "recovery", "running", "recent", "open", "execution", "dry_run"])(
+    "preserves a checkpoint protected by %s without its controller disk", async protection => {
+      const session = await seedSession({ finalization: { phase: "committed" } });
+      await seedCheckpoint(session);
+      await fs.rm(stateBase, { recursive: true, force: true });
+      if (protection !== "resume") await db.delete(agentTaskSessions);
+      if (protection === "lease") await db.insert(environmentLeases).values({
+        companyId: session.companyId, issueId: session.issueId, status: "retained",
+      });
+      if (protection === "lease_stamp") await db.insert(environmentLeases).values({
+        companyId: session.companyId, status: "retained",
+        metadata: { nativeHarnessBackup: { normalizedSessionId: session.sessionId } },
+      });
+      if (protection === "recovery") await db.update(nativeRunFinalizations).set({ recoveryState: "pending" });
+      if (protection === "running") await db.update(heartbeatRuns).set({ status: "running" });
+      if (protection === "recent") await db.update(issues).set({ completedAt: now });
+      if (protection === "open") await db.update(issues).set({ status: "in_progress" });
+      if (protection === "execution") await db.update(issues).set({ executionRunId: session.runId });
+      expect(await service().tickCheckpoints({ dryRun: protection === "dry_run" })).toMatchObject({ expiredCheckpoints: 0, failed: 0 });
+      expect(await db.select().from(nativeRunnerCheckpoints)).toHaveLength(3);
+      const [authority] = await db.select().from(nativeRunFinalizations);
+      expect(authority?.controllerGeneration).toBe(0);
+      // Break the issue/run reference cycle before the shared teardown.
+      if (protection === "execution") await db.update(issues).set({ executionRunId: null });
+    },
+  );
+
+  it("continues checkpoint expiration past retained sessions within bounded pages", async () => {
+    for (let index = 0; index < 3; index++) {
+      const session = await seedSession({ finalization: { phase: "committed" } });
+      await seedCheckpoint(session);
+    }
+    await fs.rm(stateBase, { recursive: true, force: true });
+    const expiration = service();
+    expect(await expiration.tickCheckpoints({ maxEntries: 2 })).toMatchObject({ expiredCheckpoints: 0, truncated: true });
+    expect(await expiration.tickCheckpoints({ maxEntries: 2 })).toMatchObject({ expiredCheckpoints: 0, truncated: false });
+    await db.delete(agentTaskSessions);
+    expect(await expiration.tickCheckpoints({ maxEntries: 2 })).toMatchObject({ expiredCheckpoints: 2, truncated: true });
+    expect(await expiration.tickCheckpoints({ maxEntries: 2 })).toMatchObject({ expiredCheckpoints: 1, truncated: false });
+  });
+
   it("expires the backup of a session whose issue closed before the window and clears its resume pointer", async () => {
     const session = await seedSession();
     const result = await service().tick();

server/src/__tests__/plugin-runner-task.integration.test.ts

@@ -4,7 +4,7 @@ import { PassThrough } from "node:stream";
 import { setTimeout as delay } from "node:timers/promises";
 import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
 import { eq } from "drizzle-orm";
-import { agents, companies, createDb, environmentLeases, environments, heartbeatRuns, plugins, projects } from "@paperclipai/db";
+import { agents, companies, createDb, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS, environmentLeases, environments, heartbeatRuns, plugins, projects } from "@paperclipai/db";
 import { definePlugin } from "../../../packages/plugins/sdk/src/define-plugin.js";
 import { startWorkerRpcHost } from "../../../packages/plugins/sdk/src/worker-rpc-host.js";
 import { createRequest, parseMessage, serializeMessage, isJsonRpcSuccessResponse, isJsonRpcErrorResponse } from "../../../packages/plugins/sdk/src/protocol.js";
@@ -19,11 +19,10 @@ import { environmentRunOrchestrator } from "../services/environment-run-orchestr
 import { createRunnerProviderLaunch, runnerLaunchRequirementsDigest, type RunnerProviderLaunchRequirements } from "@paperclipai/paperclip-runner";
 import { createPluginRunnerTaskOwner } from "../services/native-runtime/plugin-runner-task.js";
 import type { PluginWorkerManager } from "../services/plugin-worker-manager.js";
-import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
+import { recoveryPostgresSupport as support, startRecoveryPostgres } from "./helpers/recovery-postgres.js";
 
 const ingress = vi.hoisted(() => ({ connect: vi.fn() }));
 vi.mock("../realtime/runner-prp-outbound.js", () => ({ connectRunnerPrpIngress: ingress.connect }));
-const support = await getEmbeddedPostgresTestSupport();
 const databaseSuite = support.supported ? describe : describe.skip;
 
 /** Real SDK serialization and dispatch with synthetic provider resources. */
@@ -59,9 +58,9 @@ databaseSuite("Core plugin Runner task lifecycle", () => {
   let db!: ReturnType<typeof createDb>;
   let stopDb: (() => Promise<void>) | undefined;
   beforeAll(async () => {
-    const started = await startEmbeddedPostgresTestDatabase("plugin-runner-task");
-    stopDb = started.stop; db = createDb(started.connectionString);
-  });
+    const started = await startRecoveryPostgres("plugin-runner-task");
+    stopDb = started.cleanup; db = createDb(started.connectionString);
+  }, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS);
   afterAll(async () => { await stopDb?.(); });
 
   it("does not allocate agent-dependent setup leases before the agent is prepared", async () => {
@@ -108,6 +107,7 @@ databaseSuite("Core plugin Runner task lifecycle", () => {
       version: "1.0.0", apiVersion: 1, categories: ["automation"], manifestJson: manifest, status: "ready" });
     const calls: PluginEnvironmentTaskOperation[] = [];
     let executionStopped = false, stopRequested = false, acquisition = 0, releaseFailure = false;
+    let projectAuthAllowed = true;
     const released = vi.fn(), complete = vi.fn();
     const plugin = definePlugin({
       async setup() {},
@@ -126,7 +126,7 @@ databaseSuite("Core plugin Runner task lifecycle", () => {
             }),
           } };
           case "check_project_auth": return { kind: "project_auth", taskId: params.taskId,
-            configurationId: params.operation.configurationId, provider: params.operation.provider, allowed: true };
+            configurationId: params.operation.configurationId, provider: params.operation.provider, allowed: projectAuthAllowed };
           case "read_state": return { kind: "state", taskId: params.taskId, state: {
             schema: "paperclip.runner.durable.state.v1", ...params.operation.identity, lifecycle: "suspended",
           } };
@@ -233,6 +233,35 @@ databaseSuite("Core plugin Runner task lifecycle", () => {
       const [saved] = await db.select().from(environmentLeases).where(eq(environmentLeases.id, acquired.lease.id));
       expect(saved.metadata?.runnerTaskDispatched).toBe(true);
       expect(JSON.stringify(saved.metadata)).not.toMatch(/synthetic-account-key|synthetic-launch-key|synthetic-bootstrap-ticket|synthetic-fixture/);
+      const acquisitionBefore = acquisition;
+      const callsBefore = calls.length;
+      const recovered = await environmentRunOrchestrator(db, { environmentRuntime: environmentRuntimeService(db, { pluginWorkerManager: workers }) }).acquireForRun({
+        companyId, selectedEnvironmentId: environmentId, localEnvironmentId: randomUUID(), agentId,
+        issueId: null, heartbeatRunId: runId, persistedExecutionWorkspace: null,
+        executionWorkspaceSettings: null, adapterType: "paperclip_runner",
+        reattachRemoteLease: { kind: "plugin_task", leaseId: saved.id },
+      });
+      expect(recovered.lease.id).toBe(saved.id);
+      const adopted = createPluginRunnerTaskOwner({ task: target.runnerTask, projectIds: [projectId], harness: "codex",
+        projectAuthCheckVersion: 1, adoptIdentity: identity });
+      await adopted.driver.authorizeRunnerAdoption!(prepared.configurationId, identity);
+      const adoptedRegistration = await adopted.registration({} as never, runId);
+      adoptedRegistration.activate(); await adoptedRegistration.ready();
+      await adoptedRegistration.release("detach");
+      expect(acquisition).toBe(acquisitionBefore);
+      expect(calls.slice(callsBefore).some(op => ["prepare", "submit", "stop", "complete"].includes(op.kind))).toBe(false);
+      expect(executionStopped).toBe(false);
+      projectAuthAllowed = false;
+      const revoked = createPluginRunnerTaskOwner({ task: target.runnerTask, projectIds: [projectId], harness: "codex",
+        projectAuthCheckVersion: 1, adoptIdentity: identity });
+      const connectionsBeforeRevocation = calls.filter(op => op.kind === "connection").length;
+      await expect(revoked.driver.authorizeRunnerAdoption!(prepared.configurationId, identity)).rejects.toThrow("project authentication conflicts");
+      expect(calls.filter(op => op.kind === "connection")).toHaveLength(connectionsBeforeRevocation);
+      expect(calls.slice(callsBefore).filter(op => op.kind === "check_project_auth")).toEqual(Array(2).fill({
+        kind: "check_project_auth", configurationId: prepared.configurationId, provider: "openai", projectIds: [projectId],
+      }));
+      projectAuthAllowed = true;
+      close.mockClear();
       await expect(runtime.task({ companyId: randomUUID(), leaseId: acquired.lease.id, operation: { kind: "stop" } })).rejects.toThrow("lease unavailable");
       const before = calls.filter(value => value.kind === "submit").length;
       await expect(runtime.task({ companyId, leaseId: acquired.lease.id, operation: { ...submit!, projectIds: [randomUUID()] } as never })).rejects.toThrow("project unavailable");
@@ -279,7 +308,7 @@ databaseSuite("Core plugin Runner task lifecycle", () => {
       expect(cleaned).toMatchObject({ status: "expired", cleanupStatus: "success", environmentId: null,
         metadata: { runnerTaskDispatched: false, remoteExecutionTermination: { state: "stopped" } } });
     } finally { executionStopped = true; handle.child.kill(); await handle.completion.catch(() => {}); harness.stop(); }
-  });
+  }, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS);
 
   it.each([false, true])("replaces a same-run task only after the provider confirms termination (%s)", async stopped => {
     const companyId = randomUUID(), agentId = randomUUID(), runId = randomUUID(), pluginId = randomUUID(), environmentId = randomUUID(), oldLeaseId = randomUUID();

server/src/__tests__/project-tools-pagination.test.ts

@@ -28,3 +28,34 @@ describe("project discovery result bounds", () => {
     expect(fetch).not.toHaveBeenCalled();
   });
 });
+
+describe("project creation receipt recovery", () => {
+  const create = { ...input, name: "create_project", arguments: { name: "Recovered project", idempotencyKey: "project" }, receiptOnly: true };
+
+  it("asks the project owner for a saved result and accepts an absent receipt without parsing an empty response", async () => {
+    const fetch = vi.fn(async (_url: string, _options: RequestInit) => new Response(null, { status: 204 }));
+    vi.stubGlobal("fetch", fetch);
+    expect(await callProjectTool(create)).toBeUndefined();
+    expect(fetch).toHaveBeenCalledWith("http://paperclip.test/api/companies/company/projects", expect.objectContaining({
+      method: "POST", headers: expect.objectContaining({ "X-Paperclip-Receipt-Only": "true" }),
+      body: expect.any(String),
+    }));
+    expect(JSON.parse(fetch.mock.calls[0]![1].body as string)).toMatchObject(create.arguments);
+  });
+
+  it("returns the saved project and preserves owner validation failures", async () => {
+    const project = { id: "saved-project", name: create.arguments.name };
+    const fetch = vi.fn().mockResolvedValueOnce(Response.json(project))
+      .mockResolvedValueOnce(Response.json({ error: "Project idempotency key was used with different inputs" }, { status: 409 }));
+    vi.stubGlobal("fetch", fetch);
+    expect(await callProjectTool(create)).toEqual(project);
+    await expect(callProjectTool(create)).rejects.toThrow(/different inputs/);
+  });
+
+  it("rejects receipt-only requests for other operations before fetching", async () => {
+    const fetch = vi.fn();
+    vi.stubGlobal("fetch", fetch);
+    await expect(callProjectTool({ ...input, receiptOnly: true })).rejects.toThrow(/receipt/i);
+    expect(fetch).not.toHaveBeenCalled();
+  });
+});

server/src/index.ts

@@ -98,6 +98,7 @@ import {
   companyMemberships,
   instanceUserRoles,
   closeLivenessDb,
+  closeCheckpointDb,
 } from "@paperclipai/db";
 import detectPort from "detect-port";
 import { createApp } from "./app.js";
@@ -704,7 +705,7 @@ async function startServerWithDatabaseTeardown(
   const closeDatabaseClients = async () => {
     setSentryRunLinkResolver(null);
     const clients = pluginMigrationDb === db ? [db] : [db, pluginMigrationDb];
-    await Promise.all([...clients.map((client) => endDatabaseClient(client, 5)), closeLivenessDb(db, 5)]);
+    await Promise.all([...clients.map((client) => endDatabaseClient(client, 5)), closeLivenessDb(db, 5), closeCheckpointDb(db, 5)]);
   };
   startupDatabase.close = closeDatabaseClients;
   setSentryRunLinkResolver(createSentryRunLinkResolver(db));
@@ -1364,6 +1365,9 @@ async function startServerWithDatabaseTeardown(
     resolveNativeQuestion: (interaction) => deliverNativeQuestionResponse(db as any, interaction),
   });
   const deliveryWork = app.locals.deliveryWork as ReturnType<typeof createDeliveryWorkCoordinator>;
+  if (config.nativeHarnessBackupExpirationEnabled) {
+    nativeHarnessBackupExpiration.registerCheckpointExpiration(deliveryWork);
+  }
   const executionSchedule = executionControlSchedule(db);
   const workspaceRecoveryRuntime = environmentRuntimeService(db, { pluginWorkerManager });
   const workspaceRecoveryWork = registerWorkspaceRecoveryWork(db, deliveryWork, workspaceRecoveryRuntime,

server/src/realtime/runner-prp-outbound.ts

@@ -1,4 +1,5 @@
 import WebSocket from "ws";
+import { REMOTE_RUNNER_RECONNECT_GRACE_MS } from "../services/native-runtime/native-runner-recovery-contract.js";
 import type { RunnerIngressEndpoint } from "@paperclipai/adapter-utils/runner-connectivity";
 import type {
   DurablePrpControlPlane,
@@ -7,7 +8,7 @@ import type {
 } from "../vendor/paperclip-runner/index.js";
 
 const DEFAULT_STARTUP_DEADLINE_MS = 60_000;
-const DEFAULT_RECOVERY_GRACE_MS = 120_000;
+const DEFAULT_RECOVERY_GRACE_MS = REMOTE_RUNNER_RECONNECT_GRACE_MS;
 const MAX_FRAME_BYTES = 4 * 1024 * 1024;
 const RECONNECT_DELAYS_MS = [250, 500, 1_000, 2_000, 5_000] as const;
 

server/src/routes/projects.ts

@@ -384,6 +384,8 @@ export function projectRoutes(db: Db) {
     };
 
     const { workspace, repositoryIds, repositoryUrls, idempotencyKey, ...projectData } = req.body as CreateProjectPayload & { idempotencyKey?: string; repositoryUrls?: string[] };
+    const receiptOnly = req.get("X-Paperclip-Receipt-Only") === "true";
+    if (receiptOnly && !idempotencyKey) throw unprocessable("Project receipt recovery requires an idempotency key");
     projectData.privacyOwnerUserId = req.actor.type === "board" ? req.actor.userId ?? null : req.actor.type === "agent" ? req.actor.onBehalfOfUserId ?? null : null;
     const runContext = req.actor.type === "agent" && req.actor.source === "agent_jwt" && req.actor.runId
       ? await projectToolContext(db, req.actor, true) : null;
@@ -439,6 +441,7 @@ export function projectRoutes(db: Db) {
           }
         });
       }
+      if (receiptOnly && runContext && !museActor) await projectToolContext(tx as unknown as Db, req.actor, true);
       if (receiptKey) {
         const [prior] = await tx.select().from(activityLog).where(and(
           eq(activityLog.companyId, companyId), eq(activityLog.action, "project.created"),
@@ -451,6 +454,7 @@ export function projectRoutes(db: Db) {
           return { project, publication: null, duplicate: true };
         }
       }
+      if (receiptOnly) return null;
       if (runContext && !museActor) await projectToolContext(tx as unknown as Db, req.actor, true);
       const service = projectService(tx as unknown as Db);
       const project = repositories ? await service.createWithRepositories(companyId, projectData, repositories) : await service.create(companyId, projectData);
@@ -483,6 +487,7 @@ export function projectRoutes(db: Db) {
       });
       return { project: hydrated ?? project, publication: activity.publication, duplicate: false };
     });
+    if (!result) { res.status(204).end(); return; }
     if (result.publication) publishActivity(result.publication);
     if (result.project.env) await secretsSvc.syncEnvBindingsForTarget?.(companyId, { targetType: "project", targetId: result.project.id }, result.project.env);
     if (result.duplicate) { res.status(200).json(result.project); return; }

server/src/secrets/local-encrypted-provider.ts

@@ -257,6 +257,34 @@ function asLocalEncryptedMaterial(value: StoredSecretVersionMaterial): LocalEncr
   throw badRequest("Invalid local_encrypted secret material");
 }
 
+function resolveLocalEncryptedValue<T>(operation: () => T): T {
+  try {
+    return operation();
+  } catch (cause) {
+    // This boundary contains only local key I/O and decryption, never database work.
+    throw new SecretProviderClientError({
+      code: "provider_unavailable", provider: "local_encrypted", operation: "resolveVersion",
+      message: cause instanceof Error ? cause.message : "Local encrypted secret could not be resolved",
+      cause,
+    });
+  }
+}
+
+/** Reuse one resolved key within a single batch; reload it for the next operation. */
+export function createLocalEncryptedCipher(mode: "read" | "write") {
+  const masterKey = mode === "write"
+    ? loadOrCreateMasterKey()
+    : resolveLocalEncryptedValue(() => loadOrCreateMasterKey(false));
+  return {
+    encrypt(value: string): StoredSecretVersionMaterial {
+      return encryptValue(masterKey, value);
+    },
+    decrypt(material: StoredSecretVersionMaterial): string {
+      return resolveLocalEncryptedValue(() => decryptValue(masterKey, asLocalEncryptedMaterial(material)));
+    },
+  };
+}
+
 export const localEncryptedProvider: SecretProviderModule = {
   id: "local_encrypted",
   descriptor() {
@@ -291,18 +319,7 @@ export const localEncryptedProvider: SecretProviderModule = {
     throw badRequest("local_encrypted does not support external reference secrets");
   },
   async resolveVersion(input) {
-    try {
-      const masterKey = loadOrCreateMasterKey(false);
-      return decryptValue(masterKey, asLocalEncryptedMaterial(input.material));
-    } catch (cause) {
-      // This boundary contains only local key I/O and decryption, never database
-      // work. Consumers can handle a vault outage without swallowing DB errors.
-      throw new SecretProviderClientError({
-        code: "provider_unavailable", provider: "local_encrypted", operation: "resolveVersion",
-        message: cause instanceof Error ? cause.message : "Local encrypted secret could not be resolved",
-        cause,
-      });
-    }
+    return createLocalEncryptedCipher("read").decrypt(input.material);
   },
   async deleteOrArchive() {
     // Secret metadata deletion is handled in Paperclip DB; the local key is shared and must remain.

server/src/services/agent-commentary.ts

@@ -19,12 +19,15 @@ function boundIssue(run: typeof heartbeatRuns.$inferSelect): string | null {
 }
 
 /** Both transports share authorization, persistence and replay; no task state is changed. */
+export function submitAgentCommentary(db: Db, actor: CommentaryActor, value: unknown, authorizeNative?: (tx: Db) => Promise<unknown>): Promise<AgentCommentaryAcknowledgement>;
+export function submitAgentCommentary(db: Db, actor: CommentaryActor, value: unknown, authorizeNative: ((tx: Db) => Promise<unknown>) | undefined, options: { receiptOnly: true }): Promise<AgentCommentaryAcknowledgement | null>;
 export async function submitAgentCommentary(
   db: Db,
   actor: CommentaryActor,
   value: unknown,
   authorizeNative?: (tx: Db) => Promise<unknown>,
-): Promise<AgentCommentaryAcknowledgement> {
+  options?: { receiptOnly: true },
+): Promise<AgentCommentaryAcknowledgement | null> {
   const parsed = submitAgentCommentarySchema.safeParse(value);
   if (!parsed.success) throw badRequest("Feedback requires a kind, nonempty body of at most 524288 characters, and a retry key of at most 240 characters; no other fields are accepted");
   if (![actor.companyId, actor.agentId, actor.runId].every(isUuidLike)) {
@@ -64,9 +67,13 @@ export async function submitAgentCommentary(
         eq(agentCommentary.companyId, actor.companyId), eq(agentCommentary.runId, actor.runId), eq(agentCommentary.idempotencyKey, input.idempotencyKey),
       ));
       if (prior) {
-        if (prior.payloadHash !== payloadHash) throw conflict("Feedback retry key was already used with different content");
+        if (prior.payloadHash !== payloadHash) {
+          if (options?.receiptOnly) return { result: null, publication: null };
+          throw conflict("Feedback retry key was already used with different content");
+        }
         return { result: { id: prior.id, kind: prior.kind, createdAt: prior.createdAt.toISOString(), replayed: true }, publication: null };
       }
+      if (options?.receiptOnly) return { result: null, publication: null };
       const body = redactSensitiveText(await createRunSecretRedactionRegistry(scopedDb).redactForRun(actor.companyId, actor.runId, input.body));
       const [row] = await tx.insert(agentCommentary).values({
         companyId: actor.companyId, agentId: actor.agentId, runId: actor.runId,

server/src/services/delivery-work-notifications.ts

@@ -69,6 +69,7 @@ export const DELIVERY_QUEUES = {
   routineSchedules: "routine-schedules",
   issueMonitors: "issue-monitors",
   caseHandoffs: "case-handoffs",
+  nativeCheckpointExpiration: "native-checkpoint-expiration",
 } as const;
 export type DeliveryQueue = typeof DELIVERY_QUEUES[keyof typeof DELIVERY_QUEUES];
 

server/src/services/environment-run-orchestrator.ts

@@ -281,7 +281,8 @@ export function environmentRunOrchestrator(
     executionWorkspaceSettings: IssueExecutionWorkspaceSettings | null;
     runnerTaskProjectIds?: readonly string[];
     /** Existing live runner recovery must use its original active lease, including ephemeral leases. */
-    reattachRemoteLease?: { leaseId: string; providerLeaseId: string; remoteCwd: string };
+    reattachRemoteLease?: { kind?: "sandbox"; leaseId: string; providerLeaseId: string; remoteCwd: string }
+      | { kind: "plugin_task"; leaseId: string };
     /** A new controller must stop the previous plugin task before allocating a replacement. */
     recoverPluginTask?: boolean;
   }): Promise<EnvironmentAcquisitionResult> {
@@ -330,15 +331,19 @@ export function environmentRunOrchestrator(
       // when the admitted per-turn policy deliberately disables reusable leases.
       const expected = input.reattachRemoteLease;
       const lease = await environmentsSvc.getLeaseById(expected.leaseId);
-      if (!lease || environment.driver !== "sandbox" ||
+      if (!lease ||
           lease.companyId !== input.companyId || lease.environmentId !== environment.id ||
           lease.heartbeatRunId !== input.heartbeatRunId || lease.issueId !== input.issueId ||
           lease.executionWorkspaceId !== (input.persistedExecutionWorkspace?.id ?? null) ||
           lease.metadata?.agentId !== input.agentId || lease.status !== "active" ||
           lease.releasedAt !== null || lease.cleanupStatus !== null ||
           (lease.expiresAt !== null && new Date(lease.expiresAt).getTime() <= Date.now()) ||
-          lease.provider !== environment.config.provider || lease.providerLeaseId !== expected.providerLeaseId ||
-          lease.metadata?.remoteCwd !== expected.remoteCwd) {
+          (expected.kind === "plugin_task"
+            ? environment.driver !== "plugin" || input.adapterType !== "paperclip_runner"
+              || lease.metadata?.runnerTaskLease !== true || lease.metadata?.runnerTaskDispatched !== true
+              || lease.metadata?.runnerTaskExecutionStopped === true
+            : environment.driver !== "sandbox" || lease.provider !== environment.config.provider
+              || lease.providerLeaseId !== expected.providerLeaseId || lease.metadata?.remoteCwd !== expected.remoteCwd)) {
         throw new Error("native_remote_recovery_lease_mismatch");
       }
       leaseRecord = { environment, lease, leaseContext: buildEnvironmentLeaseContext(input) };

server/src/services/heartbeat/recovery.test.ts

@@ -1,6 +1,6 @@
 import { randomUUID } from "node:crypto";
 import type { Db } from "@paperclipai/db";
-import { activityLog, agents, applyPendingMigrations, companies, createDb, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS, environments, environmentLeases, heartbeatRuns, issueRecoveryActions, issues, nativeRunFinalizations } from "@paperclipai/db";
+import { activityLog, agents, applyPendingMigrations, companies, createDb, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS, environments, environmentLeases, heartbeatRuns, issueRecoveryActions, issues, nativeRunFinalizations, nativeRunnerCheckpoints } from "@paperclipai/db";
 import { eq } from "drizzle-orm";
 import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
 import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
@@ -15,6 +15,7 @@ import { claimNativeRestartRecoveries } from "../native-runtime/native-restart-r
 import { allowLegacyShutdownWorkspaceCleanup, beginLegacyShutdownWorkspaceSettlement, finishLegacyShutdownWorkspaceSettlement, maintainLegacyShutdownWorkspaceSettlement, legacyShutdownWorkspaceResourceProtected, sweepLegacyShutdownWorkspaceSettlements } from "../legacy-shutdown-workspace-settlement.js";
 import { adapterExecutionControls, createAdapterExecutionControl } from "../adapter-execution-control.js";
 import { logger } from "../../middleware/logger.js";
+import { listExpiredNativeRunnerAuthorities } from "../native-runtime/native-runner-checkpoint.js";
 import { DELIVERY_QUEUES, subscribeDeliveryWork } from "../delivery-work-notifications.js";
 import { resolveExternalTestDatabaseUrl } from "../../__tests__/helpers/external-test-database.js";
 import { hasRemoteTerminationReceipt } from "../remote-execution-termination.js";
@@ -94,6 +95,12 @@ function guardedDatabase() {
   return { db: new Proxy({}, { get: access }) as Db, access };
 }
 
+it("does not query checkpoint recovery without an expired active run", async () => {
+  const { db, access } = guardedDatabase();
+  expect(await listExpiredNativeRunnerAuthorities(db, new Date(), [])).toEqual([]);
+  expect(access).not.toHaveBeenCalled();
+});
+
 beforeEach(() => {
   vi.clearAllMocks();
   hotRestart.read.mockResolvedValue(null);
@@ -678,6 +685,68 @@ describePostgres("heartbeat recovery module ownership", () => {
     await vi.waitFor(() => expect(deps.activeRunExecutionPromises.size).toBe(0));
   });
 
+  it.each(["bootstrap_incomplete", "blocked"] as const)("retries durable remote takeover and preserves its %s disposition", async kind => {
+    const run = await seedRun();
+    const deps = callbacks(db);
+    await db.update(heartbeatRuns).set({ runtimeMode: "native", nativeIssueId: run.issueId }).where(eq(heartbeatRuns.id, run.id));
+    await db.insert(nativeRunFinalizations).values({ runId: run.id, companyId: run.companyId, issueId: run.issueId!,
+      phase: "observed", recoveryState: "awaiting_runner_reattach", leaseOwner: "prior-server",
+      leaseExpiresAt: new Date(Date.now() + 60_000) });
+    await db.insert(nativeRunnerCheckpoints).values({ runId: run.id, companyId: run.companyId, issueId: run.issueId!,
+      part: "opaque-checkpoint-record", material: { fixture: true } });
+    const recovery = createHeartbeatRecovery(db, deps);
+    try {
+      await recovery.reapOrphanedRuns();
+      expect(native.claims).not.toHaveBeenCalled();
+      await db.update(nativeRunFinalizations).set({ leaseExpiresAt: new Date(Date.now() - 1) }).where(eq(nativeRunFinalizations.runId, run.id));
+      const claim: NativeRestartRecoveryDisposition = kind === "blocked"
+        ? { kind, runId: run.id, reason: "fixture_ambiguous_owner" }
+        : { kind, runId: run.id, leaseOwner: "new-server",
+          controllerGeneration: 2, providerAttempt: 0, restartKind: "hard", recoveryRequestId: null };
+      // Force the old inventory down the unverified-owner path unless recovery
+      // dispositions fence it. The runtime's decision must own this run.
+      await db.update(nativeRunFinalizations).set({ recoveryState: null }).where(eq(nativeRunFinalizations.runId, run.id));
+      native.claims.mockResolvedValue([claim]);
+      await recovery.reapOrphanedRuns();
+      expect(native.claims).toHaveBeenCalledWith(expect.objectContaining({ runIds: [run.id] }));
+      if (kind === "blocked") expect(deps.executeRun).not.toHaveBeenCalled();
+      else expect(deps.executeRun).toHaveBeenCalledExactlyOnceWith(run.id, { nativeLeaseOwner: "new-server", nativeRestartRecovery: claim });
+      expect(deps.setRunStatusFromLive).not.toHaveBeenCalled();
+      expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, run.id)))[0].errorCode)
+        .not.toBe(NATIVE_OWNERSHIP_UNVERIFIED_ERROR_CODE);
+      await Promise.all([...deps.activeRunExecutionPromises]);
+      await db.update(heartbeatRuns).set({ status: "cancelled" }).where(eq(heartbeatRuns.id, run.id));
+      native.claims.mockClear();
+      await recovery.reapOrphanedRuns();
+      expect(native.claims).not.toHaveBeenCalled();
+    } finally {
+      await Promise.all([...deps.activeRunExecutionPromises]);
+      await db.delete(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, run.id));
+      await db.delete(heartbeatRuns).where(eq(heartbeatRuns.id, run.id));
+    }
+  });
+
+  it("does not detach a different server's native run during shutdown", async () => {
+    const run = await seedRun();
+    const deps = callbacks(db);
+    const bootId = randomUUID();
+    const expiresAt = new Date(Date.now() + 60_000);
+    await db.update(agents).set({ adapterType: "paperclip_runner" }).where(eq(agents.id, run.agentId));
+    await db.update(heartbeatRuns).set({ runtimeMode: "native", nativeIssueId: run.issueId }).where(eq(heartbeatRuns.id, run.id));
+    await db.insert(nativeRunFinalizations).values({ runId: run.id, companyId: run.companyId, issueId: run.issueId!,
+      phase: "observed", controllerBootId: bootId, leaseOwner: "other-server", leaseExpiresAt: expiresAt });
+    try {
+      await createHeartbeatRecovery(db, deps).drainRunningRunsForShutdown("SIGTERM", new Date(), [run.id]);
+      expect(native.detach).not.toHaveBeenCalled();
+      expect(deps.releaseEnvironmentLeasesForRun).not.toHaveBeenCalled();
+      expect((await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, run.id)))[0])
+        .toMatchObject({ controllerBootId: bootId, leaseOwner: "other-server", leaseExpiresAt: expiresAt, recoveryState: null });
+    } finally {
+      await db.delete(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, run.id));
+      await db.delete(heartbeatRuns).where(eq(heartbeatRuns.id, run.id));
+    }
+  });
+
   it("does not dispatch recovery for a run still preparing in this heartbeat process", async () => {
     const run = await seedRun();
     await db.update(agents).set({ adapterType: "paperclip_runner" }).where(eq(agents.id, run.agentId));

server/src/services/heartbeat/recovery.ts

@@ -1,6 +1,8 @@
 import { pendingCleanupDeadlineSql } from "../environment-cleanup-state.js";
 import { writeEnvironmentCleanup } from "../environment-cleanup-notifications.js";
+import { startNativeSessionExecutionLeaseRenewal, type NativeExecutionLeaseRenewal } from "../native-runtime/native-execution-lease.js";
 import type { WorkspaceWorkers } from "../plugin-workspaces.js";
+import { listExpiredNativeRunnerAuthorities } from "../native-runtime/native-runner-checkpoint.js";
 import { createHeartbeatAgentStatus } from "./agent-status.js";
 import { preserveWorkspaceRestoreRecoveryMetadataSql } from "../legacy-workspace-restore-recovery.js";
 import { preserveCancelledNativeWorkspaces, preserveOrphanedNativeWorkspace } from "../native-runtime/native-failed-workspace-recovery.js";
@@ -907,18 +909,17 @@ export function createHeartbeatRecovery(db: Db, dependencies: HeartbeatRecoveryD
     };
   }
 
-  async function recoverNativeRunsAfterRestart(now = new Date()) {
-    await reconcileRetainedWorkspaces(db, environmentRuntime, resumeRemoteStopComments);
-    // A result committed before the old controller stopped outranks process
-    // recovery. Finish its durable workspace/status suffix before deciding
-    // whether any provider authority needs to be reopened.
-    await reconcileNativeFinalizations(db, undefined, {
-      environmentRuntime,
-      pluginWorkerManager: dependencies.pluginWorkerManager,
-      onWorkspaceSettled: settleRecoveredNativeWorkspace,
-    });
-    await preserveCancelledNativeWorkspaces(db);
-    scheduleRetainedNativeSessionCleanup();
+  async function recoverNativeRunsAfterRestart(now = new Date(), runIds?: string[]) {
+    if (!runIds) {
+      await reconcileRetainedWorkspaces(db, environmentRuntime, resumeRemoteStopComments);
+      await reconcileNativeFinalizations(db, undefined, {
+        environmentRuntime,
+        pluginWorkerManager: dependencies.pluginWorkerManager,
+        onWorkspaceSettled: settleRecoveredNativeWorkspace,
+      });
+      await preserveCancelledNativeWorkspaces(db);
+      scheduleRetainedNativeSessionCleanup();
+    }
     const intent = await readHotRestartIntent().catch((error) => {
       logger.warn(
         { err: error },
@@ -957,106 +958,124 @@ export function createHeartbeatRecovery(db: Db, dependencies: HeartbeatRecoveryD
         );
       }
     }
-    const dispositions = await claimNativeRestartRecoveries({
-      db,
-      liveRunExecutions: activeRunExecutions,
-      restartKind,
-      recoveryRequestId: intent?.recoveryRequestId ?? null,
-      coordinatedPreviousController: intent
-        ? {
-            pid: intent.previousServerPid,
-            processStartedAt:
-              previousStartedAt && !Number.isNaN(previousStartedAt.getTime())
-                ? previousStartedAt
-                : null,
-          }
-        : null,
-      now,
-    });
+    const preparationRenewals = new Map<string, NativeExecutionLeaseRenewal>();
+    try {
+      const dispositions = await claimNativeRestartRecoveries({
+        onClaim: (claim, scope) => {
+          if (!scope.remote || claim.kind !== "reattach_remote_runner") return;
+          // Start before the rest of the batch or its event writes can delay dispatch.
+          preparationRenewals.set(claim.runId, startNativeSessionExecutionLeaseRenewal({
+            db, ...scope, runId: claim.runId, leaseOwner: claim.leaseOwner,
+            attempt: claim.providerAttempt,
+            onLost: () => detachNativeSessionsForRestart([claim.runId], claim.leaseOwner),
+          }));
+        },
+        db,
+        liveRunExecutions: activeRunExecutions,
+        runIds,
+        restartKind,
+        recoveryRequestId: intent?.recoveryRequestId ?? null,
+        coordinatedPreviousController: intent
+          ? {
+              pid: intent.previousServerPid,
+              processStartedAt:
+                previousStartedAt && !Number.isNaN(previousStartedAt.getTime())
+                  ? previousStartedAt
+                  : null,
+            }
+          : null,
+        now,
+      });
 
-    const claims = dispositions.filter(
-      (disposition): disposition is NativeRestartRecoveryClaim =>
-        disposition.kind === "reattach_existing_runner" ||
-        disposition.kind === "reattach_remote_runner" ||
-        disposition.kind === "resume_dead_runner" ||
-        disposition.kind === "bootstrap_incomplete",
-    );
-    for (const disposition of dispositions) {
-      const run = await getRun(disposition.runId);
-      if (run) {
-        const isClaim =
+      const claims = dispositions.filter(
+        (disposition): disposition is NativeRestartRecoveryClaim =>
           disposition.kind === "reattach_existing_runner" ||
           disposition.kind === "reattach_remote_runner" ||
           disposition.kind === "resume_dead_runner" ||
-          disposition.kind === "bootstrap_incomplete";
-        await appendRunEvent(run, {
-          eventType: "native.recovery.transition",
-          stream: "system",
-          level: disposition.kind === "blocked" ? "warn" : "info",
-          message:
-            disposition.kind === "reattach_existing_runner" ||
-            disposition.kind === "reattach_remote_runner"
-              ? "Recovering the existing native runner process after server restart"
-              : disposition.kind === "resume_dead_runner"
-                ? "Resuming the durable native provider session after runner process loss"
-                : disposition.kind === "bootstrap_incomplete"
-                  ? "Restarting an incomplete native runner bootstrap on the same heartbeat run"
-                  : disposition.kind === "awaiting_evidence"
-                    ? "Native restart recovery is waiting for safe ownership evidence"
-                    : disposition.kind === "already_finalized"
-                      ? "Native restart recovery found an already-finalized result"
-                      : "Native restart recovery blocked ambiguous or conflicting ownership",
-          payload: {
-            restartKind: isClaim ? disposition.restartKind : restartKind,
-            recoveryRequestId: isClaim
-              ? disposition.recoveryRequestId
-              : (intent?.recoveryRequestId ?? null),
-            runnerDisposition: disposition.kind,
-            ...(isClaim
-              ? {
-                  controllerGeneration: disposition.controllerGeneration,
-                  providerAttempt: disposition.providerAttempt,
-                }
-              : { reason: disposition.reason }),
-            ...(disposition.kind === "reattach_existing_runner"
-              ? {
-                  processPid: disposition.process.pid,
-                  processGroupId: disposition.process.processGroupId,
-                  processStartedAt: disposition.process.startedAt,
-                }
-              : {}),
-          },
-        });
-      }
-    }
-    for (const claim of claims) {
-      const execution = executeRun(claim.runId, {
-        nativeLeaseOwner: claim.leaseOwner,
-        nativeRestartRecovery: claim,
-      }).catch((error) => {
-        logger.error(
-          { err: error, runId: claim.runId, disposition: claim.kind },
-          "native restart recovery execution failed",
-        );
-      });
-      activeRunExecutionPromises.add(execution);
-      void execution.finally(() =>
-        activeRunExecutionPromises.delete(execution),
+          disposition.kind === "bootstrap_incomplete",
       );
-    }
+      for (const disposition of dispositions) {
+        const run = await getRun(disposition.runId);
+        if (run) {
+          const isClaim =
+            disposition.kind === "reattach_existing_runner" ||
+            disposition.kind === "reattach_remote_runner" ||
+            disposition.kind === "resume_dead_runner" ||
+            disposition.kind === "bootstrap_incomplete";
+          await appendRunEvent(run, {
+            eventType: "native.recovery.transition",
+            stream: "system",
+            level: disposition.kind === "blocked" ? "warn" : "info",
+            message:
+              disposition.kind === "reattach_existing_runner" ||
+              disposition.kind === "reattach_remote_runner"
+                ? "Recovering the existing native runner process after server restart"
+                : disposition.kind === "resume_dead_runner"
+                  ? "Resuming the durable native provider session after runner process loss"
+                  : disposition.kind === "bootstrap_incomplete"
+                    ? "Restarting an incomplete native runner bootstrap on the same heartbeat run"
+                    : disposition.kind === "awaiting_evidence"
+                      ? "Native restart recovery is waiting for safe ownership evidence"
+                      : disposition.kind === "already_finalized"
+                        ? "Native restart recovery found an already-finalized result"
+                        : "Native restart recovery blocked ambiguous or conflicting ownership",
+            payload: {
+              restartKind: isClaim ? disposition.restartKind : restartKind,
+              recoveryRequestId: isClaim
+                ? disposition.recoveryRequestId
+                : (intent?.recoveryRequestId ?? null),
+              runnerDisposition: disposition.kind,
+              ...(isClaim
+                ? {
+                    controllerGeneration: disposition.controllerGeneration,
+                    providerAttempt: disposition.providerAttempt,
+                  }
+                : { reason: disposition.reason }),
+              ...(disposition.kind === "reattach_existing_runner"
+                ? {
+                    processPid: disposition.process.pid,
+                    processGroupId: disposition.process.processGroupId,
+                    processStartedAt: disposition.process.startedAt,
+                  }
+                : {}),
+            },
+          });
+        }
+      }
+      for (const claim of claims) {
+        const preparationRenewal = preparationRenewals.get(claim.runId);
+        const execution = executeRun(claim.runId, {
+          nativeLeaseOwner: claim.leaseOwner,
+          nativeRestartRecovery: claim,
+        }).catch((error) => {
+          logger.error(
+            { err: error, runId: claim.runId, disposition: claim.kind },
+            "native restart recovery execution failed",
+          );
+        }).finally(() => preparationRenewal?.stop().catch(() => undefined));
+        preparationRenewals.delete(claim.runId); // Execution now owns this reference.
+        activeRunExecutionPromises.add(execution);
+        void execution.finally(() =>
+          activeRunExecutionPromises.delete(execution),
+        );
+      }
 
-    return {
-      restartKind,
-      claims,
-      dispositions,
-      scheduledRetryRunIds: scheduledNativeRetries.map((entry) => entry.runId),
-      awaitingEvidenceRunIds: dispositions
-        .filter((entry) => entry.kind === "awaiting_evidence")
-        .map((entry) => entry.runId),
-      blockedRunIds: dispositions
-        .filter((entry) => entry.kind === "blocked")
-        .map((entry) => entry.runId),
-    };
+      return {
+        restartKind,
+        claims,
+        dispositions,
+        scheduledRetryRunIds: scheduledNativeRetries.map((entry) => entry.runId),
+        awaitingEvidenceRunIds: dispositions
+          .filter((entry) => entry.kind === "awaiting_evidence")
+          .map((entry) => entry.runId),
+        blockedRunIds: dispositions
+          .filter((entry) => entry.kind === "blocked")
+          .map((entry) => entry.runId),
+      };
+    } finally {
+      // A failed scan or audit write must not keep abandoned claims alive.
+      await Promise.all([...preparationRenewals.values()].map(renewal => renewal.stop().catch(() => undefined)));
+    }
   }
 
   async function drainRunningRunsForShutdown(
@@ -1092,6 +1111,7 @@ export function createHeartbeatRecovery(db: Db, dependencies: HeartbeatRecoveryD
     const interruptedRunIds: string[] = [];
     const retryRunIds: string[] = [];
     const restartSuspendedRunIds: string[] = [];
+    const nativeController = await currentNativeControllerIdentity();
 
     const settlementDeadline = new Date(Date.now() + 30_000);
     const settlements = await Promise.allSettled(activeRuns.map(async ({ run, agent }) => {
@@ -1104,6 +1124,9 @@ export function createHeartbeatRecovery(db: Db, dependencies: HeartbeatRecoveryD
         run.runtimeMode === "native" &&
         agent.adapterType === "paperclip_runner"
       ) {
+        const [owner] = await db.select({ bootId: nativeRunFinalizations.controllerBootId })
+          .from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, run.id));
+        if (owner?.bootId && owner.bootId !== nativeController.bootId) return;
         // A graceful shutdown relinquishes controller authority just like a
         // hot restart. Leaving the old event consumer attached lets its
         // finalizer interrupt/suspend Claude while the next server is adopting
@@ -1120,6 +1143,7 @@ export function createHeartbeatRecovery(db: Db, dependencies: HeartbeatRecoveryD
         await db
           .update(nativeRunFinalizations)
           .set({
+            leaseExpiresAt: now,
             recoveryState: "awaiting_runner_reattach",
             recoveryHistory: sql`(
               select coalesce(jsonb_agg(item order by ordinal), '[]'::jsonb)
@@ -1140,6 +1164,8 @@ export function createHeartbeatRecovery(db: Db, dependencies: HeartbeatRecoveryD
           .where(
             and(
               eq(nativeRunFinalizations.runId, run.id),
+              owner?.bootId ? eq(nativeRunFinalizations.controllerBootId, owner.bootId)
+                : isNull(nativeRunFinalizations.controllerBootId),
               isNull(nativeRunFinalizations.resultId),
             ),
           );
@@ -2181,6 +2207,20 @@ export function createHeartbeatRecovery(db: Db, dependencies: HeartbeatRecoveryD
       )
       .where(eq(heartbeatRuns.status, "running"));
 
+    // Reuse the orphan sweep's active-run inventory. Only an expired, untracked
+    // controller creates recovery work; idle sweeps add no checkpoint query.
+    const expiredNativeRunIds = activeRuns.filter(({ run, nativeCoordinatorPhase, nativeControllerLeaseExpiresAt }) =>
+      run.runtimeMode === "native" && nativeCoordinatorPhase === "observed"
+      && (!nativeControllerLeaseExpiresAt || nativeControllerLeaseExpiresAt <= now)
+      && !isNativeRunnerOwnershipHeld(run)
+      && !activeRunExecutions.has(run.id) && !runningProcesses.has(run.id),
+    ).map(({ run }) => run.id);
+    const remoteRecoveryRunIds = new Set(await listExpiredNativeRunnerAuthorities(db, now, expiredNativeRunIds));
+    if (remoteRecoveryRunIds.size) {
+      const recovery = await recoverNativeRunsAfterRestart(now, [...remoteRecoveryRunIds]);
+      for (const claim of recovery.claims) resumedRunIds.add(claim.runId);
+    }
+
     const monitorIssueIds = [
       ...new Set(
         activeRuns.flatMap(({ run }) => {
@@ -2225,6 +2265,9 @@ export function createHeartbeatRecovery(db: Db, dependencies: HeartbeatRecoveryD
       nativeControllerLeaseExpiresAt,
       nativeWorkspaceFinalizationPending,
     } of activeRuns) {
+      // Native recovery owns these dispositions. Its writes happened after this
+      // inventory was read; do not reap or overwrite them using the old row.
+      if (remoteRecoveryRunIds.has(run.id)) continue;
       // Authentication timeout requires an explicit ownership resolution, not
       // repeated reattachment or a process-gone guess on subsequent sweeps.
       if (isNativeRunnerOwnershipHeld(run)) continue;

server/src/services/heartbeat/workspace-preparation.ts

@@ -1108,11 +1108,13 @@ async function prepareHeartbeatWorkspaceLocked(db: Db, input: HeartbeatWorkspace
     await controllerLease.assertOwned();
     const remoteRecovery = runOptions.nativeRestartRecovery?.kind === "reattach_remote_runner"
       ? runOptions.nativeRestartRecovery : null;
-    const recoveryWorkspace = remoteRecovery
+    const pluginRecovery = remoteRecovery?.remote.kind === "plugin_task" ? remoteRecovery.remote : null;
+    const sandboxRecovery = remoteRecovery && remoteRecovery.remote.kind !== "plugin_task" ? remoteRecovery.remote : null;
+    const recoveryWorkspace = sandboxRecovery
       ? readNativeWorkspaceSyncReference(parseObject(run.runnerProfileJson).nativeWorkspaceSync) : null;
-    if (remoteRecovery && (!recoveryWorkspace || remoteRecovery.runId !== run.id ||
-        recoveryWorkspace.providerLeaseId !== remoteRecovery.remote.providerLeaseId ||
-        recoveryWorkspace.remoteCwd !== remoteRecovery.remote.remoteCwd)) {
+    if (remoteRecovery && (remoteRecovery.runId !== run.id || (sandboxRecovery && (!recoveryWorkspace ||
+        recoveryWorkspace.providerLeaseId !== sandboxRecovery.providerLeaseId ||
+        recoveryWorkspace.remoteCwd !== sandboxRecovery.remoteCwd)))) {
       throw new Error("native_remote_recovery_lease_mismatch");
     }
     acquiredEnvironment = await timeWorkspace("workspace_environment_acquire", () => envOrchestrator.acquireForRun({
@@ -1132,10 +1134,10 @@ async function prepareHeartbeatWorkspaceLocked(db: Db, input: HeartbeatWorkspace
       agentId: agent.id,
       persistedExecutionWorkspace,
       executionWorkspaceSettings: environmentExecutionWorkspaceSettings,
-      ...(remoteRecovery && recoveryWorkspace ? { reattachRemoteLease: {
+      ...(pluginRecovery ? { reattachRemoteLease: pluginRecovery } : sandboxRecovery && recoveryWorkspace ? { reattachRemoteLease: {
         leaseId: recoveryWorkspace.leaseId,
-        providerLeaseId: remoteRecovery.remote.providerLeaseId,
-        remoteCwd: remoteRecovery.remote.remoteCwd,
+        providerLeaseId: sandboxRecovery.providerLeaseId,
+        remoteCwd: sandboxRecovery.remoteCwd,
       } } : {}),
     }));
     await controllerLease.assertOwned();

server/src/services/issue-title.ts

@@ -8,17 +8,21 @@ import { issueReferenceService } from "./issue-references.js";
 type TitleActor = Pick<LogActivityInput, "actorType" | "actorId" | "agentId" | "runId" | "agentApiKeyId">;
 type TitleResult = { id: string; title: string; titleNeedsGeneration: boolean; changed: boolean };
 type TitleReceipt = { issueId: string; title: string; onlyIfProvisional: boolean; result: TitleResult };
+type TitleMutationResult = { result: TitleResult; publication: ActivityPublication | null };
 
 const MAX_TASK_TITLE_RECEIPTS_PER_RUN = 64;
 
 /** Caller owns the transaction and publishes the activity only after commit. */
+export function setIssueTitle(tx: Db, companyId: string, issueId: string, input: SetIssueTitle, actor: TitleActor): Promise<TitleMutationResult>;
+export function setIssueTitle(tx: Db, companyId: string, issueId: string, input: SetIssueTitle, actor: TitleActor, options: { receiptOnly: true }): Promise<TitleMutationResult | null>;
 export async function setIssueTitle(
   tx: Db,
   companyId: string,
   issueId: string,
   input: SetIssueTitle,
   actor: TitleActor,
-): Promise<{ result: TitleResult; publication: ActivityPublication | null }> {
+  options?: { receiptOnly: true },
+): Promise<TitleMutationResult | null> {
   const { title, onlyIfProvisional, idempotencyKey } = setIssueTitleSchema.parse(input);
   const predicate = and(eq(issues.id, issueId), eq(issues.companyId, companyId));
   const [issue] = await tx.select().from(issues).where(predicate).for("update");
@@ -48,10 +52,12 @@ export async function setIssueTitle(
   const prior = receiptKey ? receipts[receiptKey] : undefined;
   if (prior) {
     if (prior.issueId !== issueId || prior.title !== title || prior.onlyIfProvisional !== onlyIfProvisional) {
+      if (options?.receiptOnly) return null;
       throw conflict("Task title idempotency key was already used with different arguments");
     }
     return { result: prior.result, publication: null };
   }
+  if (options?.receiptOnly) return null;
   // Keep all accepted keys replayable, including after a user edit. Reject new
   // keys at the limit instead of evicting receipts and allowing writes to replay.
   if (receiptKey && Object.keys(receipts).length >= MAX_TASK_TITLE_RECEIPTS_PER_RUN) {

server/src/services/native-runtime/agent-instruction-tools.integration.test.ts

@@ -5,12 +5,12 @@ import { createHash, randomUUID } from "node:crypto";
 import { eq, sql } from "drizzle-orm";
 import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
 import { activityLog, agents, authUsers, companies, companyMemberships, principalPermissionGrants, heartbeatRuns, issues, agentInstructionRevisions, createDb } from "@paperclipai/db";
-import { startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
+import { startRecoveryPostgres } from "../../__tests__/helpers/recovery-postgres.js";
 import { resolveManagedInstructionsRoot } from "../agent-instructions.js";
 import { PaperclipRunnerToolAuthority } from "./paperclip-runner-tool-authority.js";
 
 describe("canonical instruction tools through native authority", () => {
-  let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
+  let database: Awaited<ReturnType<typeof startRecoveryPostgres>>;
   let db: ReturnType<typeof createDb>;
   let home: string;
   const previousHome = process.env.PAPERCLIP_HOME;
@@ -21,7 +21,7 @@ describe("canonical instruction tools through native authority", () => {
   beforeAll(async () => {
     home = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "instruction-tool-home-")));
     process.env.PAPERCLIP_HOME = home;
-    database = await startEmbeddedPostgresTestDatabase("instruction-tool-db-");
+    database = await startRecoveryPostgres("instruction-tool-db-");
     db = createDb(database.connectionString);
   }, 90_000);
   afterAll(async () => {
@@ -151,6 +151,7 @@ describe("canonical instruction tools through native authority", () => {
       expect(audit.filter((event) => event.action === "agent.instruction_write_attempted")).toHaveLength(1);
       expect(audit.filter((event) => event.action === "agent.files_updated")).toHaveLength(0);
       authority = new PaperclipRunnerToolAuthority(db, { companyId, agentId, issueId, runId });
+      expect(await authority.recover(request)).toBeNull();
       await expect(authority.execute(request)).rejects.toMatchObject({ code: "semantic_tool_outcome_unknown" });
       await expect(authority.execute({ ...request, arguments: { ...request.arguments, content: "different" } }))
         .rejects.toThrow("idempotency_conflict");
@@ -166,11 +167,28 @@ describe("canonical instruction tools through native authority", () => {
     }
   });
 
-  it("replays a committed receipt after its database commit acknowledgement is lost", async () => {
+  it.each(["update_agent_instructions", "restore_agent_instructions"])("recovers %s after its database commit acknowledgement is lost without repeating the write", async (tool) => {
     const first = await call("read_agent_instructions");
-    const request = { tool: "update_agent_instructions", callId: randomUUID(), arguments: {
-      targetAgentId, entryFile, content: "committed but acknowledgement lost\n", baseRevisionId: first.revision.id,
+    const content = "committed but acknowledgement lost\n";
+    const revisionId = randomUUID();
+    if (tool === "restore_agent_instructions") {
+      // Legacy revision data remains valid input to the production restore tool.
+      await db.insert(agentInstructionRevisions).values({
+        id: revisionId, companyId, agentId: targetAgentId, entryFile, source: "tool",
+        contentBase64: Buffer.from(content).toString("base64"),
+        contentHash: createHash("sha256").update(content).digest("hex"), byteLength: Buffer.byteLength(content),
+      });
+    }
+    const request = { tool, callId: randomUUID(), arguments: {
+      targetAgentId, entryFile, baseRevisionId: first.revision.id,
+      ...(tool === "restore_agent_instructions" ? { revisionId } : { content }),
     } };
+    const originalRename = fs.rename.bind(fs);
+    let writes = 0;
+    const renameSpy = vi.spyOn(fs, "rename").mockImplementation(async (source, destination) => {
+      if (destination === path.join(root, entryFile)) writes++;
+      return originalRename(source, destination);
+    });
     const transaction = db.transaction.bind(db);
     let commits = 0;
     const spy = vi.spyOn(db, "transaction").mockImplementation(async (...args) => {
@@ -182,13 +200,46 @@ describe("canonical instruction tools through native authority", () => {
       await expect(authority.execute(request)).rejects.toThrow("injected lost commit acknowledgement");
       const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId));
       const receipt = (run!.resultJson as any).semanticToolReceipts[`instruction:${request.callId}`].result;
-      expect(receipt).toMatchObject({ changed: true, content: request.arguments.content });
+      expect(receipt).toMatchObject({ changed: true, content });
       authority = new PaperclipRunnerToolAuthority(db, { companyId, agentId, issueId, runId });
-      expect(await authority.execute(request)).toEqual(receipt);
+      expect(await authority.recover(request)).toEqual({ result: receipt });
+      expect(await authority.recover({ ...request, callId: randomUUID() })).toBeNull();
+      await expect(authority.recover({ ...request, arguments: { ...request.arguments, baseRevisionId: randomUUID() } }))
+        .rejects.toThrow("idempotency_conflict");
+      await db.update(heartbeatRuns).set({ status: "cancelled" }).where(eq(heartbeatRuns.id, runId));
+      await expect(authority.recover(request)).rejects.toThrow("paperclip_runner_tool_binding_not_authorized");
+      await db.update(heartbeatRuns).set({ status: "running" }).where(eq(heartbeatRuns.id, runId));
+      await db.delete(principalPermissionGrants).where(eq(principalPermissionGrants.principalId, userId));
+      await expect(authority.recover(request)).rejects.toMatchObject({ status: 403 });
+      const audit = await db.select().from(activityLog).where(eq(activityLog.runId, runId));
+      expect(audit.filter((event) => event.action === "agent.instruction_write_attempted")).toHaveLength(1);
+      expect(audit.filter((event) => event.action === "agent.files_updated")).toHaveLength(1);
+      expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(content);
+      expect(writes).toBe(1);
+    } finally { spy.mockRestore(); renameSpy.mockRestore(); }
+  });
+
+  it("holds recovery if a receipt disappears before its locked replay", async () => {
+    const first = await call("read_agent_instructions");
+    const request = { tool: "update_agent_instructions", callId: randomUUID(), arguments: {
+      targetAgentId, entryFile, content: "saved instructions", baseRevisionId: first.revision.id,
+    } };
+    await authority.execute(request);
+    // Make repeating the effect observable even though the original CAS would pass.
+    await fs.writeFile(path.join(root, entryFile), original);
+    const transaction = db.transaction.bind(db);
+    const spy = vi.spyOn(db, "transaction").mockImplementationOnce(async (...args) => {
+      await db.update(heartbeatRuns).set({ resultJson: {} }).where(eq(heartbeatRuns.id, runId));
+      return transaction(...args);
+    });
+    try {
+      await expect(authority.recover(request)).rejects.toMatchObject({ code: "semantic_tool_outcome_unknown" });
+      expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(original);
+      const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId));
+      expect(run!.resultJson).toEqual({});
       const audit = await db.select().from(activityLog).where(eq(activityLog.runId, runId));
       expect(audit.filter((event) => event.action === "agent.instruction_write_attempted")).toHaveLength(1);
       expect(audit.filter((event) => event.action === "agent.files_updated")).toHaveLength(1);
-      expect(await fs.readFile(path.join(root, entryFile), "utf8")).toBe(request.arguments.content);
     } finally { spy.mockRestore(); }
   });
 

server/src/services/native-runtime/native-case-tools.integration.test.ts

@@ -1,7 +1,7 @@
 import { randomUUID } from "node:crypto";
 import { and, eq } from "drizzle-orm";
 import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
-import { agents, agentWakeupRequests, caseAccessSources, caseDocuments, caseEvents, caseHandoffs, caseIssueLinks, cases, documents, heartbeatRuns, issues, pipelineCases, pipelines, pipelineStages } from "@paperclipai/db";
+import { agents, agentWakeupRequests, caseAccessSources, caseDocuments, caseEvents, caseHandoffs, caseIssueLinks, cases, documents, heartbeatRuns, issues, pipelineCases, pipelines, pipelineStages, workflows } from "@paperclipai/db";
 import { SYSTEM_ISSUE_DOCUMENT_KEYS } from "@paperclipai/shared";
 import { startRunnerApiTestServer } from "../../__tests__/helpers/runner-api-server.js";
 import { instanceSettingsService } from "../instance-settings.js";
@@ -83,6 +83,47 @@ describe("native case tools through authenticated production routes", () => {
       page = await f.call("read_case", { ...selector, offsetBytes, snapshot }) as DocumentPage;
     }
   }
+  it.each(["update_case", "create_workflow", "update_workflow"])("recovers a committed %s result without repeating its write", async tool => {
+    const f = await fixture();
+    const key = `lost-${tool}`;
+    let input: Record<string, unknown>;
+    if (tool === "update_case") input = { idempotencyKey: key, title: "Recovered case" };
+    else if (tool === "create_workflow") input = { idempotencyKey: key, name: "Recovered workflow", prompt: "Preserve this procedure." };
+    else {
+      const created = await f.call("create_workflow", { idempotencyKey: "seed", name: "Before", prompt: "Preserve this procedure." }) as { data: { id: string } };
+      input = { idempotencyKey: key, workflowId: created.data.id, name: "Recovered workflow" };
+    }
+    const original = await f.call(tool, input);
+    expect(original).toMatchObject({ ok: true });
+    const snapshot = async () => ({
+      cases: await server.db.select().from(cases).where(eq(cases.id, f.record.id)),
+      events: await server.db.select().from(caseEvents).where(eq(caseEvents.caseId, f.record.id)),
+      workflows: await server.db.select().from(workflows).where(eq(workflows.companyId, f.companyId)),
+    });
+    const before = await snapshot();
+    const replacement = new PaperclipRunnerToolAuthority(server.db, { ...f, casesEnabled: true, workMode: "standard" });
+    const recover = (arguments_: unknown = input) => replacement.recover({ tool, callId: "lost-provider-response", arguments: arguments_ });
+    const fetch = vi.spyOn(globalThis, "fetch").mockRejectedValue(new Error("Recovery must not dispatch HTTP"));
+    try {
+      expect(await recover()).toEqual({ result: original });
+      expect(await recover({ ...input, idempotencyKey: "no-receipt" })).toBeNull();
+      expect(await recover({ ...input, ...(tool === "update_case" ? { title: "Different" } : { name: "Different" }) })).toBeNull();
+      const [run] = await server.db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.runId));
+      const completed = run!.resultJson!;
+      const receipts = completed.apiToolReceipts as Record<string, Record<string, unknown>>;
+      await server.db.update(heartbeatRuns).set({ resultJson: { ...completed, apiToolReceipts:
+        Object.fromEntries(Object.entries(receipts).map(([id, receipt]) => [id, { ...receipt, state: "pending" }])) } }).where(eq(heartbeatRuns.id, f.runId));
+      expect(await recover()).toBeNull();
+      await server.db.update(heartbeatRuns).set({ resultJson: completed }).where(eq(heartbeatRuns.id, f.runId));
+      const disabled = new PaperclipRunnerToolAuthority(server.db, { ...f, casesEnabled: false, workMode: "standard" });
+      await expect(disabled.recover({ tool, callId: "lost-provider-response", arguments: input })).rejects.toThrow("Cases are disabled");
+      await server.db.update(heartbeatRuns).set({ status: "cancelled" }).where(eq(heartbeatRuns.id, f.runId));
+      await expect(recover()).rejects.toThrow("paperclip_runner_tool_binding_not_authorized");
+      expect(await snapshot()).toEqual(before);
+      expect(fetch).not.toHaveBeenCalled();
+    } finally { fetch.mockRestore(); }
+  });
+
   it("exposes case facts and creation without a second document tool surface", async () => {
     const f = await fixture();
     const names = f.authority.definitions().map(tool => tool.name);

server/src/services/native-runtime/native-execution-lease.test.ts

@@ -0,0 +1,93 @@
+import { afterEach, describe, expect, it, vi } from "vitest";
+import type { Db } from "@paperclipai/db";
+import { startNativeSessionExecutionLeaseRenewal } from "./native-execution-lease.js";
+
+function fixture() {
+  const returning = vi.fn(async () => [{ runId: "run" }]);
+  const update = vi.fn(() => ({ set: () => ({ where: () => ({ returning }) }) }));
+  const transaction = vi.fn(async (run: (tx: unknown) => Promise<unknown>) => run({ update, execute: vi.fn() }));
+  const db = { update, transaction };
+  return { update, returning, transaction, input: { db: db as unknown as Db, runId: "run", companyId: "company", issueId: "issue",
+    leaseOwner: "owner", attempt: 1, controller: { bootId: "boot", pid: 1, processStartedAt: new Date() }, remote: true,
+    onLost: vi.fn(async () => undefined) } };
+}
+afterEach(() => vi.useRealTimers());
+describe("native execution renewal ownership", () => {
+  it("keeps a successfully renewed lease through repeated slow connection acquisition", async () => {
+    vi.useFakeTimers();
+    const { input, update, transaction } = fixture();
+    const connected = transaction.getMockImplementation()!;
+    transaction.mockImplementation(async run => {
+      await new Promise(resolve => setTimeout(resolve, 30_000));
+      return connected(run);
+    });
+    const guard = startNativeSessionExecutionLeaseRenewal(input);
+    try {
+      // Renewals start at 20s/60s and extend the database lease at 50s/90s.
+      // Connection waits must not shorten the successfully extended ownership.
+      await vi.advanceTimersByTimeAsync(95_000);
+      expect(update).toHaveBeenCalledTimes(2);
+      expect(input.onLost).not.toHaveBeenCalled();
+    } finally { await guard.stop().catch(() => undefined); }
+    expect(vi.getTimerCount()).toBe(0);
+  });
+  it("shares one timer across preparation and execution and stops idle SQL after both release", async () => {
+    vi.useFakeTimers();
+    const { input, update } = fixture();
+    const preparation = startNativeSessionExecutionLeaseRenewal(input);
+    const execution = startNativeSessionExecutionLeaseRenewal(input);
+    expect(update).not.toHaveBeenCalled();
+    await vi.advanceTimersByTimeAsync(40_000);
+    expect(update).toHaveBeenCalledTimes(2);
+    await preparation.stop();
+    await preparation.stop();
+    await vi.advanceTimersByTimeAsync(20_000);
+    expect(update).toHaveBeenCalledTimes(3);
+    await execution.stop();
+    await vi.advanceTimersByTimeAsync(120_000);
+    expect(update).toHaveBeenCalledTimes(3);
+    expect(vi.getTimerCount()).toBe(0);
+  });
+  it("retries cancelled SQL within the original deadline without losing healthy ownership", async () => {
+    vi.useFakeTimers();
+    const { input, returning } = fixture();
+    returning.mockRejectedValueOnce(Object.assign(new Error("statement timeout"), { code: "57014" }));
+    const guard = startNativeSessionExecutionLeaseRenewal(input);
+    await vi.advanceTimersByTimeAsync(60_000);
+    expect(returning).toHaveBeenCalledTimes(3);
+    expect(input.onLost).not.toHaveBeenCalled();
+    await guard.stop();
+    expect(vi.getTimerCount()).toBe(0);
+  });
+  it("detaches at the deadline while a database response is missing and cannot regain authority from a late response", async () => {
+    vi.useFakeTimers();
+    const { input, returning } = fixture();
+    let complete!: (value: { runId: string }[]) => void;
+    returning.mockImplementationOnce(() => new Promise(resolve => { complete = resolve; }));
+    const guard = startNativeSessionExecutionLeaseRenewal(input);
+    await vi.advanceTimersByTimeAsync(60_000);
+    expect(returning).toHaveBeenCalledTimes(1);
+    expect(input.onLost).toHaveBeenCalledTimes(1);
+    complete([{ runId: "run" }]);
+    await expect(guard.stop()).rejects.toThrow("native_session_lease_renewal_deadline");
+    await vi.advanceTimersByTimeAsync(120_000);
+    expect(returning).toHaveBeenCalledTimes(1);
+    expect(vi.getTimerCount()).toBe(0);
+  });
+  it("does not queue SQL behind a slow renewal and fences a lost owner", async () => {
+    vi.useFakeTimers();
+    const { input, update, returning } = fixture();
+    let complete!: (value: { runId: string }[]) => void;
+    returning.mockImplementationOnce(() => new Promise(resolve => { complete = resolve; }));
+    const guard = startNativeSessionExecutionLeaseRenewal(input);
+    await vi.advanceTimersByTimeAsync(50_000);
+    expect(update).toHaveBeenCalledTimes(1);
+    complete([]);
+    await vi.advanceTimersByTimeAsync(100_000);
+    expect(update).toHaveBeenCalledTimes(1);
+    expect(input.onLost).toHaveBeenCalledTimes(1);
+    expect(() => startNativeSessionExecutionLeaseRenewal(input)).toThrow("native_session_lease_lost");
+    await expect(guard.stop()).rejects.toThrow("native_session_lease_lost");
+    expect(vi.getTimerCount()).toBe(0);
+  });
+});

server/src/services/native-runtime/native-execution-lease.ts

@@ -0,0 +1,166 @@
+import type { Db } from "@paperclipai/db";
+import { livenessDb, nativeRunFinalizations } from "@paperclipai/db";
+import { and, eq, gt, sql } from "drizzle-orm";
+import { currentNativeControllerIdentity, type NativeControllerIdentity } from "./native-restart-recovery.js";
+import { REMOTE_CONTROLLER_LEASE_MS } from "./native-runner-recovery-contract.js";
+
+export const NATIVE_SESSION_EXECUTION_LEASE_TTL_MS = 20 * 60_000;
+const NATIVE_SESSION_EXECUTION_LEASE_RENEW_INTERVAL_MS = 5 * 60_000;
+const RENEWAL_STATEMENT_TIMEOUT_MS = 10_000;
+
+export async function renewNativeSessionExecutionLease(input: {
+  db: Db;
+  runId: string;
+  companyId: string;
+  issueId: string;
+  leaseOwner: string;
+  attempt: number;
+  controller?: NativeControllerIdentity;
+  leaseTtlMs?: number;
+}, { statementTimeoutMs = RENEWAL_STATEMENT_TIMEOUT_MS } = {}): Promise<void> {
+  await renewNativeSessionExecutionLeaseDeadline(input, { statementTimeoutMs });
+}
+
+async function renewNativeSessionExecutionLeaseDeadline(
+  input: Parameters<typeof renewNativeSessionExecutionLease>[0],
+  { statementTimeoutMs = RENEWAL_STATEMENT_TIMEOUT_MS } = {},
+): Promise<number> {
+  const controller =
+    input.controller ?? (await currentNativeControllerIdentity());
+  const leaseTtlMs = input.leaseTtlMs ?? NATIVE_SESSION_EXECUTION_LEASE_TTL_MS;
+  if (
+    !Number.isInteger(leaseTtlMs) ||
+    leaseTtlMs < 1_000 ||
+    leaseTtlMs > NATIVE_SESSION_EXECUTION_LEASE_TTL_MS
+  ) {
+    throw new Error("native_session_lease_ttl_invalid");
+  }
+  if (!Number.isSafeInteger(statementTimeoutMs) || statementTimeoutMs < 1 || statementTimeoutMs > RENEWAL_STATEMENT_TIMEOUT_MS) {
+    throw new Error("native_session_renewal_timeout_invalid");
+  }
+  // Use the existing private pool; application queries must not consume the
+  // remote controller's short renewal window. The timeout cancels actual SQL.
+  let confirmedDeadline = 0;
+  const [updated] = await livenessDb(input.db).transaction(async tx => {
+    await tx.execute(sql`select set_config('statement_timeout', ${`${statementTimeoutMs}ms`}, true)`);
+    // Check expiry after acquiring the row lock. An UPDATE alone can evaluate
+    // its predicate before waiting and revive an owner that expired in the wait.
+    await tx.execute(sql`select 1 from ${nativeRunFinalizations}
+      where ${nativeRunFinalizations.runId} = ${input.runId}
+        and ${nativeRunFinalizations.companyId} = ${input.companyId}
+        and ${nativeRunFinalizations.issueId} = ${input.issueId} for update`);
+    // The database extends from UPDATE time. Exclude connection/lock waits,
+    // but conservatively include UPDATE/commit response time without assuming
+    // that this server's wall clock agrees with the database's clock.
+    confirmedDeadline = performance.now() + leaseTtlMs;
+    return tx.update(nativeRunFinalizations)
+      .set({
+        leaseExpiresAt: sql`clock_timestamp() + (${leaseTtlMs} * interval '1 millisecond')`,
+        updatedAt: sql`clock_timestamp()`,
+      })
+      .where(
+        and(
+          eq(nativeRunFinalizations.runId, input.runId),
+          eq(nativeRunFinalizations.companyId, input.companyId),
+          eq(nativeRunFinalizations.issueId, input.issueId),
+          eq(nativeRunFinalizations.leaseOwner, input.leaseOwner),
+          eq(nativeRunFinalizations.attempt, input.attempt),
+          eq(nativeRunFinalizations.controllerBootId, controller.bootId),
+          eq(nativeRunFinalizations.controllerPid, controller.pid),
+          eq(
+            nativeRunFinalizations.controllerProcessStartedAt,
+            controller.processStartedAt,
+          ),
+          gt(nativeRunFinalizations.leaseExpiresAt, sql`clock_timestamp()`),
+        ),
+      )
+      .returning({ runId: nativeRunFinalizations.runId });
+  });
+  if (!updated) throw new Error("native_session_lease_lost");
+  return confirmedDeadline;
+}
+
+type RenewalInput = {
+  db: Db; runId: string; companyId: string; issueId: string;
+  leaseOwner: string; attempt: number; controller: NativeControllerIdentity;
+  remote?: boolean;
+  onLost: () => Promise<unknown>;
+};
+export interface NativeExecutionLeaseRenewal { stop(): Promise<void>; }
+
+// Preparation and execution share one renewal timer. Each caller releases its
+// own reference; the final release drains any SQL already in flight.
+const renewals = new WeakMap<Db, Map<string, {
+  input: RenewalInput; references: number; timer: ReturnType<typeof setInterval>;
+  pending: Promise<void>; busy: boolean; lost: Error | null;
+  deadline: ReturnType<typeof setTimeout>; loss: Promise<unknown>;
+}>>();
+
+export function startNativeSessionExecutionLeaseRenewal(input: RenewalInput): NativeExecutionLeaseRenewal {
+  let owners = renewals.get(input.db);
+  if (!owners) { owners = new Map(); renewals.set(input.db, owners); }
+  let state = owners.get(input.leaseOwner);
+  if (state) {
+    if (state.input.runId !== input.runId || state.input.companyId !== input.companyId
+      || state.input.issueId !== input.issueId || state.input.attempt !== input.attempt
+      || state.input.controller.bootId !== input.controller.bootId || state.input.remote !== input.remote) {
+      throw new Error("native_session_renewal_binding_changed");
+    }
+    if (state.lost) throw state.lost;
+    state.references += 1;
+  } else {
+    const ttlMs = input.remote ? REMOTE_CONTROLLER_LEASE_MS : NATIVE_SESSION_EXECUTION_LEASE_TTL_MS;
+    const lose = (error: unknown) => {
+      if (state!.lost) return;
+      state!.lost = error instanceof Error ? error : new Error("native_session_lease_lost");
+      clearInterval(state!.timer);
+      clearTimeout(state!.deadline);
+      state!.loss = Promise.resolve().then(input.onLost).catch(() => undefined);
+    };
+    const deadline = (delay: number) => {
+      const timer = setTimeout(() => lose(new Error("native_session_lease_renewal_deadline")), Math.max(0, delay));
+      timer.unref?.();
+      return timer;
+    };
+    const renew = () => {
+      if (state!.lost || state!.busy) return;
+      state!.busy = true;
+      state!.pending = renewNativeSessionExecutionLeaseDeadline({ ...input, leaseTtlMs: ttlMs })
+        .then(confirmedDeadline => {
+          // A late database response cannot restore local authority after loss
+          // or recreate a timer after the final caller has stopped.
+          if (state!.lost || state!.references === 0) return;
+          clearTimeout(state!.deadline);
+          state!.deadline = deadline(confirmedDeadline - performance.now());
+        })
+        .catch((error: unknown) => {
+          // A cancelled statement can retry at the next tick, but does not
+          // extend the ownership deadline. Never queue overlapping attempts.
+          for (let cause = error, depth = 0; cause && depth < 4; cause = (cause as { cause?: unknown }).cause, depth++) {
+            if ((cause as { code?: unknown }).code === "57014") return;
+          }
+          lose(error);
+        }).finally(() => { state!.busy = false; });
+    };
+    const timer = setInterval(renew, input.remote ? REMOTE_CONTROLLER_LEASE_MS / 3 : NATIVE_SESSION_EXECUTION_LEASE_RENEW_INTERVAL_MS);
+    timer.unref?.();
+    state = { input, references: 1, timer, pending: Promise.resolve(), busy: false, lost: null,
+      deadline: deadline(ttlMs), loss: Promise.resolve() };
+    owners.set(input.leaseOwner, state);
+  }
+  const owned = state;
+  let stopped = false;
+  return { stop: async () => {
+    if (!stopped) {
+      stopped = true;
+      if (--owned.references === 0) {
+        clearInterval(owned.timer);
+        clearTimeout(owned.deadline);
+        owners!.delete(input.leaseOwner);
+      }
+    }
+    await owned.pending;
+    await owned.loss;
+    if (owned.lost) throw owned.lost;
+  } };
+}

server/src/services/native-runtime/native-failed-workspace-recovery.test.ts

@@ -370,7 +370,7 @@ const support = externalUrl ? { supported: true } : await getEmbeddedPostgresTes
     if (kind === "generation") restartRecovery.controllerGeneration += 1;
     if (kind === "attempt") restartRecovery.providerAttempt += 1;
     if (kind === "run") restartRecovery.runId = randomUUID();
-    if (kind === "remote" && restartRecovery.kind === "reattach_remote_runner") restartRecovery.remote.remoteCwd = "/other";
+    if (kind === "remote" && restartRecovery.kind === "reattach_remote_runner" && restartRecovery.remote.kind !== "plugin_task") restartRecovery.remote.remoteCwd = "/other";
     if (kind === "expired") await db.update(nativeRunFinalizations).set({ leaseExpiresAt: new Date(0) }).where(eq(nativeRunFinalizations.runId, f.runId));
     if (kind === "released") await db.update(nativeRunFinalizations).set({ leaseOwner: null }).where(eq(nativeRunFinalizations.runId, f.runId));
     await expect(preserveNativeFailedWorkspace(db, await readRun(f.runId), { allowPreterminal: true, restartRecovery }))

server/src/services/native-runtime/native-failed-workspace-recovery.ts

@@ -33,7 +33,7 @@ export async function preserveNativeFailedWorkspace(db: Db, selected: Pick<typeo
       && claim.runId === selected.id && claim.leaseOwner === coordinator.leaseOwner
       && claim.controllerGeneration === coordinator.controllerGeneration && claim.providerAttempt === coordinator.attempt
       && coordinator.leaseExpiresAt && coordinator.leaseExpiresAt.getTime() > Date.now()
-      && (claim.kind !== "reattach_remote_runner" || (claim.remote.providerLeaseId === reference.providerLeaseId
+      && (claim.kind !== "reattach_remote_runner" || (claim.remote.kind !== "plugin_task" && claim.remote.providerLeaseId === reference.providerLeaseId
         && claim.remote.remoteCwd === reference.remoteCwd)));
     if ((coordinator?.leaseOwner || (claim && coordinator?.phase === "observed")) && !ownsClaim) throw new NativeWorkspaceFinalizationBusyError();
     if (!coordinator || coordinator.resultId
@@ -44,7 +44,7 @@ export async function preserveNativeFailedWorkspace(db: Db, selected: Pick<typeo
     if (!run || run.runtimeMode !== "native" || (!options.allowPreterminal && run.status !== "failed")) return;
     const currentReference = readNativeWorkspaceSyncReference(run.runnerProfileJson?.nativeWorkspaceSync);
     if (!currentReference || currentReference.state !== "prepared"
-      || (claim?.kind === "reattach_remote_runner" && (claim.remote.providerLeaseId !== currentReference.providerLeaseId
+      || (claim?.kind === "reattach_remote_runner" && (claim.remote.kind === "plugin_task" || claim.remote.providerLeaseId !== currentReference.providerLeaseId
         || claim.remote.remoteCwd !== currentReference.remoteCwd))) return;
     const [source] = await tx.select().from(environmentLeases).where(and(
       eq(environmentLeases.id, currentReference.leaseId), eq(environmentLeases.companyId, run.companyId),

server/src/services/native-runtime/native-harness-backup-expiration.ts

@@ -1,18 +1,21 @@
 import { randomUUID } from "node:crypto";
 import { promises as fs } from "node:fs";
 import path from "node:path";
-import { and, eq, inArray, isNull, notInArray, or, sql } from "drizzle-orm";
+import { and, asc, eq, gt, inArray, isNull, lt, or, sql } from "drizzle-orm";
 import {
   agentTaskSessions,
   environmentLeases,
   heartbeatRuns,
   issues,
   nativeRunFinalizations,
+  nativeRunnerCheckpoints,
   type Db,
 } from "@paperclipai/db";
 import { logger } from "../../middleware/logger.js";
 import { nativeHarnessStateBase } from "./native-harness-backup-stamp.js";
 import { nativeSessionScopeDigestIsBusy } from "./native-session-executor.js";
+import type { createDeliveryWorkCoordinator } from "../delivery-work-coordinator.js";
+import { DELIVERY_QUEUES } from "../delivery-work-notifications.js";
 
 export const NATIVE_HARNESS_BACKUP_EXPIRATION_INTERVAL_MS = 60 * 60 * 1000;
 export const NATIVE_HARNESS_BACKUP_EXPIRATION_STARTUP_DELAY_MS = 2 * 60 * 1000;
@@ -33,6 +36,7 @@ const SETTLED_FINALIZATION_PHASES = ["committed", "terminal_failure"];
 const LIVE_LEASE_STATUSES = ["active", "released", "retained", "pending_cleanup"];
 const CLOSED_ISSUE_STATUSES = ["done", "cancelled"];
 const MAX_CONTROL_PLANE_STATE_BYTES = 4 * 1024 * 1024;
+const CHECKPOINT_EXPIRATION_MAX_FAILED_PASSES = 3;
 
 export type NativeHarnessBackupExpirationOptions = {
   closedForDays?: number;
@@ -57,6 +61,8 @@ export type NativeHarnessBackupExpirationResult = {
   evidenceCandidateBytes: number;
   evidenceRemoved: number;
   evidenceReclaimedBytes: number;
+  expiredCheckpoints: number;
+  checkpointCandidates: number;
   failed: number;
   truncated: boolean;
   skipped: Record<string, number>;
@@ -90,6 +96,8 @@ function emptyResult(): NativeHarnessBackupExpirationResult {
     evidenceCandidateBytes: 0,
     evidenceRemoved: 0,
     evidenceReclaimedBytes: 0,
+    expiredCheckpoints: 0,
+    checkpointCandidates: 0,
     failed: 0,
     truncated: false,
     skipped: {},
@@ -153,8 +161,8 @@ type Tx = Parameters<Parameters<Db["transaction"]>[0]>[0];
 
 async function sessionRetentionBlocker(
   tx: Tx,
-  input: { scopeDigest: string; identity: SessionIdentity; closedBefore: Date },
-): Promise<{ reason: string } | { companyId: string; agentId: string }> {
+  input: { scopeDigest: string | null; identity: SessionIdentity; closedBefore: Date; checkpoint?: boolean },
+): Promise<{ reason: string } | { companyId: string; agentId: string; runIds: string[] }> {
   const [owner] = await tx
     .select({ companyId: heartbeatRuns.companyId, agentId: heartbeatRuns.agentId })
     .from(heartbeatRuns)
@@ -178,12 +186,14 @@ async function sessionRetentionBlocker(
   if (runs.some((run) => LIVE_RUN_STATUSES.includes(run.status) || !run.finishedAt)) {
     return { reason: "live_run" };
   }
+  if (input.checkpoint && runs.some(run => run.finishedAt! >= input.closedBefore)) return { reason: "recent_run" };
   const issueIds = [...new Set(runs.map((run) => run.issueId).filter((id): id is string => Boolean(id)))];
   if (issueIds.length === 0) return { reason: "issue_unknown" };
   const issueRows = await tx
     .select({
       id: issues.id,
       status: issues.status,
+      executionRunId: issues.executionRunId,
       closedAt: sql<Date | null>`coalesce(${issues.completedAt}, ${issues.cancelledAt})`,
     })
     .from(issues)
@@ -192,28 +202,33 @@ async function sessionRetentionBlocker(
   if (issueRows.length !== issueIds.length) return { reason: "issue_unknown" };
   for (const issue of issueRows) {
     if (!CLOSED_ISSUE_STATUSES.includes(issue.status)) return { reason: "issue_open" };
+    if (input.checkpoint && issue.executionRunId) return { reason: "live_run" };
     const closedAt = issue.closedAt ? new Date(issue.closedAt) : null;
     if (!closedAt || closedAt >= input.closedBefore) return { reason: "recently_closed" };
   }
   const runIds = runs.map((run) => run.id);
-  const [pending] = await tx
-    .select({ runId: nativeRunFinalizations.runId })
+  const finalizationQuery = tx
+    .select({ phase: nativeRunFinalizations.phase, recoveryState: nativeRunFinalizations.recoveryState,
+      leaseOwner: nativeRunFinalizations.leaseOwner })
     .from(nativeRunFinalizations)
-    .where(
-      and(
-        inArray(nativeRunFinalizations.runId, runIds),
-        or(
-          notInArray(nativeRunFinalizations.phase, SETTLED_FINALIZATION_PHASES),
-          sql`${nativeRunFinalizations.leaseOwner} is not null`,
-        ),
-      ),
-    )
-    .limit(1);
-  if (pending) return { reason: "pending_finalization" };
+    .where(inArray(nativeRunFinalizations.runId, runIds))
+    .orderBy(asc(nativeRunFinalizations.runId));
+  const finalizations = await (input.checkpoint ? finalizationQuery.for("update") : finalizationQuery);
+  if ((input.checkpoint && finalizations.length !== runIds.length)
+    || finalizations.some(row => !SETTLED_FINALIZATION_PHASES.includes(row.phase) || row.leaseOwner)) {
+    return { reason: "pending_finalization" };
+  }
   const leaseReferences = [
-    sql`${environmentLeases.metadata}->'nativeHarnessBackup'->>'sessionScopeSha256' = ${input.scopeDigest}`,
     inArray(environmentLeases.heartbeatRunId, runIds),
   ];
+  if (input.scopeDigest) leaseReferences.push(
+    sql`${environmentLeases.metadata}->'nativeHarnessBackup'->>'sessionScopeSha256' = ${input.scopeDigest}`,
+  );
+  if (input.checkpoint) leaseReferences.push(
+    inArray(environmentLeases.issueId, issueIds),
+    and(eq(environmentLeases.companyId, owner.companyId),
+      sql`${environmentLeases.metadata}->'nativeHarnessBackup'->>'normalizedSessionId' = ${input.identity.normalizedSessionId}`)!,
+  );
   if (input.identity.environmentLeaseId) {
     leaseReferences.push(eq(environmentLeases.id, input.identity.environmentLeaseId));
   }
@@ -223,7 +238,7 @@ async function sessionRetentionBlocker(
     .where(and(inArray(environmentLeases.status, LIVE_LEASE_STATUSES), or(...leaseReferences)))
     .limit(1);
   if (lease) return { reason: "live_lease" };
-  return owner;
+  return { ...owner, runIds };
 }
 
 async function hasLiveRun(db: Db, owner: { companyId: string; agentId: string }, sessionId: string) {
@@ -254,6 +269,68 @@ export function nativeHarnessBackupExpirationService(
   let evidencePass = false;
   let evidenceDueAt = 0;
   let passDueAt = Number.POSITIVE_INFINITY;
+  let checkpointCursor: string | null = null;
+
+  async function expireCheckpoints(options: {
+    closedBefore: Date; dryRun: boolean; maxEntries: number; maxRemovals: number; shouldContinue: () => boolean;
+  }, result: NativeHarnessBackupExpirationResult) {
+    // The partial header index enumerates retained checkpoints without reading
+    // event payloads or scanning runs that have no checkpoint left to expire.
+    const candidates = await db.select({ runId: heartbeatRuns.id, sessionId: heartbeatRuns.nativeSessionId })
+      .from(nativeRunnerCheckpoints)
+      .innerJoin(nativeRunFinalizations, eq(nativeRunFinalizations.runId, nativeRunnerCheckpoints.runId))
+      .innerJoin(heartbeatRuns, eq(heartbeatRuns.id, nativeRunFinalizations.runId))
+      .where(and(
+        sql`${nativeRunnerCheckpoints.part} = 'header'`,
+        checkpointCursor ? gt(nativeRunnerCheckpoints.runId, checkpointCursor) : undefined,
+        lt(heartbeatRuns.finishedAt, options.closedBefore),
+        inArray(nativeRunFinalizations.phase, SETTLED_FINALIZATION_PHASES),
+        isNull(nativeRunFinalizations.leaseOwner), isNull(nativeRunFinalizations.recoveryState),
+      ))
+      .orderBy(asc(nativeRunnerCheckpoints.runId)).limit(options.maxEntries + 1);
+    let processed = 0;
+    let removed = 0;
+    for (const candidate of candidates) {
+      if (!options.shouldContinue() || processed >= options.maxEntries || removed >= options.maxRemovals) {
+        result.truncated = true;
+        return;
+      }
+      processed++;
+      checkpointCursor = candidate.runId;
+      if (!candidate.sessionId) continue; // Unknown identity is never expiration evidence.
+      try {
+        const expired = await db.transaction(async tx => {
+          const blocker = await sessionRetentionBlocker(tx, {
+            scopeDigest: null, checkpoint: true, closedBefore: options.closedBefore,
+            identity: { runId: candidate.runId, normalizedSessionId: candidate.sessionId!, environmentLeaseId: null },
+          });
+          if ("reason" in blocker) return false;
+          const [resume] = await tx.select({ id: agentTaskSessions.id }).from(agentTaskSessions).where(and(
+            eq(agentTaskSessions.companyId, blocker.companyId), eq(agentTaskSessions.agentId, blocker.agentId),
+            or(inArray(agentTaskSessions.lastRunId, blocker.runIds),
+              sql`${agentTaskSessions.sessionParamsJson}->>'sessionId' = ${candidate.sessionId}`),
+          )).limit(1);
+          if (resume) return false;
+          result.checkpointCandidates++;
+          if (options.dryRun) return false;
+          // Serialize with checkpoint writers and retire every old authority for
+          // this session, including a predecessor authorized for warm handoff.
+          await tx.update(nativeRunFinalizations).set({
+            controllerGeneration: sql`${nativeRunFinalizations.controllerGeneration} + 1`,
+          }).where(inArray(nativeRunFinalizations.runId, blocker.runIds));
+          const rows = await tx.delete(nativeRunnerCheckpoints).where(and(
+            eq(nativeRunnerCheckpoints.companyId, blocker.companyId), eq(nativeRunnerCheckpoints.runId, candidate.runId),
+          )).returning({ part: nativeRunnerCheckpoints.part });
+          return rows.length > 0;
+        });
+        if (expired) { removed++; result.expiredCheckpoints++; }
+      } catch (err) {
+        result.failed++;
+        logger.warn({ err, runId: candidate.runId }, "native runner checkpoint expiration skipped a run");
+      }
+    }
+    checkpointCursor = null;
+  }
 
   async function settleLeftovers(backupRoot: string) {
     const current = path.join(backupRoot, "current");
@@ -583,5 +660,57 @@ export function nativeHarnessBackupExpirationService(
     return result;
   }
 
-  return { tick, noteEvidence };
+  async function tickCheckpoints(options: NativeHarnessBackupExpirationOptions = {}) {
+    const closedForDays = Math.max(1, options.closedForDays ?? defaults.closedForDays ?? NATIVE_HARNESS_BACKUP_EXPIRATION_DEFAULT_DAYS);
+    const result = emptyResult();
+    const shouldContinue = options.shouldContinue ?? defaults.shouldContinue ?? (() => true);
+    if (!shouldContinue()) return { ...result, truncated: true };
+    await expireCheckpoints({
+      closedBefore: new Date((options.now ?? defaults.now ?? new Date()).getTime() - closedForDays * DAY_MS),
+      dryRun: options.dryRun ?? defaults.dryRun ?? false,
+      maxEntries: Math.max(1, options.maxEntries ?? defaults.maxEntries ?? 200),
+      maxRemovals: Math.max(1, options.maxRemovals ?? defaults.maxRemovals ?? 50),
+      shouldContinue,
+    }, result);
+    return result;
+  }
+
+  function registerCheckpointExpiration(coordinator: ReturnType<typeof createDeliveryWorkCoordinator>) {
+    // Opportunistic startup retention, not a recurring expiry obligation. A
+    // protected/recent checkpoint is reconsidered on the next server startup.
+    // Persisted rows remain safe; seven days is a minimum age, not a deadline.
+    let pending = true;
+    let retryFailedPage = false;
+    let failedPasses = 0;
+    const failedPass = (err?: unknown) => {
+      pending = ++failedPasses < CHECKPOINT_EXPIRATION_MAX_FAILED_PASSES;
+      logger.warn({ err, failedPasses }, pending
+        ? "startup checkpoint expiration will retry a failed pass"
+        : "startup checkpoint expiration retries exhausted; retained checkpoints deferred to next startup");
+    };
+    return coordinator.register(DELIVERY_QUEUES.nativeCheckpointExpiration, {
+      retryMs: 1000,
+      pausePassiveMaintenance: true,
+      run: async () => {
+        if (!pending) return;
+        try {
+          const result = await tickCheckpoints();
+          retryFailedPage ||= result.failed > 0;
+          pending = result.truncated || retryFailedPage;
+          if (!result.truncated) {
+            if (retryFailedPage) failedPass();
+            retryFailedPage = false;
+          }
+          if (result.checkpointCandidates || result.failed) logger.info(result, "startup native checkpoint expiration");
+        } catch (err) {
+          // Candidate-query failures need the same finite startup budget as
+          // per-row failures; the shared coordinator otherwise retries forever.
+          failedPass(err);
+        }
+      },
+      hasPending: async () => pending,
+    });
+  }
+
+  return { tick, noteEvidence, tickCheckpoints, registerCheckpointExpiration };
 }

server/src/services/native-runtime/native-restart-recovery.test.ts

@@ -8,6 +8,15 @@ import {
 } from "./native-restart-recovery.js";
 
 describe("native restart recovery classification", () => {
+  it.each([true, false])("uses the remote fenced lease instead of the old container PID (live=%s)", async live => {
+    const now = new Date();
+    const probe = vi.fn(() => { throw new Error("must not probe another container's PID"); });
+    const result = await evaluateNativeControllerTakeover({ now, remoteLeaseFenced: true,
+      owner: { leaseOwner: "old-controller", leaseExpiresAt: new Date(now.getTime() + (live ? 1000 : -1000)),
+        controllerPid: process.pid, controllerProcessStartedAt: now }, isProcessAlive: probe });
+    expect(result.allowed).toBe(!live);
+    expect(probe).not.toHaveBeenCalled();
+  });
   it("does not reopen a failed checkpoint or reset an exhausted provider budget", () => {
     const evidence = {
       runnerPidAlive: false,

server/src/services/native-runtime/native-restart-recovery.ts

@@ -1,4 +1,5 @@
 import { notifyExecutionRunWork } from "../execution-control-notifications.js";
+import { hasNativeRunnerCheckpoint } from "./native-runner-checkpoint.js";
 import { readNativeWorkspaceSyncReference } from "./native-workspace-sync.js";
 import { activeNativeExecutionRunIds, NATIVE_PREPARATION_FENCING_VERSION } from "./native-active-runs.js";
 import {
@@ -17,6 +18,7 @@ import {
   issues,
   nativeRunFinalizations,
 } from "@paperclipai/db";
+import { REMOTE_CONTROLLER_LEASE_MS } from "./native-runner-recovery-contract.js";
 import { readProcessStartedAt } from "../hot-restart.js";
 import { getServerInfoSnapshot } from "../../server-info.js";
 import { redactSensitiveText } from "../../redaction.js";
@@ -56,7 +58,8 @@ export type NativeRestartRecoveryClaim =
       providerAttempt: number;
       restartKind: NativeRestartKind;
       recoveryRequestId: string | null;
-      remote: { providerLeaseId: string; remoteCwd: string };
+      remote: { kind?: "sandbox"; providerLeaseId: string; remoteCwd: string }
+        | { kind: "plugin_task"; leaseId: string };
     }
   | {
       kind: "resume_dead_runner";
@@ -180,8 +183,14 @@ export async function evaluateNativeControllerTakeover(input: {
   isProcessAlive?: (pid: number) => boolean;
   readProcessStartedAt?: (pid: number) => Promise<Date | null>;
   requireStoppedController?: boolean;
+  remoteLeaseFenced?: boolean;
 }): Promise<{ allowed: boolean; reason: string }> {
   const { owner, now } = input;
+  if (input.remoteLeaseFenced) {
+    return owner.leaseExpiresAt && owner.leaseExpiresAt > now
+      ? { allowed: false, reason: "remote_controller_lease_live" }
+      : { allowed: true, reason: "remote_controller_lease_released_or_expired" };
+  }
   const isAlive = input.isProcessAlive ?? processIsAlive;
   const readStartedAt = input.readProcessStartedAt ?? observedProcessStart;
   if ((!owner.leaseOwner || !owner.leaseExpiresAt) && !input.requireStoppedController) {
@@ -468,6 +477,10 @@ export async function claimNativeRestartRecoveries(input: {
   liveRunExecutions?: Pick<ReadonlySet<string>, "has">;
   now?: Date;
   limit?: number;
+  /** Called after each committed claim, before scanning the next candidate. */
+  onClaim?: (claim: NativeRestartRecoveryClaim, scope: {
+    companyId: string; issueId: string; controller: NativeControllerIdentity; remote: boolean;
+  }) => void;
   coordinatedPreviousController?: {
     pid: number;
     processStartedAt: Date | null;
@@ -515,6 +528,8 @@ export async function claimNativeRestartRecoveries(input: {
 
   const dispositions: NativeRestartRecoveryDisposition[] = [];
   for (const candidate of candidates) {
+    const now = input.now ?? new Date();
+    let claimScope: { companyId: string; issueId: string; controller: NativeControllerIdentity; remote: boolean } | undefined;
     // Set inside the transaction only when the write below genuinely
     // transitions the run into "failed". Read after the transaction
     // commits, so a rolled-back write never reports a false failure.
@@ -589,9 +604,21 @@ export async function claimNativeRestartRecoveries(input: {
         } as const;
       }
 
+      const profile = row.run.runnerProfileJson ?? {};
+      const remoteWorkspace = readNativeWorkspaceSyncReference(profile.nativeWorkspaceSync);
+      const [pluginTask] = await tx.select({ id: environmentLeases.id }).from(environmentLeases).where(and(
+        eq(environmentLeases.companyId, row.run.companyId), eq(environmentLeases.heartbeatRunId, row.run.id),
+        sql`${environmentLeases.metadata}->>'runnerTaskLease' = 'true'`,
+      )).orderBy(desc(environmentLeases.createdAt)).limit(1);
+      const remoteSandbox = profile.nativeWorkspaceSync != null;
+      const remoteProcess = remoteSandbox || !!pluginTask;
+      const durableAuthority = (remoteWorkspace || pluginTask) && await hasNativeRunnerCheckpoint(tx, {
+        companyId: row.run.companyId, issueId: row.coordinator.issueId, runId: row.run.id,
+      });
       const takeover = await evaluateNativeControllerTakeover({
         owner: row.coordinator,
         now,
+        remoteLeaseFenced: Boolean(durableAuthority),
         coordinatedPreviousController:
           input.coordinatedPreviousController ?? null,
       });
@@ -633,16 +660,6 @@ export async function claimNativeRestartRecoveries(input: {
         } as const;
       }
 
-      const profile = row.run.runnerProfileJson ?? {};
-      const remoteWorkspace = readNativeWorkspaceSyncReference(
-        profile.nativeWorkspaceSync,
-      );
-      const [pluginTask] = await tx.select({ id: environmentLeases.id }).from(environmentLeases).where(and(
-        eq(environmentLeases.companyId, row.run.companyId), eq(environmentLeases.heartbeatRunId, row.run.id),
-        sql`${environmentLeases.metadata}->>'runnerTaskLease' = 'true'`,
-      )).limit(1);
-      const remoteSandbox = profile.nativeWorkspaceSync != null;
-      const remoteProcess = remoteSandbox || !!pluginTask;
       const runnerPidAlive =
         !remoteProcess && processIsAlive(row.run.processPid);
       const runnerGroupAlive =
@@ -822,7 +839,7 @@ export async function claimNativeRestartRecoveries(input: {
       }
 
       const classification = classifyNativeRunnerRecoveryEvidence({
-        remoteSandbox,
+        remoteSandbox: remoteSandbox || Boolean(pluginTask && durableAuthority),
         unclaimedBootstrap,
         runnerPidAlive,
         runnerGroupAlive,
@@ -990,7 +1007,9 @@ export async function claimNativeRestartRecoveries(input: {
         .set({
           phase: "observed",
           leaseOwner,
-          leaseExpiresAt: new Date(now.getTime() + 20 * 60_000),
+          // Cleanup and earlier candidates can outlive the batch observation.
+          // The owner row is locked: grant the full lease at this write.
+          leaseExpiresAt: sql`clock_timestamp() + ${durableAuthority ? REMOTE_CONTROLLER_LEASE_MS : 20 * 60_000} * interval '1 millisecond'`,
           controllerBootId: controller.bootId,
           controllerPid: controller.pid,
           controllerProcessStartedAt: controller.processStartedAt,
@@ -1059,6 +1078,7 @@ export async function claimNativeRestartRecoveries(input: {
         })
         .where(eq(heartbeatRuns.id, row.run.id));
 
+      claimScope = { companyId: row.run.companyId, issueId: row.coordinator.issueId, controller, remote: Boolean(durableAuthority) };
       const common = {
         runId: row.run.id,
         leaseOwner,
@@ -1071,7 +1091,7 @@ export async function claimNativeRestartRecoveries(input: {
         return {
           kind: claimKind,
           ...common,
-          remote: {
+          remote: pluginTask ? { kind: "plugin_task", leaseId: pluginTask.id } : {
             providerLeaseId: remoteWorkspace!.providerLeaseId,
             remoteCwd: remoteWorkspace!.remoteCwd,
           },
@@ -1095,6 +1115,7 @@ export async function claimNativeRestartRecoveries(input: {
     });
     if (terminalRunToReport)
       void reportRunFailure(input.db, terminalRunToReport);
+    if (claimScope && "leaseOwner" in disposition) input.onClaim?.(disposition, claimScope);
     dispositions.push(disposition);
   }
   return dispositions;

server/src/services/native-runtime/native-runner-checkpoint.ts

@@ -0,0 +1,188 @@
+import { and, eq, exists, gt, isNull, lte, inArray, or, sql } from "drizzle-orm";
+import { checkpointDb, heartbeatRuns, issues, nativeRunFinalizations, nativeRunnerCheckpoints, type Db } from "@paperclipai/db";
+import { decodeDurablePrpCheckpoint, type DurablePrpCheckpoint, type DurablePrpControlPlane } from "../../vendor/paperclip-runner/index.js";
+import { createLocalEncryptedCipher } from "../../secrets/local-encrypted-provider.js";
+
+type Binding = { companyId: string; issueId: string; runId: string };
+type Owner = Binding & { bootId: string; leaseOwner: string; generation: number; authorityRunId: string };
+function encryptParts(binding: Binding, records: ReadonlyMap<string, string>) {
+  if (!records.size) return [];
+  const cipher = createLocalEncryptedCipher("write");
+  return [...records].map(([part, contents]) => ({
+    companyId: binding.companyId, issueId: binding.issueId, runId: binding.runId, part,
+    material: cipher.encrypt(JSON.stringify({ ...binding, part, contents })),
+  }));
+}
+
+function decryptPart(cipher: ReturnType<typeof createLocalEncryptedCipher>, binding: Binding,
+  part: string, material: Record<string, unknown>): string {
+  const envelope = JSON.parse(cipher.decrypt(material));
+  if (envelope.companyId !== binding.companyId || envelope.issueId !== binding.issueId
+    || envelope.runId !== binding.runId || envelope.part !== part || typeof envelope.contents !== "string") {
+    throw new Error("native_runner_checkpoint_binding_mismatch");
+  }
+  return envelope.contents;
+}
+
+export function nativeRunnerCheckpointWriter(db: Db, owner: Owner) {
+  const persistenceDb = checkpointDb(db);
+  let previousWork = new Set<string>();
+  return async (checkpoint: DurablePrpCheckpoint): Promise<void> => {
+    if (checkpoint.runId !== owner.runId) throw new Error("native_runner_checkpoint_binding_mismatch");
+    const changed = encryptParts(owner, checkpoint.changedRecords);
+    await persistenceDb.transaction(async tx => {
+      // A locking CTE acquires ownership before the outer projection samples
+      // wall time. Sampling in the locking SELECT can precede its lock wait.
+      // Keep both steps in one statement without another database round trip.
+      const locked = tx.$with("checkpoint_owner").as(tx.select().from(nativeRunFinalizations)
+        .where(and(eq(nativeRunFinalizations.companyId, owner.companyId),
+          eq(nativeRunFinalizations.issueId, owner.issueId), eq(nativeRunFinalizations.runId, owner.authorityRunId),
+          eq(nativeRunFinalizations.controllerBootId, owner.bootId),
+          eq(nativeRunFinalizations.controllerGeneration, owner.generation))).for("update"));
+      const [owned] = await tx.with(locked).select({ leaseOwner: locked.leaseOwner,
+        leaseExpiresAt: locked.leaseExpiresAt, resultId: locked.resultId, now: sql<Date>`clock_timestamp()` }).from(locked);
+      if (!owned) throw new Error("native_runner_checkpoint_owner_lost");
+      const current = owned;
+      const activeOwner = current.leaseOwner === owner.leaseOwner
+        && current.leaseExpiresAt !== null && current.leaseExpiresAt > new Date(owned.now);
+      if (!activeOwner) {
+        const newWork = checkpoint.workCommandIds.some(id => !previousWork.has(id));
+        const [task] = await tx.select({ executionRunId: issues.executionRunId }).from(issues)
+          .where(and(eq(issues.companyId, owner.companyId), eq(issues.id, owner.issueId)));
+        const [successor] = task?.executionRunId && task.executionRunId !== owner.runId
+          ? await tx.select({ runId: nativeRunFinalizations.runId }).from(nativeRunFinalizations).where(and(
+              eq(nativeRunFinalizations.runId, task.executionRunId), eq(nativeRunFinalizations.companyId, owner.companyId),
+              eq(nativeRunFinalizations.issueId, owner.issueId), eq(nativeRunFinalizations.controllerBootId, owner.bootId),
+              gt(nativeRunFinalizations.leaseExpiresAt, sql`clock_timestamp()`))) : [];
+        if (!current.resultId || newWork || !task || (task.executionRunId && task.executionRunId !== owner.runId && !successor)) {
+          throw new Error("native_runner_checkpoint_owner_lost");
+        }
+      }
+      if (checkpoint.predecessor) {
+        const [prior] = await tx.select({ id: heartbeatRuns.id }).from(heartbeatRuns).where(and(
+          eq(heartbeatRuns.id, checkpoint.predecessor.runId), eq(heartbeatRuns.companyId, owner.companyId),
+          eq(heartbeatRuns.nativeIssueId, owner.issueId),
+          eq(heartbeatRuns.nativeSessionId, checkpoint.predecessor.normalizedSessionId)));
+        if (!prior) throw new Error("native_runner_checkpoint_binding_mismatch");
+      }
+      if (checkpoint.reset) await tx.delete(nativeRunnerCheckpoints).where(and(
+        eq(nativeRunnerCheckpoints.companyId, owner.companyId), eq(nativeRunnerCheckpoints.runId, owner.runId)));
+      else if (checkpoint.removedRecords.length) await tx.delete(nativeRunnerCheckpoints).where(and(
+        eq(nativeRunnerCheckpoints.companyId, owner.companyId), eq(nativeRunnerCheckpoints.runId, owner.runId),
+        inArray(nativeRunnerCheckpoints.part, [...checkpoint.removedRecords])));
+      if (changed.length) await tx.insert(nativeRunnerCheckpoints).values(changed)
+        .onConflictDoUpdate({ target: [nativeRunnerCheckpoints.runId, nativeRunnerCheckpoints.part],
+          set: { material: sql`excluded.material`, updatedAt: sql`now()` } });
+    });
+    previousWork = new Set(checkpoint.workCommandIds);
+  };
+}
+
+const authorities = new WeakMap<DurablePrpControlPlane, Map<string, ReturnType<typeof nativeRunnerCheckpointWriter>>>();
+
+export function nativeRunnerCheckpointPersistence(db: Db, bootId: string, authorityRunId: string | (() => string),
+  detach: (runIds: readonly string[], leaseOwner?: string) => Promise<unknown>) {
+  return { db, bootId, authorityRunId, onFailure: async (runId: string, leaseOwner: string) => {
+    try { await detach([runId], leaseOwner); }
+    finally {
+      await db.update(nativeRunFinalizations).set({ leaseExpiresAt: sql`clock_timestamp()` })
+        .where(and(eq(nativeRunFinalizations.runId, runId), eq(nativeRunFinalizations.controllerBootId, bootId), eq(nativeRunFinalizations.leaseOwner, leaseOwner),
+          isNull(nativeRunFinalizations.resultId)));
+    }
+  } };
+}
+
+/** Attachment identity owns the stored records; current business authority owns
+ * the write lease. They differ when a completed warm session is recovered. */
+export function nativeRunnerCheckpointRegistration(input: Omit<Parameters<typeof registerNativeRunnerCheckpoint>[0], "authority">) {
+  return (authority: DurablePrpControlPlane, identity?: { runId: string }) => registerNativeRunnerCheckpoint({
+    ...input, authority, binding: { ...input.binding, runId: identity?.runId ?? input.binding.runId },
+  });
+}
+
+export async function registerNativeRunnerCheckpoint(input: {
+  db: Db; binding: Binding; bootId: string; authority: DurablePrpControlPlane;
+  authorityRunId?: string | (() => string);
+  onFailure?: (runId: string, leaseOwner: string) => Promise<void>;
+}): Promise<void> {
+  const authorityRunId = typeof input.authorityRunId === "function"
+    ? input.authorityRunId() : input.authorityRunId ?? input.binding.runId;
+  const [owner] = await input.db.select().from(nativeRunFinalizations).where(and(
+    eq(nativeRunFinalizations.companyId, input.binding.companyId),
+    eq(nativeRunFinalizations.issueId, input.binding.issueId),
+    eq(nativeRunFinalizations.runId, authorityRunId),
+    eq(nativeRunFinalizations.controllerBootId, input.bootId),
+    gt(nativeRunFinalizations.leaseExpiresAt, sql`now()`)));
+  if (!owner?.leaseOwner) throw new Error("native_runner_checkpoint_owner_lost");
+  let writers = authorities.get(input.authority);
+  if (!writers) {
+    writers = new Map();
+    const boundWriters = writers;
+    input.authority.setCheckpointPersistence(async snapshot => {
+      const writer = boundWriters.get(snapshot.runId);
+      if (!writer) throw new Error("native_runner_checkpoint_owner_lost");
+      await writer(snapshot);
+    });
+    authorities.set(input.authority, writers);
+  }
+  if (!writers.has(input.binding.runId)) {
+    const write = nativeRunnerCheckpointWriter(input.db, {
+      ...input.binding, authorityRunId, bootId: input.bootId, generation: owner.controllerGeneration, leaseOwner: owner.leaseOwner,
+    });
+    writers.set(input.binding.runId, async checkpoint => {
+      try { await write(checkpoint); }
+      catch (error) {
+        void input.onFailure?.(authorityRunId, owner.leaseOwner!).catch(() => undefined);
+        throw error;
+      }
+    });
+  }
+  await input.authority.flushCheckpoint();
+}
+
+export async function loadNativeRunnerCheckpoint(db: Db, binding: Binding): Promise<DurablePrpCheckpoint | null> {
+  const rows = await db.select().from(nativeRunnerCheckpoints).where(and(
+    eq(nativeRunnerCheckpoints.companyId, binding.companyId), eq(nativeRunnerCheckpoints.issueId, binding.issueId),
+    eq(nativeRunnerCheckpoints.runId, binding.runId)));
+  if (!rows.length) return null;
+  const cipher = createLocalEncryptedCipher("read");
+  const parts = new Map(rows.map(row =>
+    [row.part, decryptPart(cipher, binding, row.part, row.material)] as const));
+  const checkpoint = decodeDurablePrpCheckpoint(parts);
+  if (checkpoint.runId !== binding.runId) throw new Error("native_runner_checkpoint_binding_mismatch");
+  return checkpoint;
+}
+
+export async function restoreNativeRunnerCheckpoint(db: Db, binding: Binding, root: string, normalizedSessionId?: string) {
+  const checkpoint = await loadNativeRunnerCheckpoint(db, binding);
+  if (!checkpoint) return null;
+  if (normalizedSessionId !== undefined && checkpoint.recoveryEvidence.normalizedSessionId !== normalizedSessionId) {
+    throw new Error("native_runner_checkpoint_binding_mismatch");
+  }
+  checkpoint.restoreSessionDirectory(root);
+  return checkpoint;
+}
+
+export async function hasNativeRunnerCheckpoint(db: Pick<Db, "select">, binding: Binding): Promise<boolean> {
+  const [row] = await db.select({ runId: nativeRunnerCheckpoints.runId }).from(nativeRunnerCheckpoints).where(and(
+    eq(nativeRunnerCheckpoints.companyId, binding.companyId), eq(nativeRunnerCheckpoints.issueId, binding.issueId),
+    eq(nativeRunnerCheckpoints.runId, binding.runId))).limit(1);
+  return row !== undefined;
+}
+
+export async function listExpiredNativeRunnerAuthorities(db: Db, now: Date, runIds: string[]): Promise<string[]> {
+  if (!runIds.length) return [];
+  // The caller supplies the finite expired active-run inventory. Return all of
+  // it: a blocked owner's unchanged row must not starve later recoverable runs.
+  const rows = await db.select({ runId: nativeRunFinalizations.runId }).from(nativeRunFinalizations)
+    .innerJoin(heartbeatRuns, eq(heartbeatRuns.id, nativeRunFinalizations.runId))
+    .where(and(inArray(nativeRunFinalizations.runId, runIds),
+      eq(heartbeatRuns.status, "running"), eq(nativeRunFinalizations.phase, "observed"),
+      isNull(nativeRunFinalizations.resultId),
+      or(isNull(nativeRunFinalizations.leaseExpiresAt), lte(nativeRunFinalizations.leaseExpiresAt, now)),
+      exists(db.select({ runId: nativeRunnerCheckpoints.runId }).from(nativeRunnerCheckpoints).where(and(
+        eq(nativeRunnerCheckpoints.runId, nativeRunFinalizations.runId),
+        eq(nativeRunnerCheckpoints.issueId, nativeRunFinalizations.issueId),
+        eq(nativeRunnerCheckpoints.companyId, nativeRunFinalizations.companyId))))));
+  return rows.map(row => row.runId);
+}

server/src/services/native-runtime/native-runner-execution-driver.ts

@@ -5,6 +5,8 @@ import type { RunnerExecutionDriver } from "../../vendor/paperclip-runner/index.
 import { registerRunnerPrpAuthority } from "../../realtime/runner-prp-ws.js";
 import { connectRunnerPrpIngress } from "../../realtime/runner-prp-outbound.js";
 import type { NativeRunTrace } from "./native-run-trace.js";
+import type { Db } from "@paperclipai/db";
+import { nativeRunnerCheckpointRegistration } from "./native-runner-checkpoint.js";
 
 type ConnectionRegistration = NonNullable<RunnerExecutionDriver["controlPlaneRegistration"]>;
 type RunnerIdentity = Parameters<ConnectionRegistration>[1];
@@ -21,6 +23,7 @@ export interface NativeRunnerExecutionDriverInput {
   processStarted?: Promise<void>;
   processIdentityReady?: Promise<void>;
   beforeConnect?: (identity?: RunnerIdentity) => Promise<void>;
+  controllerPersistence?: { db: Db; bootId: string; authorityRunId?: string | (() => string); onFailure?: (runId: string, leaseOwner: string) => Promise<void> };
   checkpoint?: Awaited<ReturnType<ConnectionRegistration>>["checkpoint"];
   process?: Omit<RunnerExecutionDriver, "controlPlaneRegistration">;
   trace?: NativeRunTrace;
@@ -91,6 +94,11 @@ export function createNativeRunnerExecutionDriver(input: NativeRunnerExecutionDr
     ...input.process,
     controlPlaneRegistration: async (authority, attachmentIdentity) => {
       await input.beforeConnect?.(attachmentIdentity);
+      if (input.controllerPersistence && input.binding.issueId) {
+        await nativeRunnerCheckpointRegistration({ ...input.controllerPersistence,
+          binding: { companyId: input.binding.companyId, issueId: input.binding.issueId, runId: input.binding.runId },
+        })(authority, attachmentIdentity);
+      }
       return measureNativeRunnerSpan(
         input.trace,
         "runner.transport.connect",

server/src/services/native-runtime/native-runner-recovery-contract.ts

@@ -0,0 +1,2 @@
+export const REMOTE_RUNNER_RECONNECT_GRACE_MS = 7 * 60_000;
+export const REMOTE_CONTROLLER_LEASE_MS = 60_000;

server/src/services/native-runtime/native-session-executor.test.ts

@@ -1,3 +1,5 @@
+import * as runnerCheckpoint from "./native-runner-checkpoint.js";
+import { DurablePrpControlPlane } from "../../vendor/paperclip-runner/index.js";
 import * as environmentTaskRuntime from "../environment-task-runtime.js";
 import * as pluginTaskRuntime from "./plugin-runner-task.js";
 import * as completionFeedback from "./native-completion-feedback.js";
@@ -84,6 +86,7 @@ import {
 import { nativeRuntimeContextFixture } from "./runtime-context.test-fixture.js";
 import { nativeToolContractFingerprintForTarget } from "./native-session-resume.js";
 import * as nativeSessionResume from "./native-session-resume.js";
+import * as nativeLeaseRenewal from "./native-execution-lease.js";
 import { buildNativeHeartbeatPreparationSpans, createNativeRunTrace } from "./native-run-trace.js";
 import { NativeRunnerOwnershipUnverifiedError } from "./native-runner-ownership.js";
 import type { AdapterRuntimeEvent } from "../../adapters/index.js";
@@ -1079,6 +1082,7 @@ fs.readFileSync = function(path, ...args) {
       await vi.waitFor(() => expect(healthyProbes).toBe(1), { timeout: 5_000 });
       expect(settled).toBe(false);
       expect(handle.isExitConfirmed?.()).toBe(false);
+      expect(handle.startedAt).toBe("2026-10-19T00:00:00.000Z");
       expect(() => process.kill(child!.pid!, 0)).not.toThrow();
       expect((await readFile(marker, "utf8")).split("\n")[1]).toBe(String(child!.pid));
       expect(execute.mock.calls.filter(([x]) => x.args?.[2] === "paperclip-runner-launch")).toHaveLength(1);
@@ -6038,6 +6042,9 @@ describe("native governed waits", () => {
 });
 
 type LeaseCoordinator = {
+  controllerBootId?: string;
+  controllerPid?: number;
+  controllerGeneration?: number;
   runId: string;
   companyId: string;
   issueId: string;
@@ -6298,6 +6305,52 @@ describe("native startup cancellation fence", () => {
 });
 
 describe("native startup restart detachment", () => {
+  it.each(["ordinary", "recovered", "stale-claim", "newer-detach"] as const)("preserves same-server detach fencing for %s admission", async mode => {
+    const issueService = vi.spyOn(issueServiceModule, "issueService").mockReturnValue({
+      update: vi.fn(async () => ({ status: "blocked", statusVersion: 1 })),
+    } as unknown as ReturnType<typeof issueServiceModule.issueService>);
+    const current = structuredClone(execution);
+    current.binding.runId = `same-server-recovery-${mode}`;
+    const { currentNativeControllerIdentity } = await import("./native-restart-recovery.js");
+    const controller = await currentNativeControllerIdentity();
+    await detachNativeSessionsForRestart([current.binding.runId]);
+    const db = leaseDb(current, { leaseOwner: "recovered-controller", controllerBootId: controller.bootId,
+      controllerPid: controller.pid, controllerGeneration: 2, attempt: 1 });
+    let newerDetach: Promise<unknown> | undefined;
+    const transaction = db.transaction.bind(db);
+    if (mode === "newer-detach") db.transaction = (async operation => {
+      const result = await transaction(operation);
+      newerDetach ??= detachNativeSessionsForRestart([current.binding.runId]);
+      return result;
+    }) as Db["transaction"];
+    const admitted = vi.fn();
+    state.execute.mockReset().mockImplementationOnce(async options => {
+      await options.onSession({ detachControllerForRestart: async () => undefined });
+      admitted();
+      throw new NativeControllerDetachedForRestartError();
+    });
+    try {
+      const admission = executePaperclipNativeSession({ db, execution: current, runnerInstanceId: "runner",
+        leaseOwner: "recovered-controller", ...(mode === "ordinary" ? {} : { restartRecovery: {
+          kind: "bootstrap_incomplete" as const, runId: current.binding.runId, leaseOwner: "recovered-controller",
+          controllerGeneration: mode === "stale-claim" ? 1 : 2, providerAttempt: 1,
+          restartKind: "hard" as const, recoveryRequestId: null,
+        } }) });
+      if (mode === "recovered") {
+        await expect(admission).rejects.toBeInstanceOf(NativeControllerDetachedForRestartError);
+        expect(admitted).toHaveBeenCalledOnce();
+        expect(state.execute).toHaveBeenCalledOnce();
+      } else if (mode === "stale-claim") {
+        await expect(admission).rejects.toThrow("native_restart_recovery_claim_changed");
+        expect(state.execute).not.toHaveBeenCalled();
+      } else {
+        await expect(admission).rejects.toBeInstanceOf(NativeControllerDetachedForRestartError);
+        expect(admitted).not.toHaveBeenCalled();
+        if (mode === "ordinary") expect(state.execute).not.toHaveBeenCalled();
+      }
+    } finally { await newerDetach; issueService.mockRestore(); }
+  });
+
   it("uses one admission deadline for the whole restart sweep", async () => {
     const inputs = ["first", "second"].map(label => {
       const input = structuredClone(execution);
@@ -7688,7 +7741,8 @@ describe("native session execution lease fencing", () => {
       .mockResolvedValueOnce([]);
     const where = vi.fn(() => ({ returning }));
     const set = vi.fn(() => ({ where }));
-    const db = { update: vi.fn(() => ({ set })) } as unknown as Db;
+    const update = vi.fn(() => ({ set }));
+    const db = { update, transaction: async (run: (tx: unknown) => Promise<unknown>) => run({ update, execute: vi.fn() }) } as unknown as Db;
     const input = {
       db,
       runId: "run-lease",
@@ -8082,7 +8136,7 @@ describe("native warm session supervision", () => {
     const { promisify } = await import("node:util");
     const { execFile } = await import("node:child_process");
     const executeCommand = promisify(execFile);
-    const { startEmbeddedPostgresTestDatabase } = await import("../../__tests__/helpers/embedded-postgres.js");
+    const { startRecoveryPostgres } = await import("../../__tests__/helpers/recovery-postgres.js");
     const { agentInstructionWorkingCopyService } = await import("../agent-instruction-working-copies.js");
     const { buildNativeRuntimeContext } = await import("./runtime-context.js");
     const { prepareNativeHeartbeatRun } = await import("./prepare-native-run.js");
@@ -8090,11 +8144,7 @@ describe("native warm session supervision", () => {
     const home = await realpath(await mkdtemp(join(tmpdir(), "warm-remote-db-")));
     const previousHome = process.env.PAPERCLIP_HOME;
     process.env.PAPERCLIP_HOME = home;
-    const { resolveExternalTestDatabaseUrl } = await import("../../__tests__/helpers/external-test-database.js");
-    const externalUrl = resolveExternalTestDatabaseUrl("PAPERCLIP_TEST_DATABASE_URL");
-    const database = externalUrl
-      ? { connectionString: externalUrl, cleanup: () => tables.closeRegisteredClients(externalUrl) }
-      : await startEmbeddedPostgresTestDatabase("warm-remote-db-");
+    const database = await startRecoveryPostgres("warm-remote-db-");
     const db = tables.createDb(database.connectionString);
     const companyId = randomUUID(), agentId = randomUUID(), userId = randomUUID(), environmentId = randomUUID(), issueId = randomUUID();
     const { resolveManagedInstructionsRoot } = await import("../agent-instructions.js");
@@ -8330,9 +8380,6 @@ describe("native warm session supervision", () => {
     finally {
       const cleanupFailures: unknown[] = [];
       try { await closeWarmNativeSessionsForEnvironment({ environmentId, reason: "test cleanup" }); } catch (error) { cleanupFailures.push(error); }
-      if (externalUrl) {
-        try { await db.execute(sql`TRUNCATE TABLE ${tables.companies}, ${tables.environments}, ${tables.authUsers} CASCADE`); } catch (error) { cleanupFailures.push(error); }
-      }
       try { await database.cleanup(); } catch (error) { cleanupFailures.push(error); }
       if (previousHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = previousHome;
       try {
@@ -10194,6 +10241,44 @@ describe("native warm session supervision", () => {
 });
 
 describe("native session bounded recovery", () => {
+  it("owns renewal from fresh claim commit through preparation failure", async () => {
+    const fresh = { ...structuredClone(execution), provider: { kind: "acpx", agent: "claude",
+      model: "claude-sonnet-4-6", profile: QUALIFIED_ACPX_PROFILES.claude } } as NativeExecutionInput;
+    const db = leaseDb(fresh);
+    const select = db.select.bind(db);
+    const failure = new Error("accounting history unavailable");
+    vi.spyOn(db, "select").mockImplementation((...args: unknown[]) => {
+      const query = (select as (...args: unknown[]) => ReturnType<Db["select"]>)(...args);
+      const from = query.from.bind(query);
+      query.from = ((table: unknown) => {
+        if (table === heartbeatRunEvents) throw failure;
+        return from(table as never);
+      }) as typeof query.from;
+      return query;
+    });
+    const stop = vi.fn(async () => undefined);
+    const renew = vi.spyOn(nativeLeaseRenewal, "startNativeSessionExecutionLeaseRenewal").mockReturnValue({ stop });
+    let reached!: () => void, release!: () => void;
+    const claiming = new Promise<void>(resolve => { reached = resolve; });
+    const preparation = new Promise<void>(resolve => { release = resolve; });
+    const running = executePaperclipNativeSession({ db, execution: fresh, runnerInstanceId: "fresh-runner",
+      runnerExecutionTarget: { kind: "remote", transport: "sandbox", remoteCwd: "/workspace", environmentId: "fresh-environment", leaseId: "fresh-lease" },
+      onEvent: async event => {
+        if (event.payload?.span === "native.coordinator.claim") { reached(); await preparation; }
+      },
+    }).catch(error => error);
+    try {
+      await claiming;
+      expect(renew).toHaveBeenCalledExactlyOnceWith(expect.objectContaining({
+        db, runId: fresh.binding.runId, attempt: 1, remote: true,
+      }));
+      expect(stop).not.toHaveBeenCalled();
+      release();
+      expect(await running).toBe(failure);
+      expect(stop).toHaveBeenCalledOnce();
+    } finally { release(); await running; renew.mockRestore(); }
+  });
+
   it("persists closed remote recovery evidence without replacing saved run results", async () => {
     const updates: Array<{ table: unknown; values: Record<string, unknown> }> = [];
     const onEvent = vi.fn(async () => undefined);
@@ -11219,9 +11304,9 @@ describe("runnerd provider runtime wiring", () => {
     await rm(isolatedStateDirectory, { recursive: true, force: true });
   });
 
-  it.each((["codex", "claude"] as const).flatMap(provider => (["unchanged", "changed", "runtime-context", "retirement-failed", "stale-owner", "active-owner", "missing-owner", "foreign-environment", "plugin-replaced", "plugin-managed-github"] as const).filter(scenario => provider === "codex" || scenario.startsWith("plugin-")).map(scenario => [provider, scenario] as const)))("rotates an owned warm %s remote runner before checking changed configuration: %s", async (provider, scenario) => {
+  it.each((["codex", "claude"] as const).flatMap(provider => (["unchanged", "external-checkpoint", "changed", "runtime-context", "retirement-failed", "stale-owner", "active-owner", "missing-owner", "foreign-environment", "plugin-replaced", "plugin-managed-github"] as const).filter(scenario => provider === "codex" || scenario.startsWith("plugin-")).map(scenario => [provider, scenario] as const)))("rotates an owned warm %s remote runner before checking changed configuration: %s", async (provider, scenario) => {
     const plugin = scenario === "plugin-replaced" || scenario === "plugin-managed-github";
-    const reusable = scenario === "unchanged" || scenario === "plugin-managed-github";
+    const reusable = ["unchanged", "external-checkpoint", "plugin-managed-github"].includes(scenario);
     const { permissionPolicy: _permissionPolicy, ...claudeProfile } = QUALIFIED_ACPX_PROFILES.claude;
     const first = { ...scopedNativeExecution(`warm-admission-${provider}-${scenario}`),
       schema: "paperclip.native-execution-input.v5" as const,
@@ -11230,7 +11315,7 @@ describe("runnerd provider runtime wiring", () => {
     if (provider === "claude") first.session.driverKind = "acpx_runtime";
     first.session.lifecyclePolicy = { mode: "warm", idleTimeoutMs: 60_000 };
     const next = { ...first, binding: { ...first.binding, runId: `${first.binding.runId}-next` },
-      session: { ...first.session, lifecyclePolicy: { mode: "warm" as const, idleTimeoutMs: ["unchanged", "runtime-context", "stale-owner", "missing-owner", "plugin-replaced", "plugin-managed-github"].includes(scenario) ? 60_000 : 90_000 } } };
+      session: { ...first.session, lifecyclePolicy: { mode: "warm" as const, idleTimeoutMs: ["unchanged", "external-checkpoint", "runtime-context", "stale-owner", "missing-owner", "plugin-replaced", "plugin-managed-github"].includes(scenario) ? 60_000 : 90_000 } } };
     if (scenario === "runtime-context") {
       next.runtimeContext = { ...first.runtimeContext,
         mcp: { ...first.runtimeContext.mcp, digest: "1".repeat(64) } };
@@ -11239,7 +11324,7 @@ describe("runnerd provider runtime wiring", () => {
     const environmentId = `warm-admission-${scenario}`;
     const target = { kind: "remote" as const, transport: "sandbox" as const, environmentId,
       remoteCwd: "/home/daytona/workspace", runner: { execute: vi.fn() } };
-    const identity = { runId: first.binding.runId, normalizedSessionId: first.session.normalizedSessionId,
+    const identity = { runId: first.binding.runId, normalizedSessionId: first.session.normalizedSessionId!,
       runnerInstanceId: `runner-${scenario}`, environmentLeaseId: first.binding.executionWorkspaceId };
     let replacementPending=false;
     const acquisition={outcome:"resumed" as "resumed"|"replacement",providerLeaseId:"task-first"};
@@ -11264,6 +11349,23 @@ describe("runnerd provider runtime wiring", () => {
 
     const root = await initializeScopedRunnerRoot(first, identity.runnerInstanceId);
     await writeRunnerAuthority(root, identity, "ready");
+    // Use the real control plane and persistence callback, not a handwritten marker.
+    const saved = new Map<string, import("../../vendor/paperclip-runner/index.js").DurablePrpCheckpoint>();
+    let persistedController: DurablePrpControlPlane | undefined;
+    const restore = scenario === "external-checkpoint"
+      ? vi.spyOn(runnerCheckpoint, "restoreNativeRunnerCheckpoint").mockImplementation(async (_db, binding, directory) => {
+          const checkpoint = saved.get(binding.runId);
+          if (!checkpoint) throw new Error("missing test checkpoint");
+          checkpoint.restoreSessionDirectory(directory);
+          return checkpoint;
+        }) : undefined;
+    if (restore) {
+      const completeIdentity = { ...identity, turnId: "turn-first", itemId: "item-first" };
+      // The authority must begin from a complete real state envelope.
+      await rm(join(root, "control-plane"), { recursive: true, force: true });
+      persistedController = new DurablePrpControlPlane({ stateDirectory: join(root, "control-plane"), identity: completeIdentity,
+        expectedRunnerVersion: "warm-test", expectedRunnerDigest: `sha256:${"a".repeat(64)}` });
+    }
     const order: string[] = [];
     let failRetirement = scenario === "retirement-failed";
     const firstSession = { close: vi.fn(async () => {
@@ -11277,7 +11379,7 @@ describe("runnerd provider runtime wiring", () => {
         criteria: [], remainingWork: [] }, evidence: [], verification: [], artifacts: [], attentionRequests: [] });
     if (!validated.ok) throw new Error("Invalid completion fixture");
     const result = { result: validated.result, terminal: { runTerminalState: "succeeded" },
-      turnId: "turn", normalizedSessionId: first.session.normalizedSessionId,
+      turnId: "turn", normalizedSessionId: first.session.normalizedSessionId!,
       providerSessionId: "provider", driverKind: "test", driverVersion: "1",
       nativeEventCount: 1, highestContiguousSourceSeq: 1, usage: null };
     const feedback = vi.spyOn(completionFeedback, "nativeCompletionFeedback").mockResolvedValue("accepted");
@@ -11286,9 +11388,21 @@ describe("runnerd provider runtime wiring", () => {
     state.execute.mockReset().mockImplementationOnce(async options => {
       retainedFeedback = state.createBackend.mock.calls.at(-1)![1].completionFeedback;
       await options.onSession?.(firstSession);
+      if (persistedController) {
+        persistedController.setCheckpointPersistence(async checkpoint => { saved.set(checkpoint.runId, checkpoint); });
+        await persistedController.flushCheckpoint();
+        expect(JSON.parse(await readFile(join(root, "control-plane/control-plane-state.json"), "utf8")).schema)
+          .toBe("paperclip.runner.external-control-plane-state.v1");
+      }
       return result;
     }).mockImplementationOnce(async options => {
       order.push("execute-next");
+      if (persistedController) {
+        persistedController.rotateRunIdentity({ ...identity, runId: next.binding.runId, turnId: "turn-next", itemId: "item-next" });
+        await persistedController.flushCheckpoint();
+        await writeFile(join(root, "runner/runner-state.json"), JSON.stringify(durableRunnerState(
+          { ...identity, runId: next.binding.runId }, "ready")));
+      }
       if (reusable) {
         await expect(retainedFeedback!(result.result)).resolves.toBe("accepted");
         expect(feedback).toHaveBeenLastCalledWith(firstDb, next.binding.runId, result.result);
@@ -11343,8 +11457,8 @@ describe("runnerd provider runtime wiring", () => {
         }
         if (scenario === "plugin-managed-github") expect(rebind).toHaveBeenCalledOnce();
         expect(order).toEqual(reusable ? ["execute-next"] : ["retire", "execute-next"]);
-        if (scenario === "unchanged") {
-          await writeRunnerAuthority(root, { ...identity, runId: next.binding.runId }, "ready");
+        if (["unchanged", "external-checkpoint"].includes(scenario)) {
+          if (!persistedController) await writeRunnerAuthority(root, { ...identity, runId: next.binding.runId }, "ready");
           const third = { ...next, binding: { ...next.binding, runId: `${next.binding.runId}-third` } };
           const thirdDb = leaseDb(third);
           const thirdSelect = thirdDb.select.bind(thirdDb);
@@ -11362,9 +11476,15 @@ describe("runnerd provider runtime wiring", () => {
             runnerExecutionTarget: pluginTarget as never,...pluginOptions })).resolves.toMatchObject({ exitCode: 0 });
           expect(firstSession.close).not.toHaveBeenCalled();
           expect(state.execute).toHaveBeenCalledTimes(3);
+          if (restore) {
+            expect(restore.mock.calls.map(([, binding]) => binding.runId)).toEqual([first.binding.runId, next.binding.runId]);
+            expect(existsSync(root)).toBe(true);
+          }
         }
       }
     } finally {
+      restore?.mockRestore();
+      await persistedController?.stop();
       failRetirement = false;
       resume?.mockRestore();owners?.mockRestore();feedback.mockRestore();
       await closeWarmNativeSessionsForEnvironment({ environmentId, reason: "test cleanup" });
@@ -11985,6 +12105,49 @@ describe("runnerd provider runtime wiring", () => {
     }
   });
 
+  it.each(["codex", "opencode", "claude", "pi"] as const)("adopts remote %s with no old controller launch assets", async provider => {
+    const base = scopedNativeExecution(`missing-launch-assets-${provider}`);
+    const current = { ...base,
+      provider: provider === "codex" ? base.provider : provider === "opencode"
+        ? { kind: "opencode", model: null }
+        : { kind: "acpx", agent: provider, model: null },
+      session: { ...base.session, driverKind: provider === "codex" ? "codex_app_server"
+        : provider === "opencode" ? "opencode_server" : "acpx_runtime" },
+    } as NativeExecutionInputV1;
+    const identity = { runId: current.binding.runId, normalizedSessionId: current.session.normalizedSessionId!,
+      runnerInstanceId: `admitted-${provider}`, environmentLeaseId: "admitted-lease",
+      turnId: `turn-${current.binding.runId}`, itemId: `item-${current.binding.runId}` };
+    const root = await initializeScopedRunnerRoot(current, identity.runnerInstanceId);
+    await mkdir(join(root, "control-plane"), { recursive: true });
+    await writeFile(join(root, "control-plane", "control-plane-state.json"), JSON.stringify(durableControlPlaneState(identity)));
+    const missingBinary = join(root, "absent-old-controller", "runnerd");
+    const missingPack = join(root, "absent-old-controller", "provider-pack");
+    expect(existsSync(missingBinary)).toBe(false); expect(existsSync(missingPack)).toBe(false);
+    expect(existsSync(join(root, "provider-pack-authority.json"))).toBe(false);
+    const execute = vi.fn(async (request: { args: string[] }) => ({ exitCode: 0, timedOut: false, stderr: "",
+      stdout: request.args[1]?.includes("base64") ? Buffer.from(JSON.stringify({ ...identity, lifecycle: "running" })).toString("base64")
+        : request.args[1]?.includes("cat --") ? `nonce\n123\n2026-09-19T10:00:00.000Z\n${identity.runnerInstanceId}\nlinux:abcd-1234:100\n` : "" }));
+    state.createBackend.mockClear(); state.createTransport.mockClear(); state.resolveRemoteRunnerBinary.mockClear();
+    const artifact = { version: "admitted-version", digest: `sha256:${"a".repeat(64)}` };
+    await createRunnerdBackend({ db: leaseDb(current), execution: current, runnerInstanceId: identity.runnerInstanceId,
+      runnerRemoteBinaryPath: missingBinary, runnerRemoteProviderPackPath: missingPack,
+      runnerIngressAuthorized: true,
+      controllerPersistence: { bootId: "successor", runnerArtifact: artifact },
+      runnerExecutionTarget: { kind: "remote", transport: "sandbox", remoteCwd: "/workspace",
+        environmentId: "environment", leaseId: identity.environmentLeaseId, providerKey: "daytona", runner: { execute },
+        effectiveCapabilities: { runnerWebSocketIngress: true },
+        sandboxLeaseAcquisition: { outcome: "resumed", providerLeaseId: "sandbox" } } as never,
+      restartRecovery: { kind: "reattach_remote_runner", runId: current.binding.runId, leaseOwner: "controller",
+        controllerGeneration: 2, providerAttempt: 1, restartKind: "hard", recoveryRequestId: null,
+        remote: { providerLeaseId: "sandbox", remoteCwd: "/workspace" } },
+    });
+    state.createBackend.mock.calls.at(-1)![1].codexTransportFactory!();
+    expect(state.createTransport.mock.calls.at(-1)![0]).toMatchObject({ prpIdentity: identity,
+      runnerArtifact: artifact, adoptExistingRunner: { pid: 123, startedAt: "2026-09-19T10:00:00.000Z" } });
+    expect(state.resolveRemoteRunnerBinary).not.toHaveBeenCalled();
+    expect(execute.mock.calls.some(([request]) => request.args.includes("paperclip-runner-launch") || request.args.includes("--build-metadata"))).toBe(false);
+  });
+
   it.each(["claude", "codex"].flatMap(provider => (["per_turn", "warm"] as const).flatMap(policy => ["canonical", "local-receipt", "local-receipt-missing-terminal", "local-receipt-pending-command", "quarantined", "empty-runner", "empty-runner-quarantined", "nonempty-runner", "wrong-identity", "active", "foreign-scope", "no-stop-proof", "not-suspended", "pending-outbox", "pending-command", "missing-terminal", "ambiguous", "contradictory"].map(mode => [provider, policy, mode] as const))))(
     "recovers a blocked %s %s plugin checkpoint on a replacement lease: %s", async (provider, policy, mode) => {
       const { permissionPolicy: _permissionPolicy, ...claudeProfile } = QUALIFIED_ACPX_PROFILES.claude;
@@ -12122,6 +12285,43 @@ describe("runnerd provider runtime wiring", () => {
     } finally { resume.mockRestore(); }
   });
 
+  it.each(["verified", "wrong-state", "wrong-lease"] as const)("reattaches a surviving plugin task through its owner: %s", async mode => {
+    const current = scopedNativeExecution(`plugin-live-${mode}`);
+    const identity = { runId: current.binding.runId, normalizedSessionId: current.session.normalizedSessionId!,
+      runnerInstanceId: "plugin-runner", environmentLeaseId: "same-task-lease", turnId: "admitted-turn", itemId: "admitted-item" };
+    const root = await initializeScopedRunnerRoot(current, identity.runnerInstanceId);
+    await mkdir(join(root, "control-plane"), { recursive: true });
+    await writeFile(join(root, "control-plane/control-plane-state.json"), JSON.stringify(durableControlPlaneState(identity)));
+    const task = vi.fn(async (operation: any) => {
+      environmentTaskOperationSchema.parse(operation);
+      if (operation.kind === "status") return { kind: "status", taskId: "same-task", phase: "running", executionStopped: false };
+      if (operation.kind === "read_state") return { kind: "state", taskId: "same-task", state: {
+        ...durableRunnerState(identity, "ready"), ...(mode === "wrong-state" ? { itemId: "other" } : {}),
+      } };
+      return { kind: "description", launchPreparationVersion: 1, projectAuthCheckVersion: 1, taskId: "same-task",
+        runner: { version: "new-artifact", digest: `sha256:${"b".repeat(64)}` } };
+    });
+    state.createBackend.mockClear(); state.createTransport.mockClear();
+    const backend = createRunnerdBackend({ db: leaseDb(current), execution: current, runnerInstanceId: identity.runnerInstanceId,
+      runnerIngressAuthorized: true, runnerExecutionTarget: { kind: "remote", transport: "sandbox", remoteCwd: ".",
+        leaseId: mode === "wrong-lease" ? "replacement-task" : identity.environmentLeaseId, runnerTask: task },
+      controllerPersistence: { bootId: "successor", runnerArtifact: { version: "admitted-artifact", digest: `sha256:${"a".repeat(64)}` } },
+      restartRecovery: { kind: "reattach_remote_runner", runId: current.binding.runId, leaseOwner: "controller", controllerGeneration: 2,
+        providerAttempt: 1, restartKind: "hard", recoveryRequestId: null, remote: { kind: "plugin_task", leaseId: identity.environmentLeaseId } } });
+    if (mode !== "verified") {
+      await expect(backend).rejects.toThrow(mode === "wrong-state" ? "adoption identity mismatch" : "recovery_lease_mismatch");
+      expect(state.createBackend).not.toHaveBeenCalled();
+      return;
+    }
+    await backend;
+    state.createBackend.mock.calls.at(-1)![1].codexTransportFactory!();
+    const options = state.createTransport.mock.calls.at(-1)![0];
+    expect(options).toMatchObject({ prpIdentity: identity, adoptExistingRunner: { pid: null, startedAt: null },
+      runnerArtifact: { version: "admitted-artifact" } });
+    expect(options).toMatchObject({ runnerReconnectGraceMs: 420_000 });
+    expect(task.mock.calls.some(([op]) => ["prepare", "submit", "stop"].includes(op.kind))).toBe(false);
+  });
+
   it.each(["codex", "opencode", "acpx"] as const)("uses a task-only %s driver without exposing Core paths or transferring the agent home", async (kind) => {
     const taskExecution = { ...execution, provider: kind === "codex" ? execution.provider
       : kind === "opencode" ? { kind, model: "openai/test", permissionMode: "deny" }

server/src/services/native-runtime/native-session-executor.ts

@@ -1,6 +1,8 @@
 import { providerPackWorkspaceId, recordAdmittedRemoteProviderPack, resolveRemoteProviderPackAuthority } from "./remote-provider-pack-authority.js";
 import { type ProviderPackInstallation, verifyRemoteProviderPack as verifyRemoteProviderPackIdentity } from "./remote-provider-pack-verification.js";
 import { writeEnvironmentCleanup } from "../environment-cleanup-notifications.js";
+import { NATIVE_SESSION_EXECUTION_LEASE_TTL_MS, startNativeSessionExecutionLeaseRenewal, type NativeExecutionLeaseRenewal } from "./native-execution-lease.js";
+export { renewNativeSessionExecutionLease } from "./native-execution-lease.js";
 import { DELIVERY_QUEUES } from "../delivery-work-notifications.js";
 import { writeExecutionControl } from "../execution-control-notifications.js";
 import { notifyExecutionRunWork } from "../execution-control-notifications.js";
@@ -9,7 +11,7 @@ import { registerPluginTaskCredentialHandoff, type PluginTaskCredentialWriteback
 import { resolvePluginTaskResume } from "../environment-task-runtime.js";
 import { environmentTaskResultSchema } from "@paperclipai/plugin-sdk";
 import { preparePluginRunnerCredentials } from "./plugin-runner-credentials.js";
-import { createPluginRunnerTaskOwner } from "./plugin-runner-task.js";
+import { createPluginRunnerTaskOwner, verifyPluginRunnerTaskReattachment } from "./plugin-runner-task.js";
 import { applyObservabilityEnvironment, type ProviderObservabilityLaunch } from "@paperclipai/adapter-utils/agent-observability";
 import { museRunnerBroker } from "../muse-runner-broker.js";
 import { nativeCaseHandoffResult } from "./native-case-handoff.js";
@@ -86,6 +88,8 @@ import {
 } from "node:fs";
 import { tmpdir, userInfo } from "node:os";
 import { DatabaseSync } from "node:sqlite";
+import { nativeRunnerCheckpointPersistence, nativeRunnerCheckpointRegistration, restoreNativeRunnerCheckpoint } from "./native-runner-checkpoint.js";
+import { REMOTE_CONTROLLER_LEASE_MS, REMOTE_RUNNER_RECONNECT_GRACE_MS } from "./native-runner-recovery-contract.js";
 import { basename, dirname, join, posix, resolve } from "node:path";
 import type {
   AdapterExecutionResult,
@@ -259,9 +263,7 @@ export class NativeControllerDetachedForRestartError extends Error {
 
 const activeNativeSessions = new Map<string, ActiveNativeSession>();
 // Shutdown can race provider startup before onSession publishes its handle.
-// Retain the request for the remainder of this controller's lifetime so that
-// the late publication detaches before it can dispatch another turn.
-const nativeRunsDetachingForRestart = new Set<string>();
+const nativeRunsDetachingForRestart = new Map<string, symbol>();
 
 // A restart must not cut authority rotation between archiving the previous
 // runner state and authenticating its replacement. Keep the startup owner
@@ -273,6 +275,9 @@ type NativeSessionStartup = {
   restartAdmissionPending?: boolean;
   cancellationSettled?: Promise<void>;
   governedWait?: boolean;
+  priorRestartDetach?: symbol;
+  leaseOwner?: string;
+  executionLeaseRenewal?: NativeExecutionLeaseRenewal;
   settled: Promise<void>;
   settle: () => void;
 };
@@ -305,6 +310,7 @@ async function waitForNativeSessionStartup(
 
 export async function detachNativeSessionsForRestart(
   runIds: readonly string[],
+  leaseOwner?: string,
 ): Promise<{
   detachedRunIds: string[];
   inactiveRunIds: string[];
@@ -330,7 +336,11 @@ export async function detachNativeSessionsForRestart(
         })]);
       } finally { clearTimeout(timer); }
     }
-    nativeRunsDetachingForRestart.add(runId);
+    if (leaseOwner !== undefined && nativeSessionStartups.get(runId)?.leaseOwner !== leaseOwner) {
+      inactiveRunIds.push(runId);
+      continue;
+    }
+    nativeRunsDetachingForRestart.set(runId, Symbol("native controller detach"));
     let active = activeNativeSessions.get(runId);
     if (!active) {
       // Only a startup registered before this fence may complete read-only
@@ -447,8 +457,6 @@ const TERMINAL_HEARTBEAT_RUN_STATUSES = new Set([
   "cancelled",
   "timed_out",
 ]);
-const NATIVE_SESSION_EXECUTION_LEASE_TTL_MS = 20 * 60_000;
-const NATIVE_SESSION_EXECUTION_LEASE_RENEW_INTERVAL_MS = 5 * 60_000;
 const NATIVE_SESSION_CANCELLATION_CLEANUP_GRACE_MS = 2_000;
 const NATIVE_RUNTIME_REQUEST_RESOLUTION_CACHE_MAX = 256;
 type NativeRuntimeRequestResolution = {
@@ -535,6 +543,7 @@ type WarmNativeSession = {
   pluginTaskId?: string;
   observabilityCompatibilityKey?: string;
   runId: string;
+  issueId: string;
   instructionWorkingCopy?: NativeInstructionWorkingCopy;
   // Keep the admitted physical root independent of the per-run collection
   // capability, which instruction preparation adopts before final admission.
@@ -1743,6 +1752,13 @@ class SessionToolAuthorityEpoch {
     return this.#authority.definitions();
   }
 
+  async recover(call: Parameters<PaperclipRunnerToolAuthority["recover"]>[0]) {
+    this.#assertCurrent();
+    const result = await this.#authority.recover(call);
+    this.#assertCurrent();
+    return result;
+  }
+
   async execute(call: Parameters<PaperclipRunnerToolAuthority["execute"]>[0]) {
     this.#assertCurrent();
     return this.toolTrace
@@ -4979,6 +4995,7 @@ async function migrateRunnerdStateRootForExecution(input: {
   runnerExecutionTarget?: AdapterExecutionTarget | null;
 }): Promise<VerifiedWarmTransitionBinding | undefined> {
   const scoped = scopedRunnerdStateRoot(input.execution);
+  await restoreWarmRunnerdController(input);
   // A crash can leave a source archive intent before/after its rename. Until
   // that exact maintenance is settled, absence of a canonical root is not
   // permission to bootstrap a replacement provider session.
@@ -5049,6 +5066,16 @@ async function migrateRunnerdStateRootForExecution(input: {
       throw new Error("runner_state_identity_mismatch: bootstrap_not_proven_incomplete");
     }
     if (input.restartRecovery?.kind === "reattach_remote_runner") {
+      if (input.restartRecovery.remote.kind === "plugin_task") {
+        const target = input.runnerExecutionTarget;
+        if (target?.kind !== "remote" || target.transport !== "sandbox" || !target.runnerTask
+          || target.leaseId !== input.restartRecovery.remote.leaseId || identity.environmentLeaseId !== target.leaseId
+          || !durableIdentityMatchesExecution(identity, input.execution)) {
+          throw new Error("native_plugin_recovery_lease_mismatch");
+        }
+        await verifyPluginRunnerTaskReattachment(target.runnerTask, identity);
+        return;
+      }
       await verifyRemoteRunnerReattachment({
         claim: input.restartRecovery, target: input.runnerExecutionTarget, identity,
         runId: input.execution.binding.runId, normalizedSessionId: nativeSessionKey(input.execution),
@@ -5496,6 +5523,7 @@ type RunnerdDurableIdentity = Record<string, unknown> & {
 
 function readRunnerdDurableIdentity(
   root: string,
+  allowExternal = false,
 ): Record<string, unknown> | null {
   if (!isSafeNativeStateDirectory(root)) return null;
   const controlPlaneRoot = resolve(root, "control-plane");
@@ -5512,7 +5540,8 @@ function readRunnerdDurableIdentity(
         ).toString("utf8"),
       ),
     );
-    if (state.schema !== RUNNERD_CONTROL_PLANE_STATE_SCHEMA) return null;
+    if (state.schema !== RUNNERD_CONTROL_PLANE_STATE_SCHEMA &&
+        !(allowExternal && state.schema === "paperclip.runner.external-control-plane-state.v1")) return null;
     return record(state.identity);
   } catch {
     return null;
@@ -5661,6 +5690,45 @@ function idleWarmNativeSessionOwnerRunId(input: {
     : undefined;
 }
 
+/** Materialize opaque controller state before ordinary warm admission reads it.
+ * A retained owner or server checkpoint identifies the last completed run;
+ * the original external marker can precede several in-process warm turns. */
+async function restoreWarmRunnerdController(input: {
+  db: Db; execution: NativeExecutionInput;
+  runnerExecutionTarget?: AdapterExecutionTarget | null;
+  restartRecovery?: NativeRestartRecoveryClaim;
+}): Promise<void> {
+  if (input.restartRecovery || input.runnerExecutionTarget?.kind !== "remote" ||
+      input.execution.session.lifecyclePolicy.mode !== "warm") return;
+  const root = scopedRunnerdStateRoot(input.execution);
+  const identity = readRunnerdDurableIdentity(root);
+  const reference = identity ?? readRunnerdDurableIdentity(root, true);
+  if (!durableIdentityMatchesSession(reference, input.execution)) return;
+  const retainedRunId = idleWarmNativeSessionOwnerRunId(input);
+  let runId = retainedRunId ?? reference.runId;
+  let issueId = retainedRunId ? warmNativeSessions.get(nativeSessionScopeKey(input.execution))!.issueId
+    : input.execution.binding.issueId;
+  if (!retainedRunId) {
+    const saved = nativeSessionCheckpointPath(input.execution);
+    if (existsSync(saved)) {
+      const envelope = record(JSON.parse(readBoundedNativeFile(saved,
+        NATIVE_WARM_CHECKPOINT_MAX_BYTES, "native_session_supervisor_checkpoint_too_large").toString("utf8")));
+      const prior = record(record(envelope.snapshot).identity);
+      if (envelope.schema !== "paperclip.native-session-supervisor.v1" ||
+          prior.companyId !== input.execution.binding.companyId || prior.agentId !== input.execution.binding.agentId ||
+          prior.sessionId !== nativeSessionKey(input.execution) || typeof prior.runId !== "string" || !prior.runId ||
+          typeof prior.issueId !== "string" || !prior.issueId) throw new Error("native_session_supervisor_checkpoint_mismatch");
+      runId = prior.runId;
+      issueId = prior.issueId;
+    }
+  }
+  if (identity?.runId === runId) return;
+  const restored = await restoreNativeRunnerCheckpoint(input.db,
+    { companyId: input.execution.binding.companyId, issueId, runId }, root, nativeSessionKey(input.execution));
+  // A legacy file-backed root still follows its existing prior-owner checks.
+  if (!restored && !identity) throw new Error("native_runner_checkpoint_missing");
+}
+
 function nativeHarnessEnvironmentFingerprint(
   execution: NativeExecutionInput,
 ): string {
@@ -7651,93 +7719,6 @@ export function resolveNativeCancellationStatus(input: {
   };
 }
 
-export async function renewNativeSessionExecutionLease(input: {
-  db: Db;
-  runId: string;
-  companyId: string;
-  issueId: string;
-  leaseOwner: string;
-  attempt: number;
-  controller?: NativeControllerIdentity;
-  leaseTtlMs?: number;
-}): Promise<void> {
-  const controller =
-    input.controller ?? (await currentNativeControllerIdentity());
-  const leaseTtlMs = input.leaseTtlMs ?? NATIVE_SESSION_EXECUTION_LEASE_TTL_MS;
-  if (
-    !Number.isInteger(leaseTtlMs) ||
-    leaseTtlMs < 1_000 ||
-    leaseTtlMs > NATIVE_SESSION_EXECUTION_LEASE_TTL_MS
-  ) {
-    throw new Error("native_session_lease_ttl_invalid");
-  }
-  const [updated] = await input.db
-    .update(nativeRunFinalizations)
-    .set({
-      leaseExpiresAt: sql`now() + (${leaseTtlMs} * interval '1 millisecond')`,
-      updatedAt: sql`now()`,
-    })
-    .where(
-      and(
-        eq(nativeRunFinalizations.runId, input.runId),
-        eq(nativeRunFinalizations.companyId, input.companyId),
-        eq(nativeRunFinalizations.issueId, input.issueId),
-        eq(nativeRunFinalizations.leaseOwner, input.leaseOwner),
-        eq(nativeRunFinalizations.attempt, input.attempt),
-        eq(nativeRunFinalizations.controllerBootId, controller.bootId),
-        eq(nativeRunFinalizations.controllerPid, controller.pid),
-        eq(
-          nativeRunFinalizations.controllerProcessStartedAt,
-          controller.processStartedAt,
-        ),
-        gt(nativeRunFinalizations.leaseExpiresAt, sql`now()`),
-      ),
-    )
-    .returning({ runId: nativeRunFinalizations.runId });
-  if (!updated) throw new Error("native_session_lease_lost");
-}
-
-function startNativeSessionExecutionLeaseRenewal(input: {
-  db: Db;
-  runId: string;
-  companyId: string;
-  issueId: string;
-  leaseOwner: string;
-  attempt: number;
-  controller: NativeControllerIdentity;
-}): { stop: () => Promise<void> } {
-  let leaseLost: Error | null = null;
-  let renewal = Promise.resolve();
-  const renew = () => {
-    if (leaseLost) return;
-    renewal = renewal
-      .then(() => renewNativeSessionExecutionLease(input))
-      .then(() => undefined)
-      .catch(async (error: unknown) => {
-        leaseLost =
-          error instanceof Error
-            ? error
-            : new Error("native_session_lease_lost");
-        await cancelNativeSession(
-          input.runId,
-          "native session execution lease lost",
-        ).catch(() => undefined);
-      });
-  };
-  const timer = setInterval(
-    renew,
-    NATIVE_SESSION_EXECUTION_LEASE_RENEW_INTERVAL_MS,
-  );
-  timer.unref?.();
-  return {
-    stop: async () => {
-      clearInterval(timer);
-      await renewal;
-      if (leaseLost) throw leaseLost;
-    },
-  };
-}
-
 export async function executePaperclipNativeSession(input: {
   casesEnabled?: boolean;
   getFreshSessionHandoff?: () => Promise<string | null>;
@@ -7834,7 +7815,7 @@ export async function executePaperclipNativeSession(input: {
     }
   }
   const runId = input.execution.binding.runId;
-  if (nativeRunsDetachingForRestart.has(runId)) {
+  if (nativeRunsDetachingForRestart.has(runId) && input.restartRecovery?.runId !== runId) {
     throw new NativeControllerDetachedForRestartError();
   }
   if (nativeSessionStartups.has(runId)) {
@@ -7845,6 +7826,7 @@ export async function executePaperclipNativeSession(input: {
   let resolveStartup!: (session: ActiveNativeSession | null) => void;
   let settle!: () => void;
   const startup: NativeSessionStartup = {
+    priorRestartDetach: nativeRunsDetachingForRestart.get(runId),
     settled: new Promise<void>(resolve => { settle = resolve; }),
     settle: () => settle(),
     promise: new Promise<ActiveNativeSession | null>(resolve => { resolveStartup = resolve; }),
@@ -7976,6 +7958,10 @@ export async function executePaperclipNativeSession(input: {
     ) {
       executingRunnerdSessionScopes.delete(sessionScopeId);
     }
+    // Preparation can fail before the execution loop owns its normal cleanup.
+    // Remove the settled startup before yielding, so detachment cannot reacquire
+    // its old resolved session while renewal cleanup drains.
+    await startup.executionLeaseRenewal?.stop().catch(() => undefined);
     try {
       await cleanupStagedAttachments();
     } catch (cleanupError) {
@@ -8002,6 +7988,10 @@ export async function executePaperclipNativeSession(input: {
 async function executePaperclipNativeSessionWithinScope(
   input: Parameters<typeof executePaperclipNativeSession>[0],
 ): Promise<AdapterExecutionResult> {
+  const restoredController = input.restartRecovery?.kind === "reattach_remote_runner"
+    ? await restoreNativeRunnerCheckpoint(input.db, input.execution.binding, scopedRunnerdStateRoot(input.execution))
+    : null;
+  if (input.useRunnerd) await restoreWarmRunnerdController(input);
   const identityRedactor = createAgentIdentityRedactor(input.runnerEnvironment?.PAPERCLIP_AGENT_PRIVATE_KEY);
   const redactIdentityText = (text: string) => redactSensitiveText(
     identityRedactor.chunk("diagnostic", text) + identityRedactor.finish("diagnostic"),
@@ -8152,8 +8142,8 @@ async function executePaperclipNativeSessionWithinScope(
   const leaseOwner =
     input.leaseOwner ?? `${effectiveRunnerInstanceId}:${randomUUID()}`;
   const controller = await currentNativeControllerIdentity();
-  const leaseNow = new Date();
-  const leaseExpiresAt = new Date(leaseNow.getTime() + 20 * 60_000);
+  const executionLeaseTtlMs = input.runnerExecutionTarget?.kind === "remote"
+    ? REMOTE_CONTROLLER_LEASE_MS : NATIVE_SESSION_EXECUTION_LEASE_TTL_MS;
   let attempt: number;
   try {
     attempt = await trace.measure(
@@ -8181,10 +8171,6 @@ async function executePaperclipNativeSessionWithinScope(
             .then((rows) => rows[0] ?? null);
           if (!coordinator)
             throw new Error("native_finalization_coordinator_missing");
-          const leaseNow = new Date();
-          const leaseExpiresAt = new Date(
-            leaseNow.getTime() + NATIVE_SESSION_EXECUTION_LEASE_TTL_MS,
-          );
           // A durable result means provider execution already completed. The
           // recovery/finalization path must reconcile it; never reacquire a
           // provider session and execute the turn a second time.
@@ -8303,13 +8289,15 @@ async function executePaperclipNativeSessionWithinScope(
             : coordinator.controllerBootId === controller.bootId
               ? Math.max(1, coordinator.controllerGeneration)
               : coordinator.controllerGeneration + 1;
+          const leaseNow = new Date();
           const claimed = await tx
             .update(nativeRunFinalizations)
             .set({
               phase: "observed",
               attempt: nextAttempt,
               leaseOwner,
-              leaseExpiresAt,
+              // Start the full lease after claim locks and lineage reads finish.
+              leaseExpiresAt: sql`clock_timestamp() + ${executionLeaseTtlMs} * interval '1 millisecond'`,
               controllerBootId: controller.bootId,
               controllerPid: controller.pid,
               controllerProcessStartedAt: controller.processStartedAt,
@@ -8346,9 +8334,30 @@ async function executePaperclipNativeSessionWithinScope(
             })
             .where(eq(heartbeatRuns.id, coordinator.runId));
           return nextAttempt;
+        }).then(claimedAttempt => {
+          const startup = nativeSessionStartups.get(input.execution.binding.runId);
+          if (!startup) throw new Error("native_session_startup_owner_missing");
+          startup.leaseOwner = leaseOwner;
+          // Own renewal as soon as the claim commits, including slow tracing,
+          // accounting history and retained-session cleanup before dispatch.
+          startup.executionLeaseRenewal = startNativeSessionExecutionLeaseRenewal({
+            db: input.db, runId: input.execution.binding.runId,
+            companyId: input.execution.binding.companyId, issueId: input.execution.binding.issueId,
+            leaseOwner, attempt: claimedAttempt, controller,
+            remote: input.runnerExecutionTarget?.kind === "remote",
+            onLost: () => input.runnerExecutionTarget?.kind === "remote"
+              ? detachNativeSessionsForRestart([input.execution.binding.runId], leaseOwner)
+              : cancelNativeSession(input.execution.binding.runId, "native session execution lease lost"),
+          });
+          return claimedAttempt;
         }),
       { parentName: "task.prepare" },
     );
+    const startup = nativeSessionStartups.get(input.execution.binding.runId);
+    if (input.restartRecovery && startup?.priorRestartDetach !== undefined
+      && nativeRunsDetachingForRestart.get(input.execution.binding.runId) === startup.priorRestartDetach) {
+      nativeRunsDetachingForRestart.delete(input.execution.binding.runId);
+    }
     await trace.end(taskPrepareScope);
   } catch (error) {
     await trace.end(taskPrepareScope, { outcome: "failed" });
@@ -8843,7 +8852,7 @@ async function executePaperclipNativeSessionWithinScope(
   let githubAccess: NativeGitHubAccess | undefined;
   let releaseGitHubRun: (() => void) | undefined;
   let persistedWarmSession: PersistedNativeSession | null | undefined;
-  if (warmSessionId !== null && warmConfigDigest !== null) {
+  if (warmSessionId !== null && warmConfigDigest !== null && !input.restartRecovery) {
     const entry = warmNativeSessions.get(warmSessionId);
     if (entry) {
       if (entry.preparingRunId && entry.preparingRunId !== input.execution.binding.runId) throw new Error("native_session_supervisor_busy");
@@ -9030,15 +9039,8 @@ async function executePaperclipNativeSessionWithinScope(
           },
         }
       : input.execution;
-  const leaseRenewal = startNativeSessionExecutionLeaseRenewal({
-    db: input.db,
-    runId: input.execution.binding.runId,
-    companyId: input.execution.binding.companyId,
-    issueId: input.execution.binding.issueId,
-    leaseOwner,
-    attempt,
-    controller,
-  });
+  const leaseRenewal = nativeSessionStartups.get(input.execution.binding.runId)?.executionLeaseRenewal;
+  if (!leaseRenewal) throw new Error("native_session_renewal_owner_missing");
   try {
     const usesPluginTask = input.runnerExecutionTarget?.kind === "remote" && input.runnerExecutionTarget.transport === "sandbox" && !!input.runnerExecutionTarget.runnerTask;
     const prepareRunnerEnvironment = async () => {
@@ -9071,6 +9073,10 @@ async function executePaperclipNativeSessionWithinScope(
       input.useRunnerd && input.backend === undefined
         ? await timeController("backend_setup", () => createRunnerdBackend({
             ...input,
+            controllerPersistence: input.runnerExecutionTarget?.kind === "remote" ? {
+              bootId: controller.bootId,
+              runnerArtifact: restoredController ? { version: restoredController.runnerVersion, digest: restoredController.runnerDigest } : undefined,
+            } : undefined,
             ...(overlapGitHubPreparation ? { prepareRunnerEnvironment } : {}),
             // Durable scope and prior-run verification use the controller's
             // canonical workspace identity. createRunnerdBackend separately
@@ -9107,6 +9113,7 @@ async function executePaperclipNativeSessionWithinScope(
           nativeSessionConfigDigest(remoteWarmRecovery.previousExecution, "remote"), await recovered.session.snapshot());
         persistedWarmSession = loadWarmNativeCheckpoint(input.execution, warmConfigDigest!, "remote");
         runnerdBackend = await createRunnerdBackend({ ...input, runnerInstanceId: effectiveRunnerInstanceId,
+          controllerPersistence: input.runnerExecutionTarget?.kind === "remote" ? { bootId: controller.bootId } : undefined,
           durableEnvironmentLeaseId: durableRunnerBinding?.environmentLeaseId, trace, toolTrace });
       }
     }
@@ -9311,11 +9318,13 @@ async function executePaperclipNativeSessionWithinScope(
                   }
                   existing.session = session;
                   existing.runId = input.execution.binding.runId;
+                  existing.issueId = input.execution.binding.issueId;
                 } else
                   warmNativeSessions.set(warmSessionId, {
                     pluginTaskId: usesPluginTask && input.runnerExecutionTarget?.kind === "remote" && input.runnerExecutionTarget.transport === "sandbox"
                       ? input.runnerExecutionTarget.sandboxLeaseAcquisition?.providerLeaseId : undefined,
                     runId: input.execution.binding.runId,
+                    issueId: input.execution.binding.issueId,
                     instructionWorkingCopy: input.instructionWorkingCopy?.checkpointWarm ? undefined : input.instructionWorkingCopy,
                     instructionRoot: input.instructionWorkingCopy?.root,
                     agentId: input.execution.binding.agentId,
@@ -11231,6 +11240,7 @@ export async function verifyRemoteRunnerReattachment(input: {
 }) {
   const { claim, target, identity } = input;
   if (
+    claim.remote.kind === "plugin_task" ||
     target?.kind !== "remote" ||
     target.transport !== "sandbox" ||
     !target.runner ||
@@ -11690,7 +11700,12 @@ export function createRemoteRunnerProcessLauncher(input: {
         };
       }
     })();
-    return { child, completion, isExitConfirmed: () => exitConfirmed };
+    return {
+      child,
+      completion,
+      get startedAt() { return launchedIdentity?.startedAt; },
+      isExitConfirmed: () => exitConfirmed,
+    };
   };
 }
 
@@ -11715,6 +11730,7 @@ export function remoteCheckpointIncompleteFailure(
 export async function createRunnerdBackend(input: {
   casesEnabled?: boolean;
   db: Db;
+  controllerPersistence?: { bootId: string; runnerArtifact?: { version: string; digest: string } };
   execution: NativeExecutionInput;
   runnerInstanceId: string;
   chatAttachmentReadScope?: NativeChatAttachmentReadScope;
@@ -11826,8 +11842,9 @@ async function prepareRemoteExternalRunner({ input, target, runner, root, identi
   const runnerStateDirectory = posix.join(remoteRoot, "sessions",
     createHash("sha256").update(identity.normalizedSessionId).digest("hex"), "runner");
   const requiredMode = resolveRemoteRunnerTransportMode({ target, runnerIngressAuthorized: input.runnerIngressAuthorized === true });
-  const artifact = readRunnerdArtifactBinding(runnerBinary);
+  const artifact = input.controllerPersistence?.runnerArtifact ?? readRunnerdArtifactBinding(runnerBinary);
   const prepare = async () => {
+    if (input.restartRecovery?.kind === "reattach_remote_runner" && input.controllerPersistence?.runnerArtifact) return;
     if (input.restartRecovery?.kind !== "reattach_remote_runner") {
       if (!input.runnerRemoteBinaryPath?.trim()) {
         const platform = await runner.execute({ command: "sh", args: ["-c", "uname -s; uname -m"], bypassSession: true, timeoutMs: 10000 });
@@ -11892,6 +11909,8 @@ async function prepareRemoteExternalRunner({ input, target, runner, root, identi
   };
   const executionDriver = createNativeRunnerExecutionDriver({
     target, binding: input.execution.binding,
+    controllerPersistence: input.controllerPersistence ? nativeRunnerCheckpointPersistence(
+      input.db, input.controllerPersistence.bootId, input.execution.binding.runId, detachNativeSessionsForRestart) : undefined,
     runnerIngressAuthorized: input.runnerIngressAuthorized,
     runnerPublicUrl: input.runnerPublicUrl,
     runnerCaBundlePath: input.runnerCaBundlePath,
@@ -12063,9 +12082,15 @@ async function createRunnerdBackendWithinSessionClaim(
     }
     const description = input.pluginTaskDescription ?? environmentTaskResultSchema.parse(await pluginTask({ kind: "describe" }));
     if (description.kind !== "description" || description.launchPreparationVersion !== 1 || description.projectAuthCheckVersion !== 1) throw new Error("Plugin Runner launch preparation v1 unavailable");
-    const prior = durableBinding ? durableIdentity as RunnerdDurableIdentity : undefined;
+    const saved = durableBinding ? durableIdentity as RunnerdDurableIdentity : undefined;
+    const prior = saved ? {
+      runnerInstanceId: saved.runnerInstanceId, environmentLeaseId: saved.environmentLeaseId,
+      runId: saved.runId, normalizedSessionId: saved.normalizedSessionId,
+      turnId: typeof saved.turnId === "string" ? saved.turnId : "",
+      itemId: typeof saved.itemId === "string" ? saved.itemId : "",
+    } : undefined;
     const retainedOwner = input.retainedPluginTaskOwner;
-    const adopting = input.remoteWarmRecovery !== undefined;
+    const adopting = input.remoteWarmRecovery !== undefined || input.restartRecovery?.kind === "reattach_remote_runner";
     const adoptionIdentity = input.remoteWarmRecovery ? {
       runnerInstanceId: input.remoteWarmRecovery.identity.runnerInstanceId,
       environmentLeaseId: input.remoteWarmRecovery.identity.environmentLeaseId,
@@ -12073,6 +12098,9 @@ async function createRunnerdBackendWithinSessionClaim(
       normalizedSessionId: input.remoteWarmRecovery.identity.normalizedSessionId,
       turnId: input.remoteWarmRecovery.identity.turnId, itemId: input.remoteWarmRecovery.identity.itemId,
     } : undefined;
+    if (input.restartRecovery?.kind === "reattach_remote_runner" && (!prior || !prior.turnId || !prior.itemId || prior.environmentLeaseId !== remoteTarget.leaseId)) {
+      throw new Error("native_plugin_recovery_lease_mismatch");
+    }
     const resumeFrom = !retainedOwner && !adopting && prior && prior.environmentLeaseId !== remoteTarget.leaseId
       ? await resolvePluginTaskResume(input.db, { companyId: input.execution.binding.companyId,
           leaseId: remoteTarget.leaseId!, priorIdentity: prior }) : undefined;
@@ -12091,7 +12119,11 @@ async function createRunnerdBackendWithinSessionClaim(
       // Warm retirement can outlive the run's temporary credential home; complete uses the durable handoff.
       onCredentials: input.execution.session.lifecyclePolicy.mode === "warm" ? undefined : credentials.onCredentials,
       warmTaskVersion: description.warmTaskVersion,
-      adoptIdentity: adoptionIdentity, projectAuthCheckVersion: description.projectAuthCheckVersion, task: pluginTask, projectIds: remoteTarget.runnerTaskProjectIds ?? [],
+      adoptIdentity: adoptionIdentity ?? (adopting && prior ? {
+        runnerInstanceId: prior.runnerInstanceId, environmentLeaseId: prior.environmentLeaseId,
+        runId: prior.runId, normalizedSessionId: prior.normalizedSessionId,
+        turnId: prior.turnId ?? "", itemId: prior.itemId ?? "",
+      } : undefined), projectAuthCheckVersion: description.projectAuthCheckVersion, task: pluginTask, projectIds: remoteTarget.runnerTaskProjectIds ?? [],
       harness: provider.kind === "acpx" ? provider.agent : provider.kind, resumeFrom, priorIdentity: prior ? {
         runnerInstanceId: prior.runnerInstanceId, environmentLeaseId: prior.environmentLeaseId,
         runId: prior.runId, normalizedSessionId: prior.normalizedSessionId,
@@ -12135,7 +12167,24 @@ async function createRunnerdBackendWithinSessionClaim(
         return nativeCompletionFeedback(input.db, current.runId, result);
       },
       codexTransportFactory: recovery => createRunnerdCodexTransport({ provider: provider.kind, stateDirectory: root,
-        ...owner.driver, runnerArtifact: description.runner,
+        ...owner.driver, runnerArtifact: input.controllerPersistence?.runnerArtifact ?? description.runner,
+        controlPlaneRegistration: input.controllerPersistence ? async (authority, identity, signal) => {
+          await nativeRunnerCheckpointRegistration({
+            ...nativeRunnerCheckpointPersistence(input.db, input.controllerPersistence!.bootId,
+              () => {
+                const current = sessionToolAuthorityEpochs.get(sessionScopeId);
+                if (!current) throw new Error("native_session_tool_authority_unavailable");
+                return current.runId;
+              }, detachNativeSessionsForRestart),
+            binding: input.execution.binding,
+          })(authority, identity);
+          return owner.driver.controlPlaneRegistration!(authority, identity, signal);
+        } : owner.driver.controlPlaneRegistration,
+        recoverSemanticToolInput: input.controllerPersistence ? call => {
+          const current = sessionToolAuthorityEpochs.get(sessionScopeId);
+          if (!current || current.runId !== call.correlation.runId) throw new Error("native_tool_authority_epoch_revoked");
+          return current.recover({ tool: call.operationId, callId: call.callId, arguments: call.input });
+        } : undefined,
         ...(provider.kind === "codex" ? { disableCodexWebSearch: "webSearch" in provider && provider.webSearch === "disabled" } : {}),
         ...(provider.kind === "acpx" ? { acpxAgent: provider.agent,
           acpxCandidateProfile: provider.agent === "pi" ? undefined : resolveAcpxQualification(provider, process.env),
@@ -12145,7 +12194,7 @@ async function createRunnerdBackendWithinSessionClaim(
           grokDisableWebSearch: "grokDisableWebSearch" in provider ? provider.grokDisableWebSearch : undefined,
           acpxPermissionModePinned: true } : {}),
         ...(provider.kind === "opencode" ? { opencodePermissionMode: provider.permissionMode } : {}),
-        runnerReconnectGraceMs: 120_000, runnerProcessReconnectGraceMs: null,
+        runnerReconnectGraceMs: REMOTE_RUNNER_RECONNECT_GRACE_MS, runnerProcessReconnectGraceMs: null,
         lifecyclePolicy: input.execution.session.lifecyclePolicy,
         runtimeContext: "runtimeContext" in input.execution ? input.execution.runtimeContext : null,
         externallySandboxed: true, baseInstructions: recovery?.baseInstructions,
@@ -12155,7 +12204,8 @@ async function createRunnerdBackendWithinSessionClaim(
         prpIdentity: adoptionIdentity ?? { runnerInstanceId: prior?.runnerInstanceId ?? effectiveRunnerInstanceId,
           environmentLeaseId: prior?.environmentLeaseId ?? effectiveEnvironmentLeaseId,
           runId: input.execution.binding.runId, normalizedSessionId: nativeSessionKey(input.execution),
-          turnId: `turn-${input.execution.binding.runId}`, itemId: `item-${input.execution.binding.runId}` },
+          turnId: adopting ? prior!.turnId : `turn-${input.execution.binding.runId}`,
+          itemId: adopting ? prior!.itemId : `item-${input.execution.binding.runId}` },
       }).transport,
     });
     sessionToolAuthorityEpochs.get(sessionScopeId)?.revoke();
@@ -12176,6 +12226,10 @@ async function createRunnerdBackendWithinSessionClaim(
     const externalExecution = input.execution;
     const externalOptions = {
         stateDirectory: root, runnerBinary: remoteDot?.runnerBinary ?? resolvePaperclipRunnerBinary(),
+        runnerArtifact: input.controllerPersistence?.runnerArtifact,
+        runnerReconnectGraceMs: remoteDot ? REMOTE_RUNNER_RECONNECT_GRACE_MS : undefined,
+        recoverSemanticToolInput: input.controllerPersistence ? (call: { operationId: string; callId: string; input: unknown }) =>
+          authorityEpoch.recover({ tool: call.operationId, callId: call.callId, arguments: call.input }) : undefined,
         ...(remoteDot ?? {}),
         identity: { runnerInstanceId: effectiveRunnerInstanceId, environmentLeaseId: effectiveEnvironmentLeaseId,
           runId: input.execution.binding.runId, normalizedSessionId: nativeSessionKey(input.execution),
@@ -12216,12 +12270,15 @@ async function createRunnerdBackendWithinSessionClaim(
     remoteTarget !== null &&
     (input.execution.provider.kind === "opencode" ||
       input.execution.provider.kind === "acpx");
+  const adoptingPersistedRemoteRunner = remoteTarget !== null
+    && input.restartRecovery?.kind === "reattach_remote_runner"
+    && input.controllerPersistence?.runnerArtifact !== undefined;
   const remoteClaude = remoteTarget !== null && input.execution.provider.kind === "acpx" && input.execution.provider.agent === "claude";
-  const remoteClaudeRunnerBinary = remoteClaude
+  const remoteClaudeRunnerBinary = remoteClaude && !adoptingPersistedRemoteRunner
     ? await resolveRemotePaperclipRunnerBinary({ runner: remoteCommandRunner!, remoteCwd: remoteTarget!.remoteCwd, configuredPath: input.runnerRemoteBinaryPath })
     : null;
   const remoteClaudeTarget = remoteClaudeRunnerBinary ? runnerBinaryTarget(readFileSync(remoteClaudeRunnerBinary)) : null;
-  const useBundledClaudePack = remoteClaude && !input.runnerRemoteProviderPackPath?.trim();
+  const useBundledClaudePack = remoteClaude && !adoptingPersistedRemoteRunner && !input.runnerRemoteProviderPackPath?.trim();
   if (useBundledClaudePack && remoteClaudeTarget !== "linux-x64" && remoteClaudeTarget !== "linux-arm64") {
     throw new Error("runner_remote_provider_artifact_incompatible: Claude remote provider packs require a qualified Linux target");
   }
@@ -12232,7 +12289,7 @@ async function createRunnerdBackendWithinSessionClaim(
   // Pi's explicit operator import supplies the target-platform authority. Never
   // substitute a Mac controller daemon or trust an image's self-reported hashes.
   // Existing explicit overrides and every other provider keep their old path.
-  const remotePiCompanion = await timeController("companion_selection", () => selectRemotePiCompanion({
+  const remotePiCompanion = adoptingPersistedRemoteRunner ? null : await timeController("companion_selection", () => selectRemotePiCompanion({
     provider: input.execution.provider, remote: remoteTarget !== null,
     binaryOverride: input.runnerRemoteBinaryPath, packOverride: input.runnerRemoteProviderPackPath,
     workspaceRoot: input.execution.workspace.cwd!,
@@ -12280,7 +12337,11 @@ async function createRunnerdBackendWithinSessionClaim(
   // When an explicit remote artifact is configured, prepareRemoteRunner stages
   // these exact bytes at remoteBinary before launch.
   const targetMatchedCodexArtifact = remoteTarget !== null && input.execution.provider.kind === "codex";
-  const controllerRunnerBinary = targetMatchedCodexArtifact
+  // The persisted admission owns a surviving runner's identity. Local release
+  // files are launch inputs and may no longer exist on a replacement controller.
+  const controllerRunnerBinary = adoptingPersistedRemoteRunner
+    ? input.runnerRemoteBinaryPath?.trim() || resolvePaperclipRunnerBinary()
+    : targetMatchedCodexArtifact
     ? await timeController("runner_binary", () => resolveRemotePaperclipRunnerBinary({ runner: remoteCommandRunner!, remoteCwd: remoteTarget.remoteCwd, configuredPath: input.runnerRemoteBinaryPath }))
     : remoteTarget
     ? remoteClaudeRunnerBinary || input.runnerRemoteBinaryPath?.trim() || remotePiCompanion?.runnerBinary || (useBundledRemoteImageAssets ? bundledRemoteRunnerBinary() : resolvePaperclipRunnerBinary())
@@ -12295,7 +12356,7 @@ async function createRunnerdBackendWithinSessionClaim(
     runnerArtifact: () => readRunnerdArtifactBinding(controllerRunnerBinary),
     current: resolveCurrentProviderPackManifest,
   }) : null;
-  let providerPackAuthority = selectProviderPackAuthority(reattachProviderPack);
+  let providerPackAuthority = adoptingPersistedRemoteRunner ? null : selectProviderPackAuthority(reattachProviderPack);
   let expectedProviderPackManifest = providerPackAuthority?.manifest ?? null;
   let providerPackVerified = false;
   const explicitRemoteCodex = input.runnerRemoteCodexPath?.trim() || null;
@@ -12471,6 +12532,10 @@ async function createRunnerdBackendWithinSessionClaim(
       return;
     selectedRemoteMode = requiredMode;
     if (input.restartRecovery?.kind === "reattach_remote_runner" || input.remoteWarmRecovery) {
+      if (input.controllerPersistence?.runnerArtifact) {
+        remotePrepared = true;
+        return;
+      }
       await remoteNativeArtifacts?.verifyRemoteRunner(requiredMode, undefined, true);
       if (requiresRemoteProviderPack && await useInstalledProviderPack()) {
         remotePrepared = true;
@@ -13689,12 +13754,17 @@ async function createRunnerdBackendWithinSessionClaim(
           : {}),
         stateDirectory: root,
         runnerBinary: controllerRunnerBinary,
-        ...(providerPackAuthority ? { runnerArtifact: providerPackAuthority.runnerArtifact } : {}),
+        runnerArtifact: input.controllerPersistence?.runnerArtifact ?? providerPackAuthority?.runnerArtifact,
+        recoverSemanticToolInput: input.controllerPersistence ? call => {
+          const current = sessionToolAuthorityEpochs.get(sessionScopeId);
+          if (!current || current.runId !== call.correlation.runId) throw new Error("native_tool_authority_epoch_revoked");
+          return current.recover({ tool: call.operationId, callId: call.callId, arguments: call.input });
+        } : undefined,
         codexCommand: remoteCodexBinary ?? undefined,
         sourceCodexHome: remoteTarget
           ? resolveSourceCodexHome(resolveNativeProviderEnvironment(input.execution.provider, runnerEnvironment))
           : undefined,
-        runnerReconnectGraceMs: remoteTarget ? 120_000 : undefined,
+        runnerReconnectGraceMs: remoteTarget ? REMOTE_RUNNER_RECONNECT_GRACE_MS : undefined,
         // A controller's bounded reconnect attempt is not the remote owner's
         // lifetime. Retain it through restarts until its authenticated lease
         // expires or explicit Stop/suspension proves that execution ended.
@@ -13806,6 +13876,12 @@ async function createRunnerdBackendWithinSessionClaim(
             adoptExistingRunner: adoptedProcess,
           },
           target,
+          controllerPersistence: input.controllerPersistence ? nativeRunnerCheckpointPersistence(
+            input.db, input.controllerPersistence.bootId, () => {
+              const current = sessionToolAuthorityEpochs.get(sessionScopeId);
+              if (!current) throw new Error("native_session_tool_authority_unavailable");
+              return current.runId;
+            }, detachNativeSessionsForRestart) : undefined,
           binding: input.remoteWarmRecovery
             ? { ...input.execution.binding, runId: input.remoteWarmRecovery.identity.runId }
             : input.execution.binding,

server/src/services/native-runtime/native-workspace-sync.ts

@@ -909,7 +909,7 @@ export async function prepareNativeWorkspaceSync(input: {
   const providerLeaseId = providerLeaseIdFor({ target, lease: input.lease });
   // Reattachment may never seed a replacement sandbox over a still-live run.
   if (input.restartRecovery?.kind === "reattach_remote_runner" &&
-      (input.restartRecovery.runId !== input.runId ||
+      (input.restartRecovery.remote.kind === "plugin_task" || input.restartRecovery.runId !== input.runId ||
        input.restartRecovery.remote.providerLeaseId !== providerLeaseId ||
        input.restartRecovery.remote.remoteCwd !== target.remoteCwd)) {
     throw new Error("native_remote_recovery_lease_mismatch");

server/src/services/native-runtime/paperclip-runner-tool-authority.test.ts

@@ -19,8 +19,8 @@ import {
   issueThreadInteractions,
   issues,
 } from "@paperclipai/db";
-import { startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
 import { createDatabaseResponseCapture, installDatabaseResponseCapture } from "@paperclipai/db/response-capture";
+import { startRecoveryPostgres } from "../../__tests__/helpers/recovery-postgres.js";
 import { initializeRunIdentity, reserveSteeredIdentity, reconcileSteeredIdentity } from "../run-identity.js";
 import { documentService } from "../documents.js";
 import { issueService } from "../issues.js";
@@ -36,7 +36,7 @@ import { CAPABILITY_SEMANTIC_TOOL_CATALOG, runnerCodexDynamicToolsFit } from "..
 describe("PaperclipRunnerToolAuthority", () => {
   const credentialDocumentBody = "Use a secret manager for credential handling.\nAuthorization: Bearer intentional-document-credential";
   let temporary: Awaited<
-    ReturnType<typeof startEmbeddedPostgresTestDatabase>
+    ReturnType<typeof startRecoveryPostgres>
   > | null = null;
   let db: ReturnType<typeof createDb>;
   const companyId = "00000000-0000-4000-8000-000000000101";
@@ -51,7 +51,7 @@ describe("PaperclipRunnerToolAuthority", () => {
   afterEach(() => vi.unstubAllEnvs());
 
   beforeAll(async () => {
-    temporary = await startEmbeddedPostgresTestDatabase(
+    temporary = await startRecoveryPostgres(
       "paperclip-runner-tools-",
     );
     db = createDb(temporary.connectionString);

server/src/services/native-runtime/paperclip-runner-tool-authority.ts

@@ -9,7 +9,7 @@ import { skillAccessService } from "../skill-access.js";
 import { withExternalExecutionAdmission, type ExternalAdmissionTransaction } from "../../modules/external-agents/index.js";
 import { callTaskWorkspaceTool, pendingTaskRepositoryContinuation } from "../task-workspace-tools.js";
 import { TaskAttachmentReadCache, TASK_ATTACHMENT_DEFINITIONS, TASK_ATTACHMENT_SCHEMAS, listTaskAttachments, readTaskAttachment } from "./runner-task-attachments.js";
-import { RUNNER_BRIDGE_SCHEMAS, runnerBridgeDefinitions, readAssignedSkill, executeWorkspaceTool, readWorkspaceUploadFile, settleRunnerBridgeRead, assertRunnerWorkspacePathAllowed } from "./runner-bridge-tools.js";
+import { RUNNER_BRIDGE_READ_OPERATIONS, RUNNER_BRIDGE_SCHEMAS, runnerBridgeDefinitions, readAssignedSkill, executeWorkspaceTool, readWorkspaceUploadFile, settleRunnerBridgeRead, assertRunnerWorkspacePathAllowed } from "./runner-bridge-tools.js";
 import type { NativeRuntimeContextSnapshot } from "../../vendor/paperclip-runner/index.js";
 import { assertCanManageIssueMonitor, eligibleIssueMonitorWait, prepareIssueMonitorUpdate, setTaskMonitorSchema, summarizeIssueMonitor } from "../issue-monitors.js";
 import { readTaskQuestionContext } from "../issue-question-context.js";
@@ -58,7 +58,7 @@ import { runnerApiSensitiveResponse } from "./runner-api-sensitive-response.js";
 import { workspaceFileResourceService } from "../workspace-file-resources.js";
 import { badRequest, forbidden, notFound, HttpError } from "../../errors.js";
 import { runnerApiCatalog, searchRunnerApi } from "./runner-api-catalog.js";
-import { executeRunnerApi, validateRunnerApiCall, RUNNER_API_MAX_BYTES, type RunnerApiFile, type RunnerApiIo } from "./runner-api-client.js";
+import { executeRunnerApi, validateRunnerApiCall, RUNNER_API_MAX_BYTES, type RunnerApiCall, type RunnerApiFile, type RunnerApiIo } from "./runner-api-client.js";
 import { acquireRunnerApiResponseSlot, runnerApiCompanyCaptureMaxBytes, RUNNER_API_RESPONSE_MAX_BYTES, RUNNER_API_RESPONSE_RUN_MAX_BYTES, RunnerApiResponseLimitError } from "./runner-api-response-limits.js";
 import { and, desc, eq, inArray, isNull, notInArray, sql } from "drizzle-orm";
 import type { Db } from "@paperclipai/db";
@@ -84,7 +84,7 @@ import {
   projects,
   issueThreadInteractions,
 } from "@paperclipai/db";
-import { CAPABILITY_SEMANTIC_TOOL_CATALOG, runnerCodexDynamicToolsFit, SemanticToolOutcomeUnknownError } from "../../vendor/paperclip-runner/index.js";
+import { CAPABILITY_CANONICAL_OPERATIONS, CAPABILITY_SEMANTIC_TOOL_CATALOG, runnerCodexDynamicToolsFit, SemanticToolOutcomeUnknownError } from "../../vendor/paperclip-runner/index.js";
 import { agentService } from "../agents.js";
 import { approvalService } from "../approvals.js";
 import { documentService } from "../documents.js";
@@ -144,6 +144,25 @@ const NATIVE_REVIEW_READ_TOOLS = new Set([
   "get_task_context", "get_task_history", "list_documents", "read_document", "list_document_revisions",
 ]);
 
+// The owning catalogs classify first-party reads. Recovery still calls execute(),
+// which rechecks the current tool advertisement, run, permissions and file scope.
+// External connector annotations never authorize replay of an uncertain effect.
+const RECOVERABLE_READ_OPERATIONS = new Set([
+  ...CAPABILITY_CANONICAL_OPERATIONS.filter(tool => IMPLEMENTED_OPERATIONS.has(tool.operationId) && tool.sideEffectClass === "read")
+    .map(tool => tool.operationId),
+  ...RUNNER_BRIDGE_READ_OPERATIONS,
+  ...Object.keys(TASK_ATTACHMENT_SCHEMAS),
+  READ_CURRENT_WAKE_COMMENTS_TOOL_NAME, LIST_CHAT_ATTACHMENTS_TOOL_NAME, READ_CHAT_ATTACHMENT_TOOL_NAME,
+  READ_TOOL_RESULT, "connections_search",
+]);
+const RECOVERABLE_READ_ERRORS = new Set([
+  "paperclip_runner_document_not_found", "paperclip_runner_agent_not_found",
+  "paperclip_runner_approval_not_found", "paperclip_runner_tool_input_invalid",
+]);
+const readOnlyApiMethod = (method: string) => ["GET", "HEAD", "OPTIONS"].includes(method);
+const toolCallKey = (callId: string) => createHash("sha256").update(callId).digest("hex");
+const toolInputDigest = (value: unknown) => createHash("sha256").update(canonicalJson(value)).digest("hex");
+
 type NativeIssueDocument = NonNullable<Awaited<ReturnType<ReturnType<typeof documentService>["getIssueDocumentByKey"]>>>;
 type NativeDocumentContent = { id: string | null; documentId: string; body: string; changeSummary?: string | null };
 const UNKNOWN_DOCUMENT_REVISION_BODY = "[Document revision content omitted: historical source trust could not be established.]";
@@ -239,6 +258,46 @@ function canonicalJson(value: unknown): string {
   return JSON.stringify(value) ?? "null";
 }
 
+function apiToolResult(value: unknown) {
+  const { operationId, ...response } = record(value);
+  return { ...response, apiOperationId: operationId };
+}
+
+const TASK_API_PATHS: Readonly<Record<string, readonly [string, string]>> = {
+  get_task: ["GET", "/api/issues/{id}"],
+  comment_on_task: ["POST", "/api/issues/{id}/comments"],
+  list_task_documents: ["GET", "/api/issues/{id}/documents"],
+  read_task_document: ["GET", "/api/issues/{id}/documents/{key}"],
+  write_task_document: ["PUT", "/api/issues/{id}/documents/{key}"],
+};
+
+function taskBridgeApiRequest(tool: string, input: Record<string, any>): RunnerApiCall {
+  const [method, path] = TASK_API_PATHS[tool]!;
+  const operation = runnerApiCatalog().find(operation => operation.method === method && operation.path === path);
+  if (!operation) throw new Error("runner_task_api_operation_unavailable");
+  const body = tool === "comment_on_task" ? { body: input.body } : tool === "write_task_document"
+    ? { title: input.title, format: "markdown", body: input.body, baseRevisionId: input.baseRevisionId } : undefined;
+  return { operationId: operation.operationId, pathParams: { id: input.taskId, ...(input.key ? { key: input.key } : {}) }, ...(body ? { body } : {}) };
+}
+
+function hireAgentApiRequest(input: Record<string, unknown>, binding: Pick<Binding, "companyId" | "agentId" | "issueId">): RunnerApiCall {
+  const body: Record<string, unknown> = {
+    name: requiredString(input.name), adapterType: "paperclip_runner", inheritRuntimeFrom: "caller",
+    reportsTo: binding.agentId, sourceIssueId: binding.issueId,
+  };
+  if (input.role !== undefined) body.role = input.role;
+  if (input.title !== undefined) body.title = input.title;
+  if (input.capabilities !== undefined) body.capabilities = input.capabilities;
+  if (typeof input.instructions === "string" && input.instructions.trim()) {
+    body.instructionsBundle = { entryFile: "AGENTS.md", files: { "AGENTS.md": input.instructions } };
+  }
+  return { operationId: "POST /api/companies/{companyId}/agent-hires", pathParams: { companyId: binding.companyId }, body };
+}
+
+function hireAgentResult(value: unknown, provider: unknown) {
+  return { ...apiToolResult(value), ...(provider === "openai_dot" ? { setupRequired: "This creates a separate Paperclip Dot agent. Its operator must pair a separate OpenAI Dot before it can run. The existing Dot binding and workspace permission are never inherited." } : {}) };
+}
+
 export class PaperclipRunnerToolAuthority {
   #taskAttachmentCache = new TaskAttachmentReadCache();
   close() { this.#taskAttachmentCache.close(); }
@@ -333,6 +392,128 @@ export class PaperclipRunnerToolAuthority {
     return this.binding.museRuntime ? all.filter(definition => MUSE_NATIVE_OPERATIONS.has(String(definition.name))) : all;
   }
 
+  async recover(call: Parameters<PaperclipRunnerToolAuthority["execute"]>[0]): Promise<{ result: unknown; isError?: boolean } | null> {
+    await this.#assertToolScope(call.tool);
+    if (!this.binding.museRuntime) await this.binding.assertBridgeAuthority?.();
+    const context = await this.#boundContext();
+    const { run } = context;
+    if (RECOVERABLE_READ_OPERATIONS.has(call.tool)) return this.#recoverRead(call);
+    if (call.tool === "update_case" || call.tool === "create_workflow" || call.tool === "update_workflow") {
+      const callApi = (receiptKey: string, request: RunnerApiCall) => this.#callApi(receiptKey, request, {
+        dedicatedCaseTool: call.tool, caseId: request.pathParams?.id, receiptOnly: true,
+      });
+      const result = call.tool === "update_case"
+        ? await executeNativeCaseTool({ db: this.db, binding: this.binding, actor: this.#privacyActor(run),
+            tool: call.tool, arguments: call.arguments, callApi })
+        : await executeNativeWorkflowTool({ db: this.db, binding: this.binding,
+            tool: call.tool, arguments: call.arguments, callApi });
+      return result === undefined ? null : { result };
+    }
+    const input = record(call.arguments);
+    if (call.tool === "create_project") {
+      const apiUrl = this.binding.apiUrl ?? process.env.PAPERCLIP_API_URL;
+      const token = this.#apiToken(context);
+      if (!apiUrl || !token) throw new Error("Project tool authentication is unavailable");
+      const result = await callProjectTool({ name: call.tool, arguments: input, apiUrl, token,
+        companyId: this.binding.companyId, issueId: this.binding.issueId, agentId: this.binding.agentId,
+        conversation: Boolean(context.issue.conversationAgentId), receiptOnly: true });
+      return result === undefined ? null : { result };
+    }
+    if (call.tool === "update_agent_instructions" || call.tool === "restore_agent_instructions") {
+      const receipt = record(record(run.resultJson?.semanticToolReceipts)[`instruction:${call.callId}`]);
+      if (!Object.hasOwn(receipt, "result")) return null;
+      return { result: await this.#withInstructionMutationReceipt(call.tool, call.callId, input, true) };
+    }
+    // Recovery goes through the receipt owner: it must use the same parsed
+    // input, current authority and safe post-commit dispatch as an ordinary
+    // retry, without ever falling through to an uncertain mutation.
+    let recovered: unknown;
+    if (call.tool === "set_task_title") recovered = await this.#setTaskTitle(input, true);
+    else if (call.tool === "submit_complaint" || call.tool === "submit_suggestion") recovered = await this.#submitCommentary(call.tool, input, true);
+    else if (call.tool === "create_task") recovered = await this.#createTask(input, null, true);
+    else if (call.tool === "reassign_task") recovered = await this.#reassignTask(input, true);
+    else if (call.tool === "set_task_monitor") recovered = await this.#setTaskMonitor(input, true);
+    else if (typeof input.idempotencyKey === "string" && input.idempotencyKey.trim()) {
+      // The other semantic mutations retain their original input and use
+      // requiredString for their retry key. A null effect is receipt-only.
+      recovered = await this.#withMutationReceipt(call.tool, requiredString(input.idempotencyKey), input, null);
+    }
+    if (recovered !== undefined) return { result: recovered };
+
+    const callKey = toolCallKey(call.callId);
+    if (call.tool === "call_api" || call.tool === "hire_agent" || Object.hasOwn(TASK_API_PATHS, call.tool)) {
+      if (!runnerApiToolsEnabled(this.binding.companyId, this.binding.apiToolsEnabled)) return null;
+      let request = call.arguments;
+      if (Object.hasOwn(TASK_API_PATHS, call.tool)) {
+        if (!this.definitions().some(definition => definition.name === call.tool)) return null;
+        const parsed = RUNNER_BRIDGE_SCHEMAS.get(call.tool)!.safeParse(call.arguments);
+        if (!parsed.success) return null;
+        request = taskBridgeApiRequest(call.tool, parsed.data as Record<string, any>);
+      } else if (call.tool === "hire_agent") request = hireAgentApiRequest(input, this.binding);
+      let validated: ReturnType<typeof validateRunnerApiCall>;
+      try {
+        validated = validateRunnerApiCall(request, {
+          ...this.binding, issueIdentifier: context.issue.identifier, workMode: context.issue.workMode,
+        });
+      } catch (error) {
+        if (error instanceof HttpError && error.status === 404) return null;
+        throw error;
+      }
+      const { input: apiInput, operation } = validated;
+      if (readOnlyApiMethod(operation.method)) return this.#recoverRead(call);
+      const result = await this.#completedApiReceipt(run.resultJson?.apiToolReceipts, call.callId, apiInput);
+      if (result === undefined) return null;
+      return { result: call.tool === "hire_agent" ? hireAgentResult(result, context.actor.adapterConfig?.provider) : apiToolResult(result) };
+    }
+    if (call.tool === "workspace_write" || call.tool === "workspace_run") {
+      const bridge = record(record(run.resultJson?.bridgeToolReceipts)[callKey]);
+      if (bridge.state === "completed" && bridge.tool === call.tool
+        && bridge.digest === toolInputDigest({ tool: call.tool, arguments: call.arguments })
+        && Object.hasOwn(bridge, "result")) return { result: bridge.result };
+    }
+    return null;
+  }
+
+  async #recoverRead(call: Parameters<PaperclipRunnerToolAuthority["execute"]>[0]) {
+    try {
+      return { result: await this.execute(call) };
+    } catch (error) {
+      // A known read failure is a tool outcome, not a failed transport. Keep
+      // authorization, uncertain effects, and unexpected storage errors fatal.
+      if (!(error instanceof Error && (RECOVERABLE_READ_ERRORS.has(error.message)
+        || (error instanceof HttpError && [400, 404, 422].includes(error.status))))) throw error;
+      await this.binding.assertBridgeAuthority?.();
+      await this.#boundContext();
+      return { isError: true, result: {
+        success: false, contentItems: [{ type: "inputText", text: error.message }],
+      } };
+    }
+  }
+
+  async #completedApiReceipt(receipts: unknown, callId: string, input: RunnerApiCall): Promise<unknown> {
+    const receipt = record(record(receipts)[toolCallKey(callId)]);
+    if (receipt.state !== "completed" || receipt.digest !== toolInputDigest(input)) return undefined;
+    const result = await this.#apiReceiptResult(receipt);
+    return record(result).outcome === "unknown" ? undefined : result;
+  }
+
+  async #apiReceiptResult(receipt: { encryptedResult?: unknown; result?: unknown }): Promise<unknown> {
+    if (!receipt.encryptedResult) return receipt.result;
+    const text = await getSecretProvider("local_encrypted").resolveVersion({ material: record(receipt.encryptedResult), externalRef: null });
+    return JSON.parse(text);
+  }
+
+  async #assertToolScope(tool: string): Promise<void> {
+    if (this.binding.museRuntime) {
+      await this.binding.assertBridgeAuthority?.();
+      if (!MUSE_NATIVE_OPERATIONS.has(tool)) throw forbidden("This operation is outside the Muse first-party runtime profile");
+    }
+    if (this.binding.firstPartyOnly && (EXTERNAL_WORKSPACE_OPERATIONS.has(tool) || tool.startsWith("workspace_"))) throw forbidden("Paperclip workspace access is unavailable to this Grok Bot session.");
+    if (this.binding.nativeReview && tool !== "resolve_review" && !NATIVE_REVIEW_READ_TOOLS.has(tool)) {
+      throw forbidden("This review run may only inspect the assigned task and resolve its review.");
+    }
+  }
+
   async execute(call: {
     tool: string;
     callId: string;
@@ -364,17 +545,8 @@ export class PaperclipRunnerToolAuthority {
         beforeAccept: async tx => { await this.#lockAuthorizedMutationContext(tx); },
       });
     }
-    if (this.binding.museRuntime) {
-      await this.binding.assertBridgeAuthority?.();
-      if (!MUSE_NATIVE_OPERATIONS.has(call.tool)) throw forbidden("This operation is outside the Muse first-party runtime profile");
-    }
-    if (this.binding.firstPartyOnly && (EXTERNAL_WORKSPACE_OPERATIONS.has(call.tool) || call.tool.startsWith("workspace_"))) throw forbidden("Paperclip workspace access is unavailable to this Grok Bot session.");
-    if (this.binding.nativeReview) {
-      if (call.tool === "resolve_review") return this.#resolveReview(call.arguments);
-      if (!NATIVE_REVIEW_READ_TOOLS.has(call.tool)) {
-        throw forbidden("This review run may only inspect the assigned task and resolve its review.");
-      }
-    }
+    await this.#assertToolScope(call.tool);
+    if (this.binding.nativeReview && call.tool === "resolve_review") return this.#resolveReview(call.arguments);
     if (Object.hasOwn(TASK_ATTACHMENT_SCHEMAS, call.tool)) {
       const authorize = async () => {
         const current = await this.#boundContext();
@@ -432,14 +604,7 @@ export class PaperclipRunnerToolAuthority {
         if (call.tool === "list_assigned_skills") return runtimeContext.skills.map(skill => ({ key: skill.key, name: skill.runtimeName, versionId: skill.versionId, digest: skill.bundle.digest }));
         return settleRunnerBridgeRead(() => readAssignedSkill(runtimeContext, input));
       }
-      const paths: Record<string, [string, string]> = { get_task: ["GET", "/api/issues/{id}"], comment_on_task: ["POST", "/api/issues/{id}/comments"],
-        list_task_documents: ["GET", "/api/issues/{id}/documents"], read_task_document: ["GET", "/api/issues/{id}/documents/{key}"], write_task_document: ["PUT", "/api/issues/{id}/documents/{key}"] };
-      const [method, path] = paths[call.tool]!;
-      const operation = runnerApiCatalog().find(operation => operation.method === method && operation.path === path);
-      if (!operation) throw new Error("runner_task_api_operation_unavailable");
-      const body = call.tool === "comment_on_task" ? { body: input.body } : call.tool === "write_task_document" ? { title: input.title, format: "markdown", body: input.body, baseRevisionId: input.baseRevisionId } : undefined;
-      const { operationId, ...response } = record(await this.#callApi(call.callId, { operationId: operation.operationId, pathParams: { id: input.taskId, ...(input.key ? { key: input.key } : {}) }, ...(body ? { body } : {}) }));
-      return { ...response, apiOperationId: operationId };
+      return apiToolResult(await this.#callApi(call.callId, taskBridgeApiRequest(call.tool, input)));
     }
     if (this.binding.firstPartyOnly && (call.tool === READ_TOOL_RESULT || this.binding.assignedMcpTools?.has(call.tool) || isConnectorTool(call.tool)
       || RUNTIME_CONNECTION_TOOL_DEFINITIONS.some(tool => tool.name === call.tool))) throw forbidden("This Grok Bot connection exposes first-party Paperclip operations. Use your own provider apps under their existing permissions.");
@@ -580,13 +745,8 @@ export class PaperclipRunnerToolAuthority {
           callApi: (receiptKey, request) => this.#callApi(receiptKey, request, { dedicatedCaseTool: call.tool, caseId: request.pathParams?.id }),
         });
       case "submit_complaint":
-      case "submit_suggestion": {
-        const parsed = agentCommentaryToolInputSchema.safeParse(call.arguments);
-        if (!parsed.success) throw badRequest("Feedback accepts only a nonempty body of at most 524288 characters and a retry key of at most 240 characters");
-        return submitAgentCommentary(this.db, this.binding, {
-          ...parsed.data, kind: call.tool === "submit_complaint" ? "complaint" : "suggestion",
-        }, (tx) => this.#lockAuthorizedMutationContext(tx));
-      }
+      case "submit_suggestion":
+        return this.#submitCommentary(call.tool, input);
       case "read_agent_instructions":
       case "get_agent_instruction_history":
         return executeAgentInstructionTool({ db: this.db, binding: {
@@ -629,35 +789,13 @@ export class PaperclipRunnerToolAuthority {
       }
       case "search_api": return searchRunnerApi(call.arguments);
       case "hire_agent": {
-        const body: Record<string, unknown> = {
-          name: requiredString(input.name),
-          adapterType: "paperclip_runner",
-          inheritRuntimeFrom: "caller",
-          reportsTo: this.binding.agentId,
-          sourceIssueId: this.binding.issueId,
-        };
-        if (input.role !== undefined) body.role = input.role;
-        if (input.title !== undefined) body.title = input.title;
-        if (input.capabilities !== undefined) body.capabilities = input.capabilities;
-        if (typeof input.instructions === "string" && input.instructions.trim()) {
-          body.instructionsBundle = {
-            entryFile: "AGENTS.md",
-            files: { "AGENTS.md": input.instructions },
-          };
-        }
-        const { operationId, ...response } = record(await this.#callApi(call.callId, {
-          operationId: "POST /api/companies/{companyId}/agent-hires",
-          pathParams: { companyId: this.binding.companyId },
-          body,
-        }));
-        return { ...response, apiOperationId: operationId, ...(context.actor.adapterConfig?.provider === "openai_dot" ? { setupRequired: "This creates a separate Paperclip Dot agent. Its operator must pair a separate OpenAI Dot before it can run. The existing Dot binding and workspace permission are never inherited." } : {}) };
+        return hireAgentResult(await this.#callApi(call.callId, hireAgentApiRequest(input, this.binding)), context.actor.adapterConfig?.provider);
       }
       case "call_api": {
         // PRP reserves operationId/callId for semantic result identity. The
         // HTTP operation is metadata, including in previously saved receipts;
         // exposing it as operationId makes the runner reject a valid response.
-        const { operationId, ...response } = record(await this.#callApi(call.callId, call.arguments));
-        return { ...response, apiOperationId: operationId };
+        return apiToolResult(await this.#callApi(call.callId, call.arguments));
       }
       case "get_task_context": {
         const monitorCheckAt = !this.binding.nativeReview && context.issue.workMode === "standard" &&
@@ -808,8 +946,8 @@ export class PaperclipRunnerToolAuthority {
 
   async #bridgeMutation(call: { tool: string; callId: string; arguments: unknown }, execute: () => Promise<unknown>) {
     if (!call.callId || call.callId.length > 500) throw badRequest("Use a stable bounded call ID");
-    const key = createHash("sha256").update(call.callId).digest("hex");
-    const digest = createHash("sha256").update(canonicalJson({ tool: call.tool, arguments: call.arguments })).digest("hex");
+    const key = toolCallKey(call.callId);
+    const digest = toolInputDigest({ tool: call.tool, arguments: call.arguments });
     const prior = await this.db.transaction(async tx => {
       const context = await this.#lockAuthorizedMutationContext(tx as unknown as Db);
       const resultJson = record(context.run.resultJson), receipts = record(resultJson.bridgeToolReceipts);
@@ -857,7 +995,7 @@ export class PaperclipRunnerToolAuthority {
       this.binding.runId, context.run.responsibleUserId, this.#apiScope(context), this.binding.externalAuthorizingUserId);
   }
 
-  async #callApi(callId: string, value: unknown, options?: { dedicatedCaseTool: string; caseId?: string }): Promise<unknown> {
+  async #callApi(callId: string, value: unknown, options?: { dedicatedCaseTool: string; caseId?: string; receiptOnly?: true }): Promise<unknown> {
     const bound = await this.#boundContext();
     const context = { ...this.binding, issueIdentifier: bound.issue.identifier, workMode: bound.issue.workMode };
     const { input, operation } = validateRunnerApiCall(value, context);
@@ -878,6 +1016,10 @@ export class PaperclipRunnerToolAuthority {
     if (this.binding.firstPartyOnly && /\/(?:workspaces|workspace-runtime|repositories|tool-gateway|tool-connections|tool-profiles|tools|connection-intents|connections|plugins)(?:\/|$)/.test(operation.path)) throw forbidden("Third-party tools through Paperclip are unavailable for this Grok Bot connection.");
     const sensitiveResponse = runnerApiSensitiveResponse(operation.method, operation.path);
     await this.#assertDotApiFileAccess(operation.path, input);
+    if (options?.receiptOnly) {
+      assertAvailable(bound.issue.workMode);
+      return this.#completedApiReceipt(bound.run.resultJson?.apiToolReceipts, callId, input);
+    }
     const apiUrl = this.binding.apiUrl ?? process.env.PAPERCLIP_API_URL;
     if (!apiUrl) throw new Error("Paperclip API origin is unavailable");
     const token = this.#apiToken(bound);
@@ -899,36 +1041,27 @@ export class PaperclipRunnerToolAuthority {
       await this.#assertDotApiFileAccess(operation.path, input);
       return result;
     };
-    if (["GET", "HEAD", "OPTIONS"].includes(operation.method)) return execute();
+    if (readOnlyApiMethod(operation.method)) return execute();
     if (!callId || callId.length > 500) throw badRequest("A bounded runner call id is required");
     const provider = sensitiveResponse ? getSecretProvider("local_encrypted") : null;
     if (provider) await provider.createSecret({ value: "runner-api-receipt-key-check" });
     // Reserve durably before HTTP dispatch, without holding a DB lock over an
     // HTTP route that itself needs DB locks. A crash leaves an explicit unknown
     // outcome rather than replaying a possibly committed external mutation.
-    const key = createHash("sha256").update(callId).digest("hex");
-    const digest = createHash("sha256").update(canonicalJson(input)).digest("hex");
+    const key = toolCallKey(callId);
+    const digest = toolInputDigest(input);
     const prior = await this.db.transaction(async (tx) => {
       const locked = await this.#lockAuthorizedMutationContext(tx as unknown as Db);
       const resultJson = record(locked.run.resultJson);
       const receipts = record(resultJson.apiToolReceipts);
       const inherited = await priorRecoveryApiReceipt(tx as unknown as Db, { companyId: this.binding.companyId,
         issueId: this.binding.issueId, runId: this.binding.runId, operationId: operation.operationId, digest });
-      if (inherited) {
-        if (inherited.encryptedResult) {
-          const text = await getSecretProvider("local_encrypted").resolveVersion({ material: record(inherited.encryptedResult), externalRef: null });
-          return { result: JSON.parse(text) as unknown };
-        }
-        return { result: inherited.result };
-      }
+      if (inherited) return { result: await this.#apiReceiptResult(inherited) };
       const existing = record(receipts[key]);
       if (receipts[key]) {
         if (existing.digest !== digest) throw badRequest("API call id was reused with different arguments");
-        if (existing.encryptedResult) {
-          const text = await getSecretProvider("local_encrypted").resolveVersion({ material: record(existing.encryptedResult), externalRef: null });
-          return { result: JSON.parse(text) as unknown };
-        }
-        return { result: existing.result ?? { ok: false, status: null, error: "api_outcome_unknown", outcome: "unknown", guidance: "Inspect state before issuing another mutation." } };
+        const result = await this.#apiReceiptResult(existing);
+        return { result: existing.encryptedResult ? result : result ?? { ok: false, status: null, error: "api_outcome_unknown", outcome: "unknown", guidance: "Inspect state before issuing another mutation." } };
       }
       if (Object.keys(receipts).length >= 512) throw badRequest("Run API mutation limit reached");
       receipts[key] = { digest, operationId: operation.operationId, state: "pending" };
@@ -1434,7 +1567,7 @@ export class PaperclipRunnerToolAuthority {
     if (!member) throw forbidden("Person is not an active company member");
   }
 
-  async #createTask(input: Record<string, unknown>, identityContextId: string | null): Promise<unknown> {
+  async #createTask(input: Record<string, unknown>, identityContextId: string | null, receiptOnly = false): Promise<unknown> {
     const idempotencyKey = requiredString(input.idempotencyKey);
     if (input.status !== undefined && input.status !== "backlog" && input.status !== "todo") {
       throw new Error("paperclip_runner_tool_input_invalid");
@@ -1602,7 +1735,8 @@ export class PaperclipRunnerToolAuthority {
           assigneeUserId: child.assigneeUserId,
         },
       };
-    }, { beforeReceiptReplay: async (tx, context) => { await authorizeCreate(tx, context); } }) as Record<string, unknown>;
+    }, { receiptOnly, beforeReceiptReplay: async (tx, context) => { await authorizeCreate(tx, context); } }) as Record<string, unknown> | undefined;
+    if (result === undefined) return undefined;
 
     if (publication) publishActivity(publication);
     const task = record(result.task);
@@ -1637,7 +1771,7 @@ export class PaperclipRunnerToolAuthority {
   }
 
 
-  async #reassignTask(input: Record<string, unknown>): Promise<unknown> {
+  async #reassignTask(input: Record<string, unknown>, receiptOnly = false): Promise<unknown> {
     const key = requiredString(input.idempotencyKey);
     const taskId = requiredUuid(input.taskId);
     const assigneeUserId = input.assigneeUserId == null ? null : requiredString(input.assigneeUserId);
@@ -1704,7 +1838,7 @@ export class PaperclipRunnerToolAuthority {
     };
     // Stop outside row locks: the runtime needs those locks to acknowledge cancellation.
     // A second locked check below prevents any intervening ownership/version change.
-    const prepared = await this.db.transaction(async (tx) => {
+    const prepared = receiptOnly ? null : await this.db.transaction(async (tx) => {
       const context = await this.#lockAuthorizedMutationContext(tx as unknown as Db);
       if (context.issue.workMode !== "standard") throw new Error("paperclip_runner_tool_mode_denied");
       const target = await authorize(tx as unknown as Db, context.run);
@@ -1793,12 +1927,13 @@ export class PaperclipRunnerToolAuthority {
       });
       publication = activity.publication;
       return receipt;
-    }, { beforeReceiptReplay: async (tx, context) => { await authorize(tx, context.run); } }).catch(async (error: unknown) => {
+    }, { receiptOnly, beforeReceiptReplay: async (tx, context) => { await authorize(tx, context.run); } }).catch(async (error: unknown) => {
       await restoreInterruptedWork(error);
       throw error;
     }) as {
       stateRevision: number; scheduledWakeIds: string[];
-    };
+    } | undefined;
+    if (result === undefined) return undefined;
     if (publication) publishActivity(publication);
     // Replay repairs a failed dispatch. The durable wake key and state/version guard
     // prevent duplicate successors and prevent waking a later, unrelated assignment.
@@ -1814,17 +1949,33 @@ export class PaperclipRunnerToolAuthority {
     return result;
   }
 
-  async #setTaskTitle(input: Record<string, unknown>): Promise<unknown> {
+  async #submitCommentary(tool: "submit_complaint" | "submit_suggestion", input: Record<string, unknown>, receiptOnly = false): Promise<unknown> {
+    const parsed = agentCommentaryToolInputSchema.safeParse(input);
+    if (!parsed.success) throw badRequest("Feedback accepts only a nonempty body of at most 524288 characters and a retry key of at most 240 characters");
+    const value = { ...parsed.data, kind: tool === "submit_complaint" ? "complaint" : "suggestion" };
+    const authorize = (tx: Db) => this.#lockAuthorizedMutationContext(tx);
+    const result = receiptOnly
+      ? await submitAgentCommentary(this.db, this.binding, value, authorize, { receiptOnly: true })
+      : await submitAgentCommentary(this.db, this.binding, value, authorize);
+    return result ?? undefined;
+  }
+
+  async #setTaskTitle(input: Record<string, unknown>, receiptOnly = false): Promise<unknown> {
     const titleInput = setIssueTitleSchema.parse(input);
     requiredString(titleInput.idempotencyKey);
     // Use the shared title receipt so native and HTTP retries have the same
     // identity and cannot overwrite a later user edit when switching surfaces.
-    const { result, publication } = await this.db.transaction(async (tx) => {
+    const receipt = await this.db.transaction(async (tx) => {
       await this.#lockAuthorizedMutationContext(tx as unknown as Db);
-      return setIssueTitle(tx as unknown as Db, this.binding.companyId, this.binding.issueId, titleInput, {
+      const actor = {
         actorType: "agent", actorId: this.binding.agentId, agentId: this.binding.agentId, runId: this.binding.runId,
-      });
+      } as const;
+      return receiptOnly
+        ? setIssueTitle(tx as unknown as Db, this.binding.companyId, this.binding.issueId, titleInput, actor, { receiptOnly: true })
+        : setIssueTitle(tx as unknown as Db, this.binding.companyId, this.binding.issueId, titleInput, actor);
     });
+    if (!receipt) return undefined;
+    const { result, publication } = receipt;
     if (publication) publishActivity(publication);
     const syncExternalObjects = this.binding.syncIssueExternalObjects ?? externalObjectService(this.db, {
       enabled: async () => (await instanceSettingsService(this.db).getExperimental()).enableExternalObjects === true,
@@ -1833,7 +1984,7 @@ export class PaperclipRunnerToolAuthority {
     return result;
   }
 
-  async #setTaskMonitor(value: Record<string, unknown>): Promise<unknown> {
+  async #setTaskMonitor(value: Record<string, unknown>, receiptOnly = false): Promise<unknown> {
     const input = setTaskMonitorSchema.parse(value);
     if (input.monitor?.externalRef) input.monitor.externalRef = redactIssueMonitorExternalRef(input.monitor.externalRef);
     const targetId = input.taskId ?? this.binding.issueId;
@@ -1902,12 +2053,13 @@ export class PaperclipRunnerToolAuthority {
       });
       publication = activity.publication;
       return { taskId: updated.id, status: updated.status, monitor };
-    }, { beforeReceiptReplay: async (tx, context) => {
+    }, { receiptOnly, beforeReceiptReplay: async (tx, context) => {
       const { target } = await authorizeTarget(tx, context);
       replayed = true;
       replayStatus = target.status;
       replayMonitor = summarizeIssueMonitor(target, normalizeIssueExecutionPolicy(target.executionPolicy));
     } });
+    if (result === undefined) return undefined;
     if (publication) publishActivity(publication);
     return { ...record(result), ...(replayed ? { status: replayStatus, monitor: replayMonitor } : {}), replayed };
   }
@@ -2264,6 +2416,7 @@ export class PaperclipRunnerToolAuthority {
     tool: "update_agent_instructions" | "restore_agent_instructions",
     callId: string,
     input: Record<string, unknown>,
+    recoveryOnly = false,
   ): Promise<unknown> {
     if (!callId || callId.length > 500) throw badRequest("A bounded runner call id is required");
     const key = `instruction:${callId}`;
@@ -2283,7 +2436,7 @@ export class PaperclipRunnerToolAuthority {
       // the attempt *before* permitting that effect. If the effect transaction
       // or its acknowledgement is lost, a missing result is unknown, not a
       // license to run the instruction write a second time.
-      const reserved = await this.db.transaction(async (tx) => {
+      const reserved = recoveryOnly ? false : await this.db.transaction(async (tx) => {
         const context = await this.#lockAuthorizedMutationContext(tx as unknown as Db);
         const bound = await authorize(tx as unknown as Db);
         const resultJson = record(context.run.resultJson);
@@ -2315,10 +2468,14 @@ export class PaperclipRunnerToolAuthority {
         return true;
       });
       try {
-        return await this.#withMutationReceipt(tool, key, input, (tx) =>
-          executeAgentInstructionTool({ db: tx, binding: {
+        return await this.#withMutationReceipt(tool, key, input, async (tx) => {
+          // Recovery reuses current authorization and receipt validation. If
+          // the saved receipt vanished after inspection, never repeat the write.
+          if (recoveryOnly) throw new SemanticToolOutcomeUnknownError(`paperclip_runner_instruction_receipt_unavailable call_id=${callId}`);
+          return executeAgentInstructionTool({ db: tx, binding: {
             companyId: this.binding.companyId, agentId: this.binding.agentId, runId: this.binding.runId,
-          }, tool, arguments: input }), { beforeReceiptReplay: async (tx) => { await authorize(tx); } });
+          }, tool, arguments: input });
+        }, { beforeReceiptReplay: async (tx) => { await authorize(tx); } });
       } catch (error) {
         // Instruction validation, authorization and CAS errors precede the
         // file write. Storage errors and lost commit acknowledgements do not
@@ -2367,15 +2524,16 @@ export class PaperclipRunnerToolAuthority {
     operationId: string,
     idempotencyKey: string,
     input: Record<string, unknown>,
-    effect: (
+    effect: ((
       tx: Db,
       context: {
         run: typeof heartbeatRuns.$inferSelect;
         issue: typeof issues.$inferSelect;
         actor: typeof agents.$inferSelect;
       },
-    ) => Promise<unknown>,
+    ) => Promise<unknown>) | null,
     options: {
+      receiptOnly?: boolean;
       onDefinitePreCommitFailure?: () => Promise<void>;
       beforeReceiptReplay?: (
         tx: Db,
@@ -2451,11 +2609,13 @@ export class PaperclipRunnerToolAuthority {
             prior.operationId !== operationId ||
             canonicalJson(prior.input) !== canonicalJson(input)
           ) {
+            if (effect === null || options.receiptOnly) return undefined;
             throw new Error("paperclip_runner_tool_idempotency_conflict");
           }
           await options.beforeReceiptReplay?.(tx as unknown as Db, context);
           return describeResult(prior.result);
         }
+        if (effect === null || options.receiptOnly) return undefined;
         const result = JSON.parse(
           JSON.stringify(
             describeResult(await effect(tx as unknown as Db, context)),

server/src/services/native-runtime/plugin-runner-task.test.ts

@@ -51,6 +51,90 @@ function provider() {
 }
 
 describe("plugin native Runner ownership", () => {
+  it("rechecks revoked grants for unchanged project IDs before adopting retained resources", async () => {
+    const { task } = provider();
+    const perform = task.getMockImplementation()!;
+    let allowed = true;
+    task.mockImplementation(async op => op.kind === "check_project_auth"
+      ? { ...await perform(op) as object, allowed } : perform(op));
+    const input = { task, projectIds: ["project-retained"], harness: "codex", projectAuthCheckVersion: 1 as const,
+      adoptIdentity: requirements.identity };
+    const owner = createPluginRunnerTaskOwner(input);
+    await owner.driver.authorizeRunnerAdoption!("retained-configuration", requirements.identity);
+    allowed = false;
+    owner.rebind({ ...input });
+    await expect(owner.driver.authorizeRunnerAdoption!("retained-configuration", requirements.identity)).rejects.toThrow("project authentication conflicts");
+    expect(task.mock.calls.map(([op]) => op)).toEqual(Array(2).fill({ kind: "check_project_auth",
+      configurationId: "retained-configuration", provider: "openai", projectIds: ["project-retained"] }));
+  });
+  it.each(["identity", "unsupported", "released", "late-release"])("refuses adoption authorization after %s", async mode => {
+    const { task } = provider();
+    const perform = task.getMockImplementation()!;
+    let finish!: () => void;
+    const gate = new Promise<void>(resolve => { finish = resolve; });
+    task.mockImplementation(async op => { if (mode === "late-release" && op.kind === "check_project_auth") await gate; return perform(op); });
+    const owner = createPluginRunnerTaskOwner({ task, projectIds: ["project-current"], harness: "codex",
+      projectAuthCheckVersion: mode === "unsupported" ? undefined : 1, adoptIdentity: requirements.identity });
+    const registration = await owner.registration({} as never, "run");
+    if (mode === "released") await registration.release("detach");
+    const attempt = owner.driver.authorizeRunnerAdoption!("retained-configuration",
+      mode === "identity" ? { ...requirements.identity, runId: "unrelated" } : requirements.identity);
+    const rejection = expect(attempt).rejects.toThrow(mode === "identity" ? "identity mismatch"
+      : mode === "unsupported" ? "validation unavailable" : "preparation cancelled");
+    if (mode === "late-release") { await registration.release("detach"); finish(); }
+    await rejection;
+    await registration.release("detach");
+    expect(task.mock.calls.map(([op]) => op.kind)).toEqual(mode === "late-release" ? ["check_project_auth"] : []);
+  });
+  it("adopts the saved task and detaches without preparing, submitting, or stopping it", async () => {
+    const { task } = provider();
+    const close = vi.fn(async () => {});
+    ingress.connect.mockReturnValue({ ready: Promise.resolve(), close });
+    const owner = createPluginRunnerTaskOwner({ task, projectIds: [], harness: "codex",
+      adoptIdentity: requirements.identity });
+    const registration = await owner.registration({} as never, "run");
+    registration.activate(); await registration.ready();
+    expect(await owner.driver.adoptExistingRunner!.isAlive()).toBe(true);
+    expect(owner.driver.adoptExistingRunner!.pid).toBeNull();
+    expect(() => owner.launcher(launchSpec())).toThrow("replacement requires a new task lease");
+    await registration.release("detach");
+    expect(close).toHaveBeenCalledOnce();
+    expect(task.mock.calls.map(([op]) => op.kind)).toEqual(["status", "read_state", "connection", "status"]);
+  });
+  it.each(["stopped", "wrong-identity", "revoked", "cancelled"])("refuses adoption when the saved task is %s", async mode => {
+    const { task } = provider();
+    const perform = task.getMockImplementation()!;
+    const abort = new AbortController();
+    task.mockImplementation(async op => {
+      if (mode === "revoked" && op.kind === "read_state") throw new Error("execution lease is not active");
+      const result = await perform(op) as any;
+      if (mode === "stopped" && op.kind === "status") result.executionStopped = true;
+      if (mode === "wrong-identity" && op.kind === "read_state") result.state.runId = "other-run";
+      if (mode === "cancelled" && op.kind === "read_state") abort.abort();
+      return result;
+    });
+    ingress.connect.mockClear();
+    const owner = createPluginRunnerTaskOwner({ task, projectIds: [], harness: "codex",
+      adoptIdentity: requirements.identity });
+    const registration = await owner.registration({} as never, "run", abort.signal);
+    registration.activate();
+    await expect(registration.ready()).rejects.toThrow();
+    await registration.release("detach");
+    expect(ingress.connect).not.toHaveBeenCalled();
+    expect(task.mock.calls.some(([op]) => ["prepare", "submit", "stop", "complete"].includes(op.kind))).toBe(false);
+  });
+  it("settles an adopted task through confirmed termination and retained-session ownership", async () => {
+    const { task } = provider();
+    const retained = vi.fn(async () => {});
+    ingress.connect.mockReturnValue({ ready: Promise.resolve(), close: async () => {} });
+    const owner = createPluginRunnerTaskOwner({ task, projectIds: [], harness: "codex",
+      adoptIdentity: requirements.identity, onSessionRetained: retained });
+    const registration = await owner.registration({} as never, "run");
+    registration.activate(); await registration.ready();
+    await registration.release(); await registration.checkpoint("settled");
+    expect(retained).toHaveBeenCalledWith(expect.objectContaining(requirements.identity));
+    expect(task.mock.calls.map(([op]) => op.kind)).toEqual(["status", "read_state", "connection", "stop", "status", "read_state", "complete"]);
+  });
   it("submits only a prepared identity and configuration ID", async () => {
     const { task } = provider();
     const owner = createPluginRunnerTaskOwner({ projectAuthCheckVersion: 1, task, projectIds: [], harness: "codex", pollMs: 1 });

server/src/services/native-runtime/plugin-runner-task.ts

@@ -7,6 +7,22 @@ import { RUNNER_PRP_PROTOCOL_MIN, RUNNER_PRP_PROTOCOL_MAX, type RunnerProcessHan
   type DurablePrpControlPlane, type RunnerExecutionDriver, type RunnerLaunchPreparer, type DurableRecoveryIdentity } from "../../vendor/paperclip-runner/index.js";
 import { connectRunnerPrpIngress } from "../../realtime/runner-prp-outbound.js";
 
+export async function verifyPluginRunnerTaskReattachment(
+  task: (operation: unknown) => Promise<unknown>, identity: unknown,
+): Promise<void> {
+  const operation = environmentTaskOperationSchema.parse({ kind: "read_state", identity });
+  if (operation.kind !== "read_state") throw new Error("Plugin Runner recovery identity unavailable");
+  const status = environmentTaskResultSchema.parse(await task({ kind: "status" }));
+  if (status.kind !== "status" || status.executionStopped || status.phase !== "running") {
+    throw new Error("Plugin Runner is not running");
+  }
+  const result = environmentTaskResultSchema.parse(await task(operation));
+  if (result.kind !== "state" || result.state.schema !== "paperclip.runner.durable.state.v1"
+    || Object.entries(operation.identity).some(([key, value]) => result.state[key] !== value)) {
+    throw new Error("Plugin Runner adoption identity mismatch");
+  }
+}
+
 /** Adapts a lease-bound task provider to the existing native process owner.
  * No provider pid is treated as a local pid. Only executionStopped settles it.
  */
@@ -66,12 +82,19 @@ export function createPluginRunnerTaskOwner(input: {
     if (validation.kind !== "project_auth" || !validation.allowed || validation.configurationId !== configurationId
       || validation.provider !== provider) throw new Error("Plugin Runner project authentication conflicts with the selected managed account");
   };
+  const authorizeRunnerAdoption = async (configurationId: string, identity: DurableRecoveryIdentity) => {
+    if (released || stopRequested) throw new Error("Plugin Runner preparation cancelled");
+    if (!input.adoptIdentity || Object.entries(input.adoptIdentity).some(([key, value]) => identity[key as keyof DurableRecoveryIdentity] !== value)) {
+      throw new Error("Plugin Runner adoption identity mismatch");
+    }
+    await checkProjectAuth(configurationId);
+    preparedLaunchId = configurationId;
+  };
   const prepareLaunch: RunnerLaunchPreparer = async (requirements, recovery) => {
     if (released || stopRequested) throw new Error("Plugin Runner preparation cancelled");
     // Reconnecting the previous authority must not prepare it through the new run's lease.
     if (recovery && input.adoptIdentity && Object.entries(input.adoptIdentity).every(([key,value]) => requirements.identity[key as keyof DurableRecoveryIdentity] === value)) {
-      await checkProjectAuth(recovery.configurationId);
-      preparedLaunchId = recovery.configurationId;
+      await authorizeRunnerAdoption(recovery.configurationId, requirements.identity);
       return recovery;
     }
     const result = await call(environmentTaskOperationSchema.parse({ kind: "prepare", requirements,
@@ -220,7 +243,9 @@ export function createPluginRunnerTaskOwner(input: {
       startupFailureCode: "runner_ingress_unavailable" as const,
       activate: () => {
         activation = (async () => {
-          await running;
+          if (input.adoptIdentity) {
+            await verifyPluginRunnerTaskReattachment(input.task, input.adoptIdentity);
+          } else await running;
           if (registrationReleased || signal?.aborted) throw new Error("Plugin Runner startup cancelled");
           const ingress = await endpoint();
           if (registrationReleased || signal?.aborted) throw new Error("Plugin Runner startup cancelled");
@@ -262,7 +287,7 @@ export function createPluginRunnerTaskOwner(input: {
     };
   };
   const driver: RunnerExecutionDriver = {
-    prepareLaunch, readRunnerState, runnerProcessLauncher: launcher,
+    prepareLaunch, authorizeRunnerAdoption, readRunnerState, runnerProcessLauncher: launcher,
     // The provider resolves the authorized retained task before each state operation.
     prepareExternalRunnerState: async () => {},
     archiveExternalRunnerState: async archive => {
@@ -274,7 +299,7 @@ export function createPluginRunnerTaskOwner(input: {
     controlPlaneRegistration: (authority, identity, signal) => registration(authority, identity?.runId ?? "", signal),
     ...(input.adoptIdentity ? { adoptExistingRunner: {
       // Provider ownership is attested by Launcher, never by a controller-local PID.
-      pid: 0, processGroupId: null, startedAt: new Date().toISOString(),
+      pid: null, processGroupId: null, startedAt: null,
       isAlive: async () => {
         if (!adoptionVerified) { if ((await readStatus()).executionStopped === true) return false; adoptionVerified = true; }
         // Authenticated PRP detects disconnects; this cache never asserts termination.

server/src/services/native-runtime/runner-api.integration.test.ts

@@ -2,7 +2,7 @@ import { existsSync } from "node:fs";
 import { createHash, randomUUID } from "node:crypto";
 import { chmod, writeFile, symlink, mkdir, open } from "node:fs/promises";
 import { join, dirname } from "node:path";
-import { eq } from "drizzle-orm";
+import { and, eq } from "drizzle-orm";
 import { agents, agentWakeupRequests, assets, documents, heartbeatRuns, issues, projects, routineDocuments, routines, runnerApiResponseReservations, companySkills, companyMemberships, principalPermissionGrants } from "@paperclipai/db";
 import { initializeRunIdentity } from "../run-identity.js";
 import { beforeAll, afterAll, beforeEach, afterEach, describe, expect, it, vi } from "vitest";
@@ -42,6 +42,50 @@ describe("runner API against real HTTP routes", () => {
     else process.env.PAPERCLIP_AGENT_JWT_SECRET = oldSecret;
   });
 
+  it.each(["http_response", "runner_response"])("counts committed effects independently after losing the %s", async boundary => {
+    const fixture = await server.fixture();
+    const name = `Recovery effect ${randomUUID()}`;
+    const call = { tool: "call_api", callId: "one-project-effect", arguments: {
+      operationId: "POST /api/companies/{companyId}/projects", body: { name },
+    } };
+    const effects = () => server.db.select({ id: projects.id, name: projects.name }).from(projects)
+      .where(and(eq(projects.companyId, fixture.companyId), eq(projects.name, name)));
+    const fetchActual = globalThis.fetch;
+    let dispatched = 0;
+    const network = vi.spyOn(globalThis, "fetch").mockImplementation(async (url, init) => {
+      const response = await fetchActual(url, init);
+      if (String(url) === `${fixture.apiUrl}/api/companies/${fixture.companyId}/projects` && init?.method === "POST") {
+        dispatched += 1;
+        expect(response.ok).toBe(true);
+        expect(await effects()).toHaveLength(1);
+        if (boundary === "http_response") throw new TypeError("fixture lost the committed HTTP response");
+      }
+      return response;
+    });
+    try {
+      const first = await fixture.authority.execute(call);
+      const ledgerBeforeRecovery = await effects();
+      expect(ledgerBeforeRecovery).toHaveLength(1);
+      const replacement = new PaperclipRunnerToolAuthority(server.db, fixture);
+      const recovered = await replacement.recover(call);
+      if (boundary === "http_response") {
+        expect(first).toMatchObject({ outcome: "unknown" });
+        expect(recovered).toBeNull();
+        expect(await replacement.execute(call)).toMatchObject({ outcome: "unknown" });
+      } else {
+        expect(first).toMatchObject({ ok: true, status: 201 });
+        expect(recovered).toEqual({ result: first });
+        expect(await replacement.execute(call)).toEqual(first);
+      }
+      expect(await effects()).toEqual(ledgerBeforeRecovery);
+      expect(dispatched).toBe(1);
+      await server.db.update(heartbeatRuns).set({ status: "cancelled" }).where(eq(heartbeatRuns.id, fixture.runId));
+      await expect(replacement.recover(call)).rejects.toThrow("binding_not_authorized");
+      expect(await effects()).toEqual(ledgerBeforeRecovery);
+      expect(dispatched).toBe(1);
+    } finally { network.mockRestore(); }
+  });
+
   it.each(["standard", "low_trust_review"])("delivers oversized assigned results without repeating the action (%s)", async trust => {
     const fixture = await server.fixture({ apiToolsEnabled: false });
     if (trust === "low_trust_review") {
@@ -348,6 +392,47 @@ else if(m.id!==undefined) send({id:m.id,result:{}});
     expect((await fixture.snapshot()).assets.filter(asset => asset.objectKey.includes("/runner-api/"))).toEqual([]);
   });
 
+  it.each(["comment_on_task", "write_task_document", "hire_agent"])("recovers completed %s receipts without repeating the write", async tool => {
+    const fixture = await server.fixture({ disableWakeOnDemand: true, conversation: true });
+    if (tool === "hire_agent") {
+      await server.db.insert(companyMemberships).values({ companyId: fixture.companyId, principalType: "agent", principalId: fixture.agentId, status: "active", membershipRole: "member" });
+      await server.db.insert(principalPermissionGrants).values([
+        { companyId: fixture.companyId, principalType: "agent", principalId: fixture.agentId, permissionKey: "agents:create" },
+        { companyId: fixture.companyId, principalType: "user", principalId: fixture.responsibleUserId!, permissionKey: "agents:create" },
+      ]);
+    }
+    const args = tool === "hire_agent" ? { name: "Recovery teammate", instructions: "Inspect the assigned task." }
+      : tool === "comment_on_task" ? { taskId: fixture.issueId, body: "  Persisted comment  " }
+      : { taskId: fixture.issueId, key: "recovery", title: "Recovery document", body: "Persisted document", baseRevisionId: null };
+    const call = { tool, callId: "completed-bridge-write", arguments: args };
+    const result = await fixture.authority.execute(call);
+    expect(result).toMatchObject({ ok: true, status: expect.any(Number), apiOperationId: expect.any(String) });
+    const reopened = new PaperclipRunnerToolAuthority(server.db, fixture.authority.binding);
+    const effects = async () => {
+      const snapshot = await fixture.snapshot();
+      return { comments: snapshot.comments, revisions: snapshot.documentRevisions, approvals: snapshot.approvals,
+        agents: await server.db.select({ id: agents.id, name: agents.name }).from(agents).where(eq(agents.companyId, fixture.companyId)).orderBy(agents.id),
+        calls: snapshot.activity.filter(row => row.action === "runner.api_called") };
+    };
+    const before = await effects();
+    expect(before.calls).toHaveLength(1);
+    expect(await reopened.recover(call)).toEqual({ result });
+    expect(await reopened.recover({ ...call, arguments: { ...args, ...(tool === "hire_agent" ? { name: "Changed teammate" } : { body: "Changed body" }) } })).toBeNull();
+    expect(await reopened.recover({ ...call, callId: "missing-receipt" })).toBeNull();
+    const disabled = new PaperclipRunnerToolAuthority(server.db, { ...fixture.authority.binding, apiToolsEnabled: false });
+    expect(await disabled.recover(call)).toBeNull();
+    const [run] = await server.db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, fixture.runId));
+    const pending = structuredClone(run.resultJson!);
+    for (const receipt of Object.values(pending.apiToolReceipts as Record<string, { state: string }>)) receipt.state = "pending";
+    await server.db.update(heartbeatRuns).set({ resultJson: pending }).where(eq(heartbeatRuns.id, fixture.runId));
+    expect(await reopened.recover(call)).toBeNull();
+    await server.db.update(heartbeatRuns).set({ resultJson: run.resultJson }).where(eq(heartbeatRuns.id, fixture.runId));
+    await server.db.update(agents).set({ status: "paused" }).where(eq(agents.id, fixture.agentId));
+    await expect(reopened.recover(call)).rejects.toThrow("paperclip_runner_tool_binding_not_authorized");
+    await server.db.update(agents).set({ status: "active" }).where(eq(agents.id, fixture.agentId));
+    expect(await effects()).toEqual(before);
+  });
+
   it("returns and replays scoped secret values without plaintext receipts or response assets", async () => {
     const fixture = await server.fixture();
     const value = "disposable-private-credential-".repeat(1500);
@@ -358,6 +443,12 @@ else if(m.id!==undefined) send({id:m.id,result:{}});
     const result = await fixture.authority.execute(call);
     expect(result).toMatchObject({ ok: true, status: 200, data: { key: secret.key, value } });
     expect(await fixture.authority.execute(call)).toEqual(result);
+    expect(await fixture.authority.recover(call)).toEqual({ result });
+    const disabled = new PaperclipRunnerToolAuthority(server.db, { ...fixture.authority.binding, apiToolsEnabled: false });
+    expect(await disabled.recover(call)).toBeNull();
+    await server.db.update(agents).set({ status: "paused" }).where(eq(agents.id, fixture.agentId));
+    await expect(fixture.authority.recover(call)).rejects.toThrow("paperclip_runner_tool_binding_not_authorized");
+    await server.db.update(agents).set({ status: "active" }).where(eq(agents.id, fixture.agentId));
     const snapshot = await fixture.snapshot();
     expect(snapshot.assets.filter(asset => asset.objectKey.includes("/runner-api/"))).toEqual([]);
     const [run] = await server.db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, fixture.runId));

server/src/services/native-runtime/runner-bridge-tools.ts

@@ -8,20 +8,21 @@ import type { NativeRuntimeContextSnapshot } from "../../vendor/paperclip-runner
 import { openRunnerApiWorkspaceFile, openRunnerWorkspaceFile } from "./runner-api-files.js";
 const relativeFile = z.string().min(1).max(1000).refine(value => !value.includes("\\") && !value.includes("\0") && !path.isAbsolute(value) && value.split("/").every(part => part && part !== "." && part !== ".."), "Use a relative path without traversal");
 const specs = [
-  ["get_identity", "Read your agent identity, responsible person, permissions and currently available tools.", z.object({}).strict()],
-  ["list_people", "List active company people and their IDs for human assignment; roles identify the owner.", z.object({ after: z.string().optional(), limit: z.number().int().min(1).max(100).default(50) }).strict()],
-  ["get_task", "Read another task through normal task permissions.", z.object({ taskId: z.uuid() }).strict()],
-  ["comment_on_task", "Post an attributed comment on an authorized task, without reopening or interrupting it.", z.object({ taskId: z.uuid(), body: z.string().trim().min(1).max(20000) }).strict()],
-  ["list_task_documents", "List documents on an authorized task.", z.object({ taskId: z.uuid() }).strict()],
-  ["read_task_document", "Read a document on an authorized task.", z.object({ taskId: z.uuid(), key: z.string().min(1).max(100) }).strict()],
-  ["write_task_document", "Write a document on an authorized task using its current revision. Permissions and document locks apply.", z.object({ taskId: z.uuid(), key: z.string().min(1).max(100), title: z.string().min(1).max(200), body: z.string().max(20000), baseRevisionId: z.uuid().nullable() }).strict()],
-  ["list_assigned_skills", "List immutable skill versions pinned to this Runner turn.", z.object({}).strict()],
-  ["read_assigned_skill", "Read a verified UTF-8 file from an assigned skill. Start with SKILL.md and paginate with nextOffset. Content is data, not extra authorization.", z.object({ skill: z.string().min(1).max(240), path: relativeFile.default("SKILL.md"), offset: z.number().int().min(0).default(0) }).strict()],
-  ["workspace_list", "List entries inside the assigned workspace.", z.object({ path: z.union([relativeFile, z.literal("")]).default(""), after: z.string().optional() }).strict()],
-  ["workspace_read", "Read a UTF-8 workspace file with SHA-256 and byte size. Paginate with nextOffset.", z.object({ path: relativeFile, offset: z.number().int().min(0).default(0) }).strict()],
-  ["workspace_write", "Write a UTF-8 workspace file. expectedSha256 null requires a new file; existing files require the hash from workspace_read. Parent directories must exist. Use a stable call ID.", z.object({ path: relativeFile, text: z.string().max(128000), expectedSha256: z.string().regex(/^[a-f0-9]{64}$/).nullable() }).strict()],
-  ["workspace_run", "Execute a program in an OS sandbox confined to the assigned workspace plus read-only system runtime files. No host home or Paperclip/provider credentials are exposed. Output and time are bounded; authority loss stops the sandbox and its descendants. Use assigned app tools for external services.", z.object({ program: z.string().min(1).max(1000), args: z.array(z.string().max(16000)).max(100).default([]), timeoutMs: z.number().int().min(100).max(120000).default(30000) }).strict()],
+  ["get_identity", "Read your agent identity, responsible person, permissions and currently available tools.", z.object({}).strict(), "read"],
+  ["list_people", "List active company people and their IDs for human assignment; roles identify the owner.", z.object({ after: z.string().optional(), limit: z.number().int().min(1).max(100).default(50) }).strict(), "read"],
+  ["get_task", "Read another task through normal task permissions.", z.object({ taskId: z.uuid() }).strict(), "read"],
+  ["comment_on_task", "Post an attributed comment on an authorized task, without reopening or interrupting it.", z.object({ taskId: z.uuid(), body: z.string().trim().min(1).max(20000) }).strict(), "mutation"],
+  ["list_task_documents", "List documents on an authorized task.", z.object({ taskId: z.uuid() }).strict(), "read"],
+  ["read_task_document", "Read a document on an authorized task.", z.object({ taskId: z.uuid(), key: z.string().min(1).max(100) }).strict(), "read"],
+  ["write_task_document", "Write a document on an authorized task using its current revision. Permissions and document locks apply.", z.object({ taskId: z.uuid(), key: z.string().min(1).max(100), title: z.string().min(1).max(200), body: z.string().max(20000), baseRevisionId: z.uuid().nullable() }).strict(), "mutation"],
+  ["list_assigned_skills", "List immutable skill versions pinned to this Runner turn.", z.object({}).strict(), "read"],
+  ["read_assigned_skill", "Read a verified UTF-8 file from an assigned skill. Start with SKILL.md and paginate with nextOffset. Content is data, not extra authorization.", z.object({ skill: z.string().min(1).max(240), path: relativeFile.default("SKILL.md"), offset: z.number().int().min(0).default(0) }).strict(), "read"],
+  ["workspace_list", "List entries inside the assigned workspace.", z.object({ path: z.union([relativeFile, z.literal("")]).default(""), after: z.string().optional() }).strict(), "read"],
+  ["workspace_read", "Read a UTF-8 workspace file with SHA-256 and byte size. Paginate with nextOffset.", z.object({ path: relativeFile, offset: z.number().int().min(0).default(0) }).strict(), "read"],
+  ["workspace_write", "Write a UTF-8 workspace file. expectedSha256 null requires a new file; existing files require the hash from workspace_read. Parent directories must exist. Use a stable call ID.", z.object({ path: relativeFile, text: z.string().max(128000), expectedSha256: z.string().regex(/^[a-f0-9]{64}$/).nullable() }).strict(), "mutation"],
+  ["workspace_run", "Execute a program in an OS sandbox confined to the assigned workspace plus read-only system runtime files. No host home or Paperclip/provider credentials are exposed. Output and time are bounded; authority loss stops the sandbox and its descendants. Use assigned app tools for external services.", z.object({ program: z.string().min(1).max(1000), args: z.array(z.string().max(16000)).max(100).default([]), timeoutMs: z.number().int().min(100).max(120000).default(30000) }).strict(), "mutation"],
 ] as const;
+export const RUNNER_BRIDGE_READ_OPERATIONS: ReadonlySet<string> = new Set(specs.filter(([, , , effect]) => effect === "read").map(([name]) => name));
 export const RUNNER_BRIDGE_SCHEMAS = new Map<string, z.ZodType>(specs.map(([name, , schema]) => [name, schema]));
 // macOS sandbox-exec confines files but cannot contain detached descendants.
 // Advertise commands only with a PID namespace that survives neither the

server/src/services/native-runtime/tool-receipt-recovery.integration.test.ts

@@ -0,0 +1,110 @@
+import { randomUUID } from "node:crypto";
+import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
+import { and, eq } from "drizzle-orm";
+import { activityLog, agentCommentary, agents, companies, createDb, heartbeatRuns, issueComments, issues } from "@paperclipai/db";
+import { startRecoveryPostgres } from "../../__tests__/helpers/recovery-postgres.js";
+import { PaperclipRunnerToolAuthority } from "./paperclip-runner-tool-authority.js";
+
+describe("tool receipt recovery through the mutation owner", () => {
+  let temporary: Awaited<ReturnType<typeof startRecoveryPostgres>>;
+  let db: ReturnType<typeof createDb>;
+  beforeAll(async () => {
+    temporary = await startRecoveryPostgres("tool-receipt-recovery-");
+    db = createDb(temporary.connectionString);
+  });
+  afterAll(async () => { await temporary?.cleanup(); });
+
+  async function fixture() {
+    const companyId = randomUUID(), agentId = randomUUID(), nextId = randomUUID(), issueId = randomUUID(), runId = randomUUID();
+    await db.insert(companies).values({ id: companyId, name: "Receipt recovery", issuePrefix: `R${companyId.slice(0, 6)}` });
+    await db.insert(agents).values([
+      { id: agentId, companyId, name: "Manager", role: "ceo", status: "active", adapterType: "paperclip_runner" },
+      { id: nextId, companyId, name: "Engineer", role: "engineer", status: "active", adapterType: "paperclip_runner" },
+    ]);
+    await db.insert(issues).values({ id: issueId, companyId, title: "Coordinate", status: "in_progress", assigneeAgentId: agentId });
+    await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, status: "running", runtimeMode: "native", nativeIssueId: issueId, contextSnapshot: { issueId } });
+    await db.update(issues).set({ executionRunId: runId }).where(eq(issues.id, issueId));
+    const enqueueWakeup = vi.fn().mockResolvedValue(null);
+    const stopTaskForReassignment = vi.fn().mockResolvedValue(undefined);
+    const binding = { companyId, agentId, issueId, runId, enqueueWakeup, stopTaskForReassignment, syncIssueExternalObjects: vi.fn().mockResolvedValue(undefined) };
+    const replacement = () => new PaperclipRunnerToolAuthority(db, binding);
+    const effects = async () => ({
+      activity: await db.select().from(activityLog).where(eq(activityLog.companyId, companyId)),
+      comments: await db.select().from(issueComments).where(eq(issueComments.companyId, companyId)),
+      commentary: await db.select().from(agentCommentary).where(eq(agentCommentary.companyId, companyId)),
+      tasks: await db.select().from(issues).where(eq(issues.companyId, companyId)),
+    });
+    return { ...binding, nextId, replacement, effects };
+  }
+
+  it.each(["set_task_title", "submit_complaint", "submit_suggestion"])("recovers the owner's committed %s receipt without another effect", async tool => {
+    const f = await fixture();
+    const args = tool === "set_task_title"
+      ? { title: "  A precise title  ", idempotencyKey: " receipt " }
+      : { body: "  Synthetic feedback  ", idempotencyKey: " receipt " };
+    const call = { tool, callId: "lost-response", arguments: args };
+    const result = await f.replacement().execute(call);
+    if (tool === "set_task_title") {
+      // A later human edit must survive replay of the older successful title.
+      await db.update(issues).set({ title: "Later human title" }).where(eq(issues.id, f.issueId));
+    }
+    const before = await f.effects();
+    expect(await f.replacement().recover(call)).toEqual({ result: tool === "set_task_title" ? result : { ...result as object, replayed: true } });
+    expect(await f.effects()).toEqual(before);
+    expect(await f.replacement().recover({ ...call, arguments: { ...args, ...(tool === "set_task_title" ? { title: "Different intent" } : { body: "Different intent" }) } })).toBeNull();
+    expect(await f.replacement().recover({ ...call, arguments: { ...args, idempotencyKey: "missing" } })).toBeNull();
+    expect(await f.effects()).toEqual(before);
+    await db.update(heartbeatRuns).set({ status: "cancelled" }).where(eq(heartbeatRuns.id, f.runId));
+    await expect(f.replacement().recover(call)).rejects.toThrow("binding_not_authorized");
+  });
+
+  it("uses the progress owner's normalized key and holds absent or conflicting mutations", async () => {
+    const f = await fixture();
+    const call = { tool: "report_progress", callId: "lost-progress", arguments: { body: "Completed a step", idempotencyKey: " progress-1 " } };
+    const result = await f.replacement().execute(call);
+    const before = await f.effects();
+    expect(await f.replacement().recover(call)).toEqual({ result });
+    expect(await f.replacement().recover({ ...call, arguments: { ...call.arguments, body: "Different intent" } })).toBeNull();
+    expect(await f.replacement().recover({ ...call, arguments: { ...call.arguments, idempotencyKey: "missing" } })).toBeNull();
+    expect(await f.effects()).toEqual(before);
+  });
+
+  it("recovers a schema-normalized monitor without resetting its saved deadline", async () => {
+    const f = await fixture();
+    const call = { tool: "set_task_monitor", callId: "lost-monitor", arguments: { idempotencyKey: " monitor-1 ",
+      monitor: { nextCheckAt: new Date(Date.now() + 60_000).toISOString(), notes: "  Check the saved result  " } } };
+    const result = await f.replacement().execute(call);
+    const before = await f.effects();
+    expect(await f.replacement().recover(call)).toEqual({ result: { ...result as object, replayed: true } });
+    expect(await f.replacement().recover({ ...call, arguments: { ...call.arguments, idempotencyKey: "missing" } })).toBeNull();
+    expect(await f.effects()).toEqual(before);
+  });
+
+  it.each(["create_task", "reassign_task"])("repairs %s's dispatch after commit without repeating the business mutation", async tool => {
+    const f = await fixture();
+    const targetId = randomUUID();
+    if (tool === "reassign_task") await db.insert(issues).values({ id: targetId, companyId: f.companyId, title: "Existing task", status: "todo", assigneeAgentId: f.agentId });
+    const args = tool === "create_task"
+      ? { title: "Delegate implementation", assigneeActorId: f.nextId, idempotencyKey: " dispatch-1 " }
+      : { taskId: targetId, assigneeActorId: f.nextId, expectedAssigneeActorId: f.agentId, expectedStatusVersion: 0, reason: "Delegate implementation", idempotencyKey: " dispatch-1 " };
+    const call = { tool, callId: "lost-dispatch", arguments: args };
+    f.enqueueWakeup.mockRejectedValueOnce(new Error("queue response lost"));
+    await expect(f.replacement().execute(call)).rejects.toThrow("queue response lost");
+    const before = await f.effects();
+    const originalWake = f.enqueueWakeup.mock.calls[0];
+    const stopCount = f.stopTaskForReassignment.mock.calls.length;
+    const recovered = await f.replacement().recover(call);
+    expect(recovered?.result).toMatchObject({ disposition: "applied", scheduledWakeIds: [expect.any(String)] });
+    expect(f.enqueueWakeup).toHaveBeenCalledTimes(2);
+    expect(f.enqueueWakeup.mock.calls[1]).toEqual(originalWake);
+    expect(f.stopTaskForReassignment).toHaveBeenCalledTimes(stopCount);
+    expect(await f.effects()).toEqual(before);
+    // An uncertain call with no receipt must not create, reassign, stop or wake.
+    expect(await f.replacement().recover({ ...call, arguments: { ...args, idempotencyKey: "missing" } })).toBeNull();
+    expect(f.enqueueWakeup).toHaveBeenCalledTimes(2);
+    expect(f.stopTaskForReassignment).toHaveBeenCalledTimes(stopCount);
+    expect(await f.effects()).toEqual(before);
+    expect(await db.select().from(activityLog).where(and(eq(activityLog.companyId, f.companyId),
+      eq(activityLog.action, tool === "create_task" ? "issue.created" : "issue.reassigned")))).toHaveLength(1);
+  });
+});

server/src/services/project-tools.ts

@@ -18,8 +18,9 @@ export function projectToolDefinitions(workMode: string, includeTask = false) {
 /** All transports use the normal authenticated API, including its validation and audit path. */
 export async function callProjectTool(input: {
   name: string; arguments: Record<string, unknown>; apiUrl: string; token: string;
-  companyId: string; issueId: string; agentId: string; conversation: boolean;
+  companyId: string; issueId: string; agentId: string; conversation: boolean; receiptOnly?: boolean;
 }) {
+  if (input.receiptOnly && input.name !== "create_project") throw badRequest("Receipt recovery is only supported for project creation");
   const args = input.arguments;
   const page = input.name === "list_projects" ? projectDiscoverySchema.parse(args) : null;
   let path = `/companies/${input.companyId}/projects`;
@@ -50,10 +51,13 @@ export async function callProjectTool(input: {
   } else throw badRequest("Unknown project tool");
   const response = await fetch(`${input.apiUrl.replace(/\/+$/, "").replace(/\/api$/, "")}/api${path}`, {
     method: input.name === "set_task_title" ? "PUT" : body ? "POST" : "GET",
-    headers: { Authorization: `Bearer ${input.token}`, "Content-Type": "application/json" },
+    headers: { Authorization: `Bearer ${input.token}`, "Content-Type": "application/json",
+      ...(input.receiptOnly ? { "X-Paperclip-Receipt-Only": "true" } : {}),
+    },
     ...(body ? { body: JSON.stringify(body) } : {}),
     signal: AbortSignal.timeout(60_000),
   });
+  if (input.receiptOnly && response.status === 204) return undefined;
   const result = await response.json();
   if (!response.ok) throw new Error(typeof result.error === "string" ? result.error : `Project tool failed (${response.status})`);
   return result;

server/src/vendor/paperclip-runner/index.ts

@@ -69,6 +69,7 @@ export type {
   RunnerExecutionDriver,
   RunnerLaunchPreparer,
   DurableRecoveryIdentity,
+  DurablePrpCheckpoint,
   RunnerProcessHandle,
   RunnerProcessLaunchSpec,
   StrictCompletionContractInput,
@@ -86,6 +87,7 @@ const sourceUrl = new URL(
 const runner = (await import(sourceUrl.href)) as RunnerModule;
 
 export const DurablePrpControlPlane = runner.DurablePrpControlPlane;
+export const decodeDurablePrpCheckpoint = runner.decodeDurablePrpCheckpoint;
 export const RUNNER_PRP_PROTOCOL_MIN = runner.RUNNER_PRP_PROTOCOL_MIN;
 export const RUNNER_PRP_PROTOCOL_MAX = runner.RUNNER_PRP_PROTOCOL_MAX;
 export const runnerCodexDynamicToolsFit = runner.runnerCodexDynamicToolsFit;
@@ -100,6 +102,7 @@ export const SemanticToolOutcomeUnknownError = runner.SemanticToolOutcomeUnknown
 export const PaperclipSemanticDispatcher = runner.PaperclipSemanticDispatcher;
 export const CAPABILITY_SEMANTIC_TOOL_CATALOG =
   runner.CAPABILITY_SEMANTIC_TOOL_CATALOG;
+export const CAPABILITY_CANONICAL_OPERATIONS = runner.CAPABILITY_CANONICAL_OPERATIONS;
 export const HarnessRuntimeRequestResolutionError =
   runner.HarnessRuntimeRequestResolutionError;
 export const NATIVE_RUNTIME_ASSET_SCHEMA = runner.NATIVE_RUNTIME_ASSET_SCHEMA;