fix(recovery): settle a run-less lease once its stamped holder is proven gone (#20026)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The control plane recovers in-progress records after a server
restart through a shared ownership path
> - Run-less sandbox leases (login, probe, harness verification) were
released only by a five-minute idle sweep
> - That sweep releases by age: it leaves a dead holder's sandbox for
minutes and takes a live login session's sandbox
> - This pull request settles a run-less lease only when its stamped
holder is proven gone
> - The benefit is no stuck sandbox after a restart, and no live session
loses its sandbox

## Linked Issues or Issue Description

Related: #19965 (named the owner of login/probe leases), #19964 (the
run-lease settle this mirrors). No duplicate PR was found.

**What happened?**

The final Daytona chaos wave `w-20261108T083036-s9401` recorded I2
`environment_login_probe_lease` (`pid_dead`) on 2 records. These were
the codex agent's harness-verification leases. A restart killed their
holder. The leases stayed `active` until the five-minute idle sweep
released them. The same sweep releases any run-less lease after five
idle minutes, also when its holder is alive.

**Expected behavior**

A run-less lease is released as soon as its stamped holder is proven
gone. A lease whose holder is alive is never released, however long it
is idle.

**Steps to reproduce**

1. Start a Daytona wave: `node cli/node_modules/tsx/dist/cli.mjs
tests/restart-chaos/main.ts --env daytona --daytona-image <image>`
(K1/K4 faults).
2. Let a fault kill the server while an agent's harness verification
holds its sandbox lease.
3. See the reconciler report I2 `environment_login_probe_lease` until
the idle sweep releases it.

**Paperclip version or commit**

`master` at `8475826d2d`.

## What Changed

- New `settleEnvironmentLoginProbeLease`, registered on
`environment_login_probe_lease`. It runs only for a stamped holder that
is proven gone, and compares and swaps on that exact stamp.
- It moves the lease to `pending_cleanup` with the recovered-orphan
reason, so the cleanup sweep destroys the sandbox in its next tick.
- It leaves the lease alone when a legacy shutdown fence applies, or
when another live lease holds the same provider resource.
- The idle sweep for run-less leases now covers only legacy rows saved
before holders were stamped (no `leaseController` key). Stamped rows are
never released by age.
- `holderUnchanged` is now exported from the run-lease settle so both
settles share it.
- The controller stamp (`leaseController`, and every other
`ControllerProcess` stamp) now records the platform deployment when the
platform names one (Railway: project, environment, service, deployment,
volume). `controllerStamp` passes it to `proveOwnerGone`. A container
redeploy changes the PID space, and the old stamp would read
`different_process_space` (unprovable) forever. Now the existing
`deployment_replaced` proof shows the old holder gone. This also covers
`environment_run_lease_active` and the other kinds built on
`controllerStamp`.
- `currentDeploymentIdentity` in `owner-proof.ts` now delegates to
`currentControllerDeployment`, so there is one parser.
- Data doc: added the final Daytona wave, the six Daytona-only crash
points (4 pass, 2 not reached, no violations), and the cause of the
roughly 450 s codex TTFT. Runs waited behind the agent's harness
verification, which restarted after each fault; boot itself took about
32 s.
- Restart-chaos README: corrected which Daytona crash points a plain
task reaches.

## Verification

- Red, before the change:
`environment-login-probe-lease-ownership.test.ts` failed 3 of 6. No
settle happened, and the idle sweep released 3 live holders' leases.
- Green: `pnpm exec vitest run` passes for
`environment-login-probe-lease-ownership`,
`environment-run-lease-ownership` and
`heartbeat-orphaned-active-lease-sweep` (45/45), and for
`environment-lease-owner-stamps`,
`heartbeat-observed-workspace-preservation`,
`ownership-reconciliation-report`, `heartbeat/recovery` and
`workspace-recovery-warm-handoff`.
- Redeploy: a stamp from a replaced Railway deployment settles with
`deployment_replaced`. A stamp from the same deployment in another PID
space stays `active`. This was red when `controllerStamp` dropped the
deployment.
- Live-owner counter-cases: the current process, another live PID,
another PID space, and a lease re-stamped after the proof all stay
`active`. A read-only pass never settles.
- Not re-run live: the Daytona wave and crash points ran before this
fix. The next Daytona wave should show no
`environment_login_probe_lease` I2.

## Risks

- A stamped run-less lease whose live holder never releases it, because
of a bug in that process, is no longer released after five idle minutes.
It is released once that process exits. This is the intended trade:
never take a live holder's sandbox.
- On an ephemeral server without a platform deployment identity (not
Railway), a stamp from a replaced container is still unprovable. Such a
run-less lease stays `active` until the provider expires the sandbox,
and the reconciler reports it as unprovable. Before this PR, the
five-minute sweep released it by age. This is the same open gap as
unstamped legacy leases on an ephemeral server: there is no operator
action yet. It is listed as a Phase 2 follow-up.
- Legacy unstamped run-less leases keep the age fallback. These are rows
written before #19965, and no current code writes them.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code with
tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

doc/plans/2026-11-07-phase2-reconciler-data.md

@@ -173,6 +173,44 @@ Speed: remote environments have no budget yet (#19863). These are the first remo
 
 A Daytona budget needs a dedicated remote baseline. Restart recovery is 5–10× the local values.
 
+## Final Daytona wave and Daytona-only crash points (2026-11-08)
+
+**Wave `w-20261108T083036-s9401`.** Setup:
+- controller at master `8475826d2d`;
+- signed `image_only` image `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:c24059f1…` (content ID `abc82e78…`, revision `838e9bf3d2`);
+- 6 tasks;
+- faults: K1, K4 (167 s hold, 175 s outage), K2, K1, K1, K4 (280 s hold, 287 s outage).
+
+All 6 tasks crossed a fault and all 6 passed. The sandbox sweep was clean, and no run breached a budget.
+
+The earlier attempt, `w-20261108T082021`, never started. A skopeo blob read timed out during image extraction, and the retry refused the half-written directory. This is a harness problem, fixed in #20021.
+
+**One real violation: I2 `environment_login_probe_lease` (`pid_dead`), 2 records.**
+- These are run-less leases. The codex agent's harness verification acquires one before any run is admitted.
+- A restart killed the process holding each lease. The lease stayed `active` until the five-minute idle sweep released it.
+- That same idle sweep also releases a live login session's sandbox after five idle minutes.
+- The fix is in PR fix/login-probe-lease-settle. The kind gets a settle that moves the lease to cleanup only once its stamped holder is proven gone. The idle sweep now covers only legacy rows saved before holders were stamped.
+
+**Speed.**
+- Restart recovery from `ready` to the next provider event: claude p50 23.9 s, codex 4.6 s.
+- Codex time from comment to first token on runs that crossed no fault was about 450 s, but boot was about 32 s.
+- The wait came before any run sandbox existed. The tasks launched at 09:55:46, while the first run sandboxes appeared at 10:02:43 (417 s later).
+- In between, the codex agent's harness verification ran in a fresh Daytona sandbox four times (09:55:46, 09:56:27, 09:59:43, 10:02:13). Each fault restarted it, and runs stay queued until verification completes.
+- This is lifecycle admission, not run boot. In a fault-free product it costs one verification per newly hired agent: one sandbox boot plus a hello probe (for comparison, the seed environment probe took 68 s).
+
+**Crash points `cp-20261108T104102`** (Daytona, codex A1). The run took 8.7 min; it reused the cached image.
+
+| Point | Outcome |
+|---|---|
+| `sandbox.create.after_intent_before_provider` | pass, no violations |
+| `sandbox.create.after_provider_before_lease` | pass, no violations |
+| `environment.lease_release.after_provider_before_lease` | not reached: a plain native Daytona run now releases through teardown (the two teardown points below), not this path |
+| `environment.teardown.after_claim_before_destroy` | pass, no violations |
+| `environment.teardown.after_destroy_before_lease` | pass, no violations |
+| `environment.stop_retain.after_claim_before_stop` | not reached: needs a stop-and-retain intent (a native workspace export resume), which no harness workload produces yet |
+
+The R6 sweep found no orphaned sandboxes and no held leases. The first crash-point attempt, `cp-20261108T101304`, exited 2 before any point ran. Cause: the same skopeo timeout, plus a per-run image cache that downloaded the image again. #20021 fixes both.
+
 ## Step 2: kinds moved to the shared path
 
 Order of the moves, highest risk first:
@@ -197,6 +235,7 @@ Phase 3 is not started. It needs a separate go-ahead.
 
 - **Daytona** is covered by one wave only (see "Daytona wave"). The first apparent content-ID mismatch was a misreading: CI run 42422571919 built target commit `a37a2e4`, not master. Local content IDs reproduce CI's for the same commit.
 - The remote, workspace-operation, sandbox and accounting kinds (see "Per-kind data").
+- **Follow-up: `environment.stop_retain.after_claim_before_stop` has no live coverage.** No chaos workload creates a stop-and-retain intent (a native workspace export resume on a sandbox), so the crash point is never reached. A workload that forces an export resume on Daytona would cover it. `environment.lease_release.after_provider_before_lease` is also unreached by native runs, which release through teardown.
 - Crash points that sit between two durable writes. Random waves hit 20 ms–1 s windows only by luck. The crash-point suite (Phase 5) is the tool for those.
 
 ## Reproduce

server/src/__tests__/environment-login-probe-lease-ownership.test.ts

@@ -0,0 +1,143 @@
+import { spawn } from "node:child_process";
+import { randomUUID } from "node:crypto";
+import { eq } from "drizzle-orm";
+import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
+import { companies, createDb, environmentLeases, environments } from "@paperclipai/db";
+import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
+import { currentControllerProcess, currentControllerProcessStamp, currentProcessSpaceHash, readControllerProcess } from "../services/controller-process-identity.js";
+import { ENVIRONMENT_LEASE_CONTROLLER_KEY } from "../services/environment-lease-controller.js";
+import { nextEnvironmentCleanupAt } from "../services/environment-cleanup-state.js";
+import { settleEnvironmentLoginProbeLease } from "../services/environment-login-probe-lease-settlement.js";
+import { heartbeatService } from "../services/heartbeat.js";
+import { OWNABLE_KINDS } from "../services/recovery/ownership-registry.js";
+import { createOwnershipReportState, runOwnershipReconciliationPass } from "../services/recovery/ownership-invariant-report.js";
+
+const support = await getEmbeddedPostgresTestSupport();
+const suite = support.supported ? describe : describe.skip;
+const KIND = "environment_login_probe_lease";
+
+async function exitedPid(): Promise<number> {
+  const child = spawn(process.execPath, ["-e", "setInterval(() => {}, 1000)"], { stdio: "ignore" });
+  const pid = child.pid!;
+  await new Promise<void>(resolve => { child.once("exit", () => resolve()); child.kill("SIGKILL"); });
+  return pid;
+}
+
+suite("environment_login_probe_lease on the shared ownership path", () => {
+  let db!: ReturnType<typeof createDb>;
+  let temporary: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
+  let companyId = "";
+  let environmentId = "";
+
+  beforeAll(async () => {
+    temporary = await startEmbeddedPostgresTestDatabase("paperclip-login-probe-lease-ownership-");
+    db = createDb(temporary.connectionString);
+    companyId = randomUUID();
+    environmentId = randomUUID();
+    await db.insert(companies).values({ id: companyId, name: "Paperclip", issuePrefix: "LPL",
+      requireBoardApprovalForNewAgents: false });
+    await db.insert(environments).values({ id: environmentId, name: "Fake Sandbox", driver: "sandbox",
+      status: "active", config: { provider: "fake", image: "ubuntu:24.04" } });
+  }, 30_000);
+
+  afterEach(async () => { vi.unstubAllEnvs(); await db.delete(environmentLeases); });
+  afterAll(async () => { await temporary?.cleanup(); });
+
+  async function lease(holder: Record<string, unknown> | null, providerLeaseId?: string) {
+    const id = randomUUID();
+    await db.insert(environmentLeases).values({ id, companyId, environmentId, heartbeatRunId: null, status: "active",
+      leasePolicy: "ephemeral", provider: "fake", providerLeaseId: providerLeaseId ?? `sandbox://fake/${id}`,
+      updatedAt: new Date(Date.now() - 60 * 60_000),
+      metadata: holder ? { [ENVIRONMENT_LEASE_CONTROLLER_KEY]: holder } : {} });
+    return id;
+  }
+  async function row(id: string) {
+    const [found] = await db.select().from(environmentLeases).where(eq(environmentLeases.id, id));
+    return found!;
+  }
+  async function deadHolder() {
+    return { bootId: "earlier-boot", pid: await exitedPid(), hostHash: currentProcessSpaceHash(), processStartedAt: null };
+  }
+  const pass = (record = true) => runOwnershipReconciliationPass(db, { state: createOwnershipReportState(), record });
+
+  it("moves a dead holder's lease to cleanup on the next pass, due at once", async () => {
+    const leaseId = await lease(await deadHolder());
+    const report = await pass();
+    expect(report.kinds.find(kind => kind.kind === KIND)).toMatchObject({ settled: 1, reasons: { pid_dead: 1 } });
+    const after = await row(leaseId);
+    expect(after).toMatchObject({ status: "pending_cleanup", failureReason: "orphaned_active_lease_recovered" });
+    expect(after.metadata?.recoveredRunlessLease).toEqual({ ownerProof: "pid_dead" });
+    expect(await nextEnvironmentCleanupAt(db)).toBeLessThanOrEqual(Date.now());
+    expect(report.newViolations.filter(violation => violation.kind === KIND)).toEqual([]);
+  });
+
+  it("never takes a live holder's lease, however long it has been idle", async () => {
+    const self = await currentControllerProcess();
+    const alive = await lease({ ...self, bootId: "another-boot" });
+    const mine = await lease(currentControllerProcessStamp());
+    const foreign = await lease({ bootId: "other", pid: 1, hostHash: "another-pid-space", processStartedAt: null });
+    const report = await pass();
+    expect(report.kinds.find(kind => kind.kind === KIND)).toMatchObject({ settled: 0 });
+    expect(await heartbeatService(db).sweepOrphanedActiveLeases({ backoffMs: 0 })).toEqual({ recovered: 0 });
+    for (const id of [alive, mine, foreign]) expect((await row(id)).status).toBe("active");
+  });
+
+  it("never settles on a read-only pass", async () => {
+    const leaseId = await lease(await deadHolder());
+    const report = await pass(false);
+    expect(report.kinds.find(kind => kind.kind === KIND)).toMatchObject({ settled: 0, verdicts: { gone: 1 } });
+    expect((await row(leaseId)).status).toBe("active");
+  });
+
+  it("does not settle a lease re-stamped by a new holder after the proof", async () => {
+    const leaseId = await lease(await deadHolder());
+    const { records } = await OWNABLE_KINDS.find(kind => kind.id === KIND)!.listInProgress!(db, 10, 0);
+    const record = records.find(candidate => candidate.recordId === leaseId)!;
+    await db.update(environmentLeases).set({ metadata: { [ENVIRONMENT_LEASE_CONTROLLER_KEY]: currentControllerProcessStamp() } })
+      .where(eq(environmentLeases.id, leaseId));
+    expect(await settleEnvironmentLoginProbeLease(db, record, { now: new Date(), proof: { verdict: "gone", reason: "pid_dead" } }))
+      .toEqual({ settled: false, reason: "owner_changed_since_proof" });
+    expect((await row(leaseId)).status).toBe("active");
+  });
+
+  it("keeps a lease whose provider resource another live lease still holds", async () => {
+    const shared = "sandbox://fake/shared";
+    const orphan = await lease(await deadHolder(), shared);
+    await lease(currentControllerProcessStamp(), shared);
+    await pass();
+    expect((await row(orphan)).status).toBe("active");
+  });
+
+  it("still recovers a legacy lease saved before holders were stamped, after the idle backoff", async () => {
+    const legacy = await lease(null);
+    const report = await pass();
+    expect(report.kinds.find(kind => kind.kind === KIND)).toMatchObject({ settled: 0 });
+    expect(await heartbeatService(db).sweepOrphanedActiveLeases({ backoffMs: 5 * 60_000 })).toEqual({ recovered: 1 });
+    expect((await row(legacy)).status).toBe("pending_cleanup");
+  });
+
+  function railway(deploymentId: string) {
+    vi.stubEnv("RAILWAY_PROJECT_ID", "project"); vi.stubEnv("RAILWAY_ENVIRONMENT_ID", "production");
+    vi.stubEnv("RAILWAY_SERVICE_ID", "server"); vi.stubEnv("RAILWAY_DEPLOYMENT_ID", deploymentId);
+    vi.stubEnv("RAILWAY_VOLUME_ID", "volume"); vi.stubEnv("RAILWAY_VOLUME_MOUNT_PATH", "/data");
+  }
+
+  it("stamps the platform deployment so a redeploy can prove the old holder gone", async () => {
+    railway("deployment-old");
+    const stamp = currentControllerProcessStamp();
+    expect(stamp.deployment).toEqual({ worker: { provider: "railway", projectId: "project", environmentId: "production",
+      serviceId: "server", deploymentId: "deployment-old" }, exclusiveVolume: true, volumeId: "volume" });
+    expect(readControllerProcess(JSON.parse(JSON.stringify(stamp)))).toEqual(stamp);
+    const replaced = await lease({ ...stamp, bootId: "old-boot", hostHash: "old-container", pid: 7 });
+    const sameDeployment = await lease({ ...stamp, bootId: "sibling-boot", hostHash: "sibling-container", pid: 8 });
+    await db.update(environmentLeases).set({ metadata: { leaseController: { ...stamp, bootId: "sibling-boot", hostHash: "sibling-container", pid: 8,
+      deployment: { ...stamp.deployment, worker: { ...stamp.deployment!.worker, deploymentId: "deployment-new" } } } } })
+      .where(eq(environmentLeases.id, sameDeployment));
+    railway("deployment-new");
+    const report = await pass();
+    expect(report.kinds.find(kind => kind.kind === KIND)).toMatchObject({ settled: 1,
+      reasons: { deployment_replaced: 1, different_process_space: 1 } });
+    expect((await row(replaced)).status).toBe("pending_cleanup");
+    expect((await row(sameDeployment)).status).toBe("active");
+  });
+});

server/src/__tests__/ownership-reconciliation-report.test.ts

@@ -143,7 +143,7 @@ describe("report-only ownership reconciliation", () => {
     expect(new Set(names).size).toBe(names.length);
     expect(names.length).toBeGreaterThanOrEqual(40);
     expect(OWNABLE_KINDS.filter(kind => kind.settle !== null).map(kind => kind.id))
-      .toEqual(["native_accepted_result_lease", "native_finalizer_lease", "native_cleanup_lease", "sandbox_creation_intent", "environment_run_lease_active", "environment_teardown_claim", "environment_retained_orphan_sandbox", "workspace_runtime_control_operation", "native_workspace_finalize_attempt", "execution_workspace_runtime_lease", "legacy_shutdown_settlement_fence", "copyback_owner_hold", "observed_workspace_preservation_fence", "wake_request_claimed", "legacy_controller_lease", "issue_execution_lock", "issue_checkout_lock"]);
+      .toEqual(["native_accepted_result_lease", "native_finalizer_lease", "native_cleanup_lease", "sandbox_creation_intent", "environment_run_lease_active", "environment_teardown_claim", "environment_retained_orphan_sandbox", "environment_login_probe_lease", "workspace_runtime_control_operation", "native_workspace_finalize_attempt", "execution_workspace_runtime_lease", "legacy_shutdown_settlement_fence", "copyback_owner_hold", "observed_workspace_preservation_fence", "wake_request_claimed", "legacy_controller_lease", "issue_execution_lock", "issue_checkout_lock"]);
     expect(OWNABLE_KINDS.filter(kind => !kind.listInProgress).every(kind => kind.coverageGap)).toBe(true);
   });
 

server/src/services/controller-process-identity.ts

@@ -2,13 +2,42 @@ import { createHash, randomUUID } from "node:crypto";
 import { readFileSync, readlinkSync } from "node:fs";
 import { hostname, uptime } from "node:os";
 import { readProcessStartedAt } from "./hot-restart.js";
+import { currentWorkspaceWorker, type WorkspaceWorker } from "./native-runtime/native-workspace-platform-proof.js";
 
 /**
  * Identity of the server process that started a provider side effect, saved
  * with its durable record. After a restart, recovery uses it to prove that the
  * starter has exited, so the side effect cannot still be in flight there.
  */
-export type ControllerProcess = { bootId: string; pid: number; hostHash: string; processStartedAt: string | null };
+export type ControllerDeployment = { worker: WorkspaceWorker; exclusiveVolume: boolean; volumeId: string | null };
+export type ControllerProcess = {
+  bootId: string; pid: number; hostHash: string; processStartedAt: string | null; deployment?: ControllerDeployment;
+};
+
+export function currentControllerDeployment(env: NodeJS.ProcessEnv = process.env): ControllerDeployment | null {
+  const worker = currentWorkspaceWorker(env);
+  if (!worker) return null;
+  const volumeId = env.RAILWAY_VOLUME_ID?.trim() || null;
+  return { worker, exclusiveVolume: Boolean(volumeId && env.RAILWAY_VOLUME_MOUNT_PATH?.trim()), volumeId };
+}
+
+function deploymentField(): { deployment?: ControllerDeployment } {
+  const deployment = currentControllerDeployment();
+  return deployment ? { deployment } : {};
+}
+
+function readDeployment(value: unknown): ControllerDeployment | null {
+  if (!value || typeof value !== "object" || Array.isArray(value)) return null;
+  const record = value as Record<string, unknown>;
+  const worker = record.worker as Record<string, unknown> | null | undefined;
+  if (!worker || typeof worker !== "object" || worker.provider !== "railway") return null;
+  const ids = [worker.projectId, worker.environmentId, worker.serviceId, worker.deploymentId];
+  if (ids.some(id => typeof id !== "string" || id.length === 0)) return null;
+  return { worker: { provider: "railway", projectId: ids[0] as string, environmentId: ids[1] as string,
+    serviceId: ids[2] as string, deploymentId: ids[3] as string },
+  exclusiveVolume: record.exclusiveVolume === true,
+  volumeId: typeof record.volumeId === "string" ? record.volumeId : null };
+}
 
 /**
  * The one boot id of this server process. Native coordinator claims
@@ -94,7 +123,7 @@ export async function currentControllerProcess(): Promise<ControllerProcess> {
   const startedAt = await currentProcessStartedAtMs();
   const iso = startedAt === null ? null : new Date(startedAt).toISOString();
   resolvedProcessStartedAt = iso;
-  return { bootId, pid: process.pid, hostHash, processStartedAt: iso };
+  return { bootId, pid: process.pid, hostHash, processStartedAt: iso, ...deploymentField() };
 }
 
 /**
@@ -104,7 +133,7 @@ export async function currentControllerProcess(): Promise<ControllerProcess> {
  * a live PID then stays ambiguous, which fails closed.
  */
 export function currentControllerProcessStamp(): ControllerProcess {
-  return { bootId, pid: process.pid, hostHash, processStartedAt: resolvedProcessStartedAt };
+  return { bootId, pid: process.pid, hostHash, processStartedAt: resolvedProcessStartedAt, ...deploymentField() };
 }
 
 export function readControllerProcess(value: unknown): ControllerProcess | null {
@@ -112,8 +141,10 @@ export function readControllerProcess(value: unknown): ControllerProcess | null
   const record = value as Record<string, unknown>;
   if (typeof record.bootId !== "string" || typeof record.hostHash !== "string" ||
       typeof record.pid !== "number" || !Number.isInteger(record.pid) || record.pid <= 0) return null;
+  const deployment = readDeployment(record.deployment);
   return { bootId: record.bootId, pid: record.pid, hostHash: record.hostHash,
-    processStartedAt: typeof record.processStartedAt === "string" ? record.processStartedAt : null };
+    processStartedAt: typeof record.processStartedAt === "string" ? record.processStartedAt : null,
+    ...(deployment ? { deployment } : {}) };
 }
 
 /**

server/src/services/environment-login-probe-lease-settlement.ts

@@ -0,0 +1,39 @@
+import { and, eq, inArray, isNull, ne, sql } from "drizzle-orm";
+import { environmentLeases, type Db } from "@paperclipai/db";
+import { writeEnvironmentCleanup } from "./environment-cleanup-notifications.js";
+import { pendingCleanupMetadataObjectSql } from "./environment-cleanup-state.js";
+import { holderUnchanged, ORPHANED_RUN_LEASE_FAILURE_REASON } from "./environment-run-lease-settlement.js";
+import { allowLegacyShutdownWorkspaceCleanup } from "./legacy-shutdown-workspace-settlement.js";
+import type { OwnerProof } from "./owner-proof.js";
+import type { InProgressRecord, SettleOutcome } from "./recovery/ownership-registry.js";
+
+export async function settleEnvironmentLoginProbeLease(db: Db, record: InProgressRecord,
+  context: { now: Date; proof: OwnerProof }): Promise<SettleOutcome> {
+  if (record.owner.kind !== "process") return { settled: false, reason: "owner_not_provable" };
+  return db.transaction(async transaction => {
+    const tx = transaction as unknown as Db;
+    const [lease] = await tx.select().from(environmentLeases)
+      .where(and(eq(environmentLeases.id, record.recordId), eq(environmentLeases.status, "active"),
+        isNull(environmentLeases.heartbeatRunId)))
+      .for("update");
+    if (!lease) return { settled: false, reason: "lease_no_longer_active" };
+    if (!holderUnchanged(lease, record)) return { settled: false, reason: "owner_changed_since_proof" };
+    if (!(await allowLegacyShutdownWorkspaceCleanup(tx, lease))) return { settled: false, reason: "legacy_shutdown_fence" };
+    if (lease.provider && lease.providerLeaseId) {
+      const [other] = await tx.select({ id: environmentLeases.id }).from(environmentLeases).where(and(
+        ne(environmentLeases.id, lease.id), eq(environmentLeases.provider, lease.provider),
+        eq(environmentLeases.providerLeaseId, lease.providerLeaseId),
+        inArray(environmentLeases.status, ["active", "retained", "pending_cleanup"]))).limit(1);
+      if (other) return { settled: false, reason: "provider_resource_shared" };
+    }
+    const flipped = await writeEnvironmentCleanup(tx, write => write.update(environmentLeases)
+      .set({ status: "pending_cleanup", failureReason: ORPHANED_RUN_LEASE_FAILURE_REASON,
+        metadata: sql`${pendingCleanupMetadataObjectSql()} || ${JSON.stringify({
+          pendingCleanupRecoveredAtMs: context.now.getTime(), recoveredRunlessLease: { ownerProof: context.proof.reason } })}::jsonb` })
+      .where(and(eq(environmentLeases.id, lease.id), eq(environmentLeases.status, "active"),
+        isNull(environmentLeases.heartbeatRunId),
+        sql`${environmentLeases.metadata} is not distinct from ${lease.metadata === null ? null : JSON.stringify(lease.metadata)}::jsonb`))
+      .returning({ id: environmentLeases.id }), { companyId: lease.companyId, leaseId: lease.id });
+    return flipped.length ? { settled: true, reason: context.proof.reason } : { settled: false, reason: "lease_no_longer_active" };
+  });
+}

server/src/services/environment-run-lease-settlement.ts

@@ -91,7 +91,7 @@ export async function settleEnvironmentRunLease(db: Db, record: InProgressRecord
 
 /** The proof judged a holder stamp (or its absence, for a legacy row); a lease
  * re-stamped since then belongs to someone else. */
-function holderUnchanged(lease: LeaseRow, record: InProgressRecord): boolean {
+export function holderUnchanged(lease: LeaseRow, record: InProgressRecord): boolean {
   const holder = readEnvironmentLeaseController(lease.metadata);
   if (record.owner.kind === "run") return holder === null;
   if (record.owner.kind !== "process") return false;

server/src/services/heartbeat/recovery.ts

@@ -1,5 +1,6 @@
 import { settleOwnableKind } from "../recovery/ownership-invariant-report.js";
 import { ORPHANED_RUN_LEASE_FAILURE_REASON } from "../environment-run-lease-settlement.js";
+import { ENVIRONMENT_LEASE_CONTROLLER_KEY } from "../environment-lease-controller.js";
 import { readNativeStartupCleanup, readNativeStartupContext } from "../native-runtime/native-startup-cleanup.js";
 import { legacyWorkspaceRecoveryNotMigrated } from "../legacy-workspace-recovery-migration.js";
 import { pendingFailedNativeRetryCancellationCondition } from "../native-runtime/native-cancellation-request.js";
@@ -1561,8 +1562,9 @@ export function createHeartbeatRecovery(db: Db, dependencies: HeartbeatRecoveryD
   // whose native workspace export is still pending is preserved, never
   // released, while its run is not executing here.
   //
-  // A login or probe lease has no run. It keeps the idle backoff below until
-  // its kind moves to the shared path.
+  // A run-less (login, probe, harness verification) lease is settled by the
+  // shared ownership path once its stamped holder is proven gone. Only a
+  // legacy row saved before holders were stamped keeps the idle backoff below.
   async function sweepOrphanedActiveLeases(opts: {
     /** Idle time before a run-less (login/probe) lease is recovered. */
     backoffMs: number;
@@ -1624,6 +1626,7 @@ export function createHeartbeatRecovery(db: Db, dependencies: HeartbeatRecoveryD
         opts.companyId ? eq(environmentLeases.companyId, opts.companyId) : undefined,
         opts.leaseId ? eq(environmentLeases.id, opts.leaseId) : undefined,
         isNull(environmentLeases.heartbeatRunId),
+        sql`not (coalesce(${environmentLeases.metadata}, '{}'::jsonb) ? ${ENVIRONMENT_LEASE_CONTROLLER_KEY})`,
         lte(environmentLeases.updatedAt, cutoff),
       ))
       .orderBy(asc(environmentLeases.updatedAt))

server/src/services/owner-proof.ts

@@ -1,6 +1,7 @@
 import { hostname } from "node:os";
 import {
   currentControllerBootId,
+  currentControllerDeployment,
   currentControllerProcessStamp,
   currentProcessSpaceHash,
   observedProcessStartedAt,
@@ -9,7 +10,7 @@ import {
   PROCESS_START_PRECISION_MS,
   UNKNOWN_PROCESS_SPACE,
 } from "./controller-process-identity.js";
-import { currentWorkspaceWorker, type WorkspaceWorker } from "./native-runtime/native-workspace-platform-proof.js";
+import type { WorkspaceWorker } from "./native-runtime/native-workspace-platform-proof.js";
 import { ephemeralServerEnabled } from "./ephemeral-server-mode.js";
 
 /** `processSpace` value written when a board operator confirmed that this exact
@@ -151,10 +152,7 @@ function processGroupExists(processGroupId: number): boolean {
  * the previous deployment before the new one starts, so the two never overlap.
  * A service with a volume cannot have replicas. */
 export function currentDeploymentIdentity(env: NodeJS.ProcessEnv = process.env): ProcessProbe["deployment"] {
-  const worker = currentWorkspaceWorker(env);
-  if (!worker) return null;
-  const volumeId = env.RAILWAY_VOLUME_ID?.trim() || null;
-  return { worker, exclusiveVolume: Boolean(volumeId && env.RAILWAY_VOLUME_MOUNT_PATH?.trim()), volumeId };
+  return currentControllerDeployment(env);
 }
 
 export function currentProcessProbe(): ProcessProbe {

server/src/services/recovery/ownership-registry.ts

@@ -54,6 +54,7 @@ import {
   pendingCleanupDeadlineSql,
   recoveredOrphanFirstAttemptSql,
 } from "../environment-cleanup-state.js";
+import { settleEnvironmentLoginProbeLease } from "../environment-login-probe-lease-settlement.js";
 import { settleEnvironmentRunLease } from "../environment-run-lease-settlement.js";
 import { settleEnvironmentTeardownClaim } from "../environment-teardown-claim-settlement.js";
 import { settleSandboxCreationIntent } from "../sandbox-creation-intent-settlement.js";
@@ -429,6 +430,8 @@ function controllerStamp(value: unknown, scope: string): RecordOwner {
   return { kind: "process", stamp: {
     kind: "local", scope, generation: null, host: null, processSpace: controller.hostHash,
     bootId: controller.bootId, pid: controller.pid, processStartedAt: controller.processStartedAt, processGroupId: null,
+    ...(controller.deployment ? { worker: controller.deployment.worker, exclusiveVolume: controller.deployment.exclusiveVolume,
+      volumeId: controller.deployment.volumeId } : {}),
   } };
 }
 
@@ -706,11 +709,12 @@ export const OWNABLE_KINDS: readonly OwnableKind[] = [
     nextDueAt: nextRetainedOrphanDueAt,
   },
   spoolKind("sandbox_orphan_cleanup_spool", "Orphan cleanup buffer/spool", "ownership", idleOrphanSpoolPath),
-  leaseKind("environment_login_probe_lease", "Login/probe lease", "ownership",
-    and(eq(environmentLeases.status, "active"), isNull(environmentLeases.heartbeatRunId)),
-    // Stamped by the write that makes the lease active. A lease saved before
-    // the stamp existed names no owner; it ends with its login session.
-    row => controllerStamp(json(row.metadata)[ENVIRONMENT_LEASE_CONTROLLER_KEY], "environment_login_probe_lease")),
+  {
+    ...leaseKind("environment_login_probe_lease", "Login/probe lease", "ownership",
+      and(eq(environmentLeases.status, "active"), isNull(environmentLeases.heartbeatRunId)),
+      row => controllerStamp(json(row.metadata)[ENVIRONMENT_LEASE_CONTROLLER_KEY], "environment_login_probe_lease")),
+    settle: settleEnvironmentLoginProbeLease,
+  },
 
   // Workspaces
   {

tests/restart-chaos/README.md

@@ -280,11 +280,12 @@ workload or `--nth` that does) and `harness_error`. Exit code 0, 1, 3 and 2.
 (`node scripts/crash-points.mjs --list --env daytona`). The run needs
 `DAYTONA_API_KEY` and an image matching this checkout (`--daytona-image` or
 `PAPERCLIP_E2E_DAYTONA_IMAGE`; see "Daytona" above), and refuses to start
-without them. A plain task reaches both `sandbox.create.*` points and
-`environment.lease_release.after_provider_before_lease`. The other three print
-what they need (`needs:` in `--list` and in a `not_reached` result): a failed
-release (teardown claim), a reusable lease (reusable destroy), or a
-stop-and-retain export.
+without them. A plain codex task reaches both `sandbox.create.*` points and
+both `environment.teardown.*` points (a native run releases its sandbox through
+teardown). `environment.lease_release.after_provider_before_lease` (the plain
+release path) and `environment.stop_retain.after_claim_before_stop` (a
+stop-and-retain export) were not reached by any workload on 2026-11-08; a
+`not_reached` result prints what each needs.
 
 **CI.** `.github/workflows/crash-points.yml` runs the same command on demand
 (`workflow_dispatch`, inputs `points`, `kinds`, `adapters`, `workloads`, `nth`).