fix(native): preserve closed post-result failure diagnostics (#19126)

## Thinking Path

> - Paperclip manages AI agents and their work.
> - Native execution persists a result before it completes cleanup and
workspace export.
> - A later execution error enters the failure-write path.
> - That write must reject an already selected result. The existing code
reports lease loss for every rejected write.
> - This can hide the category of the first error even when the same
controller still owns an unexpired lease.
> - This change adds closed diagnostic labels after exact ownership and
accepted-result checks. It keeps the existing control error and recovery
policy.

## Linked Issues or Issue Description

**What happened?**

A local real-PostgreSQL regression accepts a result through the
production control plane, then injects a harmless filesystem failure.
The production executor reports `native_session_lease_lost`. It loses
the original failure category because the failure transition requires a
null result ID.

**Expected behavior**

Keep the result guard and the existing lifecycle classification. Record
a closed diagnostic category when the selected result is accepted and
the controller still has exact, unexpired authority. Keep ordinary
lease-loss fencing when that proof is absent or its diagnostic lock is
busy.

**Steps to reproduce**

Run the new post-result cases in
`server/src/services/heartbeat/runtime-execution.test.ts`. They use the
real executor, control-plane result commit, heartbeat recovery path and
PostgreSQL. The provider boundary is controlled. No live provider is
used.

**Paperclip version or commit**

The regression fails on base `7d1e9b3477`. Related open PR #16917
changes legacy lease recovery. Open #19063 adds terminal finalizer
reasons, and #15310 preserves database finalization errors. They do not
change this executor diagnostic branch. This change only adds native
diagnostic evidence and does not reclaim a lease.

## What Changed

- Add a typed diagnostic with a closed stage, category and
accepted-result flag. Do not retain the source message, path, arbitrary
code or lease token.
- After a rejected failure write, check the selected accepted result and
the exact company, task, run, owner, attempt, generation, process
identity and expiry. Use a coordinator-only `NOWAIT` lock. Fall back to
the original control error if the diagnostic read fails.
- Report the closed event after rollback. Keep
`native_session_lease_lost` as the error message and leave every
failure-write predicate unchanged.
- Add real-PostgreSQL safety cases for stale and expired authority,
owner replacement, Stop state, result selection, pending accounting, one
provider invocation, closed labels and diagnostic sink failure.
- Document the closed event, unchanged authority, and possible late
diagnostic sink completion.
- Add a competing-row-lock barrier. It verifies actual PostgreSQL lock
refusal and control settlement before the competing lock is released.

## Verification

- Permanent regression on unchanged base: RED because the closed
diagnostic is absent.
- NOWAIT mutation: RED when only NOWAIT is removed. PostgreSQL reports
lock timeout instead of immediate lock refusal. The test uses barriers
and the actual database error, with no elapsed-time assertion.
- Final related suites: 56 tests passed across runtime execution and
provider failure integration tests.
- Full server source and test typecheck: passed.
- Full `pnpm -r typecheck`: passed in an independent locked-dependency
checkout.
- Full `pnpm build`: passed.
- Full `pnpm test:run`: running in the independent immutable checkout.
All six chat shards passed; the remaining suites are still pending.
- The first current-head CI attempt failed in two cases: workspace
runtime initial fixture HTTP readiness and instructions-bundle
error-message rendering. Both exact cases passed in isolation on
unchanged PR head and base. The original causes remain unknown after
bounded test-only observation. One controlled same-head rerun of each
failed job passed. Current required CI and all other completed checks
passed. Original failures remain recorded; no infrastructure or
flaky-test claim is made.
- Module boundary check and `git diff --check`: passed.
- Exact-head Greptile: 5/5 with no actionable findings or open review
threads. Automatic native review remains queued. That advisory check is
not required by current master rules; no current-head native assessment
is claimed.
- Earlier fixture setup failures remain recorded. These included missing
fixture fields and observing the Drizzle wrapper instead of its
PostgreSQL cause. They are not the regression RED. An earlier broad
typecheck was interrupted for the required NOWAIT revision; the final
full typecheck passed.
- No hosted export recovery or original hosted failure cause is claimed.

## Risks

The extra read is diagnostic only. It uses NOWAIT and cannot wait for a
competing coordinator lock. Its failure preserves the existing control
classification. The event sink uses the existing 60-second
execution-control bound. A slow sink can finish later and emit a late
event. Result selection, Stop, accounting, retry eligibility and
provider work do not depend on event success. No schema or lifecycle
contract changes are required.

## Model Used

OpenAI GPT-6.1, model ID `gpt-6.1-sol`, with reasoning, tool use and
code execution. The host did not expose the context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-11-04 17:45:53 -06:00
1 parent 0e27369fa4
commit 818d2bb661
3 files changed
+254 -5

No files matched your search

+11
View File
@@ -752,6 +752,17 @@ and current-user redaction. The task page uses one history read on mount/reconne
and the existing live transcript stream, then makes a bounded plugin readiness
check. No panel discovery timer scans runs or provider state while connected.
## Post-result failure diagnostics
`native.post_result_failure` records `stage: post_result_execution`, a closed
`category` (`io_not_found`, `io_permission_denied`, `io_failure`, or
`execution_failure`), and `resultAlreadyAccepted: true`. It requires an accepted
selected result and exact current, unexpired controller authority. A busy diagnostic
row lock is skipped. Reporting is best effort under the existing 60-second control
bound; the sink can finish later and emit a late event. The `native_session_lease_lost`
control classification stays unchanged. The labels grant no recovery, result,
Stop, provider-work or payment authority and do not establish a hosted failure cause.
## Payload storage
Structured events in this guide remain in `heartbeat_run_events`. When the
@@ -4,12 +4,12 @@ import { createNativeUsageDelivery } from "../native-runtime/native-usage-delive
import { flushBudgetEvaluationWork } from "../budgets.js";
import type { PrpEvent } from "../../vendor/paperclip-runner/index.js";
import { randomUUID } from "node:crypto";
import { mkdtemp, rm } from "node:fs/promises";
import { mkdtemp, rm, readFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { agents, budgetPolicies, companies, createDb, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS, heartbeatRuns, issues, executionWorkspaces, nativeRunFinalizations, runUsageReceipts, workspaceOperations, type Db } from "@paperclipai/db";
import { agents, budgetPolicies, companies, completionContracts, nativeRunResults, costEvents, agentWakeupRequests, createDb, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS, heartbeatRuns, issues, executionWorkspaces, nativeRunFinalizations, runUsageReceipts, workspaceOperations, type Db } from "@paperclipai/db";
import { eq, sql } from "drizzle-orm";
import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import { afterAll, beforeAll, beforeEach, describe, expect, it, onTestFinished, vi } from "vitest";
import { trace } from "@opentelemetry/api";
import { InMemorySpanExporter, SimpleSpanProcessor, TracerProvider } from "@opentelemetry/sdk-trace";
import { createRunPreparationTiming, measureRunPhase, measureWorkspaceSyncPhase, withRunExecutionTiming } from "../execution-preparation-timing.js";
@@ -21,7 +21,7 @@ import { createAdapterExecutionControl } from "../adapter-execution-control.js";
import { executionWorkspaceService } from "../execution-workspaces.js";
import { workspaceOperationService } from "../workspace-operations.js";
import { instanceSettingsService } from "../instance-settings.js";
import { executePaperclipNativeSession, NativeCancellationPendingRecoveryError, NativeControllerDetachedForRestartError } from "../native-runtime/native-session-executor.js";
import { executePaperclipNativeSession, NativeCancellationPendingRecoveryError, NativeControllerDetachedForRestartError, nativeSessionFailureSourceCode, nativeSessionFailureDisposition } from "../native-runtime/native-session-executor.js";
import { NativeRunnerOwnershipUnverifiedError } from "../native-runtime/native-runner-ownership.js";
import { NativeWorkspaceFinalizationBusyError, NativeWorkspaceFinalizationOwnershipLostError } from "../native-runtime/native-workspace-finalization-ownership.js";
import { resolveHeartbeatNativeRuntimeMode } from "../native-runtime/runtime-mode.js";
@@ -32,6 +32,16 @@ import { buildEffectiveRunSessionConfigMetadata } from "./workspaces.js";
import { revokeHeartbeatRunGatewayTokens } from "./run-preparation.js";
import { executeHeartbeatRuntime, type HeartbeatRuntimeExecutionInput } from "./runtime-execution.js";
import { prepareNativeHeartbeatRun } from "../native-runtime/prepare-native-run.js";
import { buildNativeCompletionContract } from "../native-runtime/completion-contracts.js";
import { currentNativeControllerIdentity } from "../native-runtime/native-restart-recovery.js";
import { CONTROL_PLANE_CONFORMANCE_RESULT } from "../../vendor/paperclip-runner/testing.js";
import type { ControlPlanePort, NativeExecutionInputV1 } from "@paperclipai/paperclip-runner";
const postResultProvider = vi.hoisted(() => ({ execute: vi.fn() }));
vi.mock("../../vendor/paperclip-runner/index.js", async original => ({
...await original<typeof import("../../vendor/paperclip-runner/index.js")>(), executeNativeSession: postResultProvider.execute,
}));
// Keep real accounting, workspace barriers, and task-session persistence. Providers
// and gateway delivery are external boundaries; browser acceptance covers their wiring.
vi.mock("../native-runtime/native-session-executor.js", async original => ({
@@ -115,6 +125,169 @@ describe.skipIf(!support.supported)("heartbeat runtime execution boundary", () =
};
return { input, adapter, run };
}
it.each(["current", "expired", "replaced-owner", "generation", "attempt", "boot", "pid", "process-start", "stopped", "unknown-code", "permission", "io", "diagnostic-sink-failed", "diagnostic-lock-busy"] as const)(
"preserves post-result diagnostics without changing recovery authority (%s)", async scenario => {
const { input } = await fixture(true);
const issue = input.task.issueRef!;
const [run] = await db.update(heartbeatRuns).set({ nativeIssueId: issue.id, contextSnapshot: { issueId: issue.id }, costAccountingPending: true }).where(eq(heartbeatRuns.id, input.run.id)).returning();
input.run = run;
const prepared = await prepareNativeHeartbeatRun({ db, run, issue, environmentLeaseId: randomUUID() });
const [contract] = await db.select().from(completionContracts).where(eq(completionContracts.issueId, issue.id));
const completion = buildNativeCompletionContract(issue, { revision: contract.revision });
await db.insert(nativeRunFinalizations).values({ runId: run.id, companyId: run.companyId, issueId: issue.id, phase: "observed" });
const execution: NativeExecutionInputV1 = {
schema: "paperclip.native-execution-input.v1", interactionResponses: [], credentialBindings: [], binding: { companyId: run.companyId, agentId: run.agentId, issueId: issue.id, runId: run.id, executionWorkspaceId: run.id },
completionContract: { id: contract.id, sha256: contract.canonicalSha256, schemaVersion: "paperclip.completion-contract.v1", contract: completion },
provider: { kind: "codex", model: "test-model" },
task: { identifier: issue.id, title: issue.title, description: null, prompt: "Local post-result fault", workMode: "standard" },
workspace: { cwd: home, repoUrl: null, repoRef: null, branchName: null },
session: { normalizedSessionId: prepared.normalizedSessionId, driverKind: "codex_app_server", protocolVersion: 1, lifecyclePolicy: { mode: "per_turn", idleTimeoutMs: null } },
};
input.runtime.nativeExecution = execution;
input.runtime.nativeRunnerInstanceId = prepared.runnerInstanceId;
input.workspace.nativeWorkspaceSync = { mode: "host_current", reference: {
schema: "paperclip.native-workspace-sync/v3", state: "prepared", descriptorSha256: "a".repeat(64), baselineSha256: "b".repeat(64), finalHostSha256: null,
workspaceId: run.id, leaseId: randomUUID(), providerLeaseId: "controlled-local", remoteCwd: home, resourceDisposition: null },
restoreWorkspace: vi.fn(async () => {}), cleanup: vi.fn(async () => {}) };
const diagnosticExpected = ["current", "stopped", "unknown-code", "permission", "io", "diagnostic-sink-failed"].includes(scenario);
let faultInjected = false, lockHeld = false, settledWhileLocked = false;
let releaseLock: () => void = () => {};
let lockTask: Promise<unknown> | undefined;
let diagnosticLockError: { code?: string; message?: string } | undefined;
if (scenario === "diagnostic-lock-busy") {
const transaction = db.transaction.bind(db);
let armed = false;
const transactionSpy = vi.spyOn(db, "transaction").mockImplementation(async (body, config) => {
const result = await transaction(async tx => {
// Observe real query errors; the executor still executes its actual
// diagnostic SELECT against PostgreSQL and the competing row lock.
if (faultInjected) {
const select = tx.select.bind(tx);
vi.spyOn(tx, "select").mockImplementation(fields => {
const selection = fields === undefined ? select() : select(fields);
const from = selection.from.bind(selection);
selection.from = ((table: Parameters<typeof selection.from>[0]) => {
const query = from(table);
if (table === nativeRunFinalizations && fields?.runId === nativeRunFinalizations.runId) {
const lock = query.for.bind(query);
query.for = ((...args: Parameters<typeof query.for>) => lock(...args).catch(error => {
diagnosticLockError = error.cause ?? error;
throw error;
})) as typeof query.for;
}
return query;
}) as typeof selection.from;
return selection as ReturnType<typeof select>;
});
}
return body(tx);
}, config);
// The first transaction after the injected I/O failure publishes the
// control deadline. Hold the row only after that transaction commits.
if (faultInjected && !armed) {
armed = true;
let ready!: () => void;
const acquired = new Promise<void>(resolve => { ready = resolve; });
const release = new Promise<void>(resolve => { releaseLock = resolve; });
lockTask = transaction(async locker => {
await locker.select({ runId: nativeRunFinalizations.runId }).from(nativeRunFinalizations)
.where(eq(nativeRunFinalizations.runId, run.id)).for("update");
lockHeld = true;
ready();
await release;
lockHeld = false;
});
await acquired;
}
return result;
});
onTestFinished(async () => { releaseLock(); await lockTask; transactionSpy.mockRestore(); });
}
postResultProvider.execute.mockReset().mockImplementationOnce(async (options: { controlPlane: ControlPlanePort }) => {
await options.controlPlane.openRun({ identity: { companyId: run.companyId, issueId: issue.id, runId: run.id, agentId: run.agentId, sessionId: prepared.normalizedSessionId }, backendKind: "mock", sourceInstanceId: prepared.runnerInstanceId });
await options.controlPlane.completeRun({ result: { ...CONTROL_PLANE_CONFORMANCE_RESULT, completionClaim: { ...CONTROL_PLANE_CONFORMANCE_RESULT.completionClaim, contractRevision: completion.revision } },
terminal: { schema: "paperclip.prp.terminal.v1", turnTerminalState: "completed", runTerminalState: "succeeded", reportedWorkDisposition: "done" }, turnId: prepared.turnId });
const [owner] = await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, run.id));
const controller = await currentNativeControllerIdentity();
expect(owner).toMatchObject({ phase: "workspace_finalizing", attempt: 1, controllerGeneration: 1, controllerBootId: controller.bootId, controllerPid: controller.pid, controllerProcessStartedAt: controller.processStartedAt });
expect(owner.leaseOwner).toBeTruthy();
expect(owner.leaseExpiresAt!.getTime()).toBeGreaterThan(Date.now());
const changed = scenario === "expired" ? { leaseExpiresAt: new Date(0) }
: scenario === "replaced-owner" ? { leaseOwner: "replacement-owner" }
: scenario === "generation" ? { controllerGeneration: owner.controllerGeneration + 1 }
: scenario === "attempt" ? { attempt: owner.attempt + 1 }
: scenario === "boot" ? { controllerBootId: randomUUID() }
: scenario === "pid" ? { controllerPid: controller.pid + 1 }
: scenario === "process-start" ? { controllerProcessStartedAt: new Date(controller.processStartedAt.getTime() - 1000) } : null;
if (changed) await db.update(nativeRunFinalizations).set(changed).where(eq(nativeRunFinalizations.runId, run.id));
if (scenario === "stopped") await db.update(heartbeatRuns).set({ status: "cancelled", finishedAt: new Date() }).where(eq(heartbeatRuns.id, run.id));
try { await readFile(join(home, `missing-${run.id}`, "harmless-checkpoint")); }
catch (error) {
faultInjected = true;
if (scenario === "unknown-code") (error as NodeJS.ErrnoException).code = "private-unrecognized-code";
if (scenario === "permission") (error as NodeJS.ErrnoException).code = "EACCES";
if (scenario === "io") (error as NodeJS.ErrnoException).code = "EIO";
throw error;
}
throw new Error("fixture must fail with ENOENT");
});
const events: unknown[] = [];
let controlError: unknown;
const actual = await vi.importActual<typeof import("../native-runtime/native-session-executor.js")>("../native-runtime/native-session-executor.js");
vi.mocked(executePaperclipNativeSession).mockImplementationOnce(async options => {
try { return await actual.executePaperclipNativeSession({ ...options, useRunnerd: false,
backend: { descriptor: async () => { throw new Error("Unexpected provider call"); }, openSession: async () => { throw new Error("Unexpected provider call"); } },
onEvent: async event => {
events.push(event);
if (event.eventType === "native.post_result_failure" && scenario === "diagnostic-sink-failed") throw new Error("private sink failure");
await options.onEvent?.(event);
} }); }
catch (error) {
controlError = error;
settledWhileLocked = lockHeld;
releaseLock();
await lockTask;
throw error;
}
});
await expect(executeHeartbeatRuntime(db, input)).rejects.toMatchObject({ name: "NativeWorkspaceFinalizeScheduledError", reasonCode: "workspace_sync_out_failed", terminalFailure: false });
expect((controlError as Error).message).toBe("native_session_lease_lost");
if (scenario === "diagnostic-lock-busy") {
expect(settledWhileLocked).toBe(true);
expect(diagnosticLockError?.code).toBe("55P03");
expect(diagnosticLockError?.message).toContain("could not obtain lock");
}
const source = nativeSessionFailureSourceCode(controlError);
expect(source).toBe(nativeSessionFailureSourceCode(new Error("native_session_lease_lost")));
expect(nativeSessionFailureDisposition(1, new Date(0), source)).toEqual(nativeSessionFailureDisposition(1, new Date(0), nativeSessionFailureSourceCode(new Error("native_session_lease_lost"))));
expect((controlError as { diagnostic?: unknown }).diagnostic).toEqual(diagnosticExpected
? { stage: "post_result_execution", category: scenario === "unknown-code" ? "execution_failure" : scenario === "permission" ? "io_permission_denied" : scenario === "io" ? "io_failure" : "io_not_found", resultAlreadyAccepted: true }
: undefined);
const diagnostics = events.filter(event => (event as { eventType: string }).eventType === "native.post_result_failure") as Array<{ payload: unknown }>;
expect(diagnostics).toHaveLength(diagnosticExpected ? 1 : 0);
if (diagnosticExpected) expect(diagnostics[0].payload).toEqual({ stage: "post_result_execution", category: scenario === "unknown-code" ? "execution_failure" : scenario === "permission" ? "io_permission_denied" : scenario === "io" ? "io_failure" : "io_not_found", resultAlreadyAccepted: true });
expect(JSON.stringify(diagnostics)).not.toContain(home);
expect(JSON.stringify(diagnostics)).not.toContain("private-");
const [after] = await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, run.id));
const results = await db.select().from(nativeRunResults).where(eq(nativeRunResults.runId, run.id));
expect(results).toHaveLength(1);
expect(after.resultId).toBe(results[0].id);
expect(after.phase).toBe("retryable_failure");
expect(after.failureCode).toBe("native_workspace_sync_out_failed");
expect(after.nextAttemptAt).toBeInstanceOf(Date);
const saved = await read(run.id);
expect(saved.costAccountingPending).toBe(true);
expect(saved.costAccountedAt).toBeNull();
expect(await db.select().from(costEvents).where(eq(costEvents.heartbeatRunId, run.id))).toHaveLength(0);
expect(await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.companyId, run.companyId))).toHaveLength(0);
if (scenario === "stopped") expect(saved.status).toBe("cancelled");
expect(postResultProvider.execute).toHaveBeenCalledOnce();
expect(input.effects.onNativeWorkspaceFinalizeScheduled).toHaveBeenCalledOnce();
expect(input.services.enqueueWakeup).not.toHaveBeenCalled();
expect(input.workspace.nativeWorkspaceSync.restoreWorkspace).not.toHaveBeenCalled();
},
);
const read = (id: string) => db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, id)).then(rows => rows[0]!);
const finalizations = () => db.select().from(workspaceOperations).where(eq(workspaceOperations.phase, "workspace_finalize"));
function expectCleanup(input: HeartbeatRuntimeExecutionInput) {
@@ -190,6 +190,7 @@ import {
gt,
inArray,
isNull,
isNotNull,
like,
notInArray,
or,
@@ -307,6 +308,27 @@ export class NativeCancellationPendingRecoveryError extends Error {
}
}
type NativePostResultFailureCategory = "io_not_found" | "io_permission_denied" | "io_failure" | "execution_failure";
class NativePostResultFailureError extends Error {
readonly diagnostic: Readonly<{
stage: "post_result_execution";
category: NativePostResultFailureCategory;
resultAlreadyAccepted: true;
}>;
constructor(error: unknown) {
// Keep the existing control classification. These labels explain why its
// failure write was rejected; they grant no finalization or retry authority.
super("native_session_lease_lost");
const code = record(error).code;
const category: NativePostResultFailureCategory = code === "ENOENT" ? "io_not_found"
: code === "EACCES" || code === "EPERM" ? "io_permission_denied"
: code === "EIO" ? "io_failure" : "execution_failure";
this.diagnostic = Object.freeze({ stage: "post_result_execution", category, resultAlreadyAccepted: true });
}
}
export class NativeControllerDetachedForRestartError extends Error {
constructor() {
super("native_controller_detached_for_restart");
@@ -10738,7 +10760,39 @@ async function executePaperclipNativeSessionWithinScope(
)
.returning({ runId: nativeRunFinalizations.runId, resultId: nativeRunFinalizations.resultId })
.then((rows) => rows[0] ?? null);
if (!updated) throw new Error("native_session_lease_lost");
if (!updated) {
// Result acceptance intentionally prevents the failure transition.
// Diagnose that case only under the same still-current authority;
// stale or expired owners must retain the ordinary fencing error.
const [acceptedUnderCurrentAuthority] = await tx
.select({ runId: nativeRunFinalizations.runId })
.from(nativeRunFinalizations)
.innerJoin(nativeRunResults, and(
eq(nativeRunResults.id, nativeRunFinalizations.resultId),
eq(nativeRunResults.runId, input.execution.binding.runId),
eq(nativeRunResults.companyId, input.execution.binding.companyId),
eq(nativeRunResults.issueId, input.execution.binding.issueId),
eq(nativeRunResults.schemaStatus, "accepted"),
))
.where(and(
eq(nativeRunFinalizations.runId, input.execution.binding.runId),
eq(nativeRunFinalizations.companyId, input.execution.binding.companyId),
eq(nativeRunFinalizations.issueId, input.execution.binding.issueId),
eq(nativeRunFinalizations.leaseOwner, leaseOwner),
eq(nativeRunFinalizations.attempt, attempt),
eq(nativeRunFinalizations.controllerGeneration, controllerGeneration),
eq(nativeRunFinalizations.controllerBootId, controller.bootId),
eq(nativeRunFinalizations.controllerPid, controller.pid),
eq(nativeRunFinalizations.controllerProcessStartedAt, controller.processStartedAt),
gt(nativeRunFinalizations.leaseExpiresAt, sql`clock_timestamp()`),
isNotNull(nativeRunFinalizations.resultId),
))
.for("update", { of: nativeRunFinalizations, noWait: true })
// A failed diagnostic read must retain the existing control error.
.catch(() => []);
if (acceptedUnderCurrentAuthority) throw new NativePostResultFailureError(error);
throw new Error("native_session_lease_lost");
}
const [runBeforeWrite] = await tx
.select({ status: heartbeatRuns.status, resultJson: heartbeatRuns.resultJson,
runnerInstanceId: heartbeatRuns.runnerInstanceId, nativeSessionId: heartbeatRuns.nativeSessionId,
@@ -10899,6 +10953,17 @@ async function executePaperclipNativeSessionWithinScope(
supersedeOnIdentityChange:
recoveryProjection.supersedeOnIdentityChange,
});
}).catch(async (failure: unknown) => {
// Emit after rollback so the diagnostic sink cannot deadlock on the
// coordinator transaction. Reporting failure cannot change fencing.
if (failure instanceof NativePostResultFailureError) {
await boundedExecutionCleanup(async () => {
await input.onEvent?.({ eventType: "native.post_result_failure", level: "warn",
message: "Native execution failed after its result was accepted.",
payload: { ...failure.diagnostic } });
});
}
throw failure;
});
// Pass the raw error only for closed settlement labels: its text bypasses
// the identity redactor applied to the persisted message above.