mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-01 06:42:06 +01:00
fix(runner): allow session reset after confirmed task retirement (#18373)
## Thinking Path > - Paperclip manages AI agents and their work. > - Native warm sessions retain a Runner owner between turns. > - Closing that owner must stop execution and preserve valid recovery state. > - A failed suspension can leave a rejected close promise after the task has stopped. > - A session reset then retries that same promise and blocks every fresh session. > - This PR separates task retirement proof from checkpoint reuse for a new session. > - A reset can proceed after exact stop proof, completed handoff, and file collection. ## Linked Issues or Issue Description Refs #18011. That change preserves checkpoint authority during warm preparation. This change handles a stopped owner whose checkpoint cannot be reused. Related: #14695 covers managed files retained with warm processes. A search found no duplicate stopped-owner reset fix. **What happened?** A warm Runner can stop and complete its task handoff while close reports `runner did not durably suspend before checkpoint`. The transport caches that rejected close promise. Core retains the warm owner. Each session reset retries the same promise before it can launch a provider. A controller restart clears the in-memory owner and permits recovery. **Expected behavior** A reset to a new normalized session should release the old owner when its exact task has proved that execution stopped and its handoff completed. Core must collect stopped instruction files before it admits the new session. An unconfirmed owner must continue to block replacement. **Steps to reproduce** 1. Complete a warm native run through a plugin task provider. 2. Change configuration so Core retires the warm owner. 3. Let the task stop and complete, but fail durable suspension proof. Close keeps its rejected promise. 4. Reset the session and start another run on the same task environment. 5. Observe the old close error before any new provider work starts. ## What Changed - Keep a lease-bound retirement receipt in the plugin task owner. It requires `executionStopped: true` and successful task completion after credential handoff. - Accept an unrecoverable suspension error only during retirement for a different normalized session, with that exact owner's receipt and settled Core tool results. - Collect stopped instruction files before releasing the warm owner. Preserve old checkpoint evidence. - Keep same-session, active-owner, incomplete-handoff, unrelated-owner, unresolved Core tool results, and other close-error fences. - Report Core tool settlement on the close error after ingress retirement and drain. Default missing proof to false. Expose the error through the server's Runner shim. Add regression tests and document the recovery boundary. ## Verification - The reset regression fails with the new handling disabled and passes with it enabled. - Both affected suites passed: 1,032 tests. The final 15 focused owner/reset cases pass after adding the tool-settlement guard. - The assembled regression uses the real task-owner and backend binding. Its controlled task provider proves stop and completion. Close returns the same rejected promise on every call. - Negative cases cover incomplete handoff, other close errors, same-session continuation, provider change with the same session ID, unrelated receipts, unresolved Core tool results, and failed file collection. Collection can later succeed without replaying stop or reading the old checkpoint. - Five real-process transport cases pass. They cover an insufficient suspension deadline, the cached rejected promise, late tool completion within and beyond the deadline, interruption, and result-persistence failure. Unsettled callbacks produce `semanticToolsSettled: false`. - `pnpm -r typecheck` passed with the repository's pinned Rust toolchain on PATH. After the final contract change, Runner TypeScript build and server `tsc --noEmit` passed. - The local full-suite run was stopped at the requester's direction. Full regression and build coverage are delegated to GitHub CI. - Full GitHub CI passed on `ef0040d80af679ce6d4a130468e15f75678c1f4c`. Standard Greptile reviewed the same commit: 5/5, zero comments. Apex then reviewed that same commit: 5/5, zero comments. Final checks confirm no unresolved review threads and no merge conflicts. - Initial full CI on `00c002fd5a` failed one scheduled-retry route test at 18/20 cancellations. Controlled cancellation latency reproduced the short poll-budget failure. Before the next cycle, master advanced with #18322, which coordinates that test at the first completed batch. The separate local timeout commit was dropped during rebase. The upstream test passes locally. The rebased run passed 46 targeted owner/reset and scheduled-retry route cases. This PR has no scheduled-retry source or test diff. - No deployed provider run or tenant rollout is claimed by these source tests. ## Risks The exception applies only to a new normalized session and an exact completed plugin task. The old checkpoint remains evidence and cannot authorize reuse through this path. The original suspension failure can remain visible on its run. This fix prevents repeated startup failures after a reset; it does not hide unconfirmed cleanup or make every suspension succeed. Missing task receipts and failed file collection continue to block replacement. No schema migration, plugin protocol change, Rust Runner binary change, or new polling is required. Tests use a controlled task boundary; they do not certify every external provider implementation. ## Model Used OpenAI Codex, GPT-6 family, with tool use and local code execution. The exact runtime model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal issue or instance details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] Greptile Apex is 5/5 with no unresolved findings - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
4edd10804e
commit
6174b4d7c8
9 files changed
+196
-15
No files matched your search
@@ -222,6 +222,16 @@ Core permits read-only suspension proof for the exact saved identity of a
|
||||
successful warm run after its reuse lease is retained or superseded. This does
|
||||
not authorize new work or file access through that lease.
|
||||
Temporary credential homes are not needed after the originating run ends.
|
||||
An explicit reset to a different normalized session can retire a warm owner
|
||||
whose close failed only because it could not prove checkpoint suspension. Core
|
||||
requires the exact lease-bound task's `executionStopped: true` receipt, a
|
||||
successful `complete` handoff, and settled Core tool results first. It then
|
||||
collects stopped instruction files before it releases the in-memory owner. The old checkpoint remains evidence;
|
||||
it is not reused by the new session. A failed handoff, failed file collection,
|
||||
unsettled Core tool result, other close error, or continuation of the same
|
||||
session still blocks replacement.
|
||||
A controller restart is not required for this stopped-owner reset path.
|
||||
|
||||
Termination receipts cover all run leases that owned the retired physical task.
|
||||
A stopped task gets a new provider task ID for checkpoint resumption. Inspection
|
||||
precedes in-memory session selection so a replacement cannot inherit the stopped
|
||||
|
||||
@@ -143,7 +143,11 @@ export function isSemanticToolOutcomeUnknownError(error: unknown): error is Sema
|
||||
export class NativeSessionCloseUnrecoverableError extends Error {
|
||||
readonly code = "native_session_close_unrecoverable";
|
||||
|
||||
constructor(readonly settlement?: Record<string, unknown>) {
|
||||
constructor(
|
||||
readonly settlement?: Record<string, unknown>,
|
||||
/** Exact durable results exist for every admitted Core tool callback. */
|
||||
readonly semanticToolsSettled = false,
|
||||
) {
|
||||
super(
|
||||
"provider_transport_failed: runner did not durably suspend before checkpoint",
|
||||
);
|
||||
|
||||
@@ -1356,6 +1356,7 @@ it.each(["after_budget", "within_budget", "interrupted_within_budget", "persiste
|
||||
expect(closeFailure).toBeInstanceOf(
|
||||
NativeSessionCloseUnrecoverableError,
|
||||
);
|
||||
expect(closeFailure).toHaveProperty("semanticToolsSettled", false);
|
||||
expect(closeFailure).toHaveProperty("settlement.semanticTools.pending", expect.arrayContaining([
|
||||
expect.objectContaining({ callId: expect.any(String), operationId: expect.any(String),
|
||||
sourceEventId: expect.any(String), inputDigest: expect.stringMatching(/^[a-f0-9]{64}$/) }),
|
||||
@@ -8110,9 +8111,13 @@ it("still fails closed when a real close grace period cannot fit a durable suspe
|
||||
cwd: tmpdir(),
|
||||
dynamicTools: [],
|
||||
});
|
||||
await expect(bundle.transport.close()).rejects.toThrow(
|
||||
"runner did not durably suspend before checkpoint",
|
||||
);
|
||||
const closed = bundle.transport.close();
|
||||
await expect(closed).rejects.toMatchObject({
|
||||
code: "native_session_close_unrecoverable",
|
||||
semanticToolsSettled: true,
|
||||
});
|
||||
// Retirement preserves the original rejected promise and its proof.
|
||||
expect(bundle.transport.close()).toBe(closed);
|
||||
} finally {
|
||||
await rm(stateDirectory, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
@@ -4928,7 +4928,11 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
|
||||
} catch {
|
||||
// Keep the settlement failure authoritative if its observer fails.
|
||||
}
|
||||
throw new NativeSessionCloseUnrecoverableError(settlement);
|
||||
// Sample after ingress retirement and its drain. A result observed before
|
||||
// that barrier cannot rule out one last admitted tool callback.
|
||||
throw new NativeSessionCloseUnrecoverableError(
|
||||
settlement, this.#core?.semanticToolResultsSettled() === true,
|
||||
);
|
||||
}
|
||||
if (!preparationConfirmed) {
|
||||
throw new Error(`Native runner ${attachmentPending ? "attachment" : "startup"} cleanup is unconfirmed; recovery state retained.`);
|
||||
|
||||
@@ -75,6 +75,7 @@ import { agentInstructionWorkingCopies } from "@paperclipai/db";
|
||||
import * as noLaunchProofModule from "./native-maintenance-no-launch.js";
|
||||
import {
|
||||
NativeSessionCleanupQuarantinedError,
|
||||
NativeSessionCloseUnrecoverableError,
|
||||
NativeProviderTerminalFailure,
|
||||
NativeSessionProtocolIntegrityError,
|
||||
} from "../../vendor/paperclip-runner/index.js";
|
||||
@@ -9201,6 +9202,102 @@ describe("native warm session supervision", () => {
|
||||
await closeIdleWarmNativeSessionsForRestart();
|
||||
});
|
||||
|
||||
it.each(["retired", "unconfirmed", "other-error", "collection-failed", "same-session", "same-id-provider", "unbound", "pending-semantic"] as const)(
|
||||
"handles a permanently rejected plugin close during reset (%s)", async mode => {
|
||||
const { base, next, run } = fixture(`identity-poisoned-close-${mode}`);
|
||||
const identity = { runnerInstanceId: "runner", environmentLeaseId: "lease", runId: base.binding.runId,
|
||||
normalizedSessionId: base.session.normalizedSessionId!, turnId: "turn", itemId: "item" };
|
||||
let stopped = false;
|
||||
const task = vi.fn(async (operation: any) => {
|
||||
if (operation.kind === "describe") return { kind: "description", taskId: "old-task", launchPreparationVersion: 1,
|
||||
projectAuthCheckVersion: 1, runner: { version: "fixture", digest: `sha256:${"a".repeat(64)}` } };
|
||||
if (operation.kind === "stop") stopped = true;
|
||||
if (operation.kind === "status") return { kind: "status", taskId: "old-task", phase: stopped ? "cancelled" : "running", executionStopped: stopped };
|
||||
if (operation.kind === "complete" && mode === "unconfirmed") throw new Error("handoff unconfirmed");
|
||||
return { kind: "accepted", taskId: "old-task" };
|
||||
});
|
||||
const createOwner = pluginTaskRuntime.createPluginRunnerTaskOwner;
|
||||
let owner!: ReturnType<typeof createOwner>;
|
||||
const ownerFactory = vi.spyOn(pluginTaskRuntime, "createPluginRunnerTaskOwner").mockImplementation(input => {
|
||||
owner = createOwner({ ...input, adoptIdentity: identity });
|
||||
return owner;
|
||||
});
|
||||
onTestFinished(() => ownerFactory.mockRestore());
|
||||
// Bind through the production backend so the receipt belongs to this
|
||||
// exact session object. The task boundary supplies the stop proof.
|
||||
const backend = await createRunnerdBackend({ db: leaseDb(base),
|
||||
execution: { ...base, session: { ...base.session, lifecyclePolicy: { mode: "per_turn", idleTimeoutMs: null } } },
|
||||
runnerInstanceId: "runner", runnerIngressAuthorized: true,
|
||||
runnerExecutionTarget: { kind: "remote", transport: "sandbox", remoteCwd: ".", leaseId: "lease", runnerTask: task } });
|
||||
const registration = await owner.registration({} as never, base.binding.runId);
|
||||
const failure = mode === "other-error" ? new Error("controller retirement unconfirmed") : new NativeSessionCloseUnrecoverableError(undefined, mode !== "pending-semantic");
|
||||
let closePromise: Promise<void> | undefined;
|
||||
const close = vi.fn(() => closePromise ??= (async () => {
|
||||
await registration.release();
|
||||
await registration.checkpoint("unsettled").catch(() => undefined);
|
||||
throw failure;
|
||||
})());
|
||||
const session = { close };
|
||||
if (mode !== "unbound") backend.bindManagedSession(session as never);
|
||||
const collectStopped = vi.fn(async () => {
|
||||
expect(owner.isRetired()).toBe(true);
|
||||
if (mode === "collection-failed") throw new Error("file collection failed");
|
||||
});
|
||||
const retirementFailed = vi.fn(async () => {});
|
||||
state.execute.mockReset().mockImplementationOnce(async options => {
|
||||
await options.onSession?.(session); return result;
|
||||
}).mockImplementationOnce(async options => {
|
||||
expect(options.existingSession).toBeUndefined();
|
||||
expect(collectStopped).toHaveBeenCalled();
|
||||
return result;
|
||||
});
|
||||
try {
|
||||
await executePaperclipNativeSession({ db: leaseDb(base), execution: base, runnerInstanceId: "runner",
|
||||
instructionWorkingCopy: { hasChanges: async () => false, collectStopped, retirementFailed } });
|
||||
// First poison the retained owner through the real configuration
|
||||
// retirement path. An explicit reset must not need a process restart.
|
||||
const changed = { ...base, binding: { ...base.binding, runId: `${base.binding.runId}-changed` } };
|
||||
await expect(executePaperclipNativeSession({ db: leaseDb(changed), execution: changed,
|
||||
runnerInstanceId: "runner", refreshTools: true })).rejects.toThrow(failure.message);
|
||||
expect(close).toHaveBeenCalledOnce();
|
||||
expect(collectStopped).not.toHaveBeenCalled();
|
||||
expect(retirementFailed).toHaveBeenCalledOnce();
|
||||
retirementFailed.mockClear();
|
||||
const replacement = mode === "same-session" ? { ...next, session: base.session }
|
||||
: mode === "same-id-provider" ? { ...next, session: base.session,
|
||||
provider: { kind: "opencode" as const, model: "openai/test", permissionMode: "allow" as const } } : next;
|
||||
const attempt = () => mode === "same-session"
|
||||
? executePaperclipNativeSession({ db: leaseDb(replacement), execution: replacement, runnerInstanceId: "runner", refreshTools: true })
|
||||
: run(replacement);
|
||||
if (mode === "retired") {
|
||||
await attempt();
|
||||
expect(state.execute).toHaveBeenCalledTimes(2);
|
||||
expect(retirementFailed).not.toHaveBeenCalled();
|
||||
} else {
|
||||
// Both attempts see the same rejected promise, just as the real
|
||||
// transport does; no mock succeeds on the second close.
|
||||
const expected = mode === "collection-failed" ? "file collection failed" : failure.message;
|
||||
await expect(attempt()).rejects.toThrow(expected);
|
||||
await expect(attempt()).rejects.toThrow(expected);
|
||||
expect(state.execute).toHaveBeenCalledTimes(1);
|
||||
if (mode !== "collection-failed") expect(collectStopped).not.toHaveBeenCalled();
|
||||
else {
|
||||
collectStopped.mockResolvedValue(undefined);
|
||||
await attempt();
|
||||
expect(state.execute).toHaveBeenCalledTimes(2);
|
||||
}
|
||||
}
|
||||
expect(task.mock.calls.filter(([operation]) => operation.kind === "stop")).toHaveLength(1);
|
||||
expect(task.mock.calls.some(([operation]) => operation.kind === "read_state")).toBe(false);
|
||||
} finally {
|
||||
collectStopped.mockResolvedValue(undefined);
|
||||
close.mockResolvedValue(undefined);
|
||||
await closeIdleWarmNativeSessionsForRestart();
|
||||
ownerFactory.mockRestore();
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it("retains failed retirement for retry and never launches over an uncontained owner", async () => {
|
||||
const { base, next, run } = fixture("identity-close-failure");
|
||||
const close = vi.fn().mockRejectedValueOnce(new Error("containment unavailable")).mockResolvedValue(undefined);
|
||||
@@ -12675,7 +12772,7 @@ describe("runnerd provider runtime wiring", () => {
|
||||
const driver = { prepareLaunch: vi.fn(), runnerProcessLauncher: launcher, controlPlaneRegistration: registration };
|
||||
state.toolAuthorityDefinitions.mockImplementation(async binding => binding.readRemoteTaskFile ? [{ name: "register_deliverable" }] as never : []);
|
||||
const readWorkspaceFile = vi.fn(async () => Buffer.from("proof"));
|
||||
const owner = vi.spyOn(pluginTaskRuntime, "createPluginRunnerTaskOwner").mockReturnValue({ launcher, registration, driver, readWorkspaceFile, rebind:vi.fn() });
|
||||
const owner = vi.spyOn(pluginTaskRuntime, "createPluginRunnerTaskOwner").mockReturnValue({ launcher, registration, driver, readWorkspaceFile, isRetired: () => false, rebind:vi.fn() });
|
||||
const task = vi.fn(async (operation: unknown) => {
|
||||
environmentTaskOperationSchema.parse(operation);
|
||||
return { kind: "description", launchPreparationVersion: 1, projectAuthCheckVersion: 1, ...(kind === "codex" ? { workspaceFileReadVersion: 1 } : {}), taskId: "task", runner: { version: "fixture", digest: `sha256:${"a".repeat(64)}` } };
|
||||
|
||||
@@ -127,6 +127,7 @@ import type {
|
||||
import {
|
||||
NativeProviderTerminalFailure,
|
||||
NativeSessionCleanupQuarantinedError,
|
||||
NativeSessionCloseUnrecoverableError,
|
||||
NativeSessionProtocolIntegrityError,
|
||||
completeRetainedNativeSessionCleanup,
|
||||
completeTerminatedLocalNativeSessionCleanup,
|
||||
@@ -581,6 +582,7 @@ type WarmNativeSession = {
|
||||
configDigest: string;
|
||||
configuredEnvironmentDigest: string;
|
||||
ownerScope: string;
|
||||
normalizedSessionId: string;
|
||||
companyId: string;
|
||||
environmentId: string | null;
|
||||
busy: boolean;
|
||||
@@ -589,18 +591,31 @@ type WarmNativeSession = {
|
||||
lastActivityAt: string;
|
||||
};
|
||||
|
||||
async function closeWarmNativeSession(entry: WarmNativeSession, reason: string, preserveInstructionsForRunId?: string) {
|
||||
async function closeWarmNativeSession(entry: WarmNativeSession, reason: string, options?: {
|
||||
preserveInstructionsForRunId?: string;
|
||||
discardUnsettledSession?: boolean;
|
||||
}) {
|
||||
// Revoke before awaiting process retirement/checkpoint IO.
|
||||
const stopping = entry.githubAccess?.stop();
|
||||
try {
|
||||
try { await entry.session.close({ reason }); }
|
||||
catch (error) {
|
||||
try { await entry.instructionWorkingCopy?.retirementFailed?.(); }
|
||||
catch (receiptError) { throw new AggregateError([error, receiptError], "Warm instruction retirement and receipt both failed"); }
|
||||
throw error;
|
||||
// A new conversation does not consume the old checkpoint. Its failed
|
||||
// suspension must not poison every later reset after Launcher has proved
|
||||
// that the exact task stopped and completed its credential handoff.
|
||||
// Other close errors can mean controller authority is still live.
|
||||
const retiredWithoutCheckpoint = options?.discardUnsettledSession === true
|
||||
&& error instanceof NativeSessionCloseUnrecoverableError
|
||||
&& error.semanticToolsSettled
|
||||
&& pluginTaskSessionOwners.get(entry.session)?.isRetired() === true;
|
||||
if (!retiredWithoutCheckpoint) {
|
||||
try { await entry.instructionWorkingCopy?.retirementFailed?.(); }
|
||||
catch (receiptError) { throw new AggregateError([error, receiptError], "Warm instruction retirement and receipt both failed"); }
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
await entry.instructionWorkingCopy?.collectStopped();
|
||||
if (entry.instructionCopy?.runId !== preserveInstructionsForRunId) await entry.instructionCopy?.collectStopped();
|
||||
if (entry.instructionCopy?.runId !== options?.preserveInstructionsForRunId) await entry.instructionCopy?.collectStopped();
|
||||
}
|
||||
finally { await stopping; }
|
||||
}
|
||||
@@ -1915,6 +1930,7 @@ function nativeSessionOwnerScope(
|
||||
async function retireSupersededWarmNativeSessions(
|
||||
ownerScope: string,
|
||||
nextSessionScope: string,
|
||||
nextSessionId: string,
|
||||
): Promise<void> {
|
||||
for (const [scope, entry] of warmNativeSessions) {
|
||||
if (scope === nextSessionScope || entry.ownerScope !== ownerScope) continue;
|
||||
@@ -1926,7 +1942,9 @@ async function retireSupersededWarmNativeSessions(
|
||||
entry.busy = true;
|
||||
entry.closeOnReleaseReason = "warm native session identity changed";
|
||||
try {
|
||||
await closeWarmNativeSession(entry, entry.closeOnReleaseReason);
|
||||
await closeWarmNativeSession(entry, entry.closeOnReleaseReason, {
|
||||
discardUnsettledSession: entry.normalizedSessionId !== nextSessionId,
|
||||
});
|
||||
forgetWarmNativeSession(scope, entry);
|
||||
} finally {
|
||||
// A failed close retains the owner and prevents launch. A later attempt
|
||||
@@ -7939,7 +7957,7 @@ export async function executePaperclipNativeSession(input: {
|
||||
}
|
||||
executingNativeOwnerScopes.set(ownerScope, ownerToken);
|
||||
await retireSupersededWarmNativeSessions(
|
||||
ownerScope, nativeSessionScopeKey(input.execution),
|
||||
ownerScope, nativeSessionScopeKey(input.execution), nativeSessionKey(input.execution),
|
||||
);
|
||||
if ((input.execution.schema === "paperclip.native-execution-input.v6" || input.execution.schema === "paperclip.native-execution-input.v8") && isCloudManagedInstance()
|
||||
&& (!input.useRunnerd || input.runnerExecutionTarget?.kind !== "remote"
|
||||
@@ -9025,7 +9043,7 @@ async function executePaperclipNativeSessionWithinScope(
|
||||
entry.busy = true;
|
||||
entry.closeOnReleaseReason = "warm native session configuration changed";
|
||||
try {
|
||||
await closeWarmNativeSession(entry, entry.closeOnReleaseReason, input.execution.binding.runId);
|
||||
await closeWarmNativeSession(entry, entry.closeOnReleaseReason, { preserveInstructionsForRunId: input.execution.binding.runId });
|
||||
forgetWarmNativeSession(warmSessionId, entry);
|
||||
} finally { entry.busy = false; }
|
||||
if (preparedCopy) throw new Error("native_instruction_preparation_configuration_changed");
|
||||
@@ -9481,6 +9499,7 @@ async function executePaperclipNativeSessionWithinScope(
|
||||
ownerScope: nativeSessionOwnerScope(
|
||||
input.execution, input.runnerExecutionTarget?.environmentId ?? null,
|
||||
),
|
||||
normalizedSessionId: nativeSessionKey(input.execution),
|
||||
companyId: input.execution.binding.companyId,
|
||||
environmentId:
|
||||
input.runnerExecutionTarget?.environmentId ?? null,
|
||||
|
||||
@@ -137,6 +137,39 @@ describe("plugin native Runner ownership", () => {
|
||||
expect(retained).toHaveBeenCalledWith(expect.objectContaining(requirements.identity));
|
||||
expect(task.mock.calls.map(([op]) => op.kind)).toEqual(["status", "read_state", "connection", "stop", "status", "read_state", "complete"]);
|
||||
});
|
||||
it.each(["settled", "unsettled"] as const)("proves retirement independently of %s checkpoint reuse", async settlement => {
|
||||
const { task } = provider();
|
||||
const retained = vi.fn(async () => {});
|
||||
const owner = createPluginRunnerTaskOwner({ task, projectIds: [], harness: "codex",
|
||||
adoptIdentity: requirements.identity, onSessionRetained: retained });
|
||||
const registration = await owner.registration({} as never, "run");
|
||||
expect(owner.isRetired()).toBe(false);
|
||||
await registration.release();
|
||||
expect(owner.isRetired()).toBe(false);
|
||||
await registration.checkpoint(settlement);
|
||||
expect(owner.isRetired()).toBe(true);
|
||||
expect(retained).toHaveBeenCalledTimes(settlement === "settled" ? 1 : 0);
|
||||
expect(task.mock.calls.map(([op]) => op.kind)).toEqual(settlement === "settled"
|
||||
? ["stop", "status", "read_state", "complete"] : ["stop", "status", "complete"]);
|
||||
});
|
||||
|
||||
it.each(["credentials", "read_state", "complete"])("does not prove retirement when %s obligations fail after execution stops", async failingOperation => {
|
||||
const { task } = provider();
|
||||
const perform = task.getMockImplementation()!;
|
||||
task.mockImplementation(async operation => {
|
||||
if (operation.kind === failingOperation) throw new Error("handoff failed");
|
||||
if (operation.kind === "credentials") return { kind: "credentials", taskId: "task", credential: null };
|
||||
return perform(operation);
|
||||
});
|
||||
const owner = createPluginRunnerTaskOwner({ task, projectIds: [], harness: "codex", accountIdentity: "account",
|
||||
adoptIdentity: requirements.identity, onCredentials: async () => {}, onSessionRetained: async () => {} });
|
||||
const registration = await owner.registration({} as never, "run");
|
||||
await registration.release();
|
||||
await expect(registration.checkpoint("settled")).rejects.toThrow("handoff failed");
|
||||
expect(await owner.driver.adoptExistingRunner!.isAlive()).toBe(false);
|
||||
expect(owner.isRetired()).toBe(false);
|
||||
});
|
||||
|
||||
it("submits only a prepared identity and configuration ID", async () => {
|
||||
const { task } = provider();
|
||||
const owner = createPluginRunnerTaskOwner({ projectAuthCheckVersion: 1, task, projectIds: [], harness: "codex", pollMs: 1 });
|
||||
|
||||
@@ -59,6 +59,7 @@ export function createPluginRunnerTaskOwner(input: {
|
||||
let failReady!: (error: unknown) => void;
|
||||
let confirmStopped: (code: number) => void = () => {};
|
||||
let stoppedState: Extract<PluginEnvironmentTaskResult, { kind: "status" }> | undefined;
|
||||
let completed = false;
|
||||
let rejectFailure!: (error: unknown) => void;
|
||||
const failure = new Promise<never>((_resolve, reject) => { rejectFailure = reject; });
|
||||
void failure.catch(() => undefined);
|
||||
@@ -284,7 +285,10 @@ export function createPluginRunnerTaskOwner(input: {
|
||||
}
|
||||
await input.onSessionRetained(state);
|
||||
}
|
||||
if (submitted) await call({ kind: "complete" });
|
||||
if (submitted) {
|
||||
await call({ kind: "complete" });
|
||||
completed = true;
|
||||
}
|
||||
},
|
||||
release: async (reason?: "detach") => {
|
||||
registrationReleased = true;
|
||||
@@ -328,6 +332,9 @@ export function createPluginRunnerTaskOwner(input: {
|
||||
return bytes;
|
||||
};
|
||||
return { launcher, registration, driver, readWorkspaceFile,
|
||||
// This is a lease-bound Launcher receipt, not a terminal phase or a sent
|
||||
// stop request. Completion also proves credential/writeback obligations ran.
|
||||
isRetired: () => stoppedState?.executionStopped === true && completed && !detached,
|
||||
rebind(next: typeof input) {
|
||||
if (!launched || stopRequested || stoppedState || detached || next.harness !== input.harness ||
|
||||
input.accountIdentity !== next.accountIdentity || input.credentialIdentity !== next.credentialIdentity ||
|
||||
|
||||
+2
@@ -96,6 +96,8 @@ export const runnerCodexDynamicToolsFit = runner.runnerCodexDynamicToolsFit;
|
||||
export const inspectWarmRunTransition = runner.inspectWarmRunTransition;
|
||||
export const readRunnerdArtifactBinding = runner.readRunnerdArtifactBinding;
|
||||
export const readLinuxProcessStartedAt = runner.readLinuxProcessStartedAt;
|
||||
export const NativeSessionCloseUnrecoverableError =
|
||||
runner.NativeSessionCloseUnrecoverableError;
|
||||
export const NativeSessionCleanupQuarantinedError =
|
||||
runner.NativeSessionCleanupQuarantinedError;
|
||||
export const NativeSessionProtocolIntegrityError =
|
||||
|
||||
Reference in new issue
Block a user