From 5d8d9bf83e782a962bcee9da9202adeefeecdb2c Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Thu, 5 Nov 2026 07:00:23 -0600 Subject: [PATCH] fix: share native lifecycle policy with instruction checkpoints (#19230) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Warm native Codex turns save registered agent files at each turn boundary. > - Native execution can inherit warm mode from the agent when the environment has no explicit policy. > - File preparation checked only the environment policy and could miss that inheritance. > - The new copy then had no warm checkpoint callback, even though the native session stayed warm. > - This pull request resolves one admitted policy before both owners use it. > - The benefit is a terminal file checkpoint under the selected lifecycle, with the existing ownership and permission fences. ## Linked Issues or Issue Description Refs #14735 and #19225. The first defines warm file checkpoints. The second retains coordinator ownership while cleanup joins. This change fixes the earlier lifecycle selection mismatch. **What happened?** With no explicit environment policy and an agent warm default, native selection chose warm mode with a 900,000 ms idle window. Actual file preparation registered an agent_files copy without its warm flag or checkpoint callback. An unchanged turn could have no save marker. Later retirement of a retained lease could reach the stopped-file retrieval guard and preserve pending collection debt. **Expected behavior** File preparation and native execution use the same admitted lifecycle policy. Warm native Codex copies receive their terminal checkpoint callback. Explicit per-turn overrides and pinned admissions keep their existing precedence. A pending historical copy does not become saved because policy resolution changed. **Steps to reproduce** 1. Configure a native Codex agent with a warm default and a reusable sandbox without an explicit runner lifecycle policy. 2. Prepare its managed agent-file copy through the normal instruction owner. 3. Select native execution through the normal runtime owner. 4. Compare the selected warm policy with the copy's missing warm flag and checkpoint callback. **Paperclip version or commit** Base: `3a090b815b`. The owning policy code also exists on its parent revision. **Deployment mode** Native Codex with registered agent files and a reusable sandbox that inherits its lifecycle policy. ## What Changed - Move the existing lifecycle precedence and backend gates into a narrow shared resolver. - Resolve one value before instruction preparation and pass it into runtime selection. - Select warm file ownership from that value and the existing file capability gate. - Add real-Postgres unchanged and changed terminal checkpoint cases, policy precedence cases and updated preparation coverage. - Exercise the same resolve, timed preparation and runtime-selection orchestration used by heartbeat; preserve preparation error identity and skip selection on preparation failure. - Document new-admission overrides and pinned-policy behavior. ## Verification - Red: actual preparation and runtime-selection producers disagree. The selected native mode is warm, but the real persisted copy has no warm flag. The fixture uses real PostgreSQL, managed-root authorization and a synthetic loopback transport. - Green: 85 owner tests pass. They include actual unchanged and changed terminal checkpoints plus 19 lifecycle hierarchy/capability cases. Changed synthetic bytes reach canonical storage with a saved receipt; unchanged bytes record unchanged. - Safety: 25 existing warm retirement and stopped/destroyed/foreign-owner cases pass. Six authorization, stale-adoption and adopted-owner preservation cases pass. The lease guard and native executor are unchanged. - Workspace typecheck, build and map checks pass. The final candidate's code and test blobs equal the tested source; its exact build stamp is recorded locally. - No new repository-wide local run was repeated. A separate earlier broad command on an older head failed a reproducible GitHub enrollment fixture; it is not a full local pass, established flake or proven caused by this change. Fresh full hosted CI is green on the reviewed final head; both current-head Greptile and native substantive reviews are 5/5 with no actionable findings. - Review regression: removing actual policy propagation to preparation in the shared production orchestration fails both real-Postgres cases while runtime selection still chooses warm mode. Restoring it passes. These cases invoke the production helper used by heartbeat, not the entire outer heartbeat service. - The provider-facing transport is a local loopback fixture. No model turn, actual remote platform, physical stop receipt or deployed recovery is proved by these tests. ## Risks - New Codex copies that inherit agent warm mode now use the existing terminal checkpoint path. This can add the expected file checkpoint where only an unchanged probe ran before. - The resolver preserves explicit environment per-turn overrides, pinned admissions and backend/credential gates. It does not create a timer, provider retry or new permission. - Existing pending copies remain pending. This change does not authorize retrieval from stopped or replaced leases and does not establish recovery of historical files. - No database migration, public API field, agent setting, provider change or manual rollout is included. ## Model Used OpenAI Codex, `gpt-6.1-sol`, with reasoning, source inspection and local code execution. Context window and reasoning effort are host-managed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- doc/agent-files.md | 6 ++ server/src/services/heartbeat.ts | 13 ++- .../instruction-working-copy.test.ts | 5 + .../heartbeat/instruction-working-copy.ts | 8 +- .../heartbeat/native-lifecycle.test.ts | 34 +++++++ .../services/heartbeat/native-lifecycle.ts | 28 ++++++ .../heartbeat/runtime-selection.test.ts | 92 ++++++++++++++++++- .../services/heartbeat/runtime-selection.ts | 57 +++++++----- 8 files changed, 207 insertions(+), 36 deletions(-) create mode 100644 server/src/services/heartbeat/native-lifecycle.test.ts create mode 100644 server/src/services/heartbeat/native-lifecycle.ts diff --git a/doc/agent-files.md b/doc/agent-files.md index 7e3bcd4b1a0..92bea64d590 100644 --- a/doc/agent-files.md +++ b/doc/agent-files.md @@ -151,6 +151,12 @@ provider; restart recovery preserves files until retirement is proven. the host. Retry an unstable checkpoint up to three times; if it cannot be validated, close the owned provider and perform the stopped collector. Other execution paths retain their stopped-provider collection boundary. + + File preparation uses the same admitted lifecycle policy as native execution. + An inherited environment policy can use the agent's warm default. For a new + admission, an explicit per-turn environment policy still wins. A persisted + admission keeps its pinned policy. A policy change does not mark an old + pending copy saved or grant access to a stopped or replaced lease. 4. Recheck the responsible user's current authorization. Under the same agent lock used by editor writes, apply only files changed or deleted relative to the last acknowledged baseline. For a competing edit or deletion of the same diff --git a/server/src/services/heartbeat.ts b/server/src/services/heartbeat.ts index 2f68698ef1f..d59865fcc80 100644 --- a/server/src/services/heartbeat.ts +++ b/server/src/services/heartbeat.ts @@ -44,7 +44,7 @@ import { prepareHeartbeatWorkspace } from "./heartbeat/workspace-preparation.js" import { executeHeartbeatRuntime, NativeSessionResumeScheduledError, NativeWorkspaceFinalizeScheduledError } from "./heartbeat/runtime-execution.js"; import { preserveNativeFailedWorkspace } from "./native-runtime/native-failed-workspace-recovery.js"; import { nativeRunHasNoWorkspaceAccess } from "./workspace-restore-recovery-state.js"; -import { selectHeartbeatRuntime } from "./heartbeat/runtime-selection.js"; +import { prepareAndSelectHeartbeatRuntime } from "./heartbeat/runtime-selection.js"; import { createHeartbeatRunCompletion } from "./heartbeat/run-completion.js"; export { MAX_TURN_CONTINUATION_WAKE_REASON, @@ -3319,8 +3319,7 @@ export function heartbeatService( const hasInstructionFilesystem = nativeRuntimeResolution.kind !== "native" ? adapter.supportsInstructionsBundle === true : nativeRuntimeHasInstructionFilesystem(nativeRuntimeResolution.profile.backend); - runtimeConfig = await timePreparation("instructions", () => preparedInstructions.prepare({ runtimeConfig, nativeRuntimeResolution, hasInstructionFilesystem })); - const runtimeSelection = await timePreparation("runtime_selection", () => selectHeartbeatRuntime(db, { + const runtimeSelection = await prepareAndSelectHeartbeatRuntime(db, { run, agent, nativeRuntimeResolution, @@ -3388,7 +3387,13 @@ export function heartbeatService( nativeOwnershipHeld = true; }, stopControllerLease: () => controllerLease.stop(), - })); + }, { + measure: timePreparation, + prepareInstructions: async nativeLifecyclePolicy => { + runtimeConfig = await preparedInstructions.prepare({ runtimeConfig, nativeRuntimeResolution, hasInstructionFilesystem, nativeLifecyclePolicy }); + return { runtimeConfig, instructionCopy: preparedInstructions.instructionCopy }; + }, + }); if (!runtimeSelection.selected) return; const { nativeExecution, nativeRunnerInstanceId, getNativeFreshSessionHandoff } = runtimeSelection; nativeWorkspaceSync = runtimeSelection.nativeWorkspaceSync; diff --git a/server/src/services/heartbeat/instruction-working-copy.test.ts b/server/src/services/heartbeat/instruction-working-copy.test.ts index b29d226b49e..07a01c8bb44 100644 --- a/server/src/services/heartbeat/instruction-working-copy.test.ts +++ b/server/src/services/heartbeat/instruction-working-copy.test.ts @@ -2,6 +2,7 @@ import { beforeEach, describe, expect, it, vi } from "vitest"; import { createHeartbeatInstructionWorkingCopy } from "./instruction-working-copy.js"; import { AGENT_FILES_CONTRACT } from "../agent-file-store.js"; import { AgentDirectoryReuseInvalidatedError } from "../agent-directory-working-copies.js"; +import { resolveHeartbeatNativeLifecyclePolicy } from "./native-lifecycle.js"; const native = vi.hoisted(() => ({ reserve: vi.fn(), claim: vi.fn() })); vi.mock("../native-runtime/index.js", async importOriginal => ({ @@ -39,6 +40,10 @@ function fixture(executionTarget: Input["workspace"]["executionTarget"] = null, appendRunEvent, }); const prepare: Prepare = { runtimeConfig: { retained: true }, hasInstructionFilesystem: true, + nativeLifecyclePolicy: resolveHeartbeatNativeLifecyclePolicy({ adapterType: acpxAgent ? "paperclip_runner" : "codex_local", + backend: acpxAgent ? "acpx_runtime" : "codex_app_server", acpxAgent, agentMode: "warm", agentIdleTimeoutMs: undefined, + environmentPolicy: executionTarget?.kind === "remote" && executionTarget.transport === "sandbox" ? executionTarget.runnerLifecyclePolicy : null, + managedAiSelected: false, taskOwned: Boolean(executionTarget?.kind === "remote" && executionTarget.transport === "sandbox" && executionTarget.runnerTask) }), nativeRuntimeResolution: { kind: "native", profile: { backend: acpxAgent ? "acpx_runtime" : "codex_app_server" } } as Prepare["nativeRuntimeResolution"] }; const reservation = { reuseRunId: "previous-run", adopt: vi.fn(), release: vi.fn().mockResolvedValue(undefined) }; native.reserve.mockResolvedValue(reservation); diff --git a/server/src/services/heartbeat/instruction-working-copy.ts b/server/src/services/heartbeat/instruction-working-copy.ts index 449dcbd53cf..f57dbb449bb 100644 --- a/server/src/services/heartbeat/instruction-working-copy.ts +++ b/server/src/services/heartbeat/instruction-working-copy.ts @@ -13,6 +13,7 @@ import { supportsNativeWarmFiles } from "../native-runtime/native-warm-capabilit import type { HeartbeatRuntimeSelectionInput } from "./runtime-selection.js"; import type { createHeartbeatLifecycle } from "./run-lifecycle.js"; import type { EffectiveRunSessionConfigMetadata } from "./workspaces.js"; +import type { NativeLifecyclePolicy } from "./native-lifecycle.js"; type Copies = ReturnType; type Selection = HeartbeatRuntimeSelectionInput; @@ -123,10 +124,11 @@ export function createHeartbeatInstructionWorkingCopy(input: InstructionWorkingC retirementFailed: async () => { await instructionCopies.reportRetirementUnconfirmed(agent.companyId, readRun().id); }, } : undefined; - async function prepare({ runtimeConfig, nativeRuntimeResolution, hasInstructionFilesystem }: { + async function prepare({ runtimeConfig, nativeRuntimeResolution, hasInstructionFilesystem, nativeLifecyclePolicy }: { runtimeConfig: Record; nativeRuntimeResolution: Selection["nativeRuntimeResolution"]; hasInstructionFilesystem: boolean; + nativeLifecyclePolicy: NativeLifecyclePolicy | null; }) { if (hasInstructionFilesystem && !task) { try { @@ -141,9 +143,7 @@ export function createHeartbeatInstructionWorkingCopy(input: InstructionWorkingC const priorWorkingCopy = parseObject(parseObject(parseObject(priorFileInput.runtimeContext).instructions).workingCopy); const warmFiles = nativeRuntimeResolution.kind === "native" && (supportsNativeWarmFiles({ ...nativeRuntimeResolution.profile, acpxAgent: parseObject(agent.adapterConfig).acpxAgent }) || (nativeRuntimeResolution.profile.backend === "acpx_runtime" && parseObject(agent.adapterConfig).acpxAgent === "cursor")) && - (executionTarget?.kind === "remote" && executionTarget.transport === "sandbox" - ? executionTarget.runnerLifecyclePolicy?.mode === "warm" - : parseObject(agent.adapterConfig).lifecycleMode === "warm"); + nativeLifecyclePolicy?.mode === "warm"; if (warmFiles && taskSessionForRun?.lastRunId) { nativeInstructionReservation = await reserveWarmNativeInstructionDirectory({ companyId: agent.companyId, agentId: agent.id, previousRunId: taskSessionForRun.lastRunId, runId: readRun().id, target: executionTarget, diff --git a/server/src/services/heartbeat/native-lifecycle.test.ts b/server/src/services/heartbeat/native-lifecycle.test.ts new file mode 100644 index 00000000000..b64b38a3d74 --- /dev/null +++ b/server/src/services/heartbeat/native-lifecycle.test.ts @@ -0,0 +1,34 @@ +import { describe, expect, it } from "vitest"; +import { resolveHeartbeatNativeLifecyclePolicy, type NativeLifecyclePolicy } from "./native-lifecycle.js"; + +type Input = Parameters[0]; +const warm: NativeLifecyclePolicy = { mode: "warm", idleTimeoutMs: 900_000 }; +const perTurn: NativeLifecyclePolicy = { mode: "per_turn", idleTimeoutMs: null }; +const base: Input = { adapterType: "paperclip_runner", backend: "codex_app_server", agentMode: "warm", + agentIdleTimeoutMs: 900_000, managedAiSelected: false, taskOwned: false }; + +describe("admitted native lifecycle policy", () => { + it.each<{ name: string; change: Partial; expected: NativeLifecyclePolicy }>([ + { name: "absent environment inherits agent warm", change: {}, expected: warm }, + { name: "inherited environment inherits agent warm", change: { environmentPolicy: null }, expected: warm }, + { name: "local selection inherits agent warm", change: { environmentPolicy: undefined }, expected: warm }, + { name: "explicit environment per-turn overrides agent warm", change: { environmentPolicy: perTurn }, expected: perTurn }, + { name: "explicit environment warm overrides agent per-turn", change: { agentMode: "per_turn", environmentPolicy: warm }, expected: warm }, + { name: "admitted per-turn stays pinned", change: { persistedPolicy: perTurn, environmentPolicy: warm }, expected: perTurn }, + { name: "admitted warm stays pinned", change: { persistedPolicy: warm, environmentPolicy: perTurn }, expected: warm }, + { name: "default agent is per-turn", change: { agentMode: undefined }, expected: perTurn }, + { name: "managed Codex supports warm", change: { managedAiSelected: true }, expected: warm }, + { name: "managed process provider without warm support is per-turn", change: { backend: "acpx_runtime", acpxAgent: "claude", managedAiSelected: true, environmentPolicy: warm }, expected: perTurn }, + { name: "unmanaged ACP retains its prior warm policy", change: { backend: "acpx_runtime", acpxAgent: "claude" }, expected: warm }, + { name: "Copilot API-key ownership supports warm", change: { backend: "acpx_runtime", acpxAgent: "copilot", managedAiSelected: true, + managedAiAttribution: { provider: "github", method: "api_key" } }, expected: warm }, + { name: "unqualified Copilot ownership remains per-turn", change: { backend: "acpx_runtime", acpxAgent: "copilot", managedAiSelected: true, + managedAiAttribution: { provider: "github", method: "subscription" } }, expected: perTurn }, + { name: "non-native adapter does not gain managed warm support", change: { adapterType: "codex_local", managedAiSelected: true }, expected: perTurn }, + { name: "task-owned provider keeps its existing ownership gate", change: { backend: "acpx_runtime", acpxAgent: "claude", managedAiSelected: true, taskOwned: true }, expected: warm }, + ...(["openai_dot_mcp", "grok_bot_mcp", "muse_external", "cursor_cloud_api"] as const).map(backend => + ({ name: `${backend} remains per-turn`, change: { backend, environmentPolicy: warm }, expected: perTurn })), + ])("$name", ({ change, expected }) => { + expect(resolveHeartbeatNativeLifecyclePolicy({ ...base, ...change })).toEqual(expected); + }); +}); diff --git a/server/src/services/heartbeat/native-lifecycle.ts b/server/src/services/heartbeat/native-lifecycle.ts new file mode 100644 index 00000000000..ca7c2fe659a --- /dev/null +++ b/server/src/services/heartbeat/native-lifecycle.ts @@ -0,0 +1,28 @@ +import { resolvePaperclipRunnerIdleTimeoutMs } from "@paperclipai/adapter-utils"; +import type { NativeExecutionInput } from "../../vendor/paperclip-runner/index.js"; +import { supportsManagedNativeWarmSession } from "../native-runtime/native-warm-capability.js"; + +export type NativeLifecyclePolicy = NonNullable; + +export function resolveHeartbeatNativeLifecyclePolicy(input: { + adapterType: string; + backend: Parameters[0]["backend"]; + acpxAgent?: unknown; + agentMode: unknown; + agentIdleTimeoutMs: unknown; + environmentPolicy?: NativeLifecyclePolicy | null; + persistedPolicy?: NativeLifecyclePolicy | null; + managedAiSelected: boolean; + managedAiAttribution?: { provider: string; method: string }; + taskOwned: boolean; +}): NativeLifecyclePolicy { + const agentPolicy: NativeLifecyclePolicy = input.agentMode === "warm" + ? { mode: "warm", idleTimeoutMs: resolvePaperclipRunnerIdleTimeoutMs(input.agentIdleTimeoutMs) } + : { mode: "per_turn", idleTimeoutMs: null }; + const supportsManagedWarm = input.adapterType === "paperclip_runner" && (input.taskOwned || + supportsManagedNativeWarmSession({ backend: input.backend, acpxAgent: input.acpxAgent }, input.managedAiAttribution)); + return input.persistedPolicy ?? + (["openai_dot_mcp", "grok_bot_mcp", "muse_external", "cursor_cloud_api"].includes(input.backend) || input.managedAiSelected && !supportsManagedWarm + ? { mode: "per_turn", idleTimeoutMs: null } + : input.environmentPolicy ?? agentPolicy); +} diff --git a/server/src/services/heartbeat/runtime-selection.test.ts b/server/src/services/heartbeat/runtime-selection.test.ts index b8f8697c386..5c027ea3060 100644 --- a/server/src/services/heartbeat/runtime-selection.test.ts +++ b/server/src/services/heartbeat/runtime-selection.test.ts @@ -1,8 +1,10 @@ import { randomUUID } from "node:crypto"; -import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { execFile as execFileCallback } from "node:child_process"; +import { promisify } from "node:util"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import { activityLog, agents, companies, completionContracts, createDb, applyPendingMigrations, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS, executionWorkspaces, heartbeatRuns, issues, nativeRunFinalizations, workspaceOperations, type Db } from "@paperclipai/db"; +import { activityLog, agents, authUsers, companyMemberships, principalPermissionGrants, environments, environmentLeases, companies, completionContracts, createDb, applyPendingMigrations, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS, executionWorkspaces, heartbeatRuns, issues, nativeRunFinalizations, workspaceOperations, type Db } from "@paperclipai/db"; import { eq, sql } from "drizzle-orm"; import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest"; import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js"; @@ -14,9 +16,12 @@ import { prepareNativeWorkspaceSync } from "../native-runtime/native-workspace-s import { resolveHeartbeatNativeRuntimeMode } from "../native-runtime/runtime-mode.js"; import { claimNativeRestartRecoveries } from "../native-runtime/native-restart-recovery.js"; import { buildNativeModelEnvelope, canonicalNativeRuntimeContextDigest } from "../../vendor/paperclip-runner/index.js"; -import { selectHeartbeatRuntime, type HeartbeatRuntimeSelectionInput } from "./runtime-selection.js"; +import { prepareAndSelectHeartbeatRuntime, selectHeartbeatRuntime, type HeartbeatRuntimeSelectionInput } from "./runtime-selection.js"; import { reserveRunBudget } from "../budget-reservations.js"; import { legacyControllerClaim } from "../legacy-controller-lease.js"; +import { agentInstructionWorkingCopyService } from "../agent-instruction-working-copies.js"; +import { createHeartbeatInstructionWorkingCopy } from "./instruction-working-copy.js"; +import { environmentService } from "../environments.js"; // Keep the real contract, session, and runtime-selection transactions. Replace only // filesystem materialization and sandbox synchronization at their external boundaries. @@ -82,6 +87,87 @@ describe.skipIf(!support.supported)("heartbeat runtime selection boundary", () = const coordinators = (id: string) => db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, id)); const sandbox = { kind: "remote" as const, transport: "sandbox" as const, remoteCwd: "/workspace", reusableLeaseConfigured: false }; + it("preserves preparation failure and skips runtime selection", async () => { + const { input, run } = await fixture(); + const initiating = new Error("Synthetic preparation failure"); + const phases: string[] = []; + await expect(prepareAndSelectHeartbeatRuntime(db, input, { + measure: async (phase, work) => { phases.push(phase); return work(); }, + prepareInstructions: async () => { throw initiating; }, + })).rejects.toBe(initiating); + expect(phases).toEqual(["instructions"]); + expect(await coordinators(run.id)).toEqual([]); + expect(buildNativeRuntimeContext).not.toHaveBeenCalled(); + }); + + it.each([false, true])("checkpoints the real instruction copy under the selected agent-default warm policy (changed: %s)", async changed => { + const { input, issue, agent, run } = await fixture(); + const userId = randomUUID(), environmentId = randomUUID(), leaseId = randomUUID(); + vi.stubEnv("PAPERCLIP_HOME", instructionRoot); + const { resolveManagedInstructionsRoot } = await import("../agent-instructions.js"); + const canonical = resolveManagedInstructionsRoot(agent), remoteCwd = join(instructionRoot, randomUUID(), "task"); + await mkdir(canonical, { recursive: true }); await mkdir(remoteCwd, { recursive: true }); + await writeFile(join(canonical, "AGENTS.md"), "Synthetic instructions for this local fixture only.\n"); + const execute = promisify(execFileCallback); + await execute("git", ["init", remoteCwd]); + const target = { kind: "remote" as const, transport: "sandbox" as const, environmentId, leaseId, remoteCwd, + runnerLifecyclePolicy: null, reusableLeaseConfigured: true, effectiveCapabilities: { reusableLeases: true } as never, + runner: { execute: async (command: { command: string; args?: string[]; cwd?: string; env?: Record; stdin?: string; timeoutMs?: number }) => { + const args = [...(command.args ?? [])], env = { PATH: process.env.PATH, ...command.env }; + if (command.stdin != null && (args[0] === "-c" || args[0] === "-lc")) { + Object.assign(env, { PAPERCLIP_TEST_STDIN: command.stdin }); + args[1] = `printf '%s' "$PAPERCLIP_TEST_STDIN" | (${args[1]})`; + } + const result = await execute(command.command, args, { cwd: command.cwd, env, timeout: command.timeoutMs, maxBuffer: 32 * 1024 * 1024 }); + return { exitCode: 0, signal: null, timedOut: false, ...result, pid: null, startedAt: new Date().toISOString() }; + } } }; + await db.insert(authUsers).values({ id: userId, name: "Fixture editor", email: `${userId}@example.test`, createdAt: new Date(), updatedAt: new Date() }); + await db.insert(companyMemberships).values([{ companyId: agent.companyId, principalType: "user", principalId: userId, membershipRole: "operator" }, + { companyId: agent.companyId, principalType: "agent", principalId: agent.id, membershipRole: "member" }]); + await db.insert(principalPermissionGrants).values({ companyId: agent.companyId, principalType: "user", principalId: userId, permissionKey: "agents:configure", scope: { agentIds: [agent.id] } }); + await db.insert(environments).values({ id: environmentId, name: `Loopback sandbox ${environmentId}`, driver: "sandbox" }); + const [lease] = await db.insert(environmentLeases).values({ id: leaseId, companyId: agent.companyId, environmentId, heartbeatRunId: run.id, + provider: "fixture", providerLeaseId: "local-loopback", leasePolicy: "reuse_by_environment" }).returning(); + input.agent = { ...agent, adapterConfig: { ...agent.adapterConfig, lifecycleMode: "warm", idleTimeoutMs: 900_000, + instructionsBundleMode: "managed", instructionsRootPath: canonical, instructionsEntryFile: "AGENTS.md" } }; + await db.update(agents).set({ adapterConfig: input.agent.adapterConfig }).where(eq(agents.id, agent.id)); + await db.update(heartbeatRuns).set({ responsibleUserId: userId }).where(eq(heartbeatRuns.id, run.id)); + input.run = await read(run.id); + input.workspace.executionTarget = target; + input.workspace.executionWorkspace.cwd = remoteCwd; + input.workspace.activeEnvironmentLease = { lease: (await environmentService(db).getLeaseById(lease.id))! }; + input.nativeRuntimeResolution = resolveHeartbeatNativeRuntimeMode({ persisted: input.run, enabled: true, runtimeConfig: {}, adapterConfig: input.agent.adapterConfig, + agent: input.agent, issue, target, workspaceId: null }); + expect(input.nativeRuntimeResolution.kind).toBe("native"); + const { agentInstructionsBundleMode } = await import("../agent-instructions.js"); + expect(agentInstructionsBundleMode(input.agent)).toBe("managed"); + const copies = agentInstructionWorkingCopyService(db); + const instructionEvents = vi.fn(async () => {}); + const controller = createHeartbeatInstructionWorkingCopy({ db, instructionCopies: copies, readRun: () => input.run, agent: input.agent, + context: input.task.context, session: { taskSession: null, taskSessionForRun: null, sessionConfigMetadata: {} as never }, + workspace: input.workspace, appendRunEvent: instructionEvents }); + const phases: string[] = []; + const selected = await prepareAndSelectHeartbeatRuntime(db, input, { + measure: async (phase, work) => { phases.push(phase); return work(); }, + prepareInstructions: async nativeLifecyclePolicy => { + const runtimeConfig = await controller.prepare({ runtimeConfig: {}, nativeRuntimeResolution: input.nativeRuntimeResolution, hasInstructionFilesystem: true, nativeLifecyclePolicy }); + return { runtimeConfig, instructionCopy: controller.instructionCopy }; + }, + }); + expect(phases).toEqual(["instructions", "runtime_selection"]); + expect(instructionEvents.mock.calls).toEqual([]); + expect(controller.instructionCopy).not.toBeNull(); + if (!selected.selected || !selected.nativeExecution) throw new Error("Expected native selection"); + expect(selected.nativeExecution.session.lifecyclePolicy).toEqual({ mode: "warm", idleTimeoutMs: 900_000 }); + expect((await copies.get(agent.companyId, run.id))?.receipt?.warm).toBe(true); + if (changed) await writeFile(join(controller.instructionCopy!.executionRoot, "memory.txt"), "Synthetic changed memory.\n"); + expect(await controller.nativeInstructionWorkingCopy()!.checkpointWarm!()).toBe(true); + expect((await copies.get(agent.companyId, run.id))?.state).toBe("warm_saved"); + expect(instructionEvents).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({ eventType: "instruction_save", + payload: expect.objectContaining({ state: changed ? "saved" : "unchanged", contract: "agent_files", errorCode: null }) })); + if (changed) expect(await readFile(join(canonical, "memory.txt"), "utf8")).toBe("Synthetic changed memory.\n"); + }, 45_000); + it.each(["matching", "different-session", "absent"])("persists the migrated logical cwd only for the receipt's exact session: %s", async receipt => { const sameSession = receipt === "matching"; const { input, run, agent, issue } = await fixture(); diff --git a/server/src/services/heartbeat/runtime-selection.ts b/server/src/services/heartbeat/runtime-selection.ts index 20d3c40ac8d..a8f2239e1cd 100644 --- a/server/src/services/heartbeat/runtime-selection.ts +++ b/server/src/services/heartbeat/runtime-selection.ts @@ -15,7 +15,7 @@ import { isAgentDirectoryCopy } from "../agent-directory-working-copies.js"; import { isConversation } from "../agent-conversations.js"; import { renewLegacyControllerLease } from "../legacy-controller-lease.js"; import type { prepareManagedAiRuntime } from "../ai-connection-runtime.js"; -import { supportsManagedNativeWarmSession } from "../native-runtime/native-warm-capability.js"; +import { resolveHeartbeatNativeLifecyclePolicy, type NativeLifecyclePolicy } from "./native-lifecycle.js"; import { getNativeReviewAssignment, readNativeReviewAssignmentContext, @@ -82,7 +82,6 @@ import { CHAT_CONTROL_RECOVERY_ADMISSION_KEY } from "../chat-control-recovery-st import { NATIVE_PREPARATION_FENCING_VERSION } from "../native-runtime/native-active-runs.js"; import type { issueService } from "../issues.js"; import { buildPlanReviewContext } from "../plan-review-context.js"; -import { resolvePaperclipRunnerIdleTimeoutMs } from "@paperclipai/adapter-utils"; import { normalizePaperclipWakePayload, selectPaperclipTaskMarkdown } from "@paperclipai/adapter-utils/server-utils"; import type { ProviderResourceDisposition } from "../environment-runtime.js"; @@ -132,6 +131,7 @@ export interface HeartbeatRuntimeSelectionInput { onPhase?: Parameters[0]["onPhase"]; }; config: { + nativeLifecyclePolicy?: NativeLifecyclePolicy | null; runtimeConfig: Record; runtimeSkillEntries: Parameters[0]["runtimeSkillEntries"]; runScopedMentionedSkillKeys: string[]; @@ -258,7 +258,35 @@ function readNonEmptyString(value: unknown): string | null { return typeof value === "string" && value.trim().length > 0 ? value : null; } -/** Prepare and persist the selected runtime before the executor dispatches provider work. */ +function resolveRuntimeLifecyclePolicy(input: HeartbeatRuntimeSelectionInput): NativeLifecyclePolicy | null { + if (input.nativeRuntimeResolution.kind !== "native") return null; + return resolveHeartbeatNativeLifecyclePolicy({ + adapterType: input.agent.adapterType, backend: input.nativeRuntimeResolution.profile.backend, + acpxAgent: parseObject(input.agent.adapterConfig).acpxAgent, + agentMode: parseObject(input.agent.adapterConfig).lifecycleMode, agentIdleTimeoutMs: parseObject(input.agent.adapterConfig).idleTimeoutMs, + environmentPolicy: input.workspace.executionTarget?.kind === "remote" && input.workspace.executionTarget.transport === "sandbox" ? input.workspace.executionTarget.runnerLifecyclePolicy : null, + persistedPolicy: input.session.persistedNativeExecutionInput?.session.lifecyclePolicy, + managedAiSelected: Boolean(input.config.managedAiRuntime), managedAiAttribution: input.config.managedAiRuntime?.attribution, + taskOwned: Boolean(input.workspace.executionTarget?.kind === "remote" && input.workspace.executionTarget.transport === "sandbox" && input.workspace.executionTarget.runnerTask), + }); +} + +export async function prepareAndSelectHeartbeatRuntime( + db: Db, + input: HeartbeatRuntimeSelectionInput, + preparation: { + prepareInstructions: (policy: NativeLifecyclePolicy | null) => Promise>; + measure: (phase: "instructions" | "runtime_selection", work: () => Promise) => Promise; + }, +): Promise { + const nativeLifecyclePolicy = resolveRuntimeLifecyclePolicy(input); + const prepared = await preparation.measure("instructions", () => preparation.prepareInstructions(nativeLifecyclePolicy)); + return preparation.measure("runtime_selection", () => selectHeartbeatRuntime(db, { + ...input, + config: { ...input.config, ...prepared, nativeLifecyclePolicy }, + })); +} + export async function selectHeartbeatRuntime(db: Db, input: HeartbeatRuntimeSelectionInput): Promise { const { run, agent, nativeRuntimeResolution, issuesSvc, runtimeEnv, beforeNativeRuntimeSelection, onAdapterEvent, onNativeLifecycleSelected, onProviderResourceDisposition, onNativeOwnershipHeld, stopControllerLease } = input; @@ -501,28 +529,7 @@ export async function selectHeartbeatRuntime(db: Db, input: HeartbeatRuntimeSele let nativeResumeCheckpoint: ReturnType< typeof rebindNativeSessionCheckpoint > = null; - const agentLifecyclePolicy = - parseObject(agent.adapterConfig).lifecycleMode === "warm" - ? { - mode: "warm" as const, - idleTimeoutMs: resolvePaperclipRunnerIdleTimeoutMs( - parseObject(agent.adapterConfig).idleTimeoutMs, - ), - } - : { mode: "per_turn" as const, idleTimeoutMs: null }; - const environmentLifecyclePolicy = - executionTarget?.kind === "remote" && - executionTarget.transport === "sandbox" - ? (executionTarget.runnerLifecyclePolicy ?? null) - : null; - // Task providers own credential lifetime; process-backed managed homes retain their provider-specific cleanup policy. - const supportsManagedWarmSession = agent.adapterType === "paperclip_runner" && - ((executionTarget?.kind === "remote" && executionTarget.transport === "sandbox" && !!executionTarget.runnerTask) || - supportsManagedNativeWarmSession({ ...nativeRuntimeResolution.profile, acpxAgent: parseObject(agent.adapterConfig).acpxAgent }, managedAiRuntime?.attribution)); - const effectiveLifecyclePolicy = persistedNativeExecutionInput?.session.lifecyclePolicy ?? - (["openai_dot_mcp", "grok_bot_mcp", "muse_external", "cursor_cloud_api"].includes(nativeRuntimeResolution.profile.backend) || managedAiRuntime && !supportsManagedWarmSession - ? { mode: "per_turn" as const, idleTimeoutMs: null } - : environmentLifecyclePolicy ?? agentLifecyclePolicy); + const effectiveLifecyclePolicy = input.config.nativeLifecyclePolicy ?? resolveRuntimeLifecyclePolicy(input)!; if ( effectiveLifecyclePolicy.mode === "warm" && executionTarget?.kind === "remote" &&