mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 08:22:20 +02:00
feat: integrate Pi 1.0 into the experimental Runner (#14921)
## Thinking Path > - Paperclip manages AI agents and their work. > - The experimental Runner owns provider processes and durable sessions. > - Pi needs working task execution and human controls. > - The five-PR stack must preserve changes already on master. > - Each layer now carries the complete integrated source for a safe sequential fallback. > - This PR belongs to native GitHub stack #15602, ending at #14956. ## Linked Issues or Issue Description Refs #14436, #14631, #14743 and #14956. Ship Pi 1.0 through the experimental Paperclip Runner. The five PRs are #14921, #14922, #14923, #14924 and #14956. The user authorized the complete merge after checks pass. Existing `pi_local` execution is unchanged. Accounting and wider provider/platform qualification remain deferred. ## What Changed - Recover missing final replies after workspace finalization changes owners, using accepted-turn evidence without rerunning work or granting external-chat publication. - Preserve the admitted Pi instruction root across warm runs, while retaining changed-root rejection. - Give Pi a bounded 15-second default shutdown grace so stop, drain acknowledgement and durable suspension can complete. Explicit deadlines and other providers retain their existing behavior. - Integrate the Pi 1.0 runtime and master contracts. - Use Pi profile 22. Preserve explicit caller-selected models and exact native thinking levels. Keep Pi's wrapper, helper, extension and question/control behavior unchanged from the qualified profile-19 runtime. - Preserve master's Dot lifecycle and consent fields, configured task environment, status guards and current Codex/Claude dependency versions. Cursor stays qualified. Copilot stays pending; profile 17 binds the changed shared protocol validation sources. - Exclude general AWS IAM credentials from Pi static/custom provider bindings and selected task projections; preserve the provider-scoped Bedrock bearer key. Profile 21 is retained as historical provenance. Rust and cloud install probes use the current declaration. - Patch bundled brace-expansion 5.0.9 to the exact official 5.0.12 payload. Pin the patch and complete runtime closures. Include the patch in normal installed setup tooling. Keep the upstream Pi shrinkwrap as provenance and permit only this exact security correction. - Include current attestation files in the Docker build context. Keep the repository lockfile unchanged from master. CI and private image builds resolve manifest changes before their frozen installation. ## Verification - Full local `pnpm -r typecheck` passes, including Runner Rust, server and UI. Focused integration checks pass: 194 Runner admission/environment tests, 63 profile/credential tests with one expected skip, 152 Dot/UI configuration tests, and Pi transcript/notice tests. - Full local `pnpm build` passes on the final source. - Fresh final-source checks pass: all 698 Rust workspace tests (32 binaries), 156 credential/profile/controller tests with one expected skip, Runner TypeScript typecheck, and 20 package/setup/sandbox tests. - The profile-21 Pi materializer passes on the native host with the official pinned Node 24.21.0 and its npm. It verifies all 150 locked packages, the patched dependency and the exact closure. Setup/package bundle tests and UI token gates pass. - The old hashes were reproduced for all three supported targets before calculating the patched graph. New closure hashes are darwin-arm64 `282022db10150c6632b3444df421342e7d534bdf5d5fb1097a2e79d0625a2bcf`, darwin-x64 `64e251e19009f755c0b04f73ce2138246faab71a961b0f13d75ebfcc34bef12e`, and linux-x64 `713b1fdff42fb56a1518bdc084f181d70bee8ebadc3e4b1d76321ed9108c8410`. Independent native platform execution is separate from graph identity reproduction. - Historical cloud qualification remains unchanged: all seven core cases pass on shipping source `10dc43c9ec65d88c2f782d62afb296d09494f215`, harness `1a4408a48cfb5a1f094a311141c257c92cd7a893`, image `sha256:5b3a775b383591bda1b0c1889e509acc70ce7f37c53f09733c81d59037f02280`, and accepted Sonnet 4.6/low fixture. All 215 canonical files and all seven cleanup checks pass independent verification. These are profile-19 results and are not relabeled as fresh profile-22 runs. - Current Pi digest: `sha256:e92078bee3c23bec4100aa589013a44613d054cd686826534025d8019e9f39a9`. [The readiness plan](https://github.com/paperclipai/paperclip/blob/codex/pi-production-readiness/doc/plans/2026-10-02-pi-production-readiness.md) preserves campaign and failed-attempt provenance. - Merge only after every PR's current-head CI and fresh review pass. Linux CI covers the full suites, build and browser tests. The local embedded Postgres API-authority suite cannot start on this macOS/Node 26 host, so Linux CI must confirm that suite. ### Fresh profile-22 core qualification — 2026-10-08 All seven accepted core cases pass canonically on Pi profile 22, with `openrouter/anthropic/claude-sonnet-4.6` and native-confirmed low thinking. This model is a fixture; production accepts the caller's explicit Pi provider/model. Runtime/install source: `3241a992f2a7703e59e97ed0fd3e5d6405de4401`. Frozen accepted harness: `1a4408a48cfb5a1f094a311141c257c92cd7a893`. Immutable cloud image: `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:506f22db7edd78f37c0c40bec1cc084af1850455026dbf467194bfbb8fcef141`. Pi digest: `sha256:e92078bee3c23bec4100aa589013a44613d054cd686826534025d8019e9f39a9`. [Hosted Linux image and clean-install verification](https://github.com/paperclipai/paperclip/actions/runs/37868328023) passes, including all 20 source-bound archives, normal CLI/Pi setup, companion import and the production pack reader. This exact installation source includes the latest master integration and the corrected Pi warm instruction-root fence. Full local typecheck/build and current-head hosted CI verify the final stack. All 13 focused real-root regressions pass. The full local executor suite passed 662 tests; 15 database tests could not start the Mac embedded PostgreSQL service. Hosted Linux CI passes the full required verification and E2E checks. These fresh results keep their own source identity; profile-19 results remain historical. | Core path | Canonical campaign | Retained archive SHA-256 | | --- | --- | --- | | File edit, validation, download and Done | `pi-core22-replyfix-0-1791511228` | 23 files; `a473e8603a3dd4737863291f8d3d1e392391f0b16d433c3e0e0e9d8baf7a97b0` | | Pending question and controller restart | `pi-core22-replyfix-1-1791511376` | 33 files; `6b829c4eb74e1f32a89c692a4ae7130dbfc1c6d3cf13915effe2103d9e242c8e` | | Three-turn session/process/workspace continuity | `pi-core22-replyfix-2-1791511587` | 23 files; `7a87021f8f9a3fdd3c58bb4467f8d82c635e3ea4795d6e75f144d9aa14818df8` | | Four typed questions and browser reconnects | `pi-core22-replyfix-3-1791511881` | 42 files; `9e31755252be1f4f9cb0626c984c142d4d1ae5f5bee3a7af08444db8d12c280a` | | Plan approval and completion | `pi-core22-replyfix-4-1791512031` | 22 files; `a0383ce1aab38e7b5a25ce0e9dd3bebea5c037ebd96ae6b29dae19015da2ae2c` | | Same-turn steering and permission denial | `pi-core22-replyfix-5-1791512261` | 39 files; `c929b8c7070f0b66aedc17e65ca46e6beab1e363926ac9f7e2a75fb250f05949` | | Stop during pending permission | `pi-core22-replyfix-6-1791512390` | 33 files; `7f0a58ae0f4d5bfc76149435f4e322537089c5bd16e7ffe9b5ad71f10a621a07` | All 215 canonical files (28714587 bytes) are independently hash-verified. All seven cleanup grades pass, with no owned runtime process or temporary root after each case. Automatic retries are zero. The owned cloud host stopped normally after retention. The prior profile-22 warm attempt remains failed and separately retained: archive SHA-256 `1e54eba5ec72b50cee1534b23d1d1d4f21a090006b8a64501ba70db972abfde5`. Its original canonical classification is preserved. Diagnosis reproduced a product bug comparing an agent-files root against an unset checkpoint-only field. The fix stores the admitted physical root separately from the adopted per-run collection capability. The real-root regression fails before the fix and passes afterward, including rejection of a changed physical root. Fixture, grader, model and all seven accepted case IDs are unchanged; this fresh campaign tests final-reply publication after file registration first. The intermediate restart attempt also remains failed and retained: archive SHA-256 `5dcaefdf1d17cf4cd54fd4cf810f45e736667392339b8ce7caf08bb4e225277f`. Its original canonical classification is preserved. Pi resumed, wrote the verified answer and completed its task; exact runner suspension was proven, but idle stop consumed about 5.2s and left under 3s for the drain acknowledgement. The Pi-only default shutdown grace is now 15s, preserving a full 5s drain round trip and a finite suspension reserve. Explicit caller deadlines, other provider defaults, literal drain receipts and exact suspension identity checks remain unchanged. The timing regression fails before this correction and passes afterward; all 18 focused settlement tests and Runner typecheck pass. The final-source file attempt is also preserved as failed (`candidate_failure`), archive SHA-256 `db6767b6773ea618997927ac77bdb005a5ac81492c7b9c0ffbc900449f829bc9`. Native edit, validation, exact downloadable artifact and Done/succeeded all passed, and the exact final reply was durably recorded. A workspace recovery owner completed before the live heartbeat reached presentation, leaving that reply absent from task chat. Recovery now materializes only a completed final reply from the accepted turn of an ordinary internal Done task, preserving issue/run/contract binding, suppression, external-chat authorization and same-run deduplication. The database regression covers the generated file-preparation receipt, suppression, unapproved external continuation and replay. Server typecheck and all 49 response-selection tests pass; hosted Linux verifies the database regression because embedded PostgreSQL cannot start on this Mac. The delayed-final-answer database regression passes on [the final root-source Linux server shard](https://github.com/paperclipai/paperclip/actions/runs/37868262553/job/113628594152), alongside 1,108 passing tests. The first root Runner shard had one unchanged durable-resume test exceed its 5-second timeout; the identical top-source shard and the isolated exact test passed. One rerun of that failed job and its required aggregate passed without source or test changes. The original failed job log and the single-rerun receipt remain retained. ### October 9 merge verification Current merge head: `5a8fe63512a7166aaef5cf50065a25008aa8b44b`. All current-head checks pass, including `ci / verify` and `ci / e2e`; exact-head Greptile review is 5/5 with no unresolved threads. Current master conflicts are resolved. The user authorized the maintainer override of the code-owner review gate after these checks. The seven retained live core cases remain bound to source `3241a992f2a7703e59e97ed0fd3e5d6405de4401` and its recorded cloud image. ## Risks - The security correction changes the dependency closure and profile identity. Old sessions must reopen on the new profile. Exact identities and credential bindings fail closed. - The runner remains experimental and requires explicit selection. Legacy Pi Local is unchanged. Caller model IDs pass through; the E2E model is a fixture. - Accounting and the broad platform/provider matrix remain deferred. This merge does not publish a release or deploy a service. ## Model Used OpenAI GPT-6 through Codex assisted with reasoning, repository inspection, editing and tool use. The exact serving ID and context window are not exposed in this session. Final live qualification uses Pi 1.0.0 with `openrouter/anthropic/claude-sonnet-4.6` and native-confirmed low thinking. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed
1 parent
4fb1ab53a8
commit
57e977be72
517 files changed
+61737
-1440
No files matched your search
@@ -5,6 +5,7 @@ import { PgDialect } from "drizzle-orm/pg-core";
|
||||
import type { SQL } from "drizzle-orm";
|
||||
import {
|
||||
access,
|
||||
chmod,
|
||||
cp,
|
||||
lstat,
|
||||
mkdir,
|
||||
@@ -42,12 +43,18 @@ import {
|
||||
validatePrpEvent,
|
||||
parseNativeExecutionInput,
|
||||
type NativeExecutionInputV1,
|
||||
type NativeExecutionInputV3,
|
||||
type NativeExecutionInput,
|
||||
type PrpEvent,
|
||||
} from "@paperclipai/paperclip-runner";
|
||||
import { createHash, generateKeyPairSync } from "node:crypto";
|
||||
import { DatabaseSync } from "node:sqlite";
|
||||
import { nativeSha256 } from "./canonical.js";
|
||||
import { agentDirectoryWorkingCopyService } from "../agent-directory-working-copies.js";
|
||||
import { probeAgentDirectory } from "../agent-directory-probe.js";
|
||||
import { AGENT_FILES_CONTRACT } from "../agent-file-store.js";
|
||||
import { captureDirectorySnapshot, directorySnapshotSha256, serializeDirectorySnapshot } from "@paperclipai/adapter-utils/workspace-restore-merge";
|
||||
import { agentInstructionWorkingCopies } from "@paperclipai/db";
|
||||
import * as noLaunchProofModule from "./native-maintenance-no-launch.js";
|
||||
import {
|
||||
NativeSessionCleanupQuarantinedError,
|
||||
@@ -63,6 +70,7 @@ import {
|
||||
} from "./native-harness-backup-stamp.js";
|
||||
import { nativeRuntimeContextFixture } from "./runtime-context.test-fixture.js";
|
||||
import { nativeToolContractFingerprintForTarget } from "./native-session-resume.js";
|
||||
import * as nativeSessionResume from "./native-session-resume.js";
|
||||
import { buildNativeHeartbeatPreparationSpans } from "./native-run-trace.js";
|
||||
import { NativeRunnerOwnershipUnverifiedError } from "./native-runner-ownership.js";
|
||||
import type { AdapterRuntimeEvent } from "../../adapters/index.js";
|
||||
@@ -297,6 +305,8 @@ import {
|
||||
buildNativeHarnessBackupManifest,
|
||||
cancelNativeSession,
|
||||
closeWarmNativeSessionsForEnvironment,
|
||||
claimWarmNativeInstructionCopy,
|
||||
nativeSessionWorkspaceScope,
|
||||
reserveWarmNativeInstructionDirectory,
|
||||
closeIdleWarmNativeSessionsForRestart,
|
||||
createGovernedWaitEventObservation,
|
||||
@@ -1252,22 +1262,43 @@ describe("remote provider pack manifest", () => {
|
||||
const candidatePath = "provider-assets/pi/linux-x64";
|
||||
await mkdir(join(root, candidatePath), { recursive: true });
|
||||
await writeFile(join(root, candidatePath, "runtime"), "pinned runtime");
|
||||
const candidates = { pi: { version: "0.0.33", profileDigest: digest("profile"),
|
||||
closureDigest: digest("closure"), qualification: "pending", path: candidatePath,
|
||||
const candidates = { pi: { version: "1.0.0", profileDigest: digest("profile"),
|
||||
closureDigest: digest("closure"), qualification: "qualified", path: candidatePath,
|
||||
sha256: sha256DirectoryTree(join(root, candidatePath)) } };
|
||||
Object.assign(payload, { candidateProviders: candidates });
|
||||
await writeManifest();
|
||||
expect(readRemoteProviderPackManifest(root).payload.candidateProviders?.pi?.qualification).toBe("pending");
|
||||
expect(readRemoteProviderPackManifest(root).payload.candidateProviders?.pi?.qualification).toBe("qualified");
|
||||
// The normal pack builder publishes Pi in both inventories. A Pi image
|
||||
// must pass the same reader used by real remote runtime preparation.
|
||||
Object.assign(payload, { providers: { cursor, pi: candidates.pi } });
|
||||
await writeManifest();
|
||||
expect(readRemoteProviderPackManifest(root).payload.providers?.pi?.qualification).toBe("qualified");
|
||||
await mkdir(releaseMetadata, { recursive: true });
|
||||
await cp(join(root, "provider-pack.json"), manifestPath);
|
||||
expect(readBundledRemoteProviderPackManifest(manifestPath).payload.providers?.pi?.qualification).toBe("qualified");
|
||||
Object.assign(payload, { providers: { cursor } });
|
||||
await rm(releaseMetadata, { recursive: true, force: true });
|
||||
await writeManifest();
|
||||
await writeFile(join(root, candidatePath, "runtime"), "substitute runtime");
|
||||
expect(() => readRemoteProviderPackManifest(root)).toThrow("candidate asset tree digest mismatch");
|
||||
await writeFile(join(root, candidatePath, "runtime"), "pinned runtime");
|
||||
for (const invalid of [{ path: "../outside" }, { qualification: "qualified" }]) {
|
||||
candidates.pi.qualification = "pending";
|
||||
await writeManifest();
|
||||
expect(() => readRemoteProviderPackManifest(root)).toThrow("invalid candidate identity");
|
||||
candidates.pi.qualification = "qualified";
|
||||
for (const invalid of [{ path: "../outside" }, { qualification: "unknown" }]) {
|
||||
const original = { ...candidates.pi };
|
||||
Object.assign(candidates.pi, invalid);
|
||||
await writeManifest();
|
||||
expect(() => readRemoteProviderPackManifest(root)).toThrow("invalid candidate identity");
|
||||
candidates.pi = original;
|
||||
}
|
||||
for (const provider of ["cursor", "copilot"]) {
|
||||
Object.assign(candidates, { [provider]: { ...candidates.pi, path: `provider-assets/${provider}/linux-x64` } });
|
||||
await writeManifest();
|
||||
expect(() => readRemoteProviderPackManifest(root)).toThrow(provider === "cursor" ? "Cursor profile or closure does not match this release" : "invalid candidate identity");
|
||||
delete (candidates as Record<string, unknown>)[provider];
|
||||
}
|
||||
await writeManifest();
|
||||
for (const [artifactName, substituteName] of [
|
||||
["nodeCommand", "productionLock"],
|
||||
@@ -1537,6 +1568,67 @@ describe("verified native harness backups", () => {
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it("binds Cursor mode across governed and identical recovery identities", () => {
|
||||
const execution = {
|
||||
...backupExecution,
|
||||
provider: { kind: "acpx", agent: "cursor", model: "explicit-model", mode: "plan" },
|
||||
interactionResponses: [{ interactionId: "interaction-1" }],
|
||||
} as unknown as NativeExecutionInputV1;
|
||||
const identity = (suffix: string, mode: unknown) => {
|
||||
const value = acpxIdentity(suffix);
|
||||
return { ...value, providerSessionIdentity: { ...value.providerSessionIdentity, mode } };
|
||||
};
|
||||
const previous = identity("previous", "plan");
|
||||
const current = identity("current", "plan");
|
||||
expect(providerSessionIdentityTransitionIsAllowed({ execution, previous, current })).toBe(true);
|
||||
expect(providerSessionIdentityTransitionIsAllowed({ execution, previous, current: previous })).toBe(true);
|
||||
for (const mode of [undefined, null, "agent", "ask", "autopilot"]) {
|
||||
const wrong = identity("wrong", mode);
|
||||
expect(providerSessionIdentityTransitionIsAllowed({ execution, previous, current: wrong })).toBe(false);
|
||||
expect(providerSessionIdentityTransitionIsAllowed({ execution, previous: wrong, current })).toBe(false);
|
||||
expect(providerSessionIdentityTransitionIsAllowed({ execution, previous: wrong, current: wrong })).toBe(false);
|
||||
}
|
||||
for (const provider of [
|
||||
{ kind: "acpx", agent: "cursor", model: "explicit-model" },
|
||||
{ kind: "acpx", agent: "copilot", model: "explicit-model" },
|
||||
]) {
|
||||
expect(providerSessionIdentityTransitionIsAllowed({
|
||||
execution: { ...execution, provider } as unknown as NativeExecutionInputV1, previous, current,
|
||||
})).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
it("binds Pi thinking across governed and identical recovery identities", () => {
|
||||
const execution = {
|
||||
...backupExecution,
|
||||
provider: { kind: "acpx", agent: "pi", model: "explicit-model", piThinkingLevel: "low" },
|
||||
interactionResponses: [{ interactionId: "interaction-1" }],
|
||||
} as unknown as NativeExecutionInputV1;
|
||||
const identity = (suffix: string, piThinkingLevel: unknown) => {
|
||||
const value = acpxIdentity(suffix);
|
||||
return { ...value, providerSessionIdentity: { ...value.providerSessionIdentity, piThinkingLevel } };
|
||||
};
|
||||
const previous = identity("previous", "low");
|
||||
const current = identity("current", "low");
|
||||
expect(providerSessionIdentityTransitionIsAllowed({ execution, previous, current })).toBe(true);
|
||||
expect(providerSessionIdentityTransitionIsAllowed({ execution, previous, current: previous })).toBe(true);
|
||||
for (const mode of [undefined, null, "high", "max", "medium"]) {
|
||||
const wrong = identity("wrong", mode);
|
||||
expect(providerSessionIdentityTransitionIsAllowed({ execution, previous, current: wrong })).toBe(false);
|
||||
expect(providerSessionIdentityTransitionIsAllowed({ execution, previous: wrong, current })).toBe(false);
|
||||
expect(providerSessionIdentityTransitionIsAllowed({ execution, previous: wrong, current: wrong })).toBe(false);
|
||||
}
|
||||
for (const provider of [
|
||||
{ kind: "acpx", agent: "pi", model: "explicit-model" },
|
||||
{ kind: "acpx", agent: "copilot", model: "explicit-model" },
|
||||
{ kind: "acpx", agent: "copilot", model: "explicit-model", piThinkingLevel: "low" },
|
||||
]) {
|
||||
expect(providerSessionIdentityTransitionIsAllowed({
|
||||
execution: { ...execution, provider } as unknown as NativeExecutionInputV1, previous, current,
|
||||
})).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
it("restores a verified continuation into an intentionally fresh non-reusable sandbox", () => {
|
||||
expect(
|
||||
shouldRestoreNativeHarnessBackupIntoSandbox({
|
||||
@@ -1899,6 +1991,27 @@ describe("split durable provider checkpoint identity", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("requires the exact observed Cursor mode in suspended descriptor and identity", () => {
|
||||
const profileDigest = `sha256:${"a".repeat(64)}`;
|
||||
const input = execution({ kind: "acpx", agent: "cursor", model: "explicit-model", permissionMode: "deny-all", mode: "plan" }, "acpx_runtime");
|
||||
const providerState = {
|
||||
schema: "paperclip.runner.acpx-provider-state.v3", lifecycle: "suspended", activeTurnId: null, providerExitUnconfirmed: false,
|
||||
descriptor: { kind: "acpx", provider: "acpx", driver: "acpx_runtime", agent: "cursor", model: "explicit-model", commandDigest: profileDigest, normalizedSessionId: "native-session", mode: "plan" },
|
||||
identity: { kind: "acpx", normalizedSessionId: "native-session", acpxRecordId: "record-1", backendSessionId: "backend-1", agentSessionId: "agent-1", profileDigest,
|
||||
workspaceDigest: `sha256:${"b".repeat(64)}`, requestedModel: "explicit-model", effectiveModel: "explicit-model", permissionMode: "deny-all", mode: "plan", providerLifetimeFenceCandidates: [53001, 53002, 53003] },
|
||||
};
|
||||
const read = (state: unknown, candidate = input) => providerSessionIdentityFromDurableProviderState({ execution: candidate, providerState: state });
|
||||
expect(read(providerState).providerSessionIdentity).toEqual(providerState.identity);
|
||||
for (const mode of [undefined, null, "agent", "ask", "autopilot"]) {
|
||||
expect(read({ ...providerState, identity: { ...providerState.identity, mode } }).providerSessionIdentity).toBeNull();
|
||||
expect(read({ ...providerState, descriptor: { ...providerState.descriptor, mode } }).providerSessionIdentity).toBeNull();
|
||||
expect(read(providerState, execution({ ...input.provider, mode }, "acpx_runtime")).providerSessionIdentity).toBeNull();
|
||||
}
|
||||
const foreign = { ...providerState, descriptor: { ...providerState.descriptor, agent: "copilot" } };
|
||||
expect(read(foreign, execution({ ...input.provider, agent: "copilot", mode: undefined }, "acpx_runtime")).providerSessionIdentity).toBeNull();
|
||||
expect(read(foreign, execution({ ...input.provider, agent: "copilot", mode: "plan" }, "acpx_runtime")).providerSessionIdentity).toEqual(providerState.identity);
|
||||
});
|
||||
|
||||
it.each([
|
||||
["codex", "codex_app_server"],
|
||||
["opencode", "opencode_server"],
|
||||
@@ -4841,6 +4954,16 @@ describe("native governed waits", () => {
|
||||
payload: { kind: "dynamicToolCall" },
|
||||
};
|
||||
|
||||
// The durable fallback preserves the question after provider loss, but
|
||||
// cannot turn that lost execution into a successful governed wait.
|
||||
const providerLost = { ...replayedEvent, eventType: "runtime_request.expired" as const,
|
||||
payload: { reason: "provider_process_lost", replayAllowed: false } };
|
||||
const lostObservation = createGovernedWaitEventObservation(async () => waitResult);
|
||||
await lostObservation.observe(providerLost, true);
|
||||
expect(lostObservation.consume(providerLost)).toBeNull();
|
||||
await lostObservation.observe(replayedEvent, true);
|
||||
expect(lostObservation.consume(replayedEvent)).toBeNull();
|
||||
|
||||
// The event consumer can lag the provider: a commentary event emitted
|
||||
// before the tool began may be processed after its approval exists in DB.
|
||||
// It is not proof that the approval-creating tool response has settled.
|
||||
@@ -5011,9 +5134,13 @@ function cancellationDb(options?: {
|
||||
runId: string;
|
||||
assessmentId: string | null;
|
||||
decisionId?: string | null;
|
||||
phase?: string;
|
||||
failureCode?: string | null;
|
||||
} | null;
|
||||
status?: string;
|
||||
failResultJsonUpdateAt?: number;
|
||||
ownershipHeld?: boolean;
|
||||
resultJson?: Record<string, unknown>;
|
||||
}) {
|
||||
const initialRun = {
|
||||
id: execution.binding.runId,
|
||||
@@ -5021,6 +5148,7 @@ function cancellationDb(options?: {
|
||||
companyId: execution.binding.companyId,
|
||||
nativeIssueId: execution.binding.issueId,
|
||||
runtimeMode: "native",
|
||||
status: options?.status ?? "running",
|
||||
...(options?.ownershipHeld
|
||||
? {
|
||||
status: "running",
|
||||
@@ -5033,6 +5161,7 @@ function cancellationDb(options?: {
|
||||
};
|
||||
let currentResultJson: Record<string, unknown> = {
|
||||
durableReceipt: { operationId: "operation-1" },
|
||||
...options?.resultJson,
|
||||
};
|
||||
const issue = {
|
||||
status: "in_progress",
|
||||
@@ -5079,6 +5208,7 @@ function cancellationDb(options?: {
|
||||
set: (values: Record<string, unknown>) => ({
|
||||
where: () => {
|
||||
updates.push({ table, values });
|
||||
if (table === nativeRunFinalizations && coordinator) Object.assign(coordinator, values);
|
||||
const updatesResultJson = "resultJson" in values;
|
||||
if (updatesResultJson) resultJsonUpdateCount += 1;
|
||||
const shouldFail =
|
||||
@@ -5953,6 +6083,71 @@ describe("native session cancellation", () => {
|
||||
expect(state.publishActivity).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it.each(["terminal_failure", "observed", "committed"])("rejects retry phase %s that changed after caller reservation, before any native effect", async phase => {
|
||||
const id = "11111111-1111-4111-8111-111111111111";
|
||||
const persistence = cancellationDb({ status: "failed",
|
||||
coordinator: { runId: execution.binding.runId, assessmentId: null, phase, failureCode: "board_recovery" },
|
||||
resultJson: { startupCancellation: { cancellationRequestId: id, retryCancellation: true } } });
|
||||
await expect(cancelNativeSession(execution.binding.runId, "operator Stop", { db: persistence.db,
|
||||
scope: "run", cancellationRequestId: id })).rejects.toMatchObject({ status: 409, message: "Native retry is no longer cancellable" });
|
||||
expect(persistence.updates).toEqual([]);
|
||||
expect(state.cancel).not.toHaveBeenCalled();
|
||||
expect(state.persistActivity).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does not mistake a recovered running attempt for the failed retry that reserved Stop", async () => {
|
||||
const id = "11111111-1111-4111-8111-111111111111";
|
||||
const persistence = cancellationDb({ status: "running",
|
||||
coordinator: { runId: execution.binding.runId, assessmentId: null, phase: "observed" },
|
||||
resultJson: { startupCancellation: { cancellationRequestId: id, retryCancellation: true } } });
|
||||
await expect(cancelNativeSession(execution.binding.runId, "operator Stop", { db: persistence.db,
|
||||
scope: "run", cancellationRequestId: id })).rejects.toMatchObject({ status: 409 });
|
||||
expect(persistence.updates).toEqual([]);
|
||||
expect(state.cancel).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("fences a failed retry at dispatch and preserves the same audited intent on replay", async () => {
|
||||
const id = "11111111-1111-4111-8111-111111111111";
|
||||
const persistence = cancellationDb({ status: "failed",
|
||||
coordinator: { runId: execution.binding.runId, assessmentId: null, phase: "retryable_failure" },
|
||||
resultJson: { startupCancellation: { cancellationRequestId: id, retryCancellation: true } } });
|
||||
const first = await cancelNativeSession(execution.binding.runId, "operator Stop", { db: persistence.db, scope: "run", cancellationRequestId: id });
|
||||
expect(persistence.updates).toContainEqual(expect.objectContaining({ table: nativeRunFinalizations,
|
||||
values: expect.objectContaining({ phase: "terminal_failure", failureCode: "native_retry_cancelled", nextAttemptAt: null }) }));
|
||||
const writes = persistence.getResultJsonUpdateCount();
|
||||
await expect(cancelNativeSession(execution.binding.runId, "retry Stop", { db: persistence.db,
|
||||
scope: "run", cancellationRequestId: id })).resolves.toMatchObject({ auditId: first.auditId });
|
||||
expect(persistence.getResultJsonUpdateCount()).toBe(writes);
|
||||
});
|
||||
|
||||
it("uses the reserved caller UUID and permits same-intent and default retries", async () => {
|
||||
const id = "11111111-1111-4111-8111-111111111111";
|
||||
const persistence = cancellationDb({ resultJson: { startupCancellation: { cancellationRequestId: id } } });
|
||||
await cancelNativeSession(execution.binding.runId, "operator Stop", { db: persistence.db, scope: "run", cancellationRequestId: id });
|
||||
expect(persistence.getResultJson().nativeCancellation).toMatchObject({ intentId: `native-cancellation:${id}` });
|
||||
const writes = persistence.getResultJsonUpdateCount();
|
||||
await cancelNativeSession(execution.binding.runId, "same request", { db: persistence.db, scope: "run", cancellationRequestId: id });
|
||||
await cancelNativeSession(execution.binding.runId, "default retry", { db: persistence.db, scope: "run" });
|
||||
expect(persistence.getResultJsonUpdateCount()).toBe(writes);
|
||||
await expect(cancelNativeSession(execution.binding.runId, "foreign caller", {
|
||||
db: persistence.db, scope: "run", cancellationRequestId: "22222222-2222-4222-8222-222222222222",
|
||||
})).rejects.toMatchObject({ status: 409 });
|
||||
expect(persistence.getResultJsonUpdateCount()).toBe(writes);
|
||||
});
|
||||
|
||||
it.each([{}, { startupCancellation: { requestedAt: "prior" } },
|
||||
{ startupCancellation: { cancellationRequestId: "other" } },
|
||||
{ startupCancellation: { cancellationRequestId: "11111111-1111-4111-8111-111111111111" }, nativeCancellation: { intentId: "foreign" } },
|
||||
])("rejects missing or raced caller ownership under the intent lock: %j", async resultJson => {
|
||||
const persistence = cancellationDb({ resultJson });
|
||||
await expect(cancelNativeSession(execution.binding.runId, "operator Stop", {
|
||||
db: persistence.db, scope: "run", cancellationRequestId: "11111111-1111-4111-8111-111111111111",
|
||||
})).rejects.toMatchObject({ status: 409 });
|
||||
expect(persistence.getForUpdateCount()).toBe(1);
|
||||
expect(persistence.updates).toEqual([]);
|
||||
expect(state.cancel).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("recovers a post-dispatch persistence failure without cancelling the provider twice", async () => {
|
||||
const persistence = cancellationDb({ failResultJsonUpdateAt: 2 });
|
||||
const running = executePaperclipNativeSession({
|
||||
@@ -6418,7 +6613,499 @@ describe("native session same-turn steering", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("bounded stopped instruction collection", () => {
|
||||
it.each([
|
||||
{ scenario: "deferred", attempts: [0], nextAttemptAt: null, expectedCalls: 1 },
|
||||
{ scenario: "no progress", attempts: [0, 0], nextAttemptAt: new Date(0), expectedCalls: 2 },
|
||||
{ scenario: "regressed counter", attempts: [1, 0], nextAttemptAt: new Date(0), expectedCalls: 2 },
|
||||
{ scenario: "progressing counter", attempts: [0, 1, 2], nextAttemptAt: new Date(0), expectedCalls: 3 },
|
||||
{ scenario: "exhausted", attempts: [3], nextAttemptAt: new Date(0), expectedCalls: 1 },
|
||||
])("finishes pending collection without inventing attempts: $scenario", async ({ attempts, nextAttemptAt, expectedCalls }) => {
|
||||
const { collectStoppedInstructionCopyWithRetries } = await import("../agent-instruction-working-copies.js");
|
||||
const collect = vi.fn(async () => {
|
||||
const index = collect.mock.calls.length - 1;
|
||||
if (index >= attempts.length) throw new Error("collector invoked beyond bounded progress");
|
||||
return { state: "pending_collection", attempts: attempts[index]!, nextAttemptAt };
|
||||
});
|
||||
expect(await collectStoppedInstructionCopyWithRetries(collect)).toEqual({ state: "pending_collection", attempts: attempts.at(-1), nextAttemptAt });
|
||||
expect(collect).toHaveBeenCalledTimes(expectedCalls);
|
||||
});
|
||||
it("returns completed or absent copies and propagates collection errors", async () => {
|
||||
const { collectStoppedInstructionCopyWithRetries } = await import("../agent-instruction-working-copies.js");
|
||||
const result = { state: "saved", attempts: 1, nextAttemptAt: null };
|
||||
const saved = vi.fn(async () => result), absent = vi.fn(async () => null);
|
||||
expect(await collectStoppedInstructionCopyWithRetries(saved)).toBe(result);
|
||||
expect(await collectStoppedInstructionCopyWithRetries(absent)).toBeNull();
|
||||
expect(saved).toHaveBeenCalledOnce(); expect(absent).toHaveBeenCalledOnce();
|
||||
await expect(collectStoppedInstructionCopyWithRetries(async () => { throw new Error("collection failed"); })).rejects.toThrow("collection failed");
|
||||
});
|
||||
});
|
||||
|
||||
describe("native warm session supervision", () => {
|
||||
it.each(["collect", "close-failure", "stopped", "destroyed", "missing-lease", "foreign-run", "foreign-environment", "deleted-environment", "claim-edit", "claim-add", "claim-remove", "claim-remove-entry", "claim-canonical", "claim-config", "claim-uncertain"])("composes remote warm ownership with real DB successor leases and current-run collection: %s", async ending => {
|
||||
const tables = await import("@paperclipai/db");
|
||||
const { eq } = await import("drizzle-orm");
|
||||
const { randomUUID } = await import("node:crypto");
|
||||
const { promisify } = await import("node:util");
|
||||
const { execFile } = await import("node:child_process");
|
||||
const executeCommand = promisify(execFile);
|
||||
const { startEmbeddedPostgresTestDatabase } = await import("../../__tests__/helpers/embedded-postgres.js");
|
||||
const { agentInstructionWorkingCopyService } = await import("../agent-instruction-working-copies.js");
|
||||
const { buildNativeRuntimeContext } = await import("./runtime-context.js");
|
||||
const { prepareNativeHeartbeatRun } = await import("./prepare-native-run.js");
|
||||
const { buildNativeCompletionContract } = await import("./completion-contracts.js");
|
||||
const home = await realpath(await mkdtemp(join(tmpdir(), "warm-remote-db-")));
|
||||
const previousHome = process.env.PAPERCLIP_HOME;
|
||||
process.env.PAPERCLIP_HOME = home;
|
||||
const database = await startEmbeddedPostgresTestDatabase("warm-remote-db-");
|
||||
const db = tables.createDb(database.connectionString);
|
||||
const companyId = randomUUID(), agentId = randomUUID(), userId = randomUUID(), environmentId = randomUUID(), issueId = randomUUID();
|
||||
const { resolveManagedInstructionsRoot } = await import("../agent-instructions.js");
|
||||
const canonical = resolveManagedInstructionsRoot({ id: agentId, companyId, name: "Warm remote", adapterConfig: {} }), remoteCwd = join(home, "remote");
|
||||
let currentLease = "", failCleanup = false, failProbe = false, closed = false;
|
||||
const commands: Array<{ leaseId: string; command: string }> = [];
|
||||
const commandRunner = (leaseId: string): import("@paperclipai/adapter-utils/command-managed-runtime").CommandManagedRuntimeRunner => ({ execute: async input => {
|
||||
expect(leaseId).toBe(currentLease); // The replaced collector must never execute.
|
||||
commands.push({ leaseId, command: `${input.command} ${(input.args ?? []).join(" ")}` });
|
||||
if (failProbe) {
|
||||
failProbe = false;
|
||||
expect(commands.at(-1)!.command).toContain("Agent directory probe deadline");
|
||||
return { exitCode: 1, signal: null, timedOut: false, stdout: "", stderr: "injected uncertain observation", pid: null, startedAt: new Date().toISOString() };
|
||||
}
|
||||
if (failCleanup && commands.at(-1)!.command.includes("rm -rf --")) {
|
||||
failCleanup = false;
|
||||
return { exitCode: 1, signal: null, timedOut: false, stdout: "", stderr: "injected cleanup failure", pid: null, startedAt: new Date().toISOString() };
|
||||
}
|
||||
const env = { PATH: process.env.PATH, ...input.env }, args = [...(input.args ?? [])];
|
||||
if (input.stdin != null && (args[0] === "-c" || args[0] === "-lc")) {
|
||||
Object.assign(env, { PAPERCLIP_TEST_STDIN: input.stdin });
|
||||
args[1] = `printf '%s' "$PAPERCLIP_TEST_STDIN" | (${args[1]})`;
|
||||
}
|
||||
const result = await executeCommand(input.command, args, { cwd: input.cwd, env, timeout: input.timeoutMs, maxBuffer: 32 * 1024 * 1024 });
|
||||
return { exitCode: 0, signal: null, timedOut: false, ...result, pid: null, startedAt: new Date().toISOString() };
|
||||
} });
|
||||
let copies = agentInstructionWorkingCopyService(db);
|
||||
const agent = { id: agentId, companyId, name: "Warm remote", adapterConfig: {
|
||||
instructionsBundleMode: "managed", instructionsRootPath: canonical, instructionsEntryFile: "AGENTS.md" } };
|
||||
let active: Awaited<ReturnType<typeof copies.prepare>>;
|
||||
const session = { close: vi.fn(async () => { closed = true; }) };
|
||||
state.execute.mockReset().mockImplementation(async options => {
|
||||
await options.onSession?.(session);
|
||||
return { result: { summary: "completed" }, terminal: { runTerminalState: "succeeded" },
|
||||
turnId: "warm-turn", normalizedSessionId: environmentId, providerSessionId: environmentId,
|
||||
driverKind: "test", driverVersion: "1", nativeEventCount: 1, highestContiguousSourceSeq: 1, usage: null };
|
||||
});
|
||||
let bodyFailure: unknown;
|
||||
try {
|
||||
await db.insert(tables.companies).values({ id: companyId, name: "Warm tests", issuePrefix: "WARM" });
|
||||
await db.insert(tables.authUsers).values({ id: userId, name: "Editor", email: `${userId}@example.test`, createdAt: new Date(), updatedAt: new Date() });
|
||||
await db.insert(tables.agents).values(agent);
|
||||
await db.insert(tables.companyMemberships).values([{ companyId, principalType: "user", principalId: userId, membershipRole: "operator" }, { companyId, principalType: "agent", principalId: agentId, membershipRole: "member" }]);
|
||||
await db.insert(tables.principalPermissionGrants).values({ companyId, principalType: "user", principalId: userId, permissionKey: "agents:configure", scope: { agentIds: [agentId] } });
|
||||
await db.insert(tables.environments).values({ id: environmentId, name: "Owned fake remote", driver: "sandbox" });
|
||||
const [issue] = await db.insert(tables.issues).values({ id: issueId, companyId, title: "Warm remote", status: "in_progress", assigneeAgentId: agentId }).returning();
|
||||
await mkdir(canonical, { recursive: true }); await mkdir(remoteCwd);
|
||||
await writeFile(join(canonical, "AGENTS.md"), "Retain this exact directory.\n");
|
||||
await writeFile(join(canonical, "optional-memory.txt"), "Optional memory\n");
|
||||
let prior: NativeExecutionInputV3 | undefined;
|
||||
let originalRoot = "", materializationRunId = "";
|
||||
for (let turn = 0; turn < 3; turn++) {
|
||||
const runId = randomUUID();
|
||||
if (currentLease) {
|
||||
await db.update(tables.environmentLeases).set({ status: "expired", releasedAt: new Date(), cleanupStatus: "success" }).where(eq(tables.environmentLeases.id, currentLease));
|
||||
await db.update(tables.heartbeatRuns).set({ status: "succeeded" }).where(eq(tables.heartbeatRuns.id, prior!.binding.runId));
|
||||
}
|
||||
currentLease = randomUUID();
|
||||
const [run] = await db.insert(tables.heartbeatRuns).values({ id: runId, companyId, agentId, invocationSource: "on_demand", responsibleUserId: userId,
|
||||
status: "running", runtimeMode: "native", nativeIssueId: issueId, nativeSessionId: environmentId, runnerInstanceId: environmentId,
|
||||
contextSnapshot: { issueId } }).returning();
|
||||
await db.insert(tables.nativeRunFinalizations).values({ runId, companyId, issueId, phase: "observed" });
|
||||
await db.update(tables.issues).set({ executionRunId: runId }).where(eq(tables.issues.id, issueId));
|
||||
await db.insert(tables.environmentLeases).values({ id: currentLease, companyId, environmentId, heartbeatRunId: runId,
|
||||
provider: "daytona", providerLeaseId: "same-owned-allocation", leasePolicy: "reuse_by_environment", metadata: { remoteCwd } });
|
||||
// Match the production preparation seam and existing native DB tests:
|
||||
// pin the issue's completion contract and durable run/lease identities.
|
||||
await prepareNativeHeartbeatRun({ db, run: run!, issue: issue!, environmentLeaseId: currentLease });
|
||||
const [contract] = await db.select().from(tables.completionContracts).where(eq(tables.completionContracts.issueId, issueId));
|
||||
const target = { kind: "remote" as const, transport: "sandbox" as const, providerKey: "daytona", environmentId, leaseId: currentLease, remoteCwd, runner: commandRunner(currentLease) };
|
||||
const current: NativeExecutionInputV3 = { ...execution, schema: "paperclip.native-execution-input.v3",
|
||||
executionMode: "default", planningContext: null, runtimeContext: nativeRuntimeContextFixture(),
|
||||
completionContract: { id: contract!.id, sha256: contract!.canonicalSha256, schemaVersion: "paperclip.completion-contract.v1",
|
||||
contract: buildNativeCompletionContract(issue!, { revision: contract!.revision }) }, binding: { companyId, agentId, issueId, runId, executionWorkspaceId: runId },
|
||||
workspace: { cwd: remoteCwd, repoUrl: null, repoRef: null, branchName: null },
|
||||
session: { ...execution.session, normalizedSessionId: environmentId, lifecyclePolicy: { mode: "warm", idleTimeoutMs: 60_000 } } };
|
||||
const preparationKey = turn === 1 && ending === "claim-config" ? "changed-config" : "same-config";
|
||||
let retiredReceipt: Record<string, unknown> | null = null;
|
||||
const capability = () => ({ runId, preparationKey, hasChanges: () => copies.hasChanges({ companyId, runId, target }),
|
||||
collectStopped: async () => { expect(closed).toBe(true); retiredReceipt = (await copies.get(companyId, runId))?.receipt ?? null;
|
||||
const { collectStoppedInstructionCopyWithRetries } = await import("../agent-instruction-working-copies.js");
|
||||
await collectStoppedInstructionCopyWithRetries(() => copies.collectStopped({ companyId, runId, target }));
|
||||
}, retirementFailed: async () => { await copies.reportRetirementUnconfirmed(companyId, runId); } });
|
||||
let release: (() => Promise<void>) | null = null;
|
||||
if (prior) {
|
||||
// Same environment is insufficient: reject a different physical allocation.
|
||||
await db.update(tables.environmentLeases).set({ providerLeaseId: "foreign-allocation" }).where(eq(tables.environmentLeases.id, currentLease));
|
||||
const before = commands.length;
|
||||
expect(await copies.adopt({ companyId, agentId, runId, previousRunId: prior.binding.runId, target, cwd: remoteCwd })).toBeNull();
|
||||
expect(commands).toHaveLength(before);
|
||||
await db.update(tables.environmentLeases).set({ providerLeaseId: "same-owned-allocation" }).where(eq(tables.environmentLeases.id, currentLease));
|
||||
if (ending === "claim-edit") await writeFile(join(originalRoot, "AGENTS.md"), "Remote edit before next claim\n");
|
||||
if (ending === "claim-add") await writeFile(join(originalRoot, "memory.txt"), "Remote addition before next claim\n");
|
||||
if (ending === "claim-remove") await rm(join(originalRoot, "optional-memory.txt"));
|
||||
if (ending === "claim-remove-entry") await rm(join(originalRoot, "AGENTS.md"));
|
||||
if (ending === "claim-canonical") await writeFile(join(canonical, "AGENTS.md"), "Canonical edit before next claim\n");
|
||||
if (["claim-edit", "claim-add", "claim-remove", "claim-remove-entry", "claim-canonical"].includes(ending)) {
|
||||
// Ordinary adoption remains unchanged-only and cannot alias this owner.
|
||||
expect(await copies.adopt({ companyId, agentId, runId, previousRunId: prior.binding.runId, target, cwd: remoteCwd })).toBeNull();
|
||||
expect((await copies.get(companyId, prior.binding.runId))?.receipt?.retainedByRunId).toBeUndefined();
|
||||
}
|
||||
failProbe = ending === "claim-uncertain";
|
||||
release = await claimWarmNativeInstructionCopy({ priorExecution: prior, companyId, agentId, executionWorkspaceId: runId,
|
||||
workspace: current.workspace, environmentId, runId, preparationKey, adopt: async previousRunId => {
|
||||
active = await copies.adopt({ companyId, agentId, runId, previousRunId, target, cwd: remoteCwd, allowRetirementHandoff: true });
|
||||
return active ? capability() : null;
|
||||
} });
|
||||
if (ending.startsWith("claim-")) {
|
||||
expect(release).toBeNull();
|
||||
expect(session.close).toHaveBeenCalledOnce();
|
||||
expect(state.execute).toHaveBeenCalledOnce(); // No reuse was admitted.
|
||||
if (ending !== "claim-config") expect(retiredReceipt).toMatchObject({ retirementRequired: true });
|
||||
expect(retiredReceipt).toMatchObject({ materializationRunId, cleanup: { leaseId: currentLease, remoteCwd } });
|
||||
expect((await copies.get(companyId, runId))?.processStoppedAt).toBeInstanceOf(Date);
|
||||
const compact = (await copies.get(companyId, runId))!.receipt;
|
||||
expect(compact).toMatchObject({ materializationRunId, cleanupPending: false });
|
||||
expect(compact?.baseline).toBeUndefined();
|
||||
await expect(access(originalRoot)).rejects.toMatchObject({ code: "ENOENT" });
|
||||
if (ending === "claim-edit") expect(await readFile(join(canonical, "AGENTS.md"), "utf8")).toBe("Remote edit before next claim\n");
|
||||
if (ending === "claim-add") expect(await readFile(join(canonical, "memory.txt"), "utf8")).toBe("Remote addition before next claim\n");
|
||||
if (ending === "claim-remove") await expect(access(join(canonical, "optional-memory.txt"))).rejects.toMatchObject({ code: "ENOENT" });
|
||||
if (ending === "claim-remove-entry") {
|
||||
expect((await copies.get(companyId, runId))!).toMatchObject({ state: "unavailable", errorCode: "AGENT_FILES_SAVE_FAILED" });
|
||||
expect(await readFile(join(canonical, "AGENTS.md"), "utf8")).toBe("Retain this exact directory.\n");
|
||||
}
|
||||
if (ending === "claim-canonical") expect(await readFile(join(canonical, "AGENTS.md"), "utf8")).toBe("Canonical edit before next claim\n");
|
||||
// Heartbeat rematerializes only after the successor proved retirement.
|
||||
active = await copies.prepare({ companyId, agentId, runId, target, cwd: remoteCwd });
|
||||
expect(active).toMatchObject({ state: "prepared", executionRoot: originalRoot, processStoppedAt: null });
|
||||
expect(active!.receipt?.retirementRequired).toBeUndefined();
|
||||
expect(await copies.hasChanges({ companyId, runId, target })).toBe(false);
|
||||
await copies.reportUnavailable(companyId, prior.binding.runId);
|
||||
await copies.release(companyId, prior.binding.runId);
|
||||
expect((await lstat(originalRoot)).isDirectory()).toBe(true);
|
||||
await copies.collectStopped({ companyId, runId, target });
|
||||
return;
|
||||
}
|
||||
expect(release).toBeTypeOf("function");
|
||||
await copies.reportUnavailable(companyId, prior.binding.runId);
|
||||
await copies.release(companyId, prior.binding.runId);
|
||||
expect(active!.receipt).toMatchObject({ materializationRunId, cleanup: { leaseId: currentLease, remoteCwd } });
|
||||
} else {
|
||||
active = await copies.prepare({ companyId, agentId, runId, target, cwd: remoteCwd });
|
||||
originalRoot = active!.executionRoot; materializationRunId = runId;
|
||||
}
|
||||
expect(active!.executionRoot).toBe(originalRoot);
|
||||
current.runtimeContext = await buildNativeRuntimeContext({ db, agent, runId, runtimeConfig: {}, runtimeSkillEntries: [],
|
||||
instructionWorkingCopy: { rootPath: active!.executionRoot, entryPath: "AGENTS.md", kind: "agent_files" } });
|
||||
expect(current.runtimeContext.instructions.workingCopy?.rootPath).toBe(originalRoot);
|
||||
await executePaperclipNativeSession({ db, execution: current, runnerInstanceId: environmentId, runnerExecutionTarget: target, instructionWorkingCopy: capability() });
|
||||
if (prior) expect(state.execute.mock.calls.at(-1)?.[0].existingSession).toBe(session);
|
||||
expect(closed).toBe(false);
|
||||
await release?.(); expect(closed).toBe(false);
|
||||
prior = current;
|
||||
}
|
||||
// A write after the live unchanged probe is still collected on retirement.
|
||||
await writeFile(join(originalRoot, "late-memory.txt"), "Latest owner's remote bytes");
|
||||
if (["missing-lease", "foreign-run", "foreign-environment", "deleted-environment"].includes(ending)) {
|
||||
if (ending === "missing-lease") await db.delete(tables.environmentLeases).where(eq(tables.environmentLeases.id, currentLease));
|
||||
if (ending === "foreign-run") await db.update(tables.environmentLeases).set({ heartbeatRunId: materializationRunId }).where(eq(tables.environmentLeases.id, currentLease));
|
||||
if (ending === "foreign-environment") {
|
||||
const other = randomUUID(); await db.insert(tables.environments).values({ id: other, name: "Other", driver: "sandbox" });
|
||||
await db.update(tables.environmentLeases).set({ environmentId: other }).where(eq(tables.environmentLeases.id, currentLease));
|
||||
}
|
||||
if (ending === "deleted-environment") await db.delete(tables.environments).where(eq(tables.environments.id, environmentId));
|
||||
const before = commands.length;
|
||||
await closeWarmNativeSessionsForEnvironment({ environmentId, reason: "lease identity changed" });
|
||||
expect(commands).toHaveLength(before);
|
||||
const pending = (await copies.get(companyId, prior!.binding.runId))!;
|
||||
expect(pending.state).toBe("pending_collection");
|
||||
expect(pending.receipt?.baseline).toBeDefined();
|
||||
expect(await readFile(join(originalRoot, "late-memory.txt"), "utf8")).toBe("Latest owner's remote bytes");
|
||||
return;
|
||||
}
|
||||
if (ending === "close-failure") {
|
||||
session.close.mockImplementationOnce(async () => { throw new Error("owned close not yet confirmed"); });
|
||||
const before = commands.length;
|
||||
expect(await closeWarmNativeSessionsForEnvironment({ environmentId, reason: "first retirement" })).toMatchObject({ failed: 1 });
|
||||
await copies.reportUnavailable(companyId, prior!.binding.runId);
|
||||
await copies.reportUnavailable(companyId, materializationRunId);
|
||||
await copies.release(companyId, prior!.binding.runId);
|
||||
await copies.release(companyId, materializationRunId);
|
||||
expect(await copies.get(companyId, prior!.binding.runId)).toMatchObject({ state: "pending_collection", errorCode: "INSTRUCTION_STOP_UNCONFIRMED", processStoppedAt: null });
|
||||
expect(commands).toHaveLength(before);
|
||||
expect(await readFile(join(originalRoot, "late-memory.txt"), "utf8")).toBe("Latest owner's remote bytes");
|
||||
expect(await closeWarmNativeSessionsForEnvironment({ environmentId, reason: "verified later retirement" })).toMatchObject({ closed: 1, failed: 0 });
|
||||
expect(await copies.get(companyId, prior!.binding.runId)).toMatchObject({ state: "saved", receipt: { cleanupPending: false } });
|
||||
expect(await readFile(join(canonical, "late-memory.txt"), "utf8")).toBe("Latest owner's remote bytes");
|
||||
await expect(access(originalRoot)).rejects.toMatchObject({ code: "ENOENT" });
|
||||
return;
|
||||
}
|
||||
if (ending !== "collect") {
|
||||
const { remoteTerminationReceipt } = await import("../remote-execution-termination.js");
|
||||
const [lease] = await db.select().from(tables.environmentLeases).where(eq(tables.environmentLeases.id, currentLease));
|
||||
await db.update(tables.environmentLeases).set({ status: "released", releasedAt: new Date(), cleanupStatus: "success",
|
||||
metadata: { ...lease!.metadata, remoteExecutionTermination: remoteTerminationReceipt(lease!, { providerLeaseId: lease!.providerLeaseId, state: ending }) } }).where(eq(tables.environmentLeases.id, currentLease));
|
||||
await db.update(tables.heartbeatRuns).set({ status: "succeeded" }).where(eq(tables.heartbeatRuns.id, prior!.binding.runId));
|
||||
const before = commands.length;
|
||||
await copies.recoverStopped(); // The original remote collector is still cached.
|
||||
expect(commands).toHaveLength(before);
|
||||
const recovered = (await copies.get(companyId, prior!.binding.runId))!;
|
||||
expect(recovered.state).toBe(ending === "destroyed" ? "unavailable" : "pending_collection");
|
||||
await expect(access(join(canonical, "late-memory.txt"))).rejects.toMatchObject({ code: "ENOENT" });
|
||||
if (ending === "stopped") {
|
||||
expect(await readFile(join(originalRoot, "late-memory.txt"), "utf8")).toBe("Latest owner's remote bytes");
|
||||
expect(recovered.receipt?.baseline).toBeDefined();
|
||||
await copies.collectStopped({ companyId, runId: prior!.binding.runId });
|
||||
await copies.release(companyId, prior!.binding.runId);
|
||||
expect(commands).toHaveLength(before);
|
||||
} else await expect(access(active!.localRoot)).rejects.toMatchObject({ code: "ENOENT" });
|
||||
await closeWarmNativeSessionsForEnvironment({ environmentId, reason: "already terminated owner" });
|
||||
expect(commands).toHaveLength(before);
|
||||
return;
|
||||
}
|
||||
failCleanup = true;
|
||||
await closeWarmNativeSessionsForEnvironment({ environmentId, reason: "test retirement" });
|
||||
expect(session.close).toHaveBeenCalledOnce();
|
||||
expect(await readFile(join(canonical, "late-memory.txt"), "utf8")).toBe("Latest owner's remote bytes");
|
||||
const saved = (await copies.get(companyId, prior!.binding.runId))!;
|
||||
expect(saved).toMatchObject({ state: "saved", receipt: { cleanupPending: true, materializationRunId, cleanup: { leaseId: currentLease } } });
|
||||
const cleanup = vi.fn(async input => {
|
||||
expect(input.lease.id).toBe(currentLease);
|
||||
expect(input.lease.heartbeatRunId).toBe(prior!.binding.runId);
|
||||
return commandRunner(currentLease).execute(input);
|
||||
});
|
||||
copies = agentInstructionWorkingCopyService(db, { environmentRuntime: { execute: cleanup } as never });
|
||||
await db.update(tables.agentInstructionWorkingCopies).set({ nextAttemptAt: null }).where(eq(tables.agentInstructionWorkingCopies.runId, prior!.binding.runId));
|
||||
await copies.recoverCaptured();
|
||||
expect(cleanup).toHaveBeenCalledOnce();
|
||||
await expect(access(originalRoot)).rejects.toMatchObject({ code: "ENOENT" });
|
||||
expect((await copies.get(companyId, prior!.binding.runId))!.receipt?.cleanupPending).toBe(false);
|
||||
} catch (error) { bodyFailure = error; throw error; }
|
||||
finally {
|
||||
const cleanupFailures: unknown[] = [];
|
||||
try { await closeWarmNativeSessionsForEnvironment({ environmentId, reason: "test cleanup" }); } catch (error) { cleanupFailures.push(error); }
|
||||
try { await database.cleanup(); } catch (error) { cleanupFailures.push(error); }
|
||||
if (previousHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = previousHome;
|
||||
try {
|
||||
const writable = async (directory: string): Promise<void> => {
|
||||
await chmod(directory, 0o700);
|
||||
for (const entry of await readdir(directory, { withFileTypes: true })) if (entry.isDirectory()) await writable(join(directory, entry.name));
|
||||
};
|
||||
await writable(home); await rm(home, { recursive: true, force: true });
|
||||
} catch (error) { cleanupFailures.push(error); }
|
||||
if (cleanupFailures.length) throw new AggregateError([...(bodyFailure ? [bodyFailure] : []), ...cleanupFailures], "Warm DB test body/cleanup failure");
|
||||
}
|
||||
}, 90_000);
|
||||
|
||||
it.each(["idle", "edit", "add", "remove", "projectless"])("retains the actual agent_files copy across warm turns, then retires for %s", async ending => {
|
||||
const root = await realpath(await mkdtemp(join(tmpdir(), "warm-agent-files-")));
|
||||
const localRoot = join(root, "run", "live");
|
||||
await mkdir(localRoot, { recursive: true });
|
||||
await writeFile(join(localRoot, "AGENTS.md"), "Keep the registered directory.\n");
|
||||
const snapshot = await captureDirectorySnapshot(localRoot);
|
||||
let row = { companyId: execution.binding.companyId, agentId: execution.binding.agentId,
|
||||
runId: "warm-real-agent-files", localRoot, executionRoot: localRoot, location: "local",
|
||||
entryFile: "AGENTS.md", state: "prepared", attempts: 0, processStoppedAt: null,
|
||||
baseRevisionId: null, candidateBase64: null, candidateHash: null, errorCode: null, errorMessage: null, nextAttemptAt: null,
|
||||
responsibleUserId: "test-user", createdAt: new Date(), updatedAt: new Date(),
|
||||
baseHash: directorySnapshotSha256(snapshot), receipt: { schema: AGENT_FILES_CONTRACT, baseline: serializeDirectorySnapshot(snapshot), materializationIdentity: probeAgentDirectory(localRoot).identity },
|
||||
} as typeof agentInstructionWorkingCopies.$inferSelect;
|
||||
const copies = agentDirectoryWorkingCopyService({} as Db, async () => row,
|
||||
async (_prior, values) => (row = { ...row, ...values } as typeof row));
|
||||
const order: string[] = [];
|
||||
const close = vi.fn(async () => { order.push("closed"); });
|
||||
const collectStopped = async () => {
|
||||
order.push("collected");
|
||||
expect(close).toHaveBeenCalledOnce();
|
||||
// The real DB service suite verifies changed-file persistence. Here the
|
||||
// fake backend proves stop precedes granting its collection callback.
|
||||
if (ending === "idle" || ending === "projectless") await copies.collectStopped(row);
|
||||
};
|
||||
const identity = "warm-real-agent-files";
|
||||
const warmExecution = { ...execution, binding: { ...execution.binding, runId: identity, executionWorkspaceId: ending === "projectless" ? identity : `${identity}-workspace` },
|
||||
session: { ...execution.session, normalizedSessionId: identity, lifecyclePolicy: { mode: "warm" as const, idleTimeoutMs: 60_000 } },
|
||||
} as NativeExecutionInputV1;
|
||||
const session = { close };
|
||||
state.execute.mockReset().mockImplementation(async (options) => {
|
||||
await options.onSession?.(session);
|
||||
return { result: { summary: "completed" }, terminal: { runTerminalState: "succeeded" },
|
||||
turnId: identity, normalizedSessionId: identity, providerSessionId: identity,
|
||||
driverKind: "test", driverVersion: "1", nativeEventCount: 1, highestContiguousSourceSeq: 1, usage: null };
|
||||
});
|
||||
try {
|
||||
await executePaperclipNativeSession({ db: leaseDb(warmExecution), execution: warmExecution,
|
||||
runnerInstanceId: identity, runnerExecutionTarget: { kind: "remote", transport: "sandbox", environmentId: identity, remoteCwd: `/tmp/${identity}` },
|
||||
instructionWorkingCopy: { runId: identity, root: localRoot, preparationKey: "same-preparation", hasChanges: () => copies.hasChanges(row), collectStopped } });
|
||||
expect(close).not.toHaveBeenCalled();
|
||||
expect(order).toEqual([]);
|
||||
expect(await readFile(join(localRoot, "AGENTS.md"), "utf8")).toContain("registered");
|
||||
const second = { ...warmExecution, binding: { ...warmExecution.binding, runId: `${identity}-second`,
|
||||
...(ending === "projectless" ? { executionWorkspaceId: `${identity}-second` } : {}) } };
|
||||
expect(nativeSessionWorkspaceScope(second)).toEqual(nativeSessionWorkspaceScope(warmExecution));
|
||||
const currentCopy = { runId: second.binding.runId, root: localRoot, preparationKey: "same-preparation", hasChanges: () => copies.hasChanges(row), collectStopped };
|
||||
if (ending === "projectless") {
|
||||
const adopt = vi.fn(async () => currentCopy);
|
||||
for (const changed of [
|
||||
{ ...second.workspace, cwd: "/different" }, { ...second.workspace, repoUrl: "https://example.test/other" },
|
||||
{ ...second.workspace, repoRef: "other" }, { ...second.workspace, branchName: "other" },
|
||||
]) expect(await claimWarmNativeInstructionCopy({ priorExecution: warmExecution,
|
||||
companyId: second.binding.companyId, agentId: second.binding.agentId, executionWorkspaceId: second.binding.runId,
|
||||
workspace: changed, environmentId: identity, runId: second.binding.runId, preparationKey: "same-preparation", adopt })).toBeNull();
|
||||
expect(await claimWarmNativeInstructionCopy({ priorExecution: warmExecution,
|
||||
companyId: second.binding.companyId, agentId: second.binding.agentId, executionWorkspaceId: "managed-workspace",
|
||||
workspace: second.workspace, environmentId: identity, runId: second.binding.runId, preparationKey: "same-preparation", adopt })).toBeNull();
|
||||
expect(adopt).not.toHaveBeenCalled(); expect(close).not.toHaveBeenCalled();
|
||||
}
|
||||
const release = await claimWarmNativeInstructionCopy({ priorExecution: warmExecution,
|
||||
companyId: warmExecution.binding.companyId, agentId: warmExecution.binding.agentId,
|
||||
executionWorkspaceId: second.binding.executionWorkspaceId, workspace: second.workspace,
|
||||
environmentId: identity, runId: second.binding.runId, preparationKey: "same-preparation",
|
||||
adopt: async () => currentCopy });
|
||||
expect(release).toBeTypeOf("function");
|
||||
await executePaperclipNativeSession({ db: leaseDb(second), execution: second, runnerInstanceId: identity,
|
||||
runnerExecutionTarget: { kind: "remote", transport: "sandbox", environmentId: identity, remoteCwd: `/tmp/${identity}` },
|
||||
instructionWorkingCopy: currentCopy });
|
||||
expect(state.execute.mock.calls.at(-1)?.[0].existingSession).toBe(session);
|
||||
expect(close).not.toHaveBeenCalled();
|
||||
expect(row.executionRoot).toBe(localRoot);
|
||||
await release?.(); // Consumed preparation cannot retire the retained owner.
|
||||
expect(close).not.toHaveBeenCalled();
|
||||
if (ending === "idle" || ending === "projectless") {
|
||||
await closeWarmNativeSessionsForEnvironment({ environmentId: identity, reason: "idle retirement" });
|
||||
await expect(access(localRoot)).rejects.toMatchObject({ code: "ENOENT" });
|
||||
} else {
|
||||
if (ending === "edit") await writeFile(join(localRoot, "AGENTS.md"), "Changed instructions");
|
||||
if (ending === "add") await writeFile(join(localRoot, "memory.txt"), "New memory");
|
||||
if (ending === "remove") await rm(join(localRoot, "AGENTS.md"));
|
||||
const adopt = vi.fn(async () => ({ ...currentCopy, runId: `${identity}-third` }));
|
||||
expect(await claimWarmNativeInstructionCopy({ priorExecution: second,
|
||||
companyId: second.binding.companyId, agentId: second.binding.agentId,
|
||||
executionWorkspaceId: second.binding.executionWorkspaceId, workspace: second.workspace,
|
||||
environmentId: identity, runId: `${identity}-third`, preparationKey: "same-preparation", adopt })).toBeNull();
|
||||
expect(adopt).toHaveBeenCalledOnce();
|
||||
}
|
||||
expect(order).toEqual(["closed", "collected"]);
|
||||
} finally {
|
||||
await closeWarmNativeSessionsForEnvironment({ environmentId: identity, reason: "test cleanup" });
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it.each(["final-config", "changed-root", "late-edit", "close-failure", "abandon", "concurrent", "foreign-company", "foreign-workspace"])("fences owned instruction preparation: %s", async scenario => {
|
||||
const identity = `warm-copy-${scenario}`;
|
||||
const order: string[] = [];
|
||||
let allowClose = scenario !== "close-failure";
|
||||
const session = { close: vi.fn(async () => { order.push("close"); if (!allowClose) throw new Error("retirement unconfirmed"); }) };
|
||||
const retirementFailed = vi.fn(async () => {});
|
||||
let lateEdit = false;
|
||||
const copy = { runId: identity, root: `/tmp/${identity}/instructions`, preparationKey: "key", hasChanges: async () => lateEdit,
|
||||
collectStopped: vi.fn(async () => { order.push("collect"); }), retirementFailed };
|
||||
const first = { ...execution, binding: { ...execution.binding, runId: identity, executionWorkspaceId: `${identity}-workspace` },
|
||||
session: { ...execution.session, normalizedSessionId: identity, lifecyclePolicy: { mode: "warm" as const, idleTimeoutMs: 60_000 } },
|
||||
} as NativeExecutionInputV1;
|
||||
const target = { kind: "remote" as const, transport: "sandbox" as const, environmentId: identity, remoteCwd: `/tmp/${identity}` };
|
||||
state.execute.mockReset().mockImplementation(async options => {
|
||||
await options.onSession?.(session);
|
||||
return { result: { summary: "completed" }, terminal: { runTerminalState: "succeeded" },
|
||||
turnId: identity, normalizedSessionId: identity, providerSessionId: identity,
|
||||
driverKind: "test", driverVersion: "1", nativeEventCount: 1, highestContiguousSourceSeq: 1, usage: null };
|
||||
});
|
||||
const second = { ...first, binding: { ...first.binding, runId: `${identity}-second` } };
|
||||
const current = { ...copy, runId: second.binding.runId,
|
||||
...(scenario === "changed-root" ? { root: `/tmp/${identity}/other-instructions` } : {}) };
|
||||
const adopt = vi.fn(async () => current);
|
||||
const claim = { priorExecution: first, companyId: first.binding.companyId, agentId: first.binding.agentId,
|
||||
executionWorkspaceId: first.binding.executionWorkspaceId, workspace: first.workspace, environmentId: identity,
|
||||
runId: second.binding.runId, preparationKey: "key", adopt };
|
||||
try {
|
||||
await executePaperclipNativeSession({ db: leaseDb(first), execution: first, runnerInstanceId: identity, runnerExecutionTarget: target, instructionWorkingCopy: copy });
|
||||
if (scenario.startsWith("foreign")) {
|
||||
expect(await claimWarmNativeInstructionCopy({ ...claim,
|
||||
...(scenario === "foreign-company" ? { companyId: "other" } : { executionWorkspaceId: "other" }) })).toBeNull();
|
||||
expect(adopt).not.toHaveBeenCalled(); expect(session.close).not.toHaveBeenCalled();
|
||||
} else {
|
||||
const release = await claimWarmNativeInstructionCopy(claim);
|
||||
if (scenario === "concurrent") await expect(claimWarmNativeInstructionCopy({ ...claim, runId: "competitor" })).rejects.toThrow("busy");
|
||||
if (scenario === "final-config" || scenario === "changed-root" || scenario === "late-edit") {
|
||||
lateEdit = scenario === "late-edit";
|
||||
const changed = scenario !== "final-config" ? second : { ...second, session: { ...second.session, lifecyclePolicy: { mode: "warm" as const, idleTimeoutMs: 99_000 } } };
|
||||
await expect(executePaperclipNativeSession({ db: leaseDb(changed), execution: changed, runnerInstanceId: identity,
|
||||
runnerExecutionTarget: target, instructionWorkingCopy: current })).rejects.toThrow(scenario === "late-edit"
|
||||
? "native_instruction_preparation_copy_changed" : "native_instruction_preparation_configuration_changed");
|
||||
expect(state.execute).toHaveBeenCalledOnce();
|
||||
}
|
||||
if (scenario === "close-failure") {
|
||||
await expect(release!()).rejects.toThrow("retirement unconfirmed");
|
||||
expect(retirementFailed).toHaveBeenCalledOnce();
|
||||
expect(copy.collectStopped).not.toHaveBeenCalled();
|
||||
expect(order).toEqual(["close"]);
|
||||
} else {
|
||||
await release?.();
|
||||
expect(order).toEqual(["close", "collect"]);
|
||||
}
|
||||
}
|
||||
} finally { allowClose = true; await closeWarmNativeSessionsForEnvironment({ environmentId: identity, reason: "test cleanup" }); }
|
||||
});
|
||||
|
||||
it.each(["config", "reset", "changed", "unavailable", "auth", "wrong-capability", "retirement-failure"])("hands off collection before probing or retiring a warm owner: %s", async scenario => {
|
||||
const identity = `warm-successor-${scenario}`, order: string[] = [];
|
||||
let allowClose = scenario !== "retirement-failure";
|
||||
const session = { close: vi.fn(async () => { order.push("stop"); if (!allowClose) throw new Error("stop unconfirmed"); }) };
|
||||
const priorCopy = { runId: identity, preparationKey: "before", hasChanges: vi.fn(async () => false),
|
||||
collectStopped: vi.fn(async () => { order.push("prior-pending"); }), retirementFailed: vi.fn(async () => {}) };
|
||||
const first = { ...execution, binding: { ...execution.binding, runId: identity, executionWorkspaceId: `${identity}-workspace` },
|
||||
session: { ...execution.session, normalizedSessionId: identity, lifecyclePolicy: { mode: "warm" as const, idleTimeoutMs: 60_000 } } } as NativeExecutionInputV1;
|
||||
const target = { kind: "remote" as const, transport: "sandbox" as const, environmentId: identity, remoteCwd: `/tmp/${identity}` };
|
||||
state.execute.mockReset().mockImplementation(async options => {
|
||||
await options.onSession?.(session);
|
||||
return { result: { summary: "completed" }, terminal: { runTerminalState: "succeeded" }, turnId: identity,
|
||||
normalizedSessionId: identity, providerSessionId: identity, driverKind: "test", driverVersion: "1",
|
||||
nativeEventCount: 1, highestContiguousSourceSeq: 1, usage: null };
|
||||
});
|
||||
const current = { runId: `${identity}-next`, preparationKey: scenario === "config" ? "after" : "before",
|
||||
hasChanges: vi.fn(async () => { order.push("current-probe"); return true; }),
|
||||
collectStopped: vi.fn(async () => { order.push("current-collect"); }), retirementFailed: vi.fn(async () => {}) };
|
||||
const adopt = vi.fn(async () => {
|
||||
order.push("handoff");
|
||||
if (scenario === "auth") throw new Error("authorization denied");
|
||||
if (scenario === "unavailable") return null;
|
||||
return scenario === "wrong-capability" ? { ...current, runId: "foreign" } : current;
|
||||
});
|
||||
try {
|
||||
await executePaperclipNativeSession({ db: leaseDb(first), execution: first, runnerInstanceId: identity, runnerExecutionTarget: target, instructionWorkingCopy: priorCopy });
|
||||
priorCopy.hasChanges.mockClear();
|
||||
const claiming = claimWarmNativeInstructionCopy({ priorExecution: first, companyId: first.binding.companyId, agentId: first.binding.agentId,
|
||||
executionWorkspaceId: first.binding.executionWorkspaceId, workspace: first.workspace, environmentId: identity,
|
||||
runId: current.runId, preparationKey: current.preparationKey, forceRetirement: scenario === "reset", adopt });
|
||||
if (["unavailable", "auth", "wrong-capability", "retirement-failure"].includes(scenario)) {
|
||||
await expect(claiming).rejects.toThrow(({ unavailable: "handoff_unavailable", auth: "authorization denied",
|
||||
"wrong-capability": "handoff_mismatch", "retirement-failure": "stop unconfirmed" } as Record<string, string>)[scenario]);
|
||||
} else expect(await claiming).toBeNull();
|
||||
expect(priorCopy.hasChanges).not.toHaveBeenCalled();
|
||||
expect(state.execute).toHaveBeenCalledOnce();
|
||||
if (["unavailable", "auth", "wrong-capability"].includes(scenario)) {
|
||||
expect(order).toEqual(["handoff", "stop", "prior-pending"]);
|
||||
expect(current.collectStopped).not.toHaveBeenCalled();
|
||||
} else {
|
||||
expect(priorCopy.collectStopped).not.toHaveBeenCalled();
|
||||
expect(order).toEqual(["handoff", ...(["config", "reset"].includes(scenario) ? [] : ["current-probe"]), "stop", ...(scenario === "retirement-failure" ? [] : ["current-collect"])]);
|
||||
if (scenario === "retirement-failure") {
|
||||
expect(current.retirementFailed).toHaveBeenCalledOnce();
|
||||
expect(current.collectStopped).not.toHaveBeenCalled();
|
||||
}
|
||||
}
|
||||
} finally { allowClose = true; await closeWarmNativeSessionsForEnvironment({ environmentId: identity, reason: "test cleanup" }); }
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ change: "addition", before: {}, after: { CUSTOM_TOKEN: "first" }, replaced: true },
|
||||
{ change: "rotation", before: { CUSTOM_TOKEN: "first" }, after: { CUSTOM_TOKEN: "second" }, replaced: true },
|
||||
@@ -7817,7 +8504,7 @@ describe("native warm session supervision", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it.each(["permission", "managed credential"])("replaces an idle warm provider session when its %s changes", async (change) => {
|
||||
it.each(["permission", "managed credential", "ACPX runtime contract", "unchanged Codex runtime contract"])("checks warm owner compatibility for %s", async (change) => {
|
||||
const firstClose = vi.fn(async () => undefined);
|
||||
const secondClose = vi.fn(async () => undefined);
|
||||
const firstSession = { close: firstClose };
|
||||
@@ -7825,7 +8512,9 @@ describe("native warm session supervision", () => {
|
||||
const base = {
|
||||
...execution,
|
||||
schema: "paperclip.native-execution-input.v4",
|
||||
provider: { kind: "codex", model: null, approvalPolicy: "never" },
|
||||
provider: change === "ACPX runtime contract"
|
||||
? { kind: "acpx", agent: "claude", model: "claude-sonnet-5", permissionMode: "approve-all" }
|
||||
: { kind: "codex", model: null, approvalPolicy: "never" },
|
||||
binding: {
|
||||
...execution.binding,
|
||||
runId: "run-permission-never",
|
||||
@@ -7839,7 +8528,7 @@ describe("native warm session supervision", () => {
|
||||
},
|
||||
session: {
|
||||
normalizedSessionId: "session-warm-permission",
|
||||
driverKind: "codex_app_server" as const,
|
||||
driverKind: change === "ACPX runtime contract" ? "acpx_runtime" as const : "codex_app_server" as const,
|
||||
protocolVersion: 1 as const,
|
||||
lifecyclePolicy: { mode: "warm" as const, idleTimeoutMs: 20 },
|
||||
},
|
||||
@@ -7862,6 +8551,12 @@ describe("native warm session supervision", () => {
|
||||
highestContiguousSourceSeq: 1,
|
||||
usage: null,
|
||||
};
|
||||
const runtimeChange = change === "ACPX runtime contract" || change === "unchanged Codex runtime contract";
|
||||
const keepsOwner = change === "unchanged Codex runtime contract";
|
||||
const currentRuntimeContract = nativeSessionResume.nativeRuntimeContractForProvider(base.provider);
|
||||
const contract = runtimeChange
|
||||
? vi.spyOn(nativeSessionResume, "nativeRuntimeContractForProvider").mockReturnValue(undefined)
|
||||
: undefined;
|
||||
state.execute
|
||||
.mockReset()
|
||||
.mockImplementationOnce(async (options) => {
|
||||
@@ -7870,8 +8565,8 @@ describe("native warm session supervision", () => {
|
||||
return result;
|
||||
})
|
||||
.mockImplementationOnce(async (options) => {
|
||||
expect(options.existingSession).toBeUndefined();
|
||||
options.onSession?.(secondSession);
|
||||
expect(options.existingSession).toBe(keepsOwner ? firstSession : undefined);
|
||||
if (!keepsOwner) options.onSession?.(secondSession);
|
||||
return result;
|
||||
});
|
||||
|
||||
@@ -7885,21 +8580,22 @@ describe("native warm session supervision", () => {
|
||||
managedAiCredentialIdentity: "first-identity",
|
||||
runnerInstanceId: "runner",
|
||||
});
|
||||
contract?.mockReturnValue(currentRuntimeContract);
|
||||
await executePaperclipNativeSession({
|
||||
db: leaseDb(lowered),
|
||||
execution: lowered,
|
||||
managedAiCredentialIdentity: change === "managed credential" ? "second-identity" : "first-identity",
|
||||
runnerInstanceId: "runner",
|
||||
});
|
||||
expect(firstClose).toHaveBeenCalledWith({
|
||||
reason: "warm native session configuration changed",
|
||||
});
|
||||
if (keepsOwner) expect(firstClose).not.toHaveBeenCalled();
|
||||
else expect(firstClose).toHaveBeenCalledWith({ reason: "warm native session configuration changed" });
|
||||
expect(secondClose).not.toHaveBeenCalled();
|
||||
await vi.advanceTimersByTimeAsync(20);
|
||||
expect(secondClose).toHaveBeenCalledWith({
|
||||
expect(keepsOwner ? firstClose : secondClose).toHaveBeenCalledWith({
|
||||
reason: "warm native session idle timeout",
|
||||
});
|
||||
} finally {
|
||||
contract?.mockRestore();
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
@@ -8723,6 +9419,11 @@ describe("native process ownership", () => {
|
||||
},
|
||||
"acpx_runtime",
|
||||
],
|
||||
[
|
||||
"Pi ACPX without candidate authorization",
|
||||
{ kind: "acpx", agent: "pi", model: "openrouter/deepseek/deepseek-v4-flash-0731", permissionMode: "approve-reads" },
|
||||
"acpx_runtime",
|
||||
],
|
||||
])(
|
||||
"admits the qualified %s provider",
|
||||
async (_name, provider, driverKind) => {
|
||||
@@ -8761,7 +9462,7 @@ describe("native process ownership", () => {
|
||||
},
|
||||
);
|
||||
|
||||
it.each(["pi", "copilot"])("rejects ACPX candidate %s without host authorization before constructing a backend", async (agent) => {
|
||||
it.each(["copilot"])("rejects ACPX candidate %s without host authorization before constructing a backend", async (agent) => {
|
||||
const piExecution = {
|
||||
...execution,
|
||||
binding: { ...execution.binding, runId: "run-acpx-pi-rejected" },
|
||||
|
||||
Reference in new issue
Block a user