mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 07:10:34 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native agent runs can execute inside a Daytona sandbox through the Daytona sandbox provider plugin > - The plugin runs every command for a lease in one persistent Daytona process session and keeps that session id in memory > - Daytona auto-stops an idle sandbox, and a stopped sandbox loses all of its process sessions > - Only the execute and resume hooks dropped the stored session id before a restart. Workspace realize, sync-in, sync-out, runner ingress, and interactive setup also restart a stopped sandbox, but they kept the dead id > - When host-side workspace preparation stays idle for longer than the auto-stop interval, sync-in restarts the sandbox, and the next command sends the dead id. The native run then fails with `native_session_interrupted: session not found` > - This pull request moves the session-id clear into the one shared start helper, so every restart path drops the stale id > - The benefit is that a run that waits through a long preparation opens a fresh session instead of failing ## Linked Issues or Issue Description **What happened?** Native runs on Daytona failed with `native_session_interrupted` and the error `session not found`. The failure came about 4 seconds after the provider session started. In every observed run, the workspace pack phase took longer than 15 minutes (917 s to 1442 s). This is longer than the default Daytona `autoStopInterval` of 15 minutes. **Expected behavior** After Daytona restarts an auto-stopped sandbox, the next command opens a new process session and the run continues. **Steps to reproduce** 1. Run one command on a Daytona lease, so the plugin opens and stores a session. 2. Let the sandbox stop (Daytona auto-stop after an idle gap). 3. Call workspace realize or sync-in on the lease. The hook restarts the sandbox. 4. Run the next command. It uses the stored session id, and Daytona answers `session not found`. No related open PR exists. The undeployed native fixes on master (#18992, #19025, #19026, #19004) do not touch the Daytona session store. ## What Changed - `ensureSandboxStarted` in the Daytona plugin now takes the lease scope. When the sandbox is not running, it clears the stored session id before it starts or recovers the sandbox. - All seven callers (execute, resume, realize workspace, interactive setup, runner ingress, sync-in, sync-out) pass their lease scope. The compiler enforces the scope on any new caller. - The execute hook no longer has its own copy of the clear. The resume hook keeps its clear because it also closes duplex channels before the restart. - Added three regression tests: a fresh session after realize restarts an auto-stopped sandbox, a fresh session after sync-in restarts it, and no new session when realize finds the sandbox already running. ## Verification - Red before the fix: `pnpm exec vitest run src/plugin.test.ts -t "auto-stopped sandbox|already running"` in `packages/plugins/sandbox-providers/daytona`. The two restart tests failed with `expected "vi.fn()" to be called 2 times, but got 1 times`. The running-sandbox safety test passed. - Green after the fix: the full Daytona plugin suite (`pnpm exec vitest run`) passes, 542 passed and 8 skipped. - `pnpm typecheck` in the Daytona plugin package passes. - Not done: a live Daytona run that idles past the auto-stop interval. The fix needs deployment before the staging failure can be checked again. ## Risks - Low risk. The clear runs only when the sandbox is not in the `started` state. A stopped sandbox has no live session, so no live shell can leak. A running sandbox keeps its stored session, and a test covers this case. - The recover path for a recoverable `error` state now also drops the id. A recovered sandbox restarts, so its old session is also gone. - Duplex channels on a restarted sandbox are still closed only by the resume hook. This PR does not change that behavior. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, browser reads of staging run records). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
7220 lines
322 KiB
TypeScript
7220 lines
322 KiB
TypeScript
import { createHash } from "node:crypto";
|
||
import { execFileSync } from "node:child_process";
|
||
import { promises as fs } from "node:fs";
|
||
import os from "node:os";
|
||
import path from "node:path";
|
||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||
|
||
const mockCreate = vi.hoisted(() => vi.fn());
|
||
const mockGet = vi.hoisted(() => vi.fn());
|
||
const mockSnapshotGet = vi.hoisted(() => vi.fn());
|
||
const mockSnapshotDelete = vi.hoisted(() => vi.fn());
|
||
const { MockDaytonaNotFoundError, MockDaytonaTimeoutError } = vi.hoisted(() => {
|
||
class MockDaytonaNotFoundError extends Error {}
|
||
class MockDaytonaTimeoutError extends Error {}
|
||
return { MockDaytonaNotFoundError, MockDaytonaTimeoutError };
|
||
});
|
||
|
||
vi.mock("@daytonaio/sdk", () => ({
|
||
Daytona: class MockDaytona {
|
||
create = mockCreate;
|
||
get = mockGet;
|
||
snapshot = {
|
||
get: mockSnapshotGet,
|
||
delete: mockSnapshotDelete,
|
||
};
|
||
constructor(_config?: unknown) {}
|
||
},
|
||
DaytonaNotFoundError: MockDaytonaNotFoundError,
|
||
DaytonaTimeoutError: MockDaytonaTimeoutError,
|
||
}));
|
||
|
||
import plugin, {
|
||
setDaytonaTimingClockForTest,
|
||
setDaytonaHandleFreshnessClockForTest,
|
||
__resetDaytonaSandboxHandleCacheForTest,
|
||
__getDaytonaWritableDirsForTest,
|
||
__setDaytonaPluginContextForTest,
|
||
} from "./plugin.js";
|
||
import type { PluginContext } from "@paperclipai/plugin-sdk";
|
||
import { environmentSyncErrorData, readEnvironmentSyncErrorDiagnostic, withEnvironmentSyncErrorCapture, environmentCreationCleanupErrorData, readEnvironmentCreationCleanupError, readEnvironmentAcquisitionDiagnostic } from "@paperclipai/plugin-sdk";
|
||
import manifest from "./manifest.js";
|
||
import { parseTarVerboseListingLine, splitLinkEntryOnce } from "./file-sync.js";
|
||
|
||
describe("Daytona read-only resource inspection", () => {
|
||
const params = { driverKey: "daytona", companyId: "fixture-company", environmentId: "fixture-environment",
|
||
providerLeaseId: "sandbox-123", config: { apiKey: "fixture-not-a-key" } };
|
||
afterEach(() => { vi.clearAllMocks(); vi.unstubAllEnvs(); __resetDaytonaSandboxHandleCacheForTest(); });
|
||
it.each(["started", "stopped"])("observes a %s sandbox without lifecycle calls", async state => {
|
||
const sandbox = createMockSandbox({ state }); mockGet.mockResolvedValue(sandbox);
|
||
expect(await plugin.definition.onEnvironmentInspectLease!(params)).toEqual({ providerLeaseId: "sandbox-123", state: state === "stopped" ? "stopped" : "active" });
|
||
expect(mockGet).toHaveBeenLastCalledWith("sandbox-123");
|
||
for (const method of [sandbox.start, sandbox.stop, sandbox.delete, sandbox.archive, sandbox.refreshData]) expect(method).not.toHaveBeenCalled();
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
});
|
||
it.each(["active", "destroyed"])("inspects a normal lease with creation provenance by recorded ID: %s", async state => {
|
||
const provenance = { creationOperationId: "16cd534e-27a1-45de-a4ad-5adbb6481361" };
|
||
if (state === "active") mockGet.mockResolvedValue(createMockSandbox());
|
||
else mockGet.mockRejectedValue(new MockDaytonaNotFoundError("gone"));
|
||
expect(await plugin.definition.onEnvironmentInspectLease!({ ...params, leaseMetadata: provenance }))
|
||
.toEqual({ providerLeaseId: "sandbox-123", state });
|
||
expect(mockGet).toHaveBeenCalledExactlyOnceWith("sandbox-123");
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
if (state === "destroyed") expect(await plugin.definition.onIdleDrain!(new AbortController().signal)).toBe("none");
|
||
});
|
||
it("accepts only typed provider not-found as deletion proof", async () => {
|
||
__resetDaytonaSandboxHandleCacheForTest();
|
||
mockGet.mockRejectedValue(new MockDaytonaNotFoundError("not found"));
|
||
expect(await plugin.definition.onEnvironmentInspectLease!(params)).toEqual({ providerLeaseId: "sandbox-123", state: "destroyed" });
|
||
expect(await plugin.definition.onIdleDrain!(new AbortController().signal)).toBe("none");
|
||
});
|
||
it("does not use an ambient account credential to certify historical deletion", async () => {
|
||
expect(await plugin.definition.onEnvironmentInspectLease!({ ...params, config: {} })).toEqual({ providerLeaseId: "sandbox-123", state: "unknown" });
|
||
expect(mockGet).not.toHaveBeenCalled();
|
||
});
|
||
it.each(["DAYTONA_API_URL", "DAYTONA_SERVER_URL"])("does not substitute ambient %s for the recorded API endpoint", async variable => {
|
||
vi.stubEnv(variable, "https://replacement.example.test/api");
|
||
expect((await plugin.definition.onEnvironmentInspectLease!(params)).state).toBe("unknown");
|
||
expect(mockGet).not.toHaveBeenCalled();
|
||
mockGet.mockRejectedValue(new MockDaytonaNotFoundError("gone"));
|
||
expect((await plugin.definition.onEnvironmentInspectLease!({ ...params, config: { ...params.config, apiUrl: "https://recorded.example.test/api" } })).state).toBe("destroyed");
|
||
});
|
||
it.each(["401 unauthorized", "404 proxy not found", "network unavailable"])("does not turn %s into proof", async message => {
|
||
mockGet.mockRejectedValue(new Error(message));
|
||
await expect(plugin.definition.onEnvironmentInspectLease!(params)).rejects.toThrow(message);
|
||
});
|
||
it("rejects another resource and re-reads provider state on every observation", async () => {
|
||
mockGet.mockResolvedValueOnce(createMockSandbox({ id: "another-sandbox" })).mockRejectedValueOnce(new MockDaytonaNotFoundError("gone"));
|
||
expect((await plugin.definition.onEnvironmentInspectLease!(params)).state).toBe("unknown");
|
||
expect((await plugin.definition.onEnvironmentInspectLease!(params)).state).toBe("destroyed");
|
||
expect(mockGet).toHaveBeenCalledTimes(2);
|
||
});
|
||
});
|
||
|
||
describe("Daytona durable creation identity", () => {
|
||
const id = "16cd534e-27a1-45de-a4ad-5adbb6481361";
|
||
const runId = "8cbd0194-e322-4b5b-9800-df4d5a682479";
|
||
const params = { driverKey: "daytona", companyId: "fixture-company", environmentId: "fixture-environment",
|
||
providerLeaseId: id, config: { apiKey: "fixture-not-a-key" },
|
||
creationOperation: { id, runId } };
|
||
beforeEach(() => { mockCreate.mockReset(); mockGet.mockReset(); __resetDaytonaSandboxHandleCacheForTest(); });
|
||
afterEach(() => { vi.unstubAllEnvs(); __resetDaytonaSandboxHandleCacheForTest(); });
|
||
function resource() { return { ...createMockSandbox({ name: `paperclip-create-${id}` }), labels: {
|
||
"paperclip-provider": "daytona", "paperclip-create-attempt": id,
|
||
"paperclip-company-id": params.companyId, "paperclip-environment-id": params.environmentId,
|
||
"paperclip-run-id": runId,
|
||
} }; }
|
||
it("locates the exact resource by stable name without creation or lifecycle calls", async () => {
|
||
const sandbox = resource(); mockGet.mockResolvedValue(sandbox);
|
||
const observed = await plugin.definition.onEnvironmentInspectLease!(params);
|
||
expect(mockGet).toHaveBeenCalledWith(`paperclip-create-${id}`);
|
||
expect(observed).toMatchObject({ providerLeaseId: `paperclip-create-${id}`, state: "active",
|
||
creationCleanup: { observedProviderLeaseId: sandbox.id, attemptId: id, runId } });
|
||
expect(mockCreate).not.toHaveBeenCalled(); expect(sandbox.delete).not.toHaveBeenCalled();
|
||
await plugin.definition.onEnvironmentDestroyLease!({ ...params, providerLeaseId: observed.providerLeaseId,
|
||
leaseMetadata: { failedCreateCleanup: observed.creationCleanup } });
|
||
expect(sandbox.delete).toHaveBeenCalledOnce();
|
||
});
|
||
it("retains uncertainty when a creation name is absent", async () => {
|
||
mockGet.mockRejectedValue(new MockDaytonaNotFoundError("not yet visible"));
|
||
expect(await plugin.definition.onEnvironmentInspectLease!(params)).toEqual({ providerLeaseId: `paperclip-create-${id}`, state: "unknown" });
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
});
|
||
it("rejects a name or label owned by another creation", async () => {
|
||
mockGet.mockResolvedValue({ ...resource(), labels: { "paperclip-company-id": "foreign" } });
|
||
await expect(plugin.definition.onEnvironmentInspectLease!(params)).rejects.toThrow("ownership does not match");
|
||
});
|
||
it("requires the recorded endpoint when ambient configuration changes", async () => {
|
||
vi.stubEnv("DAYTONA_API_URL", "https://changed.example.test");
|
||
expect((await plugin.definition.onEnvironmentInspectLease!(params)).state).toBe("unknown");
|
||
expect(mockGet).not.toHaveBeenCalled();
|
||
});
|
||
});
|
||
|
||
describe("Daytona idle eligibility", () => {
|
||
const params = { driverKey: "daytona", companyId: "fixture-company", environmentId: "fixture-environment",
|
||
config: { apiKey: "fixture-not-a-key", reuseLease: true } };
|
||
const signal = new AbortController().signal;
|
||
beforeEach(() => { mockCreate.mockReset(); mockGet.mockReset(); __resetDaytonaSandboxHandleCacheForTest(); });
|
||
afterEach(() => { vi.restoreAllMocks(); __resetDaytonaSandboxHandleCacheForTest(); });
|
||
it("permits a fresh unused provider but remains conservative after provider access", async () => {
|
||
__resetDaytonaSandboxHandleCacheForTest();
|
||
const signal = new AbortController().signal;
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("none");
|
||
mockGet.mockRejectedValueOnce(new Error("provider unreachable"));
|
||
await plugin.definition.onEnvironmentProbe!({
|
||
driverKey: "daytona", companyId: "fixture-company", environmentId: "fixture-environment",
|
||
config: { apiKey: "fixture-not-a-key" },
|
||
}).catch(() => undefined);
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("present");
|
||
__resetDaytonaSandboxHandleCacheForTest();
|
||
});
|
||
|
||
it("permits idle after a probe's exact sandbox deletion succeeds", async () => {
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
expect((await plugin.definition.onEnvironmentProbe!(params)).ok).toBe(true);
|
||
expect(sandbox.delete).toHaveBeenCalledTimes(1);
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("none");
|
||
});
|
||
|
||
it.each(["probe", "failed acquisition", "failed interactive setup", "cancelled setup"] as const)(
|
||
"waits for remote deletion before allowing idle after %s",
|
||
async kind => {
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockResolvedValue(sandbox); mockGet.mockResolvedValue(sandbox);
|
||
if (kind === "failed acquisition" || kind === "failed interactive setup") {
|
||
sandbox.fs.createFolder.mockRejectedValueOnce(new Error("workspace setup failed"));
|
||
}
|
||
let finishDeletion!: () => void;
|
||
let remotelyDestroyed = false;
|
||
const deletion = new Promise<void>(resolve => {
|
||
finishDeletion = () => { remotelyDestroyed = true; resolve(); };
|
||
});
|
||
// The SDK's default resolves after acceptance. Only wait=true observes
|
||
// completed deletion, which can happen after the request is acknowledged.
|
||
sandbox.delete.mockImplementation(async (_timeout?: number, wait = false) => {
|
||
if (wait) await deletion;
|
||
});
|
||
const input = { ...params, config: { ...params.config, reuseLease: false } };
|
||
const pending = kind === "probe" ? plugin.definition.onEnvironmentProbe!(input)
|
||
: kind === "failed acquisition" ? plugin.definition.onEnvironmentAcquireLease!({ ...input, runId: "fixture-run" })
|
||
: kind === "failed interactive setup" ? plugin.definition.onEnvironmentStartInteractiveSetup!({ ...input, sessionId: "fixture-setup" })
|
||
: plugin.definition.onEnvironmentCancelInteractiveSetup!({ ...input, providerLeaseId: sandbox.id, reason: "user_cancelled" });
|
||
let settled = false;
|
||
const outcome = pending.then(() => { settled = true; }, () => { settled = true; });
|
||
try {
|
||
await vi.waitFor(() => expect(sandbox.delete).toHaveBeenCalledTimes(1));
|
||
await new Promise<void>(resolve => setImmediate(resolve));
|
||
expect(remotelyDestroyed).toBe(false);
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("present");
|
||
expect(settled).toBe(false);
|
||
finishDeletion(); await outcome;
|
||
expect(remotelyDestroyed).toBe(true);
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("none");
|
||
} finally {
|
||
finishDeletion(); await outcome;
|
||
}
|
||
},
|
||
);
|
||
|
||
it("retains a failed probe deletion until exact provider cleanup succeeds", async () => {
|
||
const sandbox = createMockSandbox();
|
||
sandbox.delete.mockRejectedValueOnce(new Error("delete unavailable"));
|
||
mockCreate.mockResolvedValue(sandbox); mockGet.mockResolvedValue(sandbox);
|
||
await plugin.definition.onEnvironmentProbe!(params);
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("present");
|
||
await plugin.definition.onEnvironmentDestroyLease!({ ...params, providerLeaseId: sandbox.id });
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("none");
|
||
});
|
||
|
||
it.each(["stop", "release", "retain", "destroy"])("clears verified %s without clearing another account", async kind => {
|
||
const first = createMockSandbox({ id: "sandbox-a" });
|
||
const second = createMockSandbox({ id: "sandbox-b" });
|
||
mockCreate.mockResolvedValueOnce(first).mockResolvedValueOnce(second);
|
||
first.delete.mockRejectedValueOnce(new Error("deferred"));
|
||
second.delete.mockRejectedValueOnce(new Error("deferred"));
|
||
await plugin.definition.onEnvironmentProbe!(params);
|
||
const other = { ...params, config: { ...params.config, apiKey: "fixture-other-account" } };
|
||
await plugin.definition.onEnvironmentProbe!(other);
|
||
mockGet.mockResolvedValue(first);
|
||
const cleanup = { ...params, providerLeaseId: first.id };
|
||
if (kind === "stop") await plugin.definition.onEnvironmentStopLease!(cleanup);
|
||
else if (kind === "release") await plugin.definition.onEnvironmentReleaseLease!(cleanup);
|
||
else if (kind === "retain") await plugin.definition.onEnvironmentReleaseLease!({ ...cleanup, resourceDisposition: "stop_and_retain" });
|
||
else await plugin.definition.onEnvironmentDestroyLease!(cleanup);
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("present");
|
||
mockGet.mockResolvedValue(second);
|
||
await plugin.definition.onEnvironmentDestroyLease!({ ...other, providerLeaseId: second.id });
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("none");
|
||
});
|
||
|
||
it("keeps a pending create after exact cleanup until it settles and absence is re-proven", async () => {
|
||
let finishCreate!: (sandbox: ReturnType<typeof createMockSandbox>) => void;
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockImplementation(() => new Promise(resolve => { finishCreate = resolve; }));
|
||
const probe = plugin.definition.onEnvironmentProbe!(params);
|
||
await vi.waitFor(() => expect(finishCreate).toBeTypeOf("function"));
|
||
mockGet.mockRejectedValue(new MockDaytonaNotFoundError("gone"));
|
||
await plugin.definition.onEnvironmentDestroyLease!({ ...params, providerLeaseId: sandbox.id });
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("present");
|
||
// The late create must finish its own deletion. Its tracked raw request
|
||
// cannot be confused with a completed RPC or another resource's receipt.
|
||
finishCreate(sandbox); await probe;
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("none");
|
||
});
|
||
|
||
it("can clear verified absence after a failed deletion without starting the sandbox", async () => {
|
||
const sandbox = createMockSandbox(); sandbox.delete.mockRejectedValue(new Error("unavailable"));
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
await plugin.definition.onEnvironmentProbe!(params);
|
||
mockGet.mockRejectedValue(new MockDaytonaNotFoundError("gone"));
|
||
await plugin.definition.onEnvironmentInspectLease!({ ...params, providerLeaseId: sandbox.id });
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("none");
|
||
expect(sandbox.start).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("clears both sandboxes when their successful stops overlap", async () => {
|
||
const first = createMockSandbox({ id: "sandbox-a" });
|
||
const second = createMockSandbox({ id: "sandbox-b" });
|
||
for (const sandbox of [first, second]) sandbox.delete.mockRejectedValueOnce(new Error("deferred"));
|
||
mockCreate.mockResolvedValueOnce(first).mockResolvedValueOnce(second);
|
||
await plugin.definition.onEnvironmentProbe!(params);
|
||
await plugin.definition.onEnvironmentProbe!(params);
|
||
mockGet.mockImplementation(async id => id === first.id ? first : second);
|
||
let finishFirst!: () => void, finishSecond!: () => void;
|
||
first.stop.mockImplementation(() => new Promise<void>(resolve => { finishFirst = resolve; }));
|
||
second.stop.mockImplementation(() => new Promise<void>(resolve => { finishSecond = resolve; }));
|
||
const stopFirst = plugin.definition.onEnvironmentStopLease!({ ...params, providerLeaseId: first.id });
|
||
const stopSecond = plugin.definition.onEnvironmentStopLease!({ ...params, providerLeaseId: second.id });
|
||
await vi.waitFor(() => {
|
||
expect(finishFirst).toBeTypeOf("function"); expect(finishSecond).toBeTypeOf("function");
|
||
});
|
||
finishFirst(); await stopFirst;
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("present");
|
||
finishSecond(); await stopSecond;
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("none");
|
||
});
|
||
|
||
it("keeps a stale stopped reply from clearing a start that finished during the read", async () => {
|
||
const sandbox = createMockSandbox({ state: "stopped" });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
let finishStart!: () => void, finishRead!: () => void;
|
||
let remoteRunning = false;
|
||
sandbox.start.mockImplementation(async () => {
|
||
await new Promise<void>(resolve => { finishStart = resolve; });
|
||
remoteRunning = true; sandbox.state = "started";
|
||
});
|
||
sandbox.stop.mockImplementation(async () => { remoteRunning = false; sandbox.state = "stopped"; });
|
||
const execute = plugin.definition.onEnvironmentExecute!({ ...params, bypassSession: true,
|
||
lease: { providerLeaseId: sandbox.id, metadata: {} }, command: "printf", args: ["hi"], timeoutMs: 1000 });
|
||
await vi.waitFor(() => expect(finishStart).toBeTypeOf("function"));
|
||
sandbox.refreshData.mockImplementationOnce(async () => {
|
||
await new Promise<void>(resolve => { finishRead = resolve; });
|
||
sandbox.state = "stopped"; // The reply contains the state captured before Start.
|
||
});
|
||
const stopped = plugin.definition.onEnvironmentStopLease!({ ...params, providerLeaseId: sandbox.id });
|
||
await vi.waitFor(() => expect(finishRead).toBeTypeOf("function"));
|
||
finishStart(); await execute;
|
||
finishRead(); await stopped;
|
||
expect(remoteRunning).toBe(true);
|
||
expect(sandbox.stop).not.toHaveBeenCalled();
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("present");
|
||
sandbox.state = "started"; // A subsequent fresh read observes the real state.
|
||
await plugin.definition.onEnvironmentStopLease!({ ...params, providerLeaseId: sandbox.id });
|
||
expect(remoteRunning).toBe(false);
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("none");
|
||
});
|
||
|
||
it("retains a raw command dispatch after its deadline and an intervening stop", async () => {
|
||
const sandbox = createMockSandbox(); mockGet.mockResolvedValue(sandbox);
|
||
let finishDispatch!: (value: { cmdId: string }) => void;
|
||
sandbox.process.executeSessionCommand.mockImplementation(() => new Promise(resolve => { finishDispatch = resolve; }));
|
||
vi.useFakeTimers();
|
||
try {
|
||
const execution = plugin.definition.onEnvironmentExecute!({ ...params,
|
||
lease: { providerLeaseId: sandbox.id, metadata: {} }, command: "printf", args: ["hi"],
|
||
timeoutMs: 1000, commandTimeoutMs: 1000 });
|
||
await vi.advanceTimersByTimeAsync(0);
|
||
expect(finishDispatch).toBeTypeOf("function");
|
||
await vi.advanceTimersByTimeAsync(1000);
|
||
expect(await execution).toMatchObject({ timedOut: true });
|
||
await plugin.definition.onEnvironmentStopLease!({ ...params, providerLeaseId: sandbox.id });
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("present");
|
||
finishDispatch({ cmdId: "late-command" });
|
||
await vi.advanceTimersByTimeAsync(0);
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("present");
|
||
await plugin.definition.onEnvironmentStopLease!({ ...params, providerLeaseId: sandbox.id });
|
||
expect(sandbox.stop).toHaveBeenCalledTimes(2);
|
||
expect(await plugin.definition.onIdleDrain!(signal)).toBe("none");
|
||
} finally { vi.useRealTimers(); }
|
||
});
|
||
});
|
||
|
||
function createMockSandbox(overrides: {
|
||
id?: string;
|
||
name?: string;
|
||
state?: string;
|
||
recoverable?: boolean;
|
||
workDir?: string;
|
||
autoDestroyAt?: string | null;
|
||
updatedAt?: string;
|
||
} = {}) {
|
||
return {
|
||
id: overrides.id ?? "sandbox-123",
|
||
name: overrides.name ?? "paperclip-sandbox",
|
||
state: overrides.state ?? "started",
|
||
recoverable: overrides.recoverable ?? false,
|
||
target: "us",
|
||
errorReason: null,
|
||
// A configured provider TTL populates `autoDestroyAt` after `setTtl` +
|
||
// `refreshData`. The default mock leaves it unset (no TTL configured).
|
||
autoDestroyAt: overrides.autoDestroyAt ?? undefined,
|
||
updatedAt: overrides.updatedAt,
|
||
autoDeleteInterval: -1,
|
||
getWorkDir: vi.fn().mockResolvedValue(overrides.workDir ?? "/home/daytona"),
|
||
getUserHomeDir: vi.fn().mockResolvedValue("/home/daytona"),
|
||
start: vi.fn().mockResolvedValue(undefined),
|
||
stop: vi.fn().mockResolvedValue(undefined),
|
||
recover: vi.fn().mockResolvedValue(undefined),
|
||
// Real `refreshData` re-reads live provider state and mutates `state` in
|
||
// place; the default mock leaves state untouched, and tests that exercise a
|
||
// provider-initiated auto-stop override it to flip `state` to "stopped".
|
||
refreshData: vi.fn().mockResolvedValue(undefined),
|
||
resize: vi.fn().mockResolvedValue(undefined),
|
||
delete: vi.fn().mockResolvedValue(undefined),
|
||
archive: vi.fn().mockResolvedValue(undefined),
|
||
setTtl: vi.fn().mockResolvedValue(undefined),
|
||
setAutoDeleteInterval: vi.fn().mockResolvedValue(undefined),
|
||
createSshAccess: vi.fn().mockResolvedValue({
|
||
token: "ssh-token-secret",
|
||
command: "ssh ssh-token-secret@ssh.app.daytona.io",
|
||
}),
|
||
getPreviewLink: vi.fn().mockResolvedValue({
|
||
url: "https://43127-sandbox-123.proxy.daytona.test",
|
||
token: "preview-token-secret",
|
||
}),
|
||
_experimental_createSnapshot: vi.fn().mockResolvedValue(undefined),
|
||
fs: {
|
||
createFolder: vi.fn().mockResolvedValue(undefined),
|
||
uploadFile: vi.fn().mockResolvedValue(undefined),
|
||
deleteFile: vi.fn().mockResolvedValue(undefined),
|
||
// Native batch file-transfer primitives (Daytona SDK 0.171.0). Extended
|
||
// here so the sync hooks can be exercised without a real SDK install.
|
||
uploadFiles: vi.fn().mockResolvedValue(undefined),
|
||
downloadFiles: vi.fn().mockResolvedValue([]),
|
||
setFilePermissions: vi.fn().mockResolvedValue(undefined),
|
||
},
|
||
process: {
|
||
executeCommand: vi.fn().mockResolvedValue({
|
||
exitCode: 0,
|
||
result: "bash",
|
||
artifacts: { stdout: "bash" },
|
||
}),
|
||
// Session API (Daytona SDK 0.203.0). The exec hook opens one session per
|
||
// lease and dispatches every command into it. `executeSessionCommand`
|
||
// returns a `cmdId`; `getSessionCommand` reports the exit code; and
|
||
// `getSessionCommandLogs` returns separated `stdout` and `stderr`.
|
||
createSession: vi.fn().mockResolvedValue(undefined),
|
||
executeSessionCommand: vi.fn().mockResolvedValue({ cmdId: "cmd-1" }),
|
||
getSessionCommand: vi.fn().mockResolvedValue({ id: "cmd-1", command: "", exitCode: 0 }),
|
||
getSessionCommandLogs: vi.fn().mockResolvedValue({ stdout: "", stderr: "" }),
|
||
deleteSession: vi.fn().mockResolvedValue(undefined),
|
||
},
|
||
};
|
||
}
|
||
|
||
describe("Daytona sandbox provider plugin", () => {
|
||
beforeEach(() => {
|
||
mockCreate.mockReset();
|
||
mockGet.mockReset();
|
||
mockSnapshotGet.mockReset();
|
||
mockSnapshotDelete.mockReset();
|
||
vi.restoreAllMocks();
|
||
delete process.env.DAYTONA_API_KEY;
|
||
// The started-sandbox handle cache is process-scoped; clear it between tests
|
||
// so a handle memoized under a reused composite key never leaks forward.
|
||
__resetDaytonaSandboxHandleCacheForTest();
|
||
});
|
||
|
||
const snapshotRecord = { id: "a6b9e7d0-f886-471a-b461-640dd03e8d0f", name: "managed-snapshot", organizationId: "provider-account", createdAt: "2026-10-07T23:19:57.133Z" };
|
||
const snapshotProbe = { driverKey: "daytona", companyId: "company-1", environmentId: "env-1", runId: "probe-1",
|
||
resolveSnapshotIdentity: true, config: { apiKey: "fixture-key", snapshot: "managed-snapshot", reuseLease: false } };
|
||
|
||
it("qualifies a named snapshot by creating from its immutable provider ID", async () => {
|
||
mockSnapshotGet.mockResolvedValue(snapshotRecord);
|
||
mockCreate.mockResolvedValue(createMockSandbox());
|
||
const lease = await plugin.definition.onEnvironmentAcquireLease!(snapshotProbe);
|
||
expect(mockSnapshotGet).toHaveBeenCalledWith("managed-snapshot");
|
||
expect(mockCreate.mock.calls[0][0].snapshot).toBe(snapshotRecord.id);
|
||
expect(lease.metadata).toMatchObject({ snapshot: "managed-snapshot", snapshotIdentity: {
|
||
...snapshotRecord, provider: "daytona", requestedRef: "managed-snapshot",
|
||
} });
|
||
});
|
||
|
||
it("does not resolve snapshot names during ordinary acquisition", async () => {
|
||
mockCreate.mockResolvedValue(createMockSandbox());
|
||
await plugin.definition.onEnvironmentAcquireLease!({ ...snapshotProbe, resolveSnapshotIdentity: undefined });
|
||
expect(mockSnapshotGet).not.toHaveBeenCalled();
|
||
expect(mockCreate.mock.calls[0][0].snapshot).toBe("managed-snapshot");
|
||
});
|
||
|
||
it.each([{ id: "mutable-name" }, { organizationId: "" }, { name: "different-name" }, { createdAt: "invalid" }])(
|
||
"rejects unverified snapshot identity %j before allocation", async invalid => {
|
||
mockSnapshotGet.mockResolvedValue({ ...snapshotRecord, ...invalid });
|
||
await expect(plugin.definition.onEnvironmentAcquireLease!(snapshotProbe)).rejects.toThrow("immutable snapshot identity");
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
},
|
||
);
|
||
|
||
it("never allocates after a snapshot identity lookup times out", async () => {
|
||
vi.useFakeTimers();
|
||
try {
|
||
let resolveLookup!: (value: typeof snapshotRecord) => void;
|
||
mockSnapshotGet.mockReturnValue(new Promise(resolve => { resolveLookup = resolve; }));
|
||
const result = plugin.definition.onEnvironmentAcquireLease!(snapshotProbe).catch(error => error);
|
||
await vi.advanceTimersByTimeAsync(10_001);
|
||
expect(await result).toBeInstanceOf(Error);
|
||
resolveLookup(snapshotRecord);
|
||
await vi.advanceTimersByTimeAsync(1);
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
} finally { vi.useRealTimers(); }
|
||
});
|
||
|
||
it("declares environment lifecycle handlers", async () => {
|
||
expect(await plugin.definition.onHealth?.()).toEqual({
|
||
status: "ok",
|
||
message: "Daytona sandbox provider plugin healthy",
|
||
});
|
||
expect(plugin.definition.onEnvironmentAcquireLease).toBeTypeOf("function");
|
||
expect(plugin.definition.onEnvironmentExecute).toBeTypeOf("function");
|
||
expect(plugin.definition.onEnvironmentStartInteractiveSetup).toBeTypeOf("function");
|
||
expect(plugin.definition.onEnvironmentCaptureTemplate).toBeTypeOf("function");
|
||
expect(manifest.environmentDrivers?.[0]).toMatchObject({
|
||
defaultAcquireTimeoutMs: 300_000,
|
||
supportsInteractiveSetup: true,
|
||
interactiveSetupConnectionTypes: ["ssh"],
|
||
supportsTemplateCapture: true,
|
||
templateRefKind: "snapshot",
|
||
supportsTemplateDelete: true,
|
||
// Daytona streams incremental session output, so it declares the opt-in
|
||
// capability that selects the session-output streaming path.
|
||
sandboxCapabilities: { incrementalSessionOutput: true },
|
||
});
|
||
});
|
||
|
||
it("declares the concurrent-sync-operations capability so the host may parallelize sync operations", () => {
|
||
// Daytona runs file transfers into and out of the sandbox in parallel, so it
|
||
// declares the opt-in capability. The host resolves it `true` only when the
|
||
// worker also verifies both sync verbs, which the sync hooks provide.
|
||
expect(manifest.environmentDrivers?.[0]?.sandboxCapabilities).toMatchObject({
|
||
concurrentSyncOperations: true,
|
||
});
|
||
});
|
||
|
||
it("declares the duplex-command-stream capability and the four channel handlers", () => {
|
||
// Daytona carries the callback bridge on one duplex channel, so it declares
|
||
// the opt-in capability. The host resolves it `true` only when the worker also
|
||
// verifies the `duplexChannelOpen` handler, which the four handlers provide.
|
||
expect(manifest.environmentDrivers?.[0]?.sandboxCapabilities).toMatchObject({
|
||
duplexCommandStream: true,
|
||
});
|
||
expect(plugin.definition.onDuplexChannelOpen).toBeTypeOf("function");
|
||
expect(plugin.definition.onDuplexChannelWrite).toBeTypeOf("function");
|
||
expect(plugin.definition.onDuplexChannelStop).toBeTypeOf("function");
|
||
expect(plugin.definition.onDuplexChannelClose).toBeTypeOf("function");
|
||
});
|
||
|
||
it("declares and returns private authenticated runner WebSocket ingress", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
const base = {
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { image: "node:20", timeoutMs: 300_000, reuseLease: false },
|
||
};
|
||
const lease = await plugin.definition.onEnvironmentAcquireLease?.({
|
||
...base,
|
||
runId: "00000000-0000-4000-8000-000000000001",
|
||
});
|
||
const endpoint = await plugin.definition.onEnvironmentRunnerIngressEndpoint?.({
|
||
...base,
|
||
lease: lease!,
|
||
port: 43_127,
|
||
path: "/api/runner/v1/connect/00000000-0000-4000-8000-000000000001",
|
||
});
|
||
|
||
expect(manifest.environmentDrivers?.[0]?.sandboxCapabilities).toMatchObject({
|
||
runnerWebSocketIngress: true,
|
||
});
|
||
expect(endpoint).toMatchObject({
|
||
kind: "authenticated_websocket",
|
||
websocketUrl:
|
||
"wss://43127-sandbox-123.proxy.daytona.test/api/runner/v1/connect/00000000-0000-4000-8000-000000000001",
|
||
secretHeaders: [
|
||
{ name: "X-Daytona-Preview-Token", value: "preview-token-secret" },
|
||
],
|
||
});
|
||
expect(endpoint?.websocketUrl).not.toContain("preview-token-secret");
|
||
expect(endpoint?.websocketUrl).not.toContain("host-key");
|
||
});
|
||
|
||
it("keeps ingress generation independent of token rotation and changes it after a sandbox lifecycle revision", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ updatedAt: "2026-08-25T10:00:00.000Z" });
|
||
sandbox.getPreviewLink
|
||
.mockResolvedValueOnce({
|
||
url: "https://43127-sandbox-123.proxy.daytona.test",
|
||
token: "preview-token-1",
|
||
})
|
||
.mockResolvedValueOnce({
|
||
url: "https://43127-sandbox-123.proxy.daytona.test",
|
||
token: "preview-token-2",
|
||
})
|
||
.mockResolvedValueOnce({
|
||
url: "https://43127-sandbox-123.proxy.daytona.test",
|
||
token: "preview-token-3",
|
||
});
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
const base = {
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { image: "node:20", timeoutMs: 300_000, reuseLease: false },
|
||
};
|
||
const lease = await plugin.definition.onEnvironmentAcquireLease?.({
|
||
...base,
|
||
runId: "00000000-0000-4000-8000-000000000001",
|
||
});
|
||
const request = {
|
||
...base,
|
||
lease: lease!,
|
||
port: 43_127,
|
||
path: "/api/runner/v1/connect/00000000-0000-4000-8000-000000000001",
|
||
};
|
||
const first = await plugin.definition.onEnvironmentRunnerIngressEndpoint?.(request);
|
||
const second = await plugin.definition.onEnvironmentRunnerIngressEndpoint?.(request);
|
||
expect(second?.generation).toBe(first?.generation);
|
||
expect(second?.secretHeaders).not.toEqual(first?.secretHeaders);
|
||
|
||
sandbox.updatedAt = "2026-08-25T10:05:00.000Z";
|
||
const restarted = await plugin.definition.onEnvironmentRunnerIngressEndpoint?.(request);
|
||
expect(restarted?.generation).not.toBe(first?.generation);
|
||
});
|
||
|
||
it("bumps the plugin version so the server reconciles the stored manifest", () => {
|
||
// The bundled-plugin boot reconcile refreshes the stored manifest for an
|
||
// existing install only when the version changes. The acquisition budget needs
|
||
// the bump to reach an existing install.
|
||
expect(manifest.version).toBe("0.1.10");
|
||
});
|
||
|
||
it.each([false, true])("closes duplex routes on lease release even when bridge drain hangs: %s", async (hangDrain) => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
// A fake PTY handle records each host write, drives the data stream on demand,
|
||
// and records the kill and the disconnect.
|
||
const inputs: string[] = [];
|
||
let killed = 0;
|
||
let disconnected = 0;
|
||
let ptyOnData: ((data: Uint8Array) => void) | null = null;
|
||
const handle = {
|
||
async waitForConnection() {},
|
||
async sendInput(data: string | Uint8Array) {
|
||
inputs.push(typeof data === "string" ? data : new TextDecoder().decode(data));
|
||
},
|
||
wait() {
|
||
return new Promise<{ exitCode?: number }>(() => {});
|
||
},
|
||
async kill() {
|
||
killed += 1;
|
||
},
|
||
async disconnect() {
|
||
disconnected += 1;
|
||
},
|
||
};
|
||
const sandbox = createMockSandbox();
|
||
(sandbox.process as Record<string, unknown>).createPty = vi.fn(
|
||
async (options: { onData: (data: Uint8Array) => void }) => {
|
||
ptyOnData = options.onData;
|
||
return handle;
|
||
},
|
||
);
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
|
||
// Capture the data and the exit the worker forwards through `ctx.duplexChannel`.
|
||
// `ctx.duplexChannel.data` carries raw bytes; decode each chunk to text so the
|
||
// assertion below reads the plain-text payload.
|
||
const dataChunks: Array<{ hostRouteId: string; workerSessionId: string; chunk: string }> = [];
|
||
const restore = __setDaytonaPluginContextForTest({
|
||
duplexChannel: {
|
||
data: (hostRouteId: string, workerSessionId: string, chunk: Uint8Array) =>
|
||
dataChunks.push({ hostRouteId, workerSessionId, chunk: Buffer.from(chunk).toString("utf8") }),
|
||
exit: () => {},
|
||
},
|
||
} as unknown as PluginContext);
|
||
|
||
let finishBlocked: (() => void) | undefined;
|
||
let blocked: Promise<unknown> | undefined;
|
||
try {
|
||
await plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
runId: "run-1",
|
||
agentId: "agent-1",
|
||
executionWorkspaceId: "workspace-1",
|
||
adapterType: "codex_local",
|
||
config: { image: "node:20", timeoutMs: 300000, livenessTimeoutMs: 5, reuseLease: true },
|
||
});
|
||
|
||
const open = await plugin.definition.onDuplexChannelOpen?.({
|
||
hostRouteId: "route-1",
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-123",
|
||
command: ["node", "/paperclip/gateway.mjs"],
|
||
});
|
||
expect(open?.workerSessionId).toMatch(/^duplex-/);
|
||
// The open reply echoes the host route id, so the host binds the exact pair.
|
||
expect(open?.hostRouteId).toBe("route-1");
|
||
const workerSessionId = open?.workerSessionId ?? "";
|
||
|
||
// The launch wrapper sets raw mode with echo off and redirects diagnostics.
|
||
// It quotes each command argument and the diagnostics path as a shell word.
|
||
expect(inputs[0]).toContain("stty raw -echo");
|
||
expect(inputs[0]).toContain("exec 'node' '/paperclip/gateway.mjs'");
|
||
expect(inputs[0]).toMatch(/2>'\/tmp\/paperclip-duplex-.+\.log'/);
|
||
|
||
// A host write on the exact pair reaches the process on the same channel.
|
||
// `data` arrives in the wire-safe base64 form (see `ChannelBytesWireValue`
|
||
// in the plugin SDK's protocol.ts).
|
||
await plugin.definition.onDuplexChannelWrite?.({
|
||
hostRouteId: "route-1",
|
||
workerSessionId,
|
||
data: Buffer.from('{"version":1,"type":"heartbeat"}\n', "utf8").toString("base64"),
|
||
});
|
||
expect(inputs[1]).toBe('{"version":1,"type":"heartbeat"}\n');
|
||
|
||
// A write whose pair does not match the bound entry applies no bytes. The
|
||
// worker acts only on the exact live pair.
|
||
const inputsBeforeForeign = inputs.length;
|
||
await plugin.definition.onDuplexChannelWrite?.({
|
||
hostRouteId: "route-foreign",
|
||
workerSessionId,
|
||
data: Buffer.from("foreign\n", "utf8").toString("base64"),
|
||
});
|
||
expect(inputs.length).toBe(inputsBeforeForeign);
|
||
// A stop whose pair does not match the bound entry stops nothing.
|
||
const killedBeforeForeign = killed;
|
||
await plugin.definition.onDuplexChannelStop?.({
|
||
hostRouteId: "route-foreign",
|
||
workerSessionId,
|
||
});
|
||
expect(killed).toBe(killedBeforeForeign);
|
||
|
||
// Process output reaches the host as a data notification bound to the exact
|
||
// pair, so it echoes the host route id and the worker session id.
|
||
(ptyOnData as ((data: Uint8Array) => void) | null)?.(
|
||
new TextEncoder().encode('{"version":1,"type":"ready","address":"127.0.0.1:1"}\n'),
|
||
);
|
||
expect(dataChunks).toEqual([
|
||
{
|
||
hostRouteId: "route-1",
|
||
workerSessionId,
|
||
chunk: '{"version":1,"type":"ready","address":"127.0.0.1:1"}\n',
|
||
},
|
||
]);
|
||
|
||
if (hangDrain) {
|
||
sandbox.process.executeCommand.mockImplementationOnce(async () => {
|
||
await new Promise<void>(resolve => { finishBlocked = resolve; });
|
||
return { exitCode: 0, result: "done", artifacts: { stdout: "done" } };
|
||
});
|
||
blocked = plugin.definition.onEnvironmentExecute?.({ driverKey: "daytona", companyId: "company-1",
|
||
environmentId: "env-1", config: { timeoutMs: 300000 }, bypassSession: true,
|
||
lease: { providerLeaseId: "sandbox-123", metadata: {} }, command: "printf", args: ["done"] });
|
||
await vi.waitFor(() => expect(finishBlocked).toBeTypeOf("function"));
|
||
}
|
||
sandbox.stop.mockImplementation(async () => { expect(disconnected).toBe(1); });
|
||
// Route invalidation must happen before provider stop, including after drain timeout.
|
||
await plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-123",
|
||
config: { image: "node:20", timeoutMs: 300000, livenessTimeoutMs: 5, reuseLease: true },
|
||
});
|
||
expect(killed).toBeGreaterThanOrEqual(1);
|
||
expect(disconnected).toBe(1);
|
||
|
||
// The channel entry is gone, so a later write is a no-op and reaches no
|
||
// process.
|
||
const inputsBefore = inputs.length;
|
||
await plugin.definition.onDuplexChannelWrite?.({
|
||
hostRouteId: "route-1",
|
||
workerSessionId,
|
||
data: Buffer.from("late\n", "utf8").toString("base64"),
|
||
});
|
||
expect(inputs.length).toBe(inputsBefore);
|
||
} finally {
|
||
finishBlocked?.();
|
||
await blocked;
|
||
restore();
|
||
}
|
||
});
|
||
|
||
it("normalizes config and validates the API key fallback", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
|
||
const result = await plugin.definition.onEnvironmentValidateConfig?.({
|
||
driverKey: "daytona",
|
||
config: {
|
||
apiKey: " explicit-key ",
|
||
apiUrl: " https://app.daytona.io/api ",
|
||
target: " us ",
|
||
snapshot: " base-snapshot ",
|
||
language: " typescript ",
|
||
timeoutMs: "450000.9",
|
||
autoStopInterval: "15",
|
||
autoArchiveInterval: "60",
|
||
autoDeleteInterval: "-1",
|
||
reuseLease: true,
|
||
},
|
||
});
|
||
|
||
expect(result).toEqual({
|
||
ok: true,
|
||
normalizedConfig: {
|
||
apiKey: "explicit-key",
|
||
apiUrl: "https://app.daytona.io/api",
|
||
target: "us",
|
||
snapshot: "base-snapshot",
|
||
image: null,
|
||
language: "typescript",
|
||
timeoutMs: 450000,
|
||
livenessTimeoutMs: 30000,
|
||
cpu: null,
|
||
memory: null,
|
||
disk: null,
|
||
gpu: null,
|
||
autoStopInterval: 15,
|
||
autoArchiveInterval: 60,
|
||
autoDeleteInterval: -1,
|
||
ttlMinutes: null,
|
||
reuseLease: true,
|
||
archiveOnRelease: false,
|
||
},
|
||
});
|
||
});
|
||
|
||
it("applies quota-safety auto-stop/archive/delete defaults when unset", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
|
||
const result = await plugin.definition.onEnvironmentValidateConfig?.({
|
||
driverKey: "daytona",
|
||
config: {
|
||
snapshot: "base-snapshot",
|
||
timeoutMs: 300000,
|
||
reuseLease: true,
|
||
},
|
||
});
|
||
|
||
expect(result).toMatchObject({
|
||
ok: true,
|
||
normalizedConfig: {
|
||
autoStopInterval: 15,
|
||
autoArchiveInterval: 60,
|
||
autoDeleteInterval: 10080,
|
||
},
|
||
});
|
||
});
|
||
|
||
it("preserves an explicit 0/-1 to disable auto intervals", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
|
||
const result = await plugin.definition.onEnvironmentValidateConfig?.({
|
||
driverKey: "daytona",
|
||
config: {
|
||
snapshot: "base-snapshot",
|
||
timeoutMs: 300000,
|
||
autoStopInterval: 0,
|
||
autoArchiveInterval: 0,
|
||
autoDeleteInterval: -1,
|
||
reuseLease: true,
|
||
},
|
||
});
|
||
|
||
expect(result).toMatchObject({
|
||
ok: true,
|
||
normalizedConfig: {
|
||
autoStopInterval: 0,
|
||
autoArchiveInterval: 0,
|
||
autoDeleteInterval: -1,
|
||
},
|
||
});
|
||
});
|
||
|
||
it("forwards auto-archive/auto-delete defaults to the Daytona create call", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
runId: "run-1",
|
||
config: {
|
||
image: "node:20",
|
||
timeoutMs: 300000,
|
||
reuseLease: false,
|
||
},
|
||
});
|
||
|
||
const [createParams] = mockCreate.mock.calls[0] as [Record<string, unknown>];
|
||
expect(createParams).toMatchObject({
|
||
autoStopInterval: 15,
|
||
autoArchiveInterval: 60,
|
||
autoDeleteInterval: 10080,
|
||
});
|
||
});
|
||
|
||
describe("fresh acquisition deadline", () => {
|
||
const params = {
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1", runId: "run-1",
|
||
config: { image: "node:20", reuseLease: false },
|
||
};
|
||
|
||
beforeEach(() => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
vi.useFakeTimers();
|
||
});
|
||
afterEach(() => { vi.useRealTimers(); });
|
||
|
||
it.each(["create", "workspace", "shell", "expiry", "sentinel"] as const)("records the %s acquisition deadline without advancing late setup", async phase => {
|
||
const sandbox = createMockSandbox();
|
||
const never = () => new Promise<never>(() => {});
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
if (phase === "create") mockCreate.mockImplementation(never);
|
||
if (phase === "workspace") sandbox.getWorkDir.mockImplementation(never);
|
||
if (phase === "shell") sandbox.process.executeCommand.mockImplementation(never);
|
||
if (phase === "expiry") sandbox.setTtl.mockImplementation(never);
|
||
if (phase === "sentinel") sandbox.fs.uploadFile.mockImplementation(never);
|
||
const pending = plugin.definition.onEnvironmentAcquireLease!({ ...params,
|
||
config: { ...params.config, timeoutMs: 2_000 },
|
||
...(phase === "sentinel" ? { agentId: "agent-fixture", issueId: "issue-fixture" } : {}),
|
||
...(phase === "expiry" ? { requestedExpiresAt: new Date(Date.now() + 60_000).toISOString() } : {}),
|
||
}).catch(error => error);
|
||
await vi.advanceTimersByTimeAsync(2_000);
|
||
const error = await pending;
|
||
// The default lookup mock returns no sandbox, so a pending create reports `unavailable`.
|
||
expect(readEnvironmentAcquisitionDiagnostic(error)).toEqual({ phase, elapsedMs: 2_000, budgetMs: 2_000,
|
||
...(phase === "create" ? { providerState: "unavailable" } : {}) });
|
||
expect(error.message).toBe("Daytona lease acquisition timed out; allocation cleanup is pending");
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
if (phase !== "create") expect(mockGet).not.toHaveBeenCalled();
|
||
});
|
||
|
||
function ownedCreation(state: string) {
|
||
const requested = mockCreate.mock.calls.at(-1)![0];
|
||
return { ...createMockSandbox({ state }), name: requested.name, labels: { ...requested.labels } };
|
||
}
|
||
|
||
it.each([
|
||
["pulling_snapshot", "pulling_snapshot"], ["starting", "starting"], ["pending_build", "pending_build"],
|
||
["error", "error"], ["stopped", "other"], ["11184809", "other"],
|
||
])("records provider state %s as %s when creation is still pending at the deadline", async (state, expected) => {
|
||
mockCreate.mockImplementation(() => new Promise(() => {}));
|
||
let found: ReturnType<typeof ownedCreation> | undefined;
|
||
mockGet.mockImplementation(async () => (found = ownedCreation(state)));
|
||
const pending = plugin.definition.onEnvironmentAcquireLease!({ ...params, config: { ...params.config, timeoutMs: 2_000 } }).catch(error => error);
|
||
await vi.advanceTimersByTimeAsync(2_000);
|
||
const error = await pending;
|
||
expect(readEnvironmentAcquisitionDiagnostic(error)).toEqual({ phase: "create", elapsedMs: 2_000, budgetMs: 2_000, providerState: expected });
|
||
expect(mockGet).toHaveBeenCalledOnce();
|
||
expect(mockGet).toHaveBeenCalledWith(mockCreate.mock.calls[0][0].name);
|
||
// Read-only: no deletion, and the host-owned cleanup envelope is unchanged.
|
||
expect(found!.delete).not.toHaveBeenCalled();
|
||
expect(readEnvironmentCreationCleanupError(error)?.observedProviderLeaseId).toBeUndefined();
|
||
expect(JSON.stringify(environmentCreationCleanupErrorData(error, true))).not.toContain(found!.id);
|
||
});
|
||
|
||
it.each([
|
||
["absent", () => mockGet.mockRejectedValue(new MockDaytonaNotFoundError("private-not-found"))],
|
||
["unavailable", () => mockGet.mockRejectedValue(new Error("private-provider-failure"))],
|
||
["unavailable", () => mockGet.mockImplementation(async () => {
|
||
const foreign = ownedCreation("started"); foreign.labels["paperclip-run-id"] = "foreign-run"; return foreign;
|
||
})],
|
||
])("records %s when the deadline lookup cannot attribute an allocation", async (expected, arrange) => {
|
||
mockCreate.mockImplementation(() => new Promise(() => {}));
|
||
arrange();
|
||
const pending = plugin.definition.onEnvironmentAcquireLease!({ ...params, config: { ...params.config, timeoutMs: 2_000 } }).catch(error => error);
|
||
await vi.advanceTimersByTimeAsync(2_000);
|
||
const error = await pending;
|
||
expect(readEnvironmentAcquisitionDiagnostic(error)?.providerState).toBe(expected);
|
||
expect(JSON.stringify(environmentCreationCleanupErrorData(error, true))).not.toContain("private-");
|
||
});
|
||
|
||
it("bounds a hung deadline lookup well inside the host's RPC margin", async () => {
|
||
mockCreate.mockImplementation(() => new Promise(() => {}));
|
||
mockGet.mockImplementation(() => new Promise(() => {}));
|
||
let settled = false;
|
||
const pending = plugin.definition.onEnvironmentAcquireLease!(params).catch(error => { settled = true; return error; });
|
||
await vi.advanceTimersByTimeAsync(304_999);
|
||
expect(settled).toBe(false);
|
||
await vi.advanceTimersByTimeAsync(1);
|
||
const error = await pending;
|
||
expect(readEnvironmentAcquisitionDiagnostic(error)).toEqual({ phase: "create", elapsedMs: 300_000, budgetMs: 300_000, providerState: "unavailable" });
|
||
expect(error.message).toBe("Daytona lease acquisition timed out; allocation cleanup is pending");
|
||
});
|
||
|
||
it("keeps the timeout result when a late create settles during the deadline lookup", async () => {
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockImplementation(() => new Promise(resolve => setTimeout(() => resolve(sandbox), 2_500)));
|
||
mockGet.mockImplementation(() => new Promise(resolve => setTimeout(() => resolve(ownedCreation("starting")), 4_000)));
|
||
let settled = false;
|
||
const pending = plugin.definition.onEnvironmentAcquireLease!({ ...params, config: { ...params.config, timeoutMs: 2_000 } })
|
||
.catch(error => { settled = true; return error; });
|
||
// The host serializes the error on settlement; it must wait for the lookup.
|
||
await vi.advanceTimersByTimeAsync(3_000);
|
||
expect(settled).toBe(false);
|
||
await vi.advanceTimersByTimeAsync(3_000);
|
||
const error = await pending;
|
||
expect(readEnvironmentAcquisitionDiagnostic(error)).toEqual({ phase: "create", elapsedMs: 2_000, budgetMs: 2_000, providerState: "starting" });
|
||
expect(readEnvironmentCreationCleanupError(error)?.providerLeaseId).toBe(mockCreate.mock.calls[0][0].name);
|
||
expect(sandbox.getWorkDir).not.toHaveBeenCalled();
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it.each(["create", "workspace"] as const)("keeps the original %s observation when cleanup consumes the remaining deadline", async phase => {
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
const fail = () => new Promise<never>((_resolve, reject) => setTimeout(() => reject(new Error("private-provider-failure")), 100));
|
||
if (phase === "create") { mockCreate.mockImplementation(fail); mockGet.mockImplementation(() => new Promise(() => {})); }
|
||
else { sandbox.getWorkDir.mockImplementation(fail); sandbox.delete.mockImplementation(() => new Promise(() => {})); }
|
||
const pending = plugin.definition.onEnvironmentAcquireLease!({ ...params, config: { ...params.config, timeoutMs: 2_000 } }).catch(error => error);
|
||
await vi.advanceTimersByTimeAsync(2_000);
|
||
const error = await pending;
|
||
expect(readEnvironmentAcquisitionDiagnostic(error)).toEqual({ phase, elapsedMs: 100, budgetMs: 2_000 });
|
||
expect(error.message).toBe("Daytona lease acquisition timed out; allocation cleanup is pending");
|
||
expect(JSON.stringify(environmentCreationCleanupErrorData(error, true))).not.toContain("private-");
|
||
await vi.advanceTimersByTimeAsync(20_000);
|
||
expect(readEnvironmentAcquisitionDiagnostic(error)?.elapsedMs).toBe(100);
|
||
});
|
||
|
||
it("isolates overlapping acquisition observations and does not annotate successful leases", async () => {
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockImplementationOnce(() => new Promise(() => {})).mockResolvedValueOnce(sandbox);
|
||
sandbox.getWorkDir.mockImplementation(() => new Promise(() => {}));
|
||
const first = plugin.definition.onEnvironmentAcquireLease!({ ...params, config: { ...params.config, timeoutMs: 2_000 } }).catch(error => error);
|
||
await vi.advanceTimersByTimeAsync(100);
|
||
const second = plugin.definition.onEnvironmentAcquireLease!({ ...params, config: { ...params.config, timeoutMs: 3_000 } }).catch(error => error);
|
||
await vi.advanceTimersByTimeAsync(3_000);
|
||
expect(readEnvironmentAcquisitionDiagnostic(await first)).toEqual({ phase: "create", elapsedMs: 2_000, budgetMs: 2_000, providerState: "unavailable" });
|
||
expect(readEnvironmentAcquisitionDiagnostic(await second)).toEqual({ phase: "workspace", elapsedMs: 3_000, budgetMs: 3_000 });
|
||
mockCreate.mockResolvedValue(createMockSandbox());
|
||
expect(await plugin.definition.onEnvironmentAcquireLease!(params)).not.toHaveProperty("acquisitionDiagnostic");
|
||
});
|
||
|
||
it("allows a slow create and setup that finish inside the total budget", async () => {
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockImplementation(() => new Promise(resolve => setTimeout(() => resolve(sandbox), 280_000)));
|
||
sandbox.process.executeCommand.mockImplementation(() => new Promise(resolve => setTimeout(() => resolve({ result: "bash" }), 15_000)));
|
||
const pending = plugin.definition.onEnvironmentAcquireLease!(params);
|
||
await vi.advanceTimersByTimeAsync(295_000);
|
||
expect(await pending).toMatchObject({ providerLeaseId: sandbox.id });
|
||
expect(vi.getTimerCount()).toBe(0);
|
||
});
|
||
|
||
it("journals ownership before the host deadline when setup outlasts creation", async () => {
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockImplementation(() => new Promise(resolve => setTimeout(() => resolve(sandbox), 280_000)));
|
||
sandbox.process.executeCommand.mockImplementation(() => new Promise(resolve => setTimeout(() => resolve({ result: "bash" }), 55_000)));
|
||
const pending = plugin.definition.onEnvironmentAcquireLease!(params).catch(error => error);
|
||
await vi.advanceTimersByTimeAsync(300_000);
|
||
const cleanup = readEnvironmentCreationCleanupError(await pending);
|
||
expect(cleanup).toMatchObject({ companyId: params.companyId, environmentId: params.environmentId,
|
||
runId: params.runId, observedProviderLeaseId: sandbox.id, providerLeaseId: mockCreate.mock.calls[0][0].name });
|
||
await vi.advanceTimersByTimeAsync(35_000);
|
||
expect(sandbox.setTtl).not.toHaveBeenCalled();
|
||
expect(sandbox.fs.uploadFile).not.toHaveBeenCalled();
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("records an uncertain create and rejects its late result without starting setup", async () => {
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockImplementation(() => new Promise(resolve => setTimeout(() => resolve(sandbox), 310_000)));
|
||
const pending = plugin.definition.onEnvironmentAcquireLease!(params).catch(error => error);
|
||
await vi.advanceTimersByTimeAsync(300_000);
|
||
const cleanup = readEnvironmentCreationCleanupError(await pending);
|
||
expect(cleanup?.providerLeaseId).toBe(mockCreate.mock.calls[0][0].name);
|
||
expect(cleanup?.observedProviderLeaseId).toBeUndefined();
|
||
await vi.advanceTimersByTimeAsync(10_000);
|
||
expect(sandbox.getWorkDir).not.toHaveBeenCalled();
|
||
expect(sandbox.process.executeCommand).not.toHaveBeenCalled();
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("keeps a failed setup's ownership when inline deletion does not finish", async () => {
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
sandbox.getWorkDir.mockRejectedValue(new Error("workspace unavailable"));
|
||
sandbox.delete.mockImplementation(() => new Promise(() => {}));
|
||
const pending = plugin.definition.onEnvironmentAcquireLease!({ ...params,
|
||
config: { ...params.config, timeoutMs: 2_000 },
|
||
}).catch(error => error);
|
||
await vi.advanceTimersByTimeAsync(2_000);
|
||
expect(readEnvironmentCreationCleanupError(await pending)).toMatchObject({
|
||
observedProviderLeaseId: sandbox.id, companyId: params.companyId, runId: params.runId,
|
||
});
|
||
expect(sandbox.delete).toHaveBeenCalledOnce();
|
||
});
|
||
|
||
it("keeps ownership when setup and its inline deletion both reject", async () => {
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
sandbox.getWorkDir.mockRejectedValue(new Error("workspace unavailable"));
|
||
sandbox.delete.mockRejectedValue(new Error("delete unavailable"));
|
||
const error = await plugin.definition.onEnvironmentAcquireLease!(params).catch(error => error);
|
||
expect(readEnvironmentCreationCleanupError(error)).toMatchObject({
|
||
observedProviderLeaseId: sandbox.id, companyId: params.companyId, runId: params.runId,
|
||
});
|
||
expect(vi.getTimerCount()).toBe(0);
|
||
expect(readEnvironmentAcquisitionDiagnostic(error)).toEqual({ phase: "workspace", elapsedMs: 0, budgetMs: 300_000 });
|
||
});
|
||
});
|
||
|
||
describe("failed sandbox creation cleanup", () => {
|
||
const params = {
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1", runId: "run-1",
|
||
config: { image: "node:20", timeoutMs: 300_000, reuseLease: false, livenessTimeoutMs: 100 },
|
||
};
|
||
const createError = new MockDaytonaTimeoutError("Failed to create and start sandbox within 300 seconds");
|
||
|
||
beforeEach(() => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
mockCreate.mockRejectedValue(createError);
|
||
});
|
||
|
||
function ownedSandbox() {
|
||
const requested = mockCreate.mock.calls.at(-1)![0];
|
||
return { ...createMockSandbox(), name: requested.name, labels: { ...requested.labels } };
|
||
}
|
||
|
||
async function unresolvedCreation() {
|
||
mockGet.mockRejectedValue(new MockDaytonaNotFoundError("not visible yet"));
|
||
const error = await plugin.definition.onEnvironmentAcquireLease!(params).catch(error => error);
|
||
const cleanup = readEnvironmentCreationCleanupError(error);
|
||
expect(cleanup).not.toBeNull();
|
||
return { error, cleanup: cleanup! };
|
||
}
|
||
|
||
async function journalObservation(cleanup: NonNullable<ReturnType<typeof readEnvironmentCreationCleanupError>>) {
|
||
const error = await plugin.definition.onEnvironmentDestroyLease!({ ...params,
|
||
providerLeaseId: cleanup.providerLeaseId, leaseMetadata: { failedCreateCleanup: cleanup },
|
||
}).catch(error => error);
|
||
const observed = readEnvironmentCreationCleanupError(error);
|
||
expect(observed?.observedProviderLeaseId).toBeTruthy();
|
||
return observed!;
|
||
}
|
||
|
||
it("hands non-secret cleanup ownership to the host when creation is uncertain", async () => {
|
||
const { error, cleanup } = await unresolvedCreation();
|
||
expect(cleanup).toMatchObject({ companyId: params.companyId, environmentId: params.environmentId, runId: params.runId,
|
||
providerLeaseId: mockCreate.mock.calls[0][0].name, labels: mockCreate.mock.calls[0][0].labels });
|
||
expect(cleanup.accountFingerprint).toMatch(/^[a-f0-9]{64}$/);
|
||
expect(JSON.stringify(environmentCreationCleanupErrorData(error))).not.toContain("host-key");
|
||
expect(JSON.stringify(environmentCreationCleanupErrorData(error))).not.toContain(createError.message);
|
||
});
|
||
|
||
it("preserves ownership when the SDK adds labels to its mutable create input", async () => {
|
||
mockCreate.mockImplementation(async (input) => {
|
||
// Daytona 0.203.0 writes its default language into params.labels.
|
||
input.labels["code-toolbox-language"] = "python";
|
||
throw createError;
|
||
});
|
||
const { error, cleanup } = await unresolvedCreation();
|
||
expect(environmentCreationCleanupErrorData(error)).toBeDefined();
|
||
expect(cleanup.labels).not.toHaveProperty("code-toolbox-language");
|
||
const orphan = ownedSandbox();
|
||
mockGet.mockResolvedValue(orphan);
|
||
const observed = await journalObservation(cleanup);
|
||
expect(orphan.delete).not.toHaveBeenCalled();
|
||
await expect(plugin.definition.onEnvironmentDestroyLease!({ ...params,
|
||
providerLeaseId: cleanup.providerLeaseId, leaseMetadata: { failedCreateCleanup: observed },
|
||
})).resolves.toEqual({ providerLeaseId: cleanup.providerLeaseId, state: "destroyed" });
|
||
expect(orphan.delete).toHaveBeenCalledWith(10, true);
|
||
});
|
||
|
||
it("retries a late-visible failed creation using its persisted ownership envelope", async () => {
|
||
const { cleanup } = await unresolvedCreation();
|
||
const orphan = ownedSandbox(); mockGet.mockResolvedValue(orphan);
|
||
const observed = await journalObservation(cleanup);
|
||
expect(orphan.delete).not.toHaveBeenCalled();
|
||
await expect(plugin.definition.onEnvironmentDestroyLease!({ ...params, providerLeaseId: cleanup.providerLeaseId,
|
||
leaseMetadata: { failedCreateCleanup: observed } })).resolves.toEqual({ providerLeaseId: cleanup.providerLeaseId, state: "destroyed" });
|
||
expect(mockGet).toHaveBeenLastCalledWith(orphan.id);
|
||
expect(orphan.delete).toHaveBeenCalledWith(10, true);
|
||
expect(orphan.process.executeCommand).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("keeps a failed-creation retry unresolved while the allocation is still not visible", async () => {
|
||
const { cleanup } = await unresolvedCreation();
|
||
await expect(plugin.definition.onEnvironmentDestroyLease!({ ...params, providerLeaseId: cleanup.providerLeaseId,
|
||
leaseMetadata: { failedCreateCleanup: cleanup } })).rejects.toThrow();
|
||
});
|
||
|
||
it("accepts absence only after a matching provider ID was observed", async () => {
|
||
const { cleanup } = await unresolvedCreation();
|
||
const orphan = ownedSandbox(); mockGet.mockResolvedValue(orphan);
|
||
const observed = await journalObservation(cleanup);
|
||
expect(orphan.delete).not.toHaveBeenCalled();
|
||
mockGet.mockRejectedValue(new MockDaytonaNotFoundError("already deleted"));
|
||
await expect(plugin.definition.onEnvironmentDestroyLease!({ ...params,
|
||
providerLeaseId: cleanup.providerLeaseId, leaseMetadata: { failedCreateCleanup: observed },
|
||
})).resolves.toEqual({ providerLeaseId: cleanup.providerLeaseId, state: "destroyed" });
|
||
expect(mockGet).toHaveBeenLastCalledWith(orphan.id);
|
||
});
|
||
|
||
it("retains the observed ID when immediate deletion loses its acknowledgement", async () => {
|
||
let orphan: ReturnType<typeof ownedSandbox>;
|
||
mockGet.mockImplementation(async () => {
|
||
orphan = ownedSandbox();
|
||
orphan.delete.mockRejectedValue(new Error("delete response lost"));
|
||
return orphan;
|
||
});
|
||
const error = await plugin.definition.onEnvironmentAcquireLease!(params).catch(error => error);
|
||
const cleanup = readEnvironmentCreationCleanupError(error)!;
|
||
expect(cleanup.observedProviderLeaseId).toBe(orphan!.id);
|
||
mockGet.mockRejectedValue(new MockDaytonaNotFoundError("already deleted"));
|
||
await expect(plugin.definition.onEnvironmentDestroyLease!({ ...params,
|
||
providerLeaseId: cleanup.providerLeaseId, leaseMetadata: { failedCreateCleanup: cleanup },
|
||
})).resolves.toEqual({ providerLeaseId: cleanup.providerLeaseId, state: "destroyed" });
|
||
});
|
||
|
||
it.each(["company", "account", "labels"])("fences failed-creation retries by %s", async (mismatch) => {
|
||
const { cleanup } = await unresolvedCreation();
|
||
const orphan = ownedSandbox(); mockGet.mockResolvedValue(orphan);
|
||
if (mismatch === "labels") orphan.labels["paperclip-run-id"] = "foreign-run";
|
||
const retry = { ...params, providerLeaseId: cleanup.providerLeaseId, leaseMetadata: { failedCreateCleanup: cleanup } };
|
||
if (mismatch === "company") retry.companyId = "foreign-company";
|
||
if (mismatch === "account") process.env.DAYTONA_API_KEY = "another-account-key";
|
||
await expect(plugin.definition.onEnvironmentDestroyLease!(retry)).rejects.toThrow();
|
||
expect(orphan.delete).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("deletes the exact sandbox left behind when create times out, then preserves the create error", async () => {
|
||
let orphan: ReturnType<typeof ownedSandbox>;
|
||
mockGet.mockImplementation(async () => (orphan = ownedSandbox()));
|
||
await expect(plugin.definition.onEnvironmentAcquireLease?.(params)).rejects.toBe(createError);
|
||
expect(mockCreate.mock.calls[0][0].name).toMatch(/^paperclip-create-[0-9a-f-]{36}$/);
|
||
expect(mockGet).toHaveBeenCalledWith(mockCreate.mock.calls[0][0].name);
|
||
expect(orphan!.delete).toHaveBeenCalledWith(10, true);
|
||
expect(orphan!.process.executeCommand).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it.each(["paperclip-company-id", "paperclip-environment-id", "paperclip-run-id", "paperclip-create-attempt", "paperclip-provider"])("never deletes a sandbox with a mismatched %s", async (label) => {
|
||
let orphan: ReturnType<typeof ownedSandbox>;
|
||
mockGet.mockImplementation(async () => {
|
||
orphan = ownedSandbox(); orphan.labels[label] = "different-owner"; return orphan;
|
||
});
|
||
await expect(plugin.definition.onEnvironmentAcquireLease?.(params)).rejects.toThrow("cleanup could not be confirmed");
|
||
expect(orphan!.delete).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("never deletes a sandbox returned for a different name", async () => {
|
||
const wrong = { ...createMockSandbox(), labels: {} };
|
||
mockGet.mockResolvedValue(wrong);
|
||
await expect(plugin.definition.onEnvironmentAcquireLease?.(params)).rejects.toThrow("cleanup could not be confirmed");
|
||
expect(wrong.delete).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("does not treat a not-found lookup after uncertain creation as confirmed cleanup", async () => {
|
||
mockGet.mockRejectedValue(new MockDaytonaNotFoundError("missing"));
|
||
await expect(plugin.definition.onEnvironmentAcquireLease?.(params)).rejects.toThrow("cleanup could not be confirmed");
|
||
expect(mockGet).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it.each([400, 401, 403, 429, 500])("retains only SDK HTTP %i when failed creation cleanup is unconfirmed", async statusCode => {
|
||
mockCreate.mockRejectedValue(Object.assign(new Error("Bearer provider-secret-canary"), {
|
||
statusCode, headers: { authorization: "provider-secret-canary" },
|
||
}));
|
||
mockGet.mockRejectedValue(Object.assign(new Error("cleanup-secret-canary"), { statusCode: 503 }));
|
||
const failure = await plugin.definition.onEnvironmentAcquireLease!(params).catch(error => error);
|
||
expect(failure.message).toContain(`HTTP ${statusCode}`);
|
||
expect(failure.message).not.toContain("secret-canary");
|
||
expect(JSON.stringify(environmentCreationCleanupErrorData(failure, true))).not.toContain("secret-canary");
|
||
expect(readEnvironmentCreationCleanupError(failure)?.attemptId).toBeTruthy();
|
||
});
|
||
|
||
it.each([undefined, "401", -1, 600, NaN, 401.5])("omits non-HTTP statusCode %s without reading provider text", async statusCode => {
|
||
mockCreate.mockRejectedValue(Object.assign(new Error("HTTP 401 provider-secret-canary"), { statusCode }));
|
||
mockGet.mockRejectedValue(new Error("cleanup-secret-canary"));
|
||
const failure = await plugin.definition.onEnvironmentAcquireLease!(params).catch(error => error);
|
||
expect(failure.message).not.toContain("HTTP");
|
||
expect(failure.message).not.toContain("secret-canary");
|
||
});
|
||
|
||
it("does not invoke a provider status accessor", async () => {
|
||
const getter = vi.fn(() => { throw new Error("provider-secret-canary"); });
|
||
mockCreate.mockRejectedValue(Object.defineProperty(new Error("provider-secret-canary"), "statusCode", { get: getter }));
|
||
mockGet.mockRejectedValue(new Error("cleanup-secret-canary"));
|
||
const failure = await plugin.definition.onEnvironmentAcquireLease!(params).catch(error => error);
|
||
expect(failure.message).not.toContain("HTTP");
|
||
expect(failure.message).not.toContain("secret-canary");
|
||
expect(getter).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("reports unconfirmed cleanup when the provider lookup fails", async () => {
|
||
mockGet.mockRejectedValue(new Error("provider unavailable"));
|
||
await expect(plugin.definition.onEnvironmentAcquireLease?.(params)).rejects.toThrow("cleanup could not be confirmed");
|
||
});
|
||
|
||
it("reports unconfirmed cleanup when provider deletion fails", async () => {
|
||
mockGet.mockImplementation(async () => {
|
||
const orphan = ownedSandbox(); orphan.delete.mockRejectedValue(new Error("delete failed")); return orphan;
|
||
});
|
||
await expect(plugin.definition.onEnvironmentAcquireLease?.(params)).rejects.toThrow("cleanup could not be confirmed");
|
||
});
|
||
|
||
it("bounds a hung provider lookup and preserves both failure causes", async () => {
|
||
vi.useFakeTimers();
|
||
try {
|
||
mockGet.mockImplementation(() => new Promise(() => {}));
|
||
const result = plugin.definition.onEnvironmentAcquireLease!(params).catch((error: unknown) => error);
|
||
await vi.advanceTimersByTimeAsync(10_001);
|
||
const error = await result;
|
||
expect(error).toBeInstanceOf(AggregateError);
|
||
expect((error as AggregateError).errors[0]).toBe(createError);
|
||
expect((error as Error).message).toContain(mockCreate.mock.calls[0][0].name);
|
||
} finally { vi.useRealTimers(); }
|
||
});
|
||
|
||
it("bounds a hung provider deletion without returning a lease", async () => {
|
||
vi.useFakeTimers();
|
||
try {
|
||
mockGet.mockImplementation(async () => {
|
||
const orphan = ownedSandbox(); orphan.delete.mockImplementation(() => new Promise(() => {})); return orphan;
|
||
});
|
||
const result = plugin.definition.onEnvironmentAcquireLease!(params).catch((error: unknown) => error);
|
||
await vi.advanceTimersByTimeAsync(15_001);
|
||
expect(await result).toBeInstanceOf(AggregateError);
|
||
} finally { vi.useRealTimers(); }
|
||
});
|
||
|
||
it("uses distinct creation identities for concurrent attempts on the same run", async () => {
|
||
mockGet.mockRejectedValue(new MockDaytonaNotFoundError("missing"));
|
||
await Promise.allSettled([
|
||
plugin.definition.onEnvironmentAcquireLease?.(params),
|
||
plugin.definition.onEnvironmentAcquireLease?.(params),
|
||
]);
|
||
const requests = mockCreate.mock.calls.map(([request]) => request);
|
||
expect(requests).toHaveLength(2);
|
||
expect(new Set(requests.map((r) => r.name)).size).toBe(2);
|
||
expect(new Set(requests.map((r) => r.labels["paperclip-create-attempt"])).size).toBe(2);
|
||
});
|
||
});
|
||
|
||
it("rejects ambiguous or invalid config", async () => {
|
||
await expect(plugin.definition.onEnvironmentValidateConfig?.({
|
||
driverKey: "daytona",
|
||
config: {
|
||
apiUrl: "not-a-url",
|
||
image: "node:20",
|
||
snapshot: "snapshot-a",
|
||
timeoutMs: 0,
|
||
},
|
||
})).resolves.toEqual({
|
||
ok: false,
|
||
errors: [
|
||
"Daytona sandbox environments must set either image or snapshot, not both.",
|
||
"apiUrl must be a valid URL.",
|
||
"timeoutMs must be between 1 and 86400000.",
|
||
"Daytona sandbox environments require an API key in config or DAYTONA_API_KEY.",
|
||
],
|
||
});
|
||
});
|
||
|
||
it("probes by creating and then deleting a sandbox", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentProbe?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: {
|
||
snapshot: "base-snapshot",
|
||
timeoutMs: 300000,
|
||
reuseLease: false,
|
||
},
|
||
});
|
||
|
||
expect(mockCreate).toHaveBeenCalled();
|
||
expect(sandbox.fs.createFolder).toHaveBeenCalledWith("/home/daytona/paperclip-workspace", "755");
|
||
expect(sandbox.delete).toHaveBeenCalledWith(300, true);
|
||
expect(result).toMatchObject({
|
||
ok: true,
|
||
metadata: {
|
||
provider: "daytona",
|
||
shellCommand: "bash",
|
||
sandboxId: "sandbox-123",
|
||
remoteCwd: "/home/daytona/paperclip-workspace",
|
||
},
|
||
});
|
||
});
|
||
|
||
it.each([true, false])("records allocation bytes for reuseLease=%s", async reuseLease => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
|
||
const lease = await plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
runId: "run-1",
|
||
agentId: "agent-1",
|
||
executionWorkspaceId: "workspace-1",
|
||
adapterType: "codex_local",
|
||
config: {
|
||
image: "node:20",
|
||
timeoutMs: 300000,
|
||
reuseLease,
|
||
},
|
||
});
|
||
|
||
expect(lease).toMatchObject({
|
||
providerLeaseId: "sandbox-123",
|
||
metadata: {
|
||
provider: "daytona",
|
||
shellCommand: "bash",
|
||
sandboxId: "sandbox-123",
|
||
remoteCwd: "/home/daytona/paperclip-workspace",
|
||
reuseLease,
|
||
workspaceSentinel: {
|
||
path: "/home/daytona/paperclip-workspace/.paperclip-runtime/reusable-sandbox-lease.json",
|
||
result: "written",
|
||
},
|
||
},
|
||
});
|
||
const uploaded = sandbox.fs.uploadFile.mock.calls.find(call => call[1] ===
|
||
"/home/daytona/paperclip-workspace/.paperclip-runtime/reusable-sandbox-lease.json")?.[0];
|
||
expect(Buffer.isBuffer(uploaded)).toBe(true);
|
||
expect(lease?.metadata?.workspaceAllocationFiles).toEqual({ providerLeaseId: "sandbox-123",
|
||
remoteCwd: "/home/daytona/paperclip-workspace", files: [{
|
||
relativePath: ".paperclip-runtime/reusable-sandbox-lease.json",
|
||
sha256: createHash("sha256").update(uploaded as Buffer).digest("hex"),
|
||
}] });
|
||
expect(sandbox.fs.createFolder).toHaveBeenCalledWith(
|
||
"/home/daytona/paperclip-workspace/.paperclip-runtime",
|
||
"755",
|
||
);
|
||
expect(sandbox.fs.uploadFile).toHaveBeenCalledWith(
|
||
expect.any(Buffer),
|
||
"/home/daytona/paperclip-workspace/.paperclip-runtime/reusable-sandbox-lease.json",
|
||
300,
|
||
);
|
||
});
|
||
|
||
it("does not configure a provider ttl when the acquire carries no requested expiry", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
|
||
const lease = await plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
runId: "run-1",
|
||
config: { image: "node:20", timeoutMs: 300000, reuseLease: false },
|
||
});
|
||
|
||
// A generic caller keeps the current behavior: no provider ttl, no expiry.
|
||
expect(sandbox.setTtl).not.toHaveBeenCalled();
|
||
expect(lease?.expiresAt ?? null).toBeNull();
|
||
});
|
||
|
||
it.each([{ image: "node:20" }, { snapshot: "base-snapshot" }])("sets the wall-clock ttl in the create request for %j", async source => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const autoDestroyAt = new Date(Date.now() + 30 * 60_000).toISOString();
|
||
const sandbox = createMockSandbox({ autoDestroyAt });
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
const lease = await plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1", runId: "run-1",
|
||
config: { ...source, ttlMinutes: 30, reuseLease: false },
|
||
});
|
||
expect(mockCreate).toHaveBeenCalledWith(expect.objectContaining({ ...source, ttlMinutes: 30 }), expect.anything());
|
||
expect(lease?.expiresAt).toBe(autoDestroyAt);
|
||
expect(sandbox.setTtl).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("treats a cleared creation ttl as unset", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const config = { image: "node:20", ttlMinutes: "", reuseLease: false };
|
||
expect(await plugin.definition.onEnvironmentValidateConfig?.({ driverKey: "daytona", config }))
|
||
.toMatchObject({ ok: true, normalizedConfig: { ttlMinutes: null } });
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
const lease = await plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1", runId: "run-1", config,
|
||
});
|
||
expect(mockCreate.mock.calls[0][0]).not.toHaveProperty("ttlMinutes");
|
||
expect(lease?.expiresAt ?? null).toBeNull();
|
||
});
|
||
|
||
it.each(["acquire", "probe", "interactive"])("bounds a stalled creation expiry read for %s", async mode => {
|
||
vi.useFakeTimers();
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ autoDestroyAt: new Date(Date.now() + 60_000).toISOString() });
|
||
let finishRead: (() => void) | undefined;
|
||
sandbox.refreshData.mockImplementation(() => new Promise<void>(resolve => { finishRead = resolve; }));
|
||
mockCreate.mockImplementation(async params => Object.assign(sandbox, { name: params.name, labels: params.labels }));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const params = { driverKey: "daytona", companyId: "company-1", environmentId: "env-1", runId: "run-1", sessionId: "setup-1",
|
||
config: { image: "node:20", ttlMinutes: 1, timeoutMs: 60_000, livenessTimeoutMs: 5, reuseLease: false } };
|
||
const pending = mode === "probe" ? plugin.definition.onEnvironmentProbe!(params)
|
||
: mode === "interactive" ? plugin.definition.onEnvironmentStartInteractiveSetup!(params)
|
||
: plugin.definition.onEnvironmentAcquireLease!(params);
|
||
let settled = false;
|
||
const observed = pending.then(value => ({ value, error: null }), error => ({ value: null, error }));
|
||
void observed.then(() => { settled = true; });
|
||
try {
|
||
await vi.advanceTimersByTimeAsync(20);
|
||
expect(settled).toBe(true);
|
||
const result = await observed;
|
||
expect(result.error?.message ?? (result.value as { metadata?: { error?: string } })?.metadata?.error)
|
||
.toContain("did not respond within 5 ms");
|
||
expect(sandbox.delete).toHaveBeenCalled();
|
||
expect(sandbox.process.executeCommand).not.toHaveBeenCalled();
|
||
} finally {
|
||
finishRead?.();
|
||
await observed;
|
||
vi.useRealTimers();
|
||
}
|
||
});
|
||
|
||
it.each([0, -1, 1.5, "invalid", Infinity])("rejects invalid creation ttl %s before allocation", async ttlMinutes => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const config = { image: "node:20", ttlMinutes, reuseLease: false };
|
||
expect(await plugin.definition.onEnvironmentValidateConfig?.({ driverKey: "daytona", config }))
|
||
.toMatchObject({ ok: false, errors: expect.arrayContaining([expect.stringContaining("ttlMinutes")]) });
|
||
await expect(plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1", runId: "run-1", config,
|
||
})).rejects.toThrow("ttlMinutes");
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it.each([undefined, "not-a-date", new Date(Date.now() - 60_000).toISOString(), new Date(Date.now() + 60 * 60_000).toISOString()])(
|
||
"deletes a new sandbox whose configured creation ttl is not confirmed: %s", async autoDestroyAt => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ autoDestroyAt });
|
||
mockCreate.mockImplementation(async params => Object.assign(sandbox, { name: params.name, labels: params.labels }));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
await expect(plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1", runId: "run-1",
|
||
config: { image: "node:20", ttlMinutes: 30, reuseLease: false },
|
||
})).rejects.toThrow("TTL");
|
||
expect(sandbox.delete).toHaveBeenCalled();
|
||
expect(sandbox.process.executeCommand).not.toHaveBeenCalled();
|
||
},
|
||
);
|
||
|
||
it("does not extend the creation ttl for a later caller deadline", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const autoDestroyAt = new Date(Date.now() + 5 * 60_000).toISOString();
|
||
const sandbox = createMockSandbox({ autoDestroyAt });
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
const lease = await plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1", runId: "run-1",
|
||
config: { image: "node:20", ttlMinutes: 30, reuseLease: false },
|
||
requestedExpiresAt: new Date(Date.now() + 10 * 60_000).toISOString(),
|
||
});
|
||
expect(lease?.expiresAt).toBe(autoDestroyAt);
|
||
expect(sandbox.setTtl).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("shortens a configured creation ttl for an earlier caller deadline", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ autoDestroyAt: new Date(Date.now() + 30 * 60_000).toISOString() });
|
||
sandbox.setTtl.mockImplementation(async minutes => { sandbox.autoDestroyAt = new Date(Date.now() + minutes * 60_000).toISOString(); });
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
const requestedExpiresAt = new Date(Date.now() + 5 * 60_000 + 30_000).toISOString();
|
||
const lease = await plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1", runId: "run-1",
|
||
config: { image: "node:20", ttlMinutes: 30, reuseLease: false }, requestedExpiresAt,
|
||
});
|
||
expect(sandbox.setTtl).toHaveBeenCalledWith(5);
|
||
expect(Date.parse(lease!.expiresAt!)).toBeLessThanOrEqual(Date.parse(requestedExpiresAt));
|
||
});
|
||
|
||
it("configures a provider ttl at or before the requested expiry and returns the provider expiry", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const autoDestroyAt = new Date(Date.now() + 30 * 60_000).toISOString();
|
||
const sandbox = createMockSandbox({ autoDestroyAt });
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
|
||
const requestedExpiresAt = new Date(Date.now() + 30 * 60_000 + 30_000).toISOString();
|
||
const lease = await plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
runId: "run-1",
|
||
config: { image: "node:20", timeoutMs: 300000, reuseLease: false },
|
||
requestedExpiresAt,
|
||
});
|
||
|
||
// The provider ttl is rounded DOWN to whole minutes, so the destroy time
|
||
// never lands after the requested deadline.
|
||
expect(sandbox.setTtl).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.setTtl).toHaveBeenCalledWith(30);
|
||
expect(sandbox.refreshData).toHaveBeenCalled();
|
||
// The lease carries the real provider destroy time as evidence of the bound.
|
||
expect(lease?.expiresAt).toBe(autoDestroyAt);
|
||
});
|
||
|
||
it("returns no expiry when the requested deadline is less than one minute away", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ autoDestroyAt: "must-not-be-read" });
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
|
||
const requestedExpiresAt = new Date(Date.now() + 30_000).toISOString();
|
||
const lease = await plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
runId: "run-1",
|
||
config: { image: "node:20", timeoutMs: 300000, reuseLease: false },
|
||
requestedExpiresAt,
|
||
});
|
||
|
||
// Daytona ttl granularity is one minute, so a nearer deadline maps to no
|
||
// valid provider ttl. The provider grants no expiry and the server fails
|
||
// closed on the null expiry.
|
||
expect(sandbox.setTtl).not.toHaveBeenCalled();
|
||
expect(lease?.expiresAt ?? null).toBeNull();
|
||
});
|
||
|
||
it("starts an interactive setup sandbox with redacted metadata and one-time SSH payload", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
|
||
const session = await plugin.definition.onEnvironmentStartInteractiveSetup?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
sessionId: "setup-1",
|
||
sourceTemplateRef: "existing-secret-snapshot",
|
||
sourceTemplateKind: "snapshot",
|
||
connectionExpiresInMinutes: 30,
|
||
config: {
|
||
image: "node:20",
|
||
timeoutMs: 300000,
|
||
reuseLease: false,
|
||
},
|
||
});
|
||
|
||
const [createParams] = mockCreate.mock.calls[0] as [Record<string, unknown>];
|
||
expect(createParams).toMatchObject({
|
||
snapshot: "existing-secret-snapshot",
|
||
labels: {
|
||
"paperclip-provider": "daytona",
|
||
"paperclip-setup-session-id": "setup-1",
|
||
"paperclip-purpose": "interactive_setup",
|
||
},
|
||
});
|
||
expect(createParams).not.toHaveProperty("image");
|
||
expect(sandbox.createSshAccess).toHaveBeenCalledWith(30);
|
||
expect(session).toMatchObject({
|
||
providerLeaseId: "sandbox-123",
|
||
status: "waiting_for_user",
|
||
connectionSummary: {
|
||
type: "ssh",
|
||
username: "token",
|
||
hostRedacted: true,
|
||
portRedacted: true,
|
||
commandRedacted: true,
|
||
},
|
||
connectionPayload: {
|
||
type: "ssh",
|
||
command: "ssh ssh-token-secret@ssh.app.daytona.io",
|
||
token: "ssh-token-secret",
|
||
},
|
||
metadata: {
|
||
provider: "daytona",
|
||
connectionRedacted: true,
|
||
sourceTemplateRefRedacted: true,
|
||
},
|
||
});
|
||
expect(JSON.stringify(session?.metadata)).not.toContain("ssh-token-secret");
|
||
expect(JSON.stringify(session?.metadata)).not.toContain("existing-secret-snapshot");
|
||
expect(JSON.stringify(session?.connectionSummary)).not.toContain("ssh-token-secret");
|
||
});
|
||
|
||
it("starts interactive setup from an image source when the environment is image-backed", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentStartInteractiveSetup?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
sessionId: "setup-image-1",
|
||
sourceTemplateRef: "node:20",
|
||
sourceTemplateKind: "image",
|
||
connectionExpiresInMinutes: 30,
|
||
config: {
|
||
snapshot: "base-snapshot",
|
||
timeoutMs: 300000,
|
||
reuseLease: false,
|
||
},
|
||
});
|
||
|
||
const [createParams] = mockCreate.mock.calls[0] as [Record<string, unknown>];
|
||
expect(createParams).toMatchObject({
|
||
image: "node:20",
|
||
labels: {
|
||
"paperclip-provider": "daytona",
|
||
"paperclip-setup-session-id": "setup-image-1",
|
||
"paperclip-purpose": "interactive_setup",
|
||
},
|
||
});
|
||
expect(createParams).not.toHaveProperty("snapshot");
|
||
});
|
||
|
||
it("cleans up the setup sandbox if SSH access is unsupported", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
delete (sandbox as { createSshAccess?: unknown }).createSshAccess;
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
|
||
await expect(plugin.definition.onEnvironmentStartInteractiveSetup?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
sessionId: "setup-1",
|
||
config: {
|
||
snapshot: "base-snapshot",
|
||
timeoutMs: 300000,
|
||
reuseLease: false,
|
||
},
|
||
})).rejects.toThrow("Sandbox.createSshAccess");
|
||
|
||
expect(sandbox.delete).toHaveBeenCalledWith(300, true);
|
||
});
|
||
|
||
it("returns setup status without minting SSH access unless requested", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "sandbox-setup" });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const session = await plugin.definition.onEnvironmentGetInteractiveSetup?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-setup",
|
||
includeConnectionPayload: false,
|
||
config: {
|
||
snapshot: "base-snapshot",
|
||
timeoutMs: 300000,
|
||
reuseLease: false,
|
||
},
|
||
});
|
||
|
||
expect(sandbox.createSshAccess).not.toHaveBeenCalled();
|
||
expect(session).toMatchObject({
|
||
providerLeaseId: "sandbox-setup",
|
||
status: "waiting_for_user",
|
||
connectionSummary: {
|
||
type: "ssh",
|
||
hostRedacted: true,
|
||
portRedacted: true,
|
||
},
|
||
connectionPayload: null,
|
||
});
|
||
});
|
||
|
||
it("returns missing setup status when the Daytona sandbox is gone", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
mockGet.mockRejectedValue(new MockDaytonaNotFoundError("missing"));
|
||
|
||
await expect(plugin.definition.onEnvironmentGetInteractiveSetup?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-missing",
|
||
includeConnectionPayload: true,
|
||
config: {
|
||
snapshot: "base-snapshot",
|
||
timeoutMs: 300000,
|
||
reuseLease: false,
|
||
},
|
||
})).resolves.toEqual({
|
||
providerLeaseId: null,
|
||
status: "missing",
|
||
connectionSummary: null,
|
||
connectionPayload: null,
|
||
metadata: {
|
||
provider: "daytona",
|
||
missing: true,
|
||
},
|
||
});
|
||
});
|
||
|
||
it("captures a Daytona snapshot from a live setup sandbox with redacted metadata", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "sandbox-setup" });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentCaptureTemplate?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-setup",
|
||
templateLabel: " Paperclip Env 1 ",
|
||
sourceTemplateRef: "source-secret-snapshot",
|
||
previousTemplateRef: "previous-secret-snapshot",
|
||
timeoutMs: 120000,
|
||
config: {
|
||
snapshot: "base-snapshot",
|
||
timeoutMs: 300000,
|
||
reuseLease: false,
|
||
},
|
||
});
|
||
|
||
expect(sandbox._experimental_createSnapshot).toHaveBeenCalledWith("paperclip-env-1", 120);
|
||
expect(result).toMatchObject({
|
||
templateKind: "snapshot",
|
||
templateRef: "paperclip-env-1",
|
||
metadata: {
|
||
provider: "daytona",
|
||
sandboxId: "sandbox-setup",
|
||
sourceTemplateRefRedacted: true,
|
||
previousTemplateRefRedacted: true,
|
||
},
|
||
});
|
||
expect(JSON.stringify(result?.metadata)).not.toContain("source-secret-snapshot");
|
||
expect(JSON.stringify(result?.metadata)).not.toContain("previous-secret-snapshot");
|
||
});
|
||
|
||
it("cancels an interactive setup sandbox by deleting it", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "sandbox-setup" });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentCancelInteractiveSetup?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-setup",
|
||
reason: "user_cancelled",
|
||
config: {
|
||
snapshot: "base-snapshot",
|
||
timeoutMs: 300000,
|
||
reuseLease: false,
|
||
},
|
||
});
|
||
|
||
expect(sandbox.delete).toHaveBeenCalledWith(300, true);
|
||
expect(result).toMatchObject({
|
||
status: "cancelled",
|
||
metadata: {
|
||
provider: "daytona",
|
||
sandboxId: "sandbox-setup",
|
||
reason: "user_cancelled",
|
||
},
|
||
});
|
||
});
|
||
|
||
it("deletes Daytona snapshot templates through the snapshot service", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const snapshot = { name: "captured-template" };
|
||
mockSnapshotGet.mockResolvedValue(snapshot);
|
||
|
||
const result = await plugin.definition.onEnvironmentDeleteTemplate?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
templateRef: "captured-template",
|
||
templateKind: "snapshot",
|
||
reason: "cleanup",
|
||
config: {
|
||
snapshot: "base-snapshot",
|
||
timeoutMs: 300000,
|
||
reuseLease: false,
|
||
},
|
||
});
|
||
|
||
expect(mockSnapshotGet).toHaveBeenCalledWith("captured-template");
|
||
expect(mockSnapshotDelete).toHaveBeenCalledWith(snapshot);
|
||
expect(result).toEqual({
|
||
deleted: true,
|
||
metadata: {
|
||
provider: "daytona",
|
||
templateKind: "snapshot",
|
||
templateRefRedacted: true,
|
||
reason: "cleanup",
|
||
},
|
||
});
|
||
});
|
||
|
||
it("passes configured resources to Daytona for image-based creation", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
runId: "run-1",
|
||
agentId: "agent-1",
|
||
executionWorkspaceId: "workspace-1",
|
||
adapterType: "codex_local",
|
||
config: {
|
||
image: "node:20",
|
||
cpu: 4,
|
||
memory: 8,
|
||
disk: 20,
|
||
timeoutMs: 300000,
|
||
reuseLease: true,
|
||
},
|
||
});
|
||
|
||
expect(mockCreate).toHaveBeenCalledTimes(1);
|
||
const [createParams] = mockCreate.mock.calls[0] as [Record<string, unknown>];
|
||
expect(createParams).toMatchObject({
|
||
image: "node:20",
|
||
resources: { cpu: 4, memory: 8, disk: 20, gpu: undefined },
|
||
});
|
||
expect(createParams).not.toHaveProperty("snapshot");
|
||
expect(sandbox.resize).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("drops resource settings for snapshot-backed runtime creation instead of failing", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
runId: "run-1",
|
||
agentId: "agent-1",
|
||
executionWorkspaceId: "workspace-1",
|
||
adapterType: "codex_local",
|
||
config: {
|
||
snapshot: "captured-snapshot",
|
||
cpu: 4,
|
||
memory: 8,
|
||
disk: 20,
|
||
timeoutMs: 300000,
|
||
reuseLease: true,
|
||
},
|
||
});
|
||
|
||
expect(mockCreate).toHaveBeenCalledTimes(1);
|
||
const [createParams] = mockCreate.mock.calls[0] as [Record<string, unknown>];
|
||
expect(createParams).toMatchObject({ snapshot: "captured-snapshot" });
|
||
expect(createParams).not.toHaveProperty("resources");
|
||
expect(createParams).not.toHaveProperty("image");
|
||
});
|
||
|
||
it("rejects resource settings for snapshot-backed creation", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const result = await plugin.definition.onEnvironmentValidateConfig?.({
|
||
driverKey: "daytona",
|
||
config: {
|
||
snapshot: "base-snapshot",
|
||
cpu: 4,
|
||
memory: 8,
|
||
disk: 20,
|
||
timeoutMs: 300000,
|
||
reuseLease: true,
|
||
},
|
||
});
|
||
|
||
expect(result).toEqual({
|
||
ok: false,
|
||
errors: [
|
||
"Daytona resource settings require image-backed sandbox creation; snapshot/default sandbox creation cannot override CPU, memory, disk, or GPU.",
|
||
],
|
||
});
|
||
});
|
||
|
||
it("rejects resource settings for default sandbox creation before creating a sandbox", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
|
||
await expect(plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
runId: "run-1",
|
||
agentId: "agent-1",
|
||
executionWorkspaceId: "workspace-1",
|
||
adapterType: "codex_local",
|
||
config: {
|
||
cpu: 4,
|
||
memory: 4,
|
||
timeoutMs: 300000,
|
||
reuseLease: false,
|
||
},
|
||
})).rejects.toThrow(
|
||
"Daytona resource settings require image-backed sandbox creation; default sandbox creation cannot override CPU, memory, disk, or GPU.",
|
||
);
|
||
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("records requested resources in lease metadata", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
|
||
const lease = await plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
runId: "run-1",
|
||
agentId: "agent-1",
|
||
executionWorkspaceId: "workspace-1",
|
||
adapterType: "codex_local",
|
||
config: {
|
||
image: "daytonaio/sandbox:0.8.0",
|
||
cpu: 4,
|
||
memory: 8,
|
||
timeoutMs: 300000,
|
||
reuseLease: true,
|
||
},
|
||
});
|
||
|
||
expect(lease?.metadata).toMatchObject({ cpu: 4, memory: 8 });
|
||
expect(lease?.metadata).not.toHaveProperty("disk");
|
||
expect(lease?.metadata).not.toHaveProperty("gpu");
|
||
});
|
||
|
||
it.each(["matched", "mismatch", "missing", "foreign_run", "foreign_allocation", "legacy"])("verifies an ephemeral run-bound workspace before export recovery: %s", async outcome => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "ephemeral-owned" });
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
const base = { driverKey: "daytona", companyId: "company-1", environmentId: "env-1", agentId: "agent-1",
|
||
issueId: "issue-1", adapterType: "paperclip_runner", config: { reuseLease: false } };
|
||
const lease = await plugin.definition.onEnvironmentAcquireLease!({ ...base, runId: "run-1" });
|
||
const sentinel = lease.metadata!.workspaceSentinel as { token: string; result: string };
|
||
expect(sentinel).toMatchObject({ token: expect.stringMatching(/^[a-f0-9]{64}$/), result: "written" });
|
||
const second = await plugin.definition.onEnvironmentAcquireLease!({ ...base, runId: "run-2" });
|
||
expect((second.metadata!.workspaceSentinel as { token: string }).token).not.toBe(sentinel.token);
|
||
sandbox.process.executeCommand.mockResolvedValueOnce({ exitCode: outcome === "missing" ? 1 : 0,
|
||
result: JSON.stringify({ token: outcome === "matched" ? sentinel.token : "foreign" }), artifacts: { stdout: "" } });
|
||
const metadata = { ...lease.metadata,
|
||
...(outcome === "foreign_run" ? { nativeWorkspaceExportResume: { runId: "foreign-run" } } : {}),
|
||
...(outcome === "legacy" ? { workspaceSentinel: { result: "skipped", token: null } } : {}) };
|
||
sandbox.fs.createFolder.mockClear(); sandbox.fs.uploadFile.mockClear(); sandbox.stop.mockClear();
|
||
const resumed = await plugin.definition.onEnvironmentResumeLease!({ ...base, providerLeaseId: outcome === "foreign_allocation" ? "replacement" : lease.providerLeaseId!, leaseMetadata: metadata });
|
||
expect(resumed).toMatchObject(outcome === "matched" ? { providerLeaseId: sandbox.id, metadata: { workspaceSentinel: { result: "matched" } } }
|
||
: { providerLeaseId: null, metadata: { expired: true, workspaceSentinel: { result: ["foreign_run", "foreign_allocation", "legacy"].includes(outcome) ? "mismatch" : outcome } } });
|
||
expect(sandbox.fs.createFolder).not.toHaveBeenCalled();
|
||
expect(sandbox.fs.uploadFile).not.toHaveBeenCalled();
|
||
expect(sandbox.stop).not.toHaveBeenCalled();
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("changes reusable-lease sentinel identity when resources change", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
|
||
const acquireWithCpu = async (cpu: number): Promise<string> => {
|
||
const sandbox = createMockSandbox();
|
||
mockCreate.mockResolvedValueOnce(sandbox);
|
||
await plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
runId: "run-1",
|
||
agentId: "agent-1",
|
||
executionWorkspaceId: "workspace-1",
|
||
adapterType: "codex_local",
|
||
config: {
|
||
image: "daytonaio/sandbox:0.8.0",
|
||
cpu,
|
||
timeoutMs: 300000,
|
||
reuseLease: true,
|
||
},
|
||
});
|
||
const uploadCall = sandbox.fs.uploadFile.mock.calls.find(
|
||
(call) => typeof call[1] === "string" && call[1].endsWith("reusable-sandbox-lease.json"),
|
||
) as [Buffer, string, number] | undefined;
|
||
expect(uploadCall).toBeTruthy();
|
||
const parsed = JSON.parse((uploadCall as [Buffer, string, number])[0].toString("utf8")) as { token: string };
|
||
return parsed.token;
|
||
};
|
||
|
||
const tokenCpu1 = await acquireWithCpu(1);
|
||
const tokenCpu4 = await acquireWithCpu(4);
|
||
|
||
expect(tokenCpu1).toBeTruthy();
|
||
expect(tokenCpu4).toBeTruthy();
|
||
expect(tokenCpu1).not.toEqual(tokenCpu4);
|
||
});
|
||
|
||
it("deletes the sandbox if lease setup throws after sandbox creation", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
sandbox.getWorkDir.mockRejectedValue(new Error("workdir lookup failed"));
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
|
||
await expect(
|
||
plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
runId: "run-1",
|
||
config: {
|
||
image: "node:20",
|
||
timeoutMs: 300000,
|
||
reuseLease: true,
|
||
},
|
||
}),
|
||
).rejects.toThrow("workdir lookup failed");
|
||
|
||
expect(sandbox.delete).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("falls back to sh metadata when bash is not present in the sandbox image", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
sandbox.process.executeCommand.mockResolvedValue({
|
||
exitCode: 0,
|
||
result: "sh",
|
||
artifacts: { stdout: "sh" },
|
||
});
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
|
||
const lease = await plugin.definition.onEnvironmentAcquireLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
runId: "run-1",
|
||
config: {
|
||
image: "busybox:latest",
|
||
timeoutMs: 300000,
|
||
reuseLease: true,
|
||
},
|
||
});
|
||
|
||
expect(lease).toMatchObject({
|
||
metadata: {
|
||
shellCommand: "sh",
|
||
},
|
||
});
|
||
});
|
||
|
||
it("preserves the sandbox if resume setup throws after the sandbox starts", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "sandbox-resume", state: "stopped" });
|
||
sandbox.getWorkDir.mockRejectedValue(new Error("workdir lookup failed"));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await expect(
|
||
plugin.definition.onEnvironmentResumeLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-resume",
|
||
config: {
|
||
timeoutMs: 300000,
|
||
reuseLease: true,
|
||
},
|
||
}),
|
||
).rejects.toThrow("workdir lookup failed");
|
||
|
||
expect(sandbox.start).toHaveBeenCalled();
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("marks missing reusable leases as expired on resume", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
mockGet.mockRejectedValue(new MockDaytonaNotFoundError("missing"));
|
||
|
||
await expect(plugin.definition.onEnvironmentResumeLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-123",
|
||
config: {
|
||
timeoutMs: 300000,
|
||
reuseLease: true,
|
||
},
|
||
})).resolves.toEqual({
|
||
providerLeaseId: null,
|
||
metadata: { expired: true },
|
||
});
|
||
});
|
||
|
||
it("resumes a reusable lease when the workspace sentinel matches", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "sandbox-reuse", state: "stopped" });
|
||
sandbox.process.executeCommand
|
||
.mockResolvedValueOnce({
|
||
exitCode: 0,
|
||
result: JSON.stringify({ token: "sentinel-token" }),
|
||
artifacts: { stdout: JSON.stringify({ token: "sentinel-token" }) },
|
||
})
|
||
.mockResolvedValueOnce({
|
||
exitCode: 0,
|
||
result: "bash",
|
||
artifacts: { stdout: "bash" },
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const lease = await plugin.definition.onEnvironmentResumeLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-reuse",
|
||
config: {
|
||
timeoutMs: 300000,
|
||
reuseLease: true,
|
||
},
|
||
leaseMetadata: {
|
||
workspaceSentinel: {
|
||
path: "/home/daytona/paperclip-workspace/.paperclip-runtime/reusable-sandbox-lease.json",
|
||
token: "sentinel-token",
|
||
result: "written",
|
||
},
|
||
},
|
||
});
|
||
|
||
expect(sandbox.start).toHaveBeenCalledWith(300);
|
||
expect(lease).toMatchObject({
|
||
providerLeaseId: "sandbox-reuse",
|
||
metadata: {
|
||
resumedLease: true,
|
||
resumedFromState: "stopped",
|
||
sandboxState: "started",
|
||
workspaceSentinel: {
|
||
result: "matched",
|
||
token: "sentinel-token",
|
||
},
|
||
},
|
||
});
|
||
});
|
||
|
||
it.each([true, false])("resumes the recorded workspace cwd after workspace preparation (reuse=%s)", async reuseLease => {
|
||
const sandbox = createMockSandbox({ id: "sandbox-retained", state: "stopped" });
|
||
const allocationRoot = "/home/daytona/paperclip-workspace";
|
||
const workspaceId = "6028a46b-b65a-421f-952d-66b9c002b8e2";
|
||
const remoteCwd = `${allocationRoot}/.paperclip-workspaces/${workspaceId}`;
|
||
const sentinelPath = `${allocationRoot}/.paperclip-runtime/reusable-sandbox-lease.json`;
|
||
sandbox.process.executeCommand.mockImplementation(async command => ({ exitCode: 0,
|
||
result: command.startsWith("cat ") ? JSON.stringify({ token: "sentinel-token" }) : "bash" }));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const resumed = await plugin.definition.onEnvironmentResumeLease!({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1", providerLeaseId: sandbox.id,
|
||
config: { apiKey: "fixture-key", reuseLease }, requestedCwd: remoteCwd, leaseMetadata: { remoteCwd, remoteWorkspaceId: workspaceId,
|
||
sandboxId: sandbox.id, reuseLease, workspaceSentinel: { path: sentinelPath, token: "sentinel-token",
|
||
runId: "fixture-run", providerLeaseId: sandbox.id, result: "written" } },
|
||
});
|
||
expect(resumed).toMatchObject({ providerLeaseId: sandbox.id, metadata: {
|
||
remoteCwd, workspaceSentinel: { path: sentinelPath, result: "matched" } } });
|
||
expect(sandbox.process.executeCommand.mock.calls[0][0]).toBe(`cat '${sentinelPath}'`);
|
||
expect(sandbox.fs.createFolder).not.toHaveBeenCalled();
|
||
expect(sandbox.fs.uploadFile).not.toHaveBeenCalled();
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it.each([null, 7, "", "relative/workspace", "/workspace/../other", "/workspace\0other"])(
|
||
"rejects a malformed recorded cwd before touching the provider: %j", async remoteCwd => {
|
||
await expect(plugin.definition.onEnvironmentResumeLease!({ driverKey: "daytona", companyId: "company-1",
|
||
environmentId: "env-1", providerLeaseId: "sandbox-retained", config: { apiKey: "fixture-key" },
|
||
requestedCwd: remoteCwd as string, leaseMetadata: { remoteCwd },
|
||
})).rejects.toThrow("recorded Daytona workspace directory");
|
||
expect(mockGet).not.toHaveBeenCalled();
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
},
|
||
);
|
||
|
||
it("retains a nested workspace only after the allocation sentinel matches", async () => {
|
||
const sandbox = createMockSandbox({ state: "stopped" });
|
||
sandbox.process.executeCommand.mockResolvedValue({ exitCode: 0, result: JSON.stringify({ token: "foreign" }) });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const resumed = await plugin.definition.onEnvironmentResumeLease!({ driverKey: "daytona", companyId: "company-1",
|
||
environmentId: "env-1", providerLeaseId: sandbox.id, config: { apiKey: "fixture-key" },
|
||
requestedCwd: "/home/daytona/paperclip-workspace/.paperclip-workspaces/fixture-workspace", leaseMetadata: {
|
||
remoteCwd: "/home/daytona/paperclip-workspace/.paperclip-workspaces/fixture-workspace",
|
||
workspaceSentinel: { path: "/home/daytona/paperclip-workspace/.paperclip-runtime/reusable-sandbox-lease.json",
|
||
token: "expected", result: "written" },
|
||
} });
|
||
expect(resumed).toMatchObject({ providerLeaseId: null, metadata: { expired: true, workspaceSentinel: { result: "mismatch" } } });
|
||
expect(sandbox.fs.createFolder).not.toHaveBeenCalled();
|
||
expect(sandbox.fs.uploadFile).not.toHaveBeenCalled();
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it.each(["same-workspace", "later-task-workspace"])("keeps the allocation root for ordinary %s admission", async workspaceId => {
|
||
const sandbox = createMockSandbox({ state: "stopped" });
|
||
const allocationRoot = "/home/daytona/paperclip-workspace";
|
||
sandbox.process.executeCommand.mockImplementation(async command => ({ exitCode: 0,
|
||
result: command.startsWith("cat ") ? JSON.stringify({ token: "sentinel-token" }) : "bash" }));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const resumed = await plugin.definition.onEnvironmentResumeLease!({ driverKey: "daytona", companyId: "company-1",
|
||
environmentId: "env-1", providerLeaseId: sandbox.id, config: { apiKey: "fixture-key" }, leaseMetadata: {
|
||
remoteCwd: `${allocationRoot}/.paperclip-workspaces/${workspaceId}`,
|
||
workspaceSentinel: { path: `${allocationRoot}/.paperclip-runtime/reusable-sandbox-lease.json`, token: "sentinel-token" },
|
||
} });
|
||
expect(resumed).toMatchObject({ providerLeaseId: sandbox.id, metadata: { remoteCwd: allocationRoot } });
|
||
expect(sandbox.fs.createFolder).not.toHaveBeenCalled();
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it.each([undefined, null, "/home/daytona/paperclip-workspace/.paperclip-workspaces/foreign"])(
|
||
"rejects an export cwd without an exact recorded source: %j", async recordedCwd => {
|
||
await expect(plugin.definition.onEnvironmentResumeLease!({ driverKey: "daytona", companyId: "company-1",
|
||
environmentId: "env-1", providerLeaseId: "sandbox-retained", config: { apiKey: "fixture-key" },
|
||
requestedCwd: "/home/daytona/paperclip-workspace/.paperclip-workspaces/owned",
|
||
leaseMetadata: recordedCwd === undefined ? {} : { remoteCwd: recordedCwd },
|
||
})).rejects.toThrow("recorded Daytona workspace directory");
|
||
expect(mockGet).not.toHaveBeenCalled();
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
},
|
||
);
|
||
|
||
it("opens existing files in the recorded directory after a stopped sandbox resumes", async () => {
|
||
const directory = await fs.mkdtemp(path.join(os.tmpdir(), "daytona-resume-cwd-"));
|
||
const allocationRoot = path.join(directory, "paperclip-workspace");
|
||
const remoteCwd = path.join(allocationRoot, ".paperclip-workspaces", "fixture-workspace");
|
||
const sentinelPath = path.join(allocationRoot, ".paperclip-runtime", "reusable-sandbox-lease.json");
|
||
await fs.mkdir(remoteCwd, { recursive: true });
|
||
await fs.mkdir(path.dirname(sentinelPath), { recursive: true });
|
||
await fs.writeFile(sentinelPath, JSON.stringify({ token: "file-token" }));
|
||
await fs.writeFile(path.join(remoteCwd, "saved.txt"), "retained work\n");
|
||
const sandbox = createMockSandbox({ state: "stopped", workDir: directory });
|
||
sandbox.process.executeCommand.mockImplementation(async command => ({ exitCode: 0,
|
||
result: execFileSync("sh", ["-c", command], { encoding: "utf8", cwd: directory }) }));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
try {
|
||
const resumed = await plugin.definition.onEnvironmentResumeLease!({ driverKey: "daytona", companyId: "company-1",
|
||
environmentId: "env-1", providerLeaseId: sandbox.id, config: { apiKey: "fixture-key" }, requestedCwd: remoteCwd,
|
||
leaseMetadata: { remoteCwd, workspaceSentinel: { path: sentinelPath, token: "file-token" } } });
|
||
expect(await fs.readFile(path.join(resumed.metadata!.remoteCwd as string, "saved.txt"), "utf8")).toBe("retained work\n");
|
||
expect(sandbox.fs.createFolder).not.toHaveBeenCalled();
|
||
expect(sandbox.fs.uploadFile).not.toHaveBeenCalled();
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
expect(mockGet).toHaveBeenCalledWith(sandbox.id);
|
||
} finally { await fs.rm(directory, { recursive: true, force: true }); }
|
||
});
|
||
|
||
describe("missing-container resume", () => {
|
||
const sandboxId = "00000000-0000-4000-8000-000000000001";
|
||
const missing = `not found: failed to inspect sandbox container ${sandboxId}: Error response from daemon: No such container: ${sandboxId}`;
|
||
const params = {
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1", providerLeaseId: sandboxId,
|
||
config: { apiKey: "host-key", timeoutMs: 300000, livenessTimeoutMs: 100, reuseLease: true },
|
||
leaseMetadata: { workspaceSentinel: { path: "/home/daytona/paperclip-workspace/.paperclip-runtime/reusable-sandbox-lease.json", token: "sentinel-token" } },
|
||
};
|
||
const resume = () => plugin.definition.onEnvironmentResumeLease!(params);
|
||
function missingSandbox() {
|
||
return { ...createMockSandbox({ id: sandboxId, state: "error", recoverable: false }), errorReason: missing };
|
||
}
|
||
function allowSentinel(sandbox: ReturnType<typeof missingSandbox>) {
|
||
sandbox.process.executeCommand.mockResolvedValueOnce({ exitCode: 0,
|
||
result: JSON.stringify({ token: "sentinel-token" }), artifacts: { stdout: JSON.stringify({ token: "sentinel-token" }) },
|
||
});
|
||
}
|
||
|
||
it("expires a provider record with a freshly confirmed missing container without replacing it", async () => {
|
||
const sandbox = missingSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
for (let attempt = 0; attempt < 2; attempt++) {
|
||
await expect(resume()).resolves.toEqual({ providerLeaseId: null, metadata: { expired: true } });
|
||
}
|
||
expect(mockGet).toHaveBeenCalledTimes(2);
|
||
expect(sandbox.refreshData).toHaveBeenCalledTimes(2);
|
||
expect(sandbox.start).not.toHaveBeenCalled();
|
||
expect(sandbox.recover).not.toHaveBeenCalled();
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it.each(["sandbox-opaque", "sandbox.with+[literal](characters)"])("matches the opaque sandbox ID literally: %s", async (id) => {
|
||
const sandbox = { ...missingSandbox(), id, errorReason: missing.replaceAll(sandboxId, id) };
|
||
mockGet.mockResolvedValue(sandbox);
|
||
await expect(plugin.definition.onEnvironmentResumeLease!({ ...params, providerLeaseId: id }))
|
||
.resolves.toEqual({ providerLeaseId: null, metadata: { expired: true } });
|
||
expect(sandbox.refreshData).toHaveBeenCalledOnce();
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it.each([
|
||
"not found: provider temporarily unavailable",
|
||
missing.replace("No such container:", "No such volume:"),
|
||
missing.replace(/000000000001$/, "000000000002"),
|
||
missing.replaceAll(sandboxId, "00000000-0000-4000-8000-000000000002"),
|
||
])("preserves an unexplained unrecoverable error: %s", async (errorReason) => {
|
||
const sandbox = { ...missingSandbox(), errorReason };
|
||
mockGet.mockResolvedValue(sandbox);
|
||
await expect(resume()).rejects.toThrow("unrecoverable error state");
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("uses provider recovery when the sandbox is recoverable", async () => {
|
||
const sandbox = { ...missingSandbox(), recoverable: true };
|
||
allowSentinel(sandbox);
|
||
mockGet.mockResolvedValue(sandbox);
|
||
await expect(resume()).resolves.toMatchObject({ providerLeaseId: sandboxId });
|
||
expect(sandbox.recover).toHaveBeenCalled();
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("reuses a sandbox whose fresh state disproves the cached loss", async () => {
|
||
const sandbox = missingSandbox();
|
||
sandbox.refreshData.mockImplementation(async () => { sandbox.state = "started"; });
|
||
allowSentinel(sandbox);
|
||
mockGet.mockResolvedValue(sandbox);
|
||
await expect(resume()).resolves.toMatchObject({ providerLeaseId: sandboxId });
|
||
expect(sandbox.start).not.toHaveBeenCalled();
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it.each([new MockDaytonaTimeoutError("timed out"), new Error("provider 503")])("preserves the lease if confirmation fails: %s", async (error) => {
|
||
const sandbox = missingSandbox();
|
||
sandbox.refreshData.mockRejectedValue(error);
|
||
mockGet.mockResolvedValue(sandbox);
|
||
await expect(resume()).rejects.toThrow(error.message);
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("preserves an unknown error returned by the fresh provider read", async () => {
|
||
const sandbox = missingSandbox();
|
||
sandbox.refreshData.mockImplementation(async () => { sandbox.errorReason = "provider unavailable"; });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
await expect(resume()).rejects.toThrow("unrecoverable error state");
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("rejects a refreshed handle belonging to a different sandbox", async () => {
|
||
const sandbox = missingSandbox();
|
||
sandbox.refreshData.mockImplementation(async () => { sandbox.id = "another-sandbox"; });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
await expect(resume()).rejects.toThrow("handle mismatch");
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("accepts a typed not-found result while confirming the missing container", async () => {
|
||
const sandbox = missingSandbox();
|
||
sandbox.refreshData.mockRejectedValue(new MockDaytonaNotFoundError("missing"));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
await expect(resume()).resolves.toEqual({ providerLeaseId: null, metadata: { expired: true } });
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("bounds a stalled confirmation and preserves the lease", async () => {
|
||
vi.useFakeTimers();
|
||
try {
|
||
const sandbox = missingSandbox();
|
||
sandbox.refreshData.mockImplementation(() => new Promise(() => {}));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const result = resume().then(() => null, error => error as Error);
|
||
await vi.advanceTimersByTimeAsync(101);
|
||
expect((await result)?.message).toContain("sandbox.refreshData");
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
expect(mockCreate).not.toHaveBeenCalled();
|
||
} finally { vi.useRealTimers(); }
|
||
});
|
||
});
|
||
|
||
it("expires a reusable lease when the workspace sentinel does not match", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "sandbox-reuse", state: "stopped" });
|
||
sandbox.process.executeCommand.mockResolvedValueOnce({
|
||
exitCode: 0,
|
||
result: JSON.stringify({ token: "other-token" }),
|
||
artifacts: { stdout: JSON.stringify({ token: "other-token" }) },
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await expect(plugin.definition.onEnvironmentResumeLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-reuse",
|
||
config: {
|
||
timeoutMs: 300000,
|
||
reuseLease: true,
|
||
},
|
||
leaseMetadata: {
|
||
workspaceSentinel: {
|
||
path: "/home/daytona/paperclip-workspace/.paperclip-runtime/reusable-sandbox-lease.json",
|
||
token: "sentinel-token",
|
||
result: "written",
|
||
},
|
||
},
|
||
})).resolves.toEqual({
|
||
providerLeaseId: null,
|
||
metadata: {
|
||
expired: true,
|
||
workspaceSentinel: {
|
||
path: "/home/daytona/paperclip-workspace/.paperclip-runtime/reusable-sandbox-lease.json",
|
||
token: "sentinel-token",
|
||
result: "mismatch",
|
||
},
|
||
},
|
||
});
|
||
|
||
expect(sandbox.process.executeCommand).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("refreshes a cached stopped handle before granting a termination receipt", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "sandbox-resumed", state: "stopped" });
|
||
sandbox.refreshData.mockImplementation(async () => { sandbox.state = "started"; });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
await expect(plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1",
|
||
providerLeaseId: sandbox.id, config: { reuseLease: true },
|
||
})).resolves.toEqual({ providerLeaseId: sandbox.id, state: "stopped" });
|
||
expect(sandbox.refreshData).toHaveBeenCalled();
|
||
expect(sandbox.stop).toHaveBeenCalled();
|
||
});
|
||
|
||
it("does not acknowledge termination when both provider stop and delete fail", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "sandbox-failed-stop", state: "started" });
|
||
sandbox.stop.mockRejectedValueOnce(new Error("stop failed"));
|
||
sandbox.delete.mockRejectedValueOnce(new Error("delete failed"));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
await expect(plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1",
|
||
providerLeaseId: sandbox.id, config: { reuseLease: true },
|
||
})).rejects.toThrow("delete failed");
|
||
});
|
||
|
||
it.each([false, true])("explicit stop retains an ephemeral sandbox even when stop fails: %s", async fail => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "sandbox-stop-only", state: "started" });
|
||
if (fail) sandbox.stop.mockRejectedValueOnce(new Error("stop failed"));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const stop = plugin.definition.onEnvironmentStopLease!({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1",
|
||
providerLeaseId: sandbox.id, config: { reuseLease: false },
|
||
});
|
||
if (fail) await expect(stop).rejects.toThrow("stop failed");
|
||
else await expect(stop).resolves.toEqual({ providerLeaseId: sandbox.id, state: "stopped" });
|
||
expect(sandbox.stop).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it.each([true, false])("preserves an archived source only after fresh retention confirmation: %s", async confirmed => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "sandbox-already-archived", state: "archived" });
|
||
sandbox.stop.mockRejectedValue(new Error("The provider rejects Stop for an archived sandbox"));
|
||
sandbox.refreshData.mockImplementation(async () => { sandbox.autoDeleteInterval = confirmed ? -1 : 0; });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const stop = plugin.definition.onEnvironmentStopLease!({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1",
|
||
providerLeaseId: sandbox.id, config: { reuseLease: false },
|
||
});
|
||
if (confirmed) await expect(stop).resolves.toEqual({ providerLeaseId: sandbox.id, state: "stopped" });
|
||
else await expect(stop).rejects.toThrow("retention policy was not confirmed");
|
||
expect(sandbox.setAutoDeleteInterval).toHaveBeenCalledWith(-1);
|
||
expect(sandbox.refreshData).toHaveBeenCalled();
|
||
expect(sandbox.stop).not.toHaveBeenCalled();
|
||
expect(sandbox.start).not.toHaveBeenCalled();
|
||
expect(sandbox.archive).not.toHaveBeenCalled();
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("stops an archived handle that a fresh provider read shows has resumed", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "sandbox-archive-resumed", state: "archived" });
|
||
sandbox.refreshData.mockImplementation(async () => { sandbox.state = "started"; });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
await expect(plugin.definition.onEnvironmentStopLease!({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1",
|
||
providerLeaseId: sandbox.id, config: { reuseLease: false },
|
||
})).resolves.toEqual({ providerLeaseId: sandbox.id, state: "stopped" });
|
||
expect(sandbox.stop).toHaveBeenCalledOnce();
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("stops reusable leases and deletes ephemeral leases on release", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const reusable = createMockSandbox({ id: "sandbox-reusable" });
|
||
const ephemeral = createMockSandbox({ id: "sandbox-ephemeral" });
|
||
mockGet.mockResolvedValueOnce(reusable).mockResolvedValueOnce(ephemeral);
|
||
|
||
const reusableReceipt = await plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-reusable",
|
||
config: {
|
||
timeoutMs: 300000,
|
||
reuseLease: true,
|
||
},
|
||
});
|
||
const ephemeralReceipt = await plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-ephemeral",
|
||
config: {
|
||
timeoutMs: 300000,
|
||
reuseLease: false,
|
||
},
|
||
});
|
||
|
||
expect(reusableReceipt).toEqual({ providerLeaseId: "sandbox-reusable", state: "stopped" });
|
||
expect(ephemeralReceipt).toEqual({ providerLeaseId: "sandbox-ephemeral", state: "destroyed" });
|
||
expect(reusable.stop).toHaveBeenCalledWith(300);
|
||
expect(reusable.delete).not.toHaveBeenCalled();
|
||
expect(ephemeral.delete).toHaveBeenCalledWith(300, true);
|
||
});
|
||
|
||
it.each([false, true])("preserves an ephemeral completed workspace on explicit stop-and-retain (stop fails: %s)", async (stopFails) => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "sandbox-saved-work", state: "started" });
|
||
if (stopFails) sandbox.stop.mockRejectedValueOnce(new Error("injected stop transport failure"));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const stop = plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1",
|
||
providerLeaseId: sandbox.id, config: { timeoutMs: 300000, reuseLease: false },
|
||
resourceDisposition: "stop_and_retain",
|
||
} as Parameters<NonNullable<typeof plugin.definition.onEnvironmentReleaseLease>>[0]);
|
||
if (stopFails) await expect(stop).rejects.toThrow("injected stop transport failure");
|
||
else await expect(stop).resolves.toEqual({ providerLeaseId: sandbox.id, state: "stopped" });
|
||
expect(sandbox.stop).toHaveBeenCalledOnce();
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
expect(sandbox.archive).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it.each(["confirmed", "setter_failed", "refresh_failed", "not_confirmed"])("disables provider auto-delete before preserving stopped work: %s", async kind => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = Object.assign(createMockSandbox({ id: "sandbox-auto-delete", state: "started" }), { autoDeleteInterval: 0 });
|
||
let providerAutoDelete = 0, implicitlyDeleted = false;
|
||
sandbox.setAutoDeleteInterval.mockImplementation(async () => {
|
||
if (kind === "setter_failed") throw new Error("setter unavailable");
|
||
sandbox.autoDeleteInterval = -1; // SDK setter updates its local field even without a fresh provider read.
|
||
if (kind !== "not_confirmed") providerAutoDelete = -1;
|
||
});
|
||
sandbox.refreshData.mockImplementation(async () => {
|
||
if (kind === "refresh_failed") throw new Error("refresh unavailable");
|
||
sandbox.autoDeleteInterval = providerAutoDelete;
|
||
});
|
||
sandbox.stop.mockImplementation(async () => { implicitlyDeleted = providerAutoDelete === 0; });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const stop = plugin.definition.onEnvironmentStopLease!({ driverKey: "daytona", companyId: "company-1", environmentId: "env-1",
|
||
providerLeaseId: sandbox.id, config: { reuseLease: false, autoDeleteInterval: 0 } });
|
||
if (kind === "confirmed") {
|
||
await expect(stop).resolves.toEqual({ providerLeaseId: sandbox.id, state: "stopped" });
|
||
expect(sandbox.setAutoDeleteInterval).toHaveBeenCalledWith(-1);
|
||
expect(sandbox.setAutoDeleteInterval.mock.invocationCallOrder[0]).toBeLessThan(sandbox.refreshData.mock.invocationCallOrder.at(-1)!);
|
||
expect(sandbox.refreshData.mock.invocationCallOrder.at(-1)!).toBeLessThan(sandbox.stop.mock.invocationCallOrder[0]);
|
||
} else {
|
||
await expect(stop).rejects.toThrow();
|
||
expect(sandbox.stop).not.toHaveBeenCalled();
|
||
}
|
||
expect(implicitlyDeleted).toBe(false);
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
expect(sandbox.archive).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("archives instead of deleting when the lease was acquired with archiveOnRelease", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "sandbox-test-probe", state: "started" });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-test-probe",
|
||
config: {
|
||
timeoutMs: 300000,
|
||
reuseLease: false,
|
||
archiveOnRelease: true,
|
||
},
|
||
});
|
||
|
||
expect(sandbox.stop).toHaveBeenCalledWith(300);
|
||
expect(sandbox.setAutoDeleteInterval).toHaveBeenCalledWith(60);
|
||
expect(sandbox.archive).toHaveBeenCalled();
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("falls back to delete when archiving an archiveOnRelease lease fails", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "sandbox-test-probe", state: "stopped" });
|
||
sandbox.archive.mockRejectedValueOnce(new Error("archive unsupported"));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const warnSpy = vi.spyOn(console, "warn").mockImplementation(() => undefined);
|
||
|
||
await plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-test-probe",
|
||
config: {
|
||
timeoutMs: 300000,
|
||
reuseLease: false,
|
||
archiveOnRelease: true,
|
||
},
|
||
});
|
||
|
||
expect(sandbox.stop).not.toHaveBeenCalled();
|
||
expect(sandbox.archive).toHaveBeenCalled();
|
||
expect(sandbox.delete).toHaveBeenCalledWith(300, true);
|
||
expect(warnSpy).toHaveBeenCalled();
|
||
});
|
||
|
||
it("falls back to delete when stopping a reusable lease from an error state fails", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const errored = createMockSandbox({ id: "sandbox-error", state: "error" });
|
||
errored.stop.mockRejectedValueOnce(new Error("stop failed"));
|
||
mockGet.mockResolvedValue(errored);
|
||
|
||
await plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-error",
|
||
config: {
|
||
timeoutMs: 300000,
|
||
reuseLease: true,
|
||
},
|
||
});
|
||
|
||
expect(errored.stop).toHaveBeenCalledWith(300);
|
||
expect(errored.delete).toHaveBeenCalledWith(300, true);
|
||
});
|
||
|
||
it("falls back to delete when stopping a healthy reusable lease fails mid-call", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "sandbox-running", state: "started" });
|
||
sandbox.stop.mockRejectedValueOnce(new Error("api timeout"));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const warnSpy = vi.spyOn(console, "warn").mockImplementation(() => undefined);
|
||
|
||
await plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-running",
|
||
config: {
|
||
timeoutMs: 300000,
|
||
reuseLease: true,
|
||
},
|
||
});
|
||
|
||
expect(sandbox.stop).toHaveBeenCalledWith(300);
|
||
expect(sandbox.delete).toHaveBeenCalledWith(300, true);
|
||
expect(warnSpy).toHaveBeenCalled();
|
||
});
|
||
|
||
describe("session model lifecycle (per-lease session store)", () => {
|
||
// A recording plugin tracer that captures every provider span the session
|
||
// hooks open. It satisfies the structural plugin tracer contract.
|
||
const makeRecordingTracer = () => {
|
||
const spans: Array<{
|
||
name: string;
|
||
attributes: Record<string, unknown>;
|
||
status: { code: number; message?: string } | null;
|
||
ended: boolean;
|
||
}> = [];
|
||
const tracer = {
|
||
startSpan(name: string, options?: { attributes?: Record<string, string | number | boolean> }) {
|
||
const span = {
|
||
name,
|
||
attributes: { ...(options?.attributes ?? {}) } as Record<string, unknown>,
|
||
status: null as { code: number; message?: string } | null,
|
||
ended: false,
|
||
setAttribute(key: string, value: unknown) {
|
||
span.attributes[key] = value;
|
||
},
|
||
setStatus(status: { code: number; message?: string }) {
|
||
span.status = status;
|
||
},
|
||
end() {
|
||
span.ended = true;
|
||
},
|
||
};
|
||
spans.push(span);
|
||
return span;
|
||
},
|
||
};
|
||
return { tracer, spans };
|
||
};
|
||
|
||
const sessionExecParams = (overrides: Record<string, unknown> = {}) => ({
|
||
driverKey: "daytona" as const,
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: { providerLeaseId: "sandbox-123", metadata: {} },
|
||
command: "printf",
|
||
args: ["hello"],
|
||
cwd: "/workspace",
|
||
timeoutMs: 1000,
|
||
...overrides,
|
||
});
|
||
|
||
it("opens one session when two first commands overlap", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
// Hold the first session create open, so the second first command reaches
|
||
// the session store before the first create resolves. Without a single
|
||
// flight guard, both commands would open a session for one lease and the
|
||
// first session id would leak.
|
||
let releaseCreate: () => void = () => {};
|
||
const createGate = new Promise<void>((resolve) => {
|
||
releaseCreate = resolve;
|
||
});
|
||
sandbox.process.createSession.mockImplementationOnce(async () => {
|
||
await createGate;
|
||
});
|
||
|
||
const first = plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
const second = plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
// Flush the pending microtasks, so both commands park on the held create.
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
releaseCreate();
|
||
await Promise.all([first, second]);
|
||
|
||
expect(sandbox.process.createSession).toHaveBeenCalledTimes(1);
|
||
const sessionId = sandbox.process.createSession.mock.calls[0]![0] as string;
|
||
// Teardown deletes the same single session id, so no session leaks.
|
||
await plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-123",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
});
|
||
expect(sandbox.process.deleteSession).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.process.deleteSession).toHaveBeenCalledWith(sessionId);
|
||
});
|
||
|
||
it("runs a bypassSession command one-shot and leaves the session closed", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
// The provision command runs before the run opens its trace root, so the
|
||
// host marks it `bypassSession`. The provider must not open the session for
|
||
// it, or the `session.open` span loses its run parent.
|
||
await plugin.definition.onEnvironmentExecute?.(
|
||
sessionExecParams({ bypassSession: true }),
|
||
);
|
||
|
||
expect(sandbox.process.createSession).not.toHaveBeenCalled();
|
||
expect(sandbox.process.executeCommand).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("opens the session on the first in-run command after a bypassSession command", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
// A bypassSession command runs first (no session), then an in-run command
|
||
// opens the one session. The session-setup span then parents to the run
|
||
// trace, and every later in-run command reuses the same session.
|
||
await plugin.definition.onEnvironmentExecute?.(
|
||
sessionExecParams({ bypassSession: true }),
|
||
);
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
|
||
expect(sandbox.process.createSession).toHaveBeenCalledTimes(1);
|
||
const sessionId = sandbox.process.createSession.mock.calls[0]![0] as string;
|
||
expect(sessionId).toMatch(/^paperclip-/);
|
||
});
|
||
|
||
it("deletes the session and clears the store on release", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
const sessionId = sandbox.process.createSession.mock.calls[0]![0] as string;
|
||
|
||
await plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-123",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
});
|
||
|
||
expect(sandbox.process.deleteSession).toHaveBeenCalledWith(sessionId);
|
||
});
|
||
|
||
it("deletes the session at destroy even when the sandbox delete throws", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
sandbox.delete.mockRejectedValueOnce(new Error("delete failed"));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
const sessionId = sandbox.process.createSession.mock.calls[0]![0] as string;
|
||
|
||
await expect(
|
||
plugin.definition.onEnvironmentDestroyLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-123",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
}),
|
||
).rejects.toThrow(/delete failed/);
|
||
|
||
expect(sandbox.process.deleteSession).toHaveBeenCalledWith(sessionId);
|
||
});
|
||
|
||
it("deletes the session on interactive-setup cancel", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
const sessionId = sandbox.process.createSession.mock.calls[0]![0] as string;
|
||
|
||
await plugin.definition.onEnvironmentCancelInteractiveSetup?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-123",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
});
|
||
|
||
expect(sandbox.process.deleteSession).toHaveBeenCalledWith(sessionId);
|
||
});
|
||
|
||
it("logs loudly and does not throw when the session delete fails", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
sandbox.process.deleteSession.mockRejectedValueOnce(new Error("session gone"));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const errorSpy = vi.spyOn(console, "error").mockImplementation(() => undefined);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
const sessionId = sandbox.process.createSession.mock.calls[0]![0] as string;
|
||
|
||
await plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-123",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
});
|
||
|
||
expect(errorSpy).toHaveBeenCalledWith(expect.stringContaining(sessionId));
|
||
// The rest of teardown still ran: the ephemeral sandbox was deleted.
|
||
expect(sandbox.delete).toHaveBeenCalledWith(300, true);
|
||
});
|
||
|
||
it("returns a destroy receipt only after the provider confirms deletion", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
let complete!: () => void;
|
||
sandbox.delete.mockImplementationOnce(() => new Promise<void>(resolve => { complete = resolve; }));
|
||
const release = plugin.definition.onEnvironmentDestroyLease!({ driverKey: "daytona",
|
||
companyId: "company-1", environmentId: "env-1", providerLeaseId: "sandbox-123",
|
||
config: { timeoutMs: 300000, reuseLease: false } });
|
||
let settled = false;
|
||
void Promise.resolve(release).then(() => { settled = true; });
|
||
await vi.waitFor(() => expect(sandbox.delete).toHaveBeenCalledWith(300, true));
|
||
expect(settled).toBe(false);
|
||
complete();
|
||
await expect(release).resolves.toEqual({ providerLeaseId: "sandbox-123", state: "destroyed" });
|
||
});
|
||
|
||
it("clears the session store after delete so no orphan id survives a second teardown", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const releaseParams = {
|
||
driverKey: "daytona" as const,
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-123",
|
||
config: { timeoutMs: 300000, reuseLease: true },
|
||
};
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
await plugin.definition.onEnvironmentReleaseLease?.(releaseParams);
|
||
// The store is now clear. A second teardown finds no id and does not delete
|
||
// a session again, which proves no orphan id survived the first delete.
|
||
await plugin.definition.onEnvironmentReleaseLease?.(releaseParams);
|
||
|
||
expect(sandbox.process.deleteSession).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("clears the session store when resume restarts a stopped sandbox", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
// A stopped sandbox lost its session shell, so resume must clear the stale
|
||
// id and the next execute must open a fresh session.
|
||
const sandbox = createMockSandbox({ state: "stopped" });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
expect(sandbox.process.createSession).toHaveBeenCalledTimes(1);
|
||
|
||
await plugin.definition.onEnvironmentResumeLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-123",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
leaseMetadata: {
|
||
remoteCwd: "/home/daytona/paperclip-workspace",
|
||
workspaceSentinel: {
|
||
path: "/home/daytona/paperclip-workspace/.paperclip-runtime/reusable-sandbox-lease.json",
|
||
token: "token-1",
|
||
},
|
||
},
|
||
});
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
expect(sandbox.process.createSession).toHaveBeenCalledTimes(2);
|
||
});
|
||
|
||
it("keeps the session when resume runs on a still-running sandbox", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
// A running sandbox keeps its live session shell. Resume must not clear the
|
||
// stored id, or a later command opens a second session and teardown deletes
|
||
// only one, so the first session leaks until sandbox reaping.
|
||
const sandbox = createMockSandbox({ state: "started" });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
expect(sandbox.process.createSession).toHaveBeenCalledTimes(1);
|
||
const sessionId = sandbox.process.createSession.mock.calls[0]![0] as string;
|
||
|
||
await plugin.definition.onEnvironmentResumeLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-123",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
leaseMetadata: {
|
||
remoteCwd: "/home/daytona/paperclip-workspace",
|
||
workspaceSentinel: {
|
||
path: "/home/daytona/paperclip-workspace/.paperclip-runtime/reusable-sandbox-lease.json",
|
||
token: "token-1",
|
||
},
|
||
},
|
||
});
|
||
|
||
// The next command reuses the one live session, so no second session opens.
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
expect(sandbox.process.createSession).toHaveBeenCalledTimes(1);
|
||
|
||
// Teardown deletes the one session id, so no shell leaks.
|
||
await plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-123",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
});
|
||
expect(sandbox.process.deleteSession).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.process.deleteSession).toHaveBeenCalledWith(sessionId);
|
||
});
|
||
|
||
// An idle gap longer than `autoStopInterval` (for example, a long host-side
|
||
// workspace pack) lets Daytona stop the sandbox. Workspace realize and
|
||
// sync-in then restart it before the next command. A restarted sandbox has
|
||
// no process sessions, so the next command must open a fresh session
|
||
// instead of sending the dead id and failing with "session not found".
|
||
const autoStoppedSandbox = () => {
|
||
const sandbox = createMockSandbox({ state: "started" });
|
||
sandbox.start.mockImplementation(async () => {
|
||
sandbox.state = "started";
|
||
});
|
||
return sandbox;
|
||
};
|
||
|
||
it("opens a fresh session after workspace realize restarts an auto-stopped sandbox", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = autoStoppedSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
expect(sandbox.process.createSession).toHaveBeenCalledTimes(1);
|
||
const staleSessionId = sandbox.process.createSession.mock.calls[0]![0] as string;
|
||
|
||
sandbox.state = "stopped";
|
||
await plugin.definition.onEnvironmentRealizeWorkspace?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: { providerLeaseId: "sandbox-123", metadata: { remoteCwd: "/home/daytona/paperclip-workspace" } },
|
||
workspace: { localPath: "/tmp/workspace" },
|
||
} as never);
|
||
expect(sandbox.start).toHaveBeenCalledTimes(1);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
expect(sandbox.process.createSession).toHaveBeenCalledTimes(2);
|
||
const freshSessionId = sandbox.process.createSession.mock.calls[1]![0] as string;
|
||
expect(freshSessionId).not.toBe(staleSessionId);
|
||
const lastExec = sandbox.process.executeSessionCommand.mock.calls.at(-1)!;
|
||
expect(lastExec[0]).toBe(freshSessionId);
|
||
});
|
||
|
||
it("opens a fresh session after sync-in restarts an auto-stopped sandbox", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = autoStoppedSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
expect(sandbox.process.createSession).toHaveBeenCalledTimes(1);
|
||
|
||
sandbox.state = "stopped";
|
||
await plugin.definition.onEnvironmentSyncIn?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: { providerLeaseId: "sandbox-123", metadata: { remoteCwd: "/home/daytona/paperclip-workspace" } },
|
||
operations: [],
|
||
} as never);
|
||
expect(sandbox.start).toHaveBeenCalledTimes(1);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
expect(sandbox.process.createSession).toHaveBeenCalledTimes(2);
|
||
});
|
||
|
||
it("keeps the live session when realize finds the sandbox already running", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = autoStoppedSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
await plugin.definition.onEnvironmentRealizeWorkspace?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: { providerLeaseId: "sandbox-123", metadata: { remoteCwd: "/home/daytona/paperclip-workspace" } },
|
||
workspace: { localPath: "/tmp/workspace" },
|
||
} as never);
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
|
||
expect(sandbox.start).not.toHaveBeenCalled();
|
||
expect(sandbox.process.createSession).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("emits a session.open span on create and a session.close span on delete", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const { tracer, spans } = makeRecordingTracer();
|
||
const restore = __setDaytonaPluginContextForTest({ tracer } as unknown as PluginContext);
|
||
try {
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
const setup = spans.find((span) => span.name === "session.open");
|
||
expect(setup).toBeDefined();
|
||
expect(setup!.ended).toBe(true);
|
||
expect(setup!.attributes["paperclip.sandbox.startup.provider"]).toBe("daytona");
|
||
|
||
await plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-123",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
});
|
||
const teardown = spans.find((span) => span.name === "session.close");
|
||
expect(teardown).toBeDefined();
|
||
expect(teardown!.ended).toBe(true);
|
||
expect(teardown!.attributes["paperclip.sandbox.startup.provider"]).toBe("daytona");
|
||
} finally {
|
||
restore();
|
||
}
|
||
});
|
||
|
||
it("marks the session.open span failed when the session create throws", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
sandbox.process.createSession.mockRejectedValueOnce(new Error("create boom"));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const { tracer, spans } = makeRecordingTracer();
|
||
const restore = __setDaytonaPluginContextForTest({ tracer } as unknown as PluginContext);
|
||
try {
|
||
await expect(
|
||
plugin.definition.onEnvironmentExecute?.(sessionExecParams()),
|
||
).rejects.toThrow(/create boom/);
|
||
const setup = spans.find((span) => span.name === "session.open");
|
||
expect(setup).toBeDefined();
|
||
expect(setup!.ended).toBe(true);
|
||
expect(setup!.status?.code).toBe(2);
|
||
} finally {
|
||
restore();
|
||
}
|
||
});
|
||
|
||
it("dispatches commands into the session and returns separate stdout and stderr", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
sandbox.process.getSessionCommand.mockResolvedValue({ id: "cmd-1", command: "", exitCode: 7 });
|
||
sandbox.process.getSessionCommand.mockResolvedValueOnce({ exitCode: undefined });
|
||
// A session command tries the log stream first: it delivers stdout and
|
||
// stderr from the callback log form.
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (
|
||
_sid: string,
|
||
_cmdId: string,
|
||
onStdout?: (chunk: string) => void,
|
||
onStderr?: (chunk: string) => void,
|
||
) => {
|
||
onStdout?.("out-here");
|
||
onStderr?.("err-here");
|
||
},
|
||
);
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
|
||
// The command runs through the session, not the one-shot path.
|
||
expect(sandbox.process.executeSessionCommand).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.process.executeCommand).not.toHaveBeenCalled();
|
||
const [sid, req, timeoutArg] = sandbox.process.executeSessionCommand.mock.calls[0] as [
|
||
string,
|
||
{ command: string; runAsync?: boolean },
|
||
number,
|
||
];
|
||
expect(sid).toMatch(/^paperclip-/);
|
||
expect(req.runAsync).toBe(true);
|
||
expect(timeoutArg).toBe(1);
|
||
// The built command carries the login-shell script and the user command.
|
||
expect(req.command).toMatch(/&& env .*'printf' 'hello'/);
|
||
// The session command runs plain: no bwrap wrapper and no su privilege drop.
|
||
expect(req.command).not.toContain("sudo -n bwrap");
|
||
expect(req.command).not.toContain("su -s /bin/sh");
|
||
// True separated streams come from the callback log stream.
|
||
expect(result).toMatchObject({ exitCode: 7, timedOut: false, stdout: "out-here", stderr: "err-here" });
|
||
expect(typeof (result!.metadata as Record<string, unknown>)?.durationMs).toBe("number");
|
||
});
|
||
|
||
it("wraps each user command in a subshell so a top-level exit cannot kill the session shell", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(
|
||
sessionExecParams({ command: "exit", args: ["3"] }),
|
||
);
|
||
|
||
const [, req] = sandbox.process.executeSessionCommand.mock.calls[0] as [
|
||
string,
|
||
{ command: string },
|
||
];
|
||
// The whole login-shell script (with the user `exit`) runs inside a
|
||
// subshell, so a top-level exit ends the subshell, not the session shell.
|
||
expect(req.command.trimStart()).toMatch(/^\(/);
|
||
expect(req.command.trimEnd()).toMatch(/\)$/);
|
||
expect(req.command).toMatch(/'exit' '3'/);
|
||
});
|
||
|
||
it("reuses the same session (one persistent shell) across commands", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
await plugin.definition.onEnvironmentExecute?.(sessionExecParams());
|
||
|
||
expect(sandbox.process.createSession).toHaveBeenCalledTimes(1);
|
||
const sessionId = sandbox.process.createSession.mock.calls[0]![0] as string;
|
||
expect(sessionId).toMatch(/^paperclip-/);
|
||
const firstSid = sandbox.process.executeSessionCommand.mock.calls[0]![0] as string;
|
||
const secondSid = sandbox.process.executeSessionCommand.mock.calls[1]![0] as string;
|
||
expect(firstSid).toBe(secondSid);
|
||
});
|
||
|
||
it("returns a session timeout on the poll fallback when the command never reports an exit code", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
// The log stream fails, so the dispatch falls back to the poll path. The
|
||
// command stays running there: the exit code never arrives, so the poll
|
||
// deadline fires.
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (_sid: string, _cmdId: string, onStdout?: (chunk: string) => void) => {
|
||
if (onStdout) {
|
||
throw new Error("socket error");
|
||
}
|
||
return { stdout: "", stderr: "" };
|
||
},
|
||
);
|
||
sandbox.process.getSessionCommand.mockResolvedValue({ id: "cmd-1", command: "", exitCode: undefined });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentExecute?.(
|
||
sessionExecParams({ timeoutMs: 1 }),
|
||
);
|
||
|
||
expect(result).toMatchObject({ exitCode: null, timedOut: true });
|
||
expect(result!.stderr).toMatch(/timed out/);
|
||
});
|
||
|
||
describe("log stream (default)", () => {
|
||
const createStreamingSandbox = () => {
|
||
const sandbox = createMockSandbox();
|
||
sandbox.process.getSessionCommand.mockResolvedValueOnce({ exitCode: undefined });
|
||
return sandbox;
|
||
};
|
||
|
||
// A session command tries the log stream first. It streams stdout and
|
||
// stderr from the callback log form instead of the 50-ms poll, and falls
|
||
// back to the poll only when the stream fails.
|
||
const streamExecParams = (overrides: Record<string, unknown> = {}) =>
|
||
sessionExecParams(overrides);
|
||
|
||
// Long-lived healthy streams keep producing output. Quiet streams now
|
||
// switch to snapshots, which have separate lifetime coverage below.
|
||
async function streamFor(ms: number, onStderr?: (chunk: string) => void) {
|
||
const timer = setInterval(() => onStderr?.("."), 10_000);
|
||
try { await new Promise((resolve) => setTimeout(resolve, ms)); }
|
||
finally { clearInterval(timer); }
|
||
}
|
||
|
||
it("completes a session command beyond a synchronous gateway deadline without redispatch", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createStreamingSandbox();
|
||
sandbox.process.executeCommand.mockRejectedValue(new MockDaytonaTimeoutError("opaque gateway timeout"));
|
||
sandbox.process.getSessionCommand.mockResolvedValue({ exitCode: 0 });
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (_sid: string, _cmdId: string, onStdout?: (chunk: string) => void, onStderr?: (chunk: string) => void) => {
|
||
if (!onStdout) return { stdout: "first;last;", stderr: "" };
|
||
onStdout("first;");
|
||
await streamFor(3_600_001, onStderr);
|
||
onStdout("last;");
|
||
},
|
||
);
|
||
mockGet.mockResolvedValue(sandbox);
|
||
vi.useFakeTimers();
|
||
try {
|
||
const params = streamExecParams({ timeoutMs: 14_400_000, commandTimeoutMs: 14_400_000 });
|
||
// Characterize the one-shot transport failure with the same command
|
||
// and budget, then exercise the existing asynchronous session path.
|
||
expect(await plugin.definition.onEnvironmentExecute?.({ ...params, bypassSession: true }))
|
||
.toMatchObject({ timedOut: true, exitCode: null });
|
||
sandbox.process.executeCommand.mockClear();
|
||
const pending = plugin.definition.onEnvironmentExecute?.(params);
|
||
await vi.advanceTimersByTimeAsync(3_600_002);
|
||
expect(await pending).toMatchObject({ timedOut: false, exitCode: 0, stdout: "first;last;" });
|
||
expect(sandbox.process.executeCommand).not.toHaveBeenCalled();
|
||
expect(sandbox.process.executeSessionCommand).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.process.executeSessionCommand.mock.calls[0]![1]).toMatchObject({ runAsync: true });
|
||
expect(sandbox.process.executeSessionCommand.mock.calls[0]![2]).toBe(14400);
|
||
await plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1",
|
||
providerLeaseId: "sandbox-123", config: params.config,
|
||
});
|
||
expect(sandbox.process.deleteSession).toHaveBeenCalledTimes(1);
|
||
} finally { vi.useRealTimers(); }
|
||
});
|
||
|
||
it.each([undefined, 0])("preserves the existing healthy session lifetime with command timeout %s", async (commandTimeoutMs) => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createStreamingSandbox();
|
||
sandbox.process.getSessionCommand.mockResolvedValue({ exitCode: 0 });
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(async (_s: string, _c: string, stdout?: (chunk: string) => void, stderr?: (chunk: string) => void) => {
|
||
if (!stdout) return { stdout: "done;", stderr: "" };
|
||
await streamFor(60_000, stderr);
|
||
stdout("done;");
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
vi.useFakeTimers();
|
||
try {
|
||
const pending = plugin.definition.onEnvironmentExecute?.(streamExecParams({ timeoutMs: 1000, commandTimeoutMs }));
|
||
await vi.advanceTimersByTimeAsync(60_000);
|
||
expect(await pending).toMatchObject({ timedOut: false, exitCode: 0, stdout: "done;" });
|
||
expect(vi.getTimerCount()).toBe(0);
|
||
} finally { vi.useRealTimers(); }
|
||
});
|
||
|
||
it("removes deadline listeners from completed reads during a quiet polling run", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createStreamingSandbox();
|
||
sandbox.process.getSessionCommand.mockResolvedValue({ exitCode: undefined });
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(async (_s: string, _c: string, stdout?: (chunk: string) => void) => {
|
||
if (stdout) { stdout("partial;"); return; }
|
||
return { stdout: "partial;", stderr: "" };
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const added = vi.spyOn(AbortSignal.prototype, "addEventListener");
|
||
const removed = vi.spyOn(AbortSignal.prototype, "removeEventListener");
|
||
vi.useFakeTimers();
|
||
try {
|
||
const pending = plugin.definition.onEnvironmentExecute?.(streamExecParams({ timeoutMs: 60_000, commandTimeoutMs: 60_000 }));
|
||
await vi.advanceTimersByTimeAsync(30_000);
|
||
const signal = added.mock.contexts.find((value, index) => added.mock.calls[index]?.[0] === "abort");
|
||
expect(signal).toBeDefined();
|
||
const count = (spy: typeof added | typeof removed) => spy.mock.calls.filter((call, index) => call[0] === "abort" && spy.mock.contexts[index] === signal).length;
|
||
expect(count(added)).toBeGreaterThan(50);
|
||
expect(count(added) - count(removed)).toBeLessThanOrEqual(1);
|
||
await vi.advanceTimersByTimeAsync(30_000);
|
||
expect(await pending).toMatchObject({ timedOut: true, exitCode: null, stdout: "partial;" });
|
||
expect(count(removed)).toBe(count(added));
|
||
expect(sandbox.process.executeSessionCommand).toHaveBeenCalledTimes(1);
|
||
expect(vi.getTimerCount()).toBe(0);
|
||
} finally { added.mockRestore(); removed.mockRestore(); vi.useRealTimers(); }
|
||
});
|
||
|
||
it.each(["stream", "status", "snapshot", "dispatch", "final snapshot"])("preserves a command deadline during a healthy or stalled %s observation", async (phase) => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const executionLog = vi.fn();
|
||
const restore = __setDaytonaPluginContextForTest({ execution: { log: executionLog } } as unknown as PluginContext);
|
||
const sandbox = createStreamingSandbox();
|
||
let lateOutput: ((chunk: string) => void) | undefined;
|
||
let rejectStream: ((error: Error) => void) | undefined;
|
||
let keepAlive: ReturnType<typeof setInterval> | undefined;
|
||
sandbox.process.getSessionCommand.mockResolvedValue({ exitCode: phase === "final snapshot" ? 0 : undefined });
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(async (_s: string, _c: string, stdout?: (chunk: string) => void) => {
|
||
if (!stdout) {
|
||
if (phase === "snapshot" || phase === "final snapshot") return new Promise(() => {});
|
||
return { stdout: "first;", stderr: "" };
|
||
}
|
||
stdout("first;");
|
||
if (phase === "stream") {
|
||
lateOutput = stdout;
|
||
keepAlive = setInterval(() => stdout("."), 10000);
|
||
return new Promise((_resolve, reject) => { rejectStream = reject; });
|
||
}
|
||
});
|
||
if (phase === "status") sandbox.process.getSessionCommand.mockImplementation(async () => new Promise(() => {}));
|
||
if (phase === "dispatch") sandbox.process.executeSessionCommand.mockImplementation(async () => new Promise(() => {}));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
vi.useFakeTimers();
|
||
try {
|
||
const pending = plugin.definition.onEnvironmentExecute?.(streamExecParams({ timeoutMs: 14_400_000, commandTimeoutMs: 60000 }));
|
||
let settled = false;
|
||
void pending?.then(() => { settled = true; });
|
||
await vi.advanceTimersByTimeAsync(59999);
|
||
expect(settled).toBe(false);
|
||
await vi.advanceTimersByTimeAsync(1);
|
||
const result = await pending;
|
||
expect(result).toMatchObject({ timedOut: true, exitCode: null });
|
||
if (phase === "final snapshot") {
|
||
expect(result?.metadata).toMatchObject({ timeoutScope: "session_log_observation", commandExitConfirmed: true, observedExitCode: 0 });
|
||
expect(result?.stderr).toContain("Session log observation timed out after 60 s; final output may be incomplete.");
|
||
} else expect(result?.stderr).toContain("Command timed out after 60 s.");
|
||
expect(sandbox.process.executeSessionCommand).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.process.executeCommand).not.toHaveBeenCalled();
|
||
const logged = executionLog.mock.calls.length;
|
||
lateOutput?.("late private output");
|
||
rejectStream?.(new Error("late transport rejection"));
|
||
await vi.advanceTimersByTimeAsync(1);
|
||
expect(executionLog.mock.calls.length).toBe(logged);
|
||
expect(JSON.stringify(result)).not.toContain("late private");
|
||
clearInterval(keepAlive);
|
||
await plugin.definition.onEnvironmentReleaseLease?.({ driverKey: "daytona", companyId: "company-1", environmentId: "env-1", providerLeaseId: "sandbox-123", config: streamExecParams().config });
|
||
expect(sandbox.process.deleteSession).toHaveBeenCalledTimes(1);
|
||
expect(vi.getTimerCount()).toBe(0);
|
||
} finally { clearInterval(keepAlive); restore(); vi.useRealTimers(); }
|
||
});
|
||
|
||
it.each(["", "prefix;🙂;"])("recovers unseen output and cancellation from a stuck socket after prefix %j", async (prefix) => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const executionLog = vi.fn();
|
||
const restore = __setDaytonaPluginContextForTest(
|
||
{ execution: { log: executionLog } } as unknown as PluginContext,
|
||
);
|
||
const sandbox = createStreamingSandbox();
|
||
let lateOutput!: (chunk: string) => void;
|
||
let rejectStream!: (error: Error) => void;
|
||
let snapshots = 0;
|
||
let cancelled = false;
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (_sid: string, _cmdId: string, onStdout?: (chunk: string) => void) => {
|
||
if (onStdout) {
|
||
lateOutput = onStdout;
|
||
onStdout(prefix);
|
||
return new Promise((_resolve, reject) => { rejectStream = reject; });
|
||
}
|
||
snapshots += 1;
|
||
return { stdout: prefix + "permission;" + (cancelled ? "cancelled;" : ""), stderr: "" };
|
||
},
|
||
);
|
||
sandbox.process.getSessionCommand.mockImplementation(async () => ({ exitCode: cancelled ? 0 : undefined }));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
vi.useFakeTimers();
|
||
try {
|
||
const promise = plugin.definition.onEnvironmentExecute?.(streamExecParams({ timeoutMs: 400 }));
|
||
let settled = false;
|
||
void promise?.then(() => { settled = true; });
|
||
await vi.advanceTimersByTimeAsync(14_999);
|
||
expect(snapshots).toBe(0);
|
||
expect(settled).toBe(false);
|
||
await vi.advanceTimersByTimeAsync(1);
|
||
expect(snapshots).toBe(1);
|
||
expect(executionLog).toHaveBeenCalledWith("stdout", "permission;");
|
||
// The abandoned socket cannot corrupt snapshot cursors or publish
|
||
// output after the timeout, including a delayed rejection.
|
||
lateOutput("stale bytes");
|
||
rejectStream(new Error("late socket failure"));
|
||
await vi.advanceTimersByTimeAsync(2_000);
|
||
expect(settled).toBe(false);
|
||
expect(snapshots).toBe(3);
|
||
cancelled = true;
|
||
await vi.advanceTimersByTimeAsync(1_000);
|
||
expect(await promise).toMatchObject({ exitCode: 0, timedOut: false, stdout: prefix + "permission;cancelled;" });
|
||
lateOutput("after settlement");
|
||
expect(executionLog.mock.calls).toEqual([
|
||
...(prefix ? [["stdout", prefix]] : []), ["stdout", "permission;"], ["stdout", "cancelled;"],
|
||
]);
|
||
expect(sandbox.process.executeSessionCommand).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.process.getSessionCommandLogs.mock.calls.filter((call) => call[2])).toHaveLength(1);
|
||
expect(vi.getTimerCount()).toBe(0);
|
||
} finally {
|
||
vi.useRealTimers();
|
||
restore();
|
||
}
|
||
});
|
||
|
||
it.each(["status", "snapshot"])("bounds a stuck %s read after an idle socket without claiming exit", async (stalledRead) => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createStreamingSandbox();
|
||
let releaseRead!: () => void;
|
||
const stalled = new Promise<void>((resolve) => { releaseRead = resolve; });
|
||
const executionLog = vi.fn();
|
||
const restore = __setDaytonaPluginContextForTest(
|
||
{ execution: { log: executionLog } } as unknown as PluginContext,
|
||
);
|
||
sandbox.process.getSessionCommand.mockImplementation(async () => {
|
||
if (stalledRead === "status") await stalled;
|
||
return { exitCode: undefined };
|
||
});
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (_sid: string, _cmdId: string, onStdout?: (chunk: string) => void) => {
|
||
if (onStdout) {
|
||
onStdout("prefix;");
|
||
return new Promise(() => {});
|
||
}
|
||
await stalled;
|
||
return { stdout: "prefix;late;", stderr: "" };
|
||
},
|
||
);
|
||
mockGet.mockResolvedValue(sandbox);
|
||
vi.useFakeTimers();
|
||
try {
|
||
const promise = plugin.definition.onEnvironmentExecute?.(streamExecParams({ timeoutMs: 400 }));
|
||
await vi.advanceTimersByTimeAsync(15_400);
|
||
expect(await promise).toMatchObject({
|
||
exitCode: null, timedOut: true, stdout: "prefix;",
|
||
metadata: { timeoutScope: "session_log_observation", commandExitConfirmed: false },
|
||
});
|
||
releaseRead();
|
||
await vi.advanceTimersByTimeAsync(1_000);
|
||
expect(executionLog.mock.calls).toEqual([["stdout", "prefix;"]]);
|
||
expect(sandbox.process.executeSessionCommand).toHaveBeenCalledTimes(1);
|
||
expect(vi.getTimerCount()).toBe(0);
|
||
} finally {
|
||
releaseRead();
|
||
vi.useRealTimers();
|
||
restore();
|
||
}
|
||
});
|
||
|
||
it("streams ordered stdout and stderr from the callback log form (test_log_stream_delivers_ordered_chunks)", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createStreamingSandbox();
|
||
sandbox.process.getSessionCommand.mockResolvedValue({ id: "cmd-1", command: "", exitCode: 0 });
|
||
// The callback form emits stdout and stderr chunks in order. The plugin
|
||
// keeps each stream in its own arrival order.
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (
|
||
_sid: string,
|
||
_cmdId: string,
|
||
onStdout?: (chunk: string) => void,
|
||
onStderr?: (chunk: string) => void,
|
||
) => {
|
||
onStdout?.("out-1;");
|
||
onStderr?.("err-1;");
|
||
onStdout?.("out-2;");
|
||
onStderr?.("err-2;");
|
||
},
|
||
);
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentExecute?.(streamExecParams());
|
||
|
||
// One callback stream plus a final snapshot to reconcile unseen bytes.
|
||
expect(sandbox.process.getSessionCommandLogs).toHaveBeenCalledTimes(2);
|
||
const streamCall = sandbox.process.getSessionCommandLogs.mock.calls[0]!;
|
||
expect(typeof streamCall[2]).toBe("function");
|
||
expect(typeof streamCall[3]).toBe("function");
|
||
expect(result).toMatchObject({
|
||
exitCode: 0,
|
||
timedOut: false,
|
||
stdout: "out-1;out-2;",
|
||
stderr: "err-1;err-2;",
|
||
});
|
||
expect(typeof (result!.metadata as Record<string, unknown>)?.durationMs).toBe("number");
|
||
});
|
||
|
||
it("reads the exit code once after the stream ends (test_log_stream_reads_exit_code_once_after_stream_end)", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createStreamingSandbox();
|
||
sandbox.process.getSessionCommand.mockResolvedValue({ id: "cmd-1", command: "", exitCode: 5 });
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (_sid: string, _cmdId: string, onStdout?: (chunk: string) => void) => {
|
||
onStdout?.("done");
|
||
},
|
||
);
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentExecute?.(streamExecParams());
|
||
|
||
// The exit code read runs one time after the stream promise resolves.
|
||
expect(sandbox.process.getSessionCommand).toHaveBeenCalledTimes(2);
|
||
expect(result).toMatchObject({ exitCode: 5, timedOut: false, stdout: "done" });
|
||
});
|
||
|
||
it("keeps a running command alive after a clean log-stream close", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const executionLog = vi.fn();
|
||
const restore = __setDaytonaPluginContextForTest(
|
||
{ execution: { log: executionLog } } as unknown as PluginContext,
|
||
);
|
||
try {
|
||
const sandbox = createStreamingSandbox();
|
||
let streams = 0;
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (_sid: string, _cmdId: string, onStdout?: (chunk: string) => void) => {
|
||
if (!onStdout) return { stdout: "first;last;", stderr: "" };
|
||
streams += 1;
|
||
onStdout?.("first;");
|
||
if (streams === 2) onStdout?.("last;");
|
||
// The SDK resolves on WebSocket close, including an idle close.
|
||
},
|
||
);
|
||
sandbox.process.getSessionCommand.mockImplementation(async () => ({
|
||
id: "cmd-1", command: "", exitCode: streams === 1 ? undefined : 7,
|
||
}));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentExecute?.(streamExecParams({ timeoutMs: 2000 }));
|
||
|
||
expect(result).toMatchObject({ exitCode: 7, timedOut: false, stdout: "first;last;" });
|
||
expect(executionLog.mock.calls).toEqual([["stdout", "first;"], ["stdout", "last;"]]);
|
||
expect(sandbox.process.executeSessionCommand).toHaveBeenCalledTimes(1);
|
||
expect(streams).toBe(2);
|
||
} finally {
|
||
restore();
|
||
}
|
||
});
|
||
|
||
it.each(["first;live;done;", "first;"])("polls live output after repeated clean stream closes with terminal snapshot %j", async (finalSnapshot) => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const executionLog = vi.fn();
|
||
const restore = __setDaytonaPluginContextForTest(
|
||
{ execution: { log: executionLog } } as unknown as PluginContext,
|
||
);
|
||
try {
|
||
const sandbox = createStreamingSandbox();
|
||
let snapshots = 0;
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (_sid: string, _cmdId: string, onStdout?: (chunk: string) => void) => {
|
||
if (onStdout) {
|
||
onStdout("first;");
|
||
return;
|
||
}
|
||
snapshots += 1;
|
||
return { stdout: snapshots === 1 ? "first;live;" : finalSnapshot, stderr: "" };
|
||
},
|
||
);
|
||
sandbox.process.getSessionCommand.mockImplementation(async () => {
|
||
if (snapshots > 0) {
|
||
// New output must reach the bridge while the command still runs.
|
||
expect(executionLog).toHaveBeenCalledWith("stdout", "live;");
|
||
}
|
||
return { id: "cmd-1", command: "", exitCode: snapshots > 0 ? 9 : undefined };
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentExecute?.(streamExecParams({ timeoutMs: 2000 }));
|
||
|
||
expect(result).toMatchObject({
|
||
exitCode: 9, timedOut: false, stdout: finalSnapshot.length > 11 ? finalSnapshot : "first;live;",
|
||
});
|
||
expect(executionLog.mock.calls).toEqual([
|
||
["stdout", "first;"], ["stdout", "live;"],
|
||
...(finalSnapshot.endsWith("done;") ? [["stdout", "done;"]] : []),
|
||
]);
|
||
expect(sandbox.process.getSessionCommandLogs.mock.calls.filter((call) => call[2]).length).toBe(2);
|
||
expect(sandbox.process.executeSessionCommand).toHaveBeenCalledTimes(1);
|
||
} finally {
|
||
restore();
|
||
}
|
||
});
|
||
|
||
it.each(["close", "reject"])("preserves healthy stream lifetimes after an aged %s", async (ending) => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createStreamingSandbox();
|
||
let streams = 0;
|
||
let exited = false;
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (_sid: string, _cmdId: string, onStdout?: (chunk: string) => void, onStderr?: (chunk: string) => void) => {
|
||
if (!onStdout) return { stdout: "first;last;", stderr: "" };
|
||
streams += 1;
|
||
onStdout("first;");
|
||
await streamFor(3_600_000, onStderr);
|
||
if (streams === 1 && ending === "reject") throw new Error("socket error");
|
||
if (streams === 2) { onStdout("last;"); exited = true; }
|
||
},
|
||
);
|
||
sandbox.process.getSessionCommand.mockImplementation(async () => ({ exitCode: exited ? 7 : undefined }));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
vi.useFakeTimers();
|
||
try {
|
||
const promise = plugin.definition.onEnvironmentExecute?.(streamExecParams({ timeoutMs: 300_000 }));
|
||
let settled = false;
|
||
void promise?.then(() => { settled = true; });
|
||
await vi.advanceTimersByTimeAsync(3_600_000);
|
||
expect(settled).toBe(false);
|
||
await vi.advanceTimersByTimeAsync(350);
|
||
expect(streams).toBe(2);
|
||
await vi.advanceTimersByTimeAsync(300_001);
|
||
expect(settled).toBe(false);
|
||
await vi.advanceTimersByTimeAsync(3_300_000);
|
||
expect(await promise).toMatchObject({ exitCode: 7, timedOut: false, stdout: "first;last;" });
|
||
expect(sandbox.process.executeSessionCommand).toHaveBeenCalledTimes(1);
|
||
expect(vi.getTimerCount()).toBe(0);
|
||
} finally {
|
||
vi.useRealTimers();
|
||
}
|
||
});
|
||
|
||
it("polls an aged running command after repeated EOF without inventing an exit or lifetime cap", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createStreamingSandbox();
|
||
let streams = 0;
|
||
let snapshots = 0;
|
||
let exited = false;
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (_sid: string, _cmdId: string, onStdout?: (chunk: string) => void, onStderr?: (chunk: string) => void) => {
|
||
if (!onStdout) { snapshots += 1; return { stdout: "partial;", stderr: "" }; }
|
||
onStdout("partial;");
|
||
if (++streams === 1) await streamFor(3_600_000, onStderr);
|
||
},
|
||
);
|
||
sandbox.process.getSessionCommand.mockImplementation(async () => ({ exitCode: exited ? 0 : undefined }));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
vi.useFakeTimers();
|
||
try {
|
||
const promise = plugin.definition.onEnvironmentExecute?.(streamExecParams({ timeoutMs: 400 }));
|
||
let settled = false;
|
||
void promise?.then(() => { settled = true; });
|
||
await vi.advanceTimersByTimeAsync(3_602_700);
|
||
expect(settled).toBe(false);
|
||
expect(snapshots).toBe(3);
|
||
exited = true;
|
||
await vi.advanceTimersByTimeAsync(1000);
|
||
expect(await promise).toMatchObject({ exitCode: 0, timedOut: false, stdout: "partial;" });
|
||
expect(sandbox.process.executeSessionCommand).toHaveBeenCalledTimes(1);
|
||
expect(vi.getTimerCount()).toBe(0);
|
||
} finally {
|
||
vi.useRealTimers();
|
||
}
|
||
});
|
||
|
||
it.each([0, 3_600_000])("retains the fallback budget and partial output after a failed stream lasting %i ms", async (streamLifetime) => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createStreamingSandbox();
|
||
let streams = 0;
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (_sid: string, _cmdId: string, onStdout?: (chunk: string) => void, onStderr?: (chunk: string) => void) => {
|
||
if (onStdout) {
|
||
onStdout("partial;");
|
||
if (++streams === 1 && streamLifetime > 0) await streamFor(streamLifetime, onStderr);
|
||
throw new Error("socket error");
|
||
}
|
||
return { stdout: "partial;", stderr: "" };
|
||
},
|
||
);
|
||
sandbox.process.getSessionCommand.mockResolvedValue({ exitCode: undefined });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
vi.useFakeTimers();
|
||
try {
|
||
const promise = plugin.definition.onEnvironmentExecute?.(streamExecParams({ timeoutMs: 400 }));
|
||
let settled = false;
|
||
void promise?.then(() => { settled = true; });
|
||
await vi.advanceTimersByTimeAsync(streamLifetime + 399);
|
||
expect(settled).toBe(false);
|
||
expect(streams).toBe(2);
|
||
await vi.advanceTimersByTimeAsync(1);
|
||
expect(await promise).toMatchObject({ exitCode: null, timedOut: true, stdout: "partial;" });
|
||
expect(sandbox.process.executeSessionCommand).toHaveBeenCalledTimes(1);
|
||
expect(vi.getTimerCount()).toBe(0);
|
||
} finally {
|
||
vi.useRealTimers();
|
||
}
|
||
});
|
||
|
||
it("preserves provider status errors after a clean stream close", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createStreamingSandbox();
|
||
sandbox.process.getSessionCommandLogs.mockResolvedValue(undefined);
|
||
sandbox.process.getSessionCommand.mockRejectedValue(new Error("session no longer exists"));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await expect(plugin.definition.onEnvironmentExecute?.(streamExecParams()))
|
||
.rejects.toThrow("session no longer exists");
|
||
expect(sandbox.process.executeSessionCommand).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("reads bytes written after socket close before the exit code becomes visible", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createStreamingSandbox();
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (_sid: string, _cmdId: string, onStdout?: (chunk: string) => void) => {
|
||
if (onStdout) onStdout("prefix;");
|
||
else return { stdout: "prefix;tail;", stderr: "final error output" };
|
||
},
|
||
);
|
||
sandbox.process.getSessionCommand
|
||
.mockResolvedValueOnce({ exitCode: undefined })
|
||
.mockResolvedValue({ exitCode: 3 });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentExecute?.(streamExecParams());
|
||
|
||
expect(result).toMatchObject({ exitCode: 3, stdout: "prefix;tail;", stderr: "final error output" });
|
||
expect(sandbox.process.executeSessionCommand).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it.each(["status", "snapshot", "final snapshot"])("bounds a stalled %s observation after clean close and ignores late callbacks", async (stall) => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const executionLog = vi.fn();
|
||
const restore = __setDaytonaPluginContextForTest(
|
||
{ execution: { log: executionLog } } as unknown as PluginContext,
|
||
);
|
||
vi.useFakeTimers();
|
||
try {
|
||
const sandbox = createStreamingSandbox();
|
||
let streams = 0;
|
||
let lateOutput: ((chunk: string) => void) | undefined;
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (_sid: string, _cmdId: string, onStdout?: (chunk: string) => void, onStderr?: (chunk: string) => void) => {
|
||
if (onStdout) {
|
||
streams += 1;
|
||
lateOutput = onStdout;
|
||
onStdout("partial;");
|
||
onStderr?.("partial error;");
|
||
return;
|
||
}
|
||
return new Promise(() => {});
|
||
},
|
||
);
|
||
sandbox.process.getSessionCommand.mockImplementation(async () => {
|
||
if (stall === "status") return new Promise(() => {});
|
||
return { exitCode: stall === "final snapshot" ? 0 : undefined };
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const promise = plugin.definition.onEnvironmentExecute?.(streamExecParams({ timeoutMs: 800 }));
|
||
let settled = false;
|
||
void promise?.then(() => { settled = true; });
|
||
await vi.advanceTimersByTimeAsync(799);
|
||
expect(settled).toBe(false);
|
||
await vi.advanceTimersByTimeAsync(701);
|
||
const result = await promise;
|
||
|
||
expect(result).toMatchObject({
|
||
exitCode: null,
|
||
timedOut: true,
|
||
stdout: "partial;",
|
||
metadata: {
|
||
timeoutScope: "session_log_observation",
|
||
commandExitConfirmed: stall === "final snapshot",
|
||
...(stall === "final snapshot" ? { observedExitCode: 0 } : {}),
|
||
},
|
||
});
|
||
expect(result!.stderr).toMatch(/^partial error;Session log observation timed out/);
|
||
const received = [...executionLog.mock.calls];
|
||
lateOutput?.("late output must be ignored");
|
||
expect(executionLog.mock.calls).toEqual(received);
|
||
expect(sandbox.process.executeSessionCommand).toHaveBeenCalledTimes(1);
|
||
expect(vi.getTimerCount()).toBe(0);
|
||
} finally {
|
||
vi.useRealTimers();
|
||
restore();
|
||
}
|
||
});
|
||
|
||
it.each(["status", "final snapshot", "fallback snapshot"])("preserves observation state when the SDK %s times out", async (stage) => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createStreamingSandbox();
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (_sid: string, _cmdId: string, onStdout?: (chunk: string) => void, onStderr?: (chunk: string) => void) => {
|
||
if (!onStdout) throw new MockDaytonaTimeoutError("provider timeout");
|
||
onStdout("partial;");
|
||
onStderr?.("partial error;");
|
||
if (stage === "fallback snapshot") throw new Error("socket error");
|
||
},
|
||
);
|
||
sandbox.process.getSessionCommand.mockImplementation(async () => {
|
||
if (stage === "status") throw new MockDaytonaTimeoutError("provider timeout");
|
||
return { exitCode: stage === "final snapshot" ? 0 : undefined };
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentExecute?.(streamExecParams());
|
||
|
||
expect(result).toMatchObject({
|
||
exitCode: null, timedOut: true, stdout: "partial;",
|
||
metadata: {
|
||
timeoutScope: "session_log_observation",
|
||
commandExitConfirmed: stage === "final snapshot",
|
||
...(stage === "final snapshot" ? { observedExitCode: 0 } : {}),
|
||
},
|
||
});
|
||
expect(result!.stderr).toMatch(/^partial error;Session log observation timed out/);
|
||
expect(sandbox.process.executeSessionCommand).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it.each(["observation", "legacy fallback", "SDK"])("retains pending replacement characters in a %s timeout result only", async (stage) => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const executionLog = vi.fn();
|
||
const restore = __setDaytonaPluginContextForTest(
|
||
{ execution: { log: executionLog } } as unknown as PluginContext,
|
||
);
|
||
vi.useFakeTimers();
|
||
try {
|
||
const sandbox = createStreamingSandbox();
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (_sid: string, _cmdId: string, onStdout?: (chunk: string) => void, onStderr?: (chunk: string) => void) => {
|
||
onStdout?.("A\uFFFD");
|
||
onStderr?.("E\uFFFD\uFFFD");
|
||
if (stage === "legacy fallback") throw new Error("socket error");
|
||
},
|
||
);
|
||
sandbox.process.getSessionCommand.mockImplementation(async () => {
|
||
if (stage === "SDK") throw new MockDaytonaTimeoutError("provider timeout");
|
||
return new Promise(() => {});
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const promise = plugin.definition.onEnvironmentExecute?.(streamExecParams({ timeoutMs: 400 }));
|
||
await vi.advanceTimersByTimeAsync(400);
|
||
const result = await promise;
|
||
expect(result).toMatchObject({ exitCode: null, timedOut: true, stdout: "A\uFFFD" });
|
||
expect(result!.stderr).toMatch(/^E\uFFFD\uFFFD/);
|
||
expect(executionLog.mock.calls).toEqual([["stdout", "A"], ["stderr", "E"]]);
|
||
expect(sandbox.process.executeSessionCommand).toHaveBeenCalledTimes(1);
|
||
expect(vi.getTimerCount()).toBe(0);
|
||
} finally {
|
||
vi.useRealTimers();
|
||
restore();
|
||
}
|
||
});
|
||
|
||
it.each([
|
||
{ text: "A🙂Z", snapshot: "A🙂Z" },
|
||
{ text: "A\uFFFD", snapshot: "" },
|
||
{ text: "A\uFFFD\uFFFD", snapshot: "A\uFFFD" },
|
||
{ text: "A\uFFFD\uFFFD", snapshot: "" },
|
||
])("preserves UTF-8 when the SDK decoder flushes on clean close: $text", async ({ text, snapshot }) => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const executionLog = vi.fn();
|
||
const restore = __setDaytonaPluginContextForTest(
|
||
{ execution: { log: executionLog } } as unknown as PluginContext,
|
||
);
|
||
try {
|
||
const sandbox = createStreamingSandbox();
|
||
let streams = 0;
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (_sid: string, _cmdId: string, onStdout?: (chunk: string) => void, onStderr?: (chunk: string) => void) => {
|
||
if (!onStdout) return { stdout: snapshot, stderr: "E🙂Z" };
|
||
streams += 1;
|
||
// The pinned SDK streams bytes through TextDecoder and flushes
|
||
// it on clean socket close. Model that native decoder boundary
|
||
// without importing the optional provider package in root CI.
|
||
const emitClosedStream = (bytes: Buffer, emit: (chunk: string) => void) => {
|
||
const decoder = new TextDecoder();
|
||
const text = decoder.decode(bytes, { stream: true });
|
||
if (text) emit(text);
|
||
const tail = decoder.decode();
|
||
if (tail) emit(tail);
|
||
};
|
||
const stdout = Buffer.from(text);
|
||
const stderr = Buffer.from("E🙂Z");
|
||
emitClosedStream(streams === 1 && text.includes("🙂") ? stdout.subarray(0, 2) : stdout, onStdout);
|
||
emitClosedStream(streams === 1 ? stderr.subarray(0, 2) : stderr, onStderr!);
|
||
},
|
||
);
|
||
sandbox.process.getSessionCommand.mockImplementation(async () => ({ exitCode: streams === 1 ? undefined : 0 }));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentExecute?.(streamExecParams({ timeoutMs: 2000 }));
|
||
|
||
expect(result).toMatchObject({ exitCode: 0, stdout: text, stderr: "E🙂Z" });
|
||
for (const [stream, expected] of [["stdout", text], ["stderr", "E🙂Z"]]) {
|
||
expect(executionLog.mock.calls.filter(([name]) => name === stream).map(([, chunk]) => chunk).join("")).toBe(expected);
|
||
}
|
||
expect(sandbox.process.executeSessionCommand).toHaveBeenCalledTimes(1);
|
||
} finally {
|
||
restore();
|
||
}
|
||
});
|
||
|
||
it("falls back to the poll path when the stream promise rejects (test_log_stream_disconnect_rejects_and_falls_back_to_poll)", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createStreamingSandbox();
|
||
// The callback form disconnects and rejects. The snapshot form (no
|
||
// callbacks) serves the poll fallback read.
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (
|
||
_sid: string,
|
||
_cmdId: string,
|
||
onStdout?: (chunk: string) => void,
|
||
) => {
|
||
if (onStdout) {
|
||
onStdout("partial");
|
||
throw new Error("socket error");
|
||
}
|
||
return { stdout: "partialpoll-out", stderr: "poll-err" };
|
||
},
|
||
);
|
||
// The command still runs to its exit on the server, so the poll reads it.
|
||
sandbox.process.getSessionCommand.mockResolvedValue({ id: "cmd-1", command: "", exitCode: 9 });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentExecute?.(streamExecParams());
|
||
|
||
// The poll fallback served the final result, and the command still
|
||
// yielded its exit code.
|
||
expect(result).toMatchObject({ exitCode: 9, timedOut: false, stdout: "partialpoll-out", stderr: "poll-err" });
|
||
// The snapshot form (two args) ran for the fallback read.
|
||
const snapshotCalls = sandbox.process.getSessionCommandLogs.mock.calls.filter(
|
||
(call) => call[2] === undefined,
|
||
);
|
||
expect(snapshotCalls.length).toBeGreaterThanOrEqual(1);
|
||
});
|
||
|
||
it("drops the replayed prefix by byte offset on a reconnect (test_log_stream_reconnect_drops_replayed_prefix_by_byte_offset)", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createStreamingSandbox();
|
||
let attempt = 0;
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (
|
||
_sid: string,
|
||
_cmdId: string,
|
||
onStdout?: (chunk: string) => void,
|
||
onStderr?: (chunk: string) => void,
|
||
) => {
|
||
attempt += 1;
|
||
if (attempt === 1) {
|
||
// First connection: deliver a prefix, then the socket drops.
|
||
onStdout?.("AAA");
|
||
onStderr?.("EEE");
|
||
throw new Error("socket error");
|
||
}
|
||
// Reconnect: Daytona replays the whole log from byte 0, then the new
|
||
// tail. The plugin must drop the replayed prefix.
|
||
onStdout?.("AAA");
|
||
onStdout?.("BBB");
|
||
onStderr?.("EEE");
|
||
onStderr?.("FFF");
|
||
},
|
||
);
|
||
sandbox.process.getSessionCommand.mockResolvedValue({ id: "cmd-1", command: "", exitCode: 0 });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentExecute?.(streamExecParams());
|
||
|
||
// The pre-disconnect bytes appear one time, not two.
|
||
expect(result).toMatchObject({ exitCode: 0, timedOut: false, stdout: "AAABBB", stderr: "EEEFFF" });
|
||
expect(sandbox.process.getSessionCommandLogs.mock.calls.filter((call) => call[2])).toHaveLength(2);
|
||
});
|
||
|
||
it("emits each new chunk to the host and drops a replayed prefix (test_log_stream_emits_execute_log_per_chunk)", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const executionLog = vi.fn();
|
||
const restore = __setDaytonaPluginContextForTest(
|
||
{ execution: { log: executionLog } } as unknown as PluginContext,
|
||
);
|
||
try {
|
||
const sandbox = createStreamingSandbox();
|
||
let attempt = 0;
|
||
sandbox.process.getSessionCommandLogs.mockImplementation(
|
||
async (
|
||
_sid: string,
|
||
_cmdId: string,
|
||
onStdout?: (chunk: string) => void,
|
||
onStderr?: (chunk: string) => void,
|
||
) => {
|
||
attempt += 1;
|
||
if (attempt === 1) {
|
||
onStdout?.("AAA");
|
||
onStderr?.("EEE");
|
||
throw new Error("socket error");
|
||
}
|
||
// Reconnect replays the whole log from byte 0, then the new tail.
|
||
onStdout?.("AAA");
|
||
onStdout?.("BBB");
|
||
onStderr?.("EEE");
|
||
onStderr?.("FFF");
|
||
},
|
||
);
|
||
sandbox.process.getSessionCommand.mockResolvedValue({ id: "cmd-1", command: "", exitCode: 0 });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(streamExecParams());
|
||
|
||
// Each genuinely new chunk reaches the host exactly once. The replayed
|
||
// prefix ("AAA"/"EEE") on the reconnect is not re-emitted.
|
||
expect(executionLog.mock.calls).toEqual([
|
||
["stdout", "AAA"],
|
||
["stderr", "EEE"],
|
||
["stdout", "BBB"],
|
||
["stderr", "FFF"],
|
||
]);
|
||
} finally {
|
||
restore();
|
||
}
|
||
});
|
||
});
|
||
});
|
||
|
||
it("executes commands one-shot and returns combined output via stdout", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
sandbox.process.executeCommand.mockResolvedValue({
|
||
exitCode: 7,
|
||
result: "stdout\nstderr\n",
|
||
artifacts: { stdout: "stdout\nstderr\n" },
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentExecute?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: {
|
||
timeoutMs: 300000,
|
||
reuseLease: false,
|
||
},
|
||
bypassSession: true,
|
||
lease: { providerLeaseId: "sandbox-123", metadata: {} },
|
||
command: "printf",
|
||
args: ["hello"],
|
||
cwd: "/workspace",
|
||
env: { FOO: "bar" },
|
||
timeoutMs: 1000,
|
||
});
|
||
|
||
expect(sandbox.process.executeCommand).toHaveBeenCalledTimes(1);
|
||
const [command, cwdArg, envArg, timeoutArg] = sandbox.process.executeCommand.mock.calls[0] as [string, unknown, unknown, number];
|
||
expect(command).toMatch(/\/etc\/profile/);
|
||
expect(command).toMatch(/"\$HOME\/\.profile"/);
|
||
expect(command).not.toMatch(/nvm\.sh/);
|
||
expect(command).toMatch(/&& cd '\/workspace'/);
|
||
expect(command).toMatch(/&& env GIT_TERMINAL_PROMPT='0' GCM_INTERACTIVE='Never' GIT_ASKPASS='echo' SSH_ASKPASS='echo' SSH_ASKPASS_REQUIRE='force' FOO='bar' 'printf' 'hello'$/);
|
||
expect(command).not.toMatch(/(?:^|&& )exec /);
|
||
// cwd/env are baked into the command itself; we pass undefined to the SDK
|
||
// so its own cwd argument does not run before the caller env is applied.
|
||
expect(cwdArg).toBeUndefined();
|
||
expect(envArg).toBeUndefined();
|
||
expect(timeoutArg).toBe(1);
|
||
expect(result).toMatchObject({
|
||
exitCode: 7,
|
||
timedOut: false,
|
||
stdout: "stdout\nstderr\n",
|
||
stderr: "",
|
||
});
|
||
// Provider-boundary timings ride the free-form result metadata (Open Q1).
|
||
expect(typeof (result!.metadata as Record<string, unknown>)?.durationMs).toBe("number");
|
||
expect(typeof (result!.metadata as Record<string, unknown>)?.getDurationMs).toBe("number");
|
||
});
|
||
|
||
it("reports provider executeCommand and client.get durations via result metadata (injected clock)", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
sandbox.process.executeCommand.mockResolvedValue({
|
||
exitCode: 0,
|
||
result: "ok",
|
||
artifacts: { stdout: "ok" },
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
// Deterministic clock: getSandbox spans 40ms, executeCommand spans 600ms.
|
||
// Call order across the execute path is: getStart, getEnd, execStart, execEnd.
|
||
const ticks = [1000, 1040, 1040, 1640];
|
||
let i = 0;
|
||
const restoreClock = setDaytonaTimingClockForTest(() => ticks[Math.min(i++, ticks.length - 1)]!);
|
||
try {
|
||
const result = await plugin.definition.onEnvironmentExecute?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
bypassSession: true,
|
||
lease: { providerLeaseId: "sandbox-123", metadata: {} },
|
||
command: "printf",
|
||
args: ["hello"],
|
||
cwd: "/workspace",
|
||
timeoutMs: 1000,
|
||
});
|
||
|
||
expect(result!.metadata).toMatchObject({ durationMs: 600, getDurationMs: 40 });
|
||
} finally {
|
||
restoreClock();
|
||
}
|
||
});
|
||
|
||
it("sets metadata.cacheHit false on a client.get miss and true on a warm-handle hit", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
sandbox.process.executeCommand.mockResolvedValue({
|
||
exitCode: 0,
|
||
result: "ok",
|
||
artifacts: { stdout: "ok" },
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const execParams = {
|
||
driverKey: "daytona" as const,
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
bypassSession: true,
|
||
lease: { providerLeaseId: "sandbox-123", metadata: {} },
|
||
command: "printf",
|
||
args: ["hello"],
|
||
cwd: "/workspace",
|
||
timeoutMs: 1000,
|
||
};
|
||
|
||
// First execute: the handle cache is empty, so the lookup calls `client.get`
|
||
// and reports a miss.
|
||
const first = await plugin.definition.onEnvironmentExecute?.(execParams);
|
||
expect(first!.metadata).toMatchObject({ cacheHit: false });
|
||
expect(mockGet).toHaveBeenCalledTimes(1);
|
||
|
||
// Second execute: the warm handle cache serves the handle, so the lookup
|
||
// makes no `client.get` round trip and reports a hit.
|
||
const second = await plugin.definition.onEnvironmentExecute?.(execParams);
|
||
expect(second!.metadata).toMatchObject({ cacheHit: true });
|
||
expect(mockGet).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("stages stdin in the sandbox filesystem when execution needs redirected input", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentExecute?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: {
|
||
timeoutMs: 300000,
|
||
reuseLease: false,
|
||
},
|
||
bypassSession: true,
|
||
lease: { providerLeaseId: "sandbox-123", metadata: {} },
|
||
command: "cat",
|
||
args: [],
|
||
cwd: "/workspace",
|
||
stdin: "input payload",
|
||
timeoutMs: 1000,
|
||
});
|
||
|
||
expect(sandbox.fs.uploadFile).toHaveBeenCalledWith(
|
||
Buffer.from("input payload", "utf8"),
|
||
expect.stringMatching(/^\/tmp\/paperclip-stdin-/),
|
||
1,
|
||
);
|
||
const [command] = sandbox.process.executeCommand.mock.calls[0] as [string];
|
||
expect(command).toMatch(/\/etc\/profile/);
|
||
expect(command).not.toMatch(/nvm\.sh/);
|
||
expect(command).toMatch(/&& cd '\/workspace'/);
|
||
expect(command).toMatch(/env .* 'cat' < '\/tmp\/paperclip-stdin-/);
|
||
expect(command).not.toMatch(/(?:^|&& )exec /);
|
||
expect(sandbox.fs.deleteFile).toHaveBeenCalledWith(expect.stringMatching(/^\/tmp\/paperclip-stdin-/));
|
||
expect(result).toMatchObject({
|
||
exitCode: 0,
|
||
timedOut: false,
|
||
});
|
||
});
|
||
|
||
it("runs the one-shot command plain with no bwrap or su wrapper", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
bypassSession: true,
|
||
lease: { providerLeaseId: "sandbox-123", metadata: { remoteCwd: "/home/daytona/paperclip-workspace" } },
|
||
command: "printf",
|
||
args: ["hello"],
|
||
cwd: "/workspace",
|
||
timeoutMs: 1000,
|
||
});
|
||
|
||
const [command] = sandbox.process.executeCommand.mock.calls[0] as [string];
|
||
expect(command).not.toContain("sudo -n bwrap");
|
||
expect(command).not.toContain("su -s /bin/sh");
|
||
expect(command).toMatch(/'printf' 'hello'$/);
|
||
});
|
||
|
||
it("rejects invalid shell env keys before execution", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await expect(plugin.definition.onEnvironmentExecute?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: {
|
||
timeoutMs: 300000,
|
||
reuseLease: false,
|
||
},
|
||
bypassSession: true,
|
||
lease: { providerLeaseId: "sandbox-123", metadata: {} },
|
||
command: "printf",
|
||
args: ["hello"],
|
||
env: { "BAD-KEY": "bar" },
|
||
})).rejects.toThrow("Invalid sandbox environment variable key: BAD-KEY");
|
||
|
||
expect(sandbox.process.executeCommand).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("returns a timed out execute result when the Daytona SDK times out", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
sandbox.process.executeCommand.mockRejectedValue(new MockDaytonaTimeoutError("command timed out"));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
// Injected clock: getStart=0, getEnd=10 (getDurationMs=10), execStart=20,
|
||
// execEnd/timeout=1520 (durationMs=1500). Deterministic so the timeout path's
|
||
// provider-exec attribution is asserted exactly.
|
||
const ticks = [0, 10, 20, 1520];
|
||
let i = 0;
|
||
const restoreClock = setDaytonaTimingClockForTest(() => ticks[Math.min(i++, ticks.length - 1)]!);
|
||
|
||
let result;
|
||
try {
|
||
result = await plugin.definition.onEnvironmentExecute?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: {
|
||
timeoutMs: 300000,
|
||
reuseLease: false,
|
||
},
|
||
bypassSession: true,
|
||
lease: { providerLeaseId: "sandbox-123", metadata: {} },
|
||
command: "sleep",
|
||
args: ["60"],
|
||
cwd: "/workspace",
|
||
timeoutMs: 1000,
|
||
});
|
||
} finally {
|
||
restoreClock();
|
||
}
|
||
|
||
expect(result).toMatchObject({
|
||
exitCode: null,
|
||
timedOut: true,
|
||
stdout: "",
|
||
stderr: "command timed out\n",
|
||
});
|
||
// The exec reached executeCommand before timing out, so its wall-time is
|
||
// preserved as durationMs (provider-exec attribution is not dropped on the
|
||
// timeout path); the getSandbox re-fetch duration is also reported.
|
||
expect(result!.metadata).toMatchObject({ durationMs: 1500, getDurationMs: 10 });
|
||
});
|
||
|
||
it("injects noninteractive git credential defaults for every one-shot command", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
bypassSession: true,
|
||
lease: { providerLeaseId: "sandbox-123", metadata: {} },
|
||
command: "git",
|
||
args: ["status"],
|
||
timeoutMs: 5000,
|
||
});
|
||
|
||
const [command] = sandbox.process.executeCommand.mock.calls[0] as [string];
|
||
expect(command).toContain("GIT_TERMINAL_PROMPT='0'");
|
||
expect(command).toContain("GCM_INTERACTIVE='Never'");
|
||
expect(command).toContain("GIT_ASKPASS='echo'");
|
||
expect(command).toContain("SSH_ASKPASS='echo'");
|
||
expect(command).toContain("SSH_ASKPASS_REQUIRE='force'");
|
||
});
|
||
|
||
it("caps git network commands at 120 s and returns an actionable message on timeout", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
sandbox.process.executeCommand.mockRejectedValue(new MockDaytonaTimeoutError("timed out"));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentExecute?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
bypassSession: true,
|
||
lease: { providerLeaseId: "sandbox-123", metadata: {} },
|
||
command: "git",
|
||
args: ["push", "origin", "HEAD"],
|
||
cwd: "/workspace",
|
||
timeoutMs: 300000,
|
||
});
|
||
|
||
const [, , , timeoutArg] = sandbox.process.executeCommand.mock.calls[0] as [string, unknown, unknown, number];
|
||
expect(timeoutArg).toBe(120);
|
||
expect(result).toMatchObject({ exitCode: null, timedOut: true });
|
||
expect(result?.stderr).toMatch(/unreachable|credentials/i);
|
||
});
|
||
|
||
// ─── Exec command shape ────────────────────────────────────────────────────
|
||
// The wrapper sources the login profiles so `node` resolves on the reference
|
||
// image, then runs the command. It no longer sources `nvm.sh`, while every
|
||
// other exec surface (env prefix, cwd, quoting, stdin, durationMs) stays
|
||
// intact.
|
||
it("test_exec_command_preserves_env_cwd_and_duration", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
sandbox.process.executeCommand.mockResolvedValue({
|
||
exitCode: 0,
|
||
result: "ok",
|
||
artifacts: { stdout: "ok" },
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentExecute?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
bypassSession: true,
|
||
lease: { providerLeaseId: "sandbox-123", metadata: {} },
|
||
command: "base64",
|
||
args: ["-d"],
|
||
cwd: "/workspace",
|
||
env: { FOO: "bar" },
|
||
timeoutMs: 1000,
|
||
});
|
||
|
||
const [command] = sandbox.process.executeCommand.mock.calls[0] as [string];
|
||
// The command sources the login profiles first, then runs the `cd` and the
|
||
// env prefix with the noninteractive git defaults.
|
||
expect(command).toMatch(/^if \[ -f \/etc\/profile \]/);
|
||
expect(command).toMatch(/&& cd '\/workspace' && env /);
|
||
expect(command).toMatch(/GIT_TERMINAL_PROMPT='0'/);
|
||
expect(command).toMatch(/FOO='bar' 'base64' '-d'$/);
|
||
expect(command).toMatch(/\/etc\/profile/);
|
||
expect(command).not.toMatch(/nvm\.sh/);
|
||
// durationMs attribution stays intact.
|
||
expect(typeof (result!.metadata as Record<string, unknown>)?.durationMs).toBe("number");
|
||
});
|
||
|
||
it("test_exec_command_sources_profile_without_nvm", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
// A node-launching exec resolves `node` through the login profiles, which
|
||
// Daytona's non-login `executeCommand` shell does not source on its own. The
|
||
// wrapper sources the profiles but no longer sources `nvm.sh`.
|
||
await plugin.definition.onEnvironmentExecute?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
bypassSession: true,
|
||
lease: { providerLeaseId: "sandbox-123", metadata: {} },
|
||
command: "node",
|
||
args: ["--version"],
|
||
cwd: "/workspace",
|
||
timeoutMs: 1000,
|
||
});
|
||
|
||
const [command] = sandbox.process.executeCommand.mock.calls[0] as [string];
|
||
expect(command).toMatch(/\/etc\/profile/);
|
||
expect(command).toMatch(/"\$HOME\/\.profile"/);
|
||
expect(command).not.toMatch(/nvm\.sh/);
|
||
expect(command).not.toMatch(/NVM_DIR/);
|
||
});
|
||
|
||
// ─── Per-lease started-sandbox handle cache ────────────────────────────────
|
||
// These prove the security conditions: single-fetch-per-lease, strict
|
||
// composite-key isolation (no cross-lease / cross-company / cross-env reuse),
|
||
// eviction at every teardown, no caching of failed populates, single-flight
|
||
// concurrency, and sentinel re-verification on a cached resume — plus that a
|
||
// handle left idle past the provider auto-stop window is refreshed before
|
||
// reuse so a provider-initiated stop is not hidden behind a stale snapshot.
|
||
describe("started-sandbox handle cache", () => {
|
||
function execParams(
|
||
providerLeaseId: string,
|
||
overrides: { companyId?: string; environmentId?: string; driverKey?: string } = {},
|
||
) {
|
||
return {
|
||
driverKey: overrides.driverKey ?? "daytona",
|
||
companyId: overrides.companyId ?? "company-1",
|
||
environmentId: overrides.environmentId ?? "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
bypassSession: true,
|
||
lease: { providerLeaseId, metadata: {} },
|
||
command: "printf",
|
||
args: ["hi"],
|
||
timeoutMs: 1000,
|
||
};
|
||
}
|
||
|
||
it("reuses the cached handle across execs on one lease (single client.get)", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a" });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
|
||
// Second exec is served from the cache: no second REST re-fetch.
|
||
expect(mockGet).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.process.executeCommand).toHaveBeenCalledTimes(2);
|
||
});
|
||
|
||
it("keeps getDurationMs present (≈0) on a cache hit", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a" });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
const hit = await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
|
||
// The observability contract holds even when the fetch is elided.
|
||
expect(typeof (hit!.metadata as Record<string, unknown>)?.getDurationMs).toBe("number");
|
||
});
|
||
|
||
it("never serves lease A's handle to lease B (distinct fetch per lease)", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandboxA = createMockSandbox({ id: "lease-a" });
|
||
const sandboxB = createMockSandbox({ id: "lease-b" });
|
||
mockGet.mockImplementation(async (id: string) => (id === "lease-a" ? sandboxA : sandboxB));
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-b"));
|
||
|
||
expect(mockGet).toHaveBeenCalledTimes(2);
|
||
expect(sandboxA.process.executeCommand).toHaveBeenCalledTimes(1);
|
||
expect(sandboxB.process.executeCommand).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("does not share a handle across companies or environments for the same providerLeaseId", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
mockGet.mockImplementation(async () => createMockSandbox({ id: "sandbox-x" }));
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("sandbox-x", { companyId: "company-1", environmentId: "env-1" }));
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("sandbox-x", { companyId: "company-2", environmentId: "env-1" }));
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("sandbox-x", { companyId: "company-1", environmentId: "env-2" }));
|
||
|
||
// Three distinct composite keys → three independent fetches; the bare
|
||
// providerLeaseId is never a shared cache slot.
|
||
expect(mockGet).toHaveBeenCalledTimes(3);
|
||
});
|
||
|
||
it("keeps account credentials and API endpoints isolated for the same sandbox ID", async () => {
|
||
mockGet.mockImplementation(async () => createMockSandbox({ id: "sandbox-account" }));
|
||
const params = execParams("sandbox-account");
|
||
for (const config of [
|
||
{ apiKey: "account-a", apiUrl: "https://one.daytona.test/api" },
|
||
{ apiKey: "account-b", apiUrl: "https://one.daytona.test/api" },
|
||
{ apiKey: "account-a", apiUrl: "https://two.daytona.test/api" },
|
||
]) {
|
||
await plugin.definition.onEnvironmentExecute!({
|
||
...params, config: { ...params.config, ...config },
|
||
});
|
||
}
|
||
expect(mockGet).toHaveBeenCalledTimes(3);
|
||
});
|
||
|
||
it("rejects a queued execute after release teardown closes the lease", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
mockGet.mockImplementation(async () => createMockSandbox({ id: "lease-a" }));
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a")); // miss → get #1 (cached)
|
||
const releasePromise = plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "lease-a",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
});
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
|
||
const queuedExecute = plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
await expect(queuedExecute).rejects.toThrow(/no longer active/);
|
||
await releasePromise;
|
||
|
||
// The tombstone closes the lease, so the queued execute never reacquires
|
||
// the sandbox after teardown.
|
||
expect(mockGet).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("rejects an overlapping execute after release teardown closes the lease", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a" });
|
||
let stopStarted = false;
|
||
let resolveRelease!: () => void;
|
||
const releaseGate = new Promise<void>((resolve) => {
|
||
resolveRelease = resolve;
|
||
});
|
||
sandbox.stop.mockImplementation(() => {
|
||
stopStarted = true;
|
||
return releaseGate;
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
const releasePromise = plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "lease-a",
|
||
config: { timeoutMs: 300000, reuseLease: true },
|
||
});
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
expect(stopStarted).toBe(true);
|
||
|
||
const overlappingExec = plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
await expect(overlappingExec).rejects.toThrow(/no longer active/);
|
||
resolveRelease();
|
||
await releasePromise;
|
||
|
||
expect(mockGet).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.stop).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.process.executeCommand).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
|
||
|
||
it("cancels active work before waiting for a stalled execute to drain", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a" });
|
||
let release!: () => void;
|
||
sandbox.process.executeCommand.mockImplementation(async () => {
|
||
await new Promise<void>(resolve => { release = resolve; });
|
||
return { exitCode: 0, result: "", artifacts: { stdout: "" } };
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const execute = plugin.definition.onEnvironmentExecute!(execParams("lease-a"));
|
||
await vi.waitFor(() => expect(release).toBeTypeOf("function"));
|
||
const cancellation = plugin.definition.onEnvironmentReleaseLease!({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1",
|
||
providerLeaseId: "lease-a", config: { timeoutMs: 300000, reuseLease: true },
|
||
cancelActiveWork: true,
|
||
});
|
||
try {
|
||
await vi.waitFor(() => expect(sandbox.stop).toHaveBeenCalledTimes(1), { timeout: 500 });
|
||
await expect(cancellation).resolves.toEqual({ providerLeaseId: "lease-a", state: "stopped" });
|
||
expect(await plugin.definition.onIdleDrain!(new AbortController().signal)).toBe("present");
|
||
await expect(plugin.definition.onEnvironmentExecute!(execParams("lease-a"))).rejects.toThrow(/no longer active/);
|
||
await expect(plugin.definition.onEnvironmentResumeLease!({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1",
|
||
providerLeaseId: "lease-a", config: { timeoutMs: 300000, reuseLease: true },
|
||
})).rejects.toThrow(/still settling cancelled work/);
|
||
expect(sandbox.start).not.toHaveBeenCalled();
|
||
} finally {
|
||
release();
|
||
await execute;
|
||
await cancellation;
|
||
}
|
||
// Completion of the old command is not a second stop receipt. Re-read
|
||
// the exact resource after it settles before clearing worker evidence.
|
||
expect(await plugin.definition.onIdleDrain!(new AbortController().signal)).toBe("present");
|
||
sandbox.state = "stopped";
|
||
await plugin.definition.onEnvironmentInspectLease!({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1",
|
||
providerLeaseId: "lease-a", config: { apiKey: "host-key" },
|
||
});
|
||
expect(await plugin.definition.onIdleDrain!(new AbortController().signal)).toBe("none");
|
||
});
|
||
|
||
it("does not report termination when the provider rejects Stop", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a" });
|
||
sandbox.stop.mockRejectedValue(new Error("provider unavailable"));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
await expect(plugin.definition.onEnvironmentReleaseLease!({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1",
|
||
providerLeaseId: "lease-a", config: { timeoutMs: 300000, reuseLease: true },
|
||
cancelActiveWork: true,
|
||
})).rejects.toThrow("provider unavailable");
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
await expect(plugin.definition.onEnvironmentExecute!(execParams("lease-a"))).rejects.toThrow(/no longer active/);
|
||
});
|
||
|
||
it.each(["release", "destroy"])("%s terminates through the provider when bridge activity never settles", async (kind) => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a" });
|
||
let finish!: () => void;
|
||
sandbox.process.executeCommand.mockImplementation(async () => {
|
||
await new Promise<void>(resolve => { finish = resolve; });
|
||
return { exitCode: 0, result: "bash", artifacts: { stdout: "bash" } };
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const execute = plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
await vi.waitFor(() => expect(finish).toBeTypeOf("function"));
|
||
const params = { driverKey: "daytona", companyId: "company-1", environmentId: "env-1",
|
||
providerLeaseId: "lease-a", config: { timeoutMs: 300000, livenessTimeoutMs: 5, reuseLease: true } };
|
||
try {
|
||
const receipt = kind === "release"
|
||
? await plugin.definition.onEnvironmentReleaseLease?.(params)
|
||
: await plugin.definition.onEnvironmentDestroyLease?.(params);
|
||
expect(receipt).toEqual({ providerLeaseId: "lease-a", state: kind === "release" ? "stopped" : "destroyed" });
|
||
expect(kind === "release" ? sandbox.stop : sandbox.delete).toHaveBeenCalledTimes(1);
|
||
await expect(plugin.definition.onEnvironmentExecute?.(execParams("lease-a"))).rejects.toThrow(/no longer active/);
|
||
} finally { finish(); await execute; }
|
||
});
|
||
|
||
it("waits for an in-flight execute before teardown cleanup starts", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a" });
|
||
let resolveExecute!: () => void;
|
||
sandbox.process.executeCommand.mockImplementation(async () => {
|
||
await new Promise<void>((resolve) => {
|
||
resolveExecute = resolve;
|
||
});
|
||
return {
|
||
exitCode: 0,
|
||
result: "bash",
|
||
artifacts: { stdout: "bash" },
|
||
};
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const executePromise = plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
|
||
const releasePromise = plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "lease-a",
|
||
config: { timeoutMs: 300000, reuseLease: true },
|
||
});
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
expect(sandbox.stop).not.toHaveBeenCalled();
|
||
|
||
resolveExecute();
|
||
await Promise.all([executePromise, releasePromise]);
|
||
|
||
expect(mockGet).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.process.executeCommand).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.stop).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("keeps the teardown gate closed until overlapping teardowns both finish", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a" });
|
||
let stopStarted = false;
|
||
let deleteStarted = false;
|
||
let resolveStop!: () => void;
|
||
let resolveDelete!: () => void;
|
||
const stopGate = new Promise<void>((resolve) => {
|
||
resolveStop = resolve;
|
||
});
|
||
const deleteGate = new Promise<void>((resolve) => {
|
||
resolveDelete = resolve;
|
||
});
|
||
sandbox.stop.mockImplementation(() => {
|
||
stopStarted = true;
|
||
return stopGate;
|
||
});
|
||
sandbox.delete.mockImplementation(() => {
|
||
deleteStarted = true;
|
||
return deleteGate;
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
const releasePromise = plugin.definition.onEnvironmentReleaseLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "lease-a",
|
||
config: { timeoutMs: 300000, reuseLease: true },
|
||
});
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
expect(stopStarted).toBe(true);
|
||
|
||
const destroyPromise = plugin.definition.onEnvironmentDestroyLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "lease-a",
|
||
config: { timeoutMs: 300000, reuseLease: true },
|
||
});
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
expect(deleteStarted).toBe(true);
|
||
|
||
resolveDelete();
|
||
await destroyPromise;
|
||
|
||
const overlappingExec = plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
await expect(overlappingExec).rejects.toThrow(/no longer active/);
|
||
|
||
resolveStop();
|
||
await releasePromise;
|
||
|
||
expect(mockGet).toHaveBeenCalledTimes(2);
|
||
expect(sandbox.stop).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.delete).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.process.executeCommand).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("rejects a queued execute after destroy teardown closes the lease", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
mockGet.mockImplementation(async () => createMockSandbox({ id: "lease-a" }));
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
const destroyPromise = plugin.definition.onEnvironmentDestroyLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "lease-a",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
});
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
|
||
const queuedExecute = plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
await expect(queuedExecute).rejects.toThrow(/no longer active/);
|
||
await destroyPromise;
|
||
|
||
expect(mockGet).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("rejects a queued execute after interactive cancel closes the lease", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
mockGet.mockImplementation(async () => createMockSandbox({ id: "lease-a" }));
|
||
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
const cancelPromise = plugin.definition.onEnvironmentCancelInteractiveSetup?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "lease-a",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
reason: "cancelled",
|
||
});
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
|
||
const queuedExecute = plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
await expect(queuedExecute).rejects.toThrow(/no longer active/);
|
||
await cancelPromise;
|
||
|
||
expect(mockGet).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("waits for an in-flight execute before interactive cancel cleanup starts", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a" });
|
||
let resolveExecute!: () => void;
|
||
sandbox.process.executeCommand.mockImplementation(async () => {
|
||
await new Promise<void>((resolve) => {
|
||
resolveExecute = resolve;
|
||
});
|
||
return {
|
||
exitCode: 0,
|
||
result: "bash",
|
||
artifacts: { stdout: "bash" },
|
||
};
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const executePromise = plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
|
||
const cancelPromise = plugin.definition.onEnvironmentCancelInteractiveSetup?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "lease-a",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
reason: "cancelled",
|
||
});
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
|
||
resolveExecute();
|
||
await Promise.all([executePromise, cancelPromise]);
|
||
|
||
expect(mockGet).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.process.executeCommand).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.delete).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("waits for an in-flight syncIn before interactive cancel cleanup starts", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const hostDir = await fs.mkdtemp(path.join(os.tmpdir(), "daytona-cancel-sync-"));
|
||
const source = path.join(hostDir, "payload.txt");
|
||
await fs.writeFile(source, "payload");
|
||
const remoteDir = "/home/daytona/paperclip-workspace";
|
||
|
||
const sandbox = createMockSandbox({ id: "lease-a" });
|
||
let resolveUpload!: () => void;
|
||
sandbox.fs.uploadFiles.mockImplementation(async () => {
|
||
await new Promise<void>((resolve) => {
|
||
resolveUpload = resolve;
|
||
});
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const syncPromise = plugin.definition.onEnvironmentSyncIn?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: { providerLeaseId: "lease-a", metadata: { remoteCwd: remoteDir } },
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-1",
|
||
files: [{ sourcePath: source, targetPath: `${remoteDir}/payload.txt`, kind: "file" }],
|
||
},
|
||
],
|
||
});
|
||
// Let syncIn register on the activity gate and reach the hung upload.
|
||
// The real `fs.stat` and `mkdir` round trip run before the upload call,
|
||
// so a fixed tick count can race ahead of them on a slower or busier
|
||
// host. Poll for the actual upload call instead of guessing a tick
|
||
// count, so this assertion never fires before syncIn reaches the hang.
|
||
await vi.waitFor(() => {
|
||
expect(sandbox.fs.uploadFiles).toHaveBeenCalled();
|
||
});
|
||
|
||
const cancelPromise = plugin.definition.onEnvironmentCancelInteractiveSetup?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "lease-a",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
reason: "cancelled",
|
||
});
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
// Cancel must drain the active sync before deleting the sandbox out from
|
||
// under it — the same activity-gate contract the execute path relies on.
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
|
||
resolveUpload();
|
||
await Promise.all([syncPromise, cancelPromise]);
|
||
|
||
expect(sandbox.fs.uploadFiles).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.delete).toHaveBeenCalledTimes(1);
|
||
|
||
await fs.rm(hostDir, { recursive: true, force: true });
|
||
});
|
||
|
||
it("rejects a queued execute once interactive cancel tombstones the lease", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a" });
|
||
let resolveFirstExecute!: () => void;
|
||
let cancelResolved = false;
|
||
let queuedExecuteRejected = false;
|
||
sandbox.process.executeCommand.mockImplementation(async () => {
|
||
await new Promise<void>((resolve) => {
|
||
resolveFirstExecute = resolve;
|
||
});
|
||
return {
|
||
exitCode: 0,
|
||
result: "bash",
|
||
artifacts: { stdout: "bash" },
|
||
};
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const firstExecutePromise = plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
|
||
const cancelPromise = plugin.definition.onEnvironmentCancelInteractiveSetup?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "lease-a",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
reason: "cancelled",
|
||
});
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
|
||
const queuedExecutePromise = plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
queuedExecutePromise?.catch(() => {
|
||
queuedExecuteRejected = true;
|
||
});
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
|
||
expect(mockGet).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
|
||
resolveFirstExecute();
|
||
await cancelPromise!.then(() => {
|
||
cancelResolved = true;
|
||
});
|
||
await expect(queuedExecutePromise).rejects.toThrow(/no longer active/);
|
||
await firstExecutePromise;
|
||
|
||
expect(cancelResolved).toBe(true);
|
||
expect(queuedExecuteRejected).toBe(true);
|
||
expect(sandbox.process.executeCommand).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.delete).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("waits for an in-flight snapshot capture before destroy cleanup starts", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a" });
|
||
let resolveSnapshot!: () => void;
|
||
sandbox._experimental_createSnapshot.mockImplementation(async () => {
|
||
await new Promise<void>((resolve) => {
|
||
resolveSnapshot = resolve;
|
||
});
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const capturePromise = plugin.definition.onEnvironmentCaptureTemplate?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "lease-a",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
templateLabel: "snapshot-check",
|
||
});
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
|
||
const destroyPromise = plugin.definition.onEnvironmentDestroyLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "lease-a",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
});
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
|
||
resolveSnapshot();
|
||
await Promise.all([capturePromise, destroyPromise]);
|
||
|
||
expect(sandbox._experimental_createSnapshot).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.delete).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("does not cache a failed populate (NotFound) — the next lookup re-fetches", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a" });
|
||
mockGet
|
||
.mockRejectedValueOnce(new MockDaytonaNotFoundError("missing"))
|
||
.mockResolvedValue(sandbox);
|
||
|
||
const first = await plugin.definition.onEnvironmentResumeLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "lease-a",
|
||
config: { timeoutMs: 300000, reuseLease: true },
|
||
});
|
||
expect(first).toEqual({ providerLeaseId: null, metadata: { expired: true } });
|
||
|
||
// The rejected populate must not linger; the exec re-fetches successfully.
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
expect(mockGet).toHaveBeenCalledTimes(2);
|
||
});
|
||
|
||
it("single-flights concurrent misses on one lease into a single client.get", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a" });
|
||
let resolveGet: ((value: unknown) => void) | undefined;
|
||
mockGet.mockImplementation(
|
||
() => new Promise((resolve) => { resolveGet = resolve; }),
|
||
);
|
||
|
||
const p1 = plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
const p2 = plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
// Let both execs reach the shared in-flight populate before it resolves.
|
||
await Promise.resolve();
|
||
resolveGet?.(sandbox);
|
||
await Promise.all([p1, p2]);
|
||
|
||
expect(mockGet).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.process.executeCommand).toHaveBeenCalledTimes(2);
|
||
});
|
||
|
||
it("keeps concurrent different-lease populates isolated (no promise crossing)", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandboxA = createMockSandbox({ id: "lease-a" });
|
||
const sandboxB = createMockSandbox({ id: "lease-b" });
|
||
mockGet.mockImplementation(async (id: string) => (id === "lease-a" ? sandboxA : sandboxB));
|
||
|
||
await Promise.all([
|
||
plugin.definition.onEnvironmentExecute?.(execParams("lease-a")),
|
||
plugin.definition.onEnvironmentExecute?.(execParams("lease-b")),
|
||
]);
|
||
|
||
expect(mockGet).toHaveBeenCalledTimes(2);
|
||
expect(mockGet).toHaveBeenCalledWith("lease-a");
|
||
expect(mockGet).toHaveBeenCalledWith("lease-b");
|
||
// Each lease executed in its OWN sandbox, never the other's handle.
|
||
expect(sandboxA.process.executeCommand).toHaveBeenCalledTimes(1);
|
||
expect(sandboxB.process.executeCommand).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("re-verifies the workspace sentinel on a cached resume and evicts on mismatch", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a", state: "started" });
|
||
// Every executeCommand (exec body + sentinel `cat`) returns a NON-matching token.
|
||
sandbox.process.executeCommand.mockResolvedValue({
|
||
exitCode: 0,
|
||
result: JSON.stringify({ token: "other-token" }),
|
||
artifacts: { stdout: JSON.stringify({ token: "other-token" }) },
|
||
});
|
||
mockGet.mockImplementation(async () => sandbox);
|
||
|
||
// Prime the cache with a successful exec on this lease.
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
expect(mockGet).toHaveBeenCalledTimes(1);
|
||
const sentinelCallsBefore = sandbox.process.executeCommand.mock.calls.length;
|
||
|
||
// Resume hits the cache but MUST still verify the sentinel; mismatch expires.
|
||
const resumed = await plugin.definition.onEnvironmentResumeLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "lease-a",
|
||
config: { timeoutMs: 300000, reuseLease: true },
|
||
leaseMetadata: {
|
||
workspaceSentinel: {
|
||
path: "/home/daytona/paperclip-workspace/.paperclip-runtime/reusable-sandbox-lease.json",
|
||
token: "expected-token",
|
||
result: "written",
|
||
},
|
||
},
|
||
});
|
||
expect(resumed).toMatchObject({
|
||
providerLeaseId: null,
|
||
metadata: { expired: true, workspaceSentinel: { result: "mismatch" } },
|
||
});
|
||
// The sentinel `cat` ran on the cached handle — verification was not skipped.
|
||
expect(sandbox.process.executeCommand.mock.calls.length).toBeGreaterThan(sentinelCallsBefore);
|
||
|
||
// The mismatched entry was evicted, so the next exec re-fetches.
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
expect(mockGet).toHaveBeenCalledTimes(2);
|
||
});
|
||
|
||
it("refreshes a handle left idle past the auto-stop window and restarts a provider-stopped sandbox", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a", state: "started" });
|
||
// Daytona auto-stopped the sandbox while our cached handle sat idle: a live
|
||
// refresh reveals the true "stopped" state that the cached snapshot hid.
|
||
sandbox.refreshData.mockImplementation(async () => {
|
||
sandbox.state = "stopped";
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
let nowMs = 1_000_000;
|
||
const restoreFreshness = setDaytonaHandleFreshnessClockForTest(() => nowMs);
|
||
try {
|
||
// Prime the cache (single fetch, snapshot "started").
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
expect(sandbox.refreshData).not.toHaveBeenCalled();
|
||
expect(sandbox.start).not.toHaveBeenCalled();
|
||
|
||
// Idle past half of the default 15-min auto-stop interval (> 7.5 min).
|
||
nowMs += 8 * 60_000;
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
} finally {
|
||
restoreFreshness();
|
||
}
|
||
|
||
// The stale handle was refreshed in place (no second REST fetch — the same
|
||
// authenticated handle), the refresh exposed the stopped state, and the
|
||
// sandbox was restarted before the exec instead of running against a
|
||
// stopped sandbox.
|
||
expect(mockGet).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.refreshData).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.start).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.process.executeCommand).toHaveBeenCalledTimes(2);
|
||
});
|
||
|
||
it("does not refresh a handle reused within the auto-stop window", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a" });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
let nowMs = 5_000_000;
|
||
const restoreFreshness = setDaytonaHandleFreshnessClockForTest(() => nowMs);
|
||
try {
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
// Two more execs, each 6 min after the previous — always inside the
|
||
// 7.5-min window measured from the last reuse.
|
||
nowMs += 6 * 60_000;
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
nowMs += 6 * 60_000;
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
} finally {
|
||
restoreFreshness();
|
||
}
|
||
|
||
// Each reuse resets the freshness marker (an operation follows, resetting
|
||
// the provider idle clock), so an actively-used lease never pays a refresh.
|
||
expect(sandbox.refreshData).not.toHaveBeenCalled();
|
||
expect(mockGet).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("does not advance freshness when an execute fails before succeeding", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a" });
|
||
sandbox.process.executeCommand
|
||
.mockRejectedValueOnce(new Error("command failed"))
|
||
.mockResolvedValue({
|
||
exitCode: 0,
|
||
result: "bash",
|
||
artifacts: { stdout: "bash" },
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
let nowMs = 7_000_000;
|
||
const restoreFreshness = setDaytonaHandleFreshnessClockForTest(() => nowMs);
|
||
try {
|
||
await expect(plugin.definition.onEnvironmentExecute?.(execParams("lease-a"))).rejects.toThrow(
|
||
"command failed",
|
||
);
|
||
nowMs += 8 * 60_000;
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
} finally {
|
||
restoreFreshness();
|
||
}
|
||
|
||
expect(sandbox.refreshData).toHaveBeenCalledTimes(1);
|
||
expect(mockGet).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.process.executeCommand).toHaveBeenCalledTimes(2);
|
||
});
|
||
|
||
it("does not advance freshness when an execute times out before succeeding", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a" });
|
||
sandbox.process.executeCommand
|
||
.mockRejectedValueOnce(new MockDaytonaTimeoutError("timed out"))
|
||
.mockResolvedValue({
|
||
exitCode: 0,
|
||
result: "bash",
|
||
artifacts: { stdout: "bash" },
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
let nowMs = 8_000_000;
|
||
const restoreFreshness = setDaytonaHandleFreshnessClockForTest(() => nowMs);
|
||
try {
|
||
const first = await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
expect(first).toMatchObject({ timedOut: true, exitCode: null });
|
||
nowMs += 8 * 60_000;
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
} finally {
|
||
restoreFreshness();
|
||
}
|
||
|
||
expect(sandbox.refreshData).toHaveBeenCalledTimes(1);
|
||
expect(mockGet).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.process.executeCommand).toHaveBeenCalledTimes(2);
|
||
});
|
||
|
||
it("never refreshes when auto-stop is disabled, even after a long idle gap", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a" });
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const disabledAutoStop = { timeoutMs: 300000, reuseLease: false, autoStopInterval: 0 };
|
||
|
||
let nowMs = 2_000_000;
|
||
const restoreFreshness = setDaytonaHandleFreshnessClockForTest(() => nowMs);
|
||
try {
|
||
await plugin.definition.onEnvironmentExecute?.({ ...execParams("lease-a"), config: disabledAutoStop });
|
||
nowMs += 60 * 60_000; // an hour idle
|
||
await plugin.definition.onEnvironmentExecute?.({ ...execParams("lease-a"), config: disabledAutoStop });
|
||
} finally {
|
||
restoreFreshness();
|
||
}
|
||
|
||
// Auto-stop off → the provider never stops the sandbox out from under the
|
||
// handle, so the cached started snapshot is trusted without a refresh.
|
||
expect(sandbox.refreshData).not.toHaveBeenCalled();
|
||
expect(mockGet).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("evicts the handle when a freshness refresh fails so the next lookup re-fetches", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const first = createMockSandbox({ id: "lease-a" });
|
||
first.refreshData.mockRejectedValue(new MockDaytonaNotFoundError("sandbox vanished"));
|
||
const second = createMockSandbox({ id: "lease-a" });
|
||
mockGet.mockResolvedValueOnce(first).mockResolvedValue(second);
|
||
|
||
let nowMs = 3_000_000;
|
||
const restoreFreshness = setDaytonaHandleFreshnessClockForTest(() => nowMs);
|
||
try {
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a")); // fetch #1 → first
|
||
nowMs += 8 * 60_000; // idle past the refresh window
|
||
// The refresh rejects; execute surfaces it (fail closed) and the bad
|
||
// entry is evicted.
|
||
await expect(
|
||
plugin.definition.onEnvironmentExecute?.(execParams("lease-a")),
|
||
).rejects.toThrow("sandbox vanished");
|
||
// Evicted → the following exec re-fetches a fresh handle.
|
||
await plugin.definition.onEnvironmentExecute?.(execParams("lease-a"));
|
||
} finally {
|
||
restoreFreshness();
|
||
}
|
||
|
||
expect(mockGet).toHaveBeenCalledTimes(2);
|
||
expect(second.process.executeCommand).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("surfaces a bounded timeout when the freshness refresh never responds", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "lease-a", state: "started" });
|
||
// The sandbox connection went silent: `refreshData` never resolves and
|
||
// never rejects. Without a per-call bound the exec would stall until the
|
||
// outer RPC ceiling fires.
|
||
sandbox.refreshData.mockImplementation(() => new Promise<void>(() => {}));
|
||
mockGet.mockResolvedValue(sandbox);
|
||
// A short liveness bound keeps the test fast and proves the config path.
|
||
const config = { timeoutMs: 300000, reuseLease: false, livenessTimeoutMs: 50 };
|
||
|
||
let nowMs = 4_000_000;
|
||
const restoreFreshness = setDaytonaHandleFreshnessClockForTest(() => nowMs);
|
||
try {
|
||
// Prime the cache (single fetch, snapshot "started").
|
||
await plugin.definition.onEnvironmentExecute?.({ ...execParams("lease-a"), config });
|
||
// Idle past half of the default 15-min auto-stop interval so the next
|
||
// lookup refreshes the stale handle.
|
||
nowMs += 8 * 60_000;
|
||
await expect(
|
||
plugin.definition.onEnvironmentExecute?.({ ...execParams("lease-a"), config }),
|
||
).rejects.toThrow(/did not respond within 50 ms/);
|
||
} finally {
|
||
restoreFreshness();
|
||
}
|
||
|
||
expect(sandbox.refreshData).toHaveBeenCalledTimes(1);
|
||
// The failed refresh evicted the handle, so the next lookup re-fetches.
|
||
await plugin.definition.onEnvironmentExecute?.({ ...execParams("lease-a"), config });
|
||
expect(mockGet).toHaveBeenCalledTimes(2);
|
||
});
|
||
|
||
it.each(["workspace", "projectless task"])("realizes a resumed %s lease after the provider fills in an unspecified target", async (scope) => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "sandbox-default-target" });
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
mockGet.mockResolvedValue(sandbox);
|
||
const base = { driverKey: "daytona", companyId: "company-1", environmentId: "env-1" };
|
||
const config = { image: "node:20", timeoutMs: 300000, reuseLease: true };
|
||
const lease = await plugin.definition.onEnvironmentAcquireLease!({
|
||
...base, runId: "run-1", agentId: "agent-1", config,
|
||
...(scope === "workspace" ? { executionWorkspaceId: "workspace-1" } : { issueId: "task-1" }),
|
||
});
|
||
// The host materializes provider metadata into later operation config,
|
||
// but resumes with the environment's original, target-less config.
|
||
const realizedConfig = { ...config, ...lease.metadata };
|
||
expect(realizedConfig).toMatchObject({ target: "us" });
|
||
await plugin.definition.onEnvironmentRealizeWorkspace!({
|
||
...base, config: realizedConfig, lease,
|
||
workspace: { remotePath: "/home/daytona/paperclip-workspace" },
|
||
});
|
||
expect(mockGet).not.toHaveBeenCalled();
|
||
await plugin.definition.onEnvironmentReleaseLease!({
|
||
...base, config: realizedConfig, providerLeaseId: lease.providerLeaseId!,
|
||
});
|
||
await expect(plugin.definition.onEnvironmentRealizeWorkspace!({
|
||
...base, config, lease,
|
||
workspace: { remotePath: "/home/daytona/paperclip-workspace" },
|
||
})).rejects.toThrow(/no longer active/);
|
||
await expect(plugin.definition.onEnvironmentRealizeWorkspace!({
|
||
...base, config: realizedConfig, lease,
|
||
workspace: { remotePath: "/home/daytona/paperclip-workspace" },
|
||
})).rejects.toThrow(/no longer active/);
|
||
|
||
sandbox.state = "stopped";
|
||
const sentinel = lease.metadata!.workspaceSentinel as { token: string };
|
||
expect(sentinel.token).toMatch(/^[a-f0-9]{64}$/);
|
||
sandbox.process.executeCommand.mockResolvedValueOnce({
|
||
exitCode: 0, result: JSON.stringify({ token: sentinel.token }),
|
||
artifacts: { stdout: JSON.stringify({ token: sentinel.token }) },
|
||
});
|
||
const resumed = await plugin.definition.onEnvironmentResumeLease!({
|
||
...base, config, providerLeaseId: lease.providerLeaseId!, leaseMetadata: lease.metadata,
|
||
});
|
||
expect(resumed.metadata).toMatchObject({
|
||
resumedLease: true, workspaceSentinel: { result: "matched" },
|
||
});
|
||
await expect(plugin.definition.onEnvironmentRealizeWorkspace!({
|
||
...base, config: { ...config, ...resumed.metadata }, lease: resumed,
|
||
workspace: { remotePath: "/home/daytona/paperclip-workspace" },
|
||
})).resolves.toMatchObject({ cwd: "/home/daytona/paperclip-workspace" });
|
||
});
|
||
|
||
it("realizes the workspace from the acquire-seeded handle without a client.get", async () => {
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
const sandbox = createMockSandbox({ id: "sandbox-seed" });
|
||
mockCreate.mockResolvedValue(sandbox);
|
||
|
||
const base = { driverKey: "daytona", companyId: "company-1", environmentId: "env-1" };
|
||
const config = { image: "node:20", timeoutMs: 300000, reuseLease: false };
|
||
|
||
const lease = await plugin.definition.onEnvironmentAcquireLease?.({
|
||
...base,
|
||
runId: "run-1",
|
||
config,
|
||
});
|
||
expect(lease?.providerLeaseId).toBe("sandbox-seed");
|
||
|
||
const realize = await plugin.definition.onEnvironmentRealizeWorkspace?.({
|
||
...base,
|
||
lease: { providerLeaseId: lease!.providerLeaseId, metadata: lease!.metadata },
|
||
workspace: { remotePath: "/home/daytona/paperclip-workspace" },
|
||
config,
|
||
});
|
||
|
||
// Acquire seeded the handle under the exact scope realize reads, so realize
|
||
// reuses it and never pays a real REST re-fetch.
|
||
expect(mockGet).not.toHaveBeenCalled();
|
||
expect(sandbox.fs.createFolder).toHaveBeenCalledWith("/home/daytona/paperclip-workspace", "755");
|
||
expect(realize?.cwd).toBe("/home/daytona/paperclip-workspace");
|
||
});
|
||
});
|
||
});
|
||
|
||
describe("daytona native file-sync hooks", () => {
|
||
const REMOTE_DIR = "/home/daytona/paperclip-workspace";
|
||
const tempDirs: string[] = [];
|
||
|
||
async function makeHostDir(): Promise<string> {
|
||
const dir = await fs.mkdtemp(path.join(os.tmpdir(), "daytona-sync-test-"));
|
||
tempDirs.push(dir);
|
||
return dir;
|
||
}
|
||
|
||
function syncLease(overrides: Record<string, unknown> = {}) {
|
||
return {
|
||
providerLeaseId: "sandbox-123",
|
||
metadata: { provider: "daytona", remoteCwd: REMOTE_DIR, ...overrides },
|
||
};
|
||
}
|
||
|
||
// A concurrency gate for a fake transfer call (uploadFiles / downloadFiles).
|
||
// The gate lets two concurrent hook calls both enter the fake, then holds them
|
||
// there until the test releases them. It records the peak number of calls that
|
||
// are in the fake at the same time, so a test proves the two calls overlap.
|
||
//
|
||
// `expected` is how many calls the test starts. `bothArrived` resolves once
|
||
// that many calls sit inside the fake at the same moment. `release()` frees
|
||
// them. `body` is the fake implementation: it marks arrival, waits for the
|
||
// release, and then runs `onRelease` to produce the fake result.
|
||
function createTransferGate<T>(expected: number, onRelease: (args: unknown[]) => Promise<T>) {
|
||
let inFlight = 0;
|
||
let peakInFlight = 0;
|
||
let signalArrived!: () => void;
|
||
const bothArrived = new Promise<void>((resolve) => {
|
||
signalArrived = resolve;
|
||
});
|
||
let signalReleased!: () => void;
|
||
const released = new Promise<void>((resolve) => {
|
||
signalReleased = resolve;
|
||
});
|
||
const body = async (...args: unknown[]): Promise<T> => {
|
||
inFlight += 1;
|
||
peakInFlight = Math.max(peakInFlight, inFlight);
|
||
if (inFlight === expected) signalArrived();
|
||
await released;
|
||
inFlight -= 1;
|
||
return onRelease(args);
|
||
};
|
||
return {
|
||
body,
|
||
bothArrived,
|
||
release: () => signalReleased(),
|
||
peak: () => peakInFlight,
|
||
};
|
||
}
|
||
|
||
// Write each download request's snapshot bytes to its host destination and
|
||
// report success, matching the real batch-download contract the outbound sync
|
||
// path expects.
|
||
async function fulfilDownload(args: unknown[]): Promise<Array<{ source: string; result: string }>> {
|
||
const requests = args[0] as Array<{ source: string; destination: string }>;
|
||
return Promise.all(
|
||
requests.map(async (request) => {
|
||
await fs.writeFile(request.destination, "bytes");
|
||
return { source: request.source, result: request.destination };
|
||
}),
|
||
);
|
||
}
|
||
|
||
function syncInParams(overrides: {
|
||
operationId: string;
|
||
sourcePath: string;
|
||
targetPath: string;
|
||
}) {
|
||
return {
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: overrides.operationId,
|
||
files: [{ sourcePath: overrides.sourcePath, targetPath: overrides.targetPath, kind: "file" as const }],
|
||
},
|
||
],
|
||
};
|
||
}
|
||
|
||
function syncOutParams(overrides: {
|
||
operationId: string;
|
||
sourcePath: string;
|
||
targetPath: string;
|
||
}) {
|
||
return {
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: overrides.operationId,
|
||
files: [{ sourcePath: overrides.sourcePath, targetPath: overrides.targetPath, kind: "file" as const }],
|
||
},
|
||
],
|
||
};
|
||
}
|
||
|
||
beforeEach(() => {
|
||
mockGet.mockReset();
|
||
process.env.DAYTONA_API_KEY = "host-key";
|
||
__resetDaytonaSandboxHandleCacheForTest();
|
||
});
|
||
|
||
afterEach(async () => {
|
||
await Promise.all(tempDirs.splice(0).map((dir) => fs.rm(dir, { recursive: true, force: true })));
|
||
});
|
||
|
||
it("declares both sync hooks so the worker advertises the native transport", () => {
|
||
expect(plugin.definition.onEnvironmentSyncIn).toBeTypeOf("function");
|
||
expect(plugin.definition.onEnvironmentSyncOut).toBeTypeOf("function");
|
||
});
|
||
|
||
it("records the writablePath destination of a staging-tar rw mapping, not the staging parent", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const source = path.join(hostDir, "workspace.tar");
|
||
await fs.writeFile(source, "bytes");
|
||
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const params = {
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-rw",
|
||
files: [
|
||
{
|
||
// The mapping uploads a staging tar under the runtime root, and a
|
||
// post-upload command extracts it into the workspace directory. So
|
||
// `writablePath` names the real read-write destination.
|
||
sourcePath: source,
|
||
targetPath: `${REMOTE_DIR}/.paperclip-runtime/workspace-upload.tar`,
|
||
kind: "file" as const,
|
||
access: "rw" as const,
|
||
writablePath: REMOTE_DIR,
|
||
},
|
||
],
|
||
},
|
||
],
|
||
};
|
||
await plugin.definition.onEnvironmentSyncIn?.(params);
|
||
|
||
// The set holds the extract destination, not the staging archive parent.
|
||
const recorded = __getDaytonaWritableDirsForTest(params);
|
||
expect(recorded).toContain(REMOTE_DIR);
|
||
expect(recorded).not.toContain(`${REMOTE_DIR}/.paperclip-runtime`);
|
||
});
|
||
|
||
it("falls back to the parent directory of an rw mapping with no writablePath", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const source = path.join(hostDir, "in-place.txt");
|
||
await fs.writeFile(source, "bytes");
|
||
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const params = {
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-rw-inplace",
|
||
files: [
|
||
{
|
||
// No post-upload extract, so the mapping writes `targetPath` in
|
||
// place and the parent directory is the read-write destination.
|
||
sourcePath: source,
|
||
targetPath: `${REMOTE_DIR}/data/in-place.txt`,
|
||
kind: "file" as const,
|
||
access: "rw" as const,
|
||
},
|
||
],
|
||
},
|
||
],
|
||
};
|
||
await plugin.definition.onEnvironmentSyncIn?.(params);
|
||
|
||
expect(__getDaytonaWritableDirsForTest(params)).toContain(`${REMOTE_DIR}/data`);
|
||
});
|
||
|
||
it("skips ro and access-absent sync targets in the advisory writable set", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const roSource = path.join(hostDir, "referenced");
|
||
const defaultSource = path.join(hostDir, "default.tar");
|
||
await fs.mkdir(roSource, { recursive: true });
|
||
await fs.writeFile(path.join(roSource, "notes.md"), "reference");
|
||
await fs.writeFile(defaultSource, "bytes");
|
||
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const params = {
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-ro",
|
||
files: [
|
||
{
|
||
sourcePath: roSource,
|
||
targetPath: `${REMOTE_DIR}/.paperclip-runtime/project-proj-first`,
|
||
kind: "directory" as const,
|
||
access: "ro" as const,
|
||
},
|
||
{
|
||
// An absent `access` defaults to read-only, so it is not recorded.
|
||
sourcePath: defaultSource,
|
||
targetPath: `${REMOTE_DIR}/.paperclip-runtime/default-upload.tar`,
|
||
kind: "file" as const,
|
||
},
|
||
],
|
||
},
|
||
],
|
||
};
|
||
await plugin.definition.onEnvironmentSyncIn?.(params);
|
||
|
||
// Neither the ro directory nor the access-absent file directory is recorded.
|
||
expect(__getDaytonaWritableDirsForTest(params)).toEqual([]);
|
||
});
|
||
|
||
it("syncIn coalesces file mappings into one uploadFiles batch to reserved temp destinations, then one batched mv, applying secret mode via setFilePermissions before the rename", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const secretSource = path.join(hostDir, "auth.json");
|
||
const plainSource = path.join(hostDir, "config.txt");
|
||
await fs.writeFile(secretSource, "credential-material");
|
||
await fs.writeFile(plainSource, "plain");
|
||
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const result = await plugin.definition.onEnvironmentSyncIn?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-1",
|
||
files: [
|
||
{ sourcePath: secretSource, targetPath: `${REMOTE_DIR}/.secret/auth.json`, kind: "file", mode: 0o600 },
|
||
{ sourcePath: plainSource, targetPath: `${REMOTE_DIR}/config.txt`, kind: "file" },
|
||
],
|
||
},
|
||
],
|
||
});
|
||
|
||
// Exactly one bulk upload for both file mappings.
|
||
expect(sandbox.fs.uploadFiles).toHaveBeenCalledTimes(1);
|
||
const [uploads] = sandbox.fs.uploadFiles.mock.calls[0] as [Array<{ source: string; destination: string }>];
|
||
expect(uploads).toHaveLength(2);
|
||
// String sources stream from the local path; destinations are reserved temps.
|
||
expect(uploads[0].source).toBe(secretSource);
|
||
for (const upload of uploads) {
|
||
expect(path.posix.basename(upload.destination)).toMatch(/^\.paperclip-upload-/);
|
||
expect(upload.destination).not.toBe(`${REMOTE_DIR}/.secret/auth.json`);
|
||
// TOCTOU-hardened: the privileged upload destination is a DIRECT child of the
|
||
// workspace root, never a sibling under the target's (sandbox-swappable)
|
||
// parent dir, so a parent symlink swap cannot redirect the write out of root.
|
||
expect(path.posix.dirname(upload.destination)).toBe(REMOTE_DIR);
|
||
}
|
||
|
||
// Secret mode applied on the TEMP path (before the rename) so the target
|
||
// never appears at a widened window; applied via setFilePermissions as "600".
|
||
expect(sandbox.fs.setFilePermissions).toHaveBeenCalledTimes(1);
|
||
const [permPath, perms] = sandbox.fs.setFilePermissions.mock.calls[0] as [string, { mode: string }];
|
||
expect(path.posix.basename(permPath)).toMatch(/^\.paperclip-upload-/);
|
||
expect(perms).toEqual({ mode: "600" });
|
||
|
||
// The setFilePermissions on the temp precedes the mv that promotes it.
|
||
const secretTemp = permPath;
|
||
const mvCall = sandbox.process.executeCommand.mock.calls.find(([cmd]) => String(cmd).includes("mv -f"));
|
||
expect(mvCall).toBeDefined();
|
||
const mvCommand = String(mvCall?.[0]);
|
||
// Each promotion is one plain `mv -f <scratch> <target>` command, batched
|
||
// together in a single sandbox invocation.
|
||
expect(mvCommand).toContain(secretTemp);
|
||
expect(mvCommand).toContain(`${REMOTE_DIR}/.secret/auth.json`);
|
||
// Both temps are promoted (one mv line per rename).
|
||
expect(mvCommand.match(/mv -f /g)).toHaveLength(2);
|
||
|
||
expect(result).toEqual({
|
||
operations: [{ operationId: "sync-op-1", filesTransferred: 2, bytesTransferred: "credential-material".length + "plain".length }],
|
||
});
|
||
});
|
||
|
||
// A recording tracer that captures every provider span the file sync opens.
|
||
// It satisfies the structural plugin tracer contract.
|
||
function createRecordingPluginTracer() {
|
||
const spans: Array<{
|
||
name: string;
|
||
attributes: Record<string, unknown>;
|
||
status: { code: number; message?: string } | null;
|
||
ended: boolean;
|
||
}> = [];
|
||
const tracer = {
|
||
startSpan(name: string, options?: { attributes?: Record<string, string | number | boolean> }) {
|
||
const span = {
|
||
name,
|
||
attributes: { ...(options?.attributes ?? {}) } as Record<string, unknown>,
|
||
status: null as { code: number; message?: string } | null,
|
||
ended: false,
|
||
setAttribute(key: string, value: unknown) {
|
||
span.attributes[key] = value;
|
||
},
|
||
setStatus(status: { code: number; message?: string }) {
|
||
span.status = status;
|
||
},
|
||
end() {
|
||
span.ended = true;
|
||
},
|
||
};
|
||
spans.push(span);
|
||
return span;
|
||
},
|
||
};
|
||
return { tracer, spans };
|
||
}
|
||
|
||
it("opens a transfer span with the guard round-trip count around the bulk file upload", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const source = path.join(hostDir, "config.txt");
|
||
await fs.writeFile(source, "plain");
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const { tracer, spans } = createRecordingPluginTracer();
|
||
const restore = __setDaytonaPluginContextForTest({ tracer } as unknown as PluginContext);
|
||
try {
|
||
await plugin.definition.onEnvironmentSyncIn?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-1",
|
||
files: [{ sourcePath: source, targetPath: `${REMOTE_DIR}/config.txt`, kind: "file" }],
|
||
},
|
||
],
|
||
});
|
||
} finally {
|
||
restore();
|
||
}
|
||
|
||
const transfer = spans.find((span) => span.name === "transfer");
|
||
expect(transfer).toBeDefined();
|
||
expect(transfer!.ended).toBe(true);
|
||
// One serial guard round trip before the transfer: mkdir (with the zstd probe).
|
||
expect(transfer!.attributes["paperclip.sandbox.startup.transfer.guard.count"]).toBe(1);
|
||
expect(transfer!.attributes["paperclip.sandbox.startup.provider"]).toBe("daytona");
|
||
expect(typeof transfer!.attributes["paperclip.sandbox.startup.transfer.wall_ms"]).toBe("number");
|
||
// A bulk file upload builds no host tarball, so it opens no pack span.
|
||
expect(spans.find((span) => span.name === "pack")).toBeUndefined();
|
||
});
|
||
|
||
it("marks the inbound transfer span with the inbound direction attribute", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const source = path.join(hostDir, "config.txt");
|
||
await fs.writeFile(source, "plain");
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const { tracer, spans } = createRecordingPluginTracer();
|
||
const restore = __setDaytonaPluginContextForTest({ tracer } as unknown as PluginContext);
|
||
try {
|
||
await plugin.definition.onEnvironmentSyncIn?.(
|
||
syncInParams({
|
||
operationId: "sync-op-in-dir",
|
||
sourcePath: source,
|
||
targetPath: `${REMOTE_DIR}/config.txt`,
|
||
}),
|
||
);
|
||
} finally {
|
||
restore();
|
||
}
|
||
|
||
// An upload to the sandbox is an inbound transfer.
|
||
const transfer = spans.find((span) => span.name === "transfer");
|
||
expect(transfer).toBeDefined();
|
||
expect(transfer!.attributes["paperclip.sandbox.startup.transfer.direction"]).toBe("inbound");
|
||
});
|
||
|
||
it("marks the outbound transfer span with the outbound direction attribute", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const sandbox = createMockSandbox();
|
||
sandbox.fs.downloadFiles.mockImplementation(async (requests: Array<{ source: string; destination?: string }>) => {
|
||
return Promise.all(
|
||
requests.map(async (req) => {
|
||
await fs.writeFile(req.destination!, "bytes");
|
||
return { source: req.source, result: req.destination };
|
||
}),
|
||
);
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const { tracer, spans } = createRecordingPluginTracer();
|
||
const restore = __setDaytonaPluginContextForTest({ tracer } as unknown as PluginContext);
|
||
try {
|
||
await plugin.definition.onEnvironmentSyncOut?.(
|
||
syncOutParams({
|
||
operationId: "sync-op-out-dir",
|
||
sourcePath: `${REMOTE_DIR}/out/result.txt`,
|
||
targetPath: path.join(hostDir, "result.txt"),
|
||
}),
|
||
);
|
||
} finally {
|
||
restore();
|
||
}
|
||
|
||
// A download from the sandbox is an outbound transfer.
|
||
const transfer = spans.find((span) => span.name === "transfer");
|
||
expect(transfer).toBeDefined();
|
||
expect(transfer!.attributes["paperclip.sandbox.startup.transfer.direction"]).toBe("outbound");
|
||
});
|
||
|
||
it("opens a pack span and a transfer span around a directory mapping sync", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const sourceDir = path.join(hostDir, "assets");
|
||
await fs.mkdir(sourceDir, { recursive: true });
|
||
await fs.writeFile(path.join(sourceDir, "a.txt"), "alpha");
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const { tracer, spans } = createRecordingPluginTracer();
|
||
const restore = __setDaytonaPluginContextForTest({ tracer } as unknown as PluginContext);
|
||
try {
|
||
await plugin.definition.onEnvironmentSyncIn?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-dir",
|
||
files: [
|
||
{ sourcePath: sourceDir, targetPath: `${REMOTE_DIR}/.paperclip-runtime/assets`, kind: "directory" },
|
||
],
|
||
},
|
||
],
|
||
});
|
||
} finally {
|
||
restore();
|
||
}
|
||
|
||
const pack = spans.find((span) => span.name === "pack");
|
||
expect(pack).toBeDefined();
|
||
expect(pack!.ended).toBe(true);
|
||
expect(pack!.attributes["paperclip.sandbox.startup.provider"]).toBe("daytona");
|
||
expect(typeof pack!.attributes["paperclip.sandbox.startup.pack.wall_ms"]).toBe("number");
|
||
|
||
const transfer = spans.find((span) => span.name === "transfer");
|
||
expect(transfer).toBeDefined();
|
||
// One serial guard round trip before the transfer: mkdir.
|
||
expect(transfer!.attributes["paperclip.sandbox.startup.transfer.guard.count"]).toBe(1);
|
||
});
|
||
|
||
it("opens ensureDirectory, transfer, promote spans in call order for a file-mapping sync", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const source = path.join(hostDir, "config.txt");
|
||
await fs.writeFile(source, "plain");
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const { tracer, spans } = createRecordingPluginTracer();
|
||
const restore = __setDaytonaPluginContextForTest({ tracer } as unknown as PluginContext);
|
||
try {
|
||
await plugin.definition.onEnvironmentSyncIn?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-order",
|
||
files: [{ sourcePath: source, targetPath: `${REMOTE_DIR}/config.txt`, kind: "file" }],
|
||
},
|
||
],
|
||
});
|
||
} finally {
|
||
restore();
|
||
}
|
||
|
||
expect(spans.map((span) => span.name)).toEqual([
|
||
"ensureDirectory",
|
||
"transfer",
|
||
"promote",
|
||
]);
|
||
for (const span of spans) {
|
||
expect(span.ended).toBe(true);
|
||
expect(span.attributes["paperclip.sandbox.startup.provider"]).toBe("daytona");
|
||
// A per-round-trip span carries no `*.wall_ms` attribute; the native span
|
||
// width carries its time. Only `pack` and `transfer` keep a wall_ms value.
|
||
if (span.name !== "transfer") {
|
||
expect(span.attributes["paperclip.sandbox.startup.ensureDirectory.wall_ms"]).toBeUndefined();
|
||
expect(span.attributes["paperclip.sandbox.startup.promote.wall_ms"]).toBeUndefined();
|
||
}
|
||
}
|
||
});
|
||
|
||
it("opens pack, ensureDirectory, transfer, extractTarball spans in call order for a directory-mapping sync", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const sourceDir = path.join(hostDir, "assets");
|
||
await fs.mkdir(sourceDir, { recursive: true });
|
||
await fs.writeFile(path.join(sourceDir, "a.txt"), "alpha");
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const { tracer, spans } = createRecordingPluginTracer();
|
||
const restore = __setDaytonaPluginContextForTest({ tracer } as unknown as PluginContext);
|
||
try {
|
||
await plugin.definition.onEnvironmentSyncIn?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-dir-order",
|
||
files: [
|
||
{ sourcePath: sourceDir, targetPath: `${REMOTE_DIR}/.paperclip-runtime/assets`, kind: "directory" },
|
||
],
|
||
},
|
||
],
|
||
});
|
||
} finally {
|
||
restore();
|
||
}
|
||
|
||
expect(spans.map((span) => span.name)).toEqual([
|
||
"pack",
|
||
"ensureDirectory",
|
||
"transfer",
|
||
"extractTarball",
|
||
]);
|
||
for (const span of spans) {
|
||
expect(span.attributes["paperclip.sandbox.startup.provider"]).toBe("daytona");
|
||
}
|
||
});
|
||
|
||
it("opens a postUploadCommand span for a post-upload command with a working directory", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const source = path.join(hostDir, "config.txt");
|
||
await fs.writeFile(source, "plain");
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const { tracer, spans } = createRecordingPluginTracer();
|
||
const restore = __setDaytonaPluginContextForTest({ tracer } as unknown as PluginContext);
|
||
try {
|
||
await plugin.definition.onEnvironmentSyncIn?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-post",
|
||
files: [{ sourcePath: source, targetPath: `${REMOTE_DIR}/config.txt`, kind: "file" }],
|
||
postUploadCommands: [{ command: "run-me", cwd: `${REMOTE_DIR}/sub` }],
|
||
},
|
||
],
|
||
});
|
||
} finally {
|
||
restore();
|
||
}
|
||
|
||
// The full order: the file mapping opens ensureDirectory, transfer, promote;
|
||
// the post-upload command then opens the postUploadCommand span.
|
||
expect(spans.map((span) => span.name)).toEqual([
|
||
"ensureDirectory",
|
||
"transfer",
|
||
"promote",
|
||
"postUploadCommand",
|
||
]);
|
||
const provision = spans.find((span) => span.name === "postUploadCommand");
|
||
expect(provision!.ended).toBe(true);
|
||
expect(provision!.attributes["paperclip.sandbox.startup.provider"]).toBe("daytona");
|
||
});
|
||
|
||
it("gzip-tars a directory mapping host-side honoring excludes and the followSymlinks flag, then extracts it in-sandbox via a single quoted tar command", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const sourceDir = path.join(hostDir, "assets");
|
||
await fs.mkdir(path.join(sourceDir, "keep"), { recursive: true });
|
||
await fs.writeFile(path.join(sourceDir, "keep", "a.txt"), "alpha");
|
||
await fs.writeFile(path.join(sourceDir, "skip.log"), "noise");
|
||
await fs.symlink("keep/a.txt", path.join(sourceDir, "link.txt"));
|
||
|
||
const sandbox = createMockSandbox();
|
||
// Capture the tar listing inside the upload mock, before withHostTempDir
|
||
// cleans the host scratch dir.
|
||
let capturedTarListing = "";
|
||
sandbox.fs.uploadFiles.mockImplementation(async (uploads: Array<{ source: string }>) => {
|
||
capturedTarListing = execFileSync("tar", ["-tvf", uploads[0].source]).toString();
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
// Preserve-symlink case (followSymlinks falsy → no -h).
|
||
await plugin.definition.onEnvironmentSyncIn?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-dir",
|
||
files: [
|
||
{
|
||
sourcePath: sourceDir,
|
||
targetPath: `${REMOTE_DIR}/.paperclip-runtime/assets`,
|
||
kind: "directory",
|
||
exclude: ["*.log"],
|
||
},
|
||
],
|
||
},
|
||
],
|
||
});
|
||
|
||
expect(sandbox.fs.uploadFiles).toHaveBeenCalledTimes(1);
|
||
const [uploads] = sandbox.fs.uploadFiles.mock.calls[0] as [Array<{ source: string; destination: string }>];
|
||
expect(uploads).toHaveLength(1);
|
||
expect(uploads[0].source).toMatch(/\.tar\.gz$/);
|
||
expect(path.posix.basename(uploads[0].destination)).toMatch(/^\.paperclip-upload-.*\.tar\.gz$/);
|
||
expect(uploads[0].destination.startsWith(`${REMOTE_DIR}/`)).toBe(true);
|
||
|
||
// Inspect the real host tar: excluded file gone; symlink preserved AS a link.
|
||
expect(capturedTarListing).toContain("keep/a.txt");
|
||
expect(capturedTarListing).not.toContain("skip.log");
|
||
expect(capturedTarListing).toMatch(/link\.txt ->|link\.txt link to/);
|
||
|
||
// The target dir is created by its own mkdir command, before the upload.
|
||
const mkdirCall = sandbox.process.executeCommand.mock.calls.find(
|
||
([cmd]) =>
|
||
String(cmd).includes("mkdir -p") &&
|
||
String(cmd).includes(`'${REMOTE_DIR}/.paperclip-runtime/assets'`) &&
|
||
!String(cmd).includes("tar -xf"),
|
||
);
|
||
expect(mkdirCall).toBeDefined();
|
||
|
||
const extractCall = sandbox.process.executeCommand.mock.calls.find(([cmd]) => String(cmd).includes("tar -xf"));
|
||
expect(extractCall).toBeDefined();
|
||
const extractCommand = String(extractCall?.[0]);
|
||
// The extract is one plain `tar -xf <scratch-tar> -C <target>` command,
|
||
// followed by removing the scratch tar.
|
||
expect(extractCommand).toContain(".paperclip-runtime/assets");
|
||
expect(extractCommand).toContain("tar -xf");
|
||
expect(extractCommand).toMatch(/rm -f .*\.paperclip-upload-.*\.tar\.gz/);
|
||
});
|
||
|
||
it("syncIn dereferences symlinks to bytes when followSymlinks is true (tar -h)", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const sourceDir = path.join(hostDir, "deref");
|
||
await fs.mkdir(sourceDir, { recursive: true });
|
||
await fs.writeFile(path.join(sourceDir, "real.txt"), "payload");
|
||
await fs.symlink("real.txt", path.join(sourceDir, "alias.txt"));
|
||
|
||
const sandbox = createMockSandbox();
|
||
let capturedTarListing = "";
|
||
sandbox.fs.uploadFiles.mockImplementation(async (uploads: Array<{ source: string }>) => {
|
||
capturedTarListing = execFileSync("tar", ["-tvf", uploads[0].source]).toString();
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentSyncIn?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-deref",
|
||
files: [{ sourcePath: sourceDir, targetPath: `${REMOTE_DIR}/deref`, kind: "directory", followSymlinks: true }],
|
||
},
|
||
],
|
||
});
|
||
|
||
// Dereferenced: alias becomes a regular file, not a link.
|
||
expect(capturedTarListing).not.toMatch(/alias\.txt ->/);
|
||
expect(capturedTarListing).toContain("alias.txt");
|
||
});
|
||
|
||
it("syncOut reads all file mappings via one downloadFiles batch, writes each to its host target, and returns per-operation counts", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const sandbox = createMockSandbox();
|
||
// The download reads sandbox-side snapshots (reserved temp names), which are
|
||
// index-aligned with the file mappings; write payloads in request order.
|
||
const payloadsInOrder = ["result-bytes", "secret-bytes"];
|
||
sandbox.fs.downloadFiles.mockImplementation(async (requests: Array<{ source: string; destination?: string }>) => {
|
||
return Promise.all(
|
||
requests.map(async (req, index) => {
|
||
await fs.writeFile(req.destination!, payloadsInOrder[index]);
|
||
return { source: req.source, result: req.destination };
|
||
}),
|
||
);
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const resultTarget = path.join(hostDir, "result.txt");
|
||
const secretTarget = path.join(hostDir, "secret.key");
|
||
const result = await plugin.definition.onEnvironmentSyncOut?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-out",
|
||
files: [
|
||
{ sourcePath: `${REMOTE_DIR}/out/result.txt`, targetPath: resultTarget, kind: "file" },
|
||
{ sourcePath: `${REMOTE_DIR}/out/secret.key`, targetPath: secretTarget, kind: "file", mode: 0o600 },
|
||
],
|
||
},
|
||
],
|
||
});
|
||
|
||
expect(sandbox.fs.downloadFiles).toHaveBeenCalledTimes(1);
|
||
const [requests] = sandbox.fs.downloadFiles.mock.calls[0] as [Array<{ source: string; destination: string }>];
|
||
expect(requests).toHaveLength(2);
|
||
for (const req of requests) {
|
||
expect(path.basename(req.destination)).toMatch(/^\.paperclip-upload-/);
|
||
// TOCTOU-closed: the download reads a reserved snapshot inside the remote
|
||
// dir, never the mutable original source path.
|
||
expect(req.source.startsWith(`${REMOTE_DIR}/`)).toBe(true);
|
||
expect(path.posix.basename(req.source)).toMatch(/^\.paperclip-upload-/);
|
||
}
|
||
expect(requests.map((req) => req.source)).not.toContain(`${REMOTE_DIR}/out/result.txt`);
|
||
expect(requests.map((req) => req.source)).not.toContain(`${REMOTE_DIR}/out/secret.key`);
|
||
|
||
expect(await fs.readFile(resultTarget, "utf8")).toBe("result-bytes");
|
||
expect(await fs.readFile(secretTarget, "utf8")).toBe("secret-bytes");
|
||
// Secret lands 0600 on the host target.
|
||
expect((await fs.stat(secretTarget)).mode & 0o777).toBe(0o600);
|
||
|
||
expect(result).toEqual({
|
||
operations: [
|
||
{ operationId: "sync-op-out", filesTransferred: 2, bytesTransferred: "result-bytes".length + "secret-bytes".length },
|
||
],
|
||
});
|
||
});
|
||
|
||
it.each(["sandbox_access", "file_download"] as const)("keeps the deleted-sandbox policy and bounded %s evidence", async (step) => {
|
||
const hostDir = await makeHostDir();
|
||
const source = Object.freeze(Object.assign(new MockDaytonaNotFoundError("private-provider-detail"), {
|
||
status: 404, cause: { code: "EIO", token: "private-provider-token" },
|
||
}));
|
||
if (step === "sandbox_access") mockGet.mockRejectedValue(source);
|
||
else {
|
||
const sandbox = createMockSandbox();
|
||
sandbox.fs.downloadFiles.mockRejectedValue(source);
|
||
mockGet.mockResolvedValue(sandbox);
|
||
}
|
||
await withEnvironmentSyncErrorCapture(async () => {
|
||
const error = await plugin.definition.onEnvironmentSyncOut!({
|
||
driverKey: "daytona", companyId: "company-1", environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: true }, lease: syncLease(),
|
||
operations: [{ operationId: "sync-op-missing-sandbox", files: [{
|
||
sourcePath: `${REMOTE_DIR}/out/result.txt`, targetPath: path.join(hostDir, "result.txt"), kind: "file",
|
||
}] }],
|
||
}).catch(error => error);
|
||
expect(error).toBeInstanceOf(Error);
|
||
expect(error.message).toBe("daytona_sandbox_not_found");
|
||
expect(error).not.toHaveProperty("cause");
|
||
const diagnostic = readEnvironmentSyncErrorDiagnostic({ data: environmentSyncErrorData(error) });
|
||
expect(diagnostic).toEqual({ errorCode: "EIO", httpStatus: 404, transferStep: step });
|
||
expect(JSON.stringify(diagnostic)).not.toContain("private-provider-");
|
||
});
|
||
});
|
||
|
||
it("syncOut snapshot guard re-checks the resolved source is a non-symlink regular file immediately before copying (validation→copy TOCTOU)", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const sandbox = createMockSandbox();
|
||
sandbox.fs.downloadFiles.mockImplementation(async (requests: Array<{ source: string; destination?: string }>) => {
|
||
return Promise.all(
|
||
requests.map(async (req) => {
|
||
await fs.writeFile(req.destination!, "bytes");
|
||
return { source: req.source, result: req.destination };
|
||
}),
|
||
);
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentSyncOut?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-out-nofollow",
|
||
files: [{ sourcePath: `${REMOTE_DIR}/out/data.txt`, targetPath: path.join(hostDir, "data.txt"), kind: "file" }],
|
||
},
|
||
],
|
||
});
|
||
|
||
// The snapshot guard runs realpath → confine → no-follow re-check → cp, all in
|
||
// one `sh -c`. The `[ -L ]`/`[ -f ]` re-check must precede the `cp` so a source
|
||
// the sandbox repointed to a symlink after realpath is refused, not followed.
|
||
const guardCall = sandbox.process.executeCommand.mock.calls.find(
|
||
([cmd]) => String(cmd).includes("_pc_resolve") && String(cmd).includes("cp --"),
|
||
);
|
||
expect(guardCall).toBeDefined();
|
||
const guardCommand = String(guardCall?.[0]);
|
||
expect(guardCommand).toContain('[ -L "$_pc_real" ]');
|
||
expect(guardCommand).toContain('[ -f "$_pc_real" ]');
|
||
const noFollowIdx = guardCommand.indexOf('[ -L "$_pc_real" ]');
|
||
const copyIdx = guardCommand.indexOf('cp -- "$_pc_real"');
|
||
expect(noFollowIdx).toBeGreaterThan(-1);
|
||
expect(copyIdx).toBeGreaterThan(noFollowIdx);
|
||
});
|
||
|
||
it("syncOut fails loud when any per-file download reports an error, and leaves no target file", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const sandbox = createMockSandbox();
|
||
// Requests read snapshots (index-aligned with mappings); the second mapping
|
||
// (`missing.txt`) reports a per-file error.
|
||
sandbox.fs.downloadFiles.mockImplementation(async (requests: Array<{ source: string; destination?: string }>) => {
|
||
return requests.map((req, index) =>
|
||
index === 1
|
||
? { source: req.source, error: "not found", errorDetails: { message: "not found", statusCode: 404 } }
|
||
: { source: req.source, result: req.destination },
|
||
);
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const okTarget = path.join(hostDir, "ok.txt");
|
||
const badTarget = path.join(hostDir, "missing.txt");
|
||
await expect(
|
||
plugin.definition.onEnvironmentSyncOut?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-err",
|
||
files: [
|
||
{ sourcePath: `${REMOTE_DIR}/ok.txt`, targetPath: okTarget, kind: "file" },
|
||
{ sourcePath: `${REMOTE_DIR}/missing.txt`, targetPath: badTarget, kind: "file" },
|
||
],
|
||
},
|
||
],
|
||
}),
|
||
).rejects.toThrow(/download failed for .*missing\.txt: not found/);
|
||
|
||
// Fail-loud: no target file is promoted when the batch has any error.
|
||
await expect(fs.stat(okTarget)).rejects.toThrow();
|
||
await expect(fs.stat(badTarget)).rejects.toThrow();
|
||
});
|
||
|
||
it("syncOut rejects an outbound source whose in-sandbox realpath escapes the workspace remote dir, before any download", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const sandbox = createMockSandbox();
|
||
// The in-sandbox realpath guard runs as a single `sh -c` probe; report the
|
||
// escape exit code (42) for that probe while leaving any other command green,
|
||
// so the guard is the only thing that can trip this test.
|
||
sandbox.process.executeCommand.mockImplementation(async (command: string) => {
|
||
if (command.includes("_pc_resolve")) {
|
||
return { exitCode: 42, result: `ESCAPE:${REMOTE_DIR}/out/link.txt`, artifacts: { stdout: "" } };
|
||
}
|
||
return { exitCode: 0, result: "bash", artifacts: { stdout: "bash" } };
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const target = path.join(hostDir, "link.txt");
|
||
await expect(
|
||
plugin.definition.onEnvironmentSyncOut?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-escape-out",
|
||
files: [{ sourcePath: `${REMOTE_DIR}/out/link.txt`, targetPath: target, kind: "file" }],
|
||
},
|
||
],
|
||
}),
|
||
).rejects.toThrow(/outbound symlink-escape guard command failed \(exit 42\)/);
|
||
|
||
// Fail-closed: the guard trips before any bytes are read, and no target lands.
|
||
expect(sandbox.fs.downloadFiles).not.toHaveBeenCalled();
|
||
await expect(fs.stat(target)).rejects.toThrow();
|
||
});
|
||
|
||
it("syncOut fails closed when the sandbox has no path canonicalizer to resolve the symlink guard", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const sandbox = createMockSandbox();
|
||
// Neither `realpath` nor `readlink` present → the probe exits 40 rather than
|
||
// silently skipping the guard the host-side string check cannot enforce.
|
||
sandbox.process.executeCommand.mockImplementation(async (command: string) => {
|
||
if (command.includes("_pc_resolve")) {
|
||
return { exitCode: 40, result: "no path canonicalizer available", artifacts: { stdout: "" } };
|
||
}
|
||
return { exitCode: 0, result: "bash", artifacts: { stdout: "bash" } };
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const target = path.join(hostDir, "data.txt");
|
||
await expect(
|
||
plugin.definition.onEnvironmentSyncOut?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-no-canon",
|
||
files: [{ sourcePath: `${REMOTE_DIR}/out/data.txt`, targetPath: target, kind: "file" }],
|
||
},
|
||
],
|
||
}),
|
||
).rejects.toThrow(/outbound symlink-escape guard command failed \(exit 40\)/);
|
||
|
||
expect(sandbox.fs.downloadFiles).not.toHaveBeenCalled();
|
||
await expect(fs.stat(target)).rejects.toThrow();
|
||
});
|
||
|
||
it("syncIn sweeps staged temps when the batched rename fails mid-promotion", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const source = path.join(hostDir, "config.txt");
|
||
await fs.writeFile(source, "plain");
|
||
|
||
const sandbox = createMockSandbox();
|
||
// mkdir + realpath guard succeed; the promoting `mv -f` fails, leaving staged
|
||
// `.paperclip-upload-*` temps that the error path must sweep with `rm -f`.
|
||
sandbox.process.executeCommand.mockImplementation(async (command: string) => {
|
||
if (command.includes("mv -f")) {
|
||
return { exitCode: 1, result: "mv: permission denied", artifacts: { stdout: "mv: permission denied" } };
|
||
}
|
||
return { exitCode: 0, result: "bash", artifacts: { stdout: "bash" } };
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await expect(
|
||
plugin.definition.onEnvironmentSyncIn?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-in-rename-fail",
|
||
files: [{ sourcePath: source, targetPath: `${REMOTE_DIR}/config.txt`, kind: "file" }],
|
||
},
|
||
],
|
||
}),
|
||
).rejects.toThrow(/syncIn rename command failed \(exit 1\)/);
|
||
|
||
// The upload happened, so a temp was staged; the error path cleans it up.
|
||
expect(sandbox.fs.uploadFiles).toHaveBeenCalledTimes(1);
|
||
const cleanupCall = sandbox.process.executeCommand.mock.calls.find(
|
||
([cmd]) => String(cmd).includes("rm -f") && String(cmd).includes(".paperclip-upload-"),
|
||
);
|
||
expect(cleanupCall).toBeDefined();
|
||
});
|
||
|
||
it("syncOut refuses a sandbox-authored tarball whose members escape the extraction dir (path traversal)", async () => {
|
||
const hostRoot = await makeHostDir();
|
||
const restored = path.join(hostRoot, "restored");
|
||
const sandbox = createMockSandbox();
|
||
sandbox.fs.downloadFiles.mockImplementation(async (requests: Array<{ source: string; destination?: string }>) => {
|
||
return Promise.all(
|
||
requests.map(async (req) => {
|
||
// Craft a tar containing a traversal member `../escape.txt`.
|
||
const staging = await fs.mkdtemp(path.join(os.tmpdir(), "daytona-evil-"));
|
||
tempDirs.push(staging);
|
||
await fs.mkdir(path.join(staging, "sub"), { recursive: true });
|
||
await fs.writeFile(path.join(staging, "sub", "escape.txt"), "escape");
|
||
// GNU spells member-name rewriting --transform; bsdtar (macOS) spells it -s.
|
||
const gnuTar = execFileSync("tar", ["--version"]).toString().includes("GNU tar");
|
||
execFileSync("tar", [
|
||
"-cf",
|
||
req.destination!,
|
||
"-C",
|
||
path.join(staging, "sub"),
|
||
...(gnuTar ? ["--transform", "s,^,../,"] : ["-s", ",^,../,"]),
|
||
"escape.txt",
|
||
]);
|
||
return { source: req.source, result: req.destination };
|
||
}),
|
||
);
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await expect(
|
||
plugin.definition.onEnvironmentSyncOut?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-out-traversal",
|
||
files: [{ sourcePath: `${REMOTE_DIR}/proj`, targetPath: restored, kind: "directory" }],
|
||
},
|
||
],
|
||
}),
|
||
).rejects.toThrow(/escapes the extraction dir/);
|
||
|
||
// The traversal member (`../escape.txt` relative to `restored`) was never
|
||
// written above the extraction dir.
|
||
await expect(fs.stat(path.join(hostRoot, "escape.txt"))).rejects.toThrow();
|
||
});
|
||
|
||
it.each(["../../outside.txt", "/tmp/paperclip-git-workspace-example/skills/demo"])("syncOut refuses a sandbox-authored tarball with escaping symlink target %s", async (linkTarget) => {
|
||
const hostRoot = await makeHostDir();
|
||
const restored = path.join(hostRoot, "restored");
|
||
const sandbox = createMockSandbox();
|
||
sandbox.fs.downloadFiles.mockImplementation(async (requests: Array<{ source: string; destination?: string }>) => {
|
||
return Promise.all(
|
||
requests.map(async (req) => {
|
||
// Craft a tar whose sole member is a symlink pointing outside the tree.
|
||
const staging = await fs.mkdtemp(path.join(os.tmpdir(), "daytona-evil-"));
|
||
tempDirs.push(staging);
|
||
await fs.mkdir(path.join(staging, "sub"), { recursive: true });
|
||
await fs.symlink(linkTarget, path.join(staging, "sub", "evil"));
|
||
execFileSync("tar", ["-cf", req.destination!, "-C", path.join(staging, "sub"), "evil"]);
|
||
return { source: req.source, result: req.destination };
|
||
}),
|
||
);
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await expect(
|
||
plugin.definition.onEnvironmentSyncOut?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-out-symlink-escape",
|
||
files: [{ sourcePath: `${REMOTE_DIR}/proj`, targetPath: restored, kind: "directory" }],
|
||
},
|
||
],
|
||
}),
|
||
).rejects.toThrow(/link whose target escapes the extraction dir/);
|
||
|
||
// Fail-closed: the confinement check runs before extraction touches disk.
|
||
await expect(fs.stat(restored)).rejects.toThrow();
|
||
});
|
||
|
||
it("syncOut refuses a symlink whose name embeds the listing delimiter (ambiguous split hides the real target)", async () => {
|
||
const hostRoot = await makeHostDir();
|
||
const restored = path.join(hostRoot, "restored");
|
||
const sandbox = createMockSandbox();
|
||
sandbox.fs.downloadFiles.mockImplementation(async (requests: Array<{ source: string; destination?: string }>) => {
|
||
return Promise.all(
|
||
requests.map(async (req) => {
|
||
// A symlink literally named "evil -> decoy" with an escaping target
|
||
// lists as "evil -> decoy -> ../../outside.txt"; splitting at the
|
||
// first delimiter would validate "decoy -> ../../outside.txt" (which
|
||
// normalizes in-tree) while tar extracts the real escaping link.
|
||
const staging = await fs.mkdtemp(path.join(os.tmpdir(), "daytona-evil-"));
|
||
tempDirs.push(staging);
|
||
await fs.mkdir(path.join(staging, "sub"), { recursive: true });
|
||
await fs.symlink("../../outside.txt", path.join(staging, "sub", "evil -> decoy"));
|
||
execFileSync("tar", ["-cf", req.destination!, "-C", path.join(staging, "sub"), "evil -> decoy"]);
|
||
return { source: req.source, result: req.destination };
|
||
}),
|
||
);
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await expect(
|
||
plugin.definition.onEnvironmentSyncOut?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-out-ambiguous-symlink",
|
||
files: [{ sourcePath: `${REMOTE_DIR}/proj`, targetPath: restored, kind: "directory" }],
|
||
},
|
||
],
|
||
}),
|
||
).rejects.toThrow(/ambiguous symlink entry/);
|
||
|
||
// Fail-closed: the confinement check runs before extraction touches disk.
|
||
await expect(fs.stat(restored)).rejects.toThrow();
|
||
});
|
||
|
||
it("round-trips a directory (syncIn then syncOut) preserving contents, a 0600 file, and a preserved symlink", async () => {
|
||
const hostRoot = await makeHostDir();
|
||
const source = path.join(hostRoot, "src");
|
||
await fs.mkdir(path.join(source, "nested"), { recursive: true });
|
||
await fs.writeFile(path.join(source, "nested", "data.txt"), "hello world");
|
||
await fs.writeFile(path.join(source, "secret"), "top-secret");
|
||
await fs.chmod(path.join(source, "secret"), 0o600);
|
||
await fs.symlink("nested/data.txt", path.join(source, "shortcut"));
|
||
await fs.mkdir(path.join(source, ".claude", "skills"), { recursive: true });
|
||
await fs.symlink("../../nested", path.join(source, ".claude", "skills", "demo"));
|
||
|
||
// Simulate the sandbox filesystem with a host-side directory the mock tar
|
||
// commands operate on, so the round-trip exercises real tar create/extract.
|
||
const sandboxFsRoot = await makeHostDir();
|
||
const remoteTargetDir = path.join(sandboxFsRoot, "materialized");
|
||
|
||
const sandbox = createMockSandbox();
|
||
// syncIn: capture the uploaded host tar and extract it into the simulated
|
||
// sandbox dir, mirroring the in-sandbox `tar -xf`.
|
||
sandbox.fs.uploadFiles.mockImplementation(async (uploads: Array<{ source: string; destination: string }>) => {
|
||
for (const upload of uploads) {
|
||
await fs.mkdir(remoteTargetDir, { recursive: true });
|
||
execFileSync("tar", ["-xpf", upload.source, "-C", remoteTargetDir]);
|
||
}
|
||
});
|
||
// syncOut: build a tar of the simulated sandbox dir and stream it to the
|
||
// requested host destination, mirroring in-sandbox `tar -c` + downloadFiles.
|
||
sandbox.fs.downloadFiles.mockImplementation(async (requests: Array<{ source: string; destination?: string }>) => {
|
||
return Promise.all(
|
||
requests.map(async (req) => {
|
||
const entries = (await fs.readdir(remoteTargetDir)).sort();
|
||
execFileSync("tar", ["-cpf", req.destination!, "-C", remoteTargetDir, "--", ...entries]);
|
||
return { source: req.source, result: req.destination };
|
||
}),
|
||
);
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentSyncIn?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{ operationId: "rt-in", files: [{ sourcePath: source, targetPath: `${REMOTE_DIR}/proj`, kind: "directory" }] },
|
||
],
|
||
});
|
||
|
||
const restored = path.join(hostRoot, "restored");
|
||
await plugin.definition.onEnvironmentSyncOut?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{ operationId: "rt-out", files: [{ sourcePath: `${REMOTE_DIR}/proj`, targetPath: restored, kind: "directory" }] },
|
||
],
|
||
});
|
||
|
||
expect(await fs.readFile(path.join(restored, "nested", "data.txt"), "utf8")).toBe("hello world");
|
||
expect(await fs.readFile(path.join(restored, "secret"), "utf8")).toBe("top-secret");
|
||
expect((await fs.stat(path.join(restored, "secret"))).mode & 0o777).toBe(0o600);
|
||
const linkStat = await fs.lstat(path.join(restored, "shortcut"));
|
||
expect(linkStat.isSymbolicLink()).toBe(true);
|
||
expect(await fs.readlink(path.join(restored, "shortcut"))).toBe("nested/data.txt");
|
||
expect(await fs.readlink(path.join(restored, ".claude", "skills", "demo"))).toBe("../../nested");
|
||
expect(await fs.readFile(path.join(restored, ".claude", "skills", "demo", "data.txt"), "utf8")).toBe("hello world");
|
||
});
|
||
|
||
// -------------------------------------------------------------------------
|
||
// Post-upload commands (Phase 3 / Security Conditions C1–C4). Daytona runs an
|
||
// operation's ordered `postUploadCommands` in-sandbox AFTER `uploadFiles`,
|
||
// fail-fast, with the command `cwd` re-confined under the workspace remote dir.
|
||
// -------------------------------------------------------------------------
|
||
|
||
it("runs post-upload commands in array order AFTER uploadFiles, each verbatim via the exec seam", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const source = path.join(hostDir, "config.txt");
|
||
await fs.writeFile(source, "plain");
|
||
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await plugin.definition.onEnvironmentSyncIn?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "op-cmd",
|
||
files: [{ sourcePath: source, targetPath: `${REMOTE_DIR}/config.txt`, kind: "file" }],
|
||
postUploadCommands: [
|
||
{ command: "codex-auth-merge --first" },
|
||
{ command: "chmod 600 config.txt" },
|
||
],
|
||
},
|
||
],
|
||
});
|
||
|
||
// Both commands ran, VERBATIM (first arg is the exact authored string — the
|
||
// provider never rewrote/concatenated a shell fragment onto it: C1/C3).
|
||
const findCall = (cmd: string) =>
|
||
sandbox.process.executeCommand.mock.calls.find(([c]) => c === cmd);
|
||
expect(findCall("codex-auth-merge --first")).toBeDefined();
|
||
expect(findCall("chmod 600 config.txt")).toBeDefined();
|
||
|
||
// Ordered: the first command's exec precedes the second's (C4 array order).
|
||
const orderOf = (cmd: string) => {
|
||
const idx = sandbox.process.executeCommand.mock.calls.findIndex(([c]) => c === cmd);
|
||
return sandbox.process.executeCommand.mock.invocationCallOrder[idx];
|
||
};
|
||
expect(orderOf("codex-auth-merge --first")).toBeLessThan(orderOf("chmod 600 config.txt"));
|
||
|
||
// Upload happened BEFORE the first command.
|
||
expect(sandbox.fs.uploadFiles.mock.invocationCallOrder[0]).toBeLessThan(
|
||
orderOf("codex-auth-merge --first"),
|
||
);
|
||
|
||
// Absent `cwd` defaults to the provider-resolved remote dir — never a process
|
||
// default cwd (C2). The command's structured cwd argument is REMOTE_DIR.
|
||
expect(findCall("codex-auth-merge --first")?.[1]).toBe(REMOTE_DIR);
|
||
});
|
||
|
||
it("aborts the operation fail-loud on a non-zero post-upload command exit, skipping the remainder (C4)", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const source = path.join(hostDir, "config.txt");
|
||
await fs.writeFile(source, "plain");
|
||
|
||
const sandbox = createMockSandbox();
|
||
// The first command exits non-zero; every transfer/guard script stays green.
|
||
sandbox.process.executeCommand.mockImplementation(async (command: string) => {
|
||
if (command === "failing-command") {
|
||
return { exitCode: 7, result: "boom", artifacts: { stdout: "boom" } };
|
||
}
|
||
return { exitCode: 0, result: "", artifacts: { stdout: "" } };
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await expect(
|
||
plugin.definition.onEnvironmentSyncIn?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "op-fail",
|
||
files: [{ sourcePath: source, targetPath: `${REMOTE_DIR}/config.txt`, kind: "file" }],
|
||
postUploadCommands: [{ command: "failing-command" }, { command: "should-not-run" }],
|
||
},
|
||
],
|
||
}),
|
||
).rejects.toThrow(/post-upload command failed \(exit 7\)/);
|
||
|
||
// Fail-fast: the command after the failing one never executed.
|
||
expect(
|
||
sandbox.process.executeCommand.mock.calls.some(([c]) => c === "should-not-run"),
|
||
).toBe(false);
|
||
});
|
||
|
||
// -------------------------------------------------------------------------
|
||
// Merged git-workspace operation. A git-backed workspace stage-sync rides ONE
|
||
// operation whose `files` carry the git-history tar and the workspace-overlay
|
||
// tar, with the two extract commands as ordered `postUploadCommands`. The
|
||
// operation shares one mkdir, one confine guard, one `uploadFiles`, and one
|
||
// rename exec.
|
||
// -------------------------------------------------------------------------
|
||
|
||
it("stages a merged git-workspace operation as one uploadFiles batch and one rename exec, both extracts in order", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const gitTar = path.join(hostDir, "git-workspace.tar");
|
||
const overlayTar = path.join(hostDir, "workspace.tar");
|
||
await fs.writeFile(gitTar, "git-bytes");
|
||
await fs.writeFile(overlayTar, "overlay-bytes");
|
||
const runtimeDir = `${REMOTE_DIR}/.paperclip-runtime/adapter`;
|
||
|
||
const sandbox = createMockSandbox();
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const gitExtract = "git-history-extract";
|
||
const overlayExtract = "workspace-overlay-extract";
|
||
const result = await plugin.definition.onEnvironmentSyncIn?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "merged-workspace",
|
||
files: [
|
||
{ sourcePath: gitTar, targetPath: `${runtimeDir}/git-workspace-upload.tar`, kind: "file" },
|
||
{ sourcePath: overlayTar, targetPath: `${runtimeDir}/workspace-upload.tar`, kind: "file" },
|
||
],
|
||
postUploadCommands: [{ command: gitExtract }, { command: overlayExtract }],
|
||
},
|
||
],
|
||
});
|
||
|
||
// One bulk upload carries BOTH tars; one rename exec promotes both temps.
|
||
expect(sandbox.fs.uploadFiles).toHaveBeenCalledTimes(1);
|
||
const [uploads] = sandbox.fs.uploadFiles.mock.calls[0] as [Array<{ source: string; destination: string }>];
|
||
expect(uploads).toHaveLength(2);
|
||
const mvCalls = sandbox.process.executeCommand.mock.calls.filter(([cmd]) =>
|
||
String(cmd).includes("mv -f"),
|
||
);
|
||
expect(mvCalls).toHaveLength(1);
|
||
expect(String(mvCalls[0][0]).match(/mv -f /g)).toHaveLength(2);
|
||
|
||
// Both extract commands ran, in array order, AFTER the upload (git first).
|
||
const orderOf = (cmd: string) => {
|
||
const idx = sandbox.process.executeCommand.mock.calls.findIndex(([c]) => c === cmd);
|
||
return sandbox.process.executeCommand.mock.invocationCallOrder[idx];
|
||
};
|
||
expect(orderOf(gitExtract)).toBeLessThan(orderOf(overlayExtract));
|
||
expect(sandbox.fs.uploadFiles.mock.invocationCallOrder[0]).toBeLessThan(orderOf(gitExtract));
|
||
|
||
expect(result).toEqual({
|
||
operations: [{
|
||
operationId: "merged-workspace",
|
||
filesTransferred: 2,
|
||
bytesTransferred: "git-bytes".length + "overlay-bytes".length,
|
||
}],
|
||
});
|
||
});
|
||
|
||
it("stops the overlay and remove-deleted commands when the git extract fails (merged operation fail-fast)", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const gitTar = path.join(hostDir, "git-workspace.tar");
|
||
const overlayTar = path.join(hostDir, "workspace.tar");
|
||
await fs.writeFile(gitTar, "git-bytes");
|
||
await fs.writeFile(overlayTar, "overlay-bytes");
|
||
const runtimeDir = `${REMOTE_DIR}/.paperclip-runtime/adapter`;
|
||
|
||
const sandbox = createMockSandbox();
|
||
// The first (git-history) extract exits non-zero; every transfer/guard script
|
||
// stays green so the fail-fast loop is the only thing that can trip this test.
|
||
sandbox.process.executeCommand.mockImplementation(async (command: string) => {
|
||
if (command === "git-history-extract") {
|
||
return { exitCode: 5, result: "boom", artifacts: { stdout: "boom" } };
|
||
}
|
||
return { exitCode: 0, result: "", artifacts: { stdout: "" } };
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
await expect(
|
||
plugin.definition.onEnvironmentSyncIn?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "merged-failfast",
|
||
files: [
|
||
{ sourcePath: gitTar, targetPath: `${runtimeDir}/git-workspace-upload.tar`, kind: "file" },
|
||
{ sourcePath: overlayTar, targetPath: `${runtimeDir}/workspace-upload.tar`, kind: "file" },
|
||
],
|
||
postUploadCommands: [
|
||
{ command: "git-history-extract" },
|
||
{ command: "workspace-overlay-extract" },
|
||
{ command: "remove-deleted-paths" },
|
||
],
|
||
},
|
||
],
|
||
}),
|
||
).rejects.toThrow(/post-upload command failed \(exit 5\)/);
|
||
|
||
// Fail-fast: the overlay extract and the remove-deleted command never ran.
|
||
const ran = (cmd: string) =>
|
||
sandbox.process.executeCommand.mock.calls.some(([c]) => c === cmd);
|
||
expect(ran("git-history-extract")).toBe(true);
|
||
expect(ran("workspace-overlay-extract")).toBe(false);
|
||
expect(ran("remove-deleted-paths")).toBe(false);
|
||
});
|
||
|
||
it("issues no extra exec when an operation has no post-upload commands (backward-compat)", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const source = path.join(hostDir, "config.txt");
|
||
await fs.writeFile(source, "plain");
|
||
|
||
const baseline = createMockSandbox();
|
||
mockGet.mockResolvedValue(baseline);
|
||
await plugin.definition.onEnvironmentSyncIn?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{ operationId: "op-plain", files: [{ sourcePath: source, targetPath: `${REMOTE_DIR}/config.txt`, kind: "file" }] },
|
||
],
|
||
});
|
||
const baselineExecCount = baseline.process.executeCommand.mock.calls.length;
|
||
|
||
// Same operation, now with an (empty) postUploadCommands array — must be
|
||
// byte-identical: an absent/empty command list adds zero execs.
|
||
// Reset the process-scoped handle cache so the second operation fetches its
|
||
// own `withEmpty` handle. Both operations reuse the same providerLeaseId, so
|
||
// without this reset the cache serves the first `baseline` handle again and
|
||
// `withEmpty` records zero execs.
|
||
__resetDaytonaSandboxHandleCacheForTest();
|
||
const withEmpty = createMockSandbox();
|
||
mockGet.mockResolvedValue(withEmpty);
|
||
await plugin.definition.onEnvironmentSyncIn?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "op-plain",
|
||
files: [{ sourcePath: source, targetPath: `${REMOTE_DIR}/config.txt`, kind: "file" }],
|
||
postUploadCommands: [],
|
||
},
|
||
],
|
||
});
|
||
expect(withEmpty.process.executeCommand.mock.calls.length).toBe(baselineExecCount);
|
||
});
|
||
|
||
it("runs two concurrent inbound syncIn calls with separate reserved scratch names", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const sourceA = path.join(hostDir, "a.txt");
|
||
const sourceB = path.join(hostDir, "b.txt");
|
||
await fs.writeFile(sourceA, "alpha");
|
||
await fs.writeFile(sourceB, "beta");
|
||
|
||
const sandbox = createMockSandbox();
|
||
// Gate the upload so both concurrent calls sit inside uploadFiles together.
|
||
const gate = createTransferGate(2, async () => undefined);
|
||
sandbox.fs.uploadFiles.mockImplementation(gate.body);
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const callA = plugin.definition.onEnvironmentSyncIn?.(
|
||
syncInParams({ operationId: "in-a", sourcePath: sourceA, targetPath: `${REMOTE_DIR}/a.txt` }),
|
||
);
|
||
const callB = plugin.definition.onEnvironmentSyncIn?.(
|
||
syncInParams({ operationId: "in-b", sourcePath: sourceB, targetPath: `${REMOTE_DIR}/b.txt` }),
|
||
);
|
||
|
||
// Both calls reached the upload before either finished, so they overlap.
|
||
await gate.bothArrived;
|
||
expect(gate.peak()).toBe(2);
|
||
expect(sandbox.fs.uploadFiles).toHaveBeenCalledTimes(2);
|
||
|
||
gate.release();
|
||
await Promise.all([callA, callB]);
|
||
|
||
// Each concurrent call staged its upload under its own reserved scratch name;
|
||
// the two calls never share a temporary destination.
|
||
const destinations = sandbox.fs.uploadFiles.mock.calls.flatMap(
|
||
([uploads]) => (uploads as Array<{ destination: string }>).map((upload) => upload.destination),
|
||
);
|
||
expect(destinations).toHaveLength(2);
|
||
for (const destination of destinations) {
|
||
expect(path.posix.basename(destination)).toMatch(/^\.paperclip-upload-/);
|
||
expect(path.posix.dirname(destination)).toBe(REMOTE_DIR);
|
||
}
|
||
expect(new Set(destinations).size).toBe(destinations.length);
|
||
});
|
||
|
||
it("runs two concurrent outbound syncOut calls that both reach downloadFiles before either opens", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const targetA = path.join(hostDir, "a.txt");
|
||
const targetB = path.join(hostDir, "b.txt");
|
||
|
||
const sandbox = createMockSandbox();
|
||
// Gate the download so both concurrent calls sit inside downloadFiles
|
||
// together before either resolves.
|
||
const gate = createTransferGate(2, fulfilDownload);
|
||
sandbox.fs.downloadFiles.mockImplementation(gate.body);
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const callA = plugin.definition.onEnvironmentSyncOut?.(
|
||
syncOutParams({ operationId: "out-a", sourcePath: `${REMOTE_DIR}/a.txt`, targetPath: targetA }),
|
||
);
|
||
const callB = plugin.definition.onEnvironmentSyncOut?.(
|
||
syncOutParams({ operationId: "out-b", sourcePath: `${REMOTE_DIR}/b.txt`, targetPath: targetB }),
|
||
);
|
||
|
||
// Both calls reached the download before either gate opened, so they overlap.
|
||
await gate.bothArrived;
|
||
expect(gate.peak()).toBe(2);
|
||
expect(sandbox.fs.downloadFiles).toHaveBeenCalledTimes(2);
|
||
|
||
gate.release();
|
||
await Promise.all([callA, callB]);
|
||
|
||
// Each concurrent call read its own reserved snapshot; the two calls never
|
||
// share a download source.
|
||
const sources = sandbox.fs.downloadFiles.mock.calls.flatMap(
|
||
([requests]) => (requests as Array<{ source: string }>).map((request) => request.source),
|
||
);
|
||
expect(sources).toHaveLength(2);
|
||
for (const source of sources) {
|
||
expect(source.startsWith(`${REMOTE_DIR}/`)).toBe(true);
|
||
expect(path.posix.basename(source)).toMatch(/^\.paperclip-upload-/);
|
||
}
|
||
expect(new Set(sources).size).toBe(sources.length);
|
||
expect(await fs.readFile(targetA, "utf8")).toBe("bytes");
|
||
expect(await fs.readFile(targetB, "utf8")).toBe("bytes");
|
||
});
|
||
|
||
it("waits for one active inbound and one active outbound call before teardown releases the sandbox", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const inboundSource = path.join(hostDir, "in.txt");
|
||
const outboundTarget = path.join(hostDir, "out.txt");
|
||
await fs.writeFile(inboundSource, "inbound");
|
||
|
||
const sandbox = createMockSandbox({ id: "sandbox-123" });
|
||
// Hold the inbound upload and the outbound download open at the same time, so
|
||
// the shared lease has two active sync calls when teardown starts.
|
||
let uploadArrived!: () => void;
|
||
const uploadStarted = new Promise<void>((resolve) => { uploadArrived = resolve; });
|
||
let releaseUpload!: () => void;
|
||
sandbox.fs.uploadFiles.mockImplementation(async () => {
|
||
await new Promise<void>((resolve) => {
|
||
releaseUpload = resolve;
|
||
uploadArrived();
|
||
});
|
||
});
|
||
let downloadArrived!: () => void;
|
||
const downloadStarted = new Promise<void>((resolve) => { downloadArrived = resolve; });
|
||
let releaseDownload!: () => void;
|
||
sandbox.fs.downloadFiles.mockImplementation(async (requests: Array<{ source: string; destination: string }>) => {
|
||
await new Promise<void>((resolve) => {
|
||
releaseDownload = resolve;
|
||
downloadArrived();
|
||
});
|
||
return Promise.all(
|
||
requests.map(async (request) => {
|
||
await fs.writeFile(request.destination, "bytes");
|
||
return { source: request.source, result: request.destination };
|
||
}),
|
||
);
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const inboundCall = plugin.definition.onEnvironmentSyncIn?.(
|
||
syncInParams({ operationId: "in-active", sourcePath: inboundSource, targetPath: `${REMOTE_DIR}/in.txt` }),
|
||
);
|
||
const outboundCall = plugin.definition.onEnvironmentSyncOut?.(
|
||
syncOutParams({ operationId: "out-active", sourcePath: `${REMOTE_DIR}/out.txt`, targetPath: outboundTarget }),
|
||
);
|
||
// Wait for the actual transfers. One event-loop tick does not guarantee
|
||
// that the inbound filesystem reads have finished on a busy runner.
|
||
await Promise.all([uploadStarted, downloadStarted]);
|
||
|
||
const destroyCall = plugin.definition.onEnvironmentDestroyLease?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
providerLeaseId: "sandbox-123",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
});
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
// Two active sync calls block teardown, so it must not delete the sandbox yet.
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
|
||
// Release only the inbound call. One outbound call is still active, so
|
||
// teardown must keep waiting.
|
||
releaseUpload();
|
||
await inboundCall;
|
||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||
expect(sandbox.delete).not.toHaveBeenCalled();
|
||
|
||
// Release the outbound call. No sync call is active now, so teardown deletes.
|
||
releaseDownload();
|
||
await Promise.all([outboundCall, destroyCall]);
|
||
|
||
expect(sandbox.fs.uploadFiles).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.fs.downloadFiles).toHaveBeenCalledTimes(1);
|
||
expect(sandbox.delete).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it("opens a transfer span with the guard round-trip count around the bulk file download", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const target = path.join(hostDir, "result.txt");
|
||
const sandbox = createMockSandbox();
|
||
sandbox.fs.downloadFiles.mockImplementation(async (requests: Array<{ source: string; destination: string }>) => {
|
||
return Promise.all(
|
||
requests.map(async (request) => {
|
||
await fs.writeFile(request.destination, "bytes");
|
||
return { source: request.source, result: request.destination };
|
||
}),
|
||
);
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const { tracer, spans } = createRecordingPluginTracer();
|
||
const restore = __setDaytonaPluginContextForTest({ tracer } as unknown as PluginContext);
|
||
try {
|
||
await plugin.definition.onEnvironmentSyncOut?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-out",
|
||
files: [{ sourcePath: `${REMOTE_DIR}/out/result.txt`, targetPath: target, kind: "file" }],
|
||
},
|
||
],
|
||
});
|
||
} finally {
|
||
restore();
|
||
}
|
||
|
||
const transfer = spans.find((span) => span.name === "transfer");
|
||
expect(transfer).toBeDefined();
|
||
expect(transfer!.ended).toBe(true);
|
||
// One serial guard round trip before the transfer: the validate-and-snapshot.
|
||
expect(transfer!.attributes["paperclip.sandbox.startup.transfer.guard.count"]).toBe(1);
|
||
expect(transfer!.attributes["paperclip.sandbox.startup.provider"]).toBe("daytona");
|
||
expect(typeof transfer!.attributes["paperclip.sandbox.startup.transfer.wall_ms"]).toBe("number");
|
||
});
|
||
|
||
it("opens a transfer span around a directory-mapping download with the guard round-trip count", async () => {
|
||
const hostDir = await makeHostDir();
|
||
const targetDir = path.join(hostDir, "assets");
|
||
const sandbox = createMockSandbox();
|
||
sandbox.fs.downloadFiles.mockImplementation(async (requests: Array<{ source: string; destination: string }>) => {
|
||
// Write a valid empty tar (1024-byte zero EOF marker) so host-side extract
|
||
// is a clean no-op.
|
||
await Promise.all(requests.map((request) => fs.writeFile(request.destination, Buffer.alloc(1024))));
|
||
return requests.map((request) => ({ source: request.source, result: request.destination }));
|
||
});
|
||
mockGet.mockResolvedValue(sandbox);
|
||
|
||
const { tracer, spans } = createRecordingPluginTracer();
|
||
const restore = __setDaytonaPluginContextForTest({ tracer } as unknown as PluginContext);
|
||
try {
|
||
await plugin.definition.onEnvironmentSyncOut?.({
|
||
driverKey: "daytona",
|
||
companyId: "company-1",
|
||
environmentId: "env-1",
|
||
config: { timeoutMs: 300000, reuseLease: false },
|
||
lease: syncLease(),
|
||
operations: [
|
||
{
|
||
operationId: "sync-op-out-dir",
|
||
files: [{ sourcePath: `${REMOTE_DIR}/out/assets`, targetPath: targetDir, kind: "directory" }],
|
||
},
|
||
],
|
||
});
|
||
} finally {
|
||
restore();
|
||
}
|
||
|
||
const transfer = spans.find((span) => span.name === "transfer");
|
||
expect(transfer).toBeDefined();
|
||
expect(transfer!.ended).toBe(true);
|
||
// Two serial guard round trips before the transfer: confinement + in-sandbox
|
||
// tar.
|
||
expect(transfer!.attributes["paperclip.sandbox.startup.transfer.guard.count"]).toBe(2);
|
||
expect(typeof transfer!.attributes["paperclip.sandbox.startup.transfer.wall_ms"]).toBe("number");
|
||
});
|
||
});
|
||
|
||
describe("daytona manifest memory config", () => {
|
||
const memorySchema = (
|
||
manifest.environmentDrivers?.[0]?.configSchema as {
|
||
properties?: Record<string, { type?: string; enum?: unknown[] }>;
|
||
required?: string[];
|
||
}
|
||
);
|
||
|
||
it("offers memory as a fixed dropdown of supported sandbox sizes", () => {
|
||
expect(memorySchema.properties?.memory?.enum).toEqual([1, 2, 4, 8]);
|
||
});
|
||
|
||
it("excludes 0 — an invalid Daytona memory configuration", () => {
|
||
expect(memorySchema.properties?.memory?.enum).not.toContain(0);
|
||
});
|
||
|
||
it("keeps memory optional so the blank/default selection stays valid", () => {
|
||
expect(memorySchema.required ?? []).not.toContain("memory");
|
||
});
|
||
});
|
||
|
||
describe("daytona manifest form defaults", () => {
|
||
const configSchema = (
|
||
manifest.environmentDrivers?.[0]?.configSchema as {
|
||
properties?: Record<
|
||
string,
|
||
{ format?: string; enum?: unknown[]; minimum?: number; default?: unknown }
|
||
>;
|
||
}
|
||
);
|
||
const properties = configSchema.properties ?? {};
|
||
|
||
it("pre-fills sizing and image fields for the environment form", () => {
|
||
expect(properties.cpu?.default).toBe(4);
|
||
expect(properties.memory?.default).toBe(4);
|
||
expect(properties.disk?.default).toBe(10);
|
||
expect(properties.image?.default).toBe("daytonaio/sandbox:0.8.0");
|
||
});
|
||
|
||
it("keeps each default within its own schema constraints", () => {
|
||
expect(properties.memory?.enum).toContain(properties.memory?.default);
|
||
expect(properties.cpu?.default as number).toBeGreaterThanOrEqual(
|
||
properties.cpu?.minimum ?? 1,
|
||
);
|
||
expect(properties.disk?.default as number).toBeGreaterThanOrEqual(
|
||
properties.disk?.minimum ?? 1,
|
||
);
|
||
});
|
||
|
||
it("declares no default on secret-ref fields, which would be persisted as a company secret", () => {
|
||
for (const prop of Object.values(properties)) {
|
||
if (prop.format === "secret-ref") {
|
||
expect(prop.default).toBeUndefined();
|
||
}
|
||
}
|
||
});
|
||
});
|
||
|
||
describe("parseTarVerboseListingLine", () => {
|
||
it("parses GNU tar listing lines (file, dir, symlink, hardlink, numeric owner)", () => {
|
||
expect(parseTarVerboseListingLine("-rw-r--r-- daytona/daytona 7560 2026-08-11 21:43 AGENTS.md")).toEqual({
|
||
typeFlag: "-",
|
||
rest: "AGENTS.md",
|
||
});
|
||
expect(parseTarVerboseListingLine("drwxr-xr-x daytona/daytona 0 2026-08-11 21:43 nested/")).toEqual({
|
||
typeFlag: "d",
|
||
rest: "nested/",
|
||
});
|
||
expect(
|
||
parseTarVerboseListingLine("lrwxrwxrwx daytona/daytona 0 2026-08-11 21:43 shortcut -> nested/data.txt"),
|
||
).toEqual({ typeFlag: "l", rest: "shortcut -> nested/data.txt" });
|
||
expect(
|
||
parseTarVerboseListingLine("hrw-r--r-- daytona/daytona 0 2026-08-11 21:43 copy.txt link to data.txt"),
|
||
).toEqual({ typeFlag: "h", rest: "copy.txt link to data.txt" });
|
||
expect(parseTarVerboseListingLine("-rw-r--r-- 0/0 12 2026-08-11 21:43 root-owned.txt")).toEqual({
|
||
typeFlag: "-",
|
||
rest: "root-owned.txt",
|
||
});
|
||
});
|
||
|
||
it("parses bsdtar (macOS) listing lines, including year-form dates", () => {
|
||
expect(parseTarVerboseListingLine("-rw-r--r-- 0 daytona daytona 7560 Aug 11 21:43 AGENTS.md")).toEqual({
|
||
typeFlag: "-",
|
||
rest: "AGENTS.md",
|
||
});
|
||
expect(parseTarVerboseListingLine("drwxr-xr-x 0 daytona daytona 0 Aug 11 21:43 nested/")).toEqual({
|
||
typeFlag: "d",
|
||
rest: "nested/",
|
||
});
|
||
expect(
|
||
parseTarVerboseListingLine("lrwxr-xr-x 0 daytona daytona 0 Aug 11 21:43 shortcut -> nested/data.txt"),
|
||
).toEqual({ typeFlag: "l", rest: "shortcut -> nested/data.txt" });
|
||
expect(
|
||
parseTarVerboseListingLine("hrw-r--r-- 0 daytona daytona 0 Aug 11 21:43 copy.txt link to data.txt"),
|
||
).toEqual({ typeFlag: "h", rest: "copy.txt link to data.txt" });
|
||
expect(parseTarVerboseListingLine("-rw-r--r-- 0 daytona daytona 7560 Aug 11 2025 old.txt")).toEqual({
|
||
typeFlag: "-",
|
||
rest: "old.txt",
|
||
});
|
||
});
|
||
|
||
it("keeps the true member name for bsdtar lines with numeric uid/gid, so traversal stays visible", () => {
|
||
// With unresolvable ids bsdtar prints bare numbers; a looser GNU-first parse
|
||
// would read this shape shifted by one field and report the member name as
|
||
// "21:43 ../escape.txt", hiding the leading "../" from the traversal check.
|
||
expect(parseTarVerboseListingLine("-rw-r--r-- 0 1001 1001 7560 Aug 11 21:43 ../escape.txt")).toEqual({
|
||
typeFlag: "-",
|
||
rest: "../escape.txt",
|
||
});
|
||
});
|
||
|
||
it("returns null (fail closed) for lines matching neither dialect", () => {
|
||
expect(parseTarVerboseListingLine("not a tar listing line")).toBeNull();
|
||
expect(parseTarVerboseListingLine("tar: Error is not recoverable: exiting now")).toBeNull();
|
||
// Device nodes carry "major,minor" instead of a byte count in both dialects.
|
||
expect(parseTarVerboseListingLine("crw-rw-rw- root/root 1,3 2026-08-11 21:43 dev/null")).toBeNull();
|
||
expect(parseTarVerboseListingLine("crw-rw-rw- 0 root wheel 1,3 Aug 11 21:43 dev/null")).toBeNull();
|
||
});
|
||
});
|
||
|
||
describe("splitLinkEntryOnce", () => {
|
||
it("splits a clean single-delimiter link field", () => {
|
||
expect(splitLinkEntryOnce("shortcut -> nested/data.txt", " -> ")).toEqual({
|
||
name: "shortcut",
|
||
target: "nested/data.txt",
|
||
});
|
||
expect(splitLinkEntryOnce("copy.txt link to data.txt", " link to ")).toEqual({
|
||
name: "copy.txt",
|
||
target: "data.txt",
|
||
});
|
||
});
|
||
|
||
it("returns null (fail closed) when the delimiter is absent or appears more than once", () => {
|
||
expect(splitLinkEntryOnce("no delimiter here", " -> ")).toBeNull();
|
||
// A link name or target embedding the delimiter makes the split point
|
||
// unresolvable; either split choice can hide an escaping target.
|
||
expect(splitLinkEntryOnce("evil -> decoy -> ../../outside.txt", " -> ")).toBeNull();
|
||
expect(splitLinkEntryOnce("a link to b link to ../../outside.txt", " link to ")).toBeNull();
|
||
});
|
||
});
|