fix: allow normal Daytona command status response latency (#16480)

Allow a normal completed Daytona status response up to 500ms within the unchanged caller deadline. Keep final separated logs, stream fallback and single dispatch.

Verified: 396 provider cases, full repository and standalone provider typecheck/build, matched real-Daytona component controls, and exact-head full CI/E2E, security and Apex 5/5. Whole-agent latency target remains open.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-10-15 14:42:36 -05:00
1 parent 2aca41464b
commit 69dded3230
3 files changed
+19 -3

No files matched your search

@@ -30,7 +30,7 @@ Notes:
- Each cold create uses a unique provider name and ownership labels. If creation fails after Daytona has allocated a sandbox, the driver looks up that exact name, checks every ownership label, and waits for deletion. Failed, missing, or timed-out lookups and failed deletions report unconfirmed cleanup with the provider name; they never return a usable lease. A missing name lookup after an uncertain create is not proof that a delayed provider request cannot create a resource.
- For plugin-backed sandbox lease acquisition, unresolved creation cleanup crosses the worker RPC as a validated ownership envelope. The host records `pending_cleanup` before retrying deletion. If only a creation name is known, the provider first returns an ownership-verified sandbox ID without deleting it. The host saves that observation before authorizing deletion. Its existing cleanup sweep and durable spool preserve retries across controller restarts and environment deletion. A missing observed ID confirms cleanup after a lost deletion reply or database update; a name that has never been observed remains unresolved. Provider exceptions and resolved credentials are excluded from that envelope. This requires the matching host and plugin SDK update; probe and custom-image interactive-setup calls still report unconfirmed immediate cleanup without this lease-recovery path.
- Reusable leases map to Daytona stop/start semantics. Non-reusable leases are deleted on release. A provider-resolved `target` does not change the identity of an existing sandbox. Release closes the same scoped lease that a later sentinel-verified resume reopens.
- A command that has already exited returns its final separated log snapshot without opening a follow socket. The initial status check is bounded to 250ms and the caller's command deadline. An unfinished command, failed status read, or slow status read keeps the existing streaming and recovery path; the command is dispatched only once.
- A command that has already exited returns its final separated log snapshot without opening a follow socket. The initial status check is bounded to 500ms and the caller's command deadline. An unfinished command, failed status read, or slow status read keeps the existing streaming and recovery path; the command is dispatched only once.
- A session log socket can close while its command still runs. The driver requires a recorded command exit before returning a completed execution. It reconnects the log stream once, then polls status and log snapshots at most once per second. Replayed output is removed by byte offset, new output continues to reach the host, and the command is never dispatched again. A socket that delivers no output for 15 seconds switches directly to polling, even if the socket stays open or its initial connection never completes. Late callbacks from that socket are ignored. This recovers pending permission requests and cancellation results from saved output without replaying the command. Healthy initial and reconnected streams retain their existing lifetime under the caller’s RPC/run guard. After a clean close or an idle socket, each status or snapshot read gets a fresh operation timeout; successful recovery polling does not impose a new command lifetime. When both stream attempts fail without a clean close, fallback retains its original timeout budget starting after the stream attempts.
- The SDK returns full log snapshots, so fallback bandwidth grows with retained output; it has no paged snapshot or stream cancellation API. A recovery-read timeout keeps partial output and explicitly reports whether command exit remains unconfirmed; it does not prove the remote command stopped. Provider/session teardown remains responsible for closing an outstanding socket. A final snapshot reconciles bytes missed by the stream before returning a recorded exit.
- A sandbox record can survive the loss of its underlying container. Resume treats it as expired only when a fresh provider read confirms the exact missing-container error for that sandbox and marks it unrecoverable. Unknown errors and failed confirmation reads preserve the lease. The host still requires a verified native-runner backup before replacement.
@@ -1808,7 +1808,7 @@ async function executeOneShot(
// Fallback reads full log snapshots as well as status. Limit the heavier
// polling path while keeping output available to interactive commands.
const SESSION_LOG_POLL_INTERVAL_MS = 1000;
const SESSION_INITIAL_STATUS_TIMEOUT_MS = 250;
const SESSION_INITIAL_STATUS_TIMEOUT_MS = 500;
// A socket can stay open after it stops delivering output. Switch to saved
// logs after this much silence; silence never proves that the command exited.
const SESSION_LOG_STREAM_IDLE_MS = 15_000;
@@ -62,6 +62,20 @@ describe("Daytona completed session commands", () => {
expect(process.getSessionCommandLogs.mock.calls.filter(call => call[2])).toHaveLength(1);
});
it("retains a completed status response that takes 300ms without opening a follow socket", async () => {
const { process, execute } = fixture();
process.getSessionCommand.mockImplementationOnce(() => new Promise(resolve => setTimeout(() => resolve({ exitCode: 0 }), 300)));
let settled = false;
const pending = execute().then(result => { settled = true; return result; });
await vi.advanceTimersByTimeAsync(600);
const settledWithinObservation = settled;
if (!settled) await vi.advanceTimersByTimeAsync(1_500);
expect(await pending).toMatchObject({ exitCode: 0, timedOut: false, stdout: "complete stdout\n", stderr: "complete stderr\n" });
expect(settledWithinObservation).toBe(true);
expect(process.getSessionCommandLogs.mock.calls.filter(call => call[2])).toHaveLength(0);
expect(process.executeSessionCommand).toHaveBeenCalledTimes(1);
});
it("bounds an unavailable initial status read and retains the working stream", async () => {
const { process, execute } = fixture();
process.getSessionCommand.mockImplementationOnce(() => new Promise(() => {}));
@@ -70,7 +84,9 @@ describe("Daytona completed session commands", () => {
return { stdout: "complete stdout\n", stderr: "complete stderr\n" };
});
const pending = execute();
await vi.advanceTimersByTimeAsync(300);
await vi.advanceTimersByTimeAsync(499);
expect(process.getSessionCommandLogs).not.toHaveBeenCalled();
await vi.advanceTimersByTimeAsync(2);
expect(await pending).toMatchObject({ exitCode: 0, timedOut: false, stdout: "complete stdout\n", stderr: "complete stderr\n" });
expect(process.getSessionCommandLogs.mock.calls.filter(call => call[2])).toHaveLength(1);
});