mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-11 05:31:46 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandbox providers carry agent work through controlled execution channels > - The Daytona callback bridge uses a bespoke line-framed protocol over its duplex channel > - The bespoke protocol adds framing work and does not use the Node transport that already supports multiplexed streams > - This pull request carries raw bytes across the channel, adds a Node HTTP/2 bridge, and selects it for Daytona > - The benefit is one authenticated, multiplexed callback session with queue_v1 as the bounded fallback ## Linked Issues or Issue Description **Subsystem affected** The packages/plugins Daytona provider and the shared duplex execution path. **Problem or motivation** The Daytona callback bridge uses a bespoke line-framed protocol over the provider duplex channel. This adds protocol work and limits stream handling. **Proposed solution** Carry raw bytes through the cross-layer channel. Add an authenticated Node HTTP/2 host server and sandbox client gateway. Select http2_v1 for Daytona and retain queue_v1 as the fallback. **Alternatives considered** Keep the current duplex_v1 protocol. This keeps the bespoke framing path and does not provide one HTTP/2 session for callback streams. **Roadmap alignment** ROADMAP.md lists Daytona under cloud and sandbox agents. This change improves the shipped Daytona provider path. **Additional context** The branch adds no dependency. Node 24 provides the http2 module. The host token check and canonical path parser remain the single dispatch path. ## What Changed - Carry raw Uint8Array chunks through the adapter, plugin, worker, runtime, and Daytona layers. - Encode bytes as base64 only across the JSON-RPC hop, because JSON has no binary type. - Add the bounded host HTTP/2 server and the in-sandbox HTTP/2 client gateway. - Authenticate every stream with the per-run bridge token before route work. - Parse the path once and reuse the canonical result for route and forwarding work. - Select http2_v1 for Daytona and fall back once to queue_v1 when the client preface is absent. - Add transport, session, stream, and fallback telemetry. - Mark HTTP/2 as the preferred transport and queue_v1 as the soft-deprecated fallback. ## Verification - `npx vitest run packages/adapter-utils/src` — 990 passed and 4 skipped. - `npx vitest run server/src/__tests__/plugin-worker-manager-duplex.test.ts` — 32 passed. - `npx vitest run --config packages/plugins/sandbox-providers/daytona/vitest.config.ts` — 220 passed and 6 skipped. - `npx tsc --noEmit` in `packages/adapter-utils`, `packages/shared`, `packages/plugins/sdk`, and `server` — clean. - No `package.json` or `pnpm-lock.yaml` file changed. - The live Daytona test skips when `DAYTONA_API_KEY` is absent. - The root `npx tsc --noEmit` command has a pre-existing missing `packages/adapters/droid-local` reference on this branch and on `master`. ## Risks - The transport change affects several duplex layers and could expose byte-boundary errors. - A missing HTTP/2 client preface falls back once to queue_v1 and records `preface_missing`. - The host token check and canonical path parser must remain on the shared dispatch path. - The live Daytona test needs `DAYTONA_API_KEY` and does not run in this agent sandbox. ## Model Used OpenAI GPT-5, tool-enabled coding agent with repository inspection, GitHub CLI, and shell execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
621 lines
30 KiB
TypeScript
621 lines
30 KiB
TypeScript
import type { Db } from "@paperclipai/db";
|
|
import type { Environment, EnvironmentLease } from "@paperclipai/shared";
|
|
import { adapterSupportsRemoteManagedEnvironments } from "@paperclipai/shared";
|
|
import {
|
|
adapterExecutionTargetToRemoteSpec,
|
|
type AdapterExecutionTarget,
|
|
} from "@paperclipai/adapter-utils/execution-target";
|
|
import type { DuplexObservabilityRecorder } from "@paperclipai/adapter-utils/duplex-observability";
|
|
import type { DuplexAggregateByteLedger } from "@paperclipai/adapter-utils/duplex-aggregate-byte-ledger";
|
|
import {
|
|
clampSpanLabel,
|
|
getActiveStepContext,
|
|
normalizeProviderFamily,
|
|
SANDBOX_STARTUP_OUTCOME,
|
|
SANDBOX_STARTUP_SPAN_ATTRS,
|
|
} from "@paperclipai/adapter-utils/acpx-engine/startup-timing";
|
|
import { parseObject } from "../adapters/utils.js";
|
|
import { getStartupTracer } from "../instrumentation.js";
|
|
import { resolveEnvironmentDriverConfigForRuntime } from "./environment-config.js";
|
|
import type { EnvironmentRuntimeService } from "./environment-runtime.js";
|
|
import { getEnvironmentDriverTraits } from "./environment-driver-traits.js";
|
|
|
|
export const DEFAULT_SANDBOX_REMOTE_CWD = "/tmp";
|
|
|
|
/** The minimal span surface the provider-exec seam calls. A real injected OTel
|
|
* span satisfies it; the no-op tracer's span satisfies it too. */
|
|
type ExecSpan = {
|
|
setAttribute(key: string, value: string | number | boolean): void;
|
|
setStatus(status: { code: number; message?: string }): void;
|
|
end(): void;
|
|
};
|
|
|
|
/**
|
|
* The value of `SpanStatusCode.ERROR` in `@opentelemetry/api`. The server injects
|
|
* a real OTel span, but this module stays OTel-free, so it uses the numeric value
|
|
* directly. A failed exec span sets this status, so a trace UI counts and filters
|
|
* the failure through the native span status, not only the `outcome` attribute.
|
|
*/
|
|
const SPAN_STATUS_CODE_ERROR = 2;
|
|
|
|
/** The minimal tracer surface the provider-exec seam calls. `getStartupTracer`
|
|
* returns a real or no-op implementation that satisfies it. The optional third
|
|
* argument is the explicit parent-context token: the seam passes the active
|
|
* step context, so the exec span parents to its step span. */
|
|
type ExecTracer = {
|
|
startSpan(name: string, options?: unknown, context?: unknown): ExecSpan;
|
|
};
|
|
|
|
/**
|
|
* Set a numeric span attribute only when the value is a finite number. A value
|
|
* that is absent, `NaN`, or `Infinity` yields no attribute — never a misleading
|
|
* `0`. This mirrors the host counter guard below and the `adapter-utils`
|
|
* startup-step guard.
|
|
*/
|
|
function setFiniteNumberAttr(span: ExecSpan, key: string, value: unknown): void {
|
|
if (typeof value === "number" && Number.isFinite(value)) {
|
|
span.setAttribute(key, value);
|
|
}
|
|
}
|
|
|
|
/** Read a free-form metadata value as a finite number, or `undefined`. The
|
|
* provider durations ride the exec result's untyped `metadata`, so a provider
|
|
* that omits or mistypes one yields no attribute — never a misleading `0`. */
|
|
function toFiniteNumber(value: unknown): number | undefined {
|
|
return typeof value === "number" && Number.isFinite(value) ? value : undefined;
|
|
}
|
|
|
|
/** Read a free-form metadata value as a boolean, or `undefined`. The provider
|
|
* cache-hit flag rides the exec result's untyped `metadata`, so a provider that
|
|
* omits or mistypes it yields no attribute — never a misleading `false`. */
|
|
function toBoolean(value: unknown): boolean | undefined {
|
|
return typeof value === "boolean" ? value : undefined;
|
|
}
|
|
|
|
/**
|
|
* Compute the tail of `final` that the provider did NOT already stream.
|
|
*
|
|
* The provider streams output chunks in order. Those chunks form `delivered`.
|
|
* The final result is `final`. In the normal path `final` continues
|
|
* `delivered`, so the tail is `final` past the delivered length.
|
|
*
|
|
* A provider can stream a prefix and then fall back to a poll that returns a
|
|
* different buffer. When `final` does not start with `delivered`, a length
|
|
* slice would drop unrelated leading output or cut a chunk mid-text, so the
|
|
* durable log would hold truncated or corrupt output. In that case this
|
|
* function returns the whole `final` instead. That can repeat the streamed
|
|
* prefix in the log, but the complete final output always reaches the log.
|
|
* Repetition is safer than a silent loss of output.
|
|
*/
|
|
function undeliveredSuffix(delivered: string, final: string): string {
|
|
if (!final) return "";
|
|
if (delivered.length === 0) return final;
|
|
if (final.startsWith(delivered)) return final.slice(delivered.length);
|
|
return final;
|
|
}
|
|
|
|
/**
|
|
* The closed input for one `sandbox.exec` span. The seam builds it from the
|
|
* exec result and the active step context. Every field is already bounded or
|
|
* numeric; the raw command clamps inside the helper below.
|
|
*/
|
|
interface SandboxExecSpanInput {
|
|
/** The low-cardinality provider family (already through `normalizeProviderFamily`). */
|
|
provider: string;
|
|
/** The raw `argv[0]`. The helper clamps it; the raw value never rides the span. */
|
|
command: string;
|
|
/** The numeric process exit code, or `null`. */
|
|
exitCode: number | null;
|
|
/** The host-measured wall time of the execution. */
|
|
wallMs: number;
|
|
/** The provider handle-fetch wait before the execution ran. */
|
|
waitBeforeMs: number | undefined;
|
|
/** The in-sandbox run time of the execution. */
|
|
sandboxMs: number | undefined;
|
|
/** Whether the execution sits on the startup critical path. */
|
|
criticalPath: boolean;
|
|
/** Whether the provider served the sandbox handle from its warm cache. */
|
|
cacheHit: boolean | undefined;
|
|
}
|
|
|
|
/**
|
|
* Assemble every `sandbox.exec` span attribute in one place. This is the single
|
|
* producer-side boundary for the exec span: it sets only the closed
|
|
* `paperclip.sandbox.startup.exec.*` allowlist. The command rides only as a
|
|
* clamped label, so a full command line, an argument, a path, an environment
|
|
* value, or any standard-stream text can never ride the span. A non-finite
|
|
* numeric input yields no attribute (fail open — never `NaN`, never a
|
|
* misleading `0`).
|
|
*/
|
|
function setSandboxExecSpanAttributes(span: ExecSpan, input: SandboxExecSpanInput): void {
|
|
const A = SANDBOX_STARTUP_SPAN_ATTRS;
|
|
span.setAttribute(A.provider, input.provider);
|
|
const command = clampSpanLabel("command", input.command);
|
|
if (command !== undefined) span.setAttribute(A.execCommand, command);
|
|
if (typeof input.exitCode === "number" && Number.isFinite(input.exitCode)) {
|
|
span.setAttribute(A.execExitCode, input.exitCode);
|
|
}
|
|
setFiniteNumberAttr(span, A.execWallMs, input.wallMs);
|
|
setFiniteNumberAttr(span, A.execWaitBeforeMs, input.waitBeforeMs);
|
|
setFiniteNumberAttr(span, A.execSandboxMs, input.sandboxMs);
|
|
// The transport time the host adds around the provider work: wall minus the
|
|
// handle-fetch wait minus the in-sandbox run. Set it only when both parts are
|
|
// present, so a provider that reports no durations yields no derived value.
|
|
if (input.waitBeforeMs !== undefined && input.sandboxMs !== undefined) {
|
|
setFiniteNumberAttr(span, A.execNetworkMs, input.wallMs - input.waitBeforeMs - input.sandboxMs);
|
|
}
|
|
span.setAttribute(A.execCriticalPath, input.criticalPath);
|
|
// The explicit provider cache-hit flag, from `result.metadata.cacheHit`. The
|
|
// plugin decides it at the handle lookup, so the span no longer infers a
|
|
// cache hit from `wait_before_ms == 0`. A provider that omits it yields no
|
|
// attribute.
|
|
if (typeof input.cacheHit === "boolean") {
|
|
span.setAttribute(A.execCacheHit, input.cacheHit);
|
|
}
|
|
const failed = input.exitCode !== 0;
|
|
span.setAttribute(
|
|
A.outcome,
|
|
failed ? SANDBOX_STARTUP_OUTCOME.failed : SANDBOX_STARTUP_OUTCOME.ok,
|
|
);
|
|
// A non-zero exit code is a failed execution, so set the native span status to
|
|
// ERROR too. The success path leaves the status unset, so a successful exec
|
|
// keeps the default OTel status. A `null` exit code counts as failed here.
|
|
if (failed) span.setStatus({ code: SPAN_STATUS_CODE_ERROR });
|
|
}
|
|
|
|
/**
|
|
* Record a failed `sandbox.exec` span when the provider execution throws. There
|
|
* is no exec result, so only the bounded provider family, the clamped command
|
|
* label, the measured wall time, the critical-path flag, and the `failed`
|
|
* outcome ride the span. The raw command never rides the span. A thrown
|
|
* execution now still produces a span, instead of no span at all.
|
|
*/
|
|
function setSandboxExecSpanFailure(
|
|
span: ExecSpan,
|
|
input: { provider: string; command: string; wallMs: number; criticalPath: boolean },
|
|
): void {
|
|
const A = SANDBOX_STARTUP_SPAN_ATTRS;
|
|
span.setAttribute(A.provider, input.provider);
|
|
const command = clampSpanLabel("command", input.command);
|
|
if (command !== undefined) span.setAttribute(A.execCommand, command);
|
|
setFiniteNumberAttr(span, A.execWallMs, input.wallMs);
|
|
span.setAttribute(A.execCriticalPath, input.criticalPath);
|
|
span.setAttribute(A.outcome, SANDBOX_STARTUP_OUTCOME.failed);
|
|
// A thrown execution is a failed execution, so set the native span status to
|
|
// ERROR too, not only the `outcome` attribute.
|
|
span.setStatus({ code: SPAN_STATUS_CODE_ERROR });
|
|
}
|
|
|
|
export async function resolveEnvironmentExecutionTarget(input: {
|
|
db: Db;
|
|
companyId: string;
|
|
adapterType: string;
|
|
environment: {
|
|
id?: string;
|
|
driver: string;
|
|
config: Record<string, unknown> | null;
|
|
};
|
|
leaseId?: string | null;
|
|
leaseMetadata: Record<string, unknown> | null;
|
|
lease?: EnvironmentLease | null;
|
|
environmentRuntime?: EnvironmentRuntimeService | null;
|
|
// The startup tracer for the provider-exec span. Defaults to the endpoint-
|
|
// gated server tracer, which is a no-op when tracing is off. Tests inject a
|
|
// recording tracer.
|
|
tracer?: ExecTracer;
|
|
// The host duplex observability recorder. The seam stamps it onto the sandbox
|
|
// target next to the runner, so the live object stays on the host and never
|
|
// enters the sandbox environment. Absent keeps the safe no-op default in the
|
|
// bridge, so the surface stays inert until the host injects a real recorder.
|
|
duplexObservabilityRecorder?: DuplexObservabilityRecorder | null;
|
|
// The process-owned aggregate byte ledger. The seam stamps it onto the sandbox
|
|
// target next to the runner, so the live object stays on the host and never
|
|
// enters the sandbox environment. Absent keeps the bridge inert for this seam.
|
|
duplexAggregateByteLedger?: DuplexAggregateByteLedger | null;
|
|
}): Promise<AdapterExecutionTarget | null> {
|
|
if (input.environment.driver === "local") {
|
|
return {
|
|
kind: "local",
|
|
environmentId: input.environment.id ?? null,
|
|
leaseId: input.leaseId ?? null,
|
|
};
|
|
}
|
|
|
|
if (input.environment.driver === "sandbox") {
|
|
// Keep this gate in lockstep with the shared capability metadata that the
|
|
// environment selector and capabilities API expose; a drift here lets the
|
|
// UI offer environments the runtime then refuses.
|
|
if (!adapterSupportsRemoteManagedEnvironments(input.adapterType)) {
|
|
return null;
|
|
}
|
|
|
|
const parsed = await resolveEnvironmentDriverConfigForRuntime(input.db, input.companyId, {
|
|
id: input.environment.id,
|
|
driver: input.environment.driver as "sandbox",
|
|
config: parseObject(input.environment.config),
|
|
});
|
|
if (parsed.driver !== "sandbox") {
|
|
return null;
|
|
}
|
|
|
|
const remoteCwd =
|
|
typeof input.leaseMetadata?.remoteCwd === "string" && input.leaseMetadata.remoteCwd.trim().length > 0
|
|
? input.leaseMetadata.remoteCwd.trim()
|
|
: DEFAULT_SANDBOX_REMOTE_CWD;
|
|
const timeoutMs = "timeoutMs" in parsed.config ? parsed.config.timeoutMs : null;
|
|
const shellCommand =
|
|
input.leaseMetadata?.shellCommand === "bash" || input.leaseMetadata?.shellCommand === "sh"
|
|
? input.leaseMetadata.shellCommand
|
|
: null;
|
|
|
|
// The low-cardinality public provider family. A plugin-backed / operator-
|
|
// defined key maps to `plugin`, so a raw unbounded key never rides a span.
|
|
const providerFamily = normalizeProviderFamily(parsed.config.provider);
|
|
// The endpoint-gated startup tracer (no-op when tracing is off). Tests inject
|
|
// a recording tracer.
|
|
const tracer = input.tracer ?? getStartupTracer();
|
|
|
|
// Resolve the read-only effective capability snapshot for this lease
|
|
// through the general resolver. Freeze it so a consumer reads it but
|
|
// never changes it. Track a resolution error apart from a genuinely
|
|
// absent snapshot: a rejected resolution must not read as an open grant.
|
|
//
|
|
// Gate the call on the `hasLeaseCapabilityModel` trait, not on whether the
|
|
// service exposes the method: the general resolver's `resolveCapabilities`
|
|
// never returns `null` for a registered driver, and it resolves every
|
|
// capability `false` for a driver with no lease capability model (see
|
|
// `ENVIRONMENT_DRIVER_CAPABILITY_SUPPORT`). Calling it unconditionally
|
|
// would turn "no snapshot" into "every capability denied" for a driver
|
|
// this branch does not otherwise gate on. Reading the trait keeps that
|
|
// behavior change out of this phase: only the `sandbox` driver has a
|
|
// lease capability model today, and this branch only runs for `sandbox`.
|
|
let effectiveCapabilities: Awaited<
|
|
ReturnType<NonNullable<EnvironmentRuntimeService["resolveCapabilities"]>>
|
|
> | null = null;
|
|
let capabilityResolutionFailed = false;
|
|
const driverHasLeaseCapabilityModel =
|
|
getEnvironmentDriverTraits(input.environment.driver)?.hasLeaseCapabilityModel ?? false;
|
|
if (driverHasLeaseCapabilityModel && input.environmentRuntime?.resolveCapabilities && input.lease) {
|
|
try {
|
|
effectiveCapabilities = await input.environmentRuntime.resolveCapabilities({
|
|
environment: input.environment as Environment,
|
|
lease: input.lease,
|
|
});
|
|
} catch {
|
|
// The runtime could not resolve the snapshot. Fail closed for the
|
|
// persistent-session gates below; never grant persistent-session
|
|
// behavior from an unknown capability set.
|
|
capabilityResolutionFailed = true;
|
|
effectiveCapabilities = null;
|
|
}
|
|
}
|
|
|
|
// Gate the sync, session, and execution decisions below on the effective
|
|
// snapshot. A genuinely absent snapshot (no runtime, no lease) keeps the
|
|
// prior behavior, so it never removes a working path. A present snapshot
|
|
// can only remove a capability, never add one back.
|
|
//
|
|
// Native file sync needs BOTH sync verbs: the runner exposes syncIn and
|
|
// syncOut both-or-neither, so a consumer either uses the native path for
|
|
// both directions or keeps the base64 fallback for both. When the snapshot
|
|
// removes either verb, keep the byte-identical base64 fallback. When the
|
|
// resolution failed, fail closed and keep the base64 fallback too: an
|
|
// unverified provider never gets the native sync path. This preserves the
|
|
// existing reusable-lease sync enforcement unchanged.
|
|
const nativeSyncAllowed =
|
|
!capabilityResolutionFailed &&
|
|
(!effectiveCapabilities ||
|
|
(effectiveCapabilities.nativeSyncIn && effectiveCapabilities.nativeSyncOut));
|
|
// A command that opts onto the persistent session needs the provider to keep
|
|
// persistent process sessions. When the snapshot removes that capability,
|
|
// never force the session; the command runs one-shot instead. When the
|
|
// resolution failed, fail closed and never force the session.
|
|
const persistentSessionsAllowed =
|
|
!capabilityResolutionFailed &&
|
|
(!effectiveCapabilities || effectiveCapabilities.persistentProcessSessions);
|
|
|
|
// Resolve the per-run duplex bridge kill switch. It rides the host-side
|
|
// sandbox target on the same seam as `effectiveCapabilities`, so the value
|
|
// stays on the host and never enters the sandbox environment. Fail closed:
|
|
// an absent runtime, an absent method, or a read error keeps the file
|
|
// bridge. The stamp never turns a read error into a grant.
|
|
let enableSandboxDuplexBridge = false;
|
|
if (input.environmentRuntime?.readSandboxDuplexBridgeInput) {
|
|
try {
|
|
const duplexBridgeInput = await input.environmentRuntime.readSandboxDuplexBridgeInput();
|
|
enableSandboxDuplexBridge = duplexBridgeInput.enableDuplexBridge === true;
|
|
} catch {
|
|
enableSandboxDuplexBridge = false;
|
|
}
|
|
}
|
|
|
|
return {
|
|
kind: "remote",
|
|
transport: "sandbox",
|
|
providerKey: parsed.config.provider,
|
|
shellCommand,
|
|
remoteCwd,
|
|
enableSandboxDuplexBridge,
|
|
// Attach the host duplex observability recorder next to the runner. The bridge
|
|
// binds it to the fixed observability surface. Absent keeps the no-op
|
|
// default, so the surface stays inert on a run with no injected recorder.
|
|
duplexObservabilityRecorder: input.duplexObservabilityRecorder ?? null,
|
|
// Attach the process-owned aggregate byte ledger next to the runner. The
|
|
// bridge passes it to the broker, the decoder, and the response-body reader.
|
|
// Absent keeps the bridge inert for this seam.
|
|
duplexAggregateByteLedger: input.duplexAggregateByteLedger ?? null,
|
|
...(effectiveCapabilities ? { effectiveCapabilities: Object.freeze({ ...effectiveCapabilities }) } : {}),
|
|
environmentId: input.environment.id ?? null,
|
|
leaseId: input.leaseId ?? null,
|
|
timeoutMs,
|
|
// Run-log streaming defaults ON for sandbox environments so agent CLI
|
|
// output reaches the UI mid-run; `streamRunLogs: false` is an explicit
|
|
// opt-out back to batch-at-end delivery.
|
|
streamRunLogs: parsed.config.streamRunLogs !== false,
|
|
// Interactive ACP output streaming is decided downstream from the effective
|
|
// capability snapshot alone: the process session bridge streams only when
|
|
// the provider keeps persistent process sessions and runs independent
|
|
// control commands. The snapshot is absent when resolution failed, so the
|
|
// bridge fails closed to the output-file poll.
|
|
runner: input.environmentRuntime && input.lease
|
|
? {
|
|
// Provider-backed sandbox RPCs do not surface bounded mid-stream
|
|
// progress for a single stdin upload, so keep the capability disabled
|
|
// here. The client falls back to the chunked upload path when this is
|
|
// false.
|
|
supportsSingleStreamStdinProgress: false,
|
|
// Carry the verified concurrent-sync opt-in to the sync client. The
|
|
// client copies it onto the native path and ignores it on the base64
|
|
// fallback, which always permits concurrency. A null snapshot or a
|
|
// provider that never opted in keeps it false, so an unverified
|
|
// provider never permits concurrent sync operations.
|
|
allowConcurrentSyncOperations: effectiveCapabilities?.concurrentSyncOperations === true,
|
|
execute: async (commandInput) => {
|
|
// Record true start and stop timestamps around the provider await,
|
|
// so the exec span and the result carry a real wall time.
|
|
const startedAtMs = Date.now();
|
|
const startedAt = new Date(startedAtMs).toISOString();
|
|
// Open one `sandbox.exec` span BEFORE the provider await, so the
|
|
// native span duration covers the whole execution and a thrown
|
|
// execution still produces a span. The span parents to the active
|
|
// step span. `startSpan` sits inside a guard; observability must
|
|
// never change execution control flow, and a no-op tracer
|
|
// (tracing off) makes the whole block inert.
|
|
const activeStep = getActiveStepContext();
|
|
const criticalPath = activeStep?.criticalPath ?? true;
|
|
let span: ExecSpan | null = null;
|
|
try {
|
|
span = tracer.startSpan("sandbox.exec", undefined, activeStep?.parentContext);
|
|
} catch {
|
|
span = null;
|
|
}
|
|
try {
|
|
// Classify the span outcome from the provider execution ONLY.
|
|
// The inner try/catch wraps just the provider await, so a thrown
|
|
// provider execution marks the span failed. A later log-callback
|
|
// rejection sits outside this block and never flips a successful
|
|
// execution to failed.
|
|
// Incremental log sink. The provider streams each output chunk
|
|
// through the execute.log notification while the command runs.
|
|
// Serialize the delivery per execute call so the runner sees the
|
|
// chunks in order, and keep the delivered text per stream, so the
|
|
// final-result delivery below emits only the un-streamed suffix
|
|
// and can detect a provider poll fallback that returns a
|
|
// different buffer.
|
|
let incrementalLogChain: Promise<void> = Promise.resolve();
|
|
let deliveredStdout = "";
|
|
let deliveredStderr = "";
|
|
const onIncrementalLog = (
|
|
stream: "stdout" | "stderr",
|
|
chunk: string,
|
|
): Promise<void> => {
|
|
if (stream === "stdout") deliveredStdout += chunk;
|
|
else deliveredStderr += chunk;
|
|
incrementalLogChain = incrementalLogChain.then(() =>
|
|
commandInput.onLog?.(stream, chunk),
|
|
);
|
|
return incrementalLogChain;
|
|
};
|
|
let result;
|
|
try {
|
|
result = await input.environmentRuntime!.execute({
|
|
environment: input.environment as Environment,
|
|
lease: input.lease!,
|
|
command: commandInput.command,
|
|
args: commandInput.args,
|
|
cwd: commandInput.cwd ?? remoteCwd,
|
|
env: commandInput.env,
|
|
stdin: commandInput.stdin,
|
|
timeoutMs: commandInput.timeoutMs,
|
|
onLog: commandInput.onLog ? onIncrementalLog : undefined,
|
|
// The ACP process session bridge sets `useSession` so its
|
|
// long-lived agent command opens the persistent session and
|
|
// streams output, even though it runs with no active step.
|
|
// The effective snapshot gates it: a provider that cannot
|
|
// keep persistent process sessions never forces the session,
|
|
// so the command runs one-shot instead.
|
|
forceSession: persistentSessionsAllowed ? commandInput.useSession : false,
|
|
// The bridge control-plane execs set `bypassSession` so they
|
|
// run one-shot and never queue behind the long-lived agent
|
|
// command on the persistent session. An explicit bypass wins
|
|
// over `forceSession` and over the active-step selection.
|
|
bypassSession: commandInput.bypassSession,
|
|
});
|
|
} catch (error) {
|
|
// The provider execution threw. Mark the span failed with the
|
|
// measured wall time, then rethrow the original error unchanged.
|
|
if (span) {
|
|
try {
|
|
setSandboxExecSpanFailure(span, {
|
|
provider: providerFamily,
|
|
command: commandInput.command,
|
|
wallMs: Date.now() - startedAtMs,
|
|
criticalPath,
|
|
});
|
|
} catch {
|
|
// Observability must not change execution control flow.
|
|
}
|
|
}
|
|
throw error;
|
|
}
|
|
// The provider execution succeeded. The span timing and outcome
|
|
// come from the command result, not from the log callbacks below.
|
|
const finishedAtMs = Date.now();
|
|
const finishedAt = new Date(finishedAtMs).toISOString();
|
|
const durationMs = finishedAtMs - startedAtMs;
|
|
// `setSandboxExecSpanAttributes` sets ONLY the closed
|
|
// `paperclip.sandbox.startup.exec.*` allowlist: the normalized
|
|
// provider family, the clamped command label, the numeric exit
|
|
// code, the wall / wait-before / sandbox / network times, the
|
|
// critical-path flag, and the outcome. The full command, args,
|
|
// env, stdout, and stderr never ride the span.
|
|
if (span) {
|
|
try {
|
|
setSandboxExecSpanAttributes(span, {
|
|
provider: providerFamily,
|
|
command: commandInput.command,
|
|
exitCode: result.exitCode,
|
|
wallMs: durationMs,
|
|
waitBeforeMs: toFiniteNumber(result.metadata?.getDurationMs),
|
|
sandboxMs: toFiniteNumber(result.metadata?.durationMs),
|
|
criticalPath,
|
|
cacheHit: toBoolean(result.metadata?.cacheHit),
|
|
});
|
|
} catch {
|
|
// Observability must not change execution control flow.
|
|
}
|
|
}
|
|
// Drain the ordered incremental delivery before the final
|
|
// result. The provider streamed chunks arrive as execute.log
|
|
// notifications while the command runs; awaiting the chain keeps
|
|
// the runner order and surfaces a log-sink rejection.
|
|
await incrementalLogChain;
|
|
// Deliver only the suffix the provider did NOT already stream.
|
|
// The streamed chunks usually form an in-order prefix of the
|
|
// final result, so the remaining output is the final text past
|
|
// the delivered text. When the provider streamed nothing, the
|
|
// whole output is the suffix. When it streamed the complete
|
|
// output, the suffix is empty and nothing repeats. When it
|
|
// streamed a prefix and then fell back to a poll whose buffer
|
|
// does not continue that prefix, `undeliveredSuffix` returns the
|
|
// whole final output, so the durable log keeps the complete
|
|
// result and never holds a truncated slice. A rejected `onLog`
|
|
// still propagates to the caller (control flow is unchanged),
|
|
// but the span already carries the successful outcome, so a log
|
|
// failure never marks the execution failed.
|
|
const stdoutSuffix = undeliveredSuffix(deliveredStdout, result.stdout ?? "");
|
|
if (stdoutSuffix) await commandInput.onLog?.("stdout", stdoutSuffix);
|
|
const stderrSuffix = undeliveredSuffix(deliveredStderr, result.stderr ?? "");
|
|
if (stderrSuffix) await commandInput.onLog?.("stderr", stderrSuffix);
|
|
return {
|
|
exitCode: result.exitCode,
|
|
signal: result.signal ?? null,
|
|
timedOut: result.timedOut,
|
|
stdout: result.stdout,
|
|
stderr: result.stderr,
|
|
pid: null,
|
|
startedAt,
|
|
finishedAt,
|
|
durationMs,
|
|
};
|
|
} finally {
|
|
if (span) {
|
|
try {
|
|
span.end();
|
|
} catch {
|
|
// Observability must not change execution control flow.
|
|
}
|
|
}
|
|
}
|
|
},
|
|
// Expose the native file-sync capability only when the provider's
|
|
// worker advertises BOTH sync verbs AND the effective snapshot still
|
|
// grants native sync; otherwise leave syncIn/syncOut undefined so
|
|
// the orchestrator keeps the byte-identical base64 path.
|
|
...(nativeSyncAllowed &&
|
|
input.environmentRuntime.supportsSync({
|
|
environment: input.environment as Environment,
|
|
lease: input.lease,
|
|
})
|
|
? {
|
|
syncIn: (operations) =>
|
|
input.environmentRuntime!.syncIn({
|
|
environment: input.environment as Environment,
|
|
lease: input.lease!,
|
|
operations,
|
|
}),
|
|
syncOut: (operations) =>
|
|
input.environmentRuntime!.syncOut({
|
|
environment: input.environment as Environment,
|
|
lease: input.lease!,
|
|
operations,
|
|
}),
|
|
}
|
|
: {}),
|
|
// Expose the duplex channel only when the effective snapshot grants
|
|
// the opt-in `duplexCommandStream` capability. A null snapshot
|
|
// (resolution failed or the snapshot is not resolvable) leaves the
|
|
// member undefined, so the caller keeps the file bridge. This mirrors
|
|
// the syncIn/syncOut gate above and fails closed.
|
|
// HTTP/2 is the preferred transport. `queue_v1` is the soft-deprecated fallback.
|
|
...(effectiveCapabilities?.duplexCommandStream
|
|
? {
|
|
openDuplexChannel: (channelInput) =>
|
|
input.environmentRuntime!.openDuplexChannel({
|
|
environment: input.environment as Environment,
|
|
lease: input.lease!,
|
|
command: channelInput.command,
|
|
}),
|
|
}
|
|
: {}),
|
|
}
|
|
: undefined,
|
|
};
|
|
}
|
|
|
|
if (
|
|
!adapterSupportsRemoteManagedEnvironments(input.adapterType) ||
|
|
input.environment.driver !== "ssh"
|
|
) {
|
|
return null;
|
|
}
|
|
|
|
const parsed = await resolveEnvironmentDriverConfigForRuntime(input.db, input.companyId, {
|
|
id: input.environment.id,
|
|
driver: input.environment.driver as "ssh",
|
|
config: parseObject(input.environment.config),
|
|
});
|
|
if (parsed.driver !== "ssh") {
|
|
return null;
|
|
}
|
|
|
|
const remoteCwd =
|
|
typeof input.leaseMetadata?.remoteCwd === "string" && input.leaseMetadata.remoteCwd.trim().length > 0
|
|
? input.leaseMetadata.remoteCwd.trim()
|
|
: parsed.config.remoteWorkspacePath;
|
|
|
|
return {
|
|
kind: "remote",
|
|
transport: "ssh",
|
|
environmentId: input.environment.id ?? null,
|
|
leaseId: input.leaseId ?? null,
|
|
remoteCwd,
|
|
spec: {
|
|
host: parsed.config.host,
|
|
port: parsed.config.port,
|
|
username: parsed.config.username,
|
|
remoteWorkspacePath: parsed.config.remoteWorkspacePath,
|
|
privateKey: parsed.config.privateKey,
|
|
knownHosts: parsed.config.knownHosts,
|
|
strictHostKeyChecking: parsed.config.strictHostKeyChecking,
|
|
remoteCwd,
|
|
},
|
|
};
|
|
}
|
|
|
|
export async function resolveEnvironmentExecutionTransport(
|
|
input: Parameters<typeof resolveEnvironmentExecutionTarget>[0],
|
|
): Promise<Record<string, unknown> | null> {
|
|
return adapterExecutionTargetToRemoteSpec(await resolveEnvironmentExecutionTarget(input)) as Record<string, unknown> | null;
|
|
}
|