From a8d32e5e61950957669784874b53b98ba71845cc Mon Sep 17 00:00:00 2001 From: Devin Foley Date: Tue, 15 Sep 2026 17:09:43 -0700 Subject: [PATCH] feat(sandbox-providers): add CreateOS sandbox provider (#13434) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent work runs in sandboxes that provider plugins supply > - Operators can choose a provider to run agent work > - CreateOS adds another provider with workspace-preserving pause and resume > - This pull request adds a CreateOS provider plugin > - The benefit is that operators can preserve a workspace between runs without keeping its compute active ## Linked Issues or Issue Description Refs #13203 and the earlier closed #13096. This continues the CreateOS contribution from @bhautikchudasama and @ashwaq06. The branch preserves the original implementation commit. Thank you to both contributors. When squash-merging, preserve the original author's credit in the squash commit body: ```text Co-Authored-By: bhautikchudasama ``` The original fork rejects maintainer pushes. This branch includes the merge-conflict resolution and review fixes. The request is described below using `adapter_request.yml`. **Agent or provider** CreateOS sandbox API (https://api.sb.createos.sh). **Why this adapter is useful** CreateOS can pause a sandbox and resume it by ID. The workspace survives the pause. This adds a reusable-lease option to the existing sandbox provider system. **How the agent is invoked** Build and install the local plugin as described in its README. Open Instance Settings, then Environments. Select the `createos` driver. Supply an API key and shape. The driver then supplies sandbox leases for agent runs. **Are you willing to implement it?** Yes. This pull request is the implementation. ## What Changed - Adds the `createos` sandbox provider under `packages/plugins/sandbox-providers/createos`. - Calls the CreateOS HTTP API directly. The package adds no vendor SDK. - Implements the environment lifecycle hooks, incremental process output, and binary workspace sync. - Registers the optional bundled provider and its trusted host credential fallback. The fallback is limited to the official API origin; custom endpoints require an explicit key. - Lists the package in the release manifest with `publishFromCi: false` until its first npm publish is bootstrapped. - Waits through delayed pause/resume state updates without duplicate action requests. - Cancels queued API requests promptly while preserving request spacing. - Uses direct CLI invocation in the setup guide so paths and IDs are passed without an extra shell expansion. - Includes current master and retains its existing Git-subfolder containment fix. ## Demo Fresh setup and a run against a CreateOS sandbox. https://github.com/user-attachments/assets/e71b9e06-c006-4fb9-b847-52dfd68f6110 https://github.com/user-attachments/assets/43b5ac75-66bd-4f76-8563-67e4c7759084 ## Verification All 25 jobs in [CI run 34884260542](https://github.com/paperclipai/paperclip/actions/runs/34884260542) passed at commit `f8d0997677024b784fdadf9d44a84c01cb4e813c`, including typecheck, build, native runner verification, server and workspace tests, browser tests, and the canary release dry run. Greptile reviewed the same commit at 5/5 with no unresolved review threads. GitHub reports no merge conflicts. The remaining merge gate is code-owner approval for the new `package.json`, as required by `.github/CODEOWNERS` and the `master` ruleset. Reviewers have been requested automatically. Local checks passed: - Provider: `pnpm typecheck`, `pnpm test` (52 passed, one live smoke skipped), and `pnpm build`. - Host: focused credential and bundled-plugin tests (17 passed), plus CLI invocation safety (39 passed). - Release: package manifest check and release policy tests (18 passed). The full local `pnpm test:run` attempt caught the README command issue; its focused rerun now passes. The full local run stopped after its general-server group: 7,804 tests passed, with unrelated embedded PostgreSQL startup failures and 10 failures in unchanged runtime-skill-cache tests (`EACCES` on directory rename on macOS). It did not reach the later test groups. Local `pnpm -r typecheck` and `pnpm build` reach the runner package and stop because this machine has no Rust/Cargo installation. The corresponding CI checks passed on provisioned runners, as linked above. The live CreateOS smoke requires explicit provider credentials and was not run during this review. It is available with `CREATEOS_LIVE_TEST=1 pnpm test` in the provider directory. The author supplied the demo links above. ## Risks The provider is opt-in and is not installed by default. It is available through a local-path install or explicit image inclusion. npm publication remains disabled until a maintainer bootstraps the package and enables publishing. Sandbox creation has no idempotency key. An ambiguous create response can leave a resource that requires provider-account inspection. Process tracking is in memory; durable lease recovery belongs to the host. The provider does not advertise guaranteed expiry, interactive login, snapshots, duplex channels, or ingress. Live native-runner qualification remains outside this PR's tested claims. ## Model Used Original provider implementation: human-authored by @bhautikchudasama, as reported in #13203. The original description reports Claude Opus 5 assistance. Review and follow-up fixes: OpenAI GPT-6 via Codex, with code review, editing, and tool execution. The precise runtime model variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused checks; full-suite environment limits documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: bhautikchudasama Co-authored-by: Paperclip --- doc/plugins/PLUGIN_AUTHORING_GUIDE.md | 11 + .../sandbox-providers/createos/.gitignore | 3 + .../sandbox-providers/createos/README.md | 146 ++++++++ .../sandbox-providers/createos/package.json | 63 ++++ .../sandbox-providers/createos/src/client.ts | 140 ++++++++ .../sandbox-providers/createos/src/config.ts | 61 ++++ .../sandbox-providers/createos/src/execute.ts | 229 ++++++++++++ .../createos/src/file-sync.test.ts | 137 +++++++ .../createos/src/file-sync.ts | 175 +++++++++ .../sandbox-providers/createos/src/index.ts | 2 + .../createos/src/manifest.ts | 46 +++ .../createos/src/plugin.live.test.ts | 50 +++ .../createos/src/plugin.test.ts | 336 ++++++++++++++++++ .../sandbox-providers/createos/src/plugin.ts | 201 +++++++++++ .../createos/src/request-pacer.test.ts | 38 ++ .../createos/src/request-pacer.ts | 39 ++ .../sandbox-providers/createos/src/worker.ts | 5 + .../sandbox-providers/createos/tsconfig.json | 11 + .../createos/vitest.config.ts | 8 + scripts/release-package-manifest.json | 5 + server/src/__tests__/bundled-plugins.test.ts | 12 + server/src/__tests__/plugin-database.test.ts | 24 ++ server/src/services/bundled-plugins.ts | 5 + server/src/services/plugin-loader.ts | 1 + 24 files changed, 1748 insertions(+) create mode 100644 packages/plugins/sandbox-providers/createos/.gitignore create mode 100644 packages/plugins/sandbox-providers/createos/README.md create mode 100644 packages/plugins/sandbox-providers/createos/package.json create mode 100644 packages/plugins/sandbox-providers/createos/src/client.ts create mode 100644 packages/plugins/sandbox-providers/createos/src/config.ts create mode 100644 packages/plugins/sandbox-providers/createos/src/execute.ts create mode 100644 packages/plugins/sandbox-providers/createos/src/file-sync.test.ts create mode 100644 packages/plugins/sandbox-providers/createos/src/file-sync.ts create mode 100644 packages/plugins/sandbox-providers/createos/src/index.ts create mode 100644 packages/plugins/sandbox-providers/createos/src/manifest.ts create mode 100644 packages/plugins/sandbox-providers/createos/src/plugin.live.test.ts create mode 100644 packages/plugins/sandbox-providers/createos/src/plugin.test.ts create mode 100644 packages/plugins/sandbox-providers/createos/src/plugin.ts create mode 100644 packages/plugins/sandbox-providers/createos/src/request-pacer.test.ts create mode 100644 packages/plugins/sandbox-providers/createos/src/request-pacer.ts create mode 100644 packages/plugins/sandbox-providers/createos/src/worker.ts create mode 100644 packages/plugins/sandbox-providers/createos/tsconfig.json create mode 100644 packages/plugins/sandbox-providers/createos/vitest.config.ts diff --git a/doc/plugins/PLUGIN_AUTHORING_GUIDE.md b/doc/plugins/PLUGIN_AUTHORING_GUIDE.md index b541ff5caa..e4d2670364 100644 --- a/doc/plugins/PLUGIN_AUTHORING_GUIDE.md +++ b/doc/plugins/PLUGIN_AUTHORING_GUIDE.md @@ -84,6 +84,17 @@ pnpm build ## Supported alpha surface +### CreateOS sandbox provider + +The in-repo [`@paperclipai/plugin-createos`](../../packages/plugins/sandbox-providers/createos/README.md) +package implements environment lifecycle hooks and incremental managed-process +output and binary workspace transfers using CreateOS's public HTTP API. It does +not advertise interactive login or template capture. Install +the built package by local path; its optional managed-image catalog key is +`createos`. The package README describes configuration and the opt-in live smoke. + +### Worker APIs + Worker: - config diff --git a/packages/plugins/sandbox-providers/createos/.gitignore b/packages/plugins/sandbox-providers/createos/.gitignore new file mode 100644 index 0000000000..0430cc3560 --- /dev/null +++ b/packages/plugins/sandbox-providers/createos/.gitignore @@ -0,0 +1,3 @@ +dist +node_modules +.paperclip-sdk diff --git a/packages/plugins/sandbox-providers/createos/README.md b/packages/plugins/sandbox-providers/createos/README.md new file mode 100644 index 0000000000..bd79ebeee7 --- /dev/null +++ b/packages/plugins/sandbox-providers/createos/README.md @@ -0,0 +1,146 @@ +# `@paperclipai/plugin-createos` + +CreateOS sandbox provider for Paperclip. This package lives alongside Daytona +and E2B, outside the root pnpm workspace, and uses CreateOS's public HTTP API. +No CreateOS SDK dependency is required; the `tar` library handles local archives. The package has not been published as +part of this change; use a local-path install for development. Its release +manifest entry has `publishFromCi: false` until a maintainer bootstraps the first +npm publish and enables CI publishing. + +## Build and install locally + +Requires Node 24.11+, pnpm, and an installed Paperclip checkout. + +```sh +cd packages/plugins/sandbox-providers/createos +pnpm install --ignore-workspace --no-lockfile +pnpm typecheck +pnpm test +pnpm build +``` + +From the Paperclip checkout, with your instance running: + +```sh +node cli/node_modules/tsx/dist/cli.mjs cli/src/index.ts plugin install /absolute/path/to/paperclip/packages/plugins/sandbox-providers/createos +node cli/node_modules/tsx/dist/cli.mjs cli/src/index.ts plugin list +node cli/node_modules/tsx/dist/cli.mjs cli/src/index.ts plugin inspect paperclip.createos-sandbox-provider +``` + +Rebuild after source changes; local plugin output watching reloads the worker. +If the running worker still uses the previous build, explicitly reload it when +no sandbox commands are active: + +```sh +node cli/node_modules/tsx/dist/cli.mjs cli/src/index.ts plugin disable paperclip.createos-sandbox-provider +node cli/node_modules/tsx/dist/cli.mjs cli/src/index.ts plugin enable paperclip.createos-sandbox-provider +``` + +The plugin uses the same package entrypoints and publish manifest helper as +other sandbox providers. Plugin workers are trusted code on the Paperclip host. + +## Configure an environment + +Configure the provider under **Instance Settings → Environments**. The plugin +has no custom UI. Required fields: + +- `apiUrl`: defaults to `https://api.sb.createos.sh`. A trailing `/v1` + is accepted and normalized. HTTPS is required except on loopback for testing. +- `shape`: choose from the dropdown of published CreateOS shapes. The bundled + choices match `https://api.sb.createos.sh/v1/shapes` as of 2026-09-08. +- `apiKey`: your CreateOS key. Paperclip saves pasted keys as company secrets; + a resolved environment key takes precedence over the optional host fallback + `CREATEOS_API_KEY` for `https://api.sb.createos.sh` only. Custom API endpoints, + including loopback fixtures, require an explicit environment key. The host + forwards this fallback only to the trusted + CreateOS package installed from the repository or bundled plugin catalog + (or from the first-party npm scope after publication). Other local plugin + paths must use an environment-configured key. + +Optional fields: + +- `rootfs`: a rootfs catalog entry or a ready template ID/name; omission uses + the provider default. The image must include `/bin/bash`, ordinary Unix + utilities including `tar`, `base64`, and GNU `realpath` (`-m` support), and the selected adapter's runtime + dependencies (such as Node and Git). The generic runtime provisions/stages + agent assets; this plugin does not build an agent image. +- `region`: must match the API endpoint. Omission uses the provider default. +- `timeoutMs`: operation/default command deadline, 300000 ms by default. This + is **not** a sandbox TTL. +- `reuseLease`: default false. False deletes on release; true waits for pause + completion and later resumes the same sandbox, preserving workspace data. + +The probe creates a sandbox, prepares its workspace, executes a managed +command, and deletes the sandbox. It therefore uses real provider resources. + +## Implemented behavior + +- Company/environment-bound lease metadata and a random workspace marker + checked before a resumed lease is trusted. API keys are not lease metadata. +- State-aware pause/resume with bounded polling. Transient errors are surfaced; + only missing/terminal sandboxes or a mismatched workspace expire a resume. +- Managed pipe processes with explicit working directory, quoted arguments, + per-command environment, staged stdin, and separate stdout/stderr. + Per-command variables are applied by the command wrapper; CreateOS's API-level + environment overrides only accept keys declared at sandbox creation. +- Incremental output via replayable NDJSON, bounded reconnect attempts, UTF-8 + decoding across frame boundaries, and explicit errors for missing output. + Returned stdout/stderr each retain at most 4 Mi characters of tail output; + `metadata.outputTruncated` reports truncation. Live log chunks are still + delivered as they arrive. +- API requests to each endpoint are spaced by at least 300 ms across this + worker's leases, keeping callback polling below the provider's 300/minute + IP limit. Other workers or applications sharing the same IP can still + exhaust that shared limit. +- Command timeout and active lease-release/shutdown cancellation explicitly + terminate the process tree. Disconnecting the stream alone is not cancellation. + Unknown process-creation outcomes and failed process cleanup prevent reuse + in the current worker; the operator/host must destroy the affected lease. + No automatic retry of process creation or command execution occurs. +- Workspace realization at `/paperclip-workspace` and native binary file sync, + including directory archives, file modes, exclusions, symlink containment, + atomic file downloads, and ordered post-upload commands. This keeps bulk data + out of CreateOS's 1 MiB managed-process output journal. Ordinary command + output still uses that journal and fails explicitly if unread data is evicted. + Outbound archives are validated before extraction and limited to 10 GiB of + declared file data; absolute/traversing paths and escaping links are rejected. + +## Capability boundaries + +Interactive login PTYs, temporary login leases, snapshot capture, +duplex channels, and provider WebSocket ingress are not advertised. +CreateOS idle auto-pause is not a guaranteed absolute expiry, so acquisition +with `requestedExpiresAt` fails before provisioning a resource. + +The host has an outbound-WSS native runner path for providers without ingress. +Using it additionally requires a reachable Paperclip runner endpoint and the +host's qualified runner/provider artifacts. This plugin's local tests do not +constitute an end-to-end native runner qualification. + +Sandbox create has no idempotency key in the inspected API. If its response is +lost before an ID arrives, the plugin cannot identify that resource for cleanup; +inspect the provider account before retrying an ambiguous creation. A plugin +crash also loses its in-memory command tracking; durable lease recovery remains +host-owned. This version does not claim a provider-side expiration guarantee. + +## Opt-in live smoke + +Default tests use mocked HTTP responses and do not contact CreateOS. To test a +chosen endpoint, export `CREATEOS_API_URL`, `CREATEOS_API_KEY`, `CREATEOS_SHAPE`, +and optionally `CREATEOS_ROOTFS`, then run: + +```sh +CREATEOS_LIVE_TEST=1 pnpm test +``` + +The live test creates a sandbox, round-trips a 5 MiB binary file through native +sync, checks stdin/env/output, writes a file, pauses +and resumes the sandbox, verifies the file, and deletes it in `finally`. +It does not print credentials. Cleanup errors fail the test. + +## Optional managed-image inclusion + +The bundled catalog key is `createos`. Include the `createos` directory in the +Docker `CLOUD_BUNDLED_PLUGINS` build argument, then include `createos` in the +managed configuration's `plugins.autoInstall` list. Adding the catalog entry +does not auto-install the plugin or change the default image contents. diff --git a/packages/plugins/sandbox-providers/createos/package.json b/packages/plugins/sandbox-providers/createos/package.json new file mode 100644 index 0000000000..9f6d75ea51 --- /dev/null +++ b/packages/plugins/sandbox-providers/createos/package.json @@ -0,0 +1,63 @@ +{ + "name": "@paperclipai/plugin-createos", + "version": "0.1.0", + "description": "CreateOS sandbox provider plugin for Paperclip environments", + "license": "MIT", + "homepage": "https://github.com/paperclipai/paperclip", + "bugs": { + "url": "https://github.com/paperclipai/paperclip/issues" + }, + "repository": { + "type": "git", + "url": "https://github.com/paperclipai/paperclip", + "directory": "packages/plugins/sandbox-providers/createos" + }, + "type": "module", + "exports": { + ".": "./src/index.ts" + }, + "publishConfig": { + "access": "public", + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + } + }, + "main": "./dist/index.js", + "types": "./dist/index.d.ts" + }, + "files": [ + "dist" + ], + "paperclipPlugin": { + "manifest": "./dist/manifest.js", + "worker": "./dist/worker.js" + }, + "keywords": [ + "paperclip", + "plugin", + "sandbox", + "createos" + ], + "scripts": { + "prebuild": "pnpm -C ../../../.. --filter @paperclipai/plugin-sdk ensure-build-deps", + "build": "rm -rf dist && tsc", + "clean": "rm -rf dist", + "typecheck": "pnpm -C ../../../.. --filter @paperclipai/plugin-sdk ensure-build-deps && tsc --noEmit", + "test": "vitest run --config vitest.config.ts", + "prepack": "rm -f package.dev.json && cp package.json package.dev.json && node ../../../../scripts/generate-plugin-package-json.mjs", + "postpack": "if [ -f package.dev.json ]; then mv package.dev.json package.json; fi" + }, + "devDependencies": { + "@types/node": "^24.0.0", + "typescript": "^5.7.3", + "vitest": "^4.1.8" + }, + "engines": { + "node": ">=24.11.0" + }, + "dependencies": { + "tar": "^7.5.22" + } +} diff --git a/packages/plugins/sandbox-providers/createos/src/client.ts b/packages/plugins/sandbox-providers/createos/src/client.ts new file mode 100644 index 0000000000..2c4dbd332d --- /dev/null +++ b/packages/plugins/sandbox-providers/createos/src/client.ts @@ -0,0 +1,140 @@ +import { setTimeout as delay } from "node:timers/promises"; +import type { CreateosConfig } from "./config.js"; +import { resolveApiKey } from "./config.js"; +import { waitForRequest } from "./request-pacer.js"; + +export class CreateosApiError extends Error { + constructor(readonly status: number, operation?: string) { + // Provider bodies may echo command input or credentials. Keep them out of + // persisted errors and probe metadata. + super(`CreateOS request failed (HTTP ${status})${operation ? ` during ${operation}` : ""}.`); + } +} + +export interface Sandbox { + id: string; + status?: string; +} + +export function object(value: unknown): Record { + if (!value || typeof value !== "object" || Array.isArray(value)) { + throw new Error("CreateOS returned an invalid response."); + } + return value as Record; +} + +export function identifier(value: unknown): string { + if (typeof value !== "string" || !/^[A-Za-z0-9_-]{1,200}$/.test(value)) { + throw new Error("CreateOS returned an invalid resource ID."); + } + return value; +} + +export class CreateosClient { + readonly apiKey: string; + constructor(readonly config: CreateosConfig) { + this.apiKey = resolveApiKey(config); + } + + async request(path: string, init: RequestInit = {}): Promise { + const signal = init.signal ?? AbortSignal.timeout(this.config.timeoutMs); + await waitForRequest(this.config.apiUrl, signal); + let response: Response; + try { + response = await fetch(`${this.config.apiUrl}/v1${path}`, { + ...init, + redirect: "error", + headers: { ...init.headers, "X-Api-Key": this.apiKey }, + signal, + }); + } catch (error) { + if (init.signal?.aborted) throw init.signal.reason; + // Do not propagate fetch causes: they can contain the configured URL. + if (error instanceof Error && error.name === "TimeoutError") throw error; + throw new Error("CreateOS connection failed."); + } + if (!response.ok) { + await response.body?.cancel(); + // Only fixed operation labels: never include paths, queries, or bodies, + // which may contain credentials or private workspace names. + const operation = path.includes("/files?") ? "file transfer" + : path.includes("/stdin/close") ? "stdin close" + : path.includes("/connect?") ? "output connection" + : path.endsWith("/processes") ? "process creation" + : path.includes("/processes/") ? "process cleanup" + : path.endsWith("/exec") ? "workspace command" + : "sandbox lifecycle"; + throw new CreateosApiError(response.status, operation); + } + return response; + } + + async json(path: string, method = "GET", body?: unknown, signal?: AbortSignal): Promise> { + const response = await this.request(path, { + method, + ...(body !== undefined ? { body: JSON.stringify(body), headers: { "Content-Type": "application/json" } } : {}), + signal, + }); + let envelope: Record; + try { envelope = object(await response.json()); } catch { throw new Error("CreateOS returned invalid JSON."); } + if (envelope.status !== "success") throw new Error("CreateOS returned an unsuccessful response."); + return object(envelope.data); + } + + async getSandbox(id: string, signal?: AbortSignal): Promise { + const data = await this.json(`/sandboxes/${identifier(id)}`, "GET", undefined, signal); + if (data.id !== id || typeof data.status !== "string") throw new Error("CreateOS sandbox identity or state is invalid."); + return { id, status: data.status }; + } + + async createSandbox(signal: AbortSignal): Promise { + const { shape, rootfs, region } = this.config; + const data = await this.json("/sandboxes", "POST", { + shape, + ...(rootfs ? { rootfs } : {}), + ...(region ? { region } : {}), + ingress_enabled: false, + // The host owns lease release. Idle pause is not a command timeout or a + // guaranteed expiry, and could suspend a quiet active agent. + }, signal); + return { id: identifier(data.id) }; + } + + async destroySandbox(id: string): Promise { + try { await this.json(`/sandboxes/${identifier(id)}`, "DELETE"); } + catch (error) { if (!(error instanceof CreateosApiError && error.status === 404)) throw error; } + } + + async transition(id: string, desired: "running" | "paused", signal: AbortSignal): Promise { + let submitted = false; + for (;;) { + signal.throwIfAborted(); + const sandbox = await this.getSandbox(id, signal); + if (sandbox.status === desired) return; + const canSubmit = desired === "running" + ? ["paused", "error"].includes(sandbox.status!) + : sandbox.status === "running"; + if (canSubmit && !submitted) { + try { + await this.json(`/sandboxes/${id}/${desired === "running" ? "resume" : "pause"}`, "POST", undefined, signal); + submitted = true; + } catch (error) { + // A concurrent state transition is reconciled by reading its state. + if (!(error instanceof CreateosApiError && error.status === 409)) throw error; + } + } else if (!canSubmit && !["creating", "pausing", "resuming"].includes(sandbox.status!)) { + throw new Error(`CreateOS sandbox did not reach ${desired}.`); + } + // An accepted transition can remain in its previous state briefly. + // Poll under the same deadline without submitting the action twice. + await delay(250, undefined, { signal }); + } + } + + async upload(id: string, path: string, content: string, signal: AbortSignal): Promise { + const response = await this.request(`/sandboxes/${identifier(id)}/files?path=${encodeURIComponent(path)}`, { + method: "PUT", headers: { "Content-Type": "application/octet-stream" }, body: content, signal, + }); + await response.body?.cancel(); + } +} diff --git a/packages/plugins/sandbox-providers/createos/src/config.ts b/packages/plugins/sandbox-providers/createos/src/config.ts new file mode 100644 index 0000000000..b1aad8d37c --- /dev/null +++ b/packages/plugins/sandbox-providers/createos/src/config.ts @@ -0,0 +1,61 @@ +export interface CreateosConfig { + apiUrl: string; + apiKey: string | null; + shape: string; + rootfs: string | null; + region: string | null; + timeoutMs: number; + reuseLease: boolean; +} + +export function parseConfig(raw: Record): CreateosConfig { + const text = (key: string): string | null => { + const value = raw[key]; + if (value == null) return null; + if (typeof value !== "string" || !value.trim() || value.includes("\0")) { + throw new Error(`${key} must be a non-empty string.`); + } + return value.trim(); + }; + const apiUrl = text("apiUrl"); + if (!apiUrl) throw new Error("CreateOS requires an API URL."); + let url: URL; + try { url = new URL(apiUrl); } catch { throw new Error("CreateOS API URL is invalid."); } + // Configuration is board-owned, but never follow redirects with the API key. + // Plain HTTP is useful for a loopback development server only. + const loopback = ["localhost", "127.0.0.1", "[::1]"].includes(url.hostname); + if ((url.protocol !== "https:" && !(url.protocol === "http:" && loopback)) || + url.username || url.password || url.search || url.hash || + !["", "/", "/v1", "/v1/"].includes(url.pathname)) { + throw new Error("CreateOS API URL must be an HTTPS origin (optionally ending in /v1); HTTP is allowed on loopback only."); + } + const shape = text("shape"); + if (!shape) throw new Error("CreateOS requires a shape from its shape catalog."); + const timeoutMs = raw.timeoutMs ?? 300_000; + if (typeof timeoutMs !== "number" || !Number.isInteger(timeoutMs) || timeoutMs < 1 || timeoutMs > 86_400_000) { + throw new Error("timeoutMs must be an integer between 1 and 86400000."); + } + if (raw.reuseLease != null && typeof raw.reuseLease !== "boolean") { + throw new Error("reuseLease must be a boolean."); + } + return { + apiUrl: url.origin, + apiKey: text("apiKey"), + shape, + rootfs: text("rootfs"), + region: text("region"), + timeoutMs, + reuseLease: raw.reuseLease === true, + }; +} + +export function resolveApiKey(config: CreateosConfig): string { + if (!config.apiKey && config.apiUrl !== "https://api.sb.createos.sh") { + throw new Error("Custom CreateOS API endpoints require an explicit environment API key; the host fallback is only available for https://api.sb.createos.sh."); + } + const key = config.apiKey ?? process.env.CREATEOS_API_KEY?.trim(); + if (!key || /[\r\n\0]/.test(key)) { + throw new Error("CreateOS requires an API key in the environment config or CREATEOS_API_KEY."); + } + return key; +} diff --git a/packages/plugins/sandbox-providers/createos/src/execute.ts b/packages/plugins/sandbox-providers/createos/src/execute.ts new file mode 100644 index 0000000000..c184dc8d3c --- /dev/null +++ b/packages/plugins/sandbox-providers/createos/src/execute.ts @@ -0,0 +1,229 @@ +import { randomUUID } from "node:crypto"; +import { StringDecoder } from "node:string_decoder"; +import { setTimeout as delay } from "node:timers/promises"; +import type { PluginEnvironmentExecuteParams, PluginEnvironmentExecuteResult } from "@paperclipai/plugin-sdk"; +import { CreateosApiError, CreateosClient, identifier, object } from "./client.js"; + +const MAX_LINE_BYTES = 1_048_576; +const MAX_CAPTURE_CHARS = 4_194_304; + +export class CreateosCleanupError extends Error {} + +export function shellQuote(value: string): string { + if (value.includes("\0")) throw new Error("Sandbox command values cannot contain NUL."); + return `'${value.replace(/'/g, `'"'"'`)}'`; +} + +function commandScript(params: PluginEnvironmentExecuteParams, stdinPath: string | null): string { + if (!params.command) throw new Error("A sandbox command is required."); + const env = Object.entries(params.env ?? {}).map(([key, value]) => { + if (!/^[A-Za-z_][A-Za-z0-9_]*$/.test(key) || typeof value !== "string") { + throw new Error("Invalid sandbox environment variable."); + } + return `${key}=${shellQuote(value)}`; + }); + const command = [params.command, ...(params.args ?? [])].map(shellQuote).join(" "); + return [ + params.cwd ? `cd -- ${shellQuote(params.cwd)} || exit` : "", + `exec env ${env.join(" ")} ${command}${stdinPath ? ` < ${shellQuote(stdinPath)}` : ""}`, + ].filter(Boolean).join("\n"); +} + +async function* events(response: Response): AsyncGenerator> { + if (!response.body) throw new Error("CreateOS returned an empty process stream."); + const reader = response.body.getReader(); + const decoder = new TextDecoder(); + let pending = ""; + try { + for (;;) { + const { value, done } = await reader.read(); + pending += done ? decoder.decode() : decoder.decode(value, { stream: true }); + let newline: number; + while ((newline = pending.indexOf("\n")) >= 0) { + const line = pending.slice(0, newline); + pending = pending.slice(newline + 1); + if (line.length > MAX_LINE_BYTES) throw new Error("CreateOS process frame is too large."); + if (line.trim()) yield parseEvent(line); + } + if (pending.length > MAX_LINE_BYTES) throw new Error("CreateOS process frame is too large."); + if (done) { + if (pending.trim()) yield parseEvent(pending); + return; + } + } + } finally { + await reader.cancel().catch(() => undefined); + reader.releaseLock(); + } +} + +function parseEvent(line: string): Record { + try { return object(JSON.parse(line)); } + catch { throw new Error("CreateOS returned an invalid process frame."); } +} + +class Output { + stdout = ""; + stderr = ""; + truncated = false; + private decoders = { stdout: new StringDecoder("utf8"), stderr: new StringDecoder("utf8") }; + constructor(private log: (stream: "stdout" | "stderr", text: string) => void) {} + + private append(stream: "stdout" | "stderr", text: string) { + if (!text) return; + this.log(stream, text); + const combined = this[stream] + text; + if (combined.length > MAX_CAPTURE_CHARS) this.truncated = true; + this[stream] = combined.slice(-MAX_CAPTURE_CHARS); + } + + write(stream: "stdout" | "stderr", bytes: Buffer) { + this.append(stream, this.decoders[stream].write(bytes)); + } + + finish() { + for (const stream of ["stdout", "stderr"] as const) this.append(stream, this.decoders[stream].end()); + } +} + +// A broken output connection does not stop a managed process. Always reconnect +// to the SAME process, never retry its creation, and explicitly terminate its +// tree on timeout/failure. Cleanup has its own deadline, independent of execute. +export async function execute( + client: CreateosClient, + params: PluginEnvironmentExecuteParams, + signal: AbortSignal, + log: (stream: "stdout" | "stderr", text: string) => void = () => {}, +): Promise { + const id = identifier(params.lease.providerLeaseId); + const stdinPath = params.stdin != null ? `/tmp/paperclip-stdin-${randomUUID()}` : null; + const script = commandScript(params, stdinPath); + const output = new Output(log); + let processId: string | null = null; + let creationMayHaveSucceeded = false; + let completed = false; + let staged = false; + let cursor = 0; + const base = `/sandboxes/${id}/processes`; + try { + if (stdinPath) { + staged = true; + await client.upload(id, stdinPath, params.stdin!, signal); + } + let created: Record; + creationMayHaveSucceeded = true; + try { + created = await client.json(base, "POST", { + cmd: "/bin/bash", args: ["-lc", script], + ...(params.cwd ? { cwd: params.cwd } : {}), + // CreateOS API overrides are limited to keys declared at sandbox + // creation. Per-command variables are already safely quoted in the + // command's `env` invocation, including on a reused sandbox. + }, signal); + } catch (error) { + if (error instanceof CreateosApiError && error.status < 500) creationMayHaveSucceeded = false; + throw error; + } + processId = identifier(created.process_id); + // No interactive stdin for ordinary commands; any supplied input comes + // from the staged file, avoiding a write-vs-fast-exit race. + try { await client.json(`${base}/${processId}/stdin/close`, "POST", undefined, signal); } + catch (error) { + if (!(error instanceof CreateosApiError && error.status === 409)) throw error; + } + let reconnects = 0; + for (;;) { + signal.throwIfAborted(); + let response: Response; + try { + response = await client.request(`${base}/${processId}/connect?after=${cursor}`, { signal }); + } catch (error) { + if (error instanceof CreateosApiError && error.status === 410) { + throw new Error("CreateOS process output was evicted before it could be read."); + } + if (signal.aborted || (error instanceof CreateosApiError && error.status < 500 && error.status !== 429)) throw error; + if (++reconnects > 3) throw new Error("CreateOS process output connection failed."); + await delay(250, undefined, { signal }); + continue; + } + try { + for await (const event of events(response)) { + if (event.type === "heartbeat") continue; + if (event.type === "data") { + const seq = event.seq; + if (typeof seq !== "number" || !Number.isSafeInteger(seq) || seq < 1) throw new Error("CreateOS process sequence is invalid."); + if (seq <= cursor) continue; + if (seq !== cursor + 1) throw new Error("CreateOS process output has a sequence gap."); + if ((event.stream !== "stdout" && event.stream !== "stderr") || + typeof event.data_base64 !== "string" || + !/^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/.test(event.data_base64)) { + throw new Error("CreateOS process output is invalid."); + } + output.write(event.stream, Buffer.from(event.data_base64, "base64")); + cursor = seq; + } else if (event.type === "exit") { + const exitCode = event.exit_code; + const exitSignal = event.signal; + if (!(typeof exitCode === "number" && Number.isInteger(exitCode)) && + !(typeof exitSignal === "string" && /^SIG[A-Z0-9]+$/.test(exitSignal))) { + throw new Error("CreateOS process exit status is missing."); + } + completed = true; + output.finish(); + return { + exitCode: typeof exitCode === "number" ? exitCode : null, + signal: typeof exitSignal === "string" ? exitSignal : null, + timedOut: false, stdout: output.stdout, stderr: output.stderr, + metadata: { processId, outputTruncated: output.truncated }, + }; + } else if (event.type === "error") { + throw new Error(event.error === "output_offset_expired" + ? "CreateOS process output was evicted before it could be read." + : "CreateOS process stream reported an error."); + } else { + throw new Error("CreateOS returned an unknown process event."); + } + } + } catch (error) { + // Network read failures can resume from the last accepted sequence. + // Protocol errors must fail closed rather than reconnect past bad data. + if (!(error instanceof TypeError) || signal.aborted) throw error; + } + if (++reconnects > 3) throw new Error("CreateOS process stream ended without an exit status."); + await delay(250, undefined, { signal }); + } + } catch (error) { + if (creationMayHaveSucceeded && !processId) { + throw new CreateosCleanupError("CreateOS process creation could not be confirmed; destroy the lease before reusing it."); + } + if (signal.aborted && signal.reason?.name === "TimeoutError") { + output.finish(); + return { + exitCode: null, timedOut: true, stdout: output.stdout, stderr: output.stderr, + metadata: { processId, outputTruncated: output.truncated }, + }; + } + if (signal.aborted) throw new Error("CreateOS command was cancelled."); + throw error; + } finally { + const cleanupSignal = AbortSignal.timeout(client.config.timeoutMs); + // Do not hide a cleanup failure: the host must know containment is unproven. + try { + if (processId && !completed) { + try { + const termination = await client.json(`${base}/${processId}?grace_ms=1000`, "DELETE", undefined, cleanupSignal); + if (termination.tree_exited !== true) throw new Error("Process tree has not exited."); + } + catch (error) { if (!(error instanceof CreateosApiError && error.status === 404)) throw new CreateosCleanupError("CreateOS command cleanup failed; process termination is unconfirmed."); } + } + } finally { + if (staged && stdinPath) { + // /files has no delete verb. /exec supplies a bounded, one-shot removal + // after the managed process finishes, without retaining another record. + await client.json(`/sandboxes/${id}/exec`, "POST", { + cmd: "/bin/rm", args: ["-f", "--", stdinPath], + }, cleanupSignal).catch(() => undefined); + } + } + } +} diff --git a/packages/plugins/sandbox-providers/createos/src/file-sync.test.ts b/packages/plugins/sandbox-providers/createos/src/file-sync.test.ts new file mode 100644 index 0000000000..4ac98c9890 --- /dev/null +++ b/packages/plugins/sandbox-providers/createos/src/file-sync.test.ts @@ -0,0 +1,137 @@ +import path from "node:path"; +import os from "node:os"; +import { promises as fs } from "node:fs"; +import * as tar from "tar"; +import { afterEach, beforeEach, expect, it, vi } from "vitest"; +import { CreateosClient } from "./client.js"; +import { parseConfig } from "./config.js"; +import { assertRemotePath, syncFiles, validateArchive } from "./file-sync.js"; + +const run = vi.hoisted(() => vi.fn()); +vi.mock("./execute.js", async (original) => ({ ...await original(), execute: run })); + +const config = { apiUrl: "https://createos.example.test", apiKey: "secret", shape: "test", timeoutMs: 5000 }; +const client = () => new CreateosClient(parseConfig(config)); +const params = { driverKey: "createos", companyId: "c", environmentId: "e", config, lease: { providerLeaseId: "sb_test" } }; +let temp: string; +let downloads: Buffer; +let uploads: Buffer[]; +let fetchMock: ReturnType; + +beforeEach(async () => { + temp = await fs.mkdtemp(path.join(os.tmpdir(), "createos-sync-test-")); + downloads = Buffer.from("data"); + uploads = []; + run.mockReset().mockResolvedValue({ exitCode: 0, timedOut: false, stdout: "", stderr: "" }); + fetchMock = vi.fn(async (url: string, init: RequestInit = {}) => { + expect(init.headers).toMatchObject({ "X-Api-Key": "secret" }); + if (init.method === "PUT") { + uploads.push(Buffer.from(await new Response(init.body).arrayBuffer())); + return Response.json({ status: "success", data: {} }); + } + if (new URL(url).pathname.endsWith("/files")) return new Response(downloads); + return Response.json({ status: "success", data: { result: { exit_code: 0 } } }); + }); + vi.stubGlobal("fetch", fetchMock); +}); + +afterEach(async () => { vi.unstubAllGlobals(); await fs.rm(temp, { recursive: true, force: true }); }); + +it("uploads binary bytes directly and applies the requested mode before promotion", async () => { + const source = path.join(temp, "secret"); + const bytes = Buffer.from([0, 255, 128, 10, 13]); + await fs.writeFile(source, bytes); + const result = await syncFiles(client(), { ...params, operations: [{ operationId: "asset", files: [{ + sourcePath: source, targetPath: "/paperclip-workspace/secret", kind: "file", mode: 0o600, + }] }] }, "in", AbortSignal.timeout(5000)); + expect(uploads).toEqual([bytes]); + expect(run.mock.calls[0][1].args[1]).toContain("chmod 600"); + expect(result.operations[0]).toMatchObject({ filesTransferred: 1, bytesTransferred: bytes.length }); +}); + +it("downloads more than the process journal limit through the binary API with exact contents and mode", async () => { + downloads = Buffer.alloc(5 * 1024 * 1024, 0xa5); + const target = path.join(temp, "result"); + const result = await syncFiles(client(), { ...params, operations: [{ operationId: "workspace", files: [{ + sourcePath: "/paperclip-workspace/big.bin", targetPath: target, kind: "file", mode: 0o600, + }] }] }, "out", AbortSignal.timeout(5000)); + expect((await fs.readFile(target)).equals(downloads)).toBe(true); + expect((await fs.stat(target)).mode & 0o777).toBe(0o600); + expect(result.operations[0].bytesTransferred).toBe(downloads.length); + expect(run.mock.calls[0][1].args[1]).toContain("cp --"); + expect(run.mock.calls[0][1].args[1]).not.toContain("base64"); +}); + +it("never replaces an existing host file with a partial download", async () => { + const target = path.join(temp, "result"); + await fs.writeFile(target, "original"); + const normal = fetchMock.getMockImplementation()!; + fetchMock.mockImplementation(async (url, init) => { + if (new URL(url).pathname.endsWith("/files") && (!init?.method || init.method === "GET")) return new Response(new ReadableStream({ + start(controller) { controller.enqueue(new Uint8Array([1, 2])); controller.error(new Error("connection lost")); }, + })); + return normal(url, init); + }); + await expect(syncFiles(client(), { ...params, operations: [{ operationId: "file", files: [{ + sourcePath: "/paperclip-workspace/result", targetPath: target, kind: "file", + }] }] }, "out", AbortSignal.timeout(5000))).rejects.toThrow(); + expect(await fs.readFile(target, "utf8")).toBe("original"); + expect(await fs.readdir(temp)).toEqual(["result"]); +}); + +it("preserves directory files, modes, and internal symlinks while honoring exclusions", async () => { + const source = path.join(temp, "source"); + await fs.mkdir(path.join(source, "node_modules"), { recursive: true }); + await fs.writeFile(path.join(source, "keep"), "hello", { mode: 0o600 }); + await fs.writeFile(path.join(source, "node_modules", "skip"), "ignored"); + await fs.symlink("keep", path.join(source, "link")); + await syncFiles(client(), { ...params, operations: [{ operationId: "directory", files: [{ + sourcePath: source, targetPath: "/paperclip-workspace/project", kind: "directory", exclude: ["node_modules"], + }] }] }, "in", AbortSignal.timeout(5000)); + downloads = uploads[0]; + const target = path.join(temp, "restored"); + await syncFiles(client(), { ...params, operations: [{ operationId: "directory", files: [{ + sourcePath: "/paperclip-workspace/project", targetPath: target, kind: "directory", + }] }] }, "out", AbortSignal.timeout(5000)); + expect(await fs.readdir(target)).toEqual(["keep", "link"]); + expect(await fs.readFile(path.join(target, "keep"), "utf8")).toBe("hello"); + expect((await fs.stat(path.join(target, "keep"))).mode & 0o777).toBe(0o600); + expect(await fs.readlink(path.join(target, "link"))).toBe("keep"); +}); + +it("rejects an archive carrying an escaping symlink before extracting any files", async () => { + const source = path.join(temp, "malicious"); + await fs.mkdir(source); + await fs.writeFile(path.join(source, "safe"), "content"); + await fs.symlink("../../outside", path.join(source, "escape")); + const file = path.join(temp, "malicious.tar"); + await tar.c({ cwd: source, file }, ["."]); + await expect(validateArchive(file)).rejects.toThrow("unsafe entries"); + downloads = await fs.readFile(file); + const target = path.join(temp, "restored"); + await expect(syncFiles(client(), { ...params, operations: [{ operationId: "directory", files: [{ + sourcePath: "/paperclip-workspace/project", targetPath: target, kind: "directory", + }] }] }, "out", AbortSignal.timeout(5000))).rejects.toThrow("unsafe entries"); + await expect(fs.stat(target)).rejects.toThrow(); +}); + +it("runs post-upload commands verbatim and stops after the first failure", async () => { + const command = "printf 'first'; printf 'second'"; + run.mockImplementation(async (_client, execution) => ({ exitCode: execution.args[1] === command ? 1 : 0, timedOut: false, stdout: "", stderr: "" })); + await expect(syncFiles(client(), { ...params, operations: [{ operationId: "commands", files: [], postUploadCommands: [ + { command, cwd: "/paperclip-workspace/project" }, { command: "must-not-run" }, + ] }] }, "in", AbortSignal.timeout(5000))).rejects.toThrow("transfer command failed"); + const commands = run.mock.calls.map((call) => call[1].args[1]); + expect(commands).toContain(command); + expect(commands).not.toContain("must-not-run"); +}); + +it.each(["/etc/passwd", "/paperclip-workspace/../secret", "relative", "/paperclip-workspace-other/file"])("rejects unconfined sandbox path %s before any API call", async (sourcePath) => { + expect(() => assertRemotePath(sourcePath)).toThrow(); + await expect(syncFiles(client(), { ...params, operations: [{ operationId: "bad", files: [{ + sourcePath, targetPath: path.join(temp, "file"), kind: "file", + }] }] }, "out", AbortSignal.timeout(5000))).rejects.toThrow(); + expect(fetchMock).not.toHaveBeenCalled(); + expect(run).not.toHaveBeenCalled(); +}); +vi.mock("./request-pacer.js", () => ({ waitForRequest: vi.fn().mockResolvedValue(undefined) })); diff --git a/packages/plugins/sandbox-providers/createos/src/file-sync.ts b/packages/plugins/sandbox-providers/createos/src/file-sync.ts new file mode 100644 index 0000000000..1acdfbd603 --- /dev/null +++ b/packages/plugins/sandbox-providers/createos/src/file-sync.ts @@ -0,0 +1,175 @@ +import path from "node:path"; +import os from "node:os"; +import { randomUUID } from "node:crypto"; +import { createReadStream, createWriteStream, promises as fs } from "node:fs"; +import { Readable } from "node:stream"; +import { pipeline } from "node:stream/promises"; +import * as tar from "tar"; +import type { PluginEnvironmentSyncInParams, PluginEnvironmentSyncResult } from "@paperclipai/plugin-sdk"; +import { CreateosClient, identifier } from "./client.js"; +import { execute, shellQuote } from "./execute.js"; + +const ROOT = "/paperclip-workspace"; + +export function assertRemotePath(value: string): void { + if (!path.posix.isAbsolute(value) || value.includes("\0") || value.split("/").includes("..")) { + throw new Error("CreateOS transfer requires a confined absolute sandbox path."); + } + const normalized = path.posix.normalize(value); + if (normalized !== ROOT && !normalized.startsWith(`${ROOT}/`)) throw new Error("CreateOS transfer path escapes the workspace."); +} + +function remoteGuard(candidate: string): string { + assertRemotePath(candidate); + // Re-check symlinks inside the sandbox immediately before use. A missing + // canonicalizer fails the command rather than weakening containment. + return `root=$(realpath -- ${shellQuote(ROOT)}) && test "$root" = ${shellQuote(ROOT)} && ` + + `resolved=$(realpath -m -- ${shellQuote(candidate)}) && ` + + `case "$resolved" in "$root"|"$root"/*) ;; *) exit 1 ;; esac`; +} + +export async function validateArchive(file: string): Promise { + let invalid = false; + let bytes = 0; + let files = 0; + const inside = (entryPath: string) => !path.posix.isAbsolute(entryPath) && + !entryPath.split("/").includes("..") && !entryPath.includes("\\") && !entryPath.includes("\0"); + await tar.t({ file, strict: true, onReadEntry(entry) { + bytes += entry.size; + if (bytes > 10 * 1024 ** 3 || !inside(entry.path)) invalid = true; + if (!["File", "OldFile", "Directory", "SymbolicLink", "Link"].includes(entry.type)) invalid = true; + if (entry.type === "File" || entry.type === "OldFile") files++; + if (entry.type === "SymbolicLink" || entry.type === "Link") { + if (!entry.linkpath) { invalid = true; return; } + const base = entry.type === "SymbolicLink" ? path.posix.dirname(entry.path) : "."; + const target = path.posix.normalize(path.posix.join(base, entry.linkpath)); + if (path.posix.isAbsolute(entry.linkpath) || entry.linkpath.includes("\\") || !inside(target)) invalid = true; + } + } }); + if (invalid) throw new Error("CreateOS archive contains unsafe entries or exceeds the extraction limit."); + return files; +} + +function excluded(name: string, patterns: string[]): boolean { + name = name.replace(/^\.\//, "").replace(/\/$/, ""); + return patterns.some((pattern) => [pattern, `${pattern}/**`, `**/${pattern}`, `**/${pattern}/**`] + .some((glob) => path.matchesGlob(name, glob))); +} + +export async function syncFiles( + client: CreateosClient, + params: PluginEnvironmentSyncInParams, + direction: "in" | "out", + signal: AbortSignal, +): Promise { + const id = identifier(params.lease.providerLeaseId); + const operations: PluginEnvironmentSyncResult["operations"] = []; + const run = async (command: string, cwd = ROOT, timeoutMs?: number) => { + assertRemotePath(cwd); + const result = await execute(client, { ...params, command: "/bin/bash", args: ["-c", command], cwd }, + timeoutMs == null ? signal : AbortSignal.any([signal, AbortSignal.timeout(timeoutMs)])); + if (result.timedOut || result.exitCode !== 0) throw new Error("CreateOS transfer command failed."); + }; + const upload = async (local: string, remote: string) => { + const source = createReadStream(local); + try { + const init: RequestInit & { duplex: "half" } = { + method: "PUT", body: Readable.toWeb(source) as ReadableStream, + duplex: "half", headers: { "Content-Type": "application/octet-stream" }, signal, + }; + const response = await client.request(`/sandboxes/${id}/files?path=${encodeURIComponent(remote)}`, init); + await response.body?.cancel(); + } finally { source.destroy(); } + }; + const download = async (remote: string, local: string, mode = 0o600) => { + const response = await client.request(`/sandboxes/${id}/files?path=${encodeURIComponent(remote)}`, { signal }); + if (!response.body) throw new Error("CreateOS file download has no body."); + await pipeline(response.body, createWriteStream(local, { flags: "wx", mode }), { signal }); + }; + + // Validate every mapping before beginning side effects. Host paths are + // orchestrator-authored and checked by its source/target-root guard. + for (const operation of params.operations) { + for (const mapping of operation.files) { + if (!["file", "directory"].includes(mapping.kind)) throw new Error("Unsupported CreateOS transfer kind."); + if (!path.isAbsolute(direction === "in" ? mapping.sourcePath : mapping.targetPath)) throw new Error("CreateOS transfer requires an absolute host path."); + if (mapping.mode != null && (!Number.isInteger(mapping.mode) || mapping.mode < 0 || mapping.mode > 0o777)) throw new Error("Invalid CreateOS file mode."); + assertRemotePath(direction === "in" ? mapping.targetPath : mapping.sourcePath); + } + for (const command of operation.postUploadCommands ?? []) { + assertRemotePath(command.cwd ?? ROOT); + if (command.timeoutMs != null && (!Number.isInteger(command.timeoutMs) || command.timeoutMs < 1 || command.timeoutMs > 86_400_000)) throw new Error("Invalid CreateOS transfer timeout."); + } + if (direction === "out" && operation.postUploadCommands?.length) throw new Error("Outbound CreateOS transfers cannot run post-upload commands."); + } + + for (const operation of params.operations) { + let bytesTransferred = 0; + let filesTransferred = 0; + for (const mapping of operation.files) { + signal.throwIfAborted(); + const local = direction === "in" ? mapping.sourcePath : mapping.targetPath; + const remote = direction === "in" ? mapping.targetPath : mapping.sourcePath; + const scratch = `/tmp/paperclip-createos-transfer-${randomUUID()}`; + // Outbound temporary files are on the target filesystem for atomic rename. + const parent = direction === "out" ? path.dirname(local) : os.tmpdir(); + await fs.mkdir(parent, { recursive: true }); + const temp = await fs.mkdtemp(path.join(parent, ".paperclip-createos-")); + const transferFile = path.join(temp, "data"); + try { + if (direction === "in") { + let source = local; + if (mapping.kind === "directory") { + await tar.c({ file: transferFile, cwd: local, follow: mapping.followSymlinks === true, + filter: (name) => !excluded(name, mapping.exclude ?? []) }, ["."]); + source = transferFile; + } + await upload(source, scratch); + const mode = mapping.mode ?? (mapping.kind === "file" ? (await fs.stat(source)).mode & 0o777 : 0o755); + await run(mapping.kind === "file" + ? `${remoteGuard(remote)} && mkdir -p -- "$(dirname -- "$resolved")" && chmod ${mode.toString(8)} -- ${shellQuote(scratch)} && mv -f -- ${shellQuote(scratch)} "$resolved"` + : `${remoteGuard(remote)} && mkdir -p -- "$resolved" && tar -xf ${shellQuote(scratch)} -C "$resolved" && chmod ${mode.toString(8)} -- "$resolved"`); + filesTransferred += mapping.kind === "file" ? 1 : await countArchiveFiles(source); + bytesTransferred += (await fs.stat(source)).size; + } else { + const excludeArgs = (mapping.exclude ?? []).map((pattern) => `--exclude=${shellQuote(pattern)}`).join(" "); + await run(mapping.kind === "file" + ? `${remoteGuard(remote)} && test -f "$resolved" && cp -- "$resolved" ${shellQuote(scratch)}` + : `${remoteGuard(remote)} && test -d "$resolved" && tar ${mapping.followSymlinks ? "-h " : ""}${excludeArgs} -cf ${shellQuote(scratch)} -C "$resolved" .`); + await download(scratch, transferFile, mapping.mode ?? 0o600); + bytesTransferred += (await fs.stat(transferFile)).size; + if (mapping.kind === "file") { + // Apply exact requested mode before promotion, including under a + // restrictive umask. Never expose secret bytes at the final path first. + await fs.chmod(transferFile, mapping.mode ?? 0o600); + await fs.rename(transferFile, local); + filesTransferred++; + } else { + filesTransferred += await validateArchive(transferFile); + await fs.mkdir(local, { recursive: true, mode: mapping.mode ?? 0o700 }); + if ((await fs.lstat(local)).isSymbolicLink()) throw new Error("CreateOS archive destination cannot be a symlink."); + // tar rejects traversal through existing symlink parents. Validate + // all archive entries first so an unsafe archive never partly lands. + await tar.x({ file: transferFile, cwd: local, strict: true, preservePaths: false }); + if (mapping.mode != null) await fs.chmod(local, mapping.mode); + } + } + } finally { + await fs.rm(temp, { recursive: true, force: true }); + await client.json(`/sandboxes/${id}/exec`, "POST", { cmd: "/bin/rm", args: ["-f", "--", scratch] }).catch(() => undefined); + } + } + for (const command of operation.postUploadCommands ?? []) { + await run(remoteGuard(command.cwd ?? ROOT)); + await run(command.command, command.cwd ?? ROOT, command.timeoutMs); + } + operations.push({ operationId: operation.operationId, filesTransferred, bytesTransferred }); + } + return { operations }; +} + +async function countArchiveFiles(file: string): Promise { + let count = 0; + await tar.t({ file, onReadEntry(entry) { if (entry.type === "File" || entry.type === "OldFile") count++; } }); + return count; +} diff --git a/packages/plugins/sandbox-providers/createos/src/index.ts b/packages/plugins/sandbox-providers/createos/src/index.ts new file mode 100644 index 0000000000..f7ce1cc157 --- /dev/null +++ b/packages/plugins/sandbox-providers/createos/src/index.ts @@ -0,0 +1,2 @@ +export { default as manifest } from "./manifest.js"; +export { default as plugin } from "./plugin.js"; diff --git a/packages/plugins/sandbox-providers/createos/src/manifest.ts b/packages/plugins/sandbox-providers/createos/src/manifest.ts new file mode 100644 index 0000000000..6f719b0503 --- /dev/null +++ b/packages/plugins/sandbox-providers/createos/src/manifest.ts @@ -0,0 +1,46 @@ +import type { PaperclipPluginManifestV1 } from "@paperclipai/plugin-sdk"; + +const manifest: PaperclipPluginManifestV1 = { + id: "paperclip.createos-sandbox-provider", + apiVersion: 1, + version: "0.1.0", + displayName: "CreateOS Sandbox Provider", + description: "Runs Paperclip agents in CreateOS sandboxes.", + author: "CreateOS", + categories: ["automation"], + capabilities: ["environment.drivers.register"], + entrypoints: { worker: "./dist/worker.js" }, + environmentDrivers: [{ + driverKey: "createos", + kind: "sandbox_provider", + displayName: "CreateOS Sandbox", + description: "CreateOS sandboxes with live command output and optional pause/resume reuse.", + supportsReusableLeases: true, + sandboxCapabilities: { incrementalSessionOutput: true }, + configSchema: { + type: "object", + required: ["apiUrl", "shape"], + properties: { + apiUrl: { type: "string", title: "API URL", default: "https://api.sb.createos.sh", description: "https://api.sb.createos.sh" }, + apiKey: { type: "string", format: "secret-ref", description: "CreateOS API key or Paperclip secret reference. Saved keys become company secrets. The official API endpoint can use CREATEOS_API_KEY from the host; custom endpoints require an explicit key." }, + shape: { + type: "string", + title: "Shape", + description: "Choose the sandbox CPU and memory size.", + // Published catalog from https://api.sb.createos.sh/v1/shapes. + enum: [ + "s-1vcpu-256mb", "s-0.25vcpu-512mb", "s-0.5vcpu-1gb", + "s-1vcpu-1gb", "s-1vcpu-2gb", "s-2vcpu-2gb", "s-2vcpu-4gb", + "s-4vcpu-4gb", "s-4vcpu-8gb", "s-8vcpu-8gb", "s-8vcpu-16gb", + ], + }, + rootfs: { type: "string", description: "Root filesystem or ready template ID/name. Omit to use the provider default. The image must supply Bash and the selected agent runtime dependencies." }, + region: { type: "string", description: "Optional region; must match the API endpoint's region." }, + timeoutMs: { type: "integer", minimum: 1, maximum: 86400000, default: 300000, description: "Operation and default command timeout in milliseconds. This is not a sandbox lifetime." }, + reuseLease: { type: "boolean", default: false, description: "Pause the sandbox after a run and resume it for subsequent runs instead of deleting it." }, + }, + }, + }], +}; + +export default manifest; diff --git a/packages/plugins/sandbox-providers/createos/src/plugin.live.test.ts b/packages/plugins/sandbox-providers/createos/src/plugin.live.test.ts new file mode 100644 index 0000000000..d3574e55b9 --- /dev/null +++ b/packages/plugins/sandbox-providers/createos/src/plugin.live.test.ts @@ -0,0 +1,50 @@ +import { expect, it } from "vitest"; +import { createPlugin } from "./plugin.js"; +import { promises as fs } from "node:fs"; +import os from "node:os"; +import path from "node:path"; + +// Explicit opt-in: this creates a billable sandbox on the supplied endpoint. +// It never selects a region or discovers credentials automatically. +it.skipIf(process.env.CREATEOS_LIVE_TEST !== "1")("CreateOS create/exec/stdin/reuse/destroy live smoke", async () => { + const { CREATEOS_API_URL: apiUrl, CREATEOS_API_KEY: apiKey, CREATEOS_SHAPE: shape, CREATEOS_ROOTFS: rootfs } = process.env; + if (!apiUrl || !apiKey || !shape) throw new Error("Set CREATEOS_API_URL, CREATEOS_API_KEY, and CREATEOS_SHAPE for the live smoke."); + const hooks = createPlugin().definition; + const params = { + driverKey: "createos", companyId: "createos-plugin-smoke", environmentId: "createos-plugin-smoke", + config: { apiUrl, apiKey, shape, ...(rootfs ? { rootfs } : {}), reuseLease: true, timeoutMs: 120_000 }, + }; + const lease = await hooks.onEnvironmentAcquireLease!({ ...params, runId: "smoke" }); + let temp: string | null = null; + try { + temp = await fs.mkdtemp(path.join(os.tmpdir(), "createos-live-")); + const workspace = await hooks.onEnvironmentRealizeWorkspace!({ ...params, lease, workspace: {} }); + const large = Buffer.alloc(5 * 1024 * 1024, 0xa5); + const source = path.join(temp, "source"); + const target = path.join(temp, "download"); + await fs.writeFile(source, large); + await hooks.onEnvironmentSyncIn!({ ...params, lease, operations: [{ operationId: "smoke-in", files: [{ + sourcePath: source, targetPath: `${workspace.cwd}/large.bin`, kind: "file", mode: 0o600, + }] }] }); + await hooks.onEnvironmentSyncOut!({ ...params, lease, operations: [{ operationId: "smoke-out", files: [{ + sourcePath: `${workspace.cwd}/large.bin`, targetPath: target, kind: "file", mode: 0o600, + }] }] }); + expect((await fs.readFile(target)).equals(large)).toBe(true); + const written = await hooks.onEnvironmentExecute!({ + ...params, lease, command: "/bin/bash", args: ["-c", "cat > smoke.txt; printf '%s' \"$SMOKE\" >&2"], + stdin: "persisted\n", env: { SMOKE: "stderr works" }, cwd: workspace.cwd, + }); + expect(written).toMatchObject({ exitCode: 0, timedOut: false, stderr: "stderr works" }); + await hooks.onEnvironmentReleaseLease!({ ...params, providerLeaseId: lease.providerLeaseId, leaseMetadata: lease.metadata }); + const resumed = await hooks.onEnvironmentResumeLease!({ ...params, providerLeaseId: lease.providerLeaseId!, leaseMetadata: lease.metadata }); + expect(resumed.providerLeaseId).toBe(lease.providerLeaseId); + const read = await hooks.onEnvironmentExecute!({ ...params, lease: resumed, command: "/bin/cat", args: ["smoke.txt"], cwd: workspace.cwd }); + expect(read).toMatchObject({ exitCode: 0, timedOut: false, stdout: "persisted\n" }); + } finally { + try { await hooks.onEnvironmentDestroyLease!({ ...params, providerLeaseId: lease.providerLeaseId, leaseMetadata: lease.metadata }); } + finally { + await hooks.onShutdown!(); + if (temp) await fs.rm(temp, { recursive: true, force: true }); + } + } +}, 300_000); diff --git a/packages/plugins/sandbox-providers/createos/src/plugin.test.ts b/packages/plugins/sandbox-providers/createos/src/plugin.test.ts new file mode 100644 index 0000000000..d45d3a087c --- /dev/null +++ b/packages/plugins/sandbox-providers/createos/src/plugin.test.ts @@ -0,0 +1,336 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { createPlugin } from "./plugin.js"; +import { parseConfig } from "./config.js"; +import manifest from "./manifest.js"; +import { execute } from "./execute.js"; +import { CreateosClient } from "./client.js"; +vi.mock("./request-pacer.js", () => ({ waitForRequest: vi.fn().mockResolvedValue(undefined) })); + +const config = { apiUrl: "https://createos.example.test", apiKey: "test-secret", shape: "test-shape", timeoutMs: 5000 }; +const base = { driverKey: "createos", companyId: "company-a", environmentId: "env-a", config }; +const success = (data: unknown = {}) => Response.json({ status: "success", data }); +const failure = (status: number) => Response.json({ status: "fail", data: "private provider diagnostics test-secret" }, { status }); +const data = (seq: number, text: string | Buffer, stream = "stdout") => ({ type: "data", seq, stream, data_base64: Buffer.from(text).toString("base64") }); +const ndjson = (frames: unknown[]) => new Response(frames.map((frame) => JSON.stringify(frame)).join("\n") + "\n", { + headers: { "Content-Type": "application/x-ndjson" }, +}); + +type Call = { path: string; method: string; body: Record; init: RequestInit }; +function provider() { + const calls: Call[] = []; + let state = "running"; + let marker = ""; + const fetchMock = vi.fn(async (url: string | URL | Request, init: RequestInit = {}) => { + const parsed = new URL(String(url)); + const path = parsed.pathname + parsed.search; + const method = init.method ?? "GET"; + const raw = typeof init.body === "string" ? init.body : ""; + const body = raw.startsWith("{") ? JSON.parse(raw) : {}; + calls.push({ path, method, body, init }); + expect(init.headers).toMatchObject({ "X-Api-Key": "test-secret" }); + expect(init.redirect).toBe("error"); + if (path === "/v1/sandboxes" && method === "POST") return success({ id: "sb_test" }); + if (path === "/v1/sandboxes/sb_test" && method === "GET") return success({ id: "sb_test", status: state }); + if (path === "/v1/sandboxes/sb_test" && method === "DELETE") { state = "destroyed"; return success({ id: "sb_test" }); } + if (path.endsWith("/pause")) { state = "paused"; return success({ status: "pausing" }); } + if (path.endsWith("/resume")) { state = "running"; return success({ status: "resuming" }); } + if (parsed.pathname.endsWith("/files") && method === "PUT") { + if (parsed.searchParams.get("path")?.endsWith(".paperclip-createos-lease")) marker = raw; + return success(); + } + if (path.endsWith("/exec")) return success({ result: { exit_code: 0, stdout: body.cmd === "/bin/cat" ? marker : "", stderr: "" } }); + if (path.endsWith("/processes") && method === "POST") { + // These sandboxes declare no creation-time env keys. Match the real + // provider's rejection of undeclared API-level overrides. + if (Object.keys((body.env ?? {}) as object).length) return failure(400); + return success({ process_id: "proc_test" }); + } + if (path.endsWith("/stdin/close")) return success(); + if (parsed.pathname.endsWith("/connect")) return ndjson([data(1, "paperclip-createos-ready\n"), { type: "exit", exit_code: 0 }]); + if (method === "DELETE" && path.includes("/processes/")) return success({ tree_exited: true }); + throw new Error(`Unhandled fixture request ${method} ${path}`); + }); + vi.stubGlobal("fetch", fetchMock); + return { calls, fetchMock, setState: (value: string) => { state = value; }, setMarker: (value: string) => { marker = value; } }; +} + +function executionParams(overrides = {}) { + return { ...base, lease: { providerLeaseId: "sb_test" }, command: "echo", args: ["hello"], ...overrides }; +} + +beforeEach(() => { vi.stubEnv("CREATEOS_API_KEY", ""); }); +afterEach(() => { vi.unstubAllGlobals(); vi.unstubAllEnvs(); }); + +describe("CreateOS lifecycle", () => { + it("acquires, executes, realizes, pauses, resumes, and destroys using public API envelopes", async () => { + const fake = provider(); + const hooks = createPlugin().definition; + const params = { ...base, config: { ...config, reuseLease: true, rootfs: "tpl_ready", region: "us" } }; + const lease = await hooks.onEnvironmentAcquireLease!({ ...params, runId: "run-a" }); + expect(fake.calls[0].body).toEqual({ shape: "test-shape", rootfs: "tpl_ready", region: "us", ingress_enabled: false }); + expect(JSON.stringify(lease)).not.toContain("test-secret"); + expect(await hooks.onEnvironmentRealizeWorkspace!({ ...params, lease, workspace: { localPath: "/private/host/path" } })).toMatchObject({ cwd: "/paperclip-workspace" }); + expect(await hooks.onEnvironmentExecute!({ ...params, lease, command: "echo" })).toMatchObject({ exitCode: 0, timedOut: false }); + await hooks.onEnvironmentReleaseLease!({ ...params, providerLeaseId: lease.providerLeaseId, leaseMetadata: lease.metadata }); + expect(fake.calls.some((call) => call.path.endsWith("/pause"))).toBe(true); + expect(await hooks.onEnvironmentResumeLease!({ ...params, providerLeaseId: lease.providerLeaseId!, leaseMetadata: lease.metadata })).toMatchObject({ providerLeaseId: "sb_test" }); + await hooks.onEnvironmentDestroyLease!({ ...params, providerLeaseId: lease.providerLeaseId, leaseMetadata: lease.metadata }); + expect(fake.calls.at(-1)).toMatchObject({ method: "DELETE", path: "/v1/sandboxes/sb_test" }); + }); + + it("cleans up failed acquisition and never hides delete failure", async () => { + const fake = provider(); + const normal = fake.fetchMock.getMockImplementation()!; + fake.fetchMock.mockImplementation(async (url, init) => String(url).endsWith("/exec") ? failure(503) : normal(url, init)); + await expect(createPlugin().definition.onEnvironmentAcquireLease!({ ...base, runId: "run" })).rejects.toThrow("HTTP 503"); + expect(fake.calls.at(-1)?.method).toBe("DELETE"); + fake.fetchMock.mockImplementation(async (url, init) => init?.method === "DELETE" ? failure(500) : String(url).endsWith("/exec") ? failure(503) : normal(url, init)); + await expect(createPlugin().definition.onEnvironmentAcquireLease!({ ...base, runId: "run" })).rejects.toThrow("cleanup is unconfirmed"); + }); + + it("rejects guaranteed-expiry leases before provisioning", async () => { + const fake = provider(); + await expect(createPlugin().definition.onEnvironmentAcquireLease!({ ...base, runId: "login", requestedExpiresAt: "2026-09-08T12:00:00Z" })).rejects.toThrow("guaranteed expiration"); + expect(fake.fetchMock).not.toHaveBeenCalled(); + }); + + it("does not execute, resume, or destroy another company's recorded lease", async () => { + const fake = provider(); + const hooks = createPlugin().definition; + const lease = await hooks.onEnvironmentAcquireLease!({ ...base, runId: "run" }); + fake.fetchMock.mockClear(); + const other = { ...base, companyId: "company-b" }; + await expect(hooks.onEnvironmentExecute!({ ...other, lease, command: "echo" })).rejects.toThrow("this environment"); + await expect(hooks.onEnvironmentResumeLease!({ ...other, providerLeaseId: "sb_test", leaseMetadata: lease.metadata })).rejects.toThrow("this environment"); + await expect(hooks.onEnvironmentDestroyLease!({ ...other, providerLeaseId: "sb_test", leaseMetadata: lease.metadata })).rejects.toThrow("this environment"); + expect(fake.fetchMock).not.toHaveBeenCalled(); + }); + + it("expires a missing or mismatched workspace but preserves transient resume failures", async () => { + const fake = provider(); + const hooks = createPlugin().definition; + const lease = await hooks.onEnvironmentAcquireLease!({ ...base, runId: "run" }); + fake.setMarker("wrong"); + const params = { ...base, providerLeaseId: "sb_test", leaseMetadata: lease.metadata }; + expect(await hooks.onEnvironmentResumeLease!(params)).toMatchObject({ providerLeaseId: null }); + fake.fetchMock.mockResolvedValueOnce(failure(404)); + expect(await hooks.onEnvironmentResumeLease!(params)).toMatchObject({ providerLeaseId: null }); + fake.fetchMock.mockResolvedValueOnce(failure(503)); + await expect(hooks.onEnvironmentResumeLease!(params)).rejects.toThrow("HTTP 503"); + expect(fake.calls.some((call) => call.method === "DELETE")).toBe(false); + }); + + it("does not POST pause/resume when already in the desired state", async () => { + const fake = provider(); + const client = new CreateosClient(parseConfig(config)); + await client.transition("sb_test", "running", AbortSignal.timeout(5000)); + fake.setState("paused"); + await client.transition("sb_test", "paused", AbortSignal.timeout(5000)); + expect(fake.calls.every((call) => call.method === "GET")).toBe(true); + }); + + it("expires a reusable lease when its requested image or shape changes", async () => { + const fake = provider(); + const hooks = createPlugin().definition; + const lease = await hooks.onEnvironmentAcquireLease!({ ...base, runId: "run" }); + fake.fetchMock.mockClear(); + for (const changed of [{ shape: "larger" }, { rootfs: "new-template" }, { region: "other" }]) { + expect(await hooks.onEnvironmentResumeLease!({ + ...base, config: { ...config, ...changed }, providerLeaseId: "sb_test", leaseMetadata: lease.metadata, + })).toMatchObject({ providerLeaseId: null }); + } + expect(fake.fetchMock).not.toHaveBeenCalled(); + }); + + it("probes actual process execution and always deletes the sandbox", async () => { + const fake = provider(); + expect(await createPlugin().definition.onEnvironmentProbe!(base)).toMatchObject({ ok: true }); + expect(fake.calls.at(-1)).toMatchObject({ method: "DELETE", path: "/v1/sandboxes/sb_test" }); + }); + + it("aborts an active command and waits for tree cleanup before pausing", async () => { + const fake = provider(); + const hooks = createPlugin().definition; + const params = { ...base, config: { ...config, reuseLease: true } }; + const lease = await hooks.onEnvironmentAcquireLease!({ ...params, runId: "run" }); + const normal = fake.fetchMock.getMockImplementation()!; + let streaming!: () => void; + const ready = new Promise((resolve) => { streaming = resolve; }); + fake.fetchMock.mockImplementation(async (url, init) => { + if (String(url).includes("/connect?")) return new Response(new ReadableStream({ + start(controller) { + init!.signal!.addEventListener("abort", () => controller.error(init!.signal!.reason), { once: true }); + streaming(); + }, + })); + return normal(url, init); + }); + const command = hooks.onEnvironmentExecute!({ ...params, lease, command: "sleep", args: ["60"] }); + const rejected = expect(command).rejects.toThrow("cancelled"); + await ready; + await hooks.onEnvironmentReleaseLease!({ ...params, providerLeaseId: "sb_test", leaseMetadata: lease.metadata }); + await rejected; + const stop = fake.calls.findIndex((call) => call.method === "DELETE" && call.path.includes("/processes/")); + const pause = fake.calls.findIndex((call) => call.path.endsWith("/pause")); + expect(stop).toBeGreaterThan(-1); + expect(pause).toBeGreaterThan(stop); + }); + + it("blocks reuse after ambiguous process creation until the sandbox is destroyed", async () => { + const fake = provider(); + const hooks = createPlugin().definition; + const params = { ...base, config: { ...config, reuseLease: true } }; + const lease = await hooks.onEnvironmentAcquireLease!({ ...params, runId: "run" }); + const normal = fake.fetchMock.getMockImplementation()!; + fake.fetchMock.mockImplementation(async (url, init) => String(url).endsWith("/processes") ? failure(502) : normal(url, init)); + await expect(hooks.onEnvironmentExecute!({ ...params, lease, command: "sleep" })).rejects.toThrow("creation could not be confirmed"); + await expect(hooks.onEnvironmentReleaseLease!({ ...params, providerLeaseId: "sb_test", leaseMetadata: lease.metadata })).rejects.toThrow("cleanup is unconfirmed"); + expect(fake.calls.some((call) => call.path.endsWith("/pause"))).toBe(false); + await hooks.onEnvironmentDestroyLease!({ ...params, providerLeaseId: "sb_test", leaseMetadata: lease.metadata }); + expect(fake.calls.at(-1)).toMatchObject({ method: "DELETE", path: "/v1/sandboxes/sb_test" }); + }); + + it("waits for asynchronous provider transitions rather than treating 202 as completion", async () => { + const fake = provider(); + const normal = fake.fetchMock.getMockImplementation()!; + let reads = 0; + fake.fetchMock.mockImplementation(async (url, init) => { + if (String(url).endsWith("/sb_test") && (!init?.method || init.method === "GET")) { + reads++; + return success({ id: "sb_test", status: reads <= 2 ? "paused" : reads === 3 ? "resuming" : "running" }); + } + return normal(url, init); + }); + await new CreateosClient(parseConfig(config)).transition("sb_test", "running", AbortSignal.timeout(5000)); + expect(reads).toBe(4); + expect(fake.calls.filter((call) => call.path.endsWith("/resume"))).toHaveLength(1); + }); +}); + +describe("CreateOS command protocol", () => { + it("quotes hostile args/env/cwd and stages stdin instead of racing the input endpoint", async () => { + const fake = provider(); + const result = await execute(new CreateosClient(parseConfig(config)), executionParams({ + command: "printf", args: ["a'; touch /bad; echo '"], cwd: "/work dir", + env: { MESSAGE: "$(touch /bad)" }, stdin: "secret input\n", + }), AbortSignal.timeout(5000)); + expect(result.exitCode).toBe(0); + const create = fake.calls.find((call) => call.path.endsWith("/processes"))!; + expect(create.body).toMatchObject({ cmd: "/bin/bash", cwd: "/work dir" }); + expect(create.body).not.toHaveProperty("env"); + expect((create.body.args as string[])[1]).toContain("MESSAGE='$(touch /bad)'"); + expect((create.body.args as string[])[1]).toContain("'a'\"'\"'; touch /bad; echo '\"'\"''"); + expect((create.body.args as string[])[1]).not.toContain("secret input"); + expect(fake.calls.some((call) => call.path.endsWith("/input"))).toBe(false); + expect(fake.calls.at(-1)?.body).toMatchObject({ cmd: "/bin/rm" }); + }); + + it("replays the same process after disconnect without duplicate logs, preserving split UTF-8", async () => { + const fake = provider(); + const normal = fake.fetchMock.getMockImplementation()!; + let connects = 0; + const euro = Buffer.from("€"); + fake.fetchMock.mockImplementation(async (url, init) => { + if (String(url).includes("/connect?")) { + connects++; + if (connects === 1) return ndjson([data(1, euro.subarray(0, 1))]); + expect(String(url)).toContain("after=1"); + return ndjson([data(1, euro.subarray(0, 1)), data(2, euro.subarray(1)), data(3, "warning", "stderr"), { type: "exit", exit_code: 7 }]); + } + return normal(url, init); + }); + const log = vi.fn(); + const result = await execute(new CreateosClient(parseConfig(config)), executionParams(), AbortSignal.timeout(5000), log); + expect(result).toMatchObject({ exitCode: 7, stdout: "€", stderr: "warning", timedOut: false }); + expect(log.mock.calls).toEqual([["stdout", "€"], ["stderr", "warning"]]); + expect(fake.calls.filter((call) => call.path.endsWith("/processes") && call.method === "POST")).toHaveLength(1); + }); + + it.each([ + ["gap", () => ndjson([data(2, "lost")])], + ["expired", () => failure(410)], + ["stream error", () => ndjson([{ type: "error", error: "output_offset_expired" }])], + ["malformed", () => new Response("{broken\n")], + ["missing exit status", () => ndjson([{ type: "exit" }])], + ])("terminates the process tree on %s output", async (_name, response) => { + const fake = provider(); + const normal = fake.fetchMock.getMockImplementation()!; + fake.fetchMock.mockImplementation(async (url, init) => String(url).includes("/connect?") ? response() : normal(url, init)); + await expect(execute(new CreateosClient(parseConfig(config)), executionParams(), AbortSignal.timeout(5000))).rejects.toThrow(); + expect(fake.calls.some((call) => call.method === "DELETE" && call.path.includes("/processes/proc_test?grace_ms=1000"))).toBe(true); + }); + + it("enforces timeout and preserves partial output while cleanup uses a fresh deadline", async () => { + const fake = provider(); + const normal = fake.fetchMock.getMockImplementation()!; + fake.fetchMock.mockImplementation(async (url, init) => { + if (String(url).includes("/connect?")) return new Response(new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode(JSON.stringify(data(1, "partial")) + "\n")); + init!.signal!.addEventListener("abort", () => controller.error(init!.signal!.reason), { once: true }); + }, + })); + if (init?.method === "DELETE") expect(init.signal?.aborted).toBe(false); + return normal(url, init); + }); + const result = await execute(new CreateosClient(parseConfig(config)), executionParams(), AbortSignal.timeout(50)); + expect(result).toMatchObject({ exitCode: null, timedOut: true, stdout: "partial" }); + expect(fake.calls.at(-1)?.method).toBe("DELETE"); + }); + + it("reports unconfirmed termination rather than hiding cleanup failure", async () => { + const fake = provider(); + const normal = fake.fetchMock.getMockImplementation()!; + fake.fetchMock.mockImplementation(async (url, init) => init?.method === "DELETE" ? failure(503) : String(url).includes("/connect?") ? ndjson([{ type: "error" }]) : normal(url, init)); + await expect(execute(new CreateosClient(parseConfig(config)), executionParams(), AbortSignal.timeout(5000))).rejects.toThrow("termination is unconfirmed"); + }); + + it("rejects invalid env names before sending a command", async () => { + const fake = provider(); + await expect(execute(new CreateosClient(parseConfig(config)), executionParams({ env: { "BAD;KEY": "x" } }), AbortSignal.timeout(5000))).rejects.toThrow("Invalid sandbox environment"); + expect(fake.fetchMock).not.toHaveBeenCalled(); + }); +}); + +describe("CreateOS configuration", () => { + it("normalizes /v1 URLs and keeps the shape explicit", () => { + expect(parseConfig({ ...config, apiUrl: config.apiUrl + "/v1/" }).apiUrl).toBe(config.apiUrl); + expect(() => parseConfig({ ...config, shape: "" })).toThrow("shape"); + expect(manifest.environmentDrivers?.[0].supportsLoginPty).not.toBe(true); + }); + + it.each(["https://user:secret@host.test", "https://host.test/?key=secret", "http://remote.test", "file:///tmp/socket", "https://host.test/other"])("rejects unsafe API URL %s", (apiUrl) => { + expect(() => parseConfig({ ...config, apiUrl })).toThrow("API URL"); + }); + + it("supports explicitly configured fixture keys and an official-endpoint host fallback", () => { + vi.stubEnv("CREATEOS_API_KEY", "fallback"); + expect(new CreateosClient(parseConfig({ ...config, apiUrl: "http://127.0.0.1:3109" })).apiKey).toBe("test-secret"); + expect(new CreateosClient(parseConfig({ ...config, apiUrl: "https://api.sb.createos.sh/v1/", apiKey: undefined })).apiKey).toBe("fallback"); + expect(new CreateosClient(parseConfig(config)).apiKey).toBe("test-secret"); + }); + + it.each([ + "https://custom.example.test", + "https://api.sb.createos.sh.attacker.test", + "https://api.sb.createos.sh:8443", + "http://127.0.0.1:3109", + ])("never sends the deployment fallback key to custom endpoint %s", async (apiUrl) => { + vi.stubEnv("CREATEOS_API_KEY", "host-secret"); + const fake = provider(); + await expect(createPlugin().definition.onEnvironmentAcquireLease!({ + ...base, config: { ...config, apiUrl, apiKey: undefined }, runId: "run", + })).rejects.toThrow("require an explicit environment API key"); + expect(fake.fetchMock).not.toHaveBeenCalled(); + }); + + it.each([0, -1, NaN, 1.5, 86_400_001, "500"])("rejects invalid timeout %s", (timeoutMs) => { + expect(() => parseConfig({ ...config, timeoutMs })).toThrow("timeoutMs"); + }); + + it("does not expose provider error bodies", async () => { + vi.stubGlobal("fetch", vi.fn().mockResolvedValue(failure(401))); + await expect(new CreateosClient(parseConfig(config)).getSandbox("sb_test")).rejects.toThrow("HTTP 401"); + await expect(new CreateosClient(parseConfig(config)).getSandbox("sb_test")).rejects.not.toThrow("test-secret"); + }); +}); diff --git a/packages/plugins/sandbox-providers/createos/src/plugin.ts b/packages/plugins/sandbox-providers/createos/src/plugin.ts new file mode 100644 index 0000000000..20f2f9d488 --- /dev/null +++ b/packages/plugins/sandbox-providers/createos/src/plugin.ts @@ -0,0 +1,201 @@ +import { randomUUID, createHash } from "node:crypto"; +import { definePlugin } from "@paperclipai/plugin-sdk"; +import type { + PluginContext, PluginEnvironmentAcquireLeaseParams, PluginEnvironmentDriverBaseParams, + PluginEnvironmentExecuteParams, PluginEnvironmentLease, PluginEnvironmentReleaseLeaseParams, +} from "@paperclipai/plugin-sdk"; +import { CreateosApiError, CreateosClient, object } from "./client.js"; +import { parseConfig, resolveApiKey } from "./config.js"; +import { CreateosCleanupError, execute, shellQuote } from "./execute.js"; +import { syncFiles } from "./file-sync.js"; + +const CWD = "/paperclip-workspace"; +// The host excludes .paperclip-runtime from workspace export, so the lease +// marker never becomes a user repository file. +const MARKER = `${CWD}/.paperclip-runtime/.paperclip-createos-lease`; + +function metadataMatches(params: PluginEnvironmentDriverBaseParams, metadata?: Record): boolean { + return metadata?.provider === "createos" && metadata.companyId === params.companyId && + metadata.environmentId === params.environmentId && metadata.apiUrl === parseConfig(params.config).apiUrl; +} + +async function acquire(params: PluginEnvironmentAcquireLeaseParams): Promise { + // An idle timeout or a host-local timer cannot supply a provider expiry. + if (params.requestedExpiresAt) throw new Error("CreateOS does not yet support leases with a guaranteed expiration deadline."); + const config = parseConfig(params.config); + const client = new CreateosClient(config); + const signal = AbortSignal.timeout(config.timeoutMs); + const sandbox = await client.createSandbox(signal); + try { + await client.transition(sandbox.id, "running", signal); + const data = await client.json(`/sandboxes/${sandbox.id}/exec`, "POST", { + cmd: "/bin/bash", args: ["-lc", `mkdir -p -- ${shellQuote(CWD)}`], + }, signal); + if (object(data.result).exit_code !== 0) throw new Error("CreateOS workspace preparation failed; the image must provide Bash."); + const marker = randomUUID(); + await client.upload(sandbox.id, MARKER, marker, signal); + return { + providerLeaseId: sandbox.id, + metadata: { + provider: "createos", apiUrl: config.apiUrl, + companyId: params.companyId, environmentId: params.environmentId, + remoteCwd: CWD, shellCommand: "bash", marker, + shape: config.shape, rootfs: config.rootfs, region: config.region, + reuseLease: config.reuseLease, + }, + }; + } catch (error) { + try { await client.destroySandbox(sandbox.id); } + catch { throw new Error(`CreateOS setup failed and cleanup is unconfirmed for sandbox ${sandbox.id}.`); } + throw error; + } +} + +// Each worker owns its own transient lifecycle state. The host owns durable leases. +export function createPlugin() { + let ctx: PluginContext | null = null; + let shuttingDown = false; + type Active = { controller: AbortController; done: Promise }; + const active = new Map>(); + const closing = new Set(); + const unconfirmedCleanup = new Set(); + + function key(params: PluginEnvironmentDriverBaseParams, id: string): string { + const config = parseConfig(params.config); + const account = createHash("sha256").update(resolveApiKey(config)).digest("hex"); + return JSON.stringify([params.companyId, params.environmentId, config.apiUrl, account, id]); + } + + async function stopActive(scope: string) { + const calls = [...(active.get(scope) ?? [])]; + for (const call of calls) call.controller.abort(); + await Promise.all(calls.map((call) => call.done)); + } + + async function track( + params: PluginEnvironmentDriverBaseParams & { lease: PluginEnvironmentLease }, + work: (client: CreateosClient, signal: AbortSignal) => Promise, + timeoutOverride?: number, + ): Promise { + if (!params.lease.providerLeaseId || !metadataMatches(params, params.lease.metadata)) throw new Error("CreateOS execution requires a lease from this environment."); + const scope = key(params, params.lease.providerLeaseId); + if (shuttingDown || closing.has(scope) || unconfirmedCleanup.has(scope)) throw new Error("CreateOS lease is closing or requires cleanup."); + const config = parseConfig(params.config); + const timeoutMs = timeoutOverride ?? config.timeoutMs; + if (!Number.isInteger(timeoutMs) || timeoutMs < 1 || timeoutMs > 86_400_000) throw new Error("Invalid CreateOS command timeout."); + const controller = new AbortController(); + let finish!: () => void; + const entry: Active = { controller, done: new Promise((resolve) => { finish = resolve; }) }; + const calls = active.get(scope) ?? new Set(); + calls.add(entry); + active.set(scope, calls); + try { + return await work(new CreateosClient(config), AbortSignal.any([controller.signal, AbortSignal.timeout(timeoutMs)])); + } catch (error) { + if (error instanceof CreateosCleanupError) unconfirmedCleanup.add(scope); + throw error; + } finally { + calls.delete(entry); + if (calls.size === 0) active.delete(scope); + finish(); + } + } + + async function release(params: PluginEnvironmentReleaseLeaseParams, destroy: boolean) { + const id = params.providerLeaseId; + if (!id) return; + if (!metadataMatches(params, params.leaseMetadata)) throw new Error("CreateOS lease does not belong to this environment."); + const scope = key(params, id); + if (closing.has(scope)) throw new Error("CreateOS lease cleanup is already in progress."); + closing.add(scope); + try { + await stopActive(scope); + const config = parseConfig(params.config); + const client = new CreateosClient(config); + if (destroy || !config.reuseLease) { + await client.destroySandbox(id); + unconfirmedCleanup.delete(scope); + } else { + if (unconfirmedCleanup.has(scope)) throw new Error("CreateOS process cleanup is unconfirmed; destroy this lease before reusing it."); + try { await client.transition(id, "paused", AbortSignal.timeout(config.timeoutMs)); } + catch (error) { if (!(error instanceof CreateosApiError && error.status === 404)) throw error; } + } + } finally { + closing.delete(scope); + } + } + + return definePlugin({ + async setup(context) { ctx = context; ctx.logger.info("CreateOS sandbox provider ready"); }, + async onHealth() { return { status: "ok", message: "CreateOS provider loaded; probe an environment to check connectivity." }; }, + async onEnvironmentValidateConfig(params) { + try { return { ok: true, normalizedConfig: { ...parseConfig(params.config) } }; } + catch (error) { return { ok: false, errors: [error instanceof Error ? error.message : "Invalid CreateOS configuration."] }; } + }, + async onEnvironmentProbe(params) { + let lease: PluginEnvironmentLease | null = null; + try { + lease = await acquire({ ...params, runId: "probe" }); + const result = await execute(new CreateosClient(parseConfig(params.config)), { + ...params, lease, command: "/bin/echo", args: ["paperclip-createos-ready"], cwd: CWD, + }, AbortSignal.timeout(parseConfig(params.config).timeoutMs)); + if (result.timedOut || result.exitCode !== 0 || !result.stdout.includes("paperclip-createos-ready")) throw new Error("CreateOS command probe failed."); + return { ok: true, summary: "CreateOS sandbox creation and command execution succeeded." }; + } catch (error) { + return { ok: false, summary: error instanceof Error ? error.message : "CreateOS probe failed." }; + } finally { + // Never leave a reusable probe sandbox behind or hide deletion failure. + if (lease?.providerLeaseId) await new CreateosClient(parseConfig(params.config)).destroySandbox(lease.providerLeaseId); + } + }, + onEnvironmentAcquireLease: acquire, + async onEnvironmentResumeLease(params) { + if (!metadataMatches(params, params.leaseMetadata)) throw new Error("CreateOS lease does not belong to this environment."); + const scope = key(params, params.providerLeaseId); + if (closing.has(scope) || unconfirmedCleanup.has(scope)) throw new Error("CreateOS lease cleanup must finish before resume."); + const marker = params.leaseMetadata?.marker; + if (typeof marker !== "string" || !/^[0-9a-f-]{36}$/.test(marker)) return { providerLeaseId: null, metadata: { expired: true } }; + const config = parseConfig(params.config); + if (params.leaseMetadata?.shape !== config.shape || params.leaseMetadata.rootfs !== config.rootfs || params.leaseMetadata.region !== config.region) { + return { providerLeaseId: null, metadata: { expired: true } }; + } + const client = new CreateosClient(config); + const signal = AbortSignal.timeout(config.timeoutMs); + try { + const sandbox = await client.getSandbox(params.providerLeaseId, signal); + if (["destroyed", "failed"].includes(sandbox.status!)) return { providerLeaseId: null, metadata: { expired: true } }; + await client.transition(params.providerLeaseId, "running", signal); + const data = await client.json(`/sandboxes/${params.providerLeaseId}/exec`, "POST", { + cmd: "/bin/cat", args: [MARKER], + }, signal); + const result = object(data.result); + if (result.exit_code !== 0 || result.stdout !== marker) return { providerLeaseId: null, metadata: { expired: true } }; + return { providerLeaseId: params.providerLeaseId, metadata: { ...params.leaseMetadata, resumedLease: true } }; + } catch (error) { + if (error instanceof CreateosApiError && error.status === 404) return { providerLeaseId: null, metadata: { expired: true } }; + // A transient error does not prove the original sandbox is lost. + throw error; + } + }, + onEnvironmentReleaseLease: (params) => release(params, false), + onEnvironmentDestroyLease: (params) => release(params, true), + async onEnvironmentRealizeWorkspace(params) { + if (!params.lease.providerLeaseId || !metadataMatches(params, params.lease.metadata)) throw new Error("CreateOS workspace requires a lease from this environment."); + // The runtime's source mappings stage into this provider workspace. + return { cwd: CWD, metadata: { provider: "createos", remoteCwd: CWD } }; + }, + async onEnvironmentExecute(params: PluginEnvironmentExecuteParams) { + return track(params, (client, signal) => execute(client, params, signal, + (stream, text) => ctx?.execution.log(stream, text)), params.timeoutMs); + }, + onEnvironmentSyncIn: (params) => track(params, (client, signal) => syncFiles(client, params, "in", signal)), + onEnvironmentSyncOut: (params) => track(params, (client, signal) => syncFiles(client, params, "out", signal)), + async onShutdown() { + shuttingDown = true; + await Promise.all([...active.keys()].map(stopActive)); + ctx = null; + }, + }); +} + +export default createPlugin(); diff --git a/packages/plugins/sandbox-providers/createos/src/request-pacer.test.ts b/packages/plugins/sandbox-providers/createos/src/request-pacer.test.ts new file mode 100644 index 0000000000..4762658727 --- /dev/null +++ b/packages/plugins/sandbox-providers/createos/src/request-pacer.test.ts @@ -0,0 +1,38 @@ +import { expect, it, vi } from "vitest"; +import { waitForRequest } from "./request-pacer.js"; + +it("spaces concurrent requests to one endpoint across clients", async () => { + const started: number[] = []; + const requests = [1, 2, 3].map(() => waitForRequest("https://pacing.test", new AbortController().signal) + .then(() => started.push(Date.now()))); + await Promise.all(requests); + expect(started).toHaveLength(3); + expect(started[1] - started[0]).toBeGreaterThanOrEqual(280); + expect(started[2] - started[1]).toBeGreaterThanOrEqual(280); +}); + +it("an aborted queued request does not block cleanup", async () => { + await waitForRequest("https://cancel.test", new AbortController().signal); + const controller = new AbortController(); + controller.abort(new Error("cancelled")); + await expect(waitForRequest("https://cancel.test", controller.signal)).rejects.toThrow("cancelled"); + const cleanup = waitForRequest("https://cancel.test", new AbortController().signal); + await cleanup; +}); + +it("cancels promptly behind other queued requests without disturbing their spacing", async () => { + await waitForRequest("https://queued-cancel.test", new AbortController().signal); + const earlierDone = vi.fn(); + const earlier = waitForRequest("https://queued-cancel.test", new AbortController().signal).then(earlierDone); + const controller = new AbortController(); + const queued = waitForRequest("https://queued-cancel.test", controller.signal); + const rejected = expect(queued).rejects.toThrow("cancelled"); + controller.abort(new Error("cancelled")); + try { + await rejected; + expect(earlierDone).not.toHaveBeenCalled(); + } finally { + await earlier; + } + await waitForRequest("https://queued-cancel.test", new AbortController().signal); +}); diff --git a/packages/plugins/sandbox-providers/createos/src/request-pacer.ts b/packages/plugins/sandbox-providers/createos/src/request-pacer.ts new file mode 100644 index 0000000000..66a6f44d34 --- /dev/null +++ b/packages/plugins/sandbox-providers/createos/src/request-pacer.ts @@ -0,0 +1,39 @@ +import { createHash } from "node:crypto"; +import { setTimeout as delay } from "node:timers/promises"; + +// The public API allows 300 requests/minute per caller IP. Callback polling +// uses several requests per command. Keep this worker below that limit and +// share the budget across leases, command clients, and cleanup clients. +const accounts = new Map; lastStarted: number }>(); +const INTERVAL_MS = 300; + +export async function waitForRequest(apiUrl: string, signal: AbortSignal): Promise { + const key = createHash("sha256").update(apiUrl).digest("hex"); + let account = accounts.get(key); + if (!account) { + account = { tail: Promise.resolve(), lastStarted: 0 }; + accounts.set(key, account); + } + const state = account; + const turn = state.tail.catch(() => {}).then(async () => { + signal.throwIfAborted(); + const remaining = state.lastStarted + INTERVAL_MS - Date.now(); + if (remaining > 0) await delay(remaining, undefined, { signal }); + signal.throwIfAborted(); + state.lastStarted = Date.now(); + }); + state.tail = turn; + // A cancelled caller must not wait for every earlier queued request's slot. + // Keep its turn in the queue so the remaining requests still stay serialized. + let onAbort!: () => void; + const aborted = new Promise((_resolve, reject) => { + onAbort = () => reject(signal.reason); + signal.addEventListener("abort", onAbort, { once: true }); + if (signal.aborted) onAbort(); + }); + try { + await Promise.race([turn, aborted]); + } finally { + signal.removeEventListener("abort", onAbort); + } +} diff --git a/packages/plugins/sandbox-providers/createos/src/worker.ts b/packages/plugins/sandbox-providers/createos/src/worker.ts new file mode 100644 index 0000000000..1e156024b7 --- /dev/null +++ b/packages/plugins/sandbox-providers/createos/src/worker.ts @@ -0,0 +1,5 @@ +import { runWorker } from "@paperclipai/plugin-sdk"; +import plugin from "./plugin.js"; + +export default plugin; +runWorker(plugin, import.meta.url); diff --git a/packages/plugins/sandbox-providers/createos/tsconfig.json b/packages/plugins/sandbox-providers/createos/tsconfig.json new file mode 100644 index 0000000000..000e3293ef --- /dev/null +++ b/packages/plugins/sandbox-providers/createos/tsconfig.json @@ -0,0 +1,11 @@ +{ + "extends": "../../../../tsconfig.json", + "compilerOptions": { + "outDir": "dist", + "rootDir": "src", + "lib": ["ES2023"], + "types": ["node"] + }, + "include": ["src"], + "exclude": ["src/**/*.test.ts"] +} diff --git a/packages/plugins/sandbox-providers/createos/vitest.config.ts b/packages/plugins/sandbox-providers/createos/vitest.config.ts new file mode 100644 index 0000000000..ce36a74267 --- /dev/null +++ b/packages/plugins/sandbox-providers/createos/vitest.config.ts @@ -0,0 +1,8 @@ +import { defineConfig } from "vitest/config"; + +export default defineConfig({ + test: { + include: ["src/**/*.test.ts"], + environment: "node", + }, +}); diff --git a/scripts/release-package-manifest.json b/scripts/release-package-manifest.json index b13049d2bd..ac73f9f675 100644 --- a/scripts/release-package-manifest.json +++ b/scripts/release-package-manifest.json @@ -129,6 +129,11 @@ "name": "@paperclipai/plugin-cloudflare-sandbox", "publishFromCi": true }, + { + "dir": "packages/plugins/sandbox-providers/createos", + "name": "@paperclipai/plugin-createos", + "publishFromCi": false + }, { "dir": "packages/plugins/sandbox-providers/daytona", "name": "@paperclipai/plugin-daytona", diff --git a/server/src/__tests__/bundled-plugins.test.ts b/server/src/__tests__/bundled-plugins.test.ts index 6dbc45f522..0015010b7e 100644 --- a/server/src/__tests__/bundled-plugins.test.ts +++ b/server/src/__tests__/bundled-plugins.test.ts @@ -31,6 +31,18 @@ afterAll(() => { // --------------------------------------------------------------------------- describe("resolveBundledPluginInstalls", () => { + it("resolves the optional CreateOS sandbox provider without making it a self-hosted default", () => { + expect(resolveBundledPluginInstalls(["createos"], { + catalogRoot: CATALOG_ROOT, + env: {}, + enforceCatalogRoot: true, + })).toEqual([{ + key: "createos", + pluginKey: "paperclip.createos-sandbox-provider", + localPath: path.join(CATALOG_ROOT, "sandbox-providers/createos"), + }]); + expect(SELF_HOSTED_AUTO_INSTALL_KEYS).not.toContain("createos"); + }); it("resolves known keys to paths inside the catalog root", () => { const resolved = resolveBundledPluginInstalls(["kubernetes", "daytona"], { catalogRoot: CATALOG_ROOT, diff --git a/server/src/__tests__/plugin-database.test.ts b/server/src/__tests__/plugin-database.test.ts index 788c6351aa..6d0417aba5 100644 --- a/server/src/__tests__/plugin-database.test.ts +++ b/server/src/__tests__/plugin-database.test.ts @@ -203,6 +203,30 @@ describe("buildPluginWorkerEnv", () => { }); }); + it.each([ + { packagePath: null, packageName: "@paperclipai/plugin-createos", driverKey: "createos", allowed: true }, + { packagePath: "/app/packages/plugins/sandbox-providers/createos", packageName: "@paperclipai/plugin-createos", driverKey: "createos", allowed: true }, + { packagePath: "/home/operator/plugins/fake-createos", packageName: "@paperclipai/plugin-createos", driverKey: "createos", allowed: false }, + { packagePath: null, packageName: "@acme/plugin-createos", driverKey: "createos", allowed: false }, + { packagePath: null, packageName: "@paperclipai/plugin-createos", driverKey: "daytona", allowed: false }, + ])("confines the CreateOS fallback credential to its trusted worker: $packageName / $packagePath / $driverKey", ({ allowed, driverKey, ...installation }) => { + const env = buildPluginWorkerEnv({ + ...installation, + manifest: { + capabilities: ["environment.drivers.register"], + environmentDrivers: [{ driverKey }], + }, + trustedLocalPluginRoots: ["/app/packages/plugins"], + instanceInfo, + processEnv: { CREATEOS_API_KEY: "createos-token", DAYTONA_API_KEY: "daytona-token" }, + }); + expect(env).toEqual({ + PAPERCLIP_DEPLOYMENT_MODE: "authenticated", + PAPERCLIP_DEPLOYMENT_EXPOSURE: "public", + ...(allowed ? { CREATEOS_API_KEY: "createos-token" } : {}), + }); + }); + it("passes a first-party sandbox provider's documented credential env var to its own worker", () => { const env = buildPluginWorkerEnv({ manifest: { diff --git a/server/src/services/bundled-plugins.ts b/server/src/services/bundled-plugins.ts index 3990934ffb..b4c5ab9bf3 100644 --- a/server/src/services/bundled-plugins.ts +++ b/server/src/services/bundled-plugins.ts @@ -61,6 +61,11 @@ export interface BundledPluginCatalogEntry { * regardless of what the managed config document says. */ export const BUNDLED_PLUGIN_CATALOG: readonly BundledPluginCatalogEntry[] = [ + { + key: "createos", + pluginKey: "paperclip.createos-sandbox-provider", + relativePath: "sandbox-providers/createos", + }, { key: "cloudflare", pluginKey: "paperclip.cloudflare-sandbox-provider", diff --git a/server/src/services/plugin-loader.ts b/server/src/services/plugin-loader.ts index a72e5db2ef..163abaebd5 100644 --- a/server/src/services/plugin-loader.ts +++ b/server/src/services/plugin-loader.ts @@ -141,6 +141,7 @@ const SANDBOX_PROVIDER_CREDENTIAL_ENV_PASSTHROUGH: Record< string, { driverKey: string; envVars: readonly string[] } > = { + "@paperclipai/plugin-createos": { driverKey: "createos", envVars: ["CREATEOS_API_KEY"] }, "@paperclipai/plugin-daytona": { driverKey: "daytona", envVars: ["DAYTONA_API_KEY"] }, "@paperclipai/plugin-e2b": { driverKey: "e2b", envVars: ["E2B_API_KEY"] }, "@paperclipai/plugin-exe-dev": { driverKey: "exe-dev", envVars: ["EXE_API_KEY"] },