Files
PaperClipAI/server/src/services/managed-environments.test.ts
Devin Foley 273315a4d0 feat(server): provision managed sandbox environments from the managed config (#10324)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Hosted/managed deployments configure instances entirely from the
control plane: `PAPERCLIP_MANAGED_CONFIG` already delivers feature flags
and `plugins.autoInstall` (bundled sandbox provider plugins), parsed
fail-closed at boot
> - A plugin alone is not usable for execution: runs need an
instance-level `driver: "sandbox"` environment row pointing at the
provider, and today only Kubernetes has a boot path for that
(`PAPERCLIP_EXECUTION_MODE` → `ensureKubernetesEnvironment`); every
other provider requires a manual product-API call the control plane
cannot make on a managed instance
> - Adding one `ensureXxxEnvironment` per provider would multiply
near-identical boot hooks and env-var surfaces
> - This pull request generalizes the existing Kubernetes machinery: the
managed-config document gains an optional `environments` section that
declares a sandbox environment for any bundled provider, ensured
idempotently at boot by a provider-agnostic service function (the
Kubernetes hook becomes a thin wrapper over it)
> - The benefit is that a managed fleet can provision any sandbox
provider (Daytona, Modal, E2B, …) purely from configuration — no
per-provider code, no manual API calls, no secrets in the document —
while self-hosted behavior is untouched

## Linked Issues or Issue Description

No public issue exists. Refs #10157 (the cloud image variant that
bundles sandbox provider plugins — this PR is the configuration half
that makes an installed provider usable).

**Problem (feature-request shape):** on a managed instance the control
plane can auto-install a bundled sandbox provider plugin via
`PAPERCLIP_MANAGED_CONFIG.plugins.autoInstall`, but cannot create the
environment row that makes the provider schedulable. The only boot-time
environment provisioning is Kubernetes-specific
(`PAPERCLIP_EXECUTION_MODE=kubernetes` + `PAPERCLIP_K8S_*`). A generic,
config-driven path is needed so any bundled provider can be provisioned
without per-plugin code or manual API calls.

## What Changed

- `server/src/services/managed-config.ts`: optional `environments`
top-level section — `[{ name, description?, provider, config? }]` —
validated fail-closed: unknown keys, more than one entry (the DB permits
exactly one Paperclip-managed sandbox row,
`environments_managed_sandbox_idx`), a `provider` not present in
`plugins.autoInstall`, `config.provider`, or secret-looking config keys
at any depth (`api_key`/`token`/`secret`/`password`/`credential`) all
refuse startup. Absent section ⇒ `environments: []`, so pre-section
documents keep booting newer builds.
- `server/src/services/environments.ts`: new provider-agnostic
`ensureManagedSandboxEnvironment({ name, description?, provider,
config?, extraMetadata? })` — idempotently owns the single managed
sandbox row: refreshes name/description/config each call, adopts the
slot across provider switches (dropping the stale
`managedKubernetesSandbox` marker), adopts a same-name unmanaged sandbox
row (stamping it managed) instead of colliding on
`environments_name_idx` every boot, and falls back to keeping the
current name if the desired name belongs to a different row.
`ensureKubernetesEnvironment` is now a thin wrapper that pins `provider:
"kubernetes"` and stamps the legacy marker.
- `server/src/services/managed-environments.ts` (new):
`applyManagedEnvironments` boot step — no-op for self-hosted/empty;
throws (fail startup) when `PAPERCLIP_EXECUTION_MODE` is also set, since
both would own the same managed sandbox row; otherwise ensures each
declared environment fail-safe per entry (log + continue boot, matching
bundled-plugin provisioning posture).
- `server/src/index.ts`: runs the new boot step right after the
execution-policy bootstrap, before the heartbeat resumes queued runs.
- `server/src/services/index.ts`: exports `applyManagedEnvironments` and
`ManagedEnvironmentSpec`.
- Secrets stay out of the document by construction: provider credentials
reach managed instances only as process env vars (each provider's
documented fallback, e.g. `DAYTONA_API_KEY` for the Daytona plugin).

## Verification

```sh
cd server
pnpm exec tsc --noEmit -p tsconfig.json
pnpm exec vitest run \
  src/__tests__/managed-config.test.ts \
  src/services/managed-environments.test.ts \
  src/services/execution-policy-bootstrap.test.ts \
  src/__tests__/environment-service.test.ts \
  src/__tests__/environment-instance-routes.test.ts \
  src/__tests__/environment-routes.test.ts \
  src/__tests__/plugin-install-guard.test.ts \
  src/__tests__/environment-execution-target.test.ts \
  src/__tests__/instance-settings-managed-overlay.test.ts \
  src/__tests__/bundled-plugins.test.ts
```

All pass locally (typecheck clean; environment-service suite runs
against embedded Postgres and exercises the refactored Kubernetes
wrapper plus the new generic ensure: create/refresh, provider switch,
unmanaged-row adoption, name-conflict fallback). New tests cover the
parser (12 cases incl. secret-key rejection at depth) and the boot step
(no-op, mutual exclusion, pass-through, fail-safe).

## Risks

- **Self-hosted: none intended.** Without `PAPERCLIP_MANAGED_CONFIG`
nothing new executes; the `PAPERCLIP_EXECUTION_MODE=kubernetes` path is
regression-covered by the existing bootstrap/service/route suites (all
green).
- **Behavioral shift in `ensureKubernetesEnvironment` (deliberate):** it
now also refreshes `name`/`description` to their managed defaults each
boot (desired-state semantics, same as config today) and adopts a
`managedByPaperclip` sandbox row that lacks the Kubernetes marker —
previously that state made the ensure throw every boot.
- **New startup failure modes are all explicit misconfigurations**
(malformed section, provider not auto-installed, secret in config,
execution-mode conflict) and fail with precise errors; DB-side ensure
failures never block boot (fail-safe per entry, logged).
- No migrations; no API surface changes.

## Model Used

Claude Fable 5 (Anthropic, model ID `claude-fable-5`) with extended
thinking and tool use, driving the change end-to-end inside a Claude
Code / agent-harness session (code, tests, and verification runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (the
managed-config module header is the contract doc)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-07-27 12:00:21 -07:00

416 lines
14 KiB
TypeScript

import { describe, expect, it, vi } from "vitest";
import type { Db } from "@paperclipai/db";
import { MANAGED_CONFIG_ENV_KEY, parseManagedConfigEnv } from "./managed-config.js";
import {
applyManagedEnvironments,
type ApplyManagedEnvironmentsOptions,
} from "./managed-environments.js";
const noDb = null as unknown as Db;
function parsedConfig(overrides: Record<string, unknown> = {}) {
const config = parseManagedConfigEnv({
[MANAGED_CONFIG_ENV_KEY]: JSON.stringify({
v: 1,
mode: "cloud",
catalogVersion: "2026.720.0",
features: {},
plugins: { autoInstall: ["daytona"] },
...overrides,
}),
});
if (!config) throw new Error("expected a parsed managed config");
return config;
}
function environmentRow(overrides: Record<string, unknown> = {}) {
const now = new Date();
return {
id: "env-1",
name: "Daytona",
description: null,
driver: "sandbox",
status: "active",
config: {},
envVars: {},
metadata: {},
createdAt: now,
updatedAt: now,
...overrides,
};
}
function readyDriverResolver(status = "ready") {
return vi.fn(async () => ({
plugin: { id: "plugin-1", pluginKey: "sandbox-providers/daytona", status },
}));
}
function runningWorkerManager(running = true) {
return { isRunning: vi.fn(() => running), getWorker: vi.fn(() => undefined) };
}
type WorkerManagerSeam = NonNullable<ApplyManagedEnvironmentsOptions["workerManager"]>;
type RecoveryHandle = NonNullable<ReturnType<WorkerManagerSeam["getWorker"]>>;
/** A worker handle fake that records `ready` listeners so tests can fire them. */
function recoveryHandle() {
const readyListeners: Array<(payload: { pluginId: string }) => void> = [];
const handle: RecoveryHandle = {
on: vi.fn((_event, listener) => {
readyListeners.push(listener);
}),
off: vi.fn((_event, listener) => {
const at = readyListeners.indexOf(listener);
if (at !== -1) readyListeners.splice(at, 1);
}),
};
return {
handle,
fireReady: () => {
for (const listener of [...readyListeners]) listener({ pluginId: "plugin-1" });
},
};
}
/** Lets the fire-and-forget recovery ensure settle. */
function tick() {
return new Promise((resolve) => setImmediate(resolve));
}
type EnvironmentsSeam = NonNullable<ApplyManagedEnvironmentsOptions["environments"]>;
function environmentsSeam(overrides: Partial<EnvironmentsSeam> = {}): EnvironmentsSeam {
return {
ensureManagedSandboxEnvironment:
overrides.ensureManagedSandboxEnvironment ?? vi.fn().mockResolvedValue(environmentRow()),
archiveManagedSandboxEnvironment:
overrides.archiveManagedSandboxEnvironment ?? vi.fn().mockResolvedValue(null),
};
}
describe("applyManagedEnvironments", () => {
it("no-ops for self-hosted instances and for documents without environments", async () => {
expect(await applyManagedEnvironments(noDb, null)).toBeNull();
expect(await applyManagedEnvironments(noDb, parsedConfig())).toBeNull();
});
it("refuses startup when a forced execution mode also claims the managed sandbox slot", async () => {
const config = parsedConfig({
environments: [{ name: "Daytona", provider: "daytona" }],
});
await expect(
applyManagedEnvironments(noDb, config, {
env: { PAPERCLIP_EXECUTION_MODE: "kubernetes" },
}),
).rejects.toThrow(/mutually exclusive/);
});
it("ensures each declared environment through the provider-agnostic service call", async () => {
const ensureManagedSandboxEnvironment = vi
.fn()
.mockResolvedValue(environmentRow());
const config = parsedConfig({
environments: [
{
name: "Daytona",
description: "Managed Daytona sandbox.",
provider: "daytona",
config: { target: "us" },
},
],
});
const resolveSandboxProviderDriver = readyDriverResolver();
const workerManager = runningWorkerManager();
const result = await applyManagedEnvironments(noDb, config, {
env: {},
workerManager,
environments: environmentsSeam({ ensureManagedSandboxEnvironment }),
resolveSandboxProviderDriver,
});
expect(result).toEqual({ ensured: 1, failed: 0 });
expect(resolveSandboxProviderDriver).toHaveBeenCalledWith({ db: noDb, driverKey: "daytona" });
expect(workerManager.isRunning).toHaveBeenCalledWith("plugin-1");
expect(ensureManagedSandboxEnvironment).toHaveBeenCalledTimes(1);
expect(ensureManagedSandboxEnvironment).toHaveBeenCalledWith({
name: "Daytona",
description: "Managed Daytona sandbox.",
provider: "daytona",
config: { target: "us" },
});
// The frozen parsed config must not leak into the service (the row's
// config is mutated downstream when the provider key is forced in).
const passedConfig = ensureManagedSandboxEnvironment.mock.calls[0]?.[0]?.config;
expect(Object.isFrozen(passedConfig)).toBe(false);
});
it("waits for the bundled-plugin startup pass before ensuring anything", async () => {
const ensureManagedSandboxEnvironment = vi
.fn()
.mockResolvedValue(environmentRow());
const config = parsedConfig({
environments: [{ name: "Daytona", provider: "daytona" }],
});
let releasePlugins!: () => void;
const pluginsReady = new Promise<void>((resolve) => {
releasePlugins = resolve;
});
const pending = applyManagedEnvironments(noDb, config, {
env: {},
pluginsReady,
workerManager: runningWorkerManager(),
environments: environmentsSeam({ ensureManagedSandboxEnvironment }),
resolveSandboxProviderDriver: readyDriverResolver(),
});
// Give the ensure every chance to (incorrectly) run early.
await new Promise((resolve) => setImmediate(resolve));
expect(ensureManagedSandboxEnvironment).not.toHaveBeenCalled();
releasePlugins();
expect(await pending).toEqual({ ensured: 1, failed: 0 });
expect(ensureManagedSandboxEnvironment).toHaveBeenCalledTimes(1);
});
it("skips an entry whose provider plugin is missing and archives its stale row", async () => {
const environments = environmentsSeam({
archiveManagedSandboxEnvironment: vi.fn().mockResolvedValue(environmentRow({ status: "archived" })),
});
const config = parsedConfig({
environments: [{ name: "Daytona", provider: "daytona" }],
});
const result = await applyManagedEnvironments(noDb, config, {
env: {},
workerManager: runningWorkerManager(),
environments,
resolveSandboxProviderDriver: vi.fn(async () => null),
});
expect(result).toEqual({ ensured: 0, failed: 1 });
expect(environments.ensureManagedSandboxEnvironment).not.toHaveBeenCalled();
expect(environments.archiveManagedSandboxEnvironment).toHaveBeenCalledWith({
provider: "daytona",
});
});
it("skips (and counts failed) an entry whose provider plugin is not ready", async () => {
const environments = environmentsSeam();
const config = parsedConfig({
environments: [{ name: "Daytona", provider: "daytona" }],
});
const result = await applyManagedEnvironments(noDb, config, {
env: {},
workerManager: runningWorkerManager(),
environments,
resolveSandboxProviderDriver: readyDriverResolver("disabled"),
});
expect(result).toEqual({ ensured: 0, failed: 1 });
expect(environments.ensureManagedSandboxEnvironment).not.toHaveBeenCalled();
expect(environments.archiveManagedSandboxEnvironment).toHaveBeenCalledWith({
provider: "daytona",
});
});
it("skips an entry whose plugin record is ready but whose worker is not running", async () => {
const environments = environmentsSeam();
const config = parsedConfig({
environments: [{ name: "Daytona", provider: "daytona" }],
});
const workerManager = runningWorkerManager(false);
const result = await applyManagedEnvironments(noDb, config, {
env: {},
workerManager,
environments,
resolveSandboxProviderDriver: readyDriverResolver(),
});
expect(result).toEqual({ ensured: 0, failed: 1 });
expect(workerManager.isRunning).toHaveBeenCalledWith("plugin-1");
expect(environments.ensureManagedSandboxEnvironment).not.toHaveBeenCalled();
expect(environments.archiveManagedSandboxEnvironment).toHaveBeenCalledWith({
provider: "daytona",
});
// No registered handle → no respawn is coming; degraded until next boot.
expect(workerManager.getWorker).toHaveBeenCalledWith("plugin-1");
});
it("reactivates the archived environment once when the provider worker recovers", async () => {
const environments = environmentsSeam();
const config = parsedConfig({
environments: [
{ name: "Daytona", provider: "daytona", config: { target: "us" } },
],
});
const { handle, fireReady } = recoveryHandle();
const workerManager = {
isRunning: vi.fn(() => false),
getWorker: vi.fn(() => handle),
};
const result = await applyManagedEnvironments(noDb, config, {
env: {},
workerManager,
environments,
resolveSandboxProviderDriver: readyDriverResolver(),
});
// The boot pass itself stays degraded: archived, counted failed.
expect(result).toEqual({ ensured: 0, failed: 1 });
expect(environments.archiveManagedSandboxEnvironment).toHaveBeenCalledWith({
provider: "daytona",
});
expect(environments.ensureManagedSandboxEnvironment).not.toHaveBeenCalled();
// Worker manager respawns the worker → the handle re-emits `ready`.
fireReady();
fireReady();
await tick();
expect(environments.ensureManagedSandboxEnvironment).toHaveBeenCalledTimes(1);
expect(environments.ensureManagedSandboxEnvironment).toHaveBeenCalledWith({
name: "Daytona",
description: undefined,
provider: "daytona",
config: { target: "us" },
});
expect(handle.off).toHaveBeenCalledTimes(1);
});
it("re-checks isRunning after subscribing so a recovery during the archive is not missed", async () => {
const environments = environmentsSeam();
const config = parsedConfig({
environments: [{ name: "Daytona", provider: "daytona" }],
});
const { handle } = recoveryHandle();
// Down at the gate check, already back up by the time the listener is
// registered — the `ready` event has been missed.
const workerManager = {
isRunning: vi
.fn()
.mockReturnValueOnce(false)
.mockReturnValue(true),
getWorker: vi.fn(() => handle),
};
const result = await applyManagedEnvironments(noDb, config, {
env: {},
workerManager,
environments,
resolveSandboxProviderDriver: readyDriverResolver(),
});
await tick();
expect(result).toEqual({ ensured: 0, failed: 1 });
expect(environments.ensureManagedSandboxEnvironment).toHaveBeenCalledTimes(1);
});
it("does not subscribe for recovery when the plugin record is missing or not ready", async () => {
const config = parsedConfig({
environments: [{ name: "Daytona", provider: "daytona" }],
});
for (const resolveSandboxProviderDriver of [
vi.fn(async () => null),
readyDriverResolver("disabled"),
]) {
const workerManager = runningWorkerManager(false);
await applyManagedEnvironments(noDb, config, {
env: {},
workerManager,
environments: environmentsSeam(),
resolveSandboxProviderDriver,
});
expect(workerManager.getWorker).not.toHaveBeenCalled();
}
});
it("logs, not throws, when the recovery re-ensure fails", async () => {
const environments = environmentsSeam({
ensureManagedSandboxEnvironment: vi.fn().mockRejectedValue(new Error("db exploded")),
});
const config = parsedConfig({
environments: [{ name: "Daytona", provider: "daytona" }],
});
const { handle, fireReady } = recoveryHandle();
const workerManager = {
isRunning: vi.fn(() => false),
getWorker: vi.fn(() => handle),
};
await applyManagedEnvironments(noDb, config, {
env: {},
workerManager,
environments,
resolveSandboxProviderDriver: readyDriverResolver(),
});
fireReady();
await tick();
// The rejection is swallowed into a log line; reaching this point without
// an unhandled rejection is the assertion.
expect(environments.ensureManagedSandboxEnvironment).toHaveBeenCalledTimes(1);
});
it("fails closed when no worker manager is provided", async () => {
const environments = environmentsSeam();
const config = parsedConfig({
environments: [{ name: "Daytona", provider: "daytona" }],
});
const result = await applyManagedEnvironments(noDb, config, {
env: {},
environments,
resolveSandboxProviderDriver: readyDriverResolver(),
});
expect(result).toEqual({ ensured: 0, failed: 1 });
expect(environments.ensureManagedSandboxEnvironment).not.toHaveBeenCalled();
});
it("counts a skipped entry once even when archiving its stale row fails", async () => {
const environments = environmentsSeam({
archiveManagedSandboxEnvironment: vi.fn().mockRejectedValue(new Error("db exploded")),
});
const config = parsedConfig({
environments: [{ name: "Daytona", provider: "daytona" }],
});
const result = await applyManagedEnvironments(noDb, config, {
env: {},
workerManager: runningWorkerManager(),
environments,
resolveSandboxProviderDriver: vi.fn(async () => null),
});
expect(result).toEqual({ ensured: 0, failed: 1 });
expect(environments.archiveManagedSandboxEnvironment).toHaveBeenCalledTimes(1);
});
it("is fail-safe per entry: an ensure failure is counted, not thrown", async () => {
const environments = environmentsSeam({
ensureManagedSandboxEnvironment: vi.fn().mockRejectedValue(new Error("db exploded")),
});
const config = parsedConfig({
environments: [{ name: "Daytona", provider: "daytona" }],
});
const result = await applyManagedEnvironments(noDb, config, {
env: {},
workerManager: runningWorkerManager(),
environments,
resolveSandboxProviderDriver: readyDriverResolver(),
});
expect(result).toEqual({ ensured: 0, failed: 1 });
expect(environments.archiveManagedSandboxEnvironment).not.toHaveBeenCalled();
});
});