Files
PaperClipAI/server/src/__tests__/managed-plugin-environments.test.ts
T
Nicky LeachandClaude Opus 5.5 98ebeefbc9 feat(environments): let managed config declare plugin environments (#19508)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A managed instance gets its instance-level environments from the
`PAPERCLIP_MANAGED_CONFIG` document
> - #19490 added the plugin environment rows, their marker index and
write floor, but nothing creates them yet
> - The managed config still accepts only one sandbox entry, so a
managed instance cannot get a plugin-provided environment without manual
setup
> - This pull request lets the managed config declare plugin entries
beside the sandbox entry, and provisions one locked row for each at boot
> - The benefit is that a managed instance gets its plugin environments
with no manual setup, and the rows recover by themselves when their
plugin restarts

## Linked Issues or Issue Description

Refs #19490 (the plugin environment rows this pull request provisions)

**Subsystem affected**
Server environments: the managed-config parser, the managed environment
bootstrap, and the environment routes' write floor.

**Problem or motivation**
A managed instance cannot get a plugin environment from its managed
config. The `environments` section accepts one entry, and that entry
always provisions the managed sandbox. Company settings cannot create a
plugin environment either. So each managed instance that needs one
requires manual setup.

**Proposed solution**
Let an `environments` entry declare `driver: "plugin"`. Boot provisions
one locked row for each plugin entry through the row layer from #19490.
A later pull request adds a one-time instance default for one plugin
entry, and advertises the new managed-config capability.

**Alternatives considered**
Manual setup on each instance, and a new control-plane endpoint. Both
need a step outside the deploy. The managed config already delivers the
sandbox entry.

**Roadmap alignment**
This extends the existing managed-config `environments` section, which
already provisions the managed sandbox. It adds no new product surface.

## What Changed

- `server/src/services/managed-config.ts`: an `environments` entry may
set `driver` to `"sandbox"` or `"plugin"`. An entry without `driver` is
still the sandbox entry, so today's documents parse unchanged.
- A plugin entry has `name`, optional `description`, `pluginKey` (the
plugin's manifest id), `driverKey` and optional `driverConfig`.
- At most one sandbox entry. Names are unique across the section, and
`pluginKey`–`driverKey` pairs are unique. Secret-like keys in
`driverConfig` refuse startup, as for sandbox `config`. An unknown key
or driver still refuses startup, and that includes `instanceDefault`
until the follow-up pull request.
- Plugin entries parse into a new `pluginEnvironments` list. The
`environments` list keeps only the sandbox entry, so its consumers do
not change.
  - `MANAGED_ENVIRONMENT_DRIVERS` lists the accepted drivers.
- `server/src/services/managed-environments.ts`
(`applyManagedEnvironments`):
- Boot refuses to start unless an auto-installed catalog entry installs
each plugin entry's `pluginKey`. `plugins.autoInstall` holds catalog
keys, not manifest ids, so the parser cannot check this. Boot uses the
manifest ids `createApp` already resolves from the bundled and
distribution catalogs (`app.locals.managedBundledPluginKeys`, passed in
by `server/src/index.ts`). Without that list it resolves the catalog
itself.
- Plugin entries run after the sandbox entry and reuse its readiness
handling. Boot awaits `pluginsReady`, then requires the plugin `ready`,
declaring the entry's driver, and running a worker. Otherwise boot
archives the row and re-ensures it on the worker's `ready`.
- Recovery re-ensures on every worker `ready` until one ensure succeeds
or the plugin rejects the config (HTTP 422).
- If the ensure fails because the worker died, boot archives the row and
waits for `ready`. If the worker is still running (for example the call
timed out), boot keeps the row as it is, retries once at once and on the
next `ready`, and otherwise at the next boot. No timer is added.
  - Removing an entry never archives its row.
- A plugin-only document does not conflict with
`PAPERCLIP_EXECUTION_MODE`, because plugin rows do not use the sandbox
slot.
- Plugin entries never change the instance default. The sandbox stamp
still fills only an unset or local default, so it never replaces a
plugin default.
- `server/src/routes/environments.ts`: `isPlatformSlotEnvironment`
treats a marked plugin row as a live slot while the document declares
its entry. So the marker-clear escape hatch is refused then, and allowed
again after the entry leaves the document.
- `doc/DEVELOPING.md`: new "Managed environments" section.

## Verification

- `cd server && npx vitest run src/__tests__/managed-config.test.ts`:
old documents parse unchanged. Sandbox plus plugin parses. Each of these
refuses startup: two sandbox entries, `driver: null`, duplicate names, a
duplicate pair, a secret-like key, an unknown key (including
`instanceDefault`), an unknown driver, a malformed `driverKey` or
`driverConfig`.
- `npx vitest run
src/services/managed-environments-plugin-entries.test.ts` (the
sandbox-entry tests stay in `managed-environments.test.ts`; shared
fixtures are in `managed-environments.test-support.ts`):
- A plugin entry is ensured with the validator's worker manager, after
the sandbox entry.
- Boot refuses a plugin entry whose `pluginKey` no auto-installed
catalog entry installs, both from the passed list and from its own
catalog lookup.
- A plugin that is down archives the row, and worker recovery re-ensures
it. A missing plugin, a plugin without the driver, or a worker manager
without `call` counts as unavailable.
- Recovery stays armed after a failed attempt and stops after one
success or a permanent config rejection (422). A `ready` during an
attempt queues one more.
- A worker that dies during the boot ensure archives the row and arms
recovery. A failure while the worker keeps running leaves the row as it
is and retries.
- Removing an entry does not archive. Plugin entries and the sandbox
stamp never change a plugin default.
  - An ensure failure is counted, not thrown.
- `npx vitest run src/__tests__/managed-plugin-environments.test.ts`
(embedded Postgres, real `applyManagedEnvironments`): the first boot
creates the row and the second changes nothing. A plugin that is down
archives the row, and recovery brings it back. Removing the entry does
not archive the row. The instance default stays unset. The row tests
from #19490 still pass.
- `npx vitest run src/__tests__/environment-routes.test.ts`: the marker
clear is refused while the document declares the entry, and allowed
after it leaves.
- `npx vitest run src/__tests__/instance-settings-service.test.ts
src/__tests__/instance-settings-cloud-defaults.test.ts`: pass with the
new `pluginEnvironments` field.
- `pnpm --filter @paperclipai/server typecheck` passes.
- I did not run the full `pnpm test:run`, `pnpm -r typecheck` or `pnpm
build`. CI covers them.

## Risks

- No schema change.
- A managed-config document with a plugin entry now provisions a row at
boot. A document without one behaves as before.
- Adoption renames an unmarked row for the same plugin and driver to the
declared name and resets its driver config to the declared one. The row
keeps its id, agents and other metadata, and it is then locked like the
managed sandbox.
- Boot now refuses to start when a plugin entry names a plugin that no
auto-installed catalog entry provides. This is the same fail-closed
posture as an unknown sandbox provider.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code
with tool use (file edits, shell, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 17:35:13 +00:00

256 lines
10 KiB
TypeScript

import { randomUUID } from "node:crypto";
import { eq } from "drizzle-orm";
import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import {
activityLog,
agents,
companies,
createDb,
environments,
instanceSettings,
plugins,
type Db,
} from "@paperclipai/db";
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
import { environmentService } from "../services/environments.js";
import { MANAGED_CONFIG_ENV_KEY, parseManagedConfigEnv } from "../services/managed-config.js";
import { applyManagedEnvironments } from "../services/managed-environments.js";
const PLUGIN_KEY = "example.remote-host-plugin";
const MARKER = `${PLUGIN_KEY}:tasks`;
type ReadyListener = (payload: { pluginId: string }) => void;
function managedConfig(entries: unknown[]) {
return parseManagedConfigEnv({
[MANAGED_CONFIG_ENV_KEY]: JSON.stringify({
v: 1,
mode: "cloud",
catalogVersion: "2026.1106.0",
features: {},
plugins: { autoInstall: ["remote-host"] },
environments: entries,
}),
});
}
const remoteHostEntry = {
name: "Remote Host",
description: "Runs the agent on a remote host",
driver: "plugin",
pluginKey: PLUGIN_KEY,
driverKey: "tasks",
driverConfig: {},
};
const support = await getEmbeddedPostgresTestSupport();
(support.supported ? describe : describe.skip)("managed plugin environment rows", () => {
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
let db: Db;
let pluginId: string;
let running: boolean;
const call = vi.fn(async (_id: string, method: string, input: { config: Record<string, unknown> }) =>
method === "environmentValidateConfig" ? { ok: true, normalizedConfig: { ...input.config, normalized: true } } : null);
let readyListeners: ReadyListener[] = [];
const workerManager = {
isRunning: vi.fn(() => running),
call,
getWorker: vi.fn(() => ({
on: (_event: "ready", listener: ReadyListener) => { readyListeners.push(listener); },
off: (_event: "ready", listener: ReadyListener) => {
readyListeners = readyListeners.filter((candidate) => candidate !== listener);
},
})),
};
beforeAll(async () => {
database = await startEmbeddedPostgresTestDatabase("paperclip-managed-plugin-environment-rows-");
db = createDb(database.connectionString);
}, 90_000);
afterAll(async () => { await database?.cleanup(); });
beforeEach(async () => {
running = true;
readyListeners = [];
call.mockClear();
await db.delete(activityLog);
await db.delete(agents);
await db.delete(instanceSettings);
await db.delete(environments);
await db.delete(plugins);
await db.delete(companies);
pluginId = randomUUID();
await db.insert(plugins).values({
id: pluginId,
pluginKey: PLUGIN_KEY,
packageName: "@example/remote-host-plugin",
version: "1.0.0",
status: "ready",
manifestJson: {
capabilities: ["environment.drivers.register"],
environmentDrivers: [{ driverKey: "tasks", requiresAgent: true, supportsTasks: true }],
} as never,
});
});
const ensure = (overrides: Record<string, unknown> = {}) =>
environmentService(db).ensureManagedPluginEnvironment({
name: "Remote Host",
description: "Runs the agent on a remote host",
pluginKey: PLUGIN_KEY,
driverKey: "tasks",
driverConfig: {},
workerManager,
...overrides,
});
const pluginRows = () => db.select().from(environments).where(eq(environments.driver, "plugin"));
const company = async () => {
const id = randomUUID();
await db.insert(companies).values({ id, name: "Managed", issuePrefix: `M${id.slice(0, 7)}` });
return id;
};
it("creates the marked row through the plugin validator, then changes nothing", async () => {
await company();
const first = await ensure();
expect(first.action).toBe("added");
expect(first.environment).toMatchObject({
name: "Remote Host",
description: "Runs the agent on a remote host",
driver: "plugin",
status: "active",
config: { pluginKey: PLUGIN_KEY, driverKey: "tasks", driverConfig: { normalized: true } },
metadata: { managedByPaperclip: true, managedPluginEnvironment: MARKER },
});
expect(call).toHaveBeenCalledWith(pluginId, "environmentValidateConfig", { driverKey: "tasks", config: {} });
expect(await db.select().from(activityLog)).toHaveLength(1);
const second = await ensure();
const rows = await pluginRows();
expect(second.action).toBe("unchanged");
expect(rows).toHaveLength(1);
expect(rows[0]!.updatedAt.getTime()).toBe(first.environment.updatedAt.getTime());
expect(await db.select().from(activityLog)).toHaveLength(1);
});
it("adopts a hand-made row for the same plugin and driver, keeping its agents", async () => {
const companyId = await company();
const [handMade] = await db.insert(environments).values({
name: "Remote Host (manual)",
driver: "plugin",
status: "active",
config: { pluginKey: PLUGIN_KEY, driverKey: "tasks", driverConfig: {} },
metadata: { note: "made by hand" },
}).returning();
const agentId = randomUUID();
await db.insert(agents).values({ id: agentId, companyId, name: "Agent", adapterType: "process", defaultEnvironmentId: handMade!.id });
const result = await ensure();
const rows = await pluginRows();
expect(result.action).toBe("adopted");
expect(rows).toHaveLength(1);
expect(rows[0]).toMatchObject({
id: handMade!.id,
name: "Remote Host",
metadata: { note: "made by hand", managedByPaperclip: true, managedPluginEnvironment: MARKER },
});
const [kept] = await db.select().from(agents).where(eq(agents.id, agentId));
expect(kept!.defaultEnvironmentId).toBe(handMade!.id);
});
it("adopts a row again after its markers were cleared", async () => {
const first = await ensure();
// What the route's marker-clear request stores.
await db.update(environments)
.set({ metadata: { managedByPaperclip: null, managedPluginEnvironment: null } })
.where(eq(environments.id, first.environment.id));
const again = await ensure();
expect(again.action).toBe("adopted");
expect(again.environment.id).toBe(first.environment.id);
expect(await pluginRows()).toHaveLength(1);
});
it("lets several rows hold a cleared marker", async () => {
const config = { pluginKey: PLUGIN_KEY, driverKey: "other", driverConfig: {} };
await db.insert(environments).values([
{ name: "Cleared one", driver: "plugin", status: "active", config, metadata: { managedPluginEnvironment: false } },
{ name: "Cleared two", driver: "plugin", status: "active", config, metadata: { managedPluginEnvironment: false } },
{ name: "Cleared three", driver: "plugin", status: "active", config, metadata: { managedPluginEnvironment: null } },
]);
expect(await pluginRows()).toHaveLength(3);
});
it("archives the active marked row and reactivates it on the next ensure", async () => {
await ensure();
const archived = await environmentService(db).archiveManagedPluginEnvironment({ pluginKey: PLUGIN_KEY, driverKey: "tasks" });
expect(archived?.status).toBe("archived");
expect(await environmentService(db).archiveManagedPluginEnvironment({ pluginKey: PLUGIN_KEY, driverKey: "tasks" })).toBeNull();
expect((await environmentService(db).findManagedPluginEnvironment({ pluginKey: PLUGIN_KEY, driverKey: "tasks" }))?.status)
.toBe("archived");
const reactivated = await ensure();
expect(reactivated.action).toBe("updated");
expect(reactivated.environment.status).toBe("active");
});
it("refuses to ensure while the plugin has no running worker", async () => {
running = false;
await expect(ensure()).rejects.toThrow(/has no running worker/);
expect(await pluginRows()).toHaveLength(0);
});
it("refuses to take a name another environment already holds", async () => {
await db.insert(environments).values({ name: "Remote Host", driver: "ssh", status: "active", config: {} });
await expect(ensure()).rejects.toThrow(/already exists with driver "ssh"/);
});
it("allows at most one row per marker", async () => {
await ensure();
await expect(db.insert(environments).values({
name: "Second remote host",
driver: "plugin",
status: "active",
config: { pluginKey: PLUGIN_KEY, driverKey: "tasks", driverConfig: {} },
metadata: { managedPluginEnvironment: MARKER },
})).rejects.toThrow();
});
describe("boot (applyManagedEnvironments)", () => {
const boot = (entries: unknown[] = [remoteHostEntry]) =>
applyManagedEnvironments(db, managedConfig(entries), { env: {}, workerManager, autoInstalledPluginKeys: [PLUGIN_KEY] });
const instanceDefault = async () =>
(await db.select().from(instanceSettings).where(eq(instanceSettings.singletonKey, "default")))[0]?.defaultEnvironmentId ?? null;
it("creates the row on the first boot and changes nothing on the second", async () => {
const first = await boot();
expect(first).toMatchObject({ ensured: 1, added: 1, failed: 0 });
const [row] = await pluginRows();
expect(row).toMatchObject({ name: "Remote Host", status: "active", metadata: { managedByPaperclip: true, managedPluginEnvironment: MARKER } });
const second = await boot();
expect(second).toMatchObject({ ensured: 1, added: 0, unchanged: 1 });
const after = await pluginRows();
expect(after).toHaveLength(1);
expect(after[0]!.updatedAt.getTime()).toBe(row!.updatedAt.getTime());
expect(await instanceDefault()).toBeNull();
});
it("archives the row while the plugin is down and reactivates it when the worker recovers", async () => {
await boot();
running = false;
expect(await boot()).toMatchObject({ ensured: 0, failed: 1 });
expect((await pluginRows())[0]!.status).toBe("archived");
running = true;
for (const listener of [...readyListeners]) listener({ pluginId });
await vi.waitFor(async () => expect((await pluginRows())[0]?.status).toBe("active"));
});
it("never archives a row whose entry left the document", async () => {
await boot();
expect(await boot([])).toBeNull();
expect((await pluginRows())[0]!.status).toBe("active");
});
});
});