feat(plugins): let managed config grant worker variables to a plugin (#19049)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A managed instance gets its plugins from the control plane, and
people do not install them
> - Some of these plugins need deployment values, such as a service
origin and a capability
> - The plugin worker environment is scrubbed, and only a fixed list of
first-party providers gets deployment variables
> - Core also sends lifecycle events and project instance secrets only
to plugins with a saved config row
> - So a provisioned plugin cannot work until a person saves
configuration for it
> - This pull request lets the managed config grant named variables to a
selected plugin
> - The benefit is that a distribution plugin can run with no tenant
configuration, and Core does not need to name it

## Linked Issues or Issue Description

No issue exists. This describes the problem. I searched open pull
requests for related work and found no duplicate.

**What existing behavior does this improve?**

How a plugin provisioned through `plugins.autoInstall` receives its
deployment settings.

**Current behavior**

`buildPluginWorkerEnv` forwards variables only from a fixed list that
names first-party packages. A distribution plugin cannot be on that list
without Core naming it. A plugin with no `pluginConfig` row gets no
resource lifecycle events, and project instance secrets refuse it.

**Proposed behavior**

The managed config can include `plugins.workerEnv`. It maps a catalog
key to variable names. The worker of that plugin receives those
variables from the server process environment. The managed config can
also include `plugins.enabledCompanies`. It maps a catalog key to
company IDs. In those companies, the plugin gets lifecycle events and
can use project instance secrets with no config row.

**Reason and benefit**

The control plane stays the only place that decides which plugin gets
which values. Tenants do not handle these values, and they are not in
the database or in the managed config document.

**Alternatives considered**

- Create a config or enablement row for each company. The control plane
would need a new tenant write API, and a row can stay after the document
removes access or return after a database restore. The document is never
saved, like the other managed settings.
- Give a grant access to every company. This has no per-company opt-in.
- Put the values in the managed config document. The document does not
hold secrets, and this rule stays true because it holds only names.

## What Changed

- `parseManagedConfigEnv` accepts an optional `plugins.workerEnv`
object. Each key must be in `plugins.autoInstall`. Names must be
uppercase variable names, with no duplicates.
- Startup fails if a name looks like a host credential, such as a name
with `KEY`, `TOKEN`, `SECRET`, `PASSWORD`, `DATABASE` or `JWT` as a
word, or if it controls how the worker loads code, such as
`NODE_OPTIONS`.
- `parseManagedConfigEnv` accepts an optional `plugins.enabledCompanies`
object. Each key must be in `plugins.autoInstall`. Values must be
lowercase company UUIDs, with no duplicates.
- `managedPluginWorkerEnvGrant` and `managedPluginEnabledCompanies`
return values by plugin key. A managed instance only starts a selected
plugin from the current image, so the saved package path is not part of
this check. It can be old or null.
- `buildPluginWorkerEnv` forwards granted variables. This does not
depend on the `environment.drivers.register` capability.
- Plugin lifecycle delivery includes a plugin with no config row only
for the companies the document names. It still skips a company that
disabled the plugin. The runtime event listener follows the same rule.
- Project instance secrets accept a plugin with no config row only for a
company the document names.
- A variable grant alone gives no company access.
- `doc/plugins/DISTRIBUTION-PLUGINS.md` describes both sections.

## Verification

- `npx vitest run` in `server/` on `managed-config`,
`managed-plugin-worker-env`, `plugin-lifecycle-delivery`,
`project-instance-secrets`, `plugin-database` and `bundled-plugins`: 201
tests pass.
- The new tests cover a digest-verified distribution plugin, a stale
saved package path, a catalog key named after an inherited property, a
company the document does not name, a company that disabled the plugin,
a named company that does not exist, a grant with no enabled company,
and host credential names.
- `tsc --noEmit` in `server/` shows no errors in the changed files. My
worktree does not have the built `@paperclipai/paperclip-runner`
package, so unrelated files show errors locally. CI runs the full
typecheck.
- I did not run the full test suite locally. CI runs it.
- I did not test this change on a running instance.

## Risks

- **Older builds:** a build without this change refuses a document with
`plugins.workerEnv` or `plugins.enabledCompanies`, because it fails
closed on unknown keys. A control plane must send these sections only to
a build that supports them.
- **Trust:** the granted plugin gets the variable values in its process.
The image owner and the control plane already trust this plugin. The
name check stops a mistake with host credentials, but a name check
cannot prove that a value is not sensitive. Core cannot use an
allowlist, because it must not name the variables of a private plugin.
- **Company scope:** the control plane sets which companies a plugin
serves. A new company needs a new document, and the server reads the
document when it starts. Self-hosted instances do not change.
- **Rotation:** a new value reaches the worker only when the server
restarts. Rotation without a restart is not part of this change.
- **Background calls:** a grant does not add proactive company access. A
plugin that makes background calls with no invocation still needs a
saved config for that company.
- This changes core plugin behavior, and I did not discuss it in `#dev`
first. Maintainers can tell me if they want a discussion before review.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), through Claude Code, with tool
use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Nicky LeachandClaude Opus 5.5 authored and GitHub committed 2026-11-04 15:24:43 -08:00
1 parent c7390bef20
commit 2c39a2866b
11 files changed
+425 -14

No files matched your search

+35
View File
@@ -48,6 +48,41 @@ Managed instances select a distribution key through the existing
compatibility, worker and health mechanisms apply. A worker or install failure
is recorded as a plugin error without taking down the application.
A managed instance can also give a selected plugin process variables through
`plugins.workerEnv`. The document holds variable names, not values. The deployment
sets the values in the server's process environment:
```json
{ "plugins": {
"autoInstall": ["example-extension"],
"workerEnv": { "example-extension": ["EXAMPLE_ORIGIN", "EXAMPLE_CAPABILITY"] }
} }
```
Each key must be in `autoInstall`. Startup fails if a name looks like a host
credential, such as one with `KEY`, `TOKEN`, `SECRET`, `PASSWORD`, `DATABASE` or
`JWT` as a word, or if it controls how the worker loads code, such as
`NODE_OPTIONS`. Give a granted variable a name for its own use, not a reused host
variable. Only the plugin with that catalog key receives the variables. A managed
instance runs a selected plugin only from the current image, so an install from
another source does not start.
A grant does not give access to any company. To let a plugin serve a company with
no saved configuration, name the company in `plugins.enabledCompanies`:
```json
{ "plugins": {
"autoInstall": ["example-extension"],
"enabledCompanies": { "example-extension": ["<company-uuid>"] }
} }
```
The plugin then receives resource lifecycle events for that company and can use
project instance secrets there. A company can still disable the plugin. This does
not give access for background worker calls. Those still need a saved company
configuration. Core reads the list from the document on each use and does not save
it, so a database restore cannot bring back access that the document removed.
The catalog alone does not auto-enable plugins on self-hosted instances.
Operators can explicitly install catalog entries through the normal plugin
CLI. The package's manifest ID and version must match the catalog.
@@ -21,6 +21,63 @@ function validDoc(overrides: Record<string, unknown> = {}) {
});
}
describe("plugins.workerEnv", () => {
const plugins = (workerEnv: unknown) => ({ plugins: { autoInstall: ["daytona"], workerEnv } });
it("is absent from documents that do not declare it", () => {
expect(parseManagedConfigEnv(envWith(validDoc()))?.plugins).toEqual({ autoInstall: ["daytona", "kubernetes"] });
});
it("accepts variable names for an auto-installed plugin", () => {
const parsed = parseManagedConfigEnv(envWith(validDoc(plugins({ daytona: ["PROVIDER_ORIGIN", "PROVIDER_CAPABILITY_2"] }))));
expect(parsed?.plugins.workerEnv).toEqual({ daytona: ["PROVIDER_ORIGIN", "PROVIDER_CAPABILITY_2"] });
});
it.each([
"ANTHROPIC_API_KEY", "AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY", "BETTER_AUTH_SECRET", "DATABASE_MIGRATION_URL", "DATABASE_URL", "DB_PASSWORD",
"GH_TOKEN", "GITHUB_TOKEN", "LD_PRELOAD", "NODE_OPTIONS", "PAPERCLIP_AGENT_JWT_SECRET", "PAPERCLIP_AUTH_SECRET", "PAPERCLIP_BACKUP_WORKER_DATABASE_URL",
"PAPERCLIP_CLOUD_CONNECTOR_SIGN_PRIVATE_KEY", "PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", "PAPERCLIP_DECISION_SIGNING_SECRET", "PAPERCLIP_MANAGED_CONFIG",
"PAPERCLIP_SECRETS_MASTER_KEY", "PAPERCLIP_SECRETS_MASTER_KEY_FILE", "PAPERCLIP_TEST_POSTGRES_ADMIN_URL", "PATH", "SENTRY_DSN",
])("refuses to start with a grant of %s", (name) => {
expect(() => parseManagedConfigEnv(envWith(validDoc(plugins({ daytona: [name] }))))).toThrow(`must not grant host credential or loader variable "${name}"`);
});
it.each(["PAPERCLIP_CLOUD_API_ORIGIN", "PAPERCLIP_PRIMARY_HOST", "PROVIDER_CAPABILITY", "MONKEY_ORIGIN"])("accepts a grant of %s", (name) => {
expect(parseManagedConfigEnv(envWith(validDoc(plugins({ daytona: [name] }))))?.plugins.workerEnv).toEqual({ daytona: [name] });
});
it.each([
["a plugin that is not auto-installed", { other: ["PROVIDER_ORIGIN"] }, /not in "plugins.autoInstall"/],
["a non-object section", ["PROVIDER_ORIGIN"], /must be an object/],
["an empty grant", { daytona: [] }, /non-empty array/],
["a value instead of a name", { daytona: ["https://example.test"] }, /uppercase variable names/],
["a lowercase name", { daytona: ["database_url"] }, /uppercase variable names/],
["a duplicate name", { daytona: ["PROVIDER_ORIGIN", "PROVIDER_ORIGIN"] }, /duplicate entry/],
])("refuses to start with %s", (_label, workerEnv, message) => {
expect(() => parseManagedConfigEnv(envWith(validDoc(plugins(workerEnv))))).toThrow(message);
});
});
describe("plugins.enabledCompanies", () => {
const companyId = "f0679247-e2be-5a76-adb6-6bd360e5a833";
const plugins = (enabledCompanies: unknown) => ({ plugins: { autoInstall: ["daytona"], enabledCompanies } });
it("accepts company ids for an auto-installed plugin", () => {
expect(parseManagedConfigEnv(envWith(validDoc(plugins({ daytona: [companyId] }))))?.plugins.enabledCompanies).toEqual({ daytona: [companyId] });
});
it.each([
["a plugin that is not auto-installed", { other: [companyId] }, /not in "plugins.autoInstall"/],
["a non-object section", [companyId], /must be an object/],
["an empty list", { daytona: [] }, /non-empty array/],
["a value that is not a UUID", { daytona: ["all"] }, /lowercase company UUIDs/],
["an uppercase UUID", { daytona: [companyId.toUpperCase()] }, /lowercase company UUIDs/],
["a duplicate id", { daytona: [companyId, companyId] }, /duplicate entry/],
])("refuses to start with %s", (_label, enabledCompanies, message) => {
expect(() => parseManagedConfigEnv(envWith(validDoc(plugins(enabledCompanies))))).toThrow(message);
});
});
describe("managedFeatureKeySet", () => {
it("contains exactly the boolean flags of the experimental schema", () => {
const keys = managedFeatureKeySet();
@@ -0,0 +1,66 @@
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { BUNDLED_CATALOG_ROOT_ENV_VAR } from "../services/bundled-plugins.js";
import { distributionBundleDigest } from "../services/distribution-plugin-catalog.js";
import { MANAGED_CONFIG_ENV_KEY } from "../services/managed-config.js";
import { managedPluginEnabledCompanies, managedPluginWorkerEnvGrant } from "../services/managed-plugin-worker-env.js";
describe("managed plugin grants", () => {
const pluginKey = "paperclip.daytona-sandbox-provider";
const companyId = "f0679247-e2be-5a76-adb6-6bd360e5a833";
let root: string;
const env = (plugins: Record<string, unknown>) => ({
[BUNDLED_CATALOG_ROOT_ENV_VAR]: root,
[MANAGED_CONFIG_ENV_KEY]: JSON.stringify({ v: 1, mode: "cloud", catalogVersion: "1", features: {}, plugins }),
});
async function distributionPlugin(key: string, id: string) {
const directory = path.join(root, "distribution", key);
await mkdir(path.join(directory, "dist"), { recursive: true });
await writeFile(path.join(directory, "package.json"), JSON.stringify({ version: "1.0.0", paperclipPlugin: { manifest: "./dist/manifest.js", worker: "./dist/worker.js" } }));
await writeFile(path.join(directory, "dist/manifest.js"), "export default {};");
await writeFile(path.join(directory, "dist/worker.js"), "export {};");
await writeFile(path.join(root, "distribution/catalog.json"), JSON.stringify({ schemaVersion: 1, plugins: [{
key, pluginKey: id, version: "1.0.0", directory: key, digest: distributionBundleDigest(directory),
}] }));
}
beforeEach(async () => {
root = await mkdtemp(path.join(tmpdir(), "managed-worker-env-"));
await mkdir(path.join(root, "sandbox-providers/daytona"), { recursive: true });
});
afterEach(async () => { await rm(root, { recursive: true, force: true }); });
it("reads each section separately", () => {
const granted = env({ autoInstall: ["daytona"], workerEnv: { daytona: ["PROVIDER_ORIGIN"] } });
expect(managedPluginWorkerEnvGrant(pluginKey, granted)).toEqual(["PROVIDER_ORIGIN"]);
expect(managedPluginEnabledCompanies(pluginKey, granted)).toEqual([]);
const enabled = env({ autoInstall: ["daytona"], enabledCompanies: { daytona: [companyId] } });
expect(managedPluginEnabledCompanies(pluginKey, enabled)).toEqual([companyId]);
expect(managedPluginWorkerEnvGrant(pluginKey, enabled)).toEqual([]);
});
it("gives nothing on a self-hosted instance or to another plugin", () => {
expect(managedPluginWorkerEnvGrant(pluginKey, { [BUNDLED_CATALOG_ROOT_ENV_VAR]: root })).toEqual([]);
const granted = env({ autoInstall: ["daytona"], workerEnv: { daytona: ["PROVIDER_ORIGIN"] }, enabledCompanies: { daytona: [companyId] } });
expect(managedPluginWorkerEnvGrant("example.other", granted)).toEqual([]);
expect(managedPluginEnabledCompanies("example.other", granted)).toEqual([]);
});
it("gives variables to a digest-verified distribution plugin", async () => {
await distributionPlugin("example-extension", "example.extension");
const granted = env({ autoInstall: ["example-extension"], workerEnv: { "example-extension": ["PROVIDER_ORIGIN"] } });
expect(managedPluginWorkerEnvGrant("example.extension", granted)).toEqual(["PROVIDER_ORIGIN"]);
});
it("does not read inherited properties for a catalog key named after one", async () => {
await distributionPlugin("constructor", "example.extension");
const granted = env({ autoInstall: ["constructor"], workerEnv: { constructor: ["PROVIDER_ORIGIN"] }, enabledCompanies: {} });
expect(managedPluginWorkerEnvGrant("example.extension", granted)).toEqual(["PROVIDER_ORIGIN"]);
expect(managedPluginEnabledCompanies("example.extension", granted)).toEqual([]);
const enabled = env({ autoInstall: ["constructor"], workerEnv: {}, enabledCompanies: { constructor: [companyId] } });
expect(managedPluginEnabledCompanies("example.extension", enabled)).toEqual([companyId]);
expect(managedPluginWorkerEnvGrant("example.extension", enabled)).toEqual([]);
});
});
@@ -148,6 +148,20 @@ describe("buildPluginWorkerEnv", () => {
deploymentExposure: "public",
};
it("forwards only granted variables to a plugin that registers no environment driver", () => {
const env = buildPluginWorkerEnv({
manifest: { capabilities: [] },
instanceInfo,
grantedEnvVars: ["PROVIDER_ORIGIN", "PROVIDER_BLANK", "PROVIDER_MISSING"],
processEnv: { PROVIDER_ORIGIN: "https://provider.test", PROVIDER_BLANK: " ", DATABASE_URL: "postgres://host", OPENAI_API_KEY: "openai-token" },
});
expect(env).toEqual({
PAPERCLIP_DEPLOYMENT_MODE: "authenticated",
PAPERCLIP_DEPLOYMENT_EXPOSURE: "public",
PROVIDER_ORIGIN: "https://provider.test",
});
});
it("passes only model provider keys through to environment driver plugins", () => {
const env = buildPluginWorkerEnv({
manifest: { capabilities: ["environment.drivers.register"] },
@@ -1,7 +1,10 @@
import { randomBytes, randomUUID } from "node:crypto";
import { EventEmitter } from "node:events";
import { mkdir, mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { eq } from "drizzle-orm";
import { companies, createDb, companySecrets, projects, pluginConfig, plugins, resourceLifecycleEvents, type Db } from "@paperclipai/db";
import { companies, createDb, companySecrets, projects, pluginCompanySettings, pluginConfig, plugins, resourceLifecycleEvents, type Db } from "@paperclipai/db";
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
import { logger } from "../middleware/logger.js";
@@ -54,6 +57,76 @@ describePostgres("durable plugin lifecycle delivery", () => {
const worker = registerPluginLifecycleDelivery(coordinator as never, db, { call, isRunning: running, onConfigApplied } as unknown as PluginWorkerManager, lifecycle as unknown as PluginLifecycleManager);
return { call, running, wake, lifecycle, worker, applied: () => { for (const listener of configApplied) listener(); }, run: () => task.run(new AbortController().signal), pending: () => task.hasPending(), nextRunAt: () => task.nextRunAt?.(), settled: () => task.onSettled?.() };
}
async function withManagedPlugin(enabledCompanies: string[], run: () => Promise<void>) {
const root = await mkdtemp(path.join(tmpdir(), "delivery-managed-"));
await mkdir(path.join(root, "sandbox-providers/daytona"), { recursive: true });
vi.stubEnv("PAPERCLIP_BUNDLED_PLUGIN_ROOT", root);
vi.stubEnv("PAPERCLIP_MANAGED_CONFIG", JSON.stringify({ v: 1, mode: "cloud", catalogVersion: "1", features: {},
plugins: { autoInstall: ["daytona"], workerEnv: { daytona: ["PROVIDER_ORIGIN"] }, enabledCompanies: { daytona: enabledCompanies } } }));
try { await run(); }
finally { vi.unstubAllEnvs(); await rm(root, { recursive: true, force: true }); }
}
it("delivers to a managed plugin with no config row and a stale saved path only in the companies the document enables", async () => {
const unlistedCompanyId = randomUUID(), disabledCompanyId = randomUUID(), absentCompanyId = randomUUID();
await withManagedPlugin([companyId, disabledCompanyId, absentCompanyId], async () => {
const managedId = randomUUID(), ungrantedId = randomUUID();
const manifestJson = { resourceLifecycle: ["project"], capabilities: ["events.subscribe"] } as never;
await db.insert(companies).values([
{ id: unlistedCompanyId, name: "Unlisted company", issuePrefix: `O${unlistedCompanyId.slice(0, 6)}` },
{ id: disabledCompanyId, name: "Disabled company", issuePrefix: `D${disabledCompanyId.slice(0, 6)}` },
]);
await db.insert(plugins).values([
{ id: managedId, pluginKey: "paperclip.daytona-sandbox-provider", packageName: "managed", version: "1.0.0", status: "ready", packagePath: "/old/image/plugins/daytona", manifestJson },
{ id: ungrantedId, pluginKey: ungrantedId, packageName: "ungranted", version: "1.0.0", status: "ready", manifestJson },
]);
await db.insert(pluginCompanySettings).values({ pluginId: managedId, companyId: disabledCompanyId, enabled: false });
const resourceId = await event();
await db.transaction(async tx => {
await recordResourceCreationEvent(tx as unknown as Db, unlistedCompanyId, "project", randomUUID());
await recordResourceCreationEvent(tx as unknown as Db, disabledCompanyId, "project", randomUUID());
});
const d = delivery();
await d.run();
const delivered = (id: string) => d.call.mock.calls.filter(([plugin]) => plugin === id).map(([, , input]) => input.resourceId);
expect(delivered(managedId)).toEqual([resourceId]);
expect(delivered(ungrantedId)).toEqual([]);
expect(delivered(pluginId)).toEqual([resourceId]);
d.worker.dispose();
});
});
it("listens for a managed plugin with no config row until its last enabled company disables it", async () => {
await withManagedPlugin([companyId], async () => {
await db.delete(plugins);
const managedId = randomUUID();
await db.insert(plugins).values({ id: managedId, pluginKey: "paperclip.daytona-sandbox-provider", packageName: "managed", version: "1.0.0", status: "ready",
manifestJson: { resourceLifecycle: ["agent_runtime"], capabilities: ["events.subscribe"] } as never });
const subscription = { unlisten: vi.fn(), state: {} };
vi.mocked(db.$client.listen).mockResolvedValueOnce(subscription as never);
const d = delivery();
await d.worker.ready;
expect(db.$client.listen).toHaveBeenCalledWith("paperclip_plugin_lifecycle", expect.any(Function), expect.any(Function));
await db.insert(pluginCompanySettings).values({ pluginId: managedId, companyId, enabled: false });
d.applied();
await vi.waitFor(() => expect(subscription.unlisten).toHaveBeenCalledTimes(1));
d.worker.dispose();
});
});
it("does not listen for a managed plugin that the document enables for no company", async () => {
const root = await mkdtemp(path.join(tmpdir(), "delivery-managed-"));
await mkdir(path.join(root, "sandbox-providers/daytona"), { recursive: true });
vi.stubEnv("PAPERCLIP_BUNDLED_PLUGIN_ROOT", root);
vi.stubEnv("PAPERCLIP_MANAGED_CONFIG", JSON.stringify({ v: 1, mode: "cloud", catalogVersion: "1", features: {},
plugins: { autoInstall: ["daytona"], workerEnv: { daytona: ["PROVIDER_ORIGIN"] } } }));
try {
await db.delete(plugins);
await db.insert(plugins).values({ id: randomUUID(), pluginKey: "paperclip.daytona-sandbox-provider", packageName: "managed", version: "1.0.0", status: "ready",
manifestJson: { resourceLifecycle: ["agent_runtime"], capabilities: ["events.subscribe"] } as never });
const d = delivery();
await d.worker.ready;
expect(db.$client.listen).not.toHaveBeenCalled();
d.worker.dispose();
} finally { vi.unstubAllEnvs(); await rm(root, { recursive: true, force: true }); }
});
it("delivers project events to a storage provider only for projects attached to it", async () => {
await db.update(plugins).set({ manifestJson: { resourceLifecycle: ["project"], capabilities: ["events.subscribe", "workspaces.manage"], workspaceProvider: { modes: ["isolated_workspace"], strategies: ["git_worktree"] } } as never }).where(eq(plugins.id, pluginId));
const [core] = await db.insert(projects).values({ companyId, name: "Core project" }).returning();
@@ -1,4 +1,7 @@
import { randomBytes, randomUUID } from "node:crypto";
import { mkdir, mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
import { and, eq } from "drizzle-orm";
import {
@@ -450,6 +453,35 @@ const support = await getEmbeddedPostgresTestSupport();
},
);
describe("project instance secret patches", () => {
it.skipIf(!support.supported)("requires a config row unless the managed config enables the plugin for this company", async () => {
const root = await mkdtemp(path.join(tmpdir(), "instance-secrets-managed-"));
await mkdir(path.join(root, "sandbox-providers/daytona"), { recursive: true });
const database = await startEmbeddedPostgresTestDatabase("paperclip-instance-secrets-managed-");
const db = createDb(database.connectionString);
try {
const companyId = randomUUID(), pluginId = randomUUID();
await db.insert(companies).values({ id: companyId, name: "Managed credentials", issuePrefix: "MCI" });
await db.insert(plugins).values({ id: pluginId, pluginKey: "paperclip.daytona-sandbox-provider", packageName: "managed", version: "1.0.0", status: "ready", packagePath: "/old/image/plugins/daytona",
manifestJson: { resourceLifecycle: ["project"], capabilities: ["events.subscribe", "projects.instance-secrets"] } as never });
const [project] = await db.insert(projects).values({ companyId, name: "Credential owner" }).returning();
await db.transaction(tx => recordResourceCreationEvent(tx as unknown as Db, companyId, "project", project.id));
const [event] = await pluginLifecycleInbox(db, pluginId).list(companyId, 100);
await expect(projectInstanceSecretService(db, pluginId).prepare(event)).rejects.toThrow("require plugin configuration");
vi.stubEnv("PAPERCLIP_BUNDLED_PLUGIN_ROOT", root);
vi.stubEnv("PAPERCLIP_MANAGED_CONFIG", JSON.stringify({ v: 1, mode: "cloud", catalogVersion: "1", features: {},
plugins: { autoInstall: ["daytona"], workerEnv: { daytona: ["PROVIDER_ORIGIN"] }, enabledCompanies: { daytona: [randomUUID()] } } }));
await expect(projectInstanceSecretService(db, pluginId).prepare(event)).rejects.toThrow("require plugin configuration");
vi.stubEnv("PAPERCLIP_MANAGED_CONFIG", JSON.stringify({ v: 1, mode: "cloud", catalogVersion: "1", features: {},
plugins: { autoInstall: ["daytona"], enabledCompanies: { daytona: [companyId] } } }));
await expect(projectInstanceSecretService(db, pluginId).prepare(event)).resolves.toMatchObject({ instanceSecrets: expect.any(Object) });
} finally {
vi.unstubAllEnvs();
await db.$client.end({ timeout: 0 });
await database.cleanup();
await rm(root, { recursive: true, force: true });
}
}, 90_000);
it("accepts bounded bundles and removal and rejects malformed patches without leaking values", () => {
expect(() =>
validateProjectInstanceSecretChanges({
+75 -4
View File
@@ -49,7 +49,11 @@ export interface ManagedInstanceConfig {
/** App feature-catalog version the document was validated against. */
catalogVersion: string;
features: Readonly<Partial<Record<ManagedExperimentalFeatureKey, boolean>>>;
plugins: { readonly autoInstall: readonly string[] };
plugins: {
readonly autoInstall: readonly string[];
readonly workerEnv?: Readonly<Record<string, readonly string[]>>;
readonly enabledCompanies?: Readonly<Record<string, readonly string[]>>;
};
/** Sandbox environments the control plane provisions at boot (empty when the section is absent). */
environments: readonly ManagedEnvironmentSpec[];
}
@@ -82,6 +86,12 @@ export interface ManagedEnvironmentSpec {
*/
export const SECRET_LIKE_CONFIG_KEY_PATTERN = /(api[-_]?key|token|secret|password|credential)/i;
const WORKER_ENV_NAME_PATTERN = /^[A-Z][A-Z0-9_]*$/;
const COMPANY_ID_PATTERN = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
const WORKER_ENV_DENIED_NAME_PATTERN =
/(^|_)(AWS|CREDENTIAL|CREDENTIALS|DATABASE|DSN|JWT|KEY|KEYS|PASSWORD|POSTGRES|PRIVATE|SECRET|SECRETS|SIGNING|TOKEN)(_|$)|^(DB|DYLD|LD|NODE)_|^(PATH|PAPERCLIP_MANAGED_CONFIG)$/;
function findSecretLikeConfigKey(value: Record<string, unknown>, path: string): string | null {
for (const [key, child] of Object.entries(value)) {
const childPath = path.length > 0 ? `${path}.${key}` : key;
@@ -224,8 +234,8 @@ export function parseManagedConfigEnv(env: ManagedConfigEnv): ManagedInstanceCon
fail(`"plugins" must be an object (got ${describeJsonValue(doc.plugins)})`);
}
for (const key of Object.keys(doc.plugins)) {
if (key !== "autoInstall") {
fail(`"plugins" has unknown key "${key}" (allowed: autoInstall)`);
if (key !== "autoInstall" && key !== "workerEnv" && key !== "enabledCompanies") {
fail(`"plugins" has unknown key "${key}" (allowed: autoInstall, workerEnv, enabledCompanies)`);
}
}
const rawAutoInstall = doc.plugins.autoInstall;
@@ -247,6 +257,63 @@ export function parseManagedConfigEnv(env: ManagedConfigEnv): ManagedInstanceCon
autoInstall.push(entry);
}
let workerEnv: Record<string, readonly string[]> | undefined;
if (doc.plugins.workerEnv !== undefined) {
if (!isPlainObject(doc.plugins.workerEnv)) {
fail(`"plugins.workerEnv" must be an object mapping plugin key → variable names (got ${describeJsonValue(doc.plugins.workerEnv)})`);
}
workerEnv = Object.create(null) as Record<string, readonly string[]>;
for (const [key, names] of Object.entries(doc.plugins.workerEnv)) {
if (!autoInstall.includes(key)) {
fail(`"plugins.workerEnv" key "${key}" is not in "plugins.autoInstall"; a grant requires its plugin to be provisioned`);
}
if (!Array.isArray(names) || names.length === 0) {
fail(`"plugins.workerEnv.${key}" must be a non-empty array of variable names (got ${describeJsonValue(names)})`);
}
const granted: string[] = [];
for (const name of names) {
if (typeof name !== "string" || !WORKER_ENV_NAME_PATTERN.test(name)) {
fail(`"plugins.workerEnv.${key}" entries must be uppercase variable names (got ${describeJsonValue(name)})`);
}
if (WORKER_ENV_DENIED_NAME_PATTERN.test(name)) {
fail(`"plugins.workerEnv.${key}" must not grant host credential or loader variable "${name}"`);
}
if (granted.includes(name)) {
fail(`"plugins.workerEnv.${key}" has duplicate entry "${name}"`);
}
granted.push(name);
}
workerEnv[key] = Object.freeze(granted);
}
}
let enabledCompanies: Record<string, readonly string[]> | undefined;
if (doc.plugins.enabledCompanies !== undefined) {
if (!isPlainObject(doc.plugins.enabledCompanies)) {
fail(`"plugins.enabledCompanies" must be an object mapping plugin key → company ids (got ${describeJsonValue(doc.plugins.enabledCompanies)})`);
}
enabledCompanies = Object.create(null) as Record<string, readonly string[]>;
for (const [key, ids] of Object.entries(doc.plugins.enabledCompanies)) {
if (!autoInstall.includes(key)) {
fail(`"plugins.enabledCompanies" key "${key}" is not in "plugins.autoInstall"; enabling a company requires its plugin to be provisioned`);
}
if (!Array.isArray(ids) || ids.length === 0) {
fail(`"plugins.enabledCompanies.${key}" must be a non-empty array of company ids (got ${describeJsonValue(ids)})`);
}
const companyIds: string[] = [];
for (const id of ids) {
if (typeof id !== "string" || !COMPANY_ID_PATTERN.test(id)) {
fail(`"plugins.enabledCompanies.${key}" entries must be lowercase company UUIDs (got ${describeJsonValue(id)})`);
}
if (companyIds.includes(id)) {
fail(`"plugins.enabledCompanies.${key}" has duplicate entry "${id}"`);
}
companyIds.push(id);
}
enabledCompanies[key] = Object.freeze(companyIds);
}
}
// `environments` is OPTIONAL, unlike `features` and `plugins`: documents
// delivered before the section existed must keep booting newer builds (a
// fleet image roll cannot be lockstepped with a config re-delivery), and
@@ -332,7 +399,11 @@ export function parseManagedConfigEnv(env: ManagedConfigEnv): ManagedInstanceCon
mode: "cloud",
catalogVersion: doc.catalogVersion,
features: Object.freeze(features),
plugins: Object.freeze({ autoInstall: Object.freeze(autoInstall) }),
plugins: Object.freeze({
autoInstall: Object.freeze(autoInstall),
...(workerEnv ? { workerEnv: Object.freeze(workerEnv) } : {}),
...(enabledCompanies ? { enabledCompanies: Object.freeze(enabledCompanies) } : {}),
}),
environments: Object.freeze(environmentSpecs),
}) as ManagedInstanceConfig;
}
@@ -0,0 +1,29 @@
import { resolveBundledCatalogRoot, resolveBundledPluginInstalls, type ResolvedBundledPlugin } from "./bundled-plugins.js";
import { getManagedInstanceConfig, type ManagedConfigEnv, type ManagedInstanceConfig } from "./managed-config.js";
let cache: { config: ManagedInstanceConfig; catalogRoot: string; installs: readonly ResolvedBundledPlugin[] } | null = null;
function managedInstalls(config: ManagedInstanceConfig, env: ManagedConfigEnv) {
const catalogRoot = resolveBundledCatalogRoot(env);
if (cache?.config === config && cache.catalogRoot === catalogRoot) return cache.installs;
const keys = new Set([...Object.keys(config.plugins.workerEnv ?? {}), ...Object.keys(config.plugins.enabledCompanies ?? {})]);
const installs = resolveBundledPluginInstalls([...keys], { catalogRoot, env, enforceCatalogRoot: true });
cache = { config, catalogRoot, installs };
return installs;
}
function managedEntry(section: "workerEnv" | "enabledCompanies", pluginKey: string, env: ManagedConfigEnv): readonly string[] {
const config = getManagedInstanceConfig(env);
const entries = config?.plugins[section];
if (!config || !entries) return [];
const key = managedInstalls(config, env).find(install => install.pluginKey === pluginKey)?.key;
return key !== undefined && Object.hasOwn(entries, key) ? entries[key]! : [];
}
export function managedPluginWorkerEnvGrant(pluginKey: string, env: ManagedConfigEnv = process.env): readonly string[] {
return managedEntry("workerEnv", pluginKey, env);
}
export function managedPluginEnabledCompanies(pluginKey: string, env: ManagedConfigEnv = process.env): readonly string[] {
return managedEntry("enabledCompanies", pluginKey, env);
}
@@ -1,12 +1,13 @@
import { projectInstanceSecretService } from "./project-instance-secrets.js";
import { and, asc, eq, isNull, or, sql } from "drizzle-orm";
import { pluginCompanySettings, pluginConfig, plugins, projects, type Db } from "@paperclipai/db";
import { and, asc, eq, inArray, isNull, or, sql } from "drizzle-orm";
import { companies, pluginCompanySettings, pluginConfig, plugins, projects, type Db } from "@paperclipai/db";
import { logger } from "../middleware/logger.js";
import { pluginLifecycleInbox } from "./plugin-lifecycle-inbox.js";
import { DELIVERY_QUEUES } from "./delivery-work-notifications.js";
import type { createDeliveryWorkCoordinator } from "./delivery-work-coordinator.js";
import { parseProjectExecutionWorkspacePolicy } from "./execution-workspace-policy.js";
import { isWorkspaceProvider } from "./project-workspace-provider.js";
import { managedPluginEnabledCompanies } from "./managed-plugin-worker-env.js";
import type { PluginWorkerManager } from "./plugin-worker-manager.js";
import type { PluginLifecycleManager } from "./plugin-lifecycle.js";
@@ -26,6 +27,34 @@ export function registerPluginLifecycleDelivery(
const retryAt = new Map<string, number>();
const refreshedRuntimeCompanies = new Set<string>();
const retryMs = 30_000;
async function managedTargets() {
const ready = await db.select().from(plugins).where(eq(plugins.status, "ready"));
const targets = ready.flatMap(plugin => managedPluginEnabledCompanies(plugin.pluginKey).map(companyId => ({ plugin, companyId })));
if (!targets.length) return targets;
const existing = await db.select({ id: companies.id }).from(companies)
.where(inArray(companies.id, targets.map(target => target.companyId)));
const existingIds = new Set(existing.map(company => company.id));
return targets.filter(target => existingIds.has(target.companyId));
}
async function enabledManagedPlugins() {
const targets = await managedTargets();
if (!targets.length) return [];
const disabled = await db.select({ pluginId: pluginCompanySettings.pluginId, companyId: pluginCompanySettings.companyId }).from(pluginCompanySettings)
.where(and(inArray(pluginCompanySettings.pluginId, targets.map(target => target.plugin.id)), eq(pluginCompanySettings.enabled, false)));
const disabledKeys = new Set(disabled.map(row => `${row.pluginId}:${row.companyId}`));
return targets.filter(target => !disabledKeys.has(`${target.plugin.id}:${target.companyId}`)).map(target => target.plugin);
}
async function deliveryTargets() {
const configured = await db.select({ plugin: plugins, companyId: pluginConfig.companyId })
.from(plugins).innerJoin(pluginConfig, eq(pluginConfig.pluginId, plugins.id))
.where(eq(plugins.status, "ready"))
.orderBy(asc(plugins.installOrder), asc(pluginConfig.companyId));
const managed = await managedTargets();
if (!managed.length) return configured;
const seen = new Set(configured.map(target => `${target.plugin.id}:${target.companyId}`));
return [...configured, ...managed.filter(target => !seen.has(`${target.plugin.id}:${target.companyId}`))].sort((a, b) =>
(a.plugin.installOrder ?? Infinity) - (b.plugin.installOrder ?? Infinity) || a.companyId.localeCompare(b.companyId));
}
const delivery = coordinator.register(DELIVERY_QUEUES.pluginLifecycle, {
retryMs,
hasPending: async () => pending,
@@ -44,11 +73,7 @@ export function registerPluginLifecycleDelivery(
try {
await ensureListening();
for (const [key, at] of retryAt) if (at <= Date.now()) retryAt.delete(key);
const configured = await db.select({ plugin: plugins, companyId: pluginConfig.companyId })
.from(plugins).innerJoin(pluginConfig, eq(pluginConfig.pluginId, plugins.id))
.where(eq(plugins.status, "ready"))
.orderBy(asc(plugins.installOrder), asc(pluginConfig.companyId));
for (const { plugin, companyId } of configured) {
for (const { plugin, companyId } of await deliveryTargets()) {
const types = plugin.manifestJson.resourceLifecycle;
if (!types?.length || !plugin.manifestJson.capabilities.includes("events.subscribe")) continue;
const [disabled] = await db.select({ id: pluginCompanySettings.id }).from(pluginCompanySettings)
@@ -134,10 +159,11 @@ export function registerPluginLifecycleDelivery(
function ensureListening() {
const update = listenerUpdate.then(async () => {
if (disposed || !db.$client?.listen) return;
const installed = await db.select({ manifest: plugins.manifestJson }).from(plugins)
const configured = await db.select({ manifest: plugins.manifestJson }).from(plugins)
.innerJoin(pluginConfig, eq(pluginConfig.pluginId, plugins.id))
.leftJoin(pluginCompanySettings, and(eq(pluginCompanySettings.pluginId, plugins.id), eq(pluginCompanySettings.companyId, pluginConfig.companyId)))
.where(and(eq(plugins.status, "ready"), or(isNull(pluginCompanySettings.enabled), eq(pluginCompanySettings.enabled, true))));
const installed = [...configured, ...(await enabledManagedPlugins()).map(plugin => ({ manifest: plugin.manifestJson }))];
if (disposed) return;
const wanted = installed.some(plugin => plugin.manifest.resourceLifecycle?.includes("agent_runtime") && plugin.manifest.capabilities?.includes("events.subscribe"));
if (wanted) listening ??= await db.$client.listen("paperclip_plugin_lifecycle", wake, wake);
+7
View File
@@ -53,6 +53,7 @@ import type { PluginToolDispatcher } from "./plugin-tool-dispatcher.js";
import type { PluginLifecycleManager } from "./plugin-lifecycle.js";
import { pluginDatabaseService } from "./plugin-database.js";
import { resolveBundledCatalogRoot } from "./bundled-plugins.js";
import { managedPluginWorkerEnvGrant } from "./managed-plugin-worker-env.js";
const execFileAsync = promisify(execFile);
const __dirname = path.dirname(fileURLToPath(import.meta.url));
@@ -160,6 +161,7 @@ export function buildPluginWorkerEnv(input: {
packagePath?: string | null;
/** Test seam; defaults to the repo plugin tree and the bundled catalog root. */
trustedLocalPluginRoots?: readonly string[];
grantedEnvVars?: readonly string[];
instanceInfo: { deploymentMode?: string | null; deploymentExposure?: string | null };
processEnv?: NodeJS.ProcessEnv;
}): Record<string, string> {
@@ -168,6 +170,10 @@ export function buildPluginWorkerEnv(input: {
PAPERCLIP_DEPLOYMENT_MODE: input.instanceInfo.deploymentMode ?? "",
PAPERCLIP_DEPLOYMENT_EXPOSURE: input.instanceInfo.deploymentExposure ?? "",
};
for (const key of input.grantedEnvVars ?? []) {
const value = processEnv[key];
if (value && value.trim().length > 0) env[key] = value;
}
const canRegisterEnvironmentDrivers = Array.isArray(input.manifest.capabilities)
&& input.manifest.capabilities.includes("environment.drivers.register");
if (!canRegisterEnvironmentDrivers) return env;
@@ -2341,6 +2347,7 @@ export function pluginLoader(
packageName: activePlugin.packageName,
packagePath: resolvePackageRoot ? packageRoot : activePlugin.packagePath,
instanceInfo,
grantedEnvVars: managedPluginWorkerEnvGrant(activePlugin.pluginKey),
}),
// Authorize the worker to act on each configured company from its
// proactive loops/timers (LOOA-629). Seeded here so it is in place
@@ -14,6 +14,7 @@ import type { ResourceLifecycleEvent } from "@paperclipai/plugin-sdk";
import { z } from "zod";
import { secretService } from "./secrets.js";
import { persistActivity, type ActivityPublication } from "./activity-log.js";
import { managedPluginEnabledCompanies } from "./managed-plugin-worker-env.js";
export type ProjectSecretSnapshot = { revision: string };
export function validateProjectInstanceSecretChanges(
@@ -82,7 +83,7 @@ export function projectInstanceSecretService(db: Db, pluginId: string) {
),
);
const [config] = await (lock ? configQuery.for("share") : configQuery);
if (!config)
if (!config && !managedPluginEnabledCompanies(plugin.pluginKey).includes(companyId))
throw new Error("Project instance secrets require plugin configuration");
const projectQuery = db
.select()