mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-18 06:01:49 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Plugins can now add connector types to the Connectors catalog (#22004), but only for transports the host already runs (remote MCP, local stdio) > - A plugin cannot yet serve a connector's tools itself, and legacy plugin tools (top-level manifest `tools`) still reach every agent without a connection or install > - A plugin should be able to ship a full connector: settings, credentials, tools, grants, policies and audit, like a built-in one > - This pull request adds the `plugin` transport, the SDK API to serve it, and moves legacy plugin tools under the same activation rules without changing today's access > - The benefit is that any connector can live in a plugin, and agents get plugin tools only when an operator enables them ## Linked Issues or Issue Description Refs #22004 **Subsystem affected** Plugins (SDK, worker protocol, manifest), Connectors (tool access, catalog discovery), tool gateway, database. **Problem or motivation** A plugin connector can only describe an existing transport. There is no way for a plugin worker to serve a connector's tools with per-connection settings and credentials. Legacy plugin tools are visible to every agent as soon as the plugin is enabled, which is the opposite of the connector activation rules. **Proposed solution** Add a `plugin` transport. The gateway keeps every decision (grant, policy, approval, rate limit, audit) and calls the plugin worker with the connection, its settings and a per-call credential. Legacy top-level tools become one implicit connector per plugin. Existing installations keep exactly their current access through a one-time migration. **Alternatives considered** Move legacy tools to connection tool names (`app.<connection>:<tool>`). Rejected: names change, so existing tool policies and agent instructions that name `plugin:tool` would stop matching (a possible widening or loss). Legacy tools keep their names and handlers; only their visibility is now gated by an install of the implicit connector. ## What Changed - `packages/db`: migration `0491` widens `tool_connections_transport_check` with `plugin` (`NOT VALID`, annotated `online_compatible`), and adds `plugin_connector_legacy_access`, seeded with one row per installed legacy-tools plugin and company. - `packages/shared`: method schema gains `settingsSchema` and `tools` (`{ kind: "static", list: [{ name, displayName?, description?, risk, parametersSchema }] } | { kind: "dynamic" }`), only for `transport: "plugin"`; tool names must match `[a-z0-9_]`; plugin-served methods use `none` or `api_key`. The connect API accepts `settings`. - `packages/plugins/sdk`: `ToolRunContext.connection` (`id`, `appSlug`, `methodKey`, `grantId`, `settings`, `credential?`); `ctx.connectors.register(appSlug, toolName, fn)` and `ctx.connectors.listTools(appSlug, fn)`; new `listConnectorTools` RPC; `executeTool` routes connector calls by `appSlug`. A worker refuses to start when a declared static tool (or dynamic lister) has no handler. The test harness gains `executeConnectorTool` and `listConnectorTools`. - `server/src/services/plugin-connector-runtime.ts`: routes one call to the plugin's worker; a stopped worker is a `PluginWorkerUnavailableError`. - Connect flow (`tool-access.ts`): a `plugin` method creates a `paperclip_plugin` application and a `plugin` connection with `config = { sourceTemplateKey, connectionMethodKey, settings }`, validated against `settingsSchema`. Credentials, grants, credential policy, installs and profiles use the existing code. - Catalog: static tools come from the definition, dynamic tools from `listConnectorTools`, with the existing version hashing and retirement. Tool risk comes from the declaration. - Gateway: plugin connections join connected-tool discovery and both execution paths (direct and approved actions). For each call the gateway resolves the grant and, for `api_key`, the credential for that call only. The run context comes only from the session. A stopped worker returns `503 plugin_connector_unavailable` with `retryable: true`. The token broker refuses to export plugin connector credentials. - Legacy tools: a plugin with top-level `tools` and no `apps` gets an implicit connector (slug `default`, named after the plugin, no authentication). Its tools keep their names, handlers and today's access decision, and an agent sees them only when a connection of that connector is installed for it or for the company. The connector's saved action rules (Off, Ask first) can only restrict them. `completeLegacyPluginToolAccess` turns each migration row into such a connection with a company install, at boot and when a disabled plugin is enabled again; it is single-flight across servers, resumes a draft left by an interrupted attempt, and never overrides an operator's later changes. - Catalog: tools a dynamic connector stops offering are retired from discovery; `listConnectorTools` carries the connection's `companyId`, and uses the acting person's grant for personal connections. - Board: the setup flow renders `settingsSchema` with the shared JSON Schema form, sends `settings`, and restores saved settings on resume and reconnect. - `create-paperclip-plugin --template connector` now scaffolds a plugin connector (`apps` + `ctx.connectors.register` + a harness test). SDK README and `PLUGIN_SPEC.md` describe connectors; top-level `tools` stay documented as legacy. - `inferToolRisk` moves to `tool-risk-inference.ts` so the catalog can reuse it. Design notes where the spec left room: - OAuth for `plugin`-transport methods is not supported in this PR (`none` and `api_key` only); OAuth plugin connectors can use `mcp_remote` today. Noted as follow-up. - Board "Test" calls on a plugin connection pass an empty `agentId`/`runId`, because there is no agent run. - Companies created after the upgrade get no automatic legacy connection; they follow the normal activation rules. ## Verification - `server`: `npx vitest run src/__tests__/plugin-connectors-plugin-transport.test.ts` (new): - rules 1-3: a ready plugin and configured accounts give no tools; an install for agent A gives only A the tools; agent B is denied on invocation; - the worker receives `runContext.connection` with the right id, method, grant, settings and the call's credential; spoofed arguments do not change it; each account calls with its own key; results do not contain the key; - invalid settings are rejected (422); a stopped worker is `503 plugin_connector_unavailable` (retryable); a disabled plugin hides the tools; - dynamic tools are listed through the worker and a tool it stops offering is retired; - rule 5: a new installation of a legacy-tools plugin gives no tools; an installation recorded by the migration keeps the same tool names for every agent, through one company-wide connection (two concurrent carry-overs make one; an interrupted draft is finished; a retry keeps an operator's restriction), and is not also exposed as connection tools; Ask first and Off on the implicit connector restrict the legacy tools; - the token broker refuses to export a plugin connection's credential. - Existing gateway plugin-tool tests (`tool-gateway`, `tool-gateway-service`, `prompt-injection-gateway`) now set up the legacy grant an existing installation has, and pass unchanged otherwise. - Regression: `plugin-connectors-circleback-parity`, `tool-gateway`, `tool-gateway-service`, `prompt-injection-gateway`, `tool-access-service` (846 passed), plus `connector-tool-policy`, `plugin-lifecycle-restart`, `remote-mcp-connectors`, `ai-connection-router` in an earlier run. - UI: `AppsConnect.test.tsx` covers sending plugin settings from the settings schema. - SDK: `tests/worker-rpc-host.test.ts` (start refusal without handlers, connector routing, tool listing). Scaffold: `entrypoints.test.ts`. - `packages/db`: `pnpm typecheck` including `check:migrations`; server layered typecheck (sources and tests); `tsc --noEmit` for shared, SDK, scaffold and UI. ## Risks - Behavior change for new installations: legacy plugin tools no longer reach agents until an operator connects the plugin's connector. Installations that exist at upgrade time keep their tools through the migration rows. - The migration widens a check constraint (`NOT VALID`) and copies plugin × company rows into a new table. Both are fast; the previous release never writes `plugin`. - Calls depend on the plugin worker; when it is down the agent gets a retryable error. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file editing, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
886 lines
32 KiB
TypeScript
886 lines
32 KiB
TypeScript
/**
|
||
* PluginLifecycleManager — state-machine controller for plugin status
|
||
* transitions and worker process coordination.
|
||
*
|
||
* Each plugin moves through a well-defined state machine:
|
||
*
|
||
* ```
|
||
* installed ──→ ready ──→ disabled
|
||
* │ │ │
|
||
* │ ├──→ error│
|
||
* │ ↓ │
|
||
* │ upgrade_pending │
|
||
* │ │ │
|
||
* ↓ ↓ ↓
|
||
* uninstalled
|
||
* ```
|
||
*
|
||
* The lifecycle manager:
|
||
*
|
||
* 1. **Validates transitions** — Only transitions defined in
|
||
* `VALID_TRANSITIONS` are allowed; invalid transitions throw.
|
||
*
|
||
* 2. **Coordinates workers** — When a plugin moves to `ready`, its
|
||
* worker process is started. When it moves out of `ready`, the
|
||
* worker is stopped gracefully.
|
||
*
|
||
* 3. **Emits events** — `plugin.loaded`, `plugin.enabled`,
|
||
* `plugin.disabled`, `plugin.unloaded`, `plugin.status_changed`
|
||
* events are emitted so that other services (job coordinator,
|
||
* tool dispatcher, event bus) can react accordingly.
|
||
*
|
||
* 4. **Persists state** — Status changes are written to the database
|
||
* through the plugin registry service.
|
||
*
|
||
* @see PLUGIN_SPEC.md §12 — Process Model
|
||
* @see PLUGIN_SPEC.md §12.5 — Graceful Shutdown Policy
|
||
*/
|
||
import { EventEmitter } from "node:events";
|
||
import type { Db } from "@paperclipai/db";
|
||
import type {
|
||
PluginStatus,
|
||
PluginRecord,
|
||
PaperclipPluginManifestV1,
|
||
} from "@paperclipai/shared";
|
||
import { pluginRegistryService } from "./plugin-registry.js";
|
||
import { pluginLoader, type PluginLoader } from "./plugin-loader.js";
|
||
import type { PluginWorkerManager, WorkerStartOptions } from "./plugin-worker-manager.js";
|
||
import { badRequest, notFound } from "../errors.js";
|
||
import { logger } from "../middleware/logger.js";
|
||
|
||
// ---------------------------------------------------------------------------
|
||
// Lifecycle state machine
|
||
// ---------------------------------------------------------------------------
|
||
|
||
/**
|
||
* Valid state transitions for the plugin lifecycle.
|
||
*
|
||
* installed → ready (initial load succeeds)
|
||
* installed → error (initial load fails)
|
||
* installed → uninstalled (abort installation)
|
||
*
|
||
* ready → disabled (operator disables plugin)
|
||
* ready → error (runtime failure)
|
||
* ready → upgrade_pending (upgrade with new capabilities)
|
||
* ready → uninstalled (uninstall)
|
||
*
|
||
* disabled → ready (operator re-enables plugin)
|
||
* disabled → uninstalled (uninstall while disabled)
|
||
*
|
||
* error → ready (retry / recovery)
|
||
* error → uninstalled (give up and uninstall)
|
||
*
|
||
* upgrade_pending → ready (operator approves new capabilities)
|
||
* upgrade_pending → error (upgrade worker fails)
|
||
* upgrade_pending → uninstalled (reject upgrade and uninstall)
|
||
*
|
||
* uninstalled → installed (reinstall)
|
||
*/
|
||
const VALID_TRANSITIONS: Record<string, readonly PluginStatus[]> = {
|
||
installed: ["ready", "error", "uninstalled"],
|
||
ready: ["ready", "disabled", "error", "upgrade_pending", "uninstalled"],
|
||
disabled: ["ready", "uninstalled"],
|
||
error: ["ready", "uninstalled"],
|
||
upgrade_pending: ["ready", "error", "uninstalled"],
|
||
uninstalled: ["installed"], // reinstall
|
||
};
|
||
|
||
/**
|
||
* Check whether a transition from `from` → `to` is valid.
|
||
*/
|
||
function isValidTransition(from: PluginStatus, to: PluginStatus): boolean {
|
||
return VALID_TRANSITIONS[from]?.includes(to) ?? false;
|
||
}
|
||
|
||
// ---------------------------------------------------------------------------
|
||
// Lifecycle events
|
||
// ---------------------------------------------------------------------------
|
||
|
||
/**
|
||
* Events emitted by the PluginLifecycleManager.
|
||
* Consumers can subscribe to these for routing-table updates, UI refresh
|
||
* notifications, and observability.
|
||
*/
|
||
export interface PluginLifecycleEvents {
|
||
/** Emitted after a plugin is loaded (installed → ready). */
|
||
"plugin.loaded": { pluginId: string; pluginKey: string };
|
||
/** Emitted after a plugin transitions to ready (enabled). */
|
||
"plugin.enabled": { pluginId: string; pluginKey: string };
|
||
/** Emitted after a plugin is disabled (ready → disabled). */
|
||
"plugin.disabled": { pluginId: string; pluginKey: string; reason?: string };
|
||
/** Emitted after a plugin is unloaded (any → uninstalled). */
|
||
"plugin.unloaded": { pluginId: string; pluginKey: string; removeData: boolean };
|
||
/** Emitted on any status change. */
|
||
"plugin.status_changed": {
|
||
pluginId: string;
|
||
pluginKey: string;
|
||
previousStatus: PluginStatus;
|
||
newStatus: PluginStatus;
|
||
};
|
||
/** Emitted when a plugin enters an error state. */
|
||
"plugin.error": { pluginId: string; pluginKey: string; error: string };
|
||
/** Emitted when a plugin enters upgrade_pending. */
|
||
"plugin.upgrade_pending": { pluginId: string; pluginKey: string };
|
||
/** Emitted when a plugin worker process has been started. */
|
||
"plugin.worker_started": { pluginId: string; pluginKey: string };
|
||
/** Emitted when a plugin worker process has been stopped. */
|
||
"plugin.worker_stopped": { pluginId: string; pluginKey: string };
|
||
}
|
||
|
||
type LifecycleEventName = keyof PluginLifecycleEvents;
|
||
type LifecycleEventPayload<K extends LifecycleEventName> = PluginLifecycleEvents[K];
|
||
|
||
// ---------------------------------------------------------------------------
|
||
// PluginLifecycleManager
|
||
// ---------------------------------------------------------------------------
|
||
|
||
export interface PluginLifecycleManager {
|
||
/**
|
||
* Load a newly installed plugin – transitions `installed` → `ready`.
|
||
*
|
||
* This is called after the registry has persisted the initial install record.
|
||
* The caller should have already spawned the worker and performed health
|
||
* checks before calling this. If the worker fails, call `markError` instead.
|
||
*/
|
||
load(pluginId: string): Promise<PluginRecord>;
|
||
|
||
/**
|
||
* Enable a plugin that is in `disabled`, `error`, or `upgrade_pending` state.
|
||
* Transitions → `ready`.
|
||
*/
|
||
enable(pluginId: string): Promise<PluginRecord>;
|
||
|
||
/**
|
||
* Disable a running plugin.
|
||
* Transitions `ready` → `disabled`.
|
||
*/
|
||
disable(pluginId: string, reason?: string): Promise<PluginRecord>;
|
||
|
||
/**
|
||
* Unload (uninstall) a plugin from any active state.
|
||
* Transitions → `uninstalled`.
|
||
*
|
||
* When `removeData` is true, the plugin row and cascaded config are
|
||
* hard-deleted. Otherwise a soft-delete sets status to `uninstalled`.
|
||
*/
|
||
unload(pluginId: string, removeData?: boolean): Promise<PluginRecord | null>;
|
||
|
||
/**
|
||
* Mark a plugin as errored (e.g. worker crash, health-check failure).
|
||
* Transitions → `error`.
|
||
*/
|
||
markError(pluginId: string, error: string): Promise<PluginRecord>;
|
||
|
||
/**
|
||
* Mark a plugin as requiring upgrade approval.
|
||
* Transitions `ready` → `upgrade_pending`.
|
||
*/
|
||
markUpgradePending(pluginId: string): Promise<PluginRecord>;
|
||
|
||
/**
|
||
* Upgrade a plugin to a newer version.
|
||
* This is a placeholder that handles the lifecycle state transition.
|
||
* The actual package installation is handled by plugin-loader.
|
||
*
|
||
* If the upgrade adds new capabilities, transitions to `upgrade_pending`.
|
||
* Otherwise, transitions to `ready` directly.
|
||
*/
|
||
upgrade(pluginId: string, version?: string): Promise<PluginRecord>;
|
||
|
||
/**
|
||
* Start the worker process for a plugin that is already in `ready` state.
|
||
*
|
||
* This is used by the server startup orchestration to start workers for
|
||
* plugins that were persisted as `ready`. It requires a `PluginWorkerManager`
|
||
* to have been provided at construction time.
|
||
*
|
||
* @param pluginId - The UUID of the plugin to start
|
||
* @param options - Worker start options (entrypoint path, config, etc.)
|
||
* @throws if no worker manager is configured or the plugin is not ready
|
||
*/
|
||
startWorker(pluginId: string, options: WorkerStartOptions): Promise<void>;
|
||
|
||
/**
|
||
* Stop the worker process for a plugin without changing lifecycle state.
|
||
*
|
||
* This is used during server shutdown to gracefully stop all workers.
|
||
* It does not transition the plugin state — plugins remain in their
|
||
* current status so they can be restarted on next server boot.
|
||
*
|
||
* @param pluginId - The UUID of the plugin to stop
|
||
*/
|
||
stopWorker(pluginId: string): Promise<void>;
|
||
|
||
/**
|
||
* Restart the worker process for a running plugin.
|
||
*
|
||
* Stops and re-starts the worker process. The plugin remains in `ready`
|
||
* state throughout. This is typically called after a config change.
|
||
*
|
||
* @param pluginId - The UUID of the plugin to restart
|
||
* @throws if no worker manager is configured or the plugin is not ready
|
||
*/
|
||
restartWorker(pluginId: string): Promise<void>;
|
||
|
||
/**
|
||
* Get the current lifecycle state for a plugin.
|
||
*/
|
||
getStatus(pluginId: string): Promise<PluginStatus | null>;
|
||
|
||
/**
|
||
* Check whether a transition is allowed from the plugin's current state.
|
||
*/
|
||
canTransition(pluginId: string, to: PluginStatus): Promise<boolean>;
|
||
|
||
/**
|
||
* Subscribe to lifecycle events.
|
||
*/
|
||
on<K extends LifecycleEventName>(
|
||
event: K,
|
||
listener: (payload: LifecycleEventPayload<K>) => void,
|
||
): void;
|
||
|
||
/**
|
||
* Unsubscribe from lifecycle events.
|
||
*/
|
||
off<K extends LifecycleEventName>(
|
||
event: K,
|
||
listener: (payload: LifecycleEventPayload<K>) => void,
|
||
): void;
|
||
|
||
/**
|
||
* Subscribe to a lifecycle event once.
|
||
*/
|
||
once<K extends LifecycleEventName>(
|
||
event: K,
|
||
listener: (payload: LifecycleEventPayload<K>) => void,
|
||
): void;
|
||
}
|
||
|
||
// ---------------------------------------------------------------------------
|
||
// Factory
|
||
// ---------------------------------------------------------------------------
|
||
|
||
/**
|
||
* Options for constructing a PluginLifecycleManager.
|
||
*/
|
||
export interface PluginLifecycleManagerOptions {
|
||
/** Plugin loader instance. Falls back to the default if omitted. */
|
||
loader?: PluginLoader;
|
||
|
||
/**
|
||
* Worker process manager. When provided, lifecycle transitions that bring
|
||
* a plugin online (load, enable, upgrade-to-ready) will start the worker
|
||
* process, and transitions that take a plugin offline (disable, unload,
|
||
* markError) will stop it.
|
||
*
|
||
* When omitted the lifecycle manager operates in state-only mode — the
|
||
* caller is responsible for managing worker processes externally.
|
||
*/
|
||
workerManager?: PluginWorkerManager;
|
||
}
|
||
|
||
/**
|
||
* Create a PluginLifecycleManager.
|
||
*
|
||
* This service orchestrates plugin state transitions on top of the
|
||
* `pluginRegistryService` (which handles raw DB persistence). It enforces
|
||
* the lifecycle state machine, emits events for downstream consumers
|
||
* (routing tables, UI, observability), and manages worker processes via
|
||
* the `PluginWorkerManager` when one is provided.
|
||
*
|
||
* Usage:
|
||
* ```ts
|
||
* const lifecycle = pluginLifecycleManager(db, {
|
||
* workerManager: createPluginWorkerManager(),
|
||
* });
|
||
* lifecycle.on("plugin.enabled", ({ pluginId }) => { ... });
|
||
* await lifecycle.load(pluginId);
|
||
* ```
|
||
*
|
||
* @see PLUGIN_SPEC.md §21.3 — `plugins.status` column
|
||
* @see PLUGIN_SPEC.md §12 — Process Model
|
||
*/
|
||
async function retireConnectorConnectionsFor(db: Db, pluginKey: string): Promise<void> {
|
||
const { retirePluginAppConnections } = await import("./plugin-app-catalog.js");
|
||
await retirePluginAppConnections(db, pluginKey);
|
||
}
|
||
|
||
export function pluginLifecycleManager(
|
||
db: Db,
|
||
options?: PluginLoader | PluginLifecycleManagerOptions,
|
||
): PluginLifecycleManager {
|
||
// Support the legacy signature: pluginLifecycleManager(db, loader)
|
||
// as well as the new options object form.
|
||
let loaderArg: PluginLoader | undefined;
|
||
let workerManager: PluginWorkerManager | undefined;
|
||
|
||
if (options && typeof options === "object" && "discoverAll" in options) {
|
||
// Legacy: second arg is a PluginLoader directly
|
||
loaderArg = options as PluginLoader;
|
||
} else if (options && typeof options === "object") {
|
||
const opts = options as PluginLifecycleManagerOptions;
|
||
loaderArg = opts.loader;
|
||
workerManager = opts.workerManager;
|
||
}
|
||
|
||
const registry = pluginRegistryService(db);
|
||
const retireConnectorConnections = (pluginKey: string) => retireConnectorConnectionsFor(db, pluginKey);
|
||
/**
|
||
* After a plugin's worker starts with possibly new code: refresh its
|
||
* connector catalogs so declared tools and risks are current, and finish
|
||
* carrying over legacy tool access recorded while it was not running.
|
||
*/
|
||
function afterConnectorPluginActivated(pluginId: string, pluginKey: string) {
|
||
void (async () => {
|
||
const [{ refreshPluginConnectionCatalogs }, { completeLegacyPluginToolAccess }] = await Promise.all([
|
||
import("./plugin-app-catalog.js"),
|
||
import("./plugin-connector-legacy-access.js"),
|
||
]);
|
||
await refreshPluginConnectionCatalogs(db, pluginKey);
|
||
await completeLegacyPluginToolAccess(db, pluginKey);
|
||
})().catch((error) => log.warn({ err: error, pluginId }, "Could not update plugin connectors after activation"));
|
||
}
|
||
const pluginLoaderInstance = loaderArg ?? pluginLoader(db);
|
||
const emitter = new EventEmitter();
|
||
emitter.setMaxListeners(100); // plugins may have many listeners; 100 is a safe upper bound
|
||
|
||
const log = logger.child({ service: "plugin-lifecycle" });
|
||
|
||
// -----------------------------------------------------------------------
|
||
// Internal helpers
|
||
// -----------------------------------------------------------------------
|
||
|
||
async function requirePlugin(pluginId: string): Promise<PluginRecord> {
|
||
const plugin = await registry.getById(pluginId);
|
||
if (!plugin) throw notFound(`Plugin not found: ${pluginId}`);
|
||
return plugin as PluginRecord;
|
||
}
|
||
|
||
function assertTransition(plugin: PluginRecord, to: PluginStatus): void {
|
||
if (!isValidTransition(plugin.status, to)) {
|
||
throw badRequest(
|
||
`Invalid lifecycle transition: ${plugin.status} → ${to} for plugin ${plugin.pluginKey}`,
|
||
);
|
||
}
|
||
}
|
||
|
||
async function transition(
|
||
pluginId: string,
|
||
to: PluginStatus,
|
||
lastError: string | null = null,
|
||
existingPlugin?: PluginRecord,
|
||
): Promise<PluginRecord> {
|
||
const plugin = existingPlugin ?? await requirePlugin(pluginId);
|
||
assertTransition(plugin, to);
|
||
|
||
const previousStatus = plugin.status;
|
||
|
||
const updated = await registry.updateStatus(pluginId, {
|
||
status: to,
|
||
lastError,
|
||
});
|
||
|
||
if (!updated) throw notFound(`Plugin not found after status update: ${pluginId}`);
|
||
const result = updated as PluginRecord;
|
||
|
||
log.info(
|
||
{ pluginId, pluginKey: result.pluginKey, from: previousStatus, to },
|
||
`plugin lifecycle: ${previousStatus} → ${to}`,
|
||
);
|
||
|
||
// Emit the generic status_changed event
|
||
emitter.emit("plugin.status_changed", {
|
||
pluginId,
|
||
pluginKey: result.pluginKey,
|
||
previousStatus,
|
||
newStatus: to,
|
||
});
|
||
|
||
return result;
|
||
}
|
||
|
||
function emitDomain(
|
||
event: LifecycleEventName,
|
||
payload: PluginLifecycleEvents[LifecycleEventName],
|
||
): void {
|
||
emitter.emit(event, payload);
|
||
}
|
||
|
||
// -----------------------------------------------------------------------
|
||
// Worker management helpers
|
||
// -----------------------------------------------------------------------
|
||
|
||
/**
|
||
* Stop the worker for a plugin if one is running.
|
||
* This is a best-effort operation — if no worker manager is configured
|
||
* or no worker is running, it silently succeeds.
|
||
*/
|
||
async function stopWorkerIfRunning(
|
||
pluginId: string,
|
||
pluginKey: string,
|
||
): Promise<void> {
|
||
if (!workerManager) return;
|
||
if (!workerManager.isRunning(pluginId) && !workerManager.getWorker(pluginId)) return;
|
||
|
||
try {
|
||
await workerManager.stopWorker(pluginId);
|
||
log.info({ pluginId, pluginKey }, "plugin lifecycle: worker stopped");
|
||
emitDomain("plugin.worker_stopped", { pluginId, pluginKey });
|
||
} catch (err) {
|
||
log.warn(
|
||
{ pluginId, pluginKey, err: err instanceof Error ? err.message : String(err) },
|
||
"plugin lifecycle: failed to stop worker (best-effort)",
|
||
);
|
||
}
|
||
}
|
||
|
||
async function activateReadyPlugin(pluginId: string): Promise<void> {
|
||
const supportsRuntimeActivation =
|
||
typeof pluginLoaderInstance.hasRuntimeServices === "function"
|
||
&& typeof pluginLoaderInstance.loadSingle === "function";
|
||
if (!supportsRuntimeActivation || !pluginLoaderInstance.hasRuntimeServices()) {
|
||
return;
|
||
}
|
||
|
||
const loadResult = await pluginLoaderInstance.loadSingle(pluginId);
|
||
if (!loadResult.success) {
|
||
throw new Error(
|
||
loadResult.error
|
||
?? `Failed to activate plugin ${loadResult.plugin.pluginKey}`,
|
||
);
|
||
}
|
||
}
|
||
|
||
async function deactivatePluginRuntime(
|
||
pluginId: string,
|
||
pluginKey: string,
|
||
): Promise<void> {
|
||
const supportsRuntimeDeactivation =
|
||
typeof pluginLoaderInstance.hasRuntimeServices === "function"
|
||
&& typeof pluginLoaderInstance.unloadSingle === "function";
|
||
|
||
if (supportsRuntimeDeactivation && pluginLoaderInstance.hasRuntimeServices()) {
|
||
await pluginLoaderInstance.unloadSingle(pluginId, pluginKey);
|
||
return;
|
||
}
|
||
|
||
await stopWorkerIfRunning(pluginId, pluginKey);
|
||
}
|
||
|
||
// -----------------------------------------------------------------------
|
||
// Public API
|
||
// -----------------------------------------------------------------------
|
||
|
||
return {
|
||
// -- load -------------------------------------------------------------
|
||
/**
|
||
* load — Transitions a plugin to 'ready' status and starts its worker.
|
||
*
|
||
* This method is called after a plugin has been successfully installed and
|
||
* validated. It marks the plugin as ready in the database and immediately
|
||
* triggers the plugin loader to start the worker process.
|
||
*
|
||
* @param pluginId - The UUID of the plugin to load.
|
||
* @returns The updated plugin record.
|
||
*/
|
||
async load(pluginId: string): Promise<PluginRecord> {
|
||
const result = await transition(pluginId, "ready");
|
||
await activateReadyPlugin(pluginId);
|
||
|
||
emitDomain("plugin.loaded", {
|
||
pluginId,
|
||
pluginKey: result.pluginKey,
|
||
});
|
||
emitDomain("plugin.enabled", {
|
||
pluginId,
|
||
pluginKey: result.pluginKey,
|
||
});
|
||
return result;
|
||
},
|
||
|
||
// -- enable -----------------------------------------------------------
|
||
/**
|
||
* enable — Re-enables a plugin that was previously in an error or upgrade state.
|
||
*
|
||
* Similar to load(), this method transitions the plugin to 'ready' and starts
|
||
* its worker, but it specifically targets plugins that are currently disabled.
|
||
*
|
||
* @param pluginId - The UUID of the plugin to enable.
|
||
* @returns The updated plugin record.
|
||
*/
|
||
async enable(pluginId: string): Promise<PluginRecord> {
|
||
const plugin = await requirePlugin(pluginId);
|
||
|
||
// Only allow enabling from disabled, error, or upgrade_pending states
|
||
if (plugin.status !== "disabled" && plugin.status !== "error" && plugin.status !== "upgrade_pending") {
|
||
throw badRequest(
|
||
`Cannot enable plugin in status '${plugin.status}'. ` +
|
||
`Plugin must be in 'disabled', 'error', or 'upgrade_pending' status to be enabled.`,
|
||
);
|
||
}
|
||
|
||
const result = await transition(pluginId, "ready", null, plugin);
|
||
await activateReadyPlugin(pluginId);
|
||
afterConnectorPluginActivated(pluginId, result.pluginKey);
|
||
emitDomain("plugin.enabled", {
|
||
pluginId,
|
||
pluginKey: result.pluginKey,
|
||
});
|
||
return result;
|
||
},
|
||
|
||
// -- disable ----------------------------------------------------------
|
||
async disable(pluginId: string, reason?: string): Promise<PluginRecord> {
|
||
const plugin = await requirePlugin(pluginId);
|
||
|
||
// Only allow disabling from ready state
|
||
if (plugin.status !== "ready") {
|
||
throw badRequest(
|
||
`Cannot disable plugin in status '${plugin.status}'. ` +
|
||
`Plugin must be in 'ready' status to be disabled.`,
|
||
);
|
||
}
|
||
|
||
await deactivatePluginRuntime(pluginId, plugin.pluginKey);
|
||
|
||
const result = await transition(pluginId, "disabled", reason ?? null, plugin);
|
||
emitDomain("plugin.disabled", {
|
||
pluginId,
|
||
pluginKey: result.pluginKey,
|
||
reason,
|
||
});
|
||
return result;
|
||
},
|
||
|
||
// -- unload -----------------------------------------------------------
|
||
async unload(
|
||
pluginId: string,
|
||
removeData = false,
|
||
): Promise<PluginRecord | null> {
|
||
const plugin = await requirePlugin(pluginId);
|
||
|
||
// If already uninstalled and removeData, hard-delete
|
||
if (plugin.status === "uninstalled") {
|
||
// Finish any connector cleanup an earlier uninstall could not complete.
|
||
await retireConnectorConnections(plugin.pluginKey);
|
||
if (removeData) {
|
||
await pluginLoaderInstance.cleanupInstallArtifacts(plugin);
|
||
const deleted = await registry.uninstall(pluginId, true);
|
||
log.info(
|
||
{ pluginId, pluginKey: plugin.pluginKey },
|
||
"plugin lifecycle: hard-deleted already-uninstalled plugin",
|
||
);
|
||
emitDomain("plugin.unloaded", {
|
||
pluginId,
|
||
pluginKey: plugin.pluginKey,
|
||
removeData: true,
|
||
});
|
||
return deleted as PluginRecord | null;
|
||
}
|
||
throw badRequest(
|
||
`Plugin ${plugin.pluginKey} is already uninstalled. ` +
|
||
`Use removeData=true to permanently delete it.`,
|
||
);
|
||
}
|
||
|
||
await deactivatePluginRuntime(pluginId, plugin.pluginKey);
|
||
await pluginLoaderInstance.cleanupInstallArtifacts(plugin);
|
||
|
||
// Mark the plugin uninstalled first, so no new connector setup can
|
||
// start. Then retire the connections made from its connector types:
|
||
// rows stay for audit, archived, with grants revoked and owned
|
||
// credentials deleted. A setup already in flight archives its own
|
||
// connection (see tool-access). A failure is reported, and uninstalling
|
||
// again finishes the cleanup; a hard delete happens only after it.
|
||
let result = await registry.uninstall(pluginId, false);
|
||
await retireConnectorConnections(plugin.pluginKey);
|
||
if (removeData) result = await registry.uninstall(pluginId, true);
|
||
|
||
|
||
log.info(
|
||
{ pluginId, pluginKey: plugin.pluginKey, removeData },
|
||
`plugin lifecycle: ${plugin.status} → uninstalled${removeData ? " (hard delete)" : ""}`,
|
||
);
|
||
|
||
emitter.emit("plugin.status_changed", {
|
||
pluginId,
|
||
pluginKey: plugin.pluginKey,
|
||
previousStatus: plugin.status,
|
||
newStatus: "uninstalled" as PluginStatus,
|
||
});
|
||
|
||
emitDomain("plugin.unloaded", {
|
||
pluginId,
|
||
pluginKey: plugin.pluginKey,
|
||
removeData,
|
||
});
|
||
|
||
return result as PluginRecord | null;
|
||
},
|
||
|
||
// -- markError --------------------------------------------------------
|
||
async markError(pluginId: string, error: string): Promise<PluginRecord> {
|
||
// Stop the worker — the plugin is in an error state and should not
|
||
// continue running. The worker manager's auto-restart is disabled
|
||
// because we are intentionally taking the plugin offline.
|
||
const plugin = await requirePlugin(pluginId);
|
||
await deactivatePluginRuntime(pluginId, plugin.pluginKey);
|
||
|
||
const result = await transition(pluginId, "error", error, plugin);
|
||
emitDomain("plugin.error", {
|
||
pluginId,
|
||
pluginKey: result.pluginKey,
|
||
error,
|
||
});
|
||
return result;
|
||
},
|
||
|
||
// -- markUpgradePending -----------------------------------------------
|
||
async markUpgradePending(pluginId: string): Promise<PluginRecord> {
|
||
const plugin = await requirePlugin(pluginId);
|
||
await deactivatePluginRuntime(pluginId, plugin.pluginKey);
|
||
|
||
const result = await transition(pluginId, "upgrade_pending", null, plugin);
|
||
emitDomain("plugin.upgrade_pending", {
|
||
pluginId,
|
||
pluginKey: result.pluginKey,
|
||
});
|
||
return result;
|
||
},
|
||
|
||
// -- upgrade ----------------------------------------------------------
|
||
/**
|
||
* Upgrade a plugin to a newer version by performing a package update and
|
||
* managing the lifecycle state transition.
|
||
*
|
||
* Following PLUGIN_SPEC.md §25.3, the upgrade process:
|
||
* 1. Stops the current worker process (if running).
|
||
* 2. Fetches and validates the new plugin package via the `PluginLoader`.
|
||
* 3. Compares the capabilities declared in the new manifest against the old one.
|
||
* 4. If new capabilities are added, transitions the plugin to `upgrade_pending`
|
||
* to await operator approval (worker stays stopped).
|
||
* 5. If no new capabilities are added, transitions the plugin back to `ready`
|
||
* with the updated version and manifest metadata.
|
||
*
|
||
* @param pluginId - The UUID of the plugin to upgrade.
|
||
* @param version - Optional target version specifier.
|
||
* @returns The updated `PluginRecord`.
|
||
* @throws {BadRequest} If the plugin is not in a ready or upgrade_pending state.
|
||
*/
|
||
async upgrade(pluginId: string, version?: string): Promise<PluginRecord> {
|
||
const plugin = await requirePlugin(pluginId);
|
||
|
||
// Can only upgrade plugins that are ready or already in upgrade_pending
|
||
if (plugin.status !== "ready" && plugin.status !== "upgrade_pending") {
|
||
throw badRequest(
|
||
`Cannot upgrade plugin in status '${plugin.status}'. ` +
|
||
`Plugin must be in 'ready' or 'upgrade_pending' status to be upgraded.`,
|
||
);
|
||
}
|
||
|
||
log.info(
|
||
{ pluginId, pluginKey: plugin.pluginKey, targetVersion: version },
|
||
"plugin lifecycle: upgrade requested",
|
||
);
|
||
|
||
await deactivatePluginRuntime(pluginId, plugin.pluginKey);
|
||
|
||
// 1. Download and validate new package via loader
|
||
const { oldManifest, newManifest, discovered } =
|
||
await pluginLoaderInstance.upgradePlugin(pluginId, { version });
|
||
|
||
log.info(
|
||
{
|
||
pluginId,
|
||
pluginKey: plugin.pluginKey,
|
||
oldVersion: oldManifest.version,
|
||
newVersion: newManifest.version,
|
||
},
|
||
"plugin lifecycle: package upgraded on disk",
|
||
);
|
||
|
||
// 2. Compare capabilities
|
||
const addedCaps = newManifest.capabilities.filter(
|
||
(cap) => !oldManifest.capabilities.includes(cap),
|
||
);
|
||
|
||
// 3. Transition state
|
||
if (addedCaps.length > 0) {
|
||
// New capabilities require operator approval — worker stays stopped
|
||
log.info(
|
||
{ pluginId, pluginKey: plugin.pluginKey, addedCaps },
|
||
"plugin lifecycle: new capabilities detected, transitioning to upgrade_pending",
|
||
);
|
||
// Skip the inner stopWorkerIfRunning since we already stopped above
|
||
const result = await transition(pluginId, "upgrade_pending", null, plugin);
|
||
emitDomain("plugin.upgrade_pending", {
|
||
pluginId,
|
||
pluginKey: result.pluginKey,
|
||
});
|
||
return result;
|
||
} else {
|
||
const result = await transition(pluginId, "ready", null, {
|
||
...plugin,
|
||
version: discovered.version,
|
||
manifestJson: newManifest,
|
||
} as PluginRecord);
|
||
await activateReadyPlugin(pluginId);
|
||
afterConnectorPluginActivated(pluginId, result.pluginKey);
|
||
|
||
emitDomain("plugin.loaded", {
|
||
pluginId,
|
||
pluginKey: result.pluginKey,
|
||
});
|
||
emitDomain("plugin.enabled", {
|
||
pluginId,
|
||
pluginKey: result.pluginKey,
|
||
});
|
||
|
||
return result;
|
||
}
|
||
},
|
||
|
||
// -- startWorker ------------------------------------------------------
|
||
async startWorker(
|
||
pluginId: string,
|
||
options: WorkerStartOptions,
|
||
): Promise<void> {
|
||
if (!workerManager) {
|
||
throw badRequest(
|
||
"Cannot start worker: no PluginWorkerManager is configured. " +
|
||
"Provide a workerManager option when constructing the lifecycle manager.",
|
||
);
|
||
}
|
||
|
||
const plugin = await requirePlugin(pluginId);
|
||
if (plugin.status !== "ready") {
|
||
throw badRequest(
|
||
`Cannot start worker for plugin in status '${plugin.status}'. ` +
|
||
`Plugin must be in 'ready' status.`,
|
||
);
|
||
}
|
||
|
||
log.info(
|
||
{ pluginId, pluginKey: plugin.pluginKey },
|
||
"plugin lifecycle: starting worker",
|
||
);
|
||
|
||
await workerManager.startWorker(pluginId, options);
|
||
emitDomain("plugin.worker_started", {
|
||
pluginId,
|
||
pluginKey: plugin.pluginKey,
|
||
});
|
||
afterConnectorPluginActivated(pluginId, plugin.pluginKey);
|
||
|
||
log.info(
|
||
{ pluginId, pluginKey: plugin.pluginKey },
|
||
"plugin lifecycle: worker started",
|
||
);
|
||
},
|
||
|
||
// -- stopWorker -------------------------------------------------------
|
||
async stopWorker(pluginId: string): Promise<void> {
|
||
if (!workerManager) return; // No worker manager — nothing to stop
|
||
|
||
const plugin = await requirePlugin(pluginId);
|
||
await stopWorkerIfRunning(pluginId, plugin.pluginKey);
|
||
},
|
||
|
||
// -- restartWorker ----------------------------------------------------
|
||
async restartWorker(pluginId: string): Promise<void> {
|
||
if (!workerManager) {
|
||
throw badRequest(
|
||
"Cannot restart worker: no PluginWorkerManager is configured.",
|
||
);
|
||
}
|
||
|
||
const plugin = await requirePlugin(pluginId);
|
||
if (plugin.status !== "ready") {
|
||
throw badRequest(
|
||
`Cannot restart worker for plugin in status '${plugin.status}'. ` +
|
||
`Plugin must be in 'ready' status.`,
|
||
);
|
||
}
|
||
|
||
const handle = workerManager.getWorker(pluginId);
|
||
if (!handle) {
|
||
throw badRequest(
|
||
`Cannot restart worker for plugin "${plugin.pluginKey}": no worker is running.`,
|
||
);
|
||
}
|
||
|
||
const supportsRuntimeActivation =
|
||
typeof pluginLoaderInstance.hasRuntimeServices === "function"
|
||
&& typeof pluginLoaderInstance.loadSingle === "function"
|
||
&& typeof pluginLoaderInstance.unloadSingle === "function"
|
||
&& pluginLoaderInstance.hasRuntimeServices();
|
||
|
||
if (supportsRuntimeActivation) {
|
||
log.info(
|
||
{ pluginId, pluginKey: plugin.pluginKey },
|
||
"plugin lifecycle: reloading plugin (re-reading manifest, re-applying pending migrations, restarting worker)",
|
||
);
|
||
|
||
// Full deactivate+reactivate cycle (not just `handle.restart()`) so that:
|
||
// - the manifest is re-read from disk, picking up newly declared
|
||
// `migrations/*.sql` files and any other manifest changes,
|
||
// - `applyMigrations` runs idempotently against the up-to-date
|
||
// migrations directory — pending migrations get applied, already-
|
||
// applied ones are skipped via the `pluginMigrations` table,
|
||
// - the worker subprocess is replaced with one loading the freshly
|
||
// built bundle.
|
||
//
|
||
// Bouncing the worker process alone (`handle.restart()`) leaves plugin
|
||
// schema out of sync with worker code whenever a hot reload adds a new
|
||
// migration, which makes downstream queries fail against missing tables.
|
||
await deactivatePluginRuntime(pluginId, plugin.pluginKey);
|
||
await activateReadyPlugin(pluginId);
|
||
afterConnectorPluginActivated(pluginId, plugin.pluginKey);
|
||
} else {
|
||
// No runtime activation services wired in (e.g. state-only test harness)
|
||
// — fall back to a bare worker subprocess bounce.
|
||
log.info(
|
||
{ pluginId, pluginKey: plugin.pluginKey },
|
||
"plugin lifecycle: restarting worker (runtime services unavailable; skipping migration re-apply)",
|
||
);
|
||
await handle.restart();
|
||
emitDomain("plugin.worker_stopped", { pluginId, pluginKey: plugin.pluginKey });
|
||
emitDomain("plugin.worker_started", { pluginId, pluginKey: plugin.pluginKey });
|
||
afterConnectorPluginActivated(pluginId, plugin.pluginKey);
|
||
}
|
||
|
||
log.info(
|
||
{ pluginId, pluginKey: plugin.pluginKey },
|
||
"plugin lifecycle: plugin reloaded",
|
||
);
|
||
},
|
||
|
||
// -- getStatus --------------------------------------------------------
|
||
async getStatus(pluginId: string): Promise<PluginStatus | null> {
|
||
const plugin = await registry.getById(pluginId);
|
||
return plugin?.status ?? null;
|
||
},
|
||
|
||
// -- canTransition ----------------------------------------------------
|
||
async canTransition(pluginId: string, to: PluginStatus): Promise<boolean> {
|
||
const plugin = await registry.getById(pluginId);
|
||
if (!plugin) return false;
|
||
return isValidTransition(plugin.status, to);
|
||
},
|
||
|
||
// -- Event subscriptions ----------------------------------------------
|
||
on(event, listener) {
|
||
emitter.on(event, listener);
|
||
},
|
||
|
||
off(event, listener) {
|
||
emitter.off(event, listener);
|
||
},
|
||
|
||
once(event, listener) {
|
||
emitter.once(event, listener);
|
||
},
|
||
};
|
||
}
|