Files
PaperClipAI/server/src/services/plugin-registry.ts
T
DottaandClaude Opus 5.5 56654adecd perf(server): reuse sandbox provider schemas for five seconds on the remote-run path (#21758)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent runs can use remote sandboxes that a sandbox provider plugin
supplies (for example Daytona)
> - Each sandbox operation resolves the environment config, and that
step reads the provider config schema to find the secret-ref fields
> - On a hosted instance with 7 remote runs, this schema read was the
most frequent statement: 17.95 per second, about 2.5 per second for each
run
> - The server runs all agents on one Node main thread, so each
statement costs main-thread time; at 1,000 agents this read alone is
about 2,500 statements per second
> - This pull request reuses the retained schema for a short, bounded
time and drops it at once when the plugin registry changes a plugin
manifest
> - The benefit is that a live remote run reads the schema about once
per 5 seconds for each provider, not once per operation

## Linked Issues or Issue Description

**What existing behavior does this improve?**
Every plugin sandbox operation (exec, file sync in/out, runner ingress
attach, capability snapshot, workspace realize, lease resume, lease
release) calls `resolveEnvironmentDriverConfigForRuntime`. That call
runs the `with selected_provider as materialized ...` statement in
`resolvePluginSandboxProviderConfigSchema`, every time.

**Subsystem affected**
Server: plugin environment driver
(`server/src/services/plugin-environment-driver.ts`) and the plugin
registry manifest writes.

**Current behavior**
The schema read runs once per operation. A census test that drives the
real runtime calls of one remote run counts 19 schema reads for 19
operations (see the table below). The hosted measurement was 17.95
executions per second with 7 runs.

**Proposed behavior**
The read runs at most once per 5 seconds for each provider and database
client. A plugin install, upgrade or uninstall through the plugin
registry in the same process takes effect on the next operation.

**Reason and benefit**
The schema is instance-wide plugin manifest data. It does not change
during normal runs. Re-reading it per operation spends main-thread and
database time for no new information.

**Breaking changes**
None. No API, schema or saved-state change.

## What Changed

- `resolvePluginSandboxProviderConfigSchema` now returns the retained
schema without a database read when the entry was checked less than
`PROVIDER_SCHEMA_REUSE_MS` (5,000 ms) ago and the plugin manifest
revision did not change. After that, it uses the existing cheap revision
check, as before.
- A lookup sequence number stops an older, slower lookup from replacing
a newer retained entry.
- A missing provider is still never retained, so a provider that becomes
available is used on the next operation.
- New module `plugin-manifest-revision.ts` holds an in-process manifest
revision counter. It is a separate module so that the registry and the
driver do not import each other.
- `plugin-registry.ts`: `install` (both the new-row and the reinstall
paths), `update` and `uninstall` announce a manifest change after the
write completes. This is 5 changed lines plus 1 import line. Status-only
writes do not announce, because the schema read does not use plugin
status.
- `plugin-loader.ts`: `installPlugin` runs `registry.install` inside an
outer transaction. It now announces the manifest change again after that
transaction settles, so a schema read made before the commit cannot be
reused after it. This is 1 changed line plus 1 import line.
- Statement census per operation, before and after. One remote run, one
secret-ref field in the provider config, embedded test database, fake
in-process plugin worker:

| Operation | Count | Schema reads before | Schema reads after | All
statements before | All statements after |
| --- | --- | --- | --- | --- | --- |
| Lease acquire | 1 | 1 | 1 | 26 | 26 |
| Runner attach (ingress endpoint) | 1 | 2 | 0 | 11 | 9 |
| Capability snapshot | 1 | 1 | 0 | 6 | 5 |
| Exec | 10 | 10 | 0 | 50 | 40 |
| File write (sync in) | 1 | 1 | 0 | 5 | 4 |
| File read (sync out) | 1 | 1 | 0 | 5 | 4 |
| Workspace realize | 1 | 1 | 0 | 5 | 4 |
| Lease resume (reattach) | 1 | 1 | 0 | 6 | 5 |
| Lease release | 1 | 1 | 0 | 18 | 17 |
| **Total** | 18 | **19** | **1** | 132 | 114 |

- Other reads that stay on each operation (reported, not changed in this
pull request): each exec resolves each secret-ref field with 3 reads
(secret, binding, version) and 1 `secret_access_events` insert. Runner
attach and the capability snapshot also read the plugin row by id. These
are part of the secret authorization and audit path, so they keep their
own checks.

## Verification

- `cd server && npx vitest run
src/__tests__/plugin-sandbox-provider-schema-census.test.ts` (new). On
master the census test fails with 19 schema reads. With this change it
passes with 1.
- The new tests also prove these points:
- A registry upgrade that changes the secret-ref fields takes effect on
the next call in the same process.
- A direct manifest change (as from another server process) takes effect
after the 5-second bound, and no read occurs inside the bound.
  - A registry uninstall takes effect at once.
- A plugin reinstall through `pluginLoader.installPlugin` takes effect
after its install transaction commits, even when a schema read ran
inside the uncommitted transaction. This test failed before the
post-commit announcement and passes with it.
- A missing provider is not retained. The next call after install reads
and uses it.
- Two companies share the provider schema but each gets only its own
secret values. A cross-company secret id still fails with "Secret must
belong to same company".
  - A new database client (a process restart) starts cold, as before.
- `cd server && npx vitest run src/__tests__/plugin-database.test.ts`:
passed (plugin install transaction paths).
- `cd server && npx vitest run
src/__tests__/sandbox-provider-schema-reads.test.ts`: the existing tests
now step the clock past the bound where they change the database
directly.
- `cd server && npx vitest run
src/__tests__/environment-runtime.test.ts`: 210 passed.
- `cd server && npx vitest run
src/__tests__/bundled-plugin-registration.test.ts
src/__tests__/external-object-refresh-work.test.ts
src/__tests__/plugin-lifecycle-restart.test.ts
src/__tests__/plugin-loader-error-retry.test.ts
src/__tests__/plugin-environment-driver-sandbox-capabilities.test.ts
src/__tests__/plugin-sandbox-provider-config-validation.test.ts
src/__tests__/environment-probe.test.ts`: 82 passed, 1 skipped.
- `cd server && pnpm typecheck`: passed.

## Risks

- Staleness bound: a plugin manifest change made by a different server
process (for example during an overlapping deploy) can take up to 5
seconds to apply here. During that time, a secret-ref field that an
upgrade adds is not resolved yet, and a field that an upgrade removes is
still resolved. The bound is short and it applies only to which config
paths are secret refs.
- The retained value is instance-wide plugin manifest data, keyed by
database client and driver key. It contains no company data and no
secret values. It does not take part in an authorization decision.
Secret resolution, the environment binding check and the access audit
still run on each operation for the calling company.
- Memory stays bounded by the existing limits: 32 entries per database
client, 64 KiB per schema.
- `plugin-registry.ts` has a small edit in `install`, `update` and
`uninstall`. Other open work in that file can cause a simple merge
conflict.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edit, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Refs #17581

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 08:09:04 -06:00

900 lines
32 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import { notifyExternalObjectRefresh } from "./external-object-refresh-notifications.js";
import { isDeepStrictEqual } from "node:util";
import { writePluginJobChange } from "./plugin-job-work-notifications.js";
import { asc, eq, isNull, ne, sql, and } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import {
plugins,
pluginConfig,
pluginCompanySettings,
pluginEntities,
pluginJobs,
pluginJobRuns,
pluginWebhookDeliveries,
} from "@paperclipai/db";
import type {
PaperclipPluginManifestV1,
PluginStatus,
InstallPlugin,
UpdatePluginStatus,
UpsertPluginConfig,
PatchPluginConfig,
PluginCompanySettings,
PluginEntityRecord,
PluginEntityQuery,
PluginJobRecord,
PluginJobRunRecord,
PluginWebhookDeliveryRecord,
PluginJobStatus,
PluginJobRunStatus,
PluginJobRunTrigger,
PluginWebhookDeliveryStatus,
} from "@paperclipai/shared";
import { conflict, notFound } from "../errors.js";
import { DELIVERY_QUEUES, notifyDeliveryWork } from "./delivery-work-notifications.js";
import { TERMINAL_WORKSPACE_PLUGIN_HINT } from "./terminal-workspace-work.js";
import { announcePluginManifestChange } from "./plugin-manifest-revision.js";
function sameBundledPluginRegistration(existing: typeof plugins.$inferSelect, packageName: string, manifest: PaperclipPluginManifestV1): boolean {
return existing.pluginKey === manifest.id && existing.packageName === packageName && existing.version === manifest.version && isDeepStrictEqual(existing.manifestJson, manifest);
}
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
/**
* Detect if a Postgres error is a unique-constraint violation on the
* `plugins_plugin_key_idx` unique index.
*/
function isPluginKeyConflict(error: unknown): boolean {
if (typeof error !== "object" || error === null) return false;
const err = error as { code?: string; constraint?: string; constraint_name?: string };
const constraint = err.constraint ?? err.constraint_name;
return err.code === "23505" && constraint === "plugins_plugin_key_idx";
}
// ---------------------------------------------------------------------------
// Service
// ---------------------------------------------------------------------------
/**
* PluginRegistry – CRUD operations for the `plugins` and `plugin_config`
* tables. Follows the same factory-function pattern used by the rest of
* the Paperclip service layer.
*
* This is the lowest-level persistence layer for plugins. Higher-level
* concerns such as lifecycle state-machine enforcement and capability
* gating are handled by {@link pluginLifecycleManager} and
* {@link pluginCapabilityValidator} respectively.
*
* @see PLUGIN_SPEC.md §21.3 — Required Tables
*/
export function pluginRegistryService(db: Db) {
// -----------------------------------------------------------------------
// Internal helpers
// -----------------------------------------------------------------------
async function getById(id: string, database: Db = db) {
return database
.select()
.from(plugins)
.where(eq(plugins.id, id))
.then((rows) => rows[0] ?? null);
}
async function getByKey(pluginKey: string) {
return db
.select()
.from(plugins)
.where(eq(plugins.pluginKey, pluginKey))
.then((rows) => rows[0] ?? null);
}
async function writeWorkspaceProviderChange<T>(reconsiderWorkspaces: boolean, write: (database: Db) => Promise<T>): Promise<T> {
return db.transaction(async tx => {
await notifyExternalObjectRefresh(tx);
if (reconsiderWorkspaces) await notifyDeliveryWork(tx, DELIVERY_QUEUES.terminalWorkspaces, TERMINAL_WORKSPACE_PLUGIN_HINT);
return write(tx as unknown as Db);
});
}
async function nextInstallOrder(): Promise<number> {
const result = await db
.select({ maxOrder: sql<number>`coalesce(max(${plugins.installOrder}), 0)` })
.from(plugins);
return (result[0]?.maxOrder ?? 0) + 1;
}
// -----------------------------------------------------------------------
// Public API
// -----------------------------------------------------------------------
return {
// ----- Read -----------------------------------------------------------
/** List all registered plugins ordered by install order. */
list: (filter?: { sandboxProviderDriverKey: string }) => {
const query = db.select().from(plugins);
const driverKey = filter?.sandboxProviderDriverKey;
return (driverKey === undefined ? query : query.where(sql`${plugins.manifestJson} @> ${JSON.stringify({
environmentDrivers: [{ driverKey, kind: "sandbox_provider" }],
})}::jsonb`)).orderBy(asc(plugins.installOrder));
},
/**
* List installed plugins (excludes soft-deleted/uninstalled).
* Use for Plugin Manager and default API list so uninstalled plugins do not appear.
*/
listInstalled: () =>
db
.select()
.from(plugins)
.where(ne(plugins.status, "uninstalled"))
.orderBy(asc(plugins.installOrder)),
/** List plugins filtered by status. */
listByStatus: (status: PluginStatus) =>
db
.select()
.from(plugins)
.where(eq(plugins.status, status))
.orderBy(asc(plugins.installOrder)),
/** Get a single plugin by primary key. */
getById,
/** Get a single plugin by its unique `pluginKey`. */
getByKey,
// ----- Install / Register --------------------------------------------
/**
* Register (install) a new plugin.
*
* The caller is expected to have already resolved and validated the
* manifest from the package. This method persists the plugin row and
* assigns the next install order.
*/
install: async (input: InstallPlugin, manifest: PaperclipPluginManifestV1, options: { idempotent?: boolean } = {}) => {
const existing = await getByKey(manifest.id);
if (existing) {
if (existing.status !== "uninstalled") {
if (options.idempotent && sameBundledPluginRegistration(existing, input.packageName, manifest)) return existing;
throw conflict(`Plugin already installed: ${manifest.id}`);
}
// Reinstall after soft-delete: reactivate the existing row so plugin-scoped
// data and references remain stable across uninstall/reinstall cycles.
return db
.update(plugins)
.set({
packageName: input.packageName,
packagePath: options.idempotent ? existing.packagePath : input.packagePath ?? null,
version: manifest.version,
apiVersion: manifest.apiVersion,
categories: manifest.categories,
manifestJson: manifest,
status: "installed" as PluginStatus,
lastError: null,
updatedAt: new Date(),
})
.where(eq(plugins.id, existing.id))
.returning()
.then((rows) => rows[0] ?? null)
.finally(announcePluginManifestChange);
}
const installOrder = await nextInstallOrder();
try {
const insertion = db
.insert(plugins)
.values({
pluginKey: manifest.id,
packageName: input.packageName,
version: manifest.version,
apiVersion: manifest.apiVersion,
categories: manifest.categories,
manifestJson: manifest,
status: "installed" as PluginStatus,
installOrder,
packagePath: input.packagePath ?? null,
});
const rows = await (options.idempotent
? insertion.onConflictDoNothing({ target: plugins.pluginKey })
: insertion).returning();
announcePluginManifestChange();
if (rows[0]) return rows[0];
const concurrent = await getByKey(manifest.id);
if (concurrent && concurrent.status !== "uninstalled" && sameBundledPluginRegistration(concurrent, input.packageName, manifest)) return concurrent;
throw conflict(`Plugin registration differs from the current image: ${manifest.id}`);
} catch (error) {
if (isPluginKeyConflict(error)) {
throw conflict(`Plugin already installed: ${manifest.id}`);
}
throw error;
}
},
// ----- Update ---------------------------------------------------------
/**
* Update a plugin's manifest and version (e.g. on upgrade).
* The plugin must already exist.
*/
update: async (
id: string,
data: {
packageName?: string;
packagePath?: string;
status?: "upgrade_pending";
version?: string;
manifest?: PaperclipPluginManifestV1;
},
) => {
const plugin = await getById(id);
if (!plugin) throw notFound("Plugin not found");
const setClause: Partial<typeof plugins.$inferInsert> & { updatedAt: Date } = {
updatedAt: new Date(),
};
if (data.packageName !== undefined) setClause.packageName = data.packageName;
if (data.packagePath !== undefined) setClause.packagePath = data.packagePath;
if (data.status !== undefined) {
setClause.status = data.status;
setClause.lastError = null;
}
if (data.version !== undefined) setClause.version = data.version;
if (data.manifest !== undefined) {
setClause.manifestJson = data.manifest;
setClause.apiVersion = data.manifest.apiVersion;
setClause.categories = data.manifest.categories;
}
const workspaceProviderChanged = data.manifest !== undefined
&& !isDeepStrictEqual(plugin.manifestJson.workspaceProvider, data.manifest.workspaceProvider);
return writeWorkspaceProviderChange(workspaceProviderChanged, database => database
.update(plugins)
.set(setClause)
.where(eq(plugins.id, id))
.returning()
.then((rows) => rows[0] ?? null)).finally(announcePluginManifestChange);
},
// ----- Status ---------------------------------------------------------
/** Update a plugin's lifecycle status and optional error message. */
updateStatus: async (id: string, input: UpdatePluginStatus) => {
const plugin = await getById(id);
if (!plugin) throw notFound("Plugin not found");
const workspaceProviderActivated = Boolean(plugin.manifestJson.workspaceProvider)
&& plugin.status !== "ready" && input.status === "ready";
return writeWorkspaceProviderChange(workspaceProviderActivated, database => database
.update(plugins)
.set({
status: input.status,
lastError: input.lastError ?? null,
updatedAt: new Date(),
})
.where(eq(plugins.id, id))
.returning()
.then((rows) => rows[0] ?? null));
},
/**
* Update status and error only while the row still has the expected status
* (and, when given, an error with the expected prefix). Returns null when a
* concurrent change such as a disable or an uninstall won.
*/
updateStatusIf: async (id: string, expected: { status: string; lastErrorPrefix?: string }, input: UpdatePluginStatus) => {
const plugin = await getById(id);
if (!plugin || plugin.status !== expected.status) return null;
const workspaceProviderActivated = Boolean(plugin.manifestJson.workspaceProvider)
&& plugin.status !== "ready" && input.status === "ready";
return writeWorkspaceProviderChange(workspaceProviderActivated, database => database
.update(plugins)
.set({ status: input.status, lastError: input.lastError ?? null, updatedAt: new Date() })
.where(and(eq(plugins.id, id), eq(plugins.status, expected.status as typeof plugin.status),
expected.lastErrorPrefix === undefined ? undefined
: sql`starts_with(${plugins.lastError}, ${expected.lastErrorPrefix})`))
.returning()
.then((rows) => rows[0] ?? null));
},
// ----- Uninstall / Remove --------------------------------------------
/**
* Uninstall a plugin.
*
* When `removeData` is true the plugin row (and cascaded config) is
* hard-deleted. Otherwise the status is set to `"uninstalled"` for
* a soft-delete that preserves the record.
*/
uninstall: async (id: string, removeData = false) => {
const plugin = await getById(id);
if (!plugin) throw notFound("Plugin not found");
return writePluginJobChange(db, async db => {
await notifyExternalObjectRefresh(db);
if (removeData) {
// Hard delete – plugin_config cascades via FK onDelete
return db
.delete(plugins)
.where(eq(plugins.id, id))
.returning()
.then((rows) => rows[0] ?? null);
}
// Soft delete – mark as uninstalled
return db
.update(plugins)
.set({
status: "uninstalled" as PluginStatus,
updatedAt: new Date(),
})
.where(eq(plugins.id, id))
.returning()
.then((rows) => rows[0] ?? null);
}).finally(announcePluginManifestChange);
},
// ----- Config ---------------------------------------------------------
/** Retrieve a plugin's company-scoped configuration. */
getConfig: (pluginId: string, companyId: string) =>
db
.select()
.from(pluginConfig)
.where(and(eq(pluginConfig.pluginId, pluginId), eq(pluginConfig.companyId, companyId)))
.then((rows) => rows[0] ?? null),
/**
* List every company-scoped configuration row for a plugin.
*
* Plugin config is company-scoped, but a worker is spawned once per plugin
* (not per company). Callers such as the plugin loader use this to replay
* each configured company's config to a freshly-started worker, so a
* proactive plugin that never runs inside a company-scoped invocation (and
* therefore cannot resolve `ctx.config.get(companyId)`) still receives its
* configuration at startup.
*
* Ordered deterministically by companyId: the startup replay delivers these
* rows to a single worker via `configChanged`, and a single-tenant worker
* binds to the first company it sees. Without a stable order the worker
* would bind to a nondeterministic (DB-dependent) company across restarts.
*/
listConfigs: (pluginId: string) =>
db
.select()
.from(pluginConfig)
.where(eq(pluginConfig.pluginId, pluginId))
.orderBy(asc(pluginConfig.companyId)),
/**
* Create or fully replace a plugin's company-scoped configuration.
* If a config row already exists for the plugin/company pair it is replaced;
* otherwise a new row is inserted.
*/
upsertConfig: async (pluginId: string, companyId: string, input: UpsertPluginConfig) => {
return db.transaction(async tx => {
const db = tx as unknown as Db;
await notifyDeliveryWork(db, DELIVERY_QUEUES.pluginLifecycle);
await notifyExternalObjectRefresh(db, companyId);
const plugin = await getById(pluginId, db);
if (!plugin) throw notFound("Plugin not found");
const existing = await db
.select()
.from(pluginConfig)
.where(and(eq(pluginConfig.pluginId, pluginId), eq(pluginConfig.companyId, companyId)))
.then((rows) => rows[0] ?? null);
if (existing) {
return db
.update(pluginConfig)
.set({
configJson: input.configJson,
lastError: null,
updatedAt: new Date(),
})
.where(and(eq(pluginConfig.pluginId, pluginId), eq(pluginConfig.companyId, companyId)))
.returning()
.then((rows) => rows[0]);
}
return db
.insert(pluginConfig)
.values({
pluginId,
companyId,
configJson: input.configJson,
})
.returning()
.then((rows) => rows[0]);
});
},
/**
* Partially update a plugin's company-scoped configuration via shallow merge.
* If no config row exists yet one is created with the supplied values.
*/
patchConfig: async (pluginId: string, companyId: string, input: PatchPluginConfig) => {
return db.transaction(async tx => {
const db = tx as unknown as Db;
await notifyDeliveryWork(db, DELIVERY_QUEUES.pluginLifecycle);
await notifyExternalObjectRefresh(db, companyId);
const plugin = await getById(pluginId, db);
if (!plugin) throw notFound("Plugin not found");
const existing = await db
.select()
.from(pluginConfig)
.where(and(eq(pluginConfig.pluginId, pluginId), eq(pluginConfig.companyId, companyId)))
.then((rows) => rows[0] ?? null);
if (existing) {
const merged = { ...existing.configJson, ...input.configJson };
return db
.update(pluginConfig)
.set({
configJson: merged,
lastError: null,
updatedAt: new Date(),
})
.where(and(eq(pluginConfig.pluginId, pluginId), eq(pluginConfig.companyId, companyId)))
.returning()
.then((rows) => rows[0]);
}
return db
.insert(pluginConfig)
.values({
pluginId,
companyId,
configJson: input.configJson,
})
.returning()
.then((rows) => rows[0]);
});
},
/**
* Record an error against a plugin's config (e.g. validation failure
* against the plugin's instanceConfigSchema).
*/
setConfigError: async (pluginId: string, companyId: string, lastError: string | null) => {
const rows = await db
.update(pluginConfig)
.set({ lastError, updatedAt: new Date() })
.where(and(eq(pluginConfig.pluginId, pluginId), eq(pluginConfig.companyId, companyId)))
.returning();
if (rows.length === 0) throw notFound("Plugin config not found");
return rows[0];
},
/** Delete a plugin's config row. */
deleteConfig: async (pluginId: string, companyId: string) => {
const rows = await db
.delete(pluginConfig)
.where(and(eq(pluginConfig.pluginId, pluginId), eq(pluginConfig.companyId, companyId)))
.returning();
return rows[0] ?? null;
},
// ----- Company settings ----------------------------------------------
/** Retrieve company-scoped plugin settings. */
getCompanySettings: (pluginId: string, companyId: string): Promise<PluginCompanySettings | null> =>
db
.select()
.from(pluginCompanySettings)
.where(and(
eq(pluginCompanySettings.pluginId, pluginId),
eq(pluginCompanySettings.companyId, companyId),
))
.then((rows) => rows[0] ?? null) as Promise<PluginCompanySettings | null>,
/** Create or replace company-scoped plugin settings. */
upsertCompanySettings: async (
pluginId: string,
companyId: string,
input: { enabled?: boolean; settingsJson: Record<string, unknown>; lastError?: string | null },
): Promise<PluginCompanySettings> => {
return db.transaction(async tx => {
const db = tx as unknown as Db;
await notifyDeliveryWork(db, DELIVERY_QUEUES.pluginLifecycle);
await notifyExternalObjectRefresh(db, companyId);
const plugin = await getById(pluginId, db);
if (!plugin) throw notFound("Plugin not found");
const existing = await db
.select()
.from(pluginCompanySettings)
.where(and(
eq(pluginCompanySettings.pluginId, pluginId),
eq(pluginCompanySettings.companyId, companyId),
))
.then((rows) => rows[0] ?? null);
if (plugin.manifestJson.workspaceProvider && existing?.enabled === false && input.enabled === true) {
await notifyDeliveryWork(db, DELIVERY_QUEUES.terminalWorkspaces, TERMINAL_WORKSPACE_PLUGIN_HINT);
}
if (existing) {
return db
.update(pluginCompanySettings)
.set({
enabled: input.enabled ?? existing.enabled,
settingsJson: input.settingsJson,
lastError: input.lastError ?? null,
updatedAt: new Date(),
})
.where(eq(pluginCompanySettings.id, existing.id))
.returning()
.then((rows) => rows[0]) as Promise<PluginCompanySettings>;
}
return db
.insert(pluginCompanySettings)
.values({
pluginId,
companyId,
enabled: input.enabled ?? true,
settingsJson: input.settingsJson,
lastError: input.lastError ?? null,
})
.returning()
.then((rows) => rows[0]) as Promise<PluginCompanySettings>;
});
},
// ----- Entities -------------------------------------------------------
/**
* List persistent entity mappings owned by a specific plugin, with filtering and pagination.
*
* @param pluginId - The UUID of the plugin.
* @param query - Optional filters (type, externalId) and pagination (limit, offset).
* @returns A list of matching `PluginEntityRecord` objects.
*/
listEntities: (pluginId: string, query?: PluginEntityQuery) => {
const conditions = [eq(pluginEntities.pluginId, pluginId)];
if (query?.entityType) conditions.push(eq(pluginEntities.entityType, query.entityType));
if (query?.externalId) conditions.push(eq(pluginEntities.externalId, query.externalId));
if (query?.scopeKind) conditions.push(eq(pluginEntities.scopeKind, query.scopeKind));
if (query?.scopeId) conditions.push(eq(pluginEntities.scopeId, query.scopeId));
return db
.select()
.from(pluginEntities)
.where(and(...conditions))
.orderBy(asc(pluginEntities.createdAt))
.limit(query?.limit ?? 100)
.offset(query?.offset ?? 0);
},
/**
* Look up a plugin-owned entity mapping by its external identifier.
*
* Scope matches `plugin_entities_external_idx` (NULLS NOT DISTINCT):
* pass the owning `companyId` (or `null` for instance-scope) to retrieve
* the row that belongs to that tenant. Two companies can share the same
* `(pluginId, entityType, externalId)` tuple — omitting `companyId` would
* return the first matched row regardless of tenant, which is unsafe.
*
* @param pluginId - The UUID of the plugin.
* @param entityType - The type of entity (e.g., 'project', 'issue').
* @param externalId - The identifier in the external system.
* @param companyId - Tenant scope; `null` for instance-scope entities.
* @returns The matching `PluginEntityRecord` or null.
*/
getEntityByExternalId: (
pluginId: string,
entityType: string,
externalId: string,
companyId: string | null,
) => {
const companyIdPredicate =
companyId == null
? isNull(pluginEntities.companyId)
: eq(pluginEntities.companyId, companyId);
return db
.select()
.from(pluginEntities)
.where(
and(
companyIdPredicate,
eq(pluginEntities.pluginId, pluginId),
eq(pluginEntities.entityType, entityType),
eq(pluginEntities.externalId, externalId),
),
)
.then((rows) => rows[0] ?? null);
},
/**
* Create or update a persistent mapping between a Paperclip object and an
* external entity.
*
* @param pluginId - The UUID of the plugin.
* @param input - The entity data to persist.
* @returns The newly created or updated `PluginEntityRecord`.
*/
upsertEntity: async (
pluginId: string,
input: Omit<typeof pluginEntities.$inferInsert, "id" | "pluginId" | "createdAt" | "updatedAt">,
) => {
// Drizzle doesn't support pg-specific onConflictDoUpdate easily in the insert() call
// with complex where clauses, so we do it manually.
// Match the per-tenant uniqueness of `plugin_entities_external_idx`
// (companyId, pluginId, entityType, externalId) with NULLS NOT DISTINCT
// semantics: two companies (and instance-scope NULLs across each other)
// may share the same (pluginId, entityType, externalId) tuple, so the
// lookup MUST scope by companyId — `isNull` for instance-scope, `eq`
// otherwise — to avoid returning and overwriting another tenant's row.
const companyIdPredicate =
input.companyId == null
? isNull(pluginEntities.companyId)
: eq(pluginEntities.companyId, input.companyId);
const existing = await db
.select()
.from(pluginEntities)
.where(
and(
companyIdPredicate,
eq(pluginEntities.pluginId, pluginId),
eq(pluginEntities.entityType, input.entityType),
eq(pluginEntities.externalId, input.externalId ?? ""),
),
)
.then((rows) => rows[0] ?? null);
if (existing) {
return db
.update(pluginEntities)
.set({
...input,
updatedAt: new Date(),
})
.where(eq(pluginEntities.id, existing.id))
.returning()
.then((rows) => rows[0]);
}
return db
.insert(pluginEntities)
.values({
...input,
pluginId,
} as any)
.returning()
.then((rows) => rows[0]);
},
/**
* Delete a specific plugin-owned entity mapping by its internal UUID.
*
* @param id - The UUID of the entity record.
* @returns The deleted record, or null if not found.
*/
deleteEntity: async (id: string) => {
const rows = await db
.delete(pluginEntities)
.where(eq(pluginEntities.id, id))
.returning();
return rows[0] ?? null;
},
// ----- Jobs -----------------------------------------------------------
/**
* List all scheduled jobs registered for a specific plugin.
*
* @param pluginId - The UUID of the plugin.
* @returns A list of `PluginJobRecord` objects.
*/
listJobs: (pluginId: string) =>
db
.select()
.from(pluginJobs)
.where(eq(pluginJobs.pluginId, pluginId))
.orderBy(asc(pluginJobs.jobKey)),
/**
* Look up a plugin job by its unique job key.
*
* @param pluginId - The UUID of the plugin.
* @param jobKey - The key defined in the plugin manifest.
* @returns The matching `PluginJobRecord` or null.
*/
getJobByKey: (pluginId: string, jobKey: string) =>
db
.select()
.from(pluginJobs)
.where(and(eq(pluginJobs.pluginId, pluginId), eq(pluginJobs.jobKey, jobKey)))
.then((rows) => rows[0] ?? null),
/**
* Register or update a scheduled job for a plugin.
*
* @param pluginId - The UUID of the plugin.
* @param jobKey - The unique key for the job.
* @param input - The schedule (cron) and optional status.
* @returns The updated or created `PluginJobRecord`.
*/
upsertJob: async (
pluginId: string,
jobKey: string,
input: { schedule: string; status?: PluginJobStatus },
) => {
return writePluginJobChange(db, async db => {
const existing = await db
.select()
.from(pluginJobs)
.where(and(eq(pluginJobs.pluginId, pluginId), eq(pluginJobs.jobKey, jobKey)))
.for("update")
.then((rows) => rows[0] ?? null);
if (existing) {
return db
.update(pluginJobs)
.set({
schedule: input.schedule,
nextRunAt: existing.schedule === input.schedule ? existing.nextRunAt : null,
status: input.status ?? existing.status,
updatedAt: new Date(),
})
.where(eq(pluginJobs.id, existing.id))
.returning()
.then((rows) => rows[0]);
}
return db
.insert(pluginJobs)
.values({
pluginId,
jobKey,
schedule: input.schedule,
status: input.status ?? "active",
})
.returning()
.then((rows) => rows[0]);
});
},
/**
* Record the start of a specific job execution.
*
* Pass the owning `companyId` so `plugin_job_runs.company_id` is populated
* and the row participates in the `ON DELETE CASCADE` from `companies`.
* `null` is the explicit instance-scope marker (cron jobs without a tenant);
* those rows survive company deletes but are still attributable.
*
* @param pluginId - The UUID of the plugin.
* @param jobId - The UUID of the parent job record.
* @param trigger - What triggered this run (e.g., 'schedule', 'manual').
* @param companyId - Tenant scope; `null` for instance-scope runs.
* @returns The newly created `PluginJobRunRecord` in 'pending' status.
*/
createJobRun: async (
pluginId: string,
jobId: string,
trigger: PluginJobRunTrigger,
companyId: string | null,
) => {
return writePluginJobChange(db, async db => {
return db
.insert(pluginJobRuns)
.values({
pluginId,
jobId,
companyId,
trigger,
status: "pending",
})
.returning()
.then((rows) => rows[0]);
});
},
/**
* Update the status, duration, and logs of a job execution record.
*
* @param runId - The UUID of the job run.
* @param input - The update fields (status, error, duration, etc.).
* @returns The updated `PluginJobRunRecord`.
*/
updateJobRun: async (
runId: string,
input: {
status: PluginJobRunStatus;
durationMs?: number;
error?: string;
logs?: string[];
startedAt?: Date;
finishedAt?: Date;
},
) => {
return writePluginJobChange(db, async db => {
return db
.update(pluginJobRuns)
.set(input)
.where(eq(pluginJobRuns.id, runId))
.returning()
.then((rows) => rows[0] ?? null);
});
},
// ----- Webhooks -------------------------------------------------------
/**
* Create a record for an incoming webhook delivery.
*
* Pass the owning `companyId` so `plugin_webhook_deliveries.company_id` is
* populated and the row participates in the `ON DELETE CASCADE` from
* `companies`. `null` is the explicit instance-scope marker (public
* webhooks without a tenant); those rows survive company deletes but are
* still attributable.
*
* @param pluginId - The UUID of the receiving plugin.
* @param webhookKey - The endpoint key defined in the manifest.
* @param companyId - Tenant scope; `null` for instance-scope deliveries.
* @param input - The payload, headers, and optional external ID.
* @returns The newly created `PluginWebhookDeliveryRecord` in 'pending' status.
*/
createWebhookDelivery: async (
pluginId: string,
webhookKey: string,
companyId: string | null,
input: {
externalId?: string;
payload: Record<string, unknown>;
headers?: Record<string, string>;
},
) => {
return db
.insert(pluginWebhookDeliveries)
.values({
pluginId,
webhookKey,
companyId,
externalId: input.externalId,
payload: input.payload,
headers: input.headers ?? {},
status: "pending",
})
.returning()
.then((rows) => rows[0]);
},
/**
* Update the status and processing metrics of a webhook delivery.
*
* @param deliveryId - The UUID of the delivery record.
* @param input - The update fields (status, error, duration, etc.).
* @returns The updated `PluginWebhookDeliveryRecord`.
*/
updateWebhookDelivery: async (
deliveryId: string,
input: {
status: PluginWebhookDeliveryStatus;
durationMs?: number;
error?: string;
startedAt?: Date;
finishedAt?: Date;
},
) => {
return db
.update(pluginWebhookDeliveries)
.set(input)
.where(eq(pluginWebhookDeliveries.id, deliveryId))
.returning()
.then((rows) => rows[0] ?? null);
},
};
}