Files
PaperClipAI/server/src/services/connector-telemetry.ts
T
DottaandClaude Opus 5.5 01451fae5b Let plugins add connector types to the Connectors catalog (#22004)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents use external services through connectors: a catalog of
connector types, connections (accounts) per type, and per-agent
installs, policies and audit in the tool gateway
> - Today only core can add a connector type. Every type is a JSON
definition compiled into `@paperclipai/shared`
> - Plugins can add agent tools, but those tools skip connections,
grants, per-agent installs and the Connectors page
> - This pull request lets a plugin add connector types with the same
definition schema, and makes them behave exactly like built-in
connectors
> - The benefit is that a connector can ship as a plugin, and an
operator controls it with the same Connectors page and the same access
rules

## Linked Issues or Issue Description

**Subsystem affected**
Plugins, Connectors (tool access), tool gateway

**Problem or motivation**
A plugin cannot add a connector type to the Connectors catalog. Plugin
tools (`tools` in the manifest) do not use connections, credentials per
account, agent installs, policies or the Connectors page.

**Proposed solution**
Add a manifest field `apps` (gated by a new `connectors.provide`
capability). Each entry uses the built-in app-definition schema. The
host namespaces the slug by plugin key and runs setup, OAuth, discovery,
grants, policies and audit through the existing code. This is the first
of two pull requests. The second adds a `plugin` transport so a plugin
worker can serve the tools itself.

**Activation rules (product requirements)**
1. Installing or enabling a plugin grants no tools. It only adds
connector types to the catalog.
2. Connecting an account grants no tools.
3. An agent gets the tools only from an agent install or a company
install of an active connection. Discovery and invocation both enforce
this.
4. A disabled or uninstalled plugin (or an upgrade that removes the app)
hides its connector types and connections from the Connectors APIs and
they reach no agent. Saved setup survives disable and comes back on
re-enable. Uninstall keeps rows archived for audit and revokes
credentials.


**Alternatives considered**
Keep plugin tools (`tools`) as the only plugin path. Rejected: they
bypass connections, per-account credentials, per-agent installs and the
Connectors page.

## What Changed

- `packages/shared`: new capability `connectors.provide`; manifest
`apps` validated with `appDefinitionSchema`. The validator rejects apps
without the capability, duplicate slugs, host-derived fields (`plugin`,
`aiConnectionRouter`, `experimentalFeatureKey`, `availability`), and
methods that need provider code in the host (REST adapters, chat
channels, AI runtimes).
- `packages/shared/src/plugin-apps.ts`: reversible namespaced slugs
(`plugin-<hex plugin key>-<slug>`), the catalog entry builder, and
`reviewedToolRisk`.
- `packages/shared/src/app-definitions.ts`: catalog lookups
(`getConnectableAppDefinition`, `getAppStoreDefinition`, ...) also
resolve the current plugin apps. The set is replaced on each read of
plugin state, so a built-in app is never shadowed.
- App definitions can declare `reviewedTools` (reviewed
read/write/destructive names and the risk for unlisted tools).
Circleback's reviewed risk moves from
`server/src/services/connectors/circleback/risk.ts` into
`circleback.json`, with the same classification. A copy of the
definition now classifies tools the same way.
- `validators/json-schema.ts`: `jsonSchemaSchema` moves to its own
module to break an import cycle between the plugin and app-definition
validators (still re-exported from `validators/plugin.ts`).
- `server/src/services/plugin-app-catalog.ts`: reads ready plugins from
the `plugins` table and answers "is this connection's connector
available". It does not trust in-memory lifecycle events, because
several lifecycle managers and processes can share one database.
- Gateway: connected-tool discovery and `resolveConnectionGrant` drop
connections from an unavailable plugin connector. Built-in connections
do not read the plugins table.
- Connectors APIs: the gallery lists ready plugin apps (with the same
Cloud connector profile filter as core); `listConnections` and
`listApplications` hide unavailable plugin connections; setup entry
points refresh plugin apps first.
- Plugin uninstall archives the plugin's connections through the
standard connection removal (grants revoked, owned credentials deleted,
rows kept).
- UI: the gallery response registers plugin apps for the setup screens;
`/apps/connect?source=plugin-…` loads the gallery first and shows an
unavailable state when the plugin is disabled.
- Docs: `doc/plugins/PLUGIN_SPEC.md` section 11A and the capability
list.

Design choices where the spec left room (picked to match "a plugin
connector behaves like a built-in connector"):
- Only `ready` plugins provide connectors. `error` and `upgrade_pending`
hide them, the same as the AI connection router plugins.
- `rest_api` is rejected for plugin apps in this PR. Core REST
connectors need reviewed adapter code in the host, so a definition alone
cannot work. The `plugin` transport in the next PR covers that case.
- Disabled plugin connections are hidden from list endpoints. Reading
one connection by id still works, so audit links keep resolving.

## Verification

- `packages/shared`: `npx vitest run src/plugin-apps.test.ts
src/validators/ src/app-definitions.test.ts` (416 passed). New tests:
manifest validation, slug namespacing, no shadowing of built-in apps,
reviewed risk.
- `server`: `npx vitest run
src/__tests__/plugin-connectors-circleback-parity.test.ts` (new). A
test-only plugin
(`server/src/__tests__/fixtures/plugin-connectors/circleback-parity/`)
ships an unmodified copy of `circleback.json`. The same journey runs
against built-in Circleback and the plugin copy with a recorded DCR
OAuth + MCP fixture, and must give identical catalog risk/status,
suggested defaults and per-agent tool sets. It covers each rule:
- Rule 1: the gallery adds only the namespaced type; no connections
exist; no agent tool changes.
  - Rule 2: two signed-in accounts, no agent sees a tool.
- Rule 3: an install for agent A gives only A the tools of that
connection; agent B is denied on invocation; each account calls the
provider with its own credential.
- Rule 4: disable hides the type, both connections and applications,
removes tools from discovery and invocation (404) and blocks new setup,
while built-in Circleback is unchanged; installs and grants are
byte-identical; re-enable restores tools with the same names; an upgrade
that removes the app hides it; uninstall archives both connections,
revokes grants and deletes owned secrets.
- Red/green: removing the gateway gate or the declared review makes the
test fail.
- An env-gated live check (`PAPERCLIP_CIRCLEBACK_LIVE=1`) starts the
real Circleback sign-in for both connectors and compares the requests.
It never runs in CI. I did not run it for this PR.
- Regression: `ai-connection-router`, `connector-tool-policy`,
`mcp-apps`, `plugin-lifecycle-restart`, `tool-access-service`,
`braintrust-connection`, `remote-mcp-connectors`,
`connection-intents-service`, `tool-gateway` (796 passed).
- `ui`: `app-connect-policy.test.ts`, `AppsConnect.test.tsx` (new plugin
cases) pass.
- `tsc --noEmit` for shared, server, ui (only pre-existing ungenerated
Storybook catalog errors), plugin SDK and CLI.
- `pnpm verify --explain` selects a broad plan because a doc under
`doc/plugins` changed; CI supplies that coverage.

## Risks

- No migration. Built-in connectors only gain one string check per
discovery; the plugins table is read only when a plugin connection is
present.
- Circleback's reviewed risk is now data. The behavior is the same, and
the existing policy tests and new tests cover it.
- Connection intents (agent connection requests in chat) still search
only built-in apps. Plugin connectors can be set up from the Connectors
page. This is a follow-up.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file editing, test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:51:39 +00:00

229 lines
8.3 KiB
TypeScript

import { eq } from "drizzle-orm";
import { toolConnections, toolInvocations, type Db } from "@paperclipai/db";
import { getConnectableAppDefinition, isPluginAppSlug } from "@paperclipai/shared";
import {
trackConnectionCreated,
trackConnectionUpdated,
trackConnectionInvoked,
} from "@paperclipai/shared/telemetry";
import { logger } from "../middleware/logger.js";
import { getTelemetryClient } from "../telemetry.js";
type ToolConnectionRow = typeof toolConnections.$inferSelect;
type ToolInvocationRow = typeof toolInvocations.$inferSelect;
export type ConnectorSetupFlow = "gallery" | "api" | "example";
export type ConnectorChangeSource =
| "api"
| "gallery"
| "oauth_callback"
| "credential_refresh"
| "archive"
| "example";
/**
* Terminal `tool_invocations.status` values. `pending`, `authorized`,
* `awaiting_approval`, and `executing` are in-flight and never emit: pending
* approval is not a failure. `cancelled` is declared in the schema enum but no
* current code path writes it to an invocation (signing-path cancellations
* land on the action request and fail the invocation); it stays here so a
* future writer is counted without a telemetry change.
*/
const TERMINAL_INVOCATION_STATUSES: ReadonlySet<string> = new Set([
"succeeded",
"failed",
"denied",
"cancelled",
"timed_out",
"rate_limited",
]);
/**
* Connector identity for telemetry is the reviewed first-party catalog slug
* only: `config.sourceTemplateKey` validated against the shared app-definitions
* catalog. Anything else — custom MCP servers, user-named connections, keys
* that no longer resolve to a catalog entry — reports the literal `custom` so
* no user- or provider-controlled string leaves the process.
*/
export function connectorKeyForConnection(
connection: Pick<ToolConnectionRow, "config">,
): string {
const raw = connection.config?.sourceTemplateKey;
const key = typeof raw === "string" ? raw : null;
// Only built-in connector keys leave the instance; a plugin's connector
// names its (possibly private) plugin, so it is reported as custom.
return key && !isPluginAppSlug(key) && getConnectableAppDefinition(key) ? key : "custom";
}
/**
* Setup-test invocations (Apps → Test tab) are separated from agent usage on
* durable invocation columns, mirroring the gateway's own test-origin
* predicate, so the split survives reloads and approval-driven completion.
*/
export function invocationOrigin(
invocation: Pick<
ToolInvocationRow,
"actorType" | "runId" | "issueId" | "gatewayId" | "connectionId"
>,
): string {
const isTestOrigin =
invocation.actorType === "user" &&
invocation.runId === null &&
invocation.issueId === null &&
invocation.gatewayId === null &&
invocation.connectionId !== null;
return isTestOrigin ? "setup_test" : invocation.actorType;
}
function isToolPurpose(connection: Pick<ToolConnectionRow, "connectionPurpose">): boolean {
return connection.connectionPurpose === "tool";
}
/**
* Emits one proposed `connection.created` event for a tool-purpose
* connection row that a caller already committed. Channel and AI connections
* never emit. This function never throws; a telemetry failure must never fail
* a connection setup path.
*/
export function emitConnectionCreated(
connection: ToolConnectionRow,
setupFlow: ConnectorSetupFlow,
): void {
try {
const client = getTelemetryClient();
if (!client) return;
if (!isToolPurpose(connection)) return;
trackConnectionCreated(client, {
connector_key: connectorKeyForConnection(connection),
transport: connection.transport,
auth_kind: connection.authKind,
setup_flow: setupFlow,
status: connection.status,
enabled: connection.enabled,
});
} catch (err) {
logger.warn(
{ err, connectionId: connection.id },
"failed to emit connection.created telemetry",
);
}
}
/**
* Emits one proposed `connection.updated` event when a committed
* write changed a tool-purpose connection's persisted lifecycle state
* (`status` or `enabled`). Metadata-only saves, health polls, credential
* rotation, and catalog refreshes do not change either field and therefore
* never emit. This function never throws.
*/
export function emitConnectionUpdated(
connection: ToolConnectionRow,
previous: Pick<ToolConnectionRow, "status" | "enabled">,
changeSource: ConnectorChangeSource,
): void {
try {
const client = getTelemetryClient();
if (!client) return;
if (!isToolPurpose(connection)) return;
if (
previous.status === connection.status &&
previous.enabled === connection.enabled
) {
return;
}
trackConnectionUpdated(client, {
connector_key: connectorKeyForConnection(connection),
transport: connection.transport,
auth_kind: connection.authKind,
change_source: changeSource,
previous_status: previous.status,
status: connection.status,
previous_enabled: previous.enabled,
enabled: connection.enabled,
});
} catch (err) {
logger.warn(
{ err, connectionId: connection.id },
"failed to emit connection.updated telemetry",
);
}
}
/**
* Emits one proposed `connection.invoked` event for an invocation
* a caller already wrote to a terminal status. Despite the name, this is a
* completion event: it records finished invocation attempts and their
* terminal status, never invocation starts. Never await it: like
* `agent.task_run`, this is best-effort background work and must not delay
* the caller's own response or lifecycle writes.
*
* Call placement inside the gateway is deliberate and asymmetric. Failure
* paths call it right after their terminal save. Success paths call it only
* after the post-save bookkeeping (action-request settlement, tool-call
* event, audit) has completed, because a bookkeeping failure lands in a catch
* that overwrites the row to `failed` and emits there — emitting the success
* beforehand would let one execution report both outcomes.
*
* Self-guards, so callers do not have to re-check anything: non-terminal
* statuses, invocations without a tool-purpose connection, and an
* unregistered (still-proposed) event name all return without emitting. The
* registration gate keeps the completion path free of telemetry reads until
* schema adoption: the runtime client drops unregistered names in `track`,
* so there is no point loading rows for an event that cannot be queued.
*
* Delivery is best-effort per execution, not exactly-once. Known residual
* duplications, accepted for the proposal stage: an outer safety-net catch
* can re-save `failed` and emit again when a failure path's own bookkeeping
* throws after its emit (test-invocation approval flow), and a crash-side
* replay that re-writes a terminal status emits again. Deduplication state
* is out of scope for this proposal.
*/
export async function emitConnectionInvoked(
db: Db,
invocationId: string,
): Promise<void> {
try {
const client = getTelemetryClient();
if (!client) return;
if (!client.isRegisteredEventName("connection.invoked")) return;
const joined = await db
.select({ invocation: toolInvocations, connection: toolConnections })
.from(toolInvocations)
.innerJoin(
toolConnections,
eq(toolInvocations.connectionId, toolConnections.id),
)
.where(eq(toolInvocations.id, invocationId))
.then((rows) => rows[0] ?? null);
if (!joined) return;
const { invocation, connection } = joined;
if (!TERMINAL_INVOCATION_STATUSES.has(invocation.status)) return;
if (!isToolPurpose(connection)) return;
const startedAtMs = invocation.startedAt
? new Date(invocation.startedAt).getTime()
: null;
const completedAtMs = invocation.completedAt
? new Date(invocation.completedAt).getTime()
: null;
const durationSeconds =
startedAtMs !== null && completedAtMs !== null
? Math.max(0, Math.round((completedAtMs - startedAtMs) / 1000))
: undefined;
trackConnectionInvoked(client, {
connector_key: connectorKeyForConnection(connection),
transport: connection.transport,
status: invocation.status,
origin: invocationOrigin(invocation),
...(durationSeconds === undefined ? {} : { duration_seconds: durationSeconds }),
});
} catch (err) {
logger.warn(
{ err, invocationId },
"failed to emit connection.invoked telemetry",
);
}
}