Files
PaperClipAI/server/src/services/plugin-secrets-handler.ts
T
DottaandPaperclip 1de0a3bb1e feat(mcp) [split 2/8]: add governed access contracts (#9557)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Governed MCP access spans contracts, runtime enforcement, adapters,
UI surfaces, and operator verification
> - The parity reference PR #9534 is too large for effective automated
or human review
> - The feature therefore needs a linear stack whose individual diffs
stay below the 100-file review limit
> - This pull request is split 2/8 and focuses on database schema and
shared governance contracts
> - The benefit is a standalone, testable review boundary while
preserving byte-for-byte parity at the top of the stack

## Linked Issues or Issue Description

- Related parity reference: #9534
- Problem: The governed access model needs additive persistence and
synchronized shared types before server enforcement can compile.
- Proposed solution: Adds migrations 0148–0169, tool-access and Smoke
Lab schema, shared types/validators/gallery helpers, and the minimal
compile-required contract consumers identified by boundary testing.
- Alternatives considered: keeping #9534 as one 403-file review, or
rewriting the feature to manufacture seams; both were rejected in favor
of path extraction plus compile-driven boundary moves.
- Roadmap alignment: this advances the existing governed MCP/tool-access
work already represented by #9534; it does not introduce a separate
roadmap initiative.
- Stack position: base branch is `pap10341-split/01-demo-servers`.
- Merge policy: merge bottom-up, in order, only after the complete
eight-PR stack has been reviewed and the top-of-stack parity gate
remains empty.
- Requested review: QA for migrations/validators; Greptile on every PR.

## What Changed

- Adds migrations 0148–0169, tool-access and Smoke Lab schema, shared
types/validators/gallery helpers, and the minimal compile-required
contract consumers identified by boundary testing.
- Keeps this PR below 100 changed files and independently typecheckable.
- Preserves the final tree from #9534 when combined with the other seven
stack levels.

## Verification

- `pnpm typecheck` — passed, including migration numbering and safety
checks
- `pnpm --filter @paperclipai/db test` — passed
- `pnpm --filter @paperclipai/shared test` — passed

## Risks

- Migration or contract mistakes could affect every upper layer; all
migrations are additive/idempotent and compile consumers are included in
this boundary.
- Stack risk: merging out of order can expose incomplete layers;
mitigate by following the documented bottom-up merge policy.
- Parity risk: later edits to an intermediate branch can drift from
#9534; mitigate by re-running the empty top-of-stack diff before merge.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, exact model ID `gpt-5.4`; runtime-managed context
window; medium reasoning with repository, shell, Git, GitHub CLI, and
code-execution tools enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] Internal references are omitted except the execution-plan link
explicitly required for this coordinated split stack
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge


## Stack Coordination

- Internal execution plan:
[PAP-13874](/PAP/issues/PAP-13874#document-plan)
- Parity reference: #9534
- Stack: #9556 → #9557 → #9558 → #9559 → #9560 → #9561 → #9562 → #9563
- Merge bottom-up only after full-stack review and an empty parity diff
at #9563.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-14 12:57:20 -05:00

286 lines
9.5 KiB
TypeScript

/**
* Plugin secrets host-side handler. Plugin workers may resolve shared
* `secret_ref` config bindings only with an explicit company context.
*/
import { and, eq } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import { companySecretBindings } from "@paperclipai/db";
import type { EnvSecretRefBinding, SecretProjectionClass, SecretVersionSelector } from "@paperclipai/shared";
import { envBindingSecretRefSchema } from "@paperclipai/shared";
import {
collectSecretRefPaths,
isUuidSecretRef,
readConfigValueAtPath,
} from "./json-schema-secret-refs.js";
import { secretService } from "./secrets.js";
import { unprocessable } from "../errors.js";
// ---------------------------------------------------------------------------
// Error helpers
// ---------------------------------------------------------------------------
function invalidSecretRef(secretRef: unknown): Error {
const rendered = typeof secretRef === "string" ? secretRef : JSON.stringify(secretRef);
const err = new Error(
`Invalid secret reference for plugin: ${rendered ?? "<empty>"}. Use { type: "secret_ref", secretId, version? }`,
);
err.name = "InvalidSecretRefError";
return err;
}
function requireCompanyId(companyId: unknown): string {
if (typeof companyId !== "string" || companyId.trim().length === 0) {
throw unprocessable("companyId is required for plugin secret resolution");
}
return companyId.trim();
}
function isPlainRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
function parseSecretRefBinding(value: unknown): EnvSecretRefBinding | null {
const parsed = envBindingSecretRefSchema.safeParse(value);
return parsed.success ? parsed.data : null;
}
function assertSecretRefBinding(
value: unknown,
path: string,
rejectLegacyUuid = false,
): EnvSecretRefBinding | null {
if (rejectLegacyUuid && typeof value === "string" && isUuidSecretRef(value)) {
throw unprocessable(
`Plugin secret ref at ${path} must use { type: "secret_ref", secretId, version? }`,
);
}
if (!isPlainRecord(value) || value.type !== "secret_ref") return null;
const parsed = parseSecretRefBinding(value);
if (!parsed) {
throw unprocessable(`Invalid secret_ref binding at ${path}`);
}
return parsed;
}
export interface PluginConfigSecretRefBinding {
secretId: string;
configPath: string;
versionSelector?: SecretVersionSelector;
required?: boolean;
label?: string | null;
projectionClass?: SecretProjectionClass;
projectionAllowlistKey?: string | null;
}
// ---------------------------------------------------------------------------
// Validation
// ---------------------------------------------------------------------------
/** Extract shared object-shaped secret refs from plugin config. */
export function extractSecretRefBindingsFromConfig(
configJson: unknown,
schema?: Record<string, unknown> | null,
): PluginConfigSecretRefBinding[] {
if (configJson == null || typeof configJson !== "object") return [];
const refsByPath = new Map<string, PluginConfigSecretRefBinding>();
const addRef = (binding: EnvSecretRefBinding, configPath: string) => {
refsByPath.set(configPath, {
secretId: binding.secretId,
configPath,
versionSelector: binding.version ?? "latest",
required: true,
label: configPath,
projectionClass: binding.projectionClass,
projectionAllowlistKey: binding.projectionAllowlistKey ?? null,
});
};
const secretPaths = collectSecretRefPaths(schema);
for (const dotPath of secretPaths) {
const current = readConfigValueAtPath(configJson as Record<string, unknown>, dotPath);
const binding = assertSecretRefBinding(current, dotPath, true);
if (binding) addRef(binding, dotPath);
}
function walk(value: unknown, path: string): void {
const binding = assertSecretRefBinding(value, path || "$");
if (binding) {
addRef(binding, path || "$");
return;
}
if (Array.isArray(value)) {
value.forEach((item, index) => walk(item, path ? `${path}.${index}` : String(index)));
return;
}
if (!isPlainRecord(value)) return;
for (const [key, child] of Object.entries(value)) {
walk(child, path ? `${path}.${key}` : key);
}
}
walk(configJson, "");
return [...refsByPath.values()];
}
/** Backward-compatible helper returning only secret IDs. */
export function extractSecretRefsFromConfig(
configJson: unknown,
schema?: Record<string, unknown> | null,
): Set<string> {
return new Set(extractSecretRefBindingsFromConfig(configJson, schema).map((ref) => ref.secretId));
}
/** Backward-compatible helper returning secret IDs grouped by config path. */
export function extractSecretRefPathsFromConfig(
configJson: unknown,
schema?: Record<string, unknown> | null,
): Map<string, Set<string>> {
const refs = new Map<string, Set<string>>();
for (const ref of extractSecretRefBindingsFromConfig(configJson, schema)) {
const paths = refs.get(ref.secretId) ?? new Set<string>();
paths.add(ref.configPath);
refs.set(ref.secretId, paths);
}
return refs;
}
// ---------------------------------------------------------------------------
// Handler factory
// ---------------------------------------------------------------------------
export interface PluginSecretsResolveParams {
/** Shared secret reference object from company-scoped plugin config. */
secretRef: string | EnvSecretRefBinding;
/** Authorized company context for this worker invocation. */
companyId?: string;
/** Config path that produced this ref. Required when a secret appears in multiple paths. */
configPath?: string;
actorType?: "agent" | "user" | "system" | "plugin";
actorId?: string | null;
issueId?: string | null;
heartbeatRunId?: string | null;
}
export interface PluginSecretsHandlerOptions {
db: Db;
pluginId: string;
}
export interface PluginSecretsService {
resolve(params: PluginSecretsResolveParams): Promise<string>;
}
function createRateLimiter(maxAttempts: number, windowMs: number) {
const attempts = new Map<string, number[]>();
return {
check(key: string): boolean {
const now = Date.now();
const windowStart = now - windowMs;
const existing = (attempts.get(key) ?? []).filter((ts) => ts > windowStart);
if (existing.length >= maxAttempts) return false;
existing.push(now);
attempts.set(key, existing);
return true;
},
};
}
export function createPluginSecretsHandler(
options: PluginSecretsHandlerOptions,
): PluginSecretsService {
const { db, pluginId } = options;
const rateLimiter = createRateLimiter(30, 60_000);
async function lookupBinding(input: {
companyId: string;
secretId: string;
versionSelector: SecretVersionSelector;
configPath?: string;
}) {
const conditions = [
eq(companySecretBindings.companyId, input.companyId),
eq(companySecretBindings.targetType, "plugin"),
eq(companySecretBindings.targetId, pluginId),
eq(companySecretBindings.secretId, input.secretId),
];
if (input.configPath) {
conditions.push(eq(companySecretBindings.configPath, input.configPath));
}
const rows = await db
.select()
.from(companySecretBindings)
.where(and(...conditions));
const matchingVersion = rows.filter(
(row) => row.versionSelector === String(input.versionSelector),
);
return matchingVersion;
}
return {
async resolve(params: PluginSecretsResolveParams): Promise<string> {
if (typeof params.secretRef === "string") {
throw invalidSecretRef(params.secretRef.trim() || "<empty>");
}
const bindingRef = parseSecretRefBinding(params.secretRef);
if (!bindingRef) throw invalidSecretRef(params.secretRef);
const companyId = requireCompanyId(params.companyId);
if (!rateLimiter.check(`${companyId}:${pluginId}`)) {
const err = new Error("Rate limit exceeded for secret resolution");
err.name = "RateLimitExceededError";
throw err;
}
const versionSelector = bindingRef.version ?? "latest";
const bindings = await lookupBinding({
companyId,
secretId: bindingRef.secretId,
versionSelector,
configPath: params.configPath,
});
if (bindings.length === 0) {
throw unprocessable(
`Secret is not bound to plugin:${pluginId}${params.configPath ? ` at ${params.configPath}` : ""}`,
{ code: "binding_missing" },
);
}
if (bindings.length > 1) {
throw unprocessable(
"Plugin secret reference is ambiguous; pass configPath when resolving this secret",
{ code: "binding_ambiguous" },
);
}
const binding = bindings[0]!;
return secretService(db).resolveSecretValue(companyId, bindingRef.secretId, versionSelector, {
bindingContext: {
consumerType: "plugin",
consumerId: pluginId,
configPath: binding.configPath,
actorType: params.actorType ?? "plugin",
actorId: params.actorId ?? pluginId,
issueId: params.issueId ?? null,
heartbeatRunId: params.heartbeatRunId ?? null,
pluginId,
},
accessContext: {
consumerType: "plugin_worker",
consumerId: pluginId,
configPath: binding.configPath,
actorType: params.actorType ?? "plugin",
actorId: params.actorId ?? pluginId,
issueId: params.issueId ?? null,
heartbeatRunId: params.heartbeatRunId ?? null,
pluginId,
},
});
},
};
}