[codex] Add built-in Hermes adapters (#8543)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent adapters are the boundary between the control plane and the
runtimes that actually do work.
> - Hermes support needs to be available as first-class local and
gateway adapters while still preserving the adapter-manager override
path for external packages.
> - The adapter work touches runtime execution, UI adapter metadata,
onboarding prompts, scoped credentials, release packaging, and smoke
coverage, so the handoff needs concrete verification rather than only
unit tests.
> - This pull request adds built-in Hermes local and Hermes gateway
support, keeps external adapter overrides compatible, and
documents/tests the gateway flow end to end.
> - The benefit is that operators can hire Hermes-backed agents without
a manual plugin install, while self-hosted installs can still
override/shadow the built-ins through Adapter manager packages.

## Linked Issues or Issue Description

No public GitHub issue exists for this exact Hermes built-in adapter,
gateway onboarding, and release-source work.

Problem description:
- Hermes local and gateway adapters need a public, reviewable source
path in the monorepo so package artifacts and built-in adapter behavior
match the application source.
- Operators need built-in `hermes_local` and `hermes_gateway` adapter
choices without losing the ability to install external Hermes packages
as overrides.
- Gateway onboarding needs secure defaults for API server URLs, API
keys, and generated agent setup text.
- Hermes-originated task bridge credentials need narrower API-key scope
configuration.
- Related public PRs found during duplicate search include #3027, #2363,
#7544, #7950, #8095, and #8543.

## What Changed

- Added the unified Hermes adapter package with local and gateway
server/UI/CLI exports, config schemas, transcript parsing, model
detection, and package metadata.
- Registered `hermes_local` and `hermes_gateway` as built-in adapters
across shared constants, server registries, CLI packaging, and UI
adapter registries.
- Kept the external adapter override path compatible so installed Hermes
packages can shadow built-ins and restore the built-in parser when
disabled.
- Added Hermes gateway onboarding docs, board-operator docs, Docker
smoke assets, and shell smoke harnesses for join/e2e validation.
- Added scoped task-bridge API-key support, authorization checks,
issue-origin handling, and tests for Hermes-created Paperclip tasks.
- Hardened gateway transport and redaction behavior for API keys,
headers, session data, and smoke diagnostics.
- Updated release packaging/bootstrap checks for the Hermes packages
while leaving `pnpm-lock.yaml` out of the PR per repository policy.

## Verification

Targeted local verification recorded before PR handoff:
- `pnpm --filter @paperclipai/hermes-paperclip-adapter exec vitest run
src/gateway/server/execute.test.ts` — 14/14 passed.
- `pnpm test:hermes-gateway-smoke` — 6/6 passed.
- Hermes package typecheck/build checks passed.
- Focused server/UI adapter tests passed — 31/31.
- Release helper Node tests passed — 18/18.
- `git diff --check origin/master..HEAD` passed.

Fresh Docker E2E smoke evidence:
- Ran `pnpm smoke:hermes-gateway-e2e` on 2026-06-26 with a fresh state
directory and fresh Docker container against a live Paperclip dev
server.
- Hermes direct execution reached `completed`.
- Hermes stop/cancel path reached `cancelled`.
- Hermes gateway created a Paperclip task, Paperclip ran the Hermes
agent, and the task reached `done` with the expected marker response.
- Temporary board auth keys, token files, smoke state, and Docker
containers were cleaned up after the run.

PR checks on head `b5eae40ce`:
- GitHub Actions passed: `policy`, `review`, `Typecheck + Release
Registry`, all general test shards, all serialized server shards,
`Build`, `Canary Dry Run`, `e2e`, and aggregate `verify`.
- External checks passed: Snyk and Socket Project Report.
- External Socket Pull Request Alerts remained pending after the
first-party CI matrix completed.

## Risks

- Medium risk: this spans adapter registration, package publishing,
gateway execution, onboarding docs, API-key scoping, and UI adapter
metadata.
- Migration risk is low: the scope-config migration adds a nullable
column and does not rewrite existing keys.
- Gateway execution depends on operator-provided Hermes API
configuration; the smoke covers the Docker gateway path but real
deployments may differ by network/auth setup.
- Direct Greptile review on the latest expanded diff is file-count
limited, although the commitperclip review gate passed.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, GPT-5 coding agent, tool use enabled in a local repository
workspace. Context window size is not exposed in this environment.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Commitperclip review gate is green; direct Greptile review is
file-count limited on the latest expanded diff
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-06-26 16:04:58 -05:00
1 parent ef1422c23e
commit fd2f82ac5b
131 files changed
+10179 -686

No files matched your search

@@ -1,4 +1,18 @@
import { envBindingSchema, type SecretVersionSelector } from "@paperclipai/shared";
interface AgentSecretBindingSyncService {
syncSecretRefsForTarget?: (
companyId: string,
target: { targetType: "agent"; targetId: string },
refs: Array<{
secretId: string;
configPath: string;
versionSelector?: SecretVersionSelector;
required?: boolean;
label?: string | null;
}>,
options?: { replaceAll?: boolean },
) => Promise<unknown>;
syncEnvBindingsForTarget?: (
companyId: string,
target: { targetType: "agent"; targetId: string; pathPrefix?: string },
@@ -11,12 +25,63 @@ function asRecord(value: unknown): Record<string, unknown> | null {
return value as Record<string, unknown>;
}
function collectSecretRefs(adapterConfig: unknown): Array<{
secretId: string;
configPath: string;
versionSelector?: SecretVersionSelector;
}> {
const config = asRecord(adapterConfig);
if (!config) return [];
const refs: Array<{
secretId: string;
configPath: string;
versionSelector?: SecretVersionSelector;
}> = [];
const envValue = asRecord(config.env);
for (const [key, rawBinding] of Object.entries(envValue ?? {})) {
const parsed = envBindingSchema.safeParse(rawBinding);
if (!parsed.success) continue;
const binding = parsed.data;
if (typeof binding !== "object" || binding === null || binding.type !== "secret_ref") continue;
refs.push({
secretId: binding.secretId,
configPath: `env.${key}`,
versionSelector: binding.version ?? "latest",
});
}
for (const [key, rawBinding] of Object.entries(config)) {
if (key === "env") continue;
const parsed = envBindingSchema.safeParse(rawBinding);
if (!parsed.success) continue;
const binding = parsed.data;
if (typeof binding !== "object" || binding === null || binding.type !== "secret_ref") continue;
refs.push({
secretId: binding.secretId,
configPath: key,
versionSelector: binding.version ?? "latest",
});
}
return refs;
}
export async function syncAgentAdapterEnvBindings(input: {
secretsSvc: AgentSecretBindingSyncService;
companyId: string;
agentId: string;
adapterConfig: unknown;
}) {
if (input.secretsSvc.syncSecretRefsForTarget) {
await input.secretsSvc.syncSecretRefsForTarget(
input.companyId,
{ targetType: "agent", targetId: input.agentId },
collectSecretRefs(input.adapterConfig),
{ replaceAll: true },
);
return;
}
const envValue = asRecord(asRecord(input.adapterConfig)?.env);
await input.secretsSvc.syncEnvBindingsForTarget?.(
input.companyId,
+48 -4
View File
@@ -20,8 +20,10 @@ import {
AGENT_DEFAULT_MAX_CONCURRENT_RUNS,
getAgentWorkEligibility,
isUuidLike,
normalizeAgentApiKeyScope,
normalizeAgentUrlKey,
type AgentEligibilityAgent,
type AgentApiKeyScope,
} from "@paperclipai/shared";
import { conflict, notFound, unprocessable } from "../errors.js";
import { syncAgentAdapterEnvBindings } from "./agent-secret-bindings.js";
@@ -428,6 +430,16 @@ export function agentService(db: Db) {
const role = (data.role ?? existing.role) as string;
normalizedPatch.permissions = normalizeAgentPermissions(data.permissions, role);
}
if (
Object.prototype.hasOwnProperty.call(normalizedPatch, "adapterConfig") &&
isPlainRecord(normalizedPatch.adapterConfig)
) {
normalizedPatch.adapterConfig = await secretsSvc.normalizeAdapterConfigForPersistence(
existing.companyId,
normalizedPatch.adapterConfig,
{ adapterType: (normalizedPatch.adapterType ?? existing.adapterType) as string },
);
}
const shouldRecordRevision = Boolean(options?.recordRevision) && hasConfigPatchFields(normalizedPatch);
const beforeConfig = shouldRecordRevision ? buildConfigSnapshot(existing) : null;
@@ -503,11 +515,24 @@ export function agentService(db: Db) {
const role = data.role ?? "general";
const normalizedPermissions = normalizeAgentPermissions(data.permissions, role);
const runtimeConfig = normalizeRuntimeConfigForNewAgent(data.runtimeConfig);
const adapterType = data.adapterType ?? "process";
const adapterConfig = isPlainRecord(data.adapterConfig)
? await secretsSvc.normalizeAdapterConfigForPersistence(companyId, data.adapterConfig, { adapterType })
: {};
return db.transaction(async (tx) => {
const txDb = tx as unknown as Db;
const created = await tx
.insert(agents)
.values({ ...data, name: uniqueName, companyId, role, permissions: normalizedPermissions, runtimeConfig })
.values({
...data,
name: uniqueName,
companyId,
role,
adapterType,
adapterConfig,
permissions: normalizedPermissions,
runtimeConfig,
})
.returning()
.then((rows) => rows[0]);
await syncAgentSecretBindings(created, txDb);
@@ -733,7 +758,7 @@ export function agentService(db: Db) {
});
},
createApiKey: async (id: string, name: string) => {
createApiKey: async (id: string, name: string, scope: AgentApiKeyScope = { kind: "standard" }) => {
const existing = await getById(id);
if (!existing) throw notFound("Agent not found");
if (existing.status === "pending_approval") {
@@ -752,6 +777,7 @@ export function agentService(db: Db) {
companyId: existing.companyId,
name,
keyHash,
scopeConfig: scope.kind === "standard" ? null : scope,
})
.returning()
.then((rows) => rows[0]);
@@ -759,6 +785,7 @@ export function agentService(db: Db) {
return {
id: created.id,
name: created.name,
scope: normalizeAgentApiKeyScope(created.scopeConfig),
token,
createdAt: created.createdAt,
};
@@ -769,11 +796,19 @@ export function agentService(db: Db) {
.select({
id: agentApiKeys.id,
name: agentApiKeys.name,
scopeConfig: agentApiKeys.scopeConfig,
createdAt: agentApiKeys.createdAt,
revokedAt: agentApiKeys.revokedAt,
})
.from(agentApiKeys)
.where(eq(agentApiKeys.agentId, id)),
.where(eq(agentApiKeys.agentId, id))
.then((rows) => rows.map((row) => ({
id: row.id,
name: row.name,
scope: normalizeAgentApiKeyScope(row.scopeConfig),
createdAt: row.createdAt,
revokedAt: row.revokedAt,
}))),
getKeyById: async (keyId: string) =>
db
@@ -782,12 +817,21 @@ export function agentService(db: Db) {
agentId: agentApiKeys.agentId,
companyId: agentApiKeys.companyId,
name: agentApiKeys.name,
scopeConfig: agentApiKeys.scopeConfig,
createdAt: agentApiKeys.createdAt,
revokedAt: agentApiKeys.revokedAt,
})
.from(agentApiKeys)
.where(eq(agentApiKeys.id, keyId))
.then((rows) => rows[0] ?? null),
.then((rows) => {
const row = rows[0] ?? null;
return row
? {
...row,
scope: normalizeAgentApiKeyScope(row.scopeConfig),
}
: null;
}),
revokeKey: async (agentId: string, keyId: string) => {
const rows = await db
+152 -1
View File
@@ -10,7 +10,7 @@ import {
principalPermissionGrants,
projects,
} from "@paperclipai/db";
import type { PermissionKey, PrincipalType } from "@paperclipai/shared";
import type { AgentApiKeyScope, PermissionKey, PrincipalType, TaskBridgeAgentKeyScope } from "@paperclipai/shared";
import { LOW_TRUST_REVIEW_PRESET, extractAgentMentionIds, type LowTrustBoundary } from "@paperclipai/shared";
import {
LOW_TRUST_ISSUE_ANCESTRY_MAX_DEPTH,
@@ -29,6 +29,8 @@ export type AuthorizationActor =
isInstanceAdmin?: boolean;
agentId?: string | null;
companyId?: string | null;
keyId?: string | null;
keyScope?: AgentApiKeyScope | null;
runId?: string | null;
source?:
| "local_implicit"
@@ -66,6 +68,8 @@ export type AuthorizationResource =
parentIssueId?: string | null;
assigneeAgentId?: string | null;
assigneeUserId?: string | null;
originKind?: string | null;
originId?: string | null;
status?: string | null;
};
@@ -211,6 +215,8 @@ type IssueAuthorizationRow = {
assigneeUserId: string | null;
status: string;
executionPolicy: unknown;
originKind: string | null;
originId: string | null;
};
function evaluateAuthorizationPolicyForAssignment(
@@ -550,6 +556,8 @@ export function authorizationService(db: Db) {
assigneeUserId: issues.assigneeUserId,
status: issues.status,
executionPolicy: issues.executionPolicy,
originKind: issues.originKind,
originId: issues.originId,
})
.from(issues)
.where(eq(issues.id, issueId))
@@ -802,6 +810,130 @@ export function authorizationService(db: Db) {
return null;
}
function taskBridgeScopeIds(
scope: TaskBridgeAgentKeyScope,
singularKey: "projectId" | "parentIssueId",
pluralKey: "projectIds" | "parentIssueIds",
) {
return [
...(typeof scope[singularKey] === "string" ? [scope[singularKey]] : []),
...(Array.isArray(scope[pluralKey]) ? scope[pluralKey] : []),
].filter((value): value is string => typeof value === "string" && value.length > 0);
}
async function parentIssueMatchesTaskBridgeBoundary(
parentIssueId: string | null | undefined,
companyId: string,
allowedParentIssueIds: string[],
) {
if (!parentIssueId || allowedParentIssueIds.length === 0) return false;
if (allowedParentIssueIds.includes(parentIssueId)) return true;
for (const rootIssueId of allowedParentIssueIds) {
if (await issueIdIsDescendantOf(parentIssueId, rootIssueId, companyId)) return true;
}
return false;
}
async function issueMatchesTaskBridgeCreateBoundary(
scope: TaskBridgeAgentKeyScope,
resource: Extract<AuthorizationResource, { type: "issue" }>,
) {
const allowedProjectIds = taskBridgeScopeIds(scope, "projectId", "projectIds");
const allowedParentIssueIds = taskBridgeScopeIds(scope, "parentIssueId", "parentIssueIds");
if (resource.projectId && allowedProjectIds.includes(resource.projectId)) return true;
if (await parentIssueMatchesTaskBridgeBoundary(resource.parentIssueId, resource.companyId, allowedParentIssueIds)) {
return true;
}
if (resource.parentIssueId && allowedProjectIds.length > 0) {
const parent = await loadIssue(resource.parentIssueId);
if (parent?.companyId === resource.companyId && parent.projectId && allowedProjectIds.includes(parent.projectId)) {
return true;
}
}
return false;
}
async function issueMatchesTaskBridgeWriteBoundary(input: {
actorAgentId: string;
keyId: string;
resource: Extract<AuthorizationResource, { type: "issue" }>;
}) {
const issue = input.resource.issueId ? await loadIssue(input.resource.issueId) : null;
const assigneeAgentId = issue?.assigneeAgentId ?? input.resource.assigneeAgentId ?? null;
if (assigneeAgentId === input.actorAgentId) return true;
const originKind = issue?.originKind ?? input.resource.originKind ?? null;
const originId = issue?.originId ?? input.resource.originId ?? null;
return originKind === "task_bridge" && originId === input.keyId;
}
async function decideTaskBridgeAccess(input: {
actorAgentId: string;
action: AuthorizationAction;
resource: AuthorizationResource;
scope: TaskBridgeAgentKeyScope;
keyId: string;
}): Promise<AuthorizationDecision | null> {
const denyBridge = (explanation: string) =>
deny({
action: input.action,
reason: "deny_scope",
explanation,
});
const allowBridge = (explanation: string) =>
allow({
action: input.action,
reason: "allow_explicit_grant",
explanation,
});
if (
input.action === "company_scope:read" ||
input.action === "agent:read" ||
input.action === "agent:wake" ||
input.action === "project:read" ||
input.action === "runtime:manage" ||
input.action === "secrets:read"
) {
return denyBridge("Task bridge keys cannot use company-wide, peer-agent, project, runtime, or secret APIs.");
}
if (input.action === "tasks:assign") {
if (input.resource.type !== "issue") {
return denyBridge("Task bridge assignment requires an issue resource.");
}
if (!(await issueMatchesTaskBridgeCreateBoundary(input.scope, input.resource))) {
return denyBridge("Task bridge key is outside its approved parent or project boundary.");
}
if (input.resource.assigneeUserId) {
return denyBridge("Task bridge keys cannot assign work to board users.");
}
const allowedAssigneeAgentIds = input.scope.allowedAssigneeAgentIds ?? [];
if (
input.resource.assigneeAgentId &&
input.resource.assigneeAgentId !== input.actorAgentId &&
!allowedAssigneeAgentIds.includes(input.resource.assigneeAgentId)
) {
return denyBridge("Task bridge key cannot assign work to that agent.");
}
return allowBridge("Allowed by task bridge create boundary.");
}
if (input.action === "issue:read" || input.action === "issue:comment" || input.action === "issue:mutate") {
if (input.resource.type !== "issue") {
return denyBridge("Task bridge issue access requires an issue resource.");
}
return await issueMatchesTaskBridgeWriteBoundary({
actorAgentId: input.actorAgentId,
keyId: input.keyId,
resource: input.resource,
})
? allowBridge("Allowed for bridge-created or assigned issue.")
: denyBridge("Task bridge key can only access assigned or bridge-created issues.");
}
return denyBridge("Task bridge key cannot use this API action.");
}
async function assignmentTargetIsInCompany(resource: AuthorizationResource) {
if (resource.type !== "issue") return true;
if (resource.assigneeAgentId) {
@@ -1183,6 +1315,25 @@ export function authorizationService(db: Db) {
});
}
if (input.actor.source === "agent_key" && input.actor.keyScope?.kind === "task_bridge") {
const keyId = input.actor.keyId ?? null;
if (!keyId) {
return deny({
action: input.action,
reason: "deny_scope",
explanation: "Task bridge key context is missing.",
});
}
const taskBridgeDecision = await decideTaskBridgeAccess({
actorAgentId,
action: input.action,
resource: input.resource,
scope: input.actor.keyScope,
keyId,
});
if (taskBridgeDecision) return taskBridgeDecision;
}
const lowTrustDecision = await decideLowTrustAccess({
actorAgentId,
action: input.action,
+1 -1
View File
@@ -3050,7 +3050,7 @@ export function companyPortabilityService(db: Db, storage?: StorageService) {
const normalizedAdapterConfig = await secrets.normalizeAdapterConfigForPersistence(
companyId,
nextAdapterConfig,
{ strictMode: strictSecretsMode },
{ strictMode: strictSecretsMode, adapterType: effectiveAdapterType },
);
await assertImportAdapterConfigConstraints(effectiveAdapterType, normalizedAdapterConfig);
return {
+42 -1
View File
@@ -412,7 +412,10 @@ const INLINE_BASE64_IMAGE_DATA_RE = /("type":"image","source":\{"type":"base64",
type RuntimeConfigSecretResolver = Pick<
ReturnType<typeof secretService>,
"resolveAdapterConfigForRuntime" | "resolveEnvBindings" | "collectMissingRuntimeBindings"
| "resolveAdapterConfigForRuntime"
| "resolveEnvBindings"
| "collectMissingRuntimeBindings"
| "collectMissingAdapterConfigRuntimeBindings"
>;
function formatMissingBindingForOperator(missing: MissingRuntimeBinding): string {
@@ -494,6 +497,7 @@ function assertLowTrustEnvConfigAllowed(envValue: unknown, source: string) {
export async function resolveExecutionRunAdapterConfig(input: {
companyId: string;
agentId?: string | null;
adapterType?: string | null;
issueId?: string | null;
heartbeatRunId?: string | null;
environmentId?: string | null;
@@ -574,6 +578,16 @@ export async function resolveExecutionRunAdapterConfig(input: {
{ consumerType: "agent", consumerId: input.agentId },
)),
);
if (typeof input.secretsSvc.collectMissingAdapterConfigRuntimeBindings === "function") {
missingBindings.push(
...(await input.secretsSvc.collectMissingAdapterConfigRuntimeBindings(
input.companyId,
executionRunConfig,
input.adapterType ?? null,
{ consumerType: "agent", consumerId: input.agentId },
)),
);
}
}
if (projectEnv && input.projectId) {
missingBindings.push(
@@ -666,6 +680,7 @@ export async function resolveExecutionRunAdapterConfig(input: {
...(lowTrustAllowedBindingIds !== undefined ? { allowedBindingIds: lowTrustAllowedBindingIds } : {}),
}
: undefined,
{ adapterType: input.adapterType ?? null },
);
if (Object.keys(environmentEnvResolution.env).length > 0) {
resolvedConfig.env = {
@@ -3140,6 +3155,13 @@ export function buildPaperclipTaskMarkdown(input: {
workMode?: string | null;
description?: string | null;
} | null;
ancestors?: Array<{
id: string;
identifier?: string | null;
title?: string | null;
status?: string | null;
priority?: string | null;
}> | null;
wakeComment?: {
id: string;
body: string;
@@ -3160,6 +3182,7 @@ export function buildPaperclipTaskMarkdown(input: {
return [fence + "text", value, fence].join("\n");
};
const issue = input.issue;
const ancestors = (input.ancestors ?? []).slice(0, 6);
const wakeComment = input.wakeComment ?? null;
const acceptedPlanContinuation =
!wakeComment &&
@@ -3212,6 +3235,19 @@ export function buildPaperclipTaskMarkdown(input: {
lines.push("", "Issue description:", fenceTaskText(description));
}
}
if (ancestors.length > 0) {
lines.push("", "Authoritative parent / ancestor context:");
for (const [index, ancestor] of ancestors.entries()) {
const label = ancestor.identifier || ancestor.id;
const status = ancestor.status ? ` (${ancestor.status})` : "";
const priority = ancestor.priority ? ` [${ancestor.priority}]` : "";
const title = ancestor.title ? ` ${ancestor.title}` : "";
lines.push(`- ${index === 0 ? "Parent" : `Ancestor ${index + 1}`}: ${label}${title}${status}${priority}`);
}
if ((input.ancestors ?? []).length > ancestors.length) {
lines.push(`- [ancestor context truncated after ${ancestors.length} entries]`);
}
}
if (wakeComment?.body.trim()) {
lines.push("", "Latest wake comment:", fenceTaskText(wakeComment.body.trim()));
}
@@ -8647,6 +8683,9 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
wakeCommentContext && !exposeLowTrustRaw
? sanitizeQuarantinedCommentForHigherTrust(wakeCommentContext)
: wakeCommentContext;
const issueAncestors = issueRef
? await issuesSvc.getAncestors(issueRef.id)
: [];
if (continuationSummary) {
context.paperclipContinuationSummary = {
key: safeContinuationSummary!.key,
@@ -8692,6 +8731,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
description: issueRef.description,
}
: null,
ancestors: issueAncestors,
wakeComment: safeWakeCommentContext,
interaction: {
kind: readNonEmptyString(context.interactionKind),
@@ -8918,6 +8958,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
const { resolvedConfig, secretKeys, secretManifest } = await resolveExecutionRunAdapterConfig({
companyId: agent.companyId,
agentId: agent.id,
adapterType: agent.adapterType,
issueId,
heartbeatRunId: run.id,
environmentId: selectedEnvironmentForConfig?.id ?? null,
+330 -42
View File
@@ -1,3 +1,4 @@
import { randomUUID } from "node:crypto";
import { and, desc, eq, inArray, like, ne, notInArray, or, sql } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import {
@@ -55,6 +56,7 @@ import type {
} from "../secrets/types.js";
import { isSecretProviderClientError } from "../secrets/types.js";
import { authorizationService } from "./authorization.js";
import { findActiveServerAdapter } from "../adapters/index.js";
const ENV_KEY_RE = /^[A-Za-z_][A-Za-z0-9_]*$/;
const SENSITIVE_ENV_KEY_RE =
@@ -64,6 +66,9 @@ const COMING_SOON_SECRET_PROVIDERS: ReadonlySet<SecretProvider> = new Set([
"gcp_secret_manager",
"vault",
]);
const FALLBACK_ADAPTER_SCHEMA_SECRET_FIELDS: Readonly<Record<string, readonly string[]>> = {
hermes_gateway: ["apiKey"],
};
type DbTransaction = Parameters<Parameters<Db["transaction"]>[0]>[0];
type SecretBindingDb = Pick<Db | DbTransaction, "select" | "delete" | "insert">;
@@ -357,6 +362,11 @@ export function secretService(db: Db) {
strictMode?: boolean;
fieldPath?: string;
};
type NormalizeAdapterConfigOptions = {
strictMode?: boolean;
adapterType?: string | null;
actor?: { userId?: string | null; agentId?: string | null };
};
async function getById(id: string, source: Pick<Db | DbTransaction, "select"> = db) {
return source
@@ -809,16 +819,210 @@ export function secretService(db: Db) {
async function normalizeAdapterConfigForPersistenceInternal(
companyId: string,
adapterConfig: Record<string, unknown>,
opts?: { strictMode?: boolean },
opts?: NormalizeAdapterConfigOptions,
) {
const normalized = { ...adapterConfig };
if (!Object.prototype.hasOwnProperty.call(adapterConfig, "env")) {
return normalized;
if (Object.prototype.hasOwnProperty.call(adapterConfig, "env")) {
normalized.env = await normalizeEnvConfig(companyId, adapterConfig.env, opts);
}
const secretFieldKeys = await listAdapterSchemaSecretFieldKeys(opts?.adapterType);
for (const key of secretFieldKeys) {
if (!Object.prototype.hasOwnProperty.call(adapterConfig, key)) continue;
const value = await normalizeSchemaSecretFieldForPersistence(companyId, {
adapterType: opts?.adapterType ?? null,
key,
rawValue: adapterConfig[key],
actor: opts?.actor,
});
if (value === undefined) {
delete normalized[key];
} else {
normalized[key] = value;
}
}
normalized.env = await normalizeEnvConfig(companyId, adapterConfig.env, opts);
return normalized;
}
async function listAdapterSchemaSecretFieldKeys(adapterType: string | null | undefined): Promise<string[]> {
if (!adapterType) return [];
const adapter = findActiveServerAdapter(adapterType);
const fallback = [...(FALLBACK_ADAPTER_SCHEMA_SECRET_FIELDS[adapterType] ?? [])];
if (!adapter?.getConfigSchema) return fallback;
try {
const schema = await adapter.getConfigSchema();
return [...new Set([
...fallback,
...schema.fields
.filter((field) => field.meta?.secret === true)
.map((field) => field.key),
])];
} catch (err) {
logger.warn({ err, adapterType }, "adapter config schema unavailable while normalizing secret fields");
return fallback;
}
}
async function normalizeSchemaSecretFieldForPersistence(
companyId: string,
input: {
adapterType: string | null;
key: string;
rawValue: unknown;
actor?: { userId?: string | null; agentId?: string | null };
},
): Promise<EnvBinding | undefined> {
if (input.rawValue === null || input.rawValue === undefined) return undefined;
const parsed = envBindingSchema.safeParse(input.rawValue);
if (!parsed.success) {
throw unprocessable(`${input.key} must be a string, plain binding, or secret reference`);
}
const binding = canonicalizeBinding(parsed.data as EnvBinding);
if (binding.type === "secret_ref") {
await assertSecretInCompany(companyId, binding.secretId);
return {
type: "secret_ref",
secretId: binding.secretId,
version: binding.version,
};
}
const value = binding.value.trim();
if (!value) return undefined;
if (value === REDACTED_SENTINEL) {
throw unprocessable(`Refusing to persist redacted placeholder for key: ${input.key}`);
}
const id = randomUUID();
const adapterPart = normalizeSecretKey(input.adapterType ?? "adapter");
const fieldPart = normalizeSecretKey(input.key);
const secret = await createManagedLocalSecret(companyId, {
name: `${adapterPart}.${fieldPart}.${id}`,
key: `${adapterPart}.${fieldPart}.${id}`,
value,
description: `Adapter config secret for ${input.adapterType ?? "adapter"}.${input.key}`,
}, input.actor);
return {
type: "secret_ref",
secretId: secret.id,
version: "latest",
};
}
async function createManagedLocalSecret(
companyId: string,
input: {
name: string;
key: string;
value: string;
description?: string | null;
},
actor?: { userId?: string | null; agentId?: string | null },
) {
const existing = await getByName(companyId, input.name);
if (existing) throw conflict(`Secret already exists: ${input.name}`);
const key = normalizeSecretKey(input.key);
if (!key) throw unprocessable("Secret key is required");
const duplicateKey = await db
.select()
.from(companySecrets)
.where(and(
eq(companySecrets.companyId, companyId),
eq(companySecrets.key, key),
ne(companySecrets.status, "deleted"),
))
.then((rows) => rows[0] ?? null);
if (duplicateKey) throw conflict(`Secret key already exists: ${key}`);
const provider = getSecretProvider("local_encrypted");
const providerConfig = await getSelectableRuntimeProviderConfig({
companyId,
provider: "local_encrypted",
providerConfigId: null,
});
const providerWriteContext = {
companyId,
secretKey: key,
secretName: input.name,
version: 1,
};
const reservedSecret = await db
.insert(companySecrets)
.values({
companyId,
key,
name: input.name,
provider: "local_encrypted",
providerConfigId: null,
status: "archived",
managedMode: "paperclip_managed",
externalRef: null,
providerMetadata: null,
latestVersion: 0,
description: input.description ?? null,
createdByAgentId: actor?.agentId ?? null,
createdByUserId: actor?.userId ?? null,
})
.returning()
.then((rows) => rows[0]);
let prepared: PreparedSecretVersion | null = null;
try {
prepared = await provider.createSecret({
value: input.value,
externalRef: null,
providerConfig,
context: providerWriteContext,
});
const preparedSecret = prepared;
await db.insert(companySecretVersions).values({
secretId: reservedSecret.id,
version: 1,
material: preparedSecret.material,
valueSha256: preparedSecret.valueSha256,
fingerprintSha256: preparedSecret.fingerprintSha256 ?? preparedSecret.valueSha256,
providerVersionRef: preparedSecret.providerVersionRef ?? null,
status: "disabled",
createdByAgentId: actor?.agentId ?? null,
createdByUserId: actor?.userId ?? null,
});
return await db.transaction(async (tx) => {
await tx
.update(companySecretVersions)
.set({ status: "current" })
.where(and(
eq(companySecretVersions.secretId, reservedSecret.id),
eq(companySecretVersions.version, 1),
));
const secret = await tx
.update(companySecrets)
.set({
status: "active",
externalRef: preparedSecret.externalRef,
latestVersion: 1,
lastRotatedAt: new Date(),
updatedAt: new Date(),
})
.where(eq(companySecrets.id, reservedSecret.id))
.returning()
.then((rows) => rows[0]);
if (!secret) throw notFound("Secret not found");
return secret;
});
} catch (error) {
if (prepared) {
await cleanupPreparedProviderWrite({
provider,
prepared,
providerConfig,
context: providerWriteContext,
mode: "delete",
operation: "adapter_config_secret.create_rollback",
}).catch(() => false);
}
await db.delete(companySecretVersions).where(eq(companySecretVersions.secretId, reservedSecret.id)).catch(() => undefined);
await db.delete(companySecrets).where(eq(companySecrets.id, reservedSecret.id)).catch(() => undefined);
throw error;
}
}
function collectTargetIds(
bindings: Array<typeof companySecretBindings.$inferSelect>,
targetType: SecretBindingTargetType,
@@ -1817,9 +2021,11 @@ export function secretService(db: Db) {
operation: "create.prepare_rollback",
});
if (cleaned) {
await db.delete(companySecretVersions).where(eq(companySecretVersions.secretId, reservedSecret.id)).catch(() => undefined);
await db.delete(companySecrets).where(eq(companySecrets.id, reservedSecret.id)).catch(() => undefined);
}
} else {
await db.delete(companySecretVersions).where(eq(companySecretVersions.secretId, reservedSecret.id)).catch(() => undefined);
await db.delete(companySecrets).where(eq(companySecrets.id, reservedSecret.id)).catch(() => undefined);
}
throw error;
@@ -1862,9 +2068,11 @@ export function secretService(db: Db) {
operation: "create.rollback",
});
if (cleaned) {
await db.delete(companySecretVersions).where(eq(companySecretVersions.secretId, reservedSecret.id)).catch(() => undefined);
await db.delete(companySecrets).where(eq(companySecrets.id, reservedSecret.id)).catch(() => undefined);
}
} else {
await db.delete(companySecretVersions).where(eq(companySecretVersions.secretId, reservedSecret.id)).catch(() => undefined);
await db.delete(companySecrets).where(eq(companySecrets.id, reservedSecret.id)).catch(() => undefined);
}
throw error;
@@ -2159,6 +2367,7 @@ export function secretService(db: Db) {
required?: boolean;
label?: string | null;
}>,
options?: { replaceAll?: boolean },
) => {
const normalizedRefs: Array<{
secretId: string;
@@ -2181,7 +2390,17 @@ export function secretService(db: Db) {
const pathPrefixes = [...new Set(normalizedRefs.map((ref) => ref.configPath.split(".")[0]))];
await db.transaction(async (tx) => {
if (pathPrefixes.length > 0) {
if (options?.replaceAll) {
await tx
.delete(companySecretBindings)
.where(
and(
eq(companySecretBindings.companyId, companyId),
eq(companySecretBindings.targetType, target.targetType),
eq(companySecretBindings.targetId, target.targetId),
),
);
} else if (pathPrefixes.length > 0) {
for (const pathPrefix of pathPrefixes) {
await tx
.delete(companySecretBindings)
@@ -2351,7 +2570,7 @@ export function secretService(db: Db) {
normalizeAdapterConfigForPersistence: async (
companyId: string,
adapterConfig: Record<string, unknown>,
opts?: { strictMode?: boolean },
opts?: NormalizeAdapterConfigOptions,
) => normalizeAdapterConfigForPersistenceInternal(companyId, adapterConfig, opts),
normalizeEnvBindingsForPersistence: async (
@@ -2363,7 +2582,7 @@ export function secretService(db: Db) {
normalizeHireApprovalPayloadForPersistence: async (
companyId: string,
payload: Record<string, unknown>,
opts?: { strictMode?: boolean },
opts?: NormalizeAdapterConfigOptions,
) => {
const normalized = { ...payload };
const adapterConfig = asRecord(payload.adapterConfig);
@@ -2473,52 +2692,121 @@ export function secretService(db: Db) {
}));
},
collectMissingAdapterConfigRuntimeBindings: async (
companyId: string,
adapterConfig: Record<string, unknown>,
adapterType: string | null | undefined,
context: Omit<SecretConsumerContext, "configPath">,
): Promise<MissingRuntimeBinding[]> => {
const secretFieldKeys = await listAdapterSchemaSecretFieldKeys(adapterType);
const secretRefs = secretFieldKeys.flatMap((key) => {
const parsed = envBindingSchema.safeParse(adapterConfig[key]);
if (!parsed.success) return [];
const binding = canonicalizeBinding(parsed.data as EnvBinding);
if (binding.type !== "secret_ref") return [];
return [{ key, configPath: key, secretId: binding.secretId }];
});
if (secretRefs.length === 0) return [];
const bindingChecks = await Promise.all(secretRefs.map(async (entry) => ({
entry,
found: await getBinding({
companyId,
secretId: entry.secretId,
consumerType: context.consumerType,
consumerId: context.consumerId,
configPath: entry.configPath,
}),
})));
const missingEntries = bindingChecks
.filter((check) => !check.found)
.map((check) => check.entry);
if (missingEntries.length === 0) return [];
const secretRows = await Promise.all(
[...new Set(missingEntries.map((entry) => entry.secretId))].map(async (secretId) => [
secretId,
await getById(secretId).catch(() => null),
] as const),
);
const secretsById = new Map(secretRows);
return missingEntries.map((entry) => ({
consumerType: context.consumerType,
consumerId: context.consumerId,
configPath: entry.configPath,
envKey: entry.key,
secretId: entry.secretId,
secretName: secretsById.get(entry.secretId)?.name ?? null,
}));
},
resolveAdapterConfigForRuntime: async (
companyId: string,
adapterConfig: Record<string, unknown>,
context?: Omit<SecretConsumerContext, "configPath">,
opts?: { adapterType?: string | null },
): Promise<{ config: Record<string, unknown>; secretKeys: Set<string>; manifest: RuntimeSecretManifestEntry[] }> => {
const resolved = { ...adapterConfig };
const secretKeys = new Set<string>();
const manifest: RuntimeSecretManifestEntry[] = [];
if (!Object.prototype.hasOwnProperty.call(adapterConfig, "env")) {
return { config: resolved, secretKeys, manifest };
}
const record = asRecord(adapterConfig.env);
if (!record) {
resolved.env = {};
return { config: resolved, secretKeys, manifest };
}
const env: Record<string, string> = {};
for (const [key, rawBinding] of Object.entries(record)) {
if (!ENV_KEY_RE.test(key)) {
throw unprocessable(`Invalid environment variable name: ${key}`);
}
const parsed = envBindingSchema.safeParse(rawBinding);
if (!parsed.success) {
throw unprocessable(`Invalid environment binding for key: ${key}`);
}
const binding = canonicalizeBinding(parsed.data as EnvBinding);
if (binding.type === "plain") {
env[key] = binding.value;
if (Object.prototype.hasOwnProperty.call(adapterConfig, "env")) {
const record = asRecord(adapterConfig.env);
if (!record) {
resolved.env = {};
} else {
const secretResolution = await resolveSecretValueInternal(
companyId,
binding.secretId,
binding.version,
context
? {
bindingContext: { ...context, configPath: `env.${key}` },
accessContext: { ...context, configPath: `env.${key}` },
}
: undefined,
);
env[key] = secretResolution.value;
manifest.push(secretResolution.manifestEntry);
secretKeys.add(key);
const env: Record<string, string> = {};
for (const [key, rawBinding] of Object.entries(record)) {
if (!ENV_KEY_RE.test(key)) {
throw unprocessable(`Invalid environment variable name: ${key}`);
}
const parsed = envBindingSchema.safeParse(rawBinding);
if (!parsed.success) {
throw unprocessable(`Invalid environment binding for key: ${key}`);
}
const binding = canonicalizeBinding(parsed.data as EnvBinding);
if (binding.type === "plain") {
env[key] = binding.value;
} else {
const secretResolution = await resolveSecretValueInternal(
companyId,
binding.secretId,
binding.version,
context
? {
bindingContext: { ...context, configPath: `env.${key}` },
accessContext: { ...context, configPath: `env.${key}` },
}
: undefined,
);
env[key] = secretResolution.value;
manifest.push(secretResolution.manifestEntry);
secretKeys.add(key);
}
}
resolved.env = env;
}
}
resolved.env = env;
const secretFieldKeys = await listAdapterSchemaSecretFieldKeys(opts?.adapterType);
for (const key of secretFieldKeys) {
const parsed = envBindingSchema.safeParse(adapterConfig[key]);
if (!parsed.success) continue;
const binding = canonicalizeBinding(parsed.data as EnvBinding);
if (binding.type === "plain") continue;
const secretResolution = await resolveSecretValueInternal(
companyId,
binding.secretId,
binding.version,
context
? {
bindingContext: { ...context, configPath: key },
accessContext: { ...context, configPath: key },
}
: undefined,
);
resolved[key] = secretResolution.value;
manifest.push(secretResolution.manifestEntry);
secretKeys.add(key);
}
return { config: resolved, secretKeys, manifest };
},
};