Files
PaperClipAI/server/src/services/environment-config.ts
T
DottaandPaperclip e34abee670 feat(mcp): connect assistants to a team with user OAuth (#14846)
## Thinking Path

> - Paperclip gives teams durable tasks, agent execution, budgets, and
approvals.
> - People also use assistants in Codex, Claude, and other MCP clients.
> - Those assistants need a scoped connection that preserves the
person’s permissions and attribution.
> - Delegating a task must not turn the assistant into the assigned
agent.
> - This PR adds opt-in user OAuth, ten first-party tools, browser
consent, and workflow packages.
> - Paid product evals verify the resulting tasks, documents,
attribution, retries, and access boundaries.
> - The team keeps working after the assistant conversation ends.

## Linked Issues or Issue Description

**Problem or motivation**

A person cannot connect an external assistant to an existing team
through browser consent and safely delegate durable work as themselves.

**Proposed solution**

Expose an opt-in `/mcp/paperclip` endpoint with individually described
first-party operations. Bind every connection to a person, client,
company, resource, and scopes. Reuse domain authorization and
scheduling. Package shared team-review, delegation, and follow-up
workflows for OpenAI/Codex and Claude.

**Alternatives considered**

Related PRs #9393 and #12549 cover earlier remote MCP and board-operator
approaches. This change uses user OAuth and a bounded public catalog. It
does not expose a generic executor, operator administration, static
shared board credentials, or external agent execution. Registry listing
work in #9851 is a separate distribution step.

**Roadmap alignment**

This maintainer-requested implementation extends the governed MCP
gateway, activity attribution, durable work products, and hosted
deployment direction in `ROADMAP.md`. It implements the first release of
the saved design plan; external agent participation and granted
third-party tools remain later releases.

## What Changed

- Add MCP 2.0 discovery and task status/comment/document Events on the
same authenticated endpoint. Persist subscriptions and delivery
receipts, verify HTTPS callbacks, sign Standard Webhooks, encrypt
callback material, recheck permissions/Cloud membership, and bound
retries/expiry. Older MCP clients keep their existing tools.
- Add discovery, dynamic client registration, S256 PKCE, resource
validation, rotating refresh tokens, revocation, and company consent.
Store credentials as hashes and recheck membership at execution.
- Add tools for connection identity, agents/projects, task
search/read/create, human comments, documents/deliverables, and
pending-approval links. Preserve current domain permissions and
scheduling.
- Add durable mutation receipts across reconnects. Matching retries
replay results; uncertain outcomes keep the same request ID and require
inspection.
- Add consent and connection-management pages, OAuth log redaction,
shared plugin workflows, and separate OpenAI/Codex and Claude package
outputs.
- Add eight paid Product E2E cases across three models, independent
durable-state grading, usage evidence, cleanup, and report integration.
Add task-document guidance and regenerate the runner capability
inventories.
- Add migrations 0301 and 0302, the dated implementation plan, result
notes, and direct-client setup instructions in `doc/public-mcp.md`.

## Verification

- Merge integration `e180b1948`: resolved conflicts with current master,
preserved both eval registries, regenerated capability catalogs, and
regenerated migrations as 0301/0302 while keeping the original
replay-safe SQL byte-identical. Local migration safety/snapshot tests
(26), MCP/OAuth tests (38), redaction/OpenAPI tests (71), and eval
catalog/grading tests (198) pass. Token and capability gates pass. Full
recursive typecheck passed. Fresh Greptile review is 5/5 with no
unresolved findings. CI is green on this exact head (55 successes, two
intentional skips, one neutral result): one unchanged Cursor sandbox
test timed out at 10 seconds, then passed locally in 856 ms. A single
retry of that failed shard and the aggregate workflow passed. Merge
remains blocked on the repository code-owner approval rule.

Earlier checks passed at `6aa0962d4fb715f2190bb7bb22efacab2e58495d`: 55
successes, two intentional skips and one neutral result. [The earlier CI
run](https://github.com/paperclipai/paperclip/actions/runs/36901592350)
includes all test shards, browser tests, typecheck, build and canary dry
run. Greptile was 5/5 on that commit with no unresolved review threads.
GitHub still requires code-owner review under the repository merge
rules; passing checks do not bypass that approval. Paid source
fingerprints remain separate below and in the dated result note.

- Paid Events qualification passes **3/3**: GPT-5.4 Mini, Claude Haiku
4.5 and Claude Sonnet 4.6. Each uses a real public HTTPS callback,
signature verification and report retrieval in a fresh conversation. A
final Mini regression passes after the quota/status fixes. All evidence
validates. Bounded tunnel startup retries occur before provider calls
and remain visible; failed earlier attempts retain their original
grades.
- The earlier complete seven-case matrix passes **21/21**, with a
separate **3/3** delegation regression. Two preceding matrices also
passed 21/21 each. A complete 24-cell matrix including Events has not
been run. [The dated
results](doc/plans/2026-10-01-public-mcp-paid-eval-results.md) retain
exact source fingerprints, failures, model IDs and partial costs.
- Node 24: repository-wide `pnpm -r typecheck` and `pnpm build` pass
after merging master. Server typecheck passes after the final
quota/status changes. Eval typecheck and all 892 eval-support tests
pass.
- All 33 real MCP/OAuth tests pass. The preceding combined MCP,
redaction, private-address and DNS-rebinding run passed 129 tests; two
later MCP regressions cover quota reuse and unchanged-status
suppression. All 28 adjacent issue-tree/stale-lock route tests pass. CI
then found a null checkout result in the existing concurrent-workspace
path; logging now uses optional status access. All 12 closed-workspace
tests and all 33 MCP tests pass after that correction. The exact-start
event calibration exposed a timestamp gap; scanning now includes the
subscription start, with all 33 MCP tests and server typecheck passing.
These two narrow corrections follow the paid regression.
- A real Core → Cloud → Core authority round trip passes OAuth, MCP 2.0
subscription/delivery, current membership loss, unsubscribe, legacy SDK
tools, refresh and revocation. Its callback transport is a fixture with
independent HMAC verification. The paid Events campaigns separately
prove public HTTPS delivery.
- Earlier component qualification passed UI 7,117 tests, CLI 502, shared
832, skills catalog 20, database 160 and OpenAPI 10. Token gates, module
boundaries, migration order and plugin regeneration passed. CI covers
general/serialized suites, eight browser shards, runner checks,
typecheck, build and canary dry run.
- **Local full-suite limitation:** the earlier monolithic run was not
clean. It encountered overlapping schema rebuilding, Mac database
shared-memory limits and isolated CLI/fixture failures. Targeted reruns
passed. The existing >32 MiB Git filename stress test still hit its
300-second Mac timeout. The additional serialized sweep stopped after 62
passing suites once CI passed. Original failures and partial logs
remain; this PR does not claim a wholly green local monolithic run.
- Local Codex CLI and Claude Code OAuth login and MCP SDK
interoperability were verified. Public-store installation, actual
ChatGPT Work Cloud Events UI, staging HTTPS client behavior and hosted
newcomer provisioning remain release gates.

Enablement is moving to **Settings → Experimental → Assistant
connections (MCP)** in the stacked follow-up
[#14933](https://github.com/paperclipai/paperclip/pull/14933). Merge
both for the intended setup experience. This foundation branch alone
still uses `PAPERCLIP_PUBLIC_MCP_ENABLED=true`. After deployment, set
`PAPERCLIP_PUBLIC_URL` to the authenticated instance's HTTPS origin, and
connect to `/mcp/paperclip`. Select a team and allow writes in browser
consent. Configure an available agent and budget, then delegate and
retrieve results later. For Events, rescan the deployed plugin catalog
in ChatGPT Work Cloud; the host supplies its webhook credentials when
the user asks to watch a task. See [the setup
runbook](doc/public-mcp.md).

## Risks

- Events are at-least-once and may arrive out of order. No replay cursor
is advertised. Clients must refresh finite subscriptions, read current
state and avoid comment feedback loops. Callback material uses the
instance secrets master key; hosted subscriptions require the updated
Cloud broker and are bounded to five minutes/the access proof expiry.
- ChatGPT Work Cloud/dot event UI, plugin rescan and a hosted staging
subscription remain deployment gates. Local signed-webhook and paid
model evidence does not claim those surfaces have been exercised.
- Disabled by default. Merging adds schema and opt-in code; it does not
deploy a public endpoint, publish a store listing, create a team, or
start paid agents.
- Migrations 0301 and 0302 are additive and idempotent. Their SQL is
unchanged from the earlier preview numbers, so hash-aware upgrade
reconciliation preserves prior staging applications. Normal instance
upgrades must apply it before enabling MCP.
- Task creation and comments can schedule paid agent work. Consent and
tool descriptions disclose that effect. Revocation blocks future calls
but does not undo delegated work.
- Public deployments need edge rate limits and credential-safe logging.
Internal dispatch is restricted to the closed catalog and carries a
request-local verified actor.
- Hosted onboarding requires the companion Cloud broker, encryption-key
configuration, and tenant rollout. Self-hosted direct connections can
use this PR alone.
- Store acceptance and agent-mode participation are not claimed.
Checked-in plugin endpoints are development defaults; rebuild packages
for a real deployment before installation.

## Model Used

OpenAI GPT-6 in Codex, with reasoning, tool use, and code execution. A
more specific serving version and context-window size were not exposed
by the session. Paid eval models: `gpt-5.4-mini-2026-03-17`,
`claude-haiku-4-5-20251001`, and `claude-sonnet-4-6`.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (targeted/component checks;
full local-run limitations are recorded above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-06 11:48:53 -05:00

827 lines
29 KiB
TypeScript

import { z } from "zod";
import { randomUUID } from "node:crypto";
import type { Db } from "@paperclipai/db";
import type {
Environment,
EnvironmentDriver,
FakeSandboxEnvironmentConfig,
LocalEnvironmentConfig,
PluginEnvironmentConfig,
PluginSandboxEnvironmentConfig,
SandboxEnvironmentConfig,
SecretProvider,
SecretVersionSelector,
SshEnvironmentConfig,
} from "@paperclipai/shared";
import { unprocessable } from "../errors.js";
import { parseObject } from "../adapters/utils.js";
import { secretService } from "./secrets.js";
import {
resolvePluginSandboxProviderDriverByKey,
validatePluginEnvironmentDriverConfig,
validatePluginSandboxProviderConfig,
} from "./plugin-environment-driver.js";
import type { PluginWorkerManager } from "./plugin-worker-manager.js";
import {
collectSecretRefPaths,
isUuidSecretRef,
parseSecretRefBindingObject,
readConfigValueAtPath,
writeConfigValueAtPath,
} from "./json-schema-secret-refs.js";
import { resolveActiveEnvironmentCustomImageTemplateForRuntime } from "./environment-custom-image-runtime.js";
const secretRefSchema = z.object({
type: z.literal("secret_ref"),
secretId: z.string().guid(),
version: z.union([z.literal("latest"), z.number().int().positive()]).optional().default("latest"),
}).strict();
const sshEnvironmentConfigSchema = z.object({
host: z.string({ error: "SSH environments require a host." }).trim().min(1, "SSH environments require a host."),
port: z.coerce.number().int().min(1).max(65535).default(22),
username: z.string({ error: "SSH environments require a username." }).trim().min(1, "SSH environments require a username."),
remoteWorkspacePath: z
.string({ error: "SSH environments require a remote workspace path." })
.trim()
.min(1, "SSH environments require a remote workspace path.")
.refine((value) => value.startsWith("/"), "SSH remote workspace path must be absolute."),
privateKey: z.null().optional().default(null),
privateKeySecretRef: secretRefSchema.optional().nullable().default(null),
knownHosts: z
.string()
.trim()
.optional()
.nullable()
.transform((value) => (value && value.length > 0 ? value : null)),
strictHostKeyChecking: z.boolean().optional().default(true),
}).strict();
const sshEnvironmentConfigProbeSchema = sshEnvironmentConfigSchema.extend({
privateKey: z
.string()
.trim()
.optional()
.nullable()
.transform((value) => (value && value.length > 0 ? value : null)),
}).strict();
const sshEnvironmentConfigPersistenceSchema = sshEnvironmentConfigProbeSchema;
const fakeSandboxEnvironmentConfigSchema = z.object({
provider: z.literal("fake").default("fake"),
image: z
.string()
.trim()
.min(1, "Fake sandbox environments require an image.")
.default("ubuntu:24.04"),
reuseLease: z.boolean().optional().default(false),
streamRunLogs: z.boolean().optional(),
runnerLifecycleMode: z.enum(["inherit", "per_turn", "warm"]).optional(),
runnerIdleTimeoutMs: z.coerce
.number()
.int()
.min(1_000)
.max(86_400_000)
.optional(),
archiveOnRelease: z.boolean().optional(),
}).strict();
const pluginSandboxProviderKeySchema = z.string()
.trim()
.min(1, "Sandbox provider is required.")
.regex(
/^[a-z0-9][a-z0-9._-]*$/,
"Sandbox provider key must start with a lowercase alphanumeric and contain only lowercase letters, digits, dots, hyphens, or underscores",
);
const pluginSandboxEnvironmentConfigSchema = z.object({
provider: pluginSandboxProviderKeySchema,
timeoutMs: z.coerce.number().int().min(1).max(86_400_000).optional(),
reuseLease: z.boolean().optional().default(false),
streamRunLogs: z.boolean().optional(),
runnerLifecycleMode: z.enum(["inherit", "per_turn", "warm"]).optional(),
runnerIdleTimeoutMs: z.coerce
.number()
.int()
.min(1_000)
.max(86_400_000)
.optional(),
archiveOnRelease: z.boolean().optional(),
}).catchall(z.unknown());
const pluginEnvironmentConfigSchema = z.object({
pluginKey: z.string().min(1),
driverKey: z.string().min(1).regex(
/^[a-z0-9][a-z0-9._-]*$/,
"Environment driver key must start with a lowercase alphanumeric and contain only lowercase letters, digits, dots, hyphens, or underscores",
),
driverConfig: z.record(z.string(), z.unknown()).optional().default({}),
}).strict();
export type ParsedEnvironmentConfig =
| { driver: "local"; config: LocalEnvironmentConfig }
| { driver: "ssh"; config: SshEnvironmentConfig }
| { driver: "sandbox"; config: SandboxEnvironmentConfig }
| { driver: "plugin"; config: PluginEnvironmentConfig };
function toErrorMessage(error: z.ZodError) {
const first = error.issues[0];
if (!first) return "Invalid environment config.";
return first.message;
}
function getSandboxProvider(raw: Record<string, unknown>) {
return typeof raw.provider === "string" && raw.provider.trim().length > 0 ? raw.provider.trim() : "fake";
}
// Operator flags removed when session-output streaming moved to the verified
// capability snapshot. A saved environment config can still carry a removed key.
// The server now decides session-output streaming from the effective capability
// snapshot alone, so no consumer reads these flags. Strip a removed key before
// validation so a strict schema (the fake sandbox) still loads an old config,
// and so the removed flag never reaches the parsed config.
const REMOVED_SANDBOX_CONFIG_KEYS = ["streamAgentSessionOutput"] as const;
function stripRemovedSandboxConfigKeys(raw: Record<string, unknown>): Record<string, unknown> {
if (!REMOVED_SANDBOX_CONFIG_KEYS.some((key) => key in raw)) {
return raw;
}
const next = { ...raw };
for (const key of REMOVED_SANDBOX_CONFIG_KEYS) {
delete next[key];
}
return next;
}
function parseSandboxEnvironmentConfig(
input: Record<string, unknown> | null | undefined,
) {
const raw = stripRemovedSandboxConfigKeys(parseObject(input));
const provider = getSandboxProvider(raw);
if (provider === "fake") {
const parsed = fakeSandboxEnvironmentConfigSchema.safeParse(raw);
return parsed.success
? ({ success: true as const, data: parsed.data satisfies FakeSandboxEnvironmentConfig })
: ({ success: false as const, error: parsed.error });
}
const parsed = pluginSandboxEnvironmentConfigSchema.safeParse(raw);
return parsed.success
? ({ success: true as const, data: parsed.data satisfies PluginSandboxEnvironmentConfig })
: ({ success: false as const, error: parsed.error });
}
async function getSandboxProviderConfigSchema(
db: Db,
provider: string,
): Promise<Record<string, unknown> | null> {
const resolved = await resolvePluginSandboxProviderDriverByKey({
db,
driverKey: provider,
});
const schema = resolved?.driver.configSchema;
return schema && typeof schema === "object" && !Array.isArray(schema)
? schema as Record<string, unknown>
: null;
}
export async function resolveSandboxProviderSecretRefPaths(
db: Db,
provider: string,
): Promise<Set<string>> {
return collectSecretRefPaths(await getSandboxProviderConfigSchema(db, provider));
}
function secretName(input: {
environmentName: string;
driver: EnvironmentDriver;
field: string;
}) {
const slug = input.environmentName
.toLowerCase()
.replace(/[^a-z0-9]+/g, "-")
.replace(/^-+|-+$/g, "")
.slice(0, 48) || "environment";
return `environment-${input.driver}-${slug}-${input.field}-${randomUUID().slice(0, 8)}`;
}
async function createEnvironmentSecret(input: {
db: Db;
companyId: string;
environmentName: string;
driver: EnvironmentDriver;
field: string;
provider: SecretProvider;
value: string;
actor?: { userId?: string | null; agentId?: string | null };
}) {
const created = await secretService(input.db).create(
input.companyId,
{
name: secretName(input),
provider: input.provider,
value: input.value,
description: `Secret for ${input.environmentName} ${input.field}.`,
},
input.actor,
);
return {
type: "secret_ref" as const,
secretId: created.id,
version: "latest" as const,
};
}
/**
* Secret pickers submit `{ type: "secret_ref", secretId, version }` binding
* objects for `format: "secret-ref"` fields, while persisted configs store the
* bare secret id. Collapse binding objects to the secret id so every consumer
* downstream deals with one shape. Sandbox provider references always resolve
* the latest version, so pinned bindings are rejected rather than silently
* resolved to a different version than the caller asked for.
*/
function canonicalizeSecretRefValue(value: unknown, path: string): unknown {
const binding = parseSecretRefBindingObject(value);
if (!binding) return value;
if (binding.version !== "latest") {
throw unprocessable(
`Secret binding at ${path} pins version ${binding.version}; sandbox provider secret references always resolve the latest version.`,
);
}
return binding.secretId;
}
async function persistConfigSecretRefs(input: {
db: Db;
companyId: string;
environmentName: string;
driver: EnvironmentDriver;
secretProvider: SecretProvider;
config: Record<string, unknown>;
schema: Record<string, unknown> | null;
actor?: { userId?: string | null; agentId?: string | null };
}): Promise<Record<string, unknown>> {
let nextConfig = { ...input.config };
for (const path of collectSecretRefPaths(input.schema)) {
const rawValue = canonicalizeSecretRefValue(readConfigValueAtPath(nextConfig, path), path);
if (typeof rawValue !== "string") continue;
const trimmed = rawValue.trim();
if (trimmed.length === 0) {
nextConfig = writeConfigValueAtPath(nextConfig, path, undefined);
continue;
}
if (isUuidSecretRef(trimmed)) {
nextConfig = writeConfigValueAtPath(nextConfig, path, trimmed);
continue;
}
const created = await createEnvironmentSecret({
db: input.db,
companyId: input.companyId,
environmentName: input.environmentName,
driver: input.driver,
field: path.replace(/[^a-z0-9]+/gi, "-").toLowerCase(),
provider: input.secretProvider,
value: trimmed,
actor: input.actor,
});
nextConfig = writeConfigValueAtPath(nextConfig, path, created.secretId);
}
return nextConfig;
}
async function resolveConfigSecretRefsForRuntime(input: {
db: Db;
companyId: string;
config: Record<string, unknown>;
schema: Record<string, unknown> | null;
context: {
consumerId: string;
issueId?: string | null;
heartbeatRunId?: string | null;
};
}): Promise<Record<string, unknown>> {
const secrets = secretService(input.db);
let nextConfig = { ...input.config };
for (const path of collectSecretRefPaths(input.schema)) {
const current = canonicalizeSecretRefValue(readConfigValueAtPath(nextConfig, path), path);
if (typeof current !== "string") continue;
const trimmed = current.trim();
if (!isUuidSecretRef(trimmed)) continue;
if (!input.context.consumerId) {
throw unprocessable("Runtime secret resolution requires an environment id");
}
nextConfig = writeConfigValueAtPath(
nextConfig,
path,
await secrets.resolveSecretValue(input.companyId, trimmed, "latest", {
consumerType: "environment",
consumerId: input.context.consumerId,
actorType: "system",
actorId: null,
issueId: input.context.issueId ?? null,
heartbeatRunId: input.context.heartbeatRunId ?? null,
configPath: path,
}),
);
}
return nextConfig;
}
async function resolveConfigSecretRefsForProbe(input: {
db: Db;
companyId: string;
config: Record<string, unknown>;
schema: Record<string, unknown> | null;
accessContext?: {
actorType: "agent" | "user";
actorId: string;
actorSource?: "local_implicit" | "session" | "board_key" | "agent_key" | "agent_jwt" | "mcp_oauth" | "cloud_tenant";
heartbeatRunId?: string | null;
};
}): Promise<Record<string, unknown>> {
const secrets = secretService(input.db);
let nextConfig = { ...input.config };
for (const path of collectSecretRefPaths(input.schema)) {
const current = canonicalizeSecretRefValue(readConfigValueAtPath(nextConfig, path), path);
if (typeof current !== "string") continue;
const trimmed = current.trim();
if (!isUuidSecretRef(trimmed)) continue;
// Unsaved draft probes do not have an environment record yet, so they
// cannot rely on environment-bound secret resolution. Resolve directly for
// this ephemeral board-only probe and never persist the plaintext value.
nextConfig = writeConfigValueAtPath(
nextConfig,
path,
await secrets.resolveSecretValueForEphemeralAccess(input.companyId, trimmed, "latest", {
consumerType: "system",
consumerId: "environment-probe-config",
configPath: path,
actorType: input.accessContext?.actorType ?? "system",
actorId: input.accessContext?.actorId ?? null,
actorSource: input.accessContext?.actorSource,
heartbeatRunId: input.accessContext?.heartbeatRunId ?? null,
}),
);
}
return nextConfig;
}
export async function collectEnvironmentSecretRefs(input: {
db: Db;
environment: Pick<Environment, "id" | "driver" | "config">;
}): Promise<Array<{ secretId: string; configPath: string; versionSelector?: SecretVersionSelector }>> {
const parsed = parseEnvironmentDriverConfig(input.environment);
if (parsed.driver === "ssh" && parsed.config.privateKeySecretRef) {
return [{
secretId: parsed.config.privateKeySecretRef.secretId,
configPath: "privateKeySecretRef",
versionSelector: parsed.config.privateKeySecretRef.version ?? "latest",
}];
}
if (parsed.driver === "sandbox" && parsed.config.provider !== "fake") {
const schema = await getSandboxProviderConfigSchema(input.db, parsed.config.provider);
const refs: Array<{ secretId: string; configPath: string; versionSelector?: SecretVersionSelector }> = [];
for (const path of collectSecretRefPaths(schema)) {
const current = readConfigValueAtPath(parsed.config as Record<string, unknown>, path);
const binding = parseSecretRefBindingObject(current);
if (binding) {
refs.push({ secretId: binding.secretId, configPath: path, versionSelector: binding.version });
continue;
}
if (typeof current === "string" && isUuidSecretRef(current.trim())) {
refs.push({ secretId: current.trim(), configPath: path, versionSelector: "latest" });
}
}
return refs;
}
return [];
}
export function stripSandboxProviderEnvelope(config: SandboxEnvironmentConfig): Record<string, unknown> {
const {
provider: _provider,
streamRunLogs: _streamRunLogs,
...driverConfig
} = config as Record<string, unknown>;
return driverConfig;
}
// The host owns these sandbox run-behavior flags, not the provider plugin. The
// host passes the whole config to the plugin, so a plugin that allowlists only
// its provider fields may drop them from its normalized config. Re-apply them
// from the parsed envelope after plugin normalization. Otherwise a saved
// environment can silently fall back to per-turn runner behavior even though
// the caller explicitly selected a warm lifecycle.
const HOST_OWNED_SANDBOX_FLAGS = [
"streamRunLogs",
"runnerLifecycleMode",
"runnerIdleTimeoutMs",
] as const;
function applyHostOwnedSandboxFlags(
normalizedConfig: Record<string, unknown>,
envelope: Record<string, unknown>,
): Record<string, unknown> {
const merged: Record<string, unknown> = { ...normalizedConfig };
for (const key of HOST_OWNED_SANDBOX_FLAGS) {
if (envelope[key] !== undefined) {
merged[key] = envelope[key];
}
}
return merged;
}
export function normalizeEnvironmentConfig(input: {
driver: EnvironmentDriver;
config: Record<string, unknown> | null | undefined;
}): Record<string, unknown> {
if (input.driver === "local") {
return { ...parseObject(input.config) };
}
if (input.driver === "ssh") {
const parsed = sshEnvironmentConfigSchema.safeParse(parseObject(input.config));
if (!parsed.success) {
throw unprocessable(toErrorMessage(parsed.error), {
issues: parsed.error.issues,
});
}
return parsed.data satisfies SshEnvironmentConfig;
}
if (input.driver === "sandbox") {
const parsed = parseSandboxEnvironmentConfig(input.config);
if (!parsed.success) {
throw unprocessable(toErrorMessage(parsed.error), {
issues: parsed.error.issues,
});
}
return parsed.data;
}
if (input.driver === "plugin") {
const parsed = pluginEnvironmentConfigSchema.safeParse(parseObject(input.config));
if (!parsed.success) {
throw unprocessable(toErrorMessage(parsed.error), {
issues: parsed.error.issues,
});
}
return parsed.data satisfies PluginEnvironmentConfig;
}
throw unprocessable(`Unsupported environment driver "${input.driver}".`);
}
export function normalizeEnvironmentConfigForProbe(input: {
db: Db;
companyId: string;
driver: EnvironmentDriver;
config: Record<string, unknown> | null | undefined;
accessContext?: {
actorType: "agent" | "user";
actorId: string;
actorSource?: "local_implicit" | "session" | "board_key" | "agent_key" | "agent_jwt" | "mcp_oauth" | "cloud_tenant";
heartbeatRunId?: string | null;
};
pluginWorkerManager?: PluginWorkerManager;
}): Promise<Record<string, unknown>> | Record<string, unknown> {
if (input.driver === "ssh") {
const parsed = sshEnvironmentConfigProbeSchema.safeParse(parseObject(input.config));
if (!parsed.success) {
throw unprocessable(toErrorMessage(parsed.error), {
issues: parsed.error.issues,
});
}
return parsed.data satisfies SshEnvironmentConfig;
}
if (input.driver === "sandbox") {
const parsed = parseSandboxEnvironmentConfig(input.config);
if (!parsed.success) {
throw unprocessable(toErrorMessage(parsed.error), {
issues: parsed.error.issues,
});
}
if (parsed.data.provider === "fake") {
return parsed.data;
}
if (!input.pluginWorkerManager) {
throw unprocessable("Sandbox provider config validation requires a running plugin worker manager.");
}
return validatePluginSandboxProviderConfig({
db: input.db,
workerManager: input.pluginWorkerManager,
provider: parsed.data.provider,
config: stripSandboxProviderEnvelope(parsed.data),
}).then(async (validated) => ({
provider: parsed.data.provider,
...(await resolveConfigSecretRefsForProbe({
db: input.db,
companyId: input.companyId,
config: applyHostOwnedSandboxFlags(
validated.normalizedConfig,
parsed.data,
),
accessContext: input.accessContext,
schema:
validated.driver.configSchema &&
typeof validated.driver.configSchema === "object" &&
!Array.isArray(validated.driver.configSchema)
? validated.driver.configSchema as Record<string, unknown>
: null,
})),
}));
}
return normalizeEnvironmentConfig({
driver: input.driver,
config: input.config,
});
}
export async function normalizeEnvironmentConfigForPersistence(input: {
db: Db;
companyId: string;
environmentName: string;
driver: EnvironmentDriver;
secretProvider: SecretProvider;
config: Record<string, unknown> | null | undefined;
actor?: { userId?: string | null; agentId?: string | null };
pluginWorkerManager?: PluginWorkerManager;
}): Promise<Record<string, unknown>> {
if (input.driver === "ssh") {
const parsed = sshEnvironmentConfigPersistenceSchema.safeParse(parseObject(input.config));
if (!parsed.success) {
throw unprocessable(toErrorMessage(parsed.error), {
issues: parsed.error.issues,
});
}
const secrets = secretService(input.db);
const { privateKey, ...stored } = parsed.data;
let nextPrivateKeySecretRef = stored.privateKeySecretRef;
if (privateKey) {
nextPrivateKeySecretRef = await createEnvironmentSecret({
db: input.db,
companyId: input.companyId,
environmentName: input.environmentName,
driver: input.driver,
field: "private-key",
provider: input.secretProvider,
value: privateKey,
actor: input.actor,
});
if (
stored.privateKeySecretRef &&
stored.privateKeySecretRef.secretId !== nextPrivateKeySecretRef.secretId
) {
await secrets.remove(stored.privateKeySecretRef.secretId);
}
}
return {
...stored,
privateKey: null,
privateKeySecretRef: nextPrivateKeySecretRef,
} satisfies SshEnvironmentConfig;
}
if (input.driver === "sandbox") {
const parsed = parseSandboxEnvironmentConfig(input.config);
if (!parsed.success) {
throw unprocessable(toErrorMessage(parsed.error), {
issues: parsed.error.issues,
});
}
if (parsed.data.provider === "fake") {
throw unprocessable(
"Built-in fake sandbox environments are reserved for internal probes and cannot be saved.",
);
}
if (!input.pluginWorkerManager) {
throw unprocessable("Sandbox provider config validation requires a running plugin worker manager.");
}
const validated = await validatePluginSandboxProviderConfig({
db: input.db,
workerManager: input.pluginWorkerManager,
provider: parsed.data.provider,
config: stripSandboxProviderEnvelope(parsed.data),
});
return await persistConfigSecretRefs({
db: input.db,
companyId: input.companyId,
environmentName: input.environmentName,
driver: input.driver,
secretProvider: input.secretProvider,
config: {
provider: parsed.data.provider,
...applyHostOwnedSandboxFlags(validated.normalizedConfig, parsed.data),
},
schema:
validated.driver.configSchema && typeof validated.driver.configSchema === "object" && !Array.isArray(validated.driver.configSchema)
? validated.driver.configSchema as Record<string, unknown>
: null,
actor: input.actor,
});
}
if (input.driver === "plugin") {
const parsed = pluginEnvironmentConfigSchema.safeParse(parseObject(input.config));
if (!parsed.success) {
throw unprocessable(toErrorMessage(parsed.error), {
issues: parsed.error.issues,
});
}
if (!input.pluginWorkerManager) {
throw unprocessable("Plugin environment config validation requires a running plugin worker manager.");
}
return { ...(await validatePluginEnvironmentDriverConfig({
db: input.db,
workerManager: input.pluginWorkerManager,
config: parsed.data,
})) };
}
return normalizeEnvironmentConfig({
driver: input.driver,
config: input.config,
});
}
export async function resolveEnvironmentDriverConfigForRuntime(
db: Db,
companyId: string | null,
environment: Pick<Environment, "driver" | "config"> & Partial<Pick<Environment, "id">>,
context?: {
issueId?: string | null;
heartbeatRunId?: string | null;
// Force applying the active custom-image template even without a run/issue
// context. Operator-initiated `Test` probes have no issueId/heartbeatRunId
// but must still resolve the active custom image as prepared runtime
// configuration and tooling so the test reflects what real agent runs use.
applyCustomImageTemplate?: boolean;
},
): Promise<ParsedEnvironmentConfig> {
const parsed = parseEnvironmentDriverConfig(environment);
const secrets = secretService(db);
const environmentId = environment.id;
if (parsed.driver === "ssh" && parsed.config.privateKeySecretRef && !environmentId) {
throw unprocessable("Runtime secret resolution requires an environment id");
}
if (parsed.driver === "ssh" && parsed.config.privateKeySecretRef) {
if (!companyId) {
throw unprocessable("Runtime secret resolution requires a companyId context");
}
return {
driver: "ssh",
config: {
...parsed.config,
privateKey: await secrets.resolveSecretValue(
companyId,
parsed.config.privateKeySecretRef.secretId,
parsed.config.privateKeySecretRef.version ?? "latest",
{
consumerType: "environment",
consumerId: environmentId!,
actorType: "system",
actorId: null,
issueId: context?.issueId ?? null,
heartbeatRunId: context?.heartbeatRunId ?? null,
configPath: "privateKeySecretRef",
},
),
},
};
}
if (parsed.driver === "sandbox" && parsed.config.provider !== "fake") {
const schema = await getSandboxProviderConfigSchema(db, parsed.config.provider);
let runtimeConfig = parsed.config;
if (companyId) {
runtimeConfig = await resolveConfigSecretRefsForRuntime({
db,
companyId,
config: parsed.config as Record<string, unknown>,
schema,
context: {
consumerId: environmentId!,
issueId: context?.issueId ?? null,
heartbeatRunId: context?.heartbeatRunId ?? null,
},
}) as SandboxEnvironmentConfig;
} else {
for (const path of collectSecretRefPaths(schema)) {
const current = readConfigValueAtPath(parsed.config as Record<string, unknown>, path);
if (parseSecretRefBindingObject(current) || (typeof current === "string" && isUuidSecretRef(current.trim()))) {
throw unprocessable("Runtime secret resolution requires a companyId context");
}
}
}
return {
driver: "sandbox",
config: environmentId && (context?.issueId || context?.heartbeatRunId || context?.applyCustomImageTemplate)
? await resolveActiveEnvironmentCustomImageTemplateForRuntime(db, {
environmentId,
baseConfig: parsed.config,
runtimeConfig,
// Match the capture-time fingerprint exclusions: secret-ref paths
// are excluded when the template's source fingerprint is computed,
// so they must be excluded when re-checking it here.
secretRefExcludePaths: collectSecretRefPaths(schema),
})
: runtimeConfig,
};
}
return parsed;
}
/**
* Resolve the connection secrets of a recorded sandbox config for a durable
* orphan-sandbox teardown. The retry reads the recorded config from the durable
* `pending_cleanup` lease row, not from the current environment. So this
* resolver must not require the environment binding: a delete removed the
* environment, or a provider change replaced the binding. It resolves each
* schema-declared secret ref by id at the latest version through
* `resolveSecretValueForSandboxCleanup`, which authorizes the read from the
* durable orphan record and skips the binding check. The resolved values are
* used once for the teardown RPC and never persisted.
*/
export async function resolveSandboxCleanupConfigSecrets(
db: Db,
companyId: string,
config: SandboxEnvironmentConfig,
context?: { issueId?: string | null; heartbeatRunId?: string | null },
): Promise<SandboxEnvironmentConfig> {
if (config.provider === "fake") return config;
const schema = await getSandboxProviderConfigSchema(db, config.provider);
const secrets = secretService(db);
let nextConfig = { ...config } as Record<string, unknown>;
for (const path of collectSecretRefPaths(schema)) {
const current = canonicalizeSecretRefValue(readConfigValueAtPath(nextConfig, path), path);
if (typeof current !== "string") continue;
const trimmed = current.trim();
if (!isUuidSecretRef(trimmed)) continue;
nextConfig = writeConfigValueAtPath(
nextConfig,
path,
await secrets.resolveSecretValueForSandboxCleanup(companyId, trimmed, "latest", {
configPath: path,
issueId: context?.issueId ?? null,
heartbeatRunId: context?.heartbeatRunId ?? null,
}),
);
}
return nextConfig as SandboxEnvironmentConfig;
}
export function readSshEnvironmentPrivateKeySecretId(
environment: Pick<Environment, "driver" | "config">,
): string | null {
if (environment.driver !== "ssh") return null;
const parsed = sshEnvironmentConfigSchema.safeParse(parseObject(environment.config));
if (!parsed.success) return null;
return parsed.data.privateKeySecretRef?.secretId ?? null;
}
export function parseEnvironmentDriverConfig(
environment: Pick<Environment, "driver" | "config">,
): ParsedEnvironmentConfig {
if (environment.driver === "local") {
return {
driver: "local",
config: { ...parseObject(environment.config) },
};
}
if (environment.driver === "ssh") {
const parsed = sshEnvironmentConfigSchema.parse(parseObject(environment.config));
return {
driver: "ssh",
config: parsed,
};
}
if (environment.driver === "sandbox") {
const parsed = parseSandboxEnvironmentConfig(environment.config);
if (!parsed.success) {
throw parsed.error;
}
return {
driver: "sandbox",
config: parsed.data,
};
}
if (environment.driver === "plugin") {
const parsed = pluginEnvironmentConfigSchema.parse(parseObject(environment.config));
return {
driver: "plugin",
config: parsed,
};
}
throw new Error(`Unsupported environment driver "${environment.driver}".`);
}