mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-20 09:50:48 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Plugins can now add connector types (#22004) and serve their tools with the `plugin` transport (#22019) > - A plugin connector method can declare `grantKinds: ["agent"]` with `auth: "none"` or `"api_key"` > - Such a connector cannot reach its agent: the connect form cannot name the agent, connect creates no agent grant, and Paperclip Runner runs never list plugin-transport tools > - The product rule is that a plugin connector behaves exactly like any existing connector > - This pull request fixes the connect form, creates the agent grant at connect, and lets runner runs list plugin-transport tools > - The benefit is that an operator can activate a per-agent plugin connector, and only the chosen agent gets its tools ## Linked Issues or Issue Description Refs #22004 Refs #22019 **Subsystem affected** Connectors (connect wizard, tool access service), Paperclip Runner assigned-tool gateway. **Problem or motivation** A plugin connector with agent grants and no sign-in cannot be activated: 1. Connect form: it says "No sign-in needed, available to all agents". "Change" shows "No identity required" and "Which agent owns this GitHub account?", with no agent picker. Connect sends `grantKind: "agent"` without `subjectAgentId`, and the server rejects it with a validation error. The "Finish setup" resume form has the same problem. 2. Grant: connect with `subjectAgentId` saves a `per_agent` connection with zero grants. Finish adds the agent install, but there is still no grant. Only the OAuth callback writes agent grants, and these methods have no OAuth step. For `api_key` methods connect fails, because the health check finds no agent grant, and the key has nowhere to go. 3. Runner: a Paperclip Runner run builds its assigned-tool gateway from an allowlist of transports (`mcp_remote`, `local_stdio`, REST, Browser Use, GitHub bot, voice). `plugin` is not in it, so the run sees no plugin-transport tools, even when the install, profile and grant allow them. **Proposed solution** Create the agent grant at connect for curated methods without sign-in. Add `plugin` to the runner assignment and its digest. Show a one-agent picker with connector-neutral copy. **Expected behavior** Connecting alone grants no tools. Finish with agent A gives only agent A the tools, in gateway discovery and in a runner run. The connection has exactly one active agent grant, for A. ## What Changed - `server/src/services/tool-access.ts`: `connectGalleryApp` writes an active agent grant for `subjectAgentId` when a curated method with `auth` `none` or `api_key` uses `grantKind: "agent"`. The API key goes on that grant, not on the connection. A resume without `subjectAgentId` keeps the agent of the existing grant and keeps its saved key. A second agent on the same connection is a 409 conflict. The grant write is audited and joins the existing revive rollback. - Finish does not create agent grants. Finish carries no credential, and it can list many agents while a per-agent connection has one identity. A credential-less grant would also claim a connected OAuth identity that does not exist. The grant comes from connect (or the OAuth callback). Finish creates only the install. - `server/src/services/heartbeat/run-preparation.ts` and `native-runtime/runtime-context.ts`: include the `plugin` transport in the runner assignment and in its digest (the two lists must match). - `ui/src/features/connections/ConnectionSetupFlow.tsx`: for an agent grant, show the agent picker (exactly one agent), keep the install choice on "specific", open the "Change" panel and disable Connect until one agent is chosen, and send `subjectAgentId`. Copy is neutral ("Which agent uses this connection?"); the GitHub copy stays for GitHub. The summary no longer says "available to all agents" for an agent grant. Finish sends the chosen agent, never `all_agents`, for an agent grant. - `doc/connections/CONNECTOR-PLAYBOOK.md`: document agent grants for methods without sign-in. ## Verification Each new test failed before its fix and passes after it: - `server/src/__tests__/plugin-connector-agent-grant.test.ts` (embedded Postgres, real routes and gateway): - `auth: "none"` + agent grants: after connect, no agent has tools; after finish with agent A, A's discovery lists the tools and B's lists none; exactly one active agent grant, for A. (Red: zero grants.) - `auth: "api_key"` + agent grants: same, the key is on A's grant, and a tool call receives it. (Red: connect returned 422.) - A second agent on the same connection is a 409. (Red: 201.) - Resume of an API-key draft without the key keeps the key on the grant. - Paperclip Runner session: `buildPaperclipRuntimeMcpServers` + `createAssignedMcpTools` list the installed plugin tools for A, and search by the manifest tool name (`update_record`) finds the gateway tool. B gets none. (Red: no tools.) - Regression: an organization-grant method keeps one default organization grant, and both agents get the tools with `all_agents`. - `ui/src/pages/apps/AppsConnect.test.tsx`: for `none` and `api_key` agent-grant plugin methods, the form shows the picker, has no GitHub or "available to all agents" copy, blocks Connect until an agent is chosen, and sends `subjectAgentId`. (Red: "available to all agents" shown.) - Regression suites: `plugin-connectors-plugin-transport`, `plugin-connectors-circleback-parity`, `heartbeat-runtime-mcp-servers`, `run-preparation`, `tool-access-service`, `tool-access-oauth-protocol.integration`, `generic-mcp-connection` (OAuth, GitHub per-agent and organization flows) pass. UI: `src/features/connections`, `src/pages/apps`, `App.deep-link-routing` pass (66 files, 1103 tests). - `tsc --noEmit` passes for `server` and `ui` source. `pnpm check:module-boundaries` passes. ## Risks - Low. The grant change applies only to curated methods with agent grants and no OAuth. Organization, personal and OAuth flows keep their code paths. - The runner change makes plugin-transport tools visible to runner runs. Access still needs an install, a profile and a grant, and every call is re-authorized by the gateway. - Connections created before this fix have no agent grant. Connecting again (or "Finish setup") with the agent creates it. - No migration. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
417 lines
26 KiB
TypeScript
417 lines
26 KiB
TypeScript
import { skillExecutionContent } from "../skill-execution-content.js";
|
||
import { withRuntimeAssetUse, removeRuntimeAssetTree } from "./runtime-asset-retention.js";
|
||
import { cachedSkillAsset, rememberSkillAsset } from "./skill-asset-cache.js";
|
||
import { verifyFileManifest, type FileManifestEntry } from "../file-manifest-cache.js";
|
||
import { measureRunPhase } from "../execution-preparation-timing.js";
|
||
import { spekoToolsForSession } from "../voice/speko-agent-tools.js";
|
||
import { githubBotConnectionIdsForRun } from "../chat-github-tools.js";
|
||
import { boundRunIssueIdSql } from "../run-task-binding.js";
|
||
import { isBrowserUseConnection } from "../browser-use-client.js";
|
||
import { restConnectorFor } from "../connectors/rest-registry.js";
|
||
import { createHash, randomUUID } from "node:crypto";
|
||
import fs from "node:fs/promises";
|
||
import path from "node:path";
|
||
import { and, desc, eq } from "drizzle-orm";
|
||
import { agentInstructionRevisions, agents, companySkillVersions, heartbeatRuns, type Db } from "@paperclipai/db";
|
||
import type { PaperclipSkillEntry } from "@paperclipai/adapter-utils/server-utils";
|
||
import { isToolConnectionAttentionHealth } from "@paperclipai/shared";
|
||
import {
|
||
PAPERCLIP_OPERATIONAL_SKILL_KEY,
|
||
resolvePaperclipDesiredSkillNames,
|
||
} from "@paperclipai/adapter-utils/server-utils";
|
||
import {
|
||
NATIVE_RUNTIME_ASSET_SCHEMA,
|
||
PAPERCLIP_EXECUTION_PROMPT,
|
||
PAPERCLIP_EXECUTION_PROMPT_REVISION,
|
||
canonicalNativeRuntimeContextDigest,
|
||
nativeRuntimePromptDigest,
|
||
parseNativeRuntimeContext,
|
||
type NativeRuntimeAssetReference,
|
||
type NativeRuntimeContextSnapshot,
|
||
} from "../../vendor/paperclip-runner/index.js";
|
||
import { resolvePaperclipInstanceRoot } from "../../home-paths.js";
|
||
import { agentInstructionsService, agentInstructionsBundleMode } from "../agent-instructions.js";
|
||
import { agentInstructionRevisionService } from "../agent-instruction-revisions.js";
|
||
import { toolAccessService } from "../tool-access.js";
|
||
import { filterResolvedGitHubConnectionsForRun } from "../git-credentials.js";
|
||
import { rehydratePreservedRuntimeSkill } from "./runtime-skill-preservation.js";
|
||
|
||
const MAX_ASSET_FILES = 10_000;
|
||
const MAX_ASSET_BYTES = 64 * 1024 * 1024;
|
||
type RuntimeAgent = { id: string; companyId: string; name: string; adapterType?: string | null; adapterConfig: unknown };
|
||
type AssetFile = { path: string; content: Buffer; mode: number };
|
||
|
||
const sha256 = (value: string | Buffer) => createHash("sha256").update(value).digest("hex");
|
||
const safeRelativePath = (value: string, label: string) => {
|
||
const normalized = value.replaceAll("\\", "/");
|
||
if (!normalized || normalized.includes("\0") || normalized.startsWith("/") || normalized.split("/").some((part) => !part || part === "." || part === "..")) {
|
||
throw new Error(`${label} must stay within its bundle root`);
|
||
}
|
||
return normalized;
|
||
};
|
||
|
||
async function collectDirectoryFiles(sourceRoot: string): Promise<AssetFile[]> {
|
||
const root = path.resolve(sourceRoot);
|
||
const rootStat = await fs.lstat(root).catch(() => null);
|
||
if (!rootStat?.isDirectory() || rootStat.isSymbolicLink()) throw new Error(`runtime context source is not a safe directory: ${root}`);
|
||
const files: AssetFile[] = [];
|
||
let totalBytes = 0;
|
||
async function visit(relativeDirectory: string): Promise<void> {
|
||
const entries = await fs.readdir(relativeDirectory ? path.join(root, relativeDirectory) : root, { withFileTypes: true });
|
||
entries.sort((a, b) => a.name.localeCompare(b.name));
|
||
for (const entry of entries) {
|
||
const relativePath = safeRelativePath(relativeDirectory ? `${relativeDirectory}/${entry.name}` : entry.name, "runtime context path");
|
||
const absolutePath = path.join(root, relativePath);
|
||
const stat = await fs.lstat(absolutePath);
|
||
if (stat.isSymbolicLink()) throw new Error(`runtime context bundles do not permit symlinks: ${relativePath}`);
|
||
if (stat.isDirectory()) { await visit(relativePath); continue; }
|
||
if (!stat.isFile()) throw new Error(`runtime context bundle contains an unsupported file: ${relativePath}`);
|
||
const content = await fs.readFile(absolutePath);
|
||
totalBytes += content.byteLength;
|
||
if (files.length + 1 > MAX_ASSET_FILES || totalBytes > MAX_ASSET_BYTES) throw new Error("runtime context bundle exceeds its bound");
|
||
files.push({ path: relativePath, content, mode: stat.mode & 0o555 });
|
||
}
|
||
}
|
||
await visit("");
|
||
if (!files.length) throw new Error(`runtime context source is empty: ${root}`);
|
||
return files;
|
||
}
|
||
|
||
async function makeDirectoriesReadOnly(directory: string): Promise<void> {
|
||
for (const entry of await fs.readdir(directory, { withFileTypes: true })) {
|
||
if (entry.isDirectory()) await makeDirectoriesReadOnly(path.join(directory, entry.name));
|
||
}
|
||
await fs.chmod(directory, 0o555);
|
||
}
|
||
|
||
async function verifyMaterializedAsset(
|
||
rootPath: string,
|
||
manifestFiles: Array<{ path: string; sha256: string; mode: number; size: number }>,
|
||
): Promise<void> {
|
||
const expected = manifestFiles.map(file => ({ ...file, executable: Boolean(file.mode & 0o111) }));
|
||
if (!await measureRunPhase("runtime_context.asset_read", () => verifyFileManifest(rootPath, expected))) {
|
||
throw new Error("runtime context asset digest mismatch");
|
||
}
|
||
}
|
||
|
||
export async function materializeAsset(files: AssetFile[]): Promise<NativeRuntimeAssetReference> {
|
||
const { sorted, manifestFiles, totalBytes, assetDigest, manifestText, manifestDigest } = await measureRunPhase("runtime_context.asset_manifest", () => {
|
||
const sorted = [...files].sort((a, b) => a.path.localeCompare(b.path));
|
||
const manifestFiles = sorted.map((file) => ({ path: safeRelativePath(file.path, "runtime context path"), sha256: sha256(file.content), mode: file.mode & 0o555, size: file.content.byteLength }));
|
||
const totalBytes = manifestFiles.reduce((sum, file) => sum + file.size, 0);
|
||
if (!manifestFiles.length || manifestFiles.length > MAX_ASSET_FILES || totalBytes > MAX_ASSET_BYTES) throw new Error("runtime context bundle is empty or exceeds its bound");
|
||
const assetDigest = sha256(JSON.stringify(manifestFiles));
|
||
const manifestText = `${JSON.stringify({ schema: "paperclip.runtime-asset-manifest.v1", digest: assetDigest, fileCount: manifestFiles.length, totalBytes, files: manifestFiles })}\n`;
|
||
const manifestDigest = sha256(manifestText);
|
||
return { sorted, manifestFiles, totalBytes, assetDigest, manifestText, manifestDigest };
|
||
});
|
||
return withRuntimeAssetUse(assetDigest, async function publishOrReuse(locked): Promise<NativeRuntimeAssetReference> {
|
||
const assetsRoot = path.join(resolvePaperclipInstanceRoot(), "runtime-context-assets");
|
||
const rootPath = path.join(assetsRoot, "bundles", assetDigest);
|
||
const manifestPath = path.join(assetsRoot, "manifests", `${assetDigest}.json`);
|
||
const existing = await measureRunPhase("runtime_context.asset_cache", () => fs.readFile(manifestPath, "utf8").catch(() => null));
|
||
const rootExists = (await fs.stat(rootPath).catch(() => null))?.isDirectory();
|
||
if (existing !== null && rootExists && sha256(existing) === manifestDigest
|
||
&& await verifyFileManifest(rootPath, manifestFiles.map(file => ({ ...file, executable: Boolean(file.mode & 0o111) })))) {
|
||
return { schema: NATIVE_RUNTIME_ASSET_SCHEMA, digest: assetDigest, manifestDigest, rootPath, fileCount: manifestFiles.length, totalBytes };
|
||
}
|
||
if (!locked) return withRuntimeAssetUse(assetDigest, publishOrReuse, true);
|
||
// The caller supplied the exact authoritative bytes for this digest. A missing
|
||
// or damaged disposable copy can be rebuilt under the publication/GC lock.
|
||
if (rootExists) await removeRuntimeAssetTree(rootPath);
|
||
await fs.rm(manifestPath, { force: true });
|
||
await measureRunPhase("runtime_context.asset_materialize", async () => {
|
||
const stagingRoot = path.join(assetsRoot, ".staging", randomUUID());
|
||
await fs.mkdir(stagingRoot, { recursive: true, mode: 0o700 });
|
||
try {
|
||
for (const file of sorted) {
|
||
const target = path.join(stagingRoot, file.path);
|
||
await fs.mkdir(path.dirname(target), { recursive: true, mode: 0o700 });
|
||
await fs.writeFile(target, file.content, { mode: file.mode & 0o555 });
|
||
await fs.chmod(target, file.mode & 0o555);
|
||
}
|
||
await fs.mkdir(path.dirname(rootPath), { recursive: true, mode: 0o700 });
|
||
let publishedRoot = false;
|
||
await fs.rename(stagingRoot, rootPath).then(() => {
|
||
publishedRoot = true;
|
||
}).catch(async (error: NodeJS.ErrnoException) => {
|
||
// Concurrent publishers can surface EACCES on macOS when the winning
|
||
// destination has already been made read-only. Treat that as a race only
|
||
// when the content-addressed destination now exists; a genuine permission
|
||
// failure with no published destination must still fail loudly.
|
||
const publishedByPeer = (await fs.stat(rootPath).catch(() => null))?.isDirectory() === true;
|
||
if (error.code !== "EEXIST" && error.code !== "ENOTEMPTY" && !publishedByPeer) throw error;
|
||
await fs.rm(stagingRoot, { recursive: true, force: true });
|
||
});
|
||
// Publish the writable staging directory first. Renaming a read-only
|
||
// directory can fail with EACCES on macOS even when both parents are owned
|
||
// and writable. Files are already immutable; directories become immutable
|
||
// immediately after this process wins publication.
|
||
if (publishedRoot) await makeDirectoriesReadOnly(rootPath);
|
||
await fs.mkdir(path.dirname(manifestPath), { recursive: true, mode: 0o700 });
|
||
await fs.writeFile(manifestPath, manifestText, { flag: "wx", mode: 0o400 }).catch(async (error: NodeJS.ErrnoException) => {
|
||
if (error.code !== "EEXIST") throw error;
|
||
if (sha256(await measureRunPhase("runtime_context.asset_cache", () => fs.readFile(manifestPath, "utf8"))) !== manifestDigest) throw new Error(`runtime context manifest race mismatch: ${assetDigest}`);
|
||
});
|
||
await measureRunPhase("runtime_context.asset_verify", () => verifyMaterializedAsset(rootPath, manifestFiles), { fileCount: manifestFiles.length, bytes: totalBytes });
|
||
} catch (error) {
|
||
await fs.rm(stagingRoot, { recursive: true, force: true }).catch(() => undefined);
|
||
throw error;
|
||
}
|
||
}, { fileCount: manifestFiles.length, bytes: totalBytes });
|
||
return { schema: NATIVE_RUNTIME_ASSET_SCHEMA, digest: assetDigest, manifestDigest, rootPath, fileCount: manifestFiles.length, totalBytes };
|
||
});
|
||
}
|
||
|
||
async function materializeInstructionBundle(db: Db, agent: RuntimeAgent, agentFiles = false) {
|
||
if (agentFiles) {
|
||
const current = await agentInstructionRevisionService(db).readCommittedForRuntime({ companyId: agent.companyId, agentId: agent.id });
|
||
if (!current) throw new Error("Configured instruction entry is missing");
|
||
const entryPath = safeRelativePath(current.revision.entryFile, "instruction entry path");
|
||
return { entryPath, bundle: await materializeAsset([{ path: entryPath, content: Buffer.from(current.content, "utf8"), mode: 0o444 }]) };
|
||
}
|
||
const exported = await agentInstructionsService(db).exportFiles(agent, { rejectSymlinks: true });
|
||
if (agentInstructionsBundleMode(agent) === "managed") {
|
||
// The database head remains authoritative when a saved revision could not
|
||
// update its disk projection. Overlay the exact committed snapshot directly;
|
||
// a repair followed by a separate export can race another pending commit.
|
||
const current = await agentInstructionRevisionService(db).readCommittedForRuntime({ companyId: agent.companyId, agentId: agent.id });
|
||
if (current) {
|
||
exported.entryFile = current.revision.entryFile;
|
||
exported.files[current.revision.entryFile] = current.content;
|
||
}
|
||
}
|
||
const entryPath = safeRelativePath(exported.entryFile, "instruction entry path");
|
||
if (!(entryPath in exported.files)) throw new Error(`configured instruction entry is missing: ${entryPath}`);
|
||
const files = Object.entries(exported.files).map(([relativePath, content]) => ({ path: safeRelativePath(relativePath, "instruction path"), content: Buffer.from(content, "utf8"), mode: 0o444 }));
|
||
return { entryPath, bundle: await materializeAsset(files) };
|
||
}
|
||
|
||
async function materializeSkillAsset(source: string) {
|
||
let bundle = await measureRunPhase("runtime_context.asset_verify", () => cachedSkillAsset(source));
|
||
if (!bundle) {
|
||
const files = await measureRunPhase("runtime_context.asset_read", () => collectDirectoryFiles(source));
|
||
bundle = await materializeAsset(files);
|
||
const manifest: FileManifestEntry[] = files.map(file => ({ path: file.path, size: file.content.length,
|
||
sha256: sha256(file.content), mode: file.mode & 0o555, executable: Boolean(file.mode & 0o111) }));
|
||
if (await verifyFileManifest(source, manifest)) rememberSkillAsset(source, manifest, bundle);
|
||
}
|
||
return bundle;
|
||
}
|
||
|
||
async function localRuntimeAssetPresent(asset: NativeRuntimeAssetReference) {
|
||
const root = path.join(resolvePaperclipInstanceRoot(), "runtime-context-assets");
|
||
const manifest = await fs.readFile(path.join(root, "manifests", `${asset.digest}.json`), "utf8").catch(() => null);
|
||
return manifest !== null && sha256(manifest) === asset.manifestDigest
|
||
&& (await fs.lstat(path.join(root, "bundles", asset.digest)).catch(() => null))?.isDirectory() === true;
|
||
}
|
||
|
||
function singleFileAssetDigest(entryPath: string, content: Buffer) {
|
||
return sha256(JSON.stringify([{ path: entryPath, sha256: sha256(content), mode: 0o444, size: content.byteLength }]));
|
||
}
|
||
|
||
export async function rehydratePinnedInstructionAsset(
|
||
db: Db,
|
||
binding: { companyId: string; agentId: string },
|
||
context: NativeRuntimeContextSnapshot,
|
||
): Promise<NativeRuntimeContextSnapshot> {
|
||
const pinned = context.instructions.bundle;
|
||
if (pinned.rootPath !== path.join(resolvePaperclipInstanceRoot(), "runtime-context-assets", "bundles", pinned.digest)) return context;
|
||
if (await localRuntimeAssetPresent(pinned)) return context;
|
||
const matches = (bundle: NativeRuntimeAssetReference) => bundle.digest === pinned.digest
|
||
&& bundle.manifestDigest === pinned.manifestDigest && bundle.fileCount === pinned.fileCount && bundle.totalBytes === pinned.totalBytes;
|
||
if (pinned.fileCount === 1) {
|
||
const rows = await db.select({ contentBase64: agentInstructionRevisions.contentBase64 }).from(agentInstructionRevisions)
|
||
.where(and(eq(agentInstructionRevisions.companyId, binding.companyId), eq(agentInstructionRevisions.agentId, binding.agentId),
|
||
eq(agentInstructionRevisions.entryFile, context.instructions.entryPath)))
|
||
.orderBy(desc(agentInstructionRevisions.createdAt)).limit(500);
|
||
for (const row of rows) {
|
||
const content = Buffer.from(row.contentBase64, "base64");
|
||
if (singleFileAssetDigest(context.instructions.entryPath, content) !== pinned.digest) continue;
|
||
const bundle = await materializeAsset([{ path: context.instructions.entryPath, content, mode: 0o444 }]);
|
||
if (matches(bundle)) return { ...context, instructions: { ...context.instructions, bundle } };
|
||
}
|
||
}
|
||
const [agent] = await db.select().from(agents).where(and(eq(agents.companyId, binding.companyId), eq(agents.id, binding.agentId)));
|
||
if (agent) {
|
||
for (const agentFiles of [true, false]) {
|
||
const rebuilt = await materializeInstructionBundle(db, agent, agentFiles).catch(() => null);
|
||
if (rebuilt && rebuilt.entryPath === context.instructions.entryPath && matches(rebuilt.bundle)) {
|
||
return { ...context, instructions: { ...context.instructions, bundle: rebuilt.bundle } };
|
||
}
|
||
}
|
||
}
|
||
return context;
|
||
}
|
||
|
||
export async function rehydratePinnedSkillAssets(
|
||
db: Db,
|
||
binding: { companyId: string; runId: string; agentId: string },
|
||
context: NativeRuntimeContextSnapshot,
|
||
): Promise<NativeRuntimeContextSnapshot> {
|
||
if (!context.skills.length) return context;
|
||
const [run] = await db.select({ responsibleUserId: heartbeatRuns.responsibleUserId,
|
||
id: heartbeatRuns.id, companyId: heartbeatRuns.companyId, agentId: heartbeatRuns.agentId,
|
||
nativeIssueId: heartbeatRuns.nativeIssueId, runnerProfileJson: heartbeatRuns.runnerProfileJson,
|
||
contextSnapshot: heartbeatRuns.contextSnapshot }).from(heartbeatRuns).where(and(
|
||
eq(heartbeatRuns.companyId, binding.companyId), eq(heartbeatRuns.id, binding.runId), eq(heartbeatRuns.agentId, binding.agentId)));
|
||
if (!run) throw new Error("Skill recovery has no matching admitted run");
|
||
const actor = { type: "agent" as const, companyId: binding.companyId, agentId: binding.agentId,
|
||
runId: binding.runId, source: "agent_jwt" as const,
|
||
onBehalfOfUserId: run.responsibleUserId === "local-board" ? null : run.responsibleUserId };
|
||
const delivery = skillExecutionContent(db, { actor });
|
||
const pins = run.contextSnapshot?.skillVersions;
|
||
const skills: NativeRuntimeContextSnapshot["skills"] = [];
|
||
for (const skill of context.skills) {
|
||
if (!skill.versionId) {
|
||
// Newly admitted local assets remain usable during an ordinary rolling
|
||
// upgrade. A migrated asset uses its exact durable receipt on cold hosts.
|
||
const preserved = run.runnerProfileJson?.nativeRuntimeSkillPreservation;
|
||
if (!preserved) { skills.push(skill); continue; }
|
||
const bundle = await rehydratePreservedRuntimeSkill(db, run, skill, materializeSkillAsset);
|
||
skills.push({ ...skill, bundle });
|
||
continue;
|
||
}
|
||
if (pins !== undefined && (!pins || typeof pins !== "object" || Array.isArray(pins)
|
||
|| (pins as Record<string, unknown>)[skill.key] !== skill.versionId)) {
|
||
throw new Error("Skill recovery differs from the admitted version selection");
|
||
}
|
||
const [version] = await db.select({ skillId: companySkillVersions.companySkillId }).from(companySkillVersions).where(and(
|
||
eq(companySkillVersions.companyId, binding.companyId), eq(companySkillVersions.id, skill.versionId)));
|
||
if (!version) throw new Error("Pinned skill version is unavailable for recovery");
|
||
const bundle = await delivery.withVersion({ companyId: binding.companyId, skillId: version.skillId, versionId: skill.versionId },
|
||
source => materializeSkillAsset(source));
|
||
if (bundle.digest !== skill.bundle.digest || bundle.manifestDigest !== skill.bundle.manifestDigest
|
||
|| bundle.fileCount !== skill.bundle.fileCount || bundle.totalBytes !== skill.bundle.totalBytes) {
|
||
throw new Error("Recovered skill projection does not match its admitted digest");
|
||
}
|
||
skills.push({ ...skill, bundle });
|
||
}
|
||
return { ...context, skills };
|
||
}
|
||
|
||
/**
|
||
* The admitted context names every skill by digest, so the Runner's copy is
|
||
* described without local files. Pinned skill files are rehydrated on this
|
||
* server now, or, when reconnecting to a live remote Runner that already holds
|
||
* them, only if a launch must stage them again.
|
||
*/
|
||
export async function prepareRuntimeContextSource(
|
||
db: Db,
|
||
binding: { companyId: string; runId: string; agentId: string },
|
||
context: NativeRuntimeContextSnapshot,
|
||
options: { deferUntilStaged: boolean },
|
||
) {
|
||
// Defer only when preparation already checked access to every admitted skill:
|
||
// each one is pinned to its exact version in the run's saved selection.
|
||
const pinned = options.deferUntilStaged && context.skills.length > 0
|
||
? await db.select({ contextSnapshot: heartbeatRuns.contextSnapshot }).from(heartbeatRuns).where(and(
|
||
eq(heartbeatRuns.companyId, binding.companyId), eq(heartbeatRuns.id, binding.runId), eq(heartbeatRuns.agentId, binding.agentId)))
|
||
.then(([run]) => {
|
||
const pins = run?.contextSnapshot?.skillVersions;
|
||
return Boolean(pins && typeof pins === "object" && !Array.isArray(pins)
|
||
&& context.skills.every(skill => skill.versionId && (pins as Record<string, unknown>)[skill.key] === skill.versionId));
|
||
})
|
||
: options.deferUntilStaged;
|
||
let staged = pinned ? null : await rehydratePinnedSkillAssets(db, binding, context);
|
||
return {
|
||
described: staged ?? context,
|
||
forStaging: async () => (staged ??= await rehydratePinnedSkillAssets(db, binding, context)),
|
||
};
|
||
}
|
||
|
||
export async function materializeSelectedSkills(runtimeConfig: Record<string, unknown>, entries: PaperclipSkillEntry[], omitLegacy: boolean) {
|
||
const desiredKeys = resolvePaperclipDesiredSkillNames(runtimeConfig, entries).filter(
|
||
(key) => !omitLegacy || key !== PAPERCLIP_OPERATIONAL_SKILL_KEY,
|
||
);
|
||
const byKey = new Map(entries.map((entry) => [entry.key, entry]));
|
||
return Promise.all(desiredKeys.sort().map(async (key) => {
|
||
const entry = byKey.get(key);
|
||
if (!entry) throw new Error(`assigned runtime skill is missing from the company library: ${key}`);
|
||
if (entry.sourceStatus === "missing") throw new Error(entry.missingDetail ?? `assigned runtime skill source is missing: ${key}`);
|
||
const bundle = await materializeSkillAsset(entry.source);
|
||
return { key: entry.key, runtimeName: safeRelativePath(entry.runtimeName, `runtime name for ${key}`), versionId: entry.versionId ?? entry.currentVersionId ?? null, bundle };
|
||
}));
|
||
}
|
||
|
||
export async function resolveNativeRuntimeMcpSnapshot(input: { db: Db; agent: Pick<RuntimeAgent, "id" | "companyId">; runId: string }) {
|
||
const effective = await toolAccessService(input.db).getEffectiveToolAssignmentsForAgent(input.agent.companyId, input.agent.id);
|
||
const permitted = new Set([...effective.entries.filter((entry) => entry.effect === "include" && entry.connectionId).map((entry) => entry.connectionId!), ...effective.allowedTools.map((tool) => tool.connectionId)]);
|
||
const hasGitHubConnection = effective.installedConnections.some((connection) => {
|
||
const config = connection.config && typeof connection.config === "object"
|
||
? connection.config as Record<string, unknown>
|
||
: {};
|
||
const transportConfig = connection.transportConfig && typeof connection.transportConfig === "object"
|
||
? connection.transportConfig as Record<string, unknown>
|
||
: {};
|
||
return config.sourceTemplateKey === "github" || transportConfig.sourceTemplateKey === "github";
|
||
});
|
||
const hasVoiceConnection = effective.installedConnections.some(connection => connection.transport === "voice");
|
||
const [runIdentity] = hasGitHubConnection || hasVoiceConnection
|
||
? await input.db.select({ issueId: boundRunIssueIdSql(), responsibleUserId: heartbeatRuns.responsibleUserId, activeIdentityContextId: heartbeatRuns.activeIdentityContextId })
|
||
.from(heartbeatRuns)
|
||
.where(and(
|
||
eq(heartbeatRuns.id, input.runId),
|
||
eq(heartbeatRuns.companyId, input.agent.companyId),
|
||
eq(heartbeatRuns.agentId, input.agent.id),
|
||
))
|
||
.limit(1)
|
||
: [];
|
||
// Broker runs pin the permitted tool catalog, not one person’s credential.
|
||
// Selection happens at each operation and can change through steering.
|
||
const resolvedInstalledConnections = runIdentity?.activeIdentityContextId
|
||
? effective.installedConnections : await filterResolvedGitHubConnectionsForRun({
|
||
db: input.db,
|
||
companyId: input.agent.companyId,
|
||
agentId: input.agent.id,
|
||
responsibleUserId: runIdentity?.responsibleUserId ?? null,
|
||
connections: effective.installedConnections,
|
||
});
|
||
// App access is optional runtime context. Keep usable assignments pinned, but
|
||
// do not stop unrelated work because an assigned app needs attention.
|
||
const githubBotConnectionIds = await githubBotConnectionIdsForRun(input.db, input.agent.companyId, input.agent.id, input.runId);
|
||
const spekoConnectionIds = hasVoiceConnection ? new Set((await spekoToolsForSession(input.db, { companyId: input.agent.companyId, agentId: input.agent.id, runId: input.runId, issueId: runIdentity?.issueId ?? null, identityContextId: runIdentity?.activeIdentityContextId })).map(tool => tool.connectionId)) : new Set<string>();
|
||
const availableConnectionIds = new Set(resolvedInstalledConnections.filter((connection) =>
|
||
permitted.has(connection.id)
|
||
&& connection.status === "active"
|
||
&& connection.enabled
|
||
&& (Boolean(runIdentity?.activeIdentityContextId) && (connection.config?.sourceTemplateKey === "github" || connection.transportConfig?.sourceTemplateKey === "github")
|
||
|| connection.credentialPolicy === "per_user"
|
||
|| connection.transport === "plugin"
|
||
|| !isToolConnectionAttentionHealth(connection.healthStatus))
|
||
// Keep this transport list equal to buildPaperclipRuntimeMcpServers.
|
||
&& (["mcp_remote", "local_stdio", "plugin"].includes(connection.transport) || isBrowserUseConnection(connection) || restConnectorFor(connection) || githubBotConnectionIds.has(connection.id) || spekoConnectionIds.has(connection.id))
|
||
).map((connection) => connection.id));
|
||
const assignment = {
|
||
version: 1,
|
||
agentId: input.agent.id,
|
||
connections: [...availableConnectionIds].sort(),
|
||
tools: effective.allowedTools
|
||
.filter((tool) => availableConnectionIds.has(tool.connectionId))
|
||
.map((tool) => tool.id)
|
||
.sort(),
|
||
};
|
||
const assignmentDigest = sha256(JSON.stringify(assignment));
|
||
return { assignmentSetId: `sha256:${assignmentDigest}`, digest: assignmentDigest, bindingId: assignment.connections.length ? `native-mcp:${input.runId}` : null };
|
||
}
|
||
|
||
export async function buildNativeRuntimeContext(input: { db: Db; agent: RuntimeAgent; runId: string; runtimeConfig: Record<string, unknown>; runtimeSkillEntries: PaperclipSkillEntry[]; instructionWorkingCopy?: { rootPath: string; entryPath: string; kind?: "agent_files" } }): Promise<NativeRuntimeContextSnapshot> {
|
||
const preconfigured = input.runtimeConfig.homeMode === "preconfigured";
|
||
const emptyMcpDigest = sha256("preconfigured");
|
||
const [instructions, skills, mcp] = await Promise.all([
|
||
measureRunPhase("runtime_context.instructions", () => materializeInstructionBundle(input.db, input.agent, input.instructionWorkingCopy?.kind === "agent_files")),
|
||
measureRunPhase("runtime_context.skills", () => preconfigured ? [] : materializeSelectedSkills(input.runtimeConfig, input.runtimeSkillEntries, input.agent.adapterType === "paperclip_runner")),
|
||
measureRunPhase("runtime_context.mcp", () => preconfigured ? { assignmentSetId: `sha256:${emptyMcpDigest}`, digest: emptyMcpDigest, bindingId: null } : resolveNativeRuntimeMcpSnapshot({ db: input.db, agent: input.agent, runId: input.runId })),
|
||
]);
|
||
const snapshot = {
|
||
prompt: { revision: PAPERCLIP_EXECUTION_PROMPT_REVISION, text: PAPERCLIP_EXECUTION_PROMPT, digest: nativeRuntimePromptDigest() },
|
||
instructions: { ...instructions, ...(input.instructionWorkingCopy ? { workingCopy: input.instructionWorkingCopy } : {}) },
|
||
skills,
|
||
mcp,
|
||
...(input.runtimeConfig.paperclipConnectionInstructions ? {
|
||
connectionInstructions: input.runtimeConfig.paperclipConnectionInstructions as { text: string; digest: string },
|
||
} : {}),
|
||
};
|
||
return parseNativeRuntimeContext({ ...snapshot, aggregateDigest: canonicalNativeRuntimeContextDigest(snapshot) });
|
||
}
|