mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-27 19:35:13 +01:00
## Thinking Path > - Paperclip is the open source control plane for agent companies. > - People choose a harness when creating an agent. > - The server should resolve that harness to a qualified runner. > - Codex is the first harness in the staged default rollout. > - Saved agents must retain their execution choices. > - New Codex agents therefore use Paperclip Runner on qualified macOS/Linux targets, with an explicit Legacy override. ## Linked Issues or Issue Description Refs #16028, #15555, #15673, #15422, #15631, #15926, #15932, #16149. The runtime/install prerequisite and release repair are merged. This slice owns Codex creation, setup, configuration and preservation. Other harness defaults remain unchanged. It adds no database migration; incoming master migrations, required maps and regression coverage are retained. ## What Changed - Resolve `runner: auto | paperclip | legacy` at the common creation boundary across onboarding, API/CLI, hires, built-ins, plugins and imports. Freeze pending hires and export resolved execution choices. - Preserve existing runners on upgrades, unrelated edits and import updates, including compatible accounts, workspace settings, Codex effort and intentional clears. - Use the underlying harness picker and branding; put the runner override in Advanced. Use the shared dropdown and update production-component stories and creation contracts. - Support qualified Apple Silicon, Intel Mac, Linux x64 and Linux ARM64. Linux requires GNU libc 2.39+. Disabled, external, unsupported and unqualified combinations retain their applicable Legacy execution. - Test the selected runtime/account/model/environment. Ordinary Connect probes eligible immutable managed targets and refreshes scoped discovery. Creation/discovery do not provision. Auth, dependency, incompatible-setting and cleanup failures remain actionable without changing runners. - Keep setup probes read-only, preserve local/SSH isolation and resolve packaged assets before staging credentials. Bound cleanup reads by the remaining absolute deadline; physically close every owned ingress socket before endpoint release. Unconfirmed cleanup retains recovery state. - Preserve validated completed-onboarding callbacks through startup recovery. First file-producing work belongs to the child task and uses the existing dependency/block handoff and task-scoped registration. - Reuse existing E2E fixtures, artifact checks, reporting and controls. Keep Docker/Cargo/native compilation remote. Paperclip tarballs contain no Codex payloads; official optional npm platform dependencies remain. ## Verification Current tested head: `1c03a8815672d82c264a9b08d4ea1984e8fae107`. **Merge-ready: required CI, fresh review, package qualification, Mac core acceptance and Test task/follow-up acceptance pass.** The existing attachment and native-continuity oracles independently verify both Test downloads, run attribution and unchanged native/provider/workspace identities. The maintainer authorized a code-owner-review exception for this PR only. The first Test setup failure and normal Board completion-review friction are retained below. Actual post-merge packages and private Cloud distribution remain separate qualification gates. - [Current CI](https://github.com/paperclipai/paperclip/actions/runs/38564241625): all 48 ordinary jobs and required verify/E2E pass. Attempt 1 retained six simultaneous host-shutdown cancellations. One failed-only retry passes those six lanes and two aggregates, preserving 40 successes. Actual integration checkout `53104c1` combines this head with master `c80`; it is not relabeled as the branch tree. Fresh exact-head Greptile 5/5, 30 resolved threads, no actionable findings. - [Exact-source package qualification](https://github.com/paperclipai/paperclip/actions/runs/38565287239): nine applicable jobs pass, nine conditional jobs skip. Four remote native producers; Linux x64 clean install; Intel Mac, Apple Silicon and Linux ARM64 native startup/physical cleanup controls. All 18 tarballs/four daemon source/hash/manifest bindings and 343 migrations through 0344 verify. No Codex payload is packed. These hosted tarballs are not a published npm canary. - Normal Apple Silicon installation and credential-free native startup/cleanup pass using the packaged daemon, with no source build or provider turn. Fresh installed DB initialization passes. Upgrade 651a→1c03 preserves all three complete execution configs, accounts, ten recorded run profiles, paused budgets, active incidents, cost summaries and two historical unpriced events. Historical recovery state is retained. - Cleanup regressions: four delayed-read baseline failures become 15 passing deadline/authority controls. No-acknowledgement and prior-error socket ownership regressions become 12 passing controls. Direct runtime/server types, maps: 35 areas / 176 IDs, diff checks and independent reviews pass. Permission/auth/reconnect policy and the 10-second setup budget remain intact. - Existing source coverage includes creation/selection/imports, pending approvals, recovery, account/lifecycle handling, mounted forms, dropdowns and E2E support. Full typechecks/tests/build/token and applicable Storybook checks pass in the recorded current CI; no duplicate suite or weakened grader. - Literal 9546 live Mac proof: ordinary Auto Codex onboarding, useful document, real completed-child callback/reply, seven successful native runs, manual follow-up and independently downloaded1,101-byte child file. Existing Legacy title edit/reload preserves execution; two real Legacy runs produce independently checked15/34-byte files. These remain historical source records. - Current literal 1c03 Mac core acceptance passes independent checks: ordinary Auto native Codex onboarding without runner override; only timeout 180 seconds/concurrency 1 saved; six successful native runs and six serialized turns (maximum overlap 1); both same-task continuity checks and five mechanical callback checks; actual rendered parent reply and exact child downloads. The useful handoff advances from revision 1 to 2. Both files are 48 bytes, SHA-256 `8b742bd4e8123a7c078f0a8d9f3449fbfa8a320d7d90be641a2818db3c53d92c` and `f42db9774fa06e804ab7d596fc8b922a2b638921c418b87d0111151d290a6afe`; exactly one byte changes and old history remains. The agent is paused with zero active runs/errors. Both artifact work products have `isPrimary=false`, and descriptive titles fail a stricter title-helper assumption; these retained results are scoped editorial follow-up, not new merge gates. Six source-bound screenshots are uploaded and verified in the private primary QA report. - [Literal 9546 managed proof](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-38485488621-1/index.html): original 19 assertions and completed judge pass, four native turns, child-bound240-byte file and session/workspace continuity. Checkout reports a verified generated target-lock mutation; full final porcelain was not retained. Signed execution-image identity remains separate from Core source. - Actual 651a Test deployment succeeds on one owned/pinned target. One ordinary Auto API-key Connect then fails native cleanup before hire. Controller turn/drain/suspend completion and service-recorded environment cleanup are observed; they do not certify physical deletion or native authority retirement. Failure/recovery/unknown holds are retained. No Legacy switch or unchanged paid retry. The two source repairs above require actual treatment. - [Current immutable Core preview](https://github.com/paperclipai/paperclip/actions/runs/38570365725) and matching DB/shared artifacts pass. One owned, pinned Test target serves exact `1c03`; identical migration history correctly skips the migrator. Ordinary Auto Codex Connect succeeds and hires native; saved native selection and QA bounds survive reload. The initial child preparation attempt fails before provider work, with actual stdout identifying `renewal_deadline`; its trigger remains unknown and the failure is retained. Normal Board **Try again** and **Resume** recover the child to Done, producing a browser-verified 44-byte file, SHA-256 `0a8458b1efcc2d7a0d815689458c4caab039a46c7c7a3e5805de6bc8b1a96ecc`. The actual parent callback reports the completed child document/file. The existing validator requires a parent-owned attachment, so recovery enters normal Review; after byte checking, normal **Approve completion** reaches parent Done. No forced status or duplicate parent file is used. The validator is unchanged from integrated and current master. Autonomous initial callback-to-Done is not proven. The same-child follow-up completes document revision 2 and publishes a second independently downloaded 44-byte file, SHA-256 `22022257300fae762784632d08d3bbe0bab61e2ce81f91864a14dcd7aa464591`; only the revision digit changes. The unchanged existing native-continuity and attachment oracles pass. All ten recorded runs are terminal: nine native successes and the retained pre-provider setup failure. The QA agent is paused with zero active runs. The pre-existing parent evidence scope mismatch is a separate follow-up, with ordinary completion review exercised rather than a forced status or relaxed validation. This unsigned Core preview remains distinct from post-merge private Cloud composition. ## Risks - Saved agents are not migrated. Explicit runner changes use normal configuration/session rules; historical runs retain their recorded execution. - Native readiness depends on verified assets and server-owned target qualification. A Legacy probe, HTTP200 or successful account save cannot prove native readiness. - Recovery exceptions remain restricted to validated completed-onboarding reports; process-loss, ownership and terminal-state safeguards remain active. Unexecuted live journeys stay open. - Linux ARM paid tasks, Windows, Alpine and 32-bit ARM remain deferred. Cold Cargo cache misses are recorded; no warm-cache speedup is claimed. - Historical invoices/physical cleanup remain unconfirmed and held. After merge, exact published npm canary, signed standard image/migrator, private composition and serving/live acceptance still require qualification. Production deployment/default promotion is not authorized. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tools, code execution and parallel agents. Exact serving snapshot and context-window size are not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have linked public issues or PRs above - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run the stated focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green on the current candidate - [x] Greptile is 5/5 with no open findings on the current candidate - [x] I will address all reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
799 lines
30 KiB
TypeScript
799 lines
30 KiB
TypeScript
/**
|
|
* @fileoverview Adapter management REST API routes
|
|
*
|
|
* This module provides Express routes for managing external adapter plugins:
|
|
* - Listing all registered adapters (built-in + external)
|
|
* - Installing external adapters from npm packages or local paths
|
|
* - Unregistering external adapters
|
|
*
|
|
* Read-only routes require board org access. Mutating adapter management
|
|
* routes require instance-admin access because they can install, reload, or
|
|
* toggle server-side adapter code for the whole Paperclip instance.
|
|
*
|
|
* @module server/routes/adapters
|
|
*/
|
|
|
|
import { execFile } from "node:child_process";
|
|
import fs from "node:fs";
|
|
import { readFile } from "node:fs/promises";
|
|
import path from "node:path";
|
|
import { promisify } from "node:util";
|
|
import { Router } from "express";
|
|
import {
|
|
listServerAdapters,
|
|
findServerAdapter,
|
|
findActiveServerAdapter,
|
|
listEnabledServerAdapters,
|
|
registerServerAdapter,
|
|
resolveExternalAdapterRegistration,
|
|
unregisterServerAdapter,
|
|
isOverridePaused,
|
|
setOverridePaused,
|
|
waitForExternalAdapters,
|
|
} from "../adapters/registry.js";
|
|
import {
|
|
listAdapterPlugins,
|
|
addAdapterPlugin,
|
|
removeAdapterPlugin,
|
|
getAdapterPluginByType,
|
|
getAdapterPluginsDir,
|
|
getDisabledAdapterTypes,
|
|
setAdapterDisabled,
|
|
} from "../services/adapter-plugin-store.js";
|
|
import type { AdapterPluginRecord } from "../services/adapter-plugin-store.js";
|
|
import type { ServerAdapterModule, AdapterConfigSchema } from "../adapters/types.js";
|
|
import type {
|
|
AdapterLoginPanelMode,
|
|
AdapterLoginTimeoutPolicy,
|
|
} from "@paperclipai/adapter-utils";
|
|
import { loadExternalAdapterPackage, getUiParserSource, getOrExtractUiParserSource, reloadExternalAdapter } from "../adapters/plugin-loader.js";
|
|
import { logger } from "../middleware/logger.js";
|
|
import { badRequest, forbidden } from "../errors.js";
|
|
import { isCloudManagedInstance } from "../services/cloud-instance.js";
|
|
import { getHiddenSettings } from "../services/settings-visibility.js";
|
|
import { assertBoardOrgAccess, assertCompanyAccess, assertInstanceAdmin } from "./authz.js";
|
|
import { BUILTIN_ADAPTER_TYPES } from "../adapters/builtin-adapter-types.js";
|
|
import { agentRunnerAvailability, type RunnerTarget } from "../services/agent-runner-selection.js";
|
|
import type { AgentRunnerAvailability } from "@paperclipai/shared";
|
|
|
|
const execFileAsync = promisify(execFile);
|
|
|
|
/**
|
|
* Floor: on cloud-managed instances adapter code is bundled into the platform
|
|
* image; fetching and loading external adapter packages at runtime stays off
|
|
* for every actor, including instance admins. Adapter code executes in the
|
|
* server process, so a runtime install would let an instance admin read the
|
|
* platform trust anchors from the process environment (mirrors the
|
|
* bundled-only plugin install floor in plugin-install-guard.ts).
|
|
*/
|
|
function assertAdapterCodeInstallAllowed() {
|
|
if (isCloudManagedInstance()) {
|
|
throw forbidden("Adapter installation is platform-managed on cloud-managed instances", {
|
|
code: "adapter_install_platform_managed",
|
|
});
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Floor: when the hosting operator hides the Adapters settings surface
|
|
* (`instance.adapters` in PAPERCLIP_HIDDEN_SETTINGS), adapter management
|
|
* writes are rejected alongside it. Reads stay open — adapter metadata is
|
|
* consumed by agent-creation UIs outside the hidden page.
|
|
*/
|
|
function assertAdapterManagementVisible() {
|
|
if (getHiddenSettings().has("instance.adapters")) {
|
|
throw forbidden("Adapter management is managed by the hosting operator on this instance", {
|
|
code: "settings_operator_managed",
|
|
});
|
|
}
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Request / Response types
|
|
// ---------------------------------------------------------------------------
|
|
|
|
interface AdapterInstallRequest {
|
|
/** npm package name (e.g., "droid-paperclip-adapter") or local path */
|
|
packageName: string;
|
|
/** True if packageName is a local filesystem path */
|
|
isLocalPath?: boolean;
|
|
/** Target version for npm packages (optional, defaults to latest) */
|
|
version?: string;
|
|
}
|
|
|
|
/**
|
|
* The safe scalar login fields the adapter listing projects to the client. It
|
|
* carries only the panel mode and the timeout policy. It carries no function
|
|
* member and no secret. The user interface reads it to pick the login flow and
|
|
* the login panel. Every login runs on a real pseudo-terminal, so the projection
|
|
* carries no transport field.
|
|
*/
|
|
interface AdapterLoginProjection {
|
|
panelMode: AdapterLoginPanelMode;
|
|
timeoutPolicy: AdapterLoginTimeoutPolicy;
|
|
}
|
|
|
|
interface AdapterCapabilities {
|
|
supportsInstructionsBundle: boolean;
|
|
supportsSkills: boolean;
|
|
supportsLocalAgentJwt: boolean;
|
|
requiresMaterializedRuntimeSkills: boolean;
|
|
supportsAcp: boolean;
|
|
/**
|
|
* The projected login capability. It is present only when the adapter
|
|
* declares an interactive login capability. It is absent otherwise.
|
|
*/
|
|
login?: AdapterLoginProjection;
|
|
}
|
|
|
|
interface AdapterInfo extends AgentRunnerAvailability {
|
|
type: string;
|
|
label: string;
|
|
source: "builtin" | "external";
|
|
modelsCount: number;
|
|
loaded: boolean;
|
|
disabled: boolean;
|
|
capabilities: AdapterCapabilities;
|
|
acp?: ServerAdapterModule["acp"];
|
|
/** True when an external plugin has replaced a built-in adapter of the same type. */
|
|
overriddenBuiltin?: boolean;
|
|
/** True when the external override for a builtin type is currently paused. */
|
|
overridePaused?: boolean;
|
|
version?: string;
|
|
packageName?: string;
|
|
isLocalPath?: boolean;
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Helpers
|
|
// ---------------------------------------------------------------------------
|
|
|
|
/**
|
|
* Resolve the adapter package directory (same rules as plugin-loader).
|
|
*/
|
|
function resolveAdapterPackageDir(record: AdapterPluginRecord): string {
|
|
return record.localPath
|
|
? path.resolve(record.localPath)
|
|
: path.resolve(getAdapterPluginsDir(), "node_modules", record.packageName);
|
|
}
|
|
|
|
/**
|
|
* Read `version` from the adapter's package.json on disk.
|
|
* This is the source of truth for what is actually installed (npm or local path).
|
|
*/
|
|
function readAdapterPackageVersionFromDisk(record: AdapterPluginRecord): string | undefined {
|
|
try {
|
|
const pkgDir = resolveAdapterPackageDir(record);
|
|
const raw = fs.readFileSync(path.join(pkgDir, "package.json"), "utf-8");
|
|
const v = JSON.parse(raw).version;
|
|
return typeof v === "string" && v.trim().length > 0 ? v.trim() : undefined;
|
|
} catch {
|
|
return undefined;
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Build the client capability view for one adapter. The login projection carries
|
|
* only the safe scalar fields; it drops the function members and the completion
|
|
* claim, so the response holds no secret and no code.
|
|
*/
|
|
export function buildAdapterCapabilities(adapter: ServerAdapterModule): AdapterCapabilities {
|
|
const login = adapter.loginCapability;
|
|
return {
|
|
supportsInstructionsBundle: adapter.supportsInstructionsBundle ?? false,
|
|
supportsSkills: Boolean(adapter.listSkills || adapter.syncSkills),
|
|
supportsLocalAgentJwt: adapter.supportsLocalAgentJwt ?? false,
|
|
requiresMaterializedRuntimeSkills: adapter.requiresMaterializedRuntimeSkills ?? false,
|
|
supportsAcp: Boolean(adapter.acp),
|
|
...(login
|
|
? {
|
|
login: {
|
|
panelMode: login.panelMode,
|
|
timeoutPolicy: login.timeoutPolicy,
|
|
},
|
|
}
|
|
: {}),
|
|
};
|
|
}
|
|
|
|
function buildAdapterInfo(adapter: ServerAdapterModule, externalRecord: AdapterPluginRecord | undefined, disabledSet: Set<string>, target?: RunnerTarget): AdapterInfo {
|
|
const fromDisk = externalRecord ? readAdapterPackageVersionFromDisk(externalRecord) : undefined;
|
|
return {
|
|
type: adapter.type,
|
|
label: adapter.type, // ServerAdapterModule doesn't have a separate "label" field; type serves as label
|
|
source: externalRecord ? "external" : "builtin",
|
|
modelsCount: (adapter.models ?? []).length,
|
|
loaded: true, // If it's in the registry, it's loaded
|
|
disabled: disabledSet.has(adapter.type),
|
|
capabilities: buildAdapterCapabilities(adapter),
|
|
...agentRunnerAvailability(adapter.type, target),
|
|
...(adapter.acp ? { acp: adapter.acp } : {}),
|
|
overriddenBuiltin: externalRecord ? BUILTIN_ADAPTER_TYPES.has(adapter.type) : undefined,
|
|
overridePaused: BUILTIN_ADAPTER_TYPES.has(adapter.type) ? isOverridePaused(adapter.type) : undefined,
|
|
// Prefer on-disk package.json so the UI reflects bumps without relying on store-only fields.
|
|
version: fromDisk ?? externalRecord?.version,
|
|
packageName: externalRecord?.packageName,
|
|
isLocalPath: externalRecord?.localPath ? true : undefined,
|
|
};
|
|
}
|
|
|
|
/**
|
|
* Normalize a local path that may be a Windows path into a WSL-compatible path.
|
|
*
|
|
* - Windows paths (e.g., "C:\\Users\\...") are converted via `wslpath -u`.
|
|
* - Paths already starting with `/mnt/` or `/` are returned as-is.
|
|
*/
|
|
async function normalizeLocalPath(rawPath: string): Promise<string> {
|
|
// Already a POSIX path (WSL or native Linux)
|
|
if (rawPath.startsWith("/")) {
|
|
return rawPath;
|
|
}
|
|
|
|
// Windows path detection: C:\ or C:/ pattern
|
|
if (/^[A-Za-z]:[\\/]/.test(rawPath)) {
|
|
try {
|
|
const { stdout } = await execFileAsync("wslpath", ["-u", rawPath]);
|
|
return stdout.trim();
|
|
} catch (err) {
|
|
logger.warn({ err, rawPath }, "wslpath conversion failed; using path as-is");
|
|
return rawPath;
|
|
}
|
|
}
|
|
|
|
return rawPath;
|
|
}
|
|
|
|
/**
|
|
* Register an external adapter module into the server registry via the
|
|
* hot-install path, resolving `sessionManagement` identically to how the
|
|
* init-time IIFE does. Module-provided `sessionManagement` is honored first,
|
|
* with fallback to the host registry by type for builtin-type overrides.
|
|
*
|
|
* Keeps the hot-install and init-time paths at parity so an adapter installed
|
|
* via `POST /api/adapters/install` has the same shape in the registry as the
|
|
* same adapter loaded on the next server restart.
|
|
*/
|
|
function registerWithSessionManagement(adapter: ServerAdapterModule): void {
|
|
registerServerAdapter(resolveExternalAdapterRegistration(adapter));
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Router
|
|
// ---------------------------------------------------------------------------
|
|
|
|
export function adapterRoutes(options: {
|
|
getNativeRunnerEnabled?: () => Promise<boolean>;
|
|
getOpenAiDotEnabled?: () => Promise<boolean>;
|
|
resolveRunnerTarget?: (companyId: string, environmentId?: string) => Promise<RunnerTarget>;
|
|
} = {}) {
|
|
const router = Router();
|
|
|
|
/**
|
|
* GET /api/adapters
|
|
*
|
|
* List all registered adapters (built-in + external).
|
|
* Each entry includes whether the adapter is built-in or external,
|
|
* its model count, and load status.
|
|
*/
|
|
router.get("/adapters", async (_req, res) => {
|
|
// Adapter inventory is needed by ordinary board members when creating or
|
|
// editing company agents. Mutating adapter management routes below remain
|
|
// instance-admin only because they affect the whole server runtime.
|
|
assertBoardOrgAccess(_req);
|
|
|
|
const companyId = _req.query.companyId;
|
|
const environmentId = _req.query.environmentId;
|
|
if ((companyId !== undefined && (typeof companyId !== "string" || !companyId.trim()))
|
|
|| (environmentId !== undefined && (typeof environmentId !== "string" || !environmentId.trim()))) {
|
|
throw badRequest("companyId and environmentId must be nonempty strings.");
|
|
}
|
|
if (environmentId !== undefined && companyId === undefined) throw badRequest("companyId is required when selecting an environment.");
|
|
let target: RunnerTarget | undefined;
|
|
if (typeof companyId === "string") {
|
|
assertCompanyAccess(_req, companyId);
|
|
if (!options.resolveRunnerTarget) throw badRequest("Scoped adapter discovery is unavailable.");
|
|
target = await options.resolveRunnerTarget(companyId, typeof environmentId === "string" ? environmentId : undefined);
|
|
}
|
|
await waitForExternalAdapters();
|
|
|
|
const registeredAdapters = listServerAdapters();
|
|
const externalRecords = new Map(
|
|
listAdapterPlugins().map((r) => [r.type, r]),
|
|
);
|
|
const disabledSet = new Set(getDisabledAdapterTypes());
|
|
// One shared implementation, with independent provider rollouts. Explicit
|
|
// adapter-admin disabling still applies to both choices.
|
|
|
|
const result: AdapterInfo[] = registeredAdapters.map((adapter) =>
|
|
buildAdapterInfo(adapter, externalRecords.get(adapter.type), disabledSet, target),
|
|
).sort((a, b) => a.type.localeCompare(b.type));
|
|
|
|
res.json(result);
|
|
});
|
|
|
|
/**
|
|
* POST /api/adapters/install
|
|
*
|
|
* Install an external adapter from an npm package or local path.
|
|
*
|
|
* Request body:
|
|
* - packageName: string (required) — npm package name or local path
|
|
* - isLocalPath?: boolean (default false)
|
|
* - version?: string — target version for npm packages
|
|
*/
|
|
router.post("/adapters/install", async (req, res) => {
|
|
assertInstanceAdmin(req);
|
|
assertAdapterCodeInstallAllowed();
|
|
assertAdapterManagementVisible();
|
|
|
|
const { packageName, isLocalPath = false, version } = req.body as AdapterInstallRequest;
|
|
|
|
if (!packageName || typeof packageName !== "string") {
|
|
res.status(400).json({ error: "packageName is required and must be a string." });
|
|
return;
|
|
}
|
|
|
|
// Strip version suffix if the UI sends "pkg@1.2.3" instead of separating it
|
|
// e.g. "@henkey/hermes-paperclip-adapter@0.3.0" → packageName + version
|
|
let canonicalName = packageName;
|
|
let explicitVersion = version;
|
|
const versionSuffix = packageName.match(/@(\d+\.\d+\.\d+.*)$/);
|
|
if (versionSuffix) {
|
|
// For scoped packages: "@scope/name@1.2.3" → "@scope/name" + "1.2.3"
|
|
// For unscoped: "name@1.2.3" → "name" + "1.2.3"
|
|
const lastAtIndex = packageName.lastIndexOf("@");
|
|
if (lastAtIndex > 0 && !explicitVersion) {
|
|
canonicalName = packageName.slice(0, lastAtIndex);
|
|
explicitVersion = versionSuffix[1];
|
|
}
|
|
}
|
|
|
|
try {
|
|
let installedVersion: string | undefined;
|
|
let moduleLocalPath: string | undefined;
|
|
|
|
if (!isLocalPath) {
|
|
// npm install into the managed directory
|
|
const pluginsDir = getAdapterPluginsDir();
|
|
const spec = explicitVersion ? `${canonicalName}@${explicitVersion}` : canonicalName;
|
|
|
|
logger.info({ spec, pluginsDir }, "Installing adapter package via npm");
|
|
|
|
await execFileAsync("npm", ["install", "--no-save", spec], {
|
|
cwd: pluginsDir,
|
|
timeout: 120_000,
|
|
});
|
|
|
|
// Read installed version from package.json
|
|
try {
|
|
const pkgJsonPath = path.join(pluginsDir, "node_modules", canonicalName, "package.json");
|
|
const pkgContent = await import("node:fs/promises");
|
|
const pkgRaw = await pkgContent.readFile(pkgJsonPath, "utf-8");
|
|
const pkg = JSON.parse(pkgRaw);
|
|
const v = pkg.version;
|
|
installedVersion =
|
|
typeof v === "string" && v.trim().length > 0 ? v.trim() : explicitVersion;
|
|
} catch {
|
|
installedVersion = explicitVersion;
|
|
}
|
|
} else {
|
|
// Local path — normalize (e.g., Windows → WSL) and use the resolved path
|
|
moduleLocalPath = path.resolve(await normalizeLocalPath(packageName));
|
|
try {
|
|
const pkgRaw = await readFile(path.join(moduleLocalPath, "package.json"), "utf-8");
|
|
const v = JSON.parse(pkgRaw).version;
|
|
if (typeof v === "string" && v.trim().length > 0) {
|
|
installedVersion = v.trim();
|
|
}
|
|
} catch {
|
|
// leave installedVersion undefined if package.json is missing
|
|
}
|
|
}
|
|
|
|
// Load and register the adapter (use canonicalName for path resolution)
|
|
const adapterModule = await loadExternalAdapterPackage(canonicalName, moduleLocalPath);
|
|
|
|
// External adapters may intentionally override built-in adapter types.
|
|
// registerServerAdapter preserves the built-in as a fallback so pausing or
|
|
// removing the override restores the original implementation.
|
|
|
|
// Check if already registered (indicates a reinstall/update).
|
|
// For built-in types the registry always returns the built-in, so we
|
|
// additionally require an existing external plugin record to
|
|
// distinguish a true reinstall from a first-time override.
|
|
const existing = findServerAdapter(adapterModule.type);
|
|
const isReinstall = existing !== null && !!getAdapterPluginByType(adapterModule.type);
|
|
if (existing) {
|
|
unregisterServerAdapter(adapterModule.type);
|
|
logger.info({ type: adapterModule.type }, "Unregistered existing adapter for replacement");
|
|
}
|
|
|
|
// Register the new adapter
|
|
registerWithSessionManagement(adapterModule);
|
|
|
|
// Persist the record (use canonicalName without version suffix)
|
|
const record: AdapterPluginRecord = {
|
|
packageName: canonicalName,
|
|
localPath: moduleLocalPath,
|
|
version: installedVersion ?? explicitVersion,
|
|
type: adapterModule.type,
|
|
installedAt: new Date().toISOString(),
|
|
};
|
|
addAdapterPlugin(record);
|
|
|
|
logger.info(
|
|
{ type: adapterModule.type, packageName: canonicalName },
|
|
"External adapter installed and registered",
|
|
);
|
|
|
|
res.status(201).json({
|
|
type: adapterModule.type,
|
|
packageName: canonicalName,
|
|
version: installedVersion ?? explicitVersion,
|
|
installedAt: record.installedAt,
|
|
requiresRestart: isReinstall,
|
|
});
|
|
} catch (err) {
|
|
const message = err instanceof Error ? err.message : String(err);
|
|
logger.error({ err, packageName }, "Failed to install external adapter");
|
|
|
|
// Distinguish npm errors from load errors
|
|
if (message.includes("npm") || message.includes("ERR!")) {
|
|
res.status(500).json({ error: `npm install failed: ${message}` });
|
|
} else {
|
|
res.status(500).json({ error: `Failed to install adapter: ${message}` });
|
|
}
|
|
}
|
|
});
|
|
|
|
router.get("/adapters/:type", async (req, res) => {
|
|
assertBoardOrgAccess(req);
|
|
|
|
const adapterType = req.params.type;
|
|
const adapter = findServerAdapter(adapterType);
|
|
if (!adapter) {
|
|
res.status(404).json({ error: `Adapter "${adapterType}" is not registered.` });
|
|
return;
|
|
}
|
|
|
|
const externalRecord = getAdapterPluginByType(adapterType);
|
|
const disabledSet = new Set(getDisabledAdapterTypes());
|
|
res.json(buildAdapterInfo(adapter, externalRecord, disabledSet));
|
|
});
|
|
|
|
/**
|
|
* PATCH /api/adapters/:type
|
|
*
|
|
* Enable or disable an adapter. Disabled adapters are hidden from agent
|
|
* creation menus but remain functional for existing agents.
|
|
*
|
|
* Request body: { "disabled": boolean }
|
|
*/
|
|
router.patch("/adapters/:type", async (req, res) => {
|
|
assertInstanceAdmin(req);
|
|
|
|
assertAdapterManagementVisible();
|
|
|
|
const adapterType = req.params.type;
|
|
const { disabled } = req.body as { disabled?: boolean };
|
|
|
|
if (typeof disabled !== "boolean") {
|
|
res.status(400).json({ error: "Request body must include { \"disabled\": true|false }." });
|
|
return;
|
|
}
|
|
|
|
// Check that the adapter exists in the registry
|
|
const existing = findServerAdapter(adapterType);
|
|
if (!existing) {
|
|
res.status(404).json({ error: `Adapter "${adapterType}" is not registered.` });
|
|
return;
|
|
}
|
|
|
|
const changed = setAdapterDisabled(adapterType, disabled);
|
|
|
|
if (changed) {
|
|
logger.info({ type: adapterType, disabled }, "Adapter enabled/disabled");
|
|
}
|
|
|
|
res.json({ type: adapterType, disabled, changed });
|
|
});
|
|
|
|
/**
|
|
* PATCH /api/adapters/:type/override
|
|
*
|
|
* Pause or resume an external adapter's override of a builtin type.
|
|
* When paused, the server returns the builtin adapter for all new requests
|
|
* (execute, listModels, config schema, etc.). Already-running sessions
|
|
* keep the adapter they started with.
|
|
*/
|
|
router.patch("/adapters/:type/override", async (req, res) => {
|
|
assertInstanceAdmin(req);
|
|
|
|
assertAdapterManagementVisible();
|
|
|
|
const adapterType = req.params.type;
|
|
const { paused } = req.body as { paused?: boolean };
|
|
|
|
if (typeof paused !== "boolean") {
|
|
res.status(400).json({ error: "\"paused\" (boolean) is required in request body." });
|
|
return;
|
|
}
|
|
|
|
if (!BUILTIN_ADAPTER_TYPES.has(adapterType)) {
|
|
res.status(400).json({ error: `Type "${adapterType}" is not a builtin adapter.` });
|
|
return;
|
|
}
|
|
|
|
const changed = setOverridePaused(adapterType, paused);
|
|
|
|
logger.info({ type: adapterType, paused, changed }, "Adapter override toggle");
|
|
|
|
res.json({ type: adapterType, paused, changed });
|
|
});
|
|
|
|
/**
|
|
* DELETE /api/adapters/:type
|
|
*
|
|
* Unregister an external adapter. Built-in adapters cannot be removed.
|
|
*/
|
|
router.delete("/adapters/:type", async (req, res) => {
|
|
assertInstanceAdmin(req);
|
|
assertAdapterManagementVisible();
|
|
|
|
const adapterType = req.params.type;
|
|
|
|
if (!adapterType) {
|
|
res.status(400).json({ error: "Adapter type is required." });
|
|
return;
|
|
}
|
|
|
|
// Prevent removal of built-in adapters, unless this built-in type is
|
|
// currently backed by an external adapter plugin override.
|
|
if (BUILTIN_ADAPTER_TYPES.has(adapterType) && !getAdapterPluginByType(adapterType)) {
|
|
res.status(403).json({
|
|
error: `Cannot remove built-in adapter "${adapterType}".`,
|
|
});
|
|
return;
|
|
}
|
|
|
|
// Check that the adapter exists in the registry
|
|
const existing = findServerAdapter(adapterType);
|
|
if (!existing) {
|
|
res.status(404).json({
|
|
error: `Adapter "${adapterType}" is not registered.`,
|
|
});
|
|
return;
|
|
}
|
|
|
|
// Check that it's an external adapter
|
|
const externalRecord = getAdapterPluginByType(adapterType);
|
|
if (!externalRecord) {
|
|
res.status(404).json({
|
|
error: `Adapter "${adapterType}" is not an externally installed adapter.`,
|
|
});
|
|
return;
|
|
}
|
|
|
|
// If installed via npm (has packageName but no localPath), run npm uninstall
|
|
if (externalRecord.packageName && !externalRecord.localPath) {
|
|
try {
|
|
const pluginsDir = getAdapterPluginsDir();
|
|
await execFileAsync("npm", ["uninstall", externalRecord.packageName], {
|
|
cwd: pluginsDir,
|
|
timeout: 60_000,
|
|
});
|
|
logger.info(
|
|
{ type: adapterType, packageName: externalRecord.packageName },
|
|
"npm uninstall completed for external adapter",
|
|
);
|
|
} catch (err) {
|
|
logger.warn(
|
|
{ err, type: adapterType, packageName: externalRecord.packageName },
|
|
"npm uninstall failed for external adapter; continuing with unregister",
|
|
);
|
|
}
|
|
}
|
|
|
|
// Unregister from the runtime registry
|
|
unregisterServerAdapter(adapterType);
|
|
|
|
// Remove from the persistent store
|
|
removeAdapterPlugin(adapterType);
|
|
|
|
logger.info({ type: adapterType }, "External adapter unregistered and removed");
|
|
|
|
res.json({ type: adapterType, removed: true });
|
|
});
|
|
|
|
/**
|
|
* POST /api/adapters/:type/reload
|
|
*
|
|
* Reload an external adapter at runtime (for dev iteration without server restart).
|
|
* Busts the ESM module cache, re-imports the adapter, and re-registers it.
|
|
*
|
|
* Cannot be used on built-in adapter types.
|
|
*/
|
|
router.post("/adapters/:type/reload", async (req, res) => {
|
|
assertInstanceAdmin(req);
|
|
assertAdapterManagementVisible();
|
|
|
|
const type = req.params.type;
|
|
|
|
// Built-in adapters cannot be reloaded unless overridden by an external one
|
|
if (BUILTIN_ADAPTER_TYPES.has(type) && !getAdapterPluginByType(type)) {
|
|
res.status(400).json({ error: "Cannot reload built-in adapter." });
|
|
return;
|
|
}
|
|
|
|
// Reload the adapter module (busts ESM cache, re-imports)
|
|
try {
|
|
const newModule = await reloadExternalAdapter(type);
|
|
|
|
// Not found in the external adapter store
|
|
if (!newModule) {
|
|
res.status(404).json({ error: `Adapter "${type}" is not an externally installed adapter.` });
|
|
return;
|
|
}
|
|
|
|
// Swap in the reloaded module
|
|
unregisterServerAdapter(type);
|
|
registerWithSessionManagement(newModule);
|
|
configSchemaCache.delete(type);
|
|
|
|
// Sync store.version from package.json (store may be missing version for local installs).
|
|
const record = getAdapterPluginByType(type);
|
|
let newVersion: string | undefined;
|
|
if (record) {
|
|
newVersion = readAdapterPackageVersionFromDisk(record);
|
|
if (newVersion) {
|
|
addAdapterPlugin({ ...record, version: newVersion });
|
|
}
|
|
}
|
|
|
|
logger.info({ type, version: newVersion }, "External adapter reloaded at runtime");
|
|
|
|
res.json({ type, version: newVersion, reloaded: true });
|
|
} catch (err) {
|
|
const message = err instanceof Error ? err.message : String(err);
|
|
logger.error({ err, type }, "Failed to reload external adapter");
|
|
res.status(500).json({ error: `Failed to reload adapter: ${message}` });
|
|
}
|
|
});
|
|
|
|
// ── POST /api/adapters/:type/reinstall ──────────────────────────────────
|
|
// Reinstall an npm-sourced external adapter (pulls latest from registry).
|
|
// Local-path adapters cannot be reinstalled — use Reload instead.
|
|
//
|
|
// This is a convenience shortcut for remove + install with the same
|
|
// package name, but without the risk of losing the store record.
|
|
router.post("/adapters/:type/reinstall", async (req, res) => {
|
|
assertInstanceAdmin(req);
|
|
assertAdapterCodeInstallAllowed();
|
|
assertAdapterManagementVisible();
|
|
|
|
const type = req.params.type;
|
|
|
|
if (BUILTIN_ADAPTER_TYPES.has(type) && !getAdapterPluginByType(type)) {
|
|
res.status(400).json({ error: "Cannot reinstall built-in adapter." });
|
|
return;
|
|
}
|
|
|
|
const record = getAdapterPluginByType(type);
|
|
if (!record) {
|
|
res.status(404).json({ error: `Adapter "${type}" is not an externally installed adapter.` });
|
|
return;
|
|
}
|
|
|
|
if (record.localPath) {
|
|
res.status(400).json({ error: "Local-path adapters cannot be reinstalled. Use Reload instead." });
|
|
return;
|
|
}
|
|
|
|
try {
|
|
const pluginsDir = getAdapterPluginsDir();
|
|
|
|
logger.info({ type, packageName: record.packageName }, "Reinstalling adapter package via npm");
|
|
|
|
await execFileAsync("npm", ["install", "--no-save", record.packageName], {
|
|
cwd: pluginsDir,
|
|
timeout: 120_000,
|
|
});
|
|
|
|
// Reload the freshly installed adapter
|
|
const newModule = await reloadExternalAdapter(type);
|
|
if (!newModule) {
|
|
res.status(500).json({ error: "npm install succeeded but adapter reload failed." });
|
|
return;
|
|
}
|
|
|
|
unregisterServerAdapter(type);
|
|
registerWithSessionManagement(newModule);
|
|
configSchemaCache.delete(type);
|
|
|
|
// Sync store version from disk
|
|
let newVersion: string | undefined;
|
|
const updatedRecord = getAdapterPluginByType(type);
|
|
if (updatedRecord) {
|
|
newVersion = readAdapterPackageVersionFromDisk(updatedRecord);
|
|
if (newVersion) {
|
|
addAdapterPlugin({ ...updatedRecord, version: newVersion });
|
|
}
|
|
}
|
|
|
|
logger.info({ type, version: newVersion }, "Adapter reinstalled from npm");
|
|
|
|
res.json({ type, version: newVersion, reinstalled: true });
|
|
} catch (err) {
|
|
const message = err instanceof Error ? err.message : String(err);
|
|
logger.error({ err, type }, "Failed to reinstall adapter");
|
|
res.status(500).json({ error: `Reinstall failed: ${message}` });
|
|
}
|
|
});
|
|
|
|
// ── GET /api/adapters/:type/config-schema ────────────────────────────────
|
|
// Serve a declarative config schema for an adapter's UI form fields.
|
|
// The adapter's getConfigSchema() resolves all options (static and dynamic)
|
|
// so the UI receives a fully hydrated schema in a single fetch.
|
|
const configSchemaCache = new Map<string, {
|
|
adapter: ServerAdapterModule;
|
|
schema: AdapterConfigSchema;
|
|
fetchedAt: number;
|
|
}>();
|
|
const CONFIG_SCHEMA_TTL_MS = 30_000;
|
|
|
|
router.get("/adapters/:type/config-schema", async (req, res) => {
|
|
// Config schemas are read-only form metadata used when org members create
|
|
// or edit agents; they do not install or execute new adapter code.
|
|
assertBoardOrgAccess(req);
|
|
const { type } = req.params;
|
|
|
|
const adapter = findActiveServerAdapter(type);
|
|
if (!adapter) {
|
|
res.status(404).json({ error: `Adapter "${type}" is not registered.` });
|
|
return;
|
|
}
|
|
if (!adapter.getConfigSchema) {
|
|
res.status(404).json({ error: `Adapter "${type}" does not provide a config schema.` });
|
|
return;
|
|
}
|
|
|
|
const cached = configSchemaCache.get(type);
|
|
if (cached && cached.adapter === adapter && Date.now() - cached.fetchedAt < CONFIG_SCHEMA_TTL_MS) {
|
|
res.json(cached.schema);
|
|
return;
|
|
}
|
|
|
|
try {
|
|
const schema = await adapter.getConfigSchema();
|
|
configSchemaCache.set(type, { adapter, schema, fetchedAt: Date.now() });
|
|
res.json(schema);
|
|
} catch (err) {
|
|
const message = err instanceof Error ? err.message : String(err);
|
|
logger.error({ err, type }, "Failed to resolve config schema");
|
|
res.status(500).json({ error: `Failed to resolve config schema: ${message}` });
|
|
}
|
|
});
|
|
|
|
// ── GET /api/adapters/:type/ui-parser.js ─────────────────────────────────
|
|
// Serve the self-contained UI parser JS for an adapter type.
|
|
// This allows external adapters to provide custom run-log parsing
|
|
// without modifying Paperclip's source code.
|
|
//
|
|
// The adapter package must export a "./ui-parser" entry in package.json
|
|
// pointing to a self-contained ESM module with zero runtime dependencies.
|
|
router.get("/adapters/:type/ui-parser.js", (req, res) => {
|
|
// UI parsers are read-only assets for displaying existing run output.
|
|
// Runtime-changing adapter management routes above require instance admin.
|
|
assertBoardOrgAccess(req);
|
|
const { type } = req.params;
|
|
const source = getOrExtractUiParserSource(type);
|
|
if (!source) {
|
|
res.status(404).json({ error: `No UI parser available for adapter "${type}".` });
|
|
return;
|
|
}
|
|
res.type("application/javascript").send(source);
|
|
});
|
|
|
|
return router;
|
|
}
|