mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-31 14:16:39 +01:00
fix(cloud): persist adapter settings in PostgreSQL
Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
1511c92965
commit
33a5c04797
16 files changed
+563
-115
No files matched your search
@@ -722,6 +722,39 @@ connection, before enabling the new server and UI.
|
||||
- The repo keeps only the newest 5 snapshots. `generate` runs `prune:snapshots` afterwards to delete older ones. Drizzle only reads the newest snapshot, and each snapshot is a full copy of the schema (over 1 MB each). Older snapshots are still in git history.
|
||||
- `packages/db/src/migration-snapshot-drift.test.ts` is the enforcement backstop. It repeats the diff that `generate` performs and fails when the newest snapshot no longer matches `packages/db/src/schema/`.
|
||||
|
||||
## Managed adapter settings
|
||||
|
||||
Cloud-managed instances keep adapter selection preferences in the private
|
||||
`instance_settings` singleton `adapter-settings:v1`. Its `general` JSON contains
|
||||
`version: 1`, a monotonic `revision`, and `disabledTypes`. It is separate from
|
||||
the public `default` settings row and never appears in the instance settings API.
|
||||
Each update locks this row in a transaction. The update and company activity
|
||||
records commit together. Discovery and new-agent admission read the database
|
||||
at their asynchronous operation boundaries; no polling or mutable settings cache
|
||||
is used. Disabled adapters remain available to existing agents.
|
||||
|
||||
`PAPERCLIP_ADAPTERS` and `PAPERCLIP_ADAPTERS_FILE` declare the effective operator
|
||||
adapter list. The declaration is parsed once for a process configuration and
|
||||
validated at startup. It does not rewrite stored preferences. A conflicting
|
||||
adapter toggle returns 403. Removing the declaration restores saved choices.
|
||||
Operator file edits require a restart. Self-hosted instances continue to use
|
||||
`adapter-settings.json` for saved preferences.
|
||||
|
||||
The server-only adapter settings service exposes `inspectStored()` and
|
||||
`seedIfAbsent({ disabledTypes }, transaction?)` for migration tooling. Seeding
|
||||
uses the unique singleton key, returns `{ seeded, current }`, and never replaces
|
||||
an existing record. The caller can use the same transaction for its import
|
||||
receipt. Before admission, `initializeAdapterSettings` checks the durable row.
|
||||
When absent, it imports a valid legacy settings file once, retaining unknown
|
||||
adapter types, or seeds empty preferences if no file exists. It rejects files
|
||||
larger than 64 KiB, malformed input, and read failures. An existing durable row
|
||||
wins over every stale file. Ordinary Cloud reads do not read a legacy file.
|
||||
|
||||
The focused PostgreSQL suite can use an owned disposable PostgreSQL database via
|
||||
`PAPERCLIP_ADAPTER_SETTINGS_TEST_DATABASE_URL` when embedded PostgreSQL is
|
||||
unavailable. It applies migrations and inserts test data, so never point this
|
||||
variable at an existing application database.
|
||||
|
||||
## Cloud runtime identity singleton
|
||||
|
||||
The private `instance_settings` row whose singleton key is
|
||||
|
||||
@@ -74,6 +74,9 @@ API: `present` — representative operation IDs:
|
||||
- `PATCH /api/instance/settings`.
|
||||
- `GET /api/instance/settings/experimental`.
|
||||
- `PATCH /api/instance/settings/experimental`.
|
||||
- `GET /api/adapters`.
|
||||
- `GET /api/adapters/{type}`.
|
||||
- `PATCH /api/adapters/{type}`.
|
||||
|
||||
Agent access: `restricted` — Board/admin instance settings; CLI configure/env; local
|
||||
configuration files.
|
||||
@@ -91,6 +94,15 @@ own policy and access review.
|
||||
Gaps: Settings APIs cover parts of configuration; pre-start database/binding/storage
|
||||
configuration is local operator tooling.
|
||||
|
||||
Adapter toggles require instance-admin authority and remain blocked when
|
||||
`instance.adapters` is hidden. Ordinary board members can read the catalog;
|
||||
agent keys cannot toggle adapters. Managed Cloud preferences are stored in a
|
||||
private database singleton. The operator adapter declaration sets the effective
|
||||
list and rejects conflicting toggles. [PostgreSQL regression coverage](../../server/src/services/adapter-settings-postgres.test.ts)
|
||||
exercises persistence, concurrent writers, admission freshness, activity audit,
|
||||
and failed writes; [authorization tests](../../server/src/__tests__/adapter-routes-authz.test.ts)
|
||||
cover admin and hidden-setting gates. These are local tests, not deployed proof.
|
||||
|
||||
Cloud stack Admins may update experimental settings through the trusted Cloud
|
||||
actor without receiving other instance-admin powers. This uses the current
|
||||
stack role on each request; company membership alone does not grant it.
|
||||
|
||||
@@ -65,6 +65,16 @@ unfinished application write must still prevent sleep after its client leaves.
|
||||
|
||||
Automated: [experimental settings](../../ui/src/pages/InstanceExperimentalSettings.test.tsx) and [database backup routes](../../server/src/__tests__/instance-database-backups-routes.test.ts) cover their UI/API contracts.
|
||||
|
||||
Managed adapters: change a visible adapter preference as an instance admin,
|
||||
then use a second application process against the same disposable database to
|
||||
confirm catalog and new-agent selection reflect it. Verify saved agents remain
|
||||
usable. Restart with an operator adapter declaration, confirm its effective
|
||||
list, and reject conflicting toggles. Remove the declaration and confirm saved
|
||||
preferences return. [Database tests](../../server/src/services/adapter-settings-postgres.test.ts)
|
||||
cover local persistence, concurrent writers, audit rollback, and failure responses;
|
||||
[local compatibility tests](../../server/src/services/adapter-settings-local.test.ts)
|
||||
cover the legacy file. No deployed walkthrough is claimed.
|
||||
|
||||
Manual: Change a harmless experimental setting, reload, and confirm the affected route honors it. Back up a disposable instance, inspect the reported file and metadata, and restore using the documented procedure into a second isolated target. Verify a known record after restoration.
|
||||
|
||||
Cloud staging: On a managed `*.staging.paperclip.app` instance, verify experiments
|
||||
|
||||
@@ -136,7 +136,6 @@ import {
|
||||
import { agentConfigurationDoc as piAgentConfigurationDoc } from "@paperclipai/adapter-pi-local";
|
||||
import { BUILTIN_ADAPTER_TYPES } from "./builtin-adapter-types.js";
|
||||
import { buildExternalAdapters } from "./plugin-loader.js";
|
||||
import { getDisabledAdapterTypes } from "../services/adapter-plugin-store.js";
|
||||
import { NativeEnvironmentProbeCleanupError } from "../vendor/paperclip-runner/index.js";
|
||||
import { processAdapter } from "./process/index.js";
|
||||
import { httpAdapter } from "./http/index.js";
|
||||
@@ -997,11 +996,6 @@ registerBuiltInAdapters();
|
||||
// applies the registry fallback for externals overriding a built-in type.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Cached sync wrapper — the store is a simple JSON file read, safe to call frequently. */
|
||||
function getDisabledAdapterTypesFromStore(): string[] {
|
||||
return getDisabledAdapterTypes();
|
||||
}
|
||||
|
||||
/**
|
||||
* Merge an external adapter module with host-provided session management.
|
||||
*
|
||||
@@ -1186,8 +1180,7 @@ export function listServerAdapters(): ServerAdapterModule[] {
|
||||
* Used for menus and agent creation flows — disabled adapters remain
|
||||
* functional for existing agents but hidden from selection.
|
||||
*/
|
||||
export function listEnabledServerAdapters(): ServerAdapterModule[] {
|
||||
const disabled = getDisabledAdapterTypesFromStore();
|
||||
export function listEnabledServerAdapters(disabled: string[]): ServerAdapterModule[] {
|
||||
const disabledSet = disabled.length > 0 ? new Set(disabled) : null;
|
||||
return disabledSet
|
||||
? Array.from(adaptersByType.values()).filter((a) => !disabledSet.has(a.type))
|
||||
|
||||
@@ -965,6 +965,7 @@ export async function createApp(
|
||||
);
|
||||
api.use(
|
||||
adapterRoutes({
|
||||
db,
|
||||
resolveRunnerTarget: (companyId, environmentId) => resolveAgentRunnerTargetForCompany(db, companyId, environmentId),
|
||||
getNativeRunnerEnabled: async () =>
|
||||
(await instanceSettingsService(db).getExperimental())
|
||||
|
||||
@@ -186,6 +186,7 @@ import {
|
||||
loadWithoutCoordinatedShutdownSignalHooks,
|
||||
} from "./shutdown.js";
|
||||
import { initializeCloudRuntimeIdentity } from "./services/cloud-runtime-identity.js";
|
||||
import { initializeAdapterSettings } from "./services/adapter-settings.js";
|
||||
import { systemdNotify } from "./services/systemd-notify.js";
|
||||
import { flushInFlightRunLogMirrors } from "./services/run-log-store.js";
|
||||
import {
|
||||
@@ -711,6 +712,7 @@ async function startServerWithDatabaseTeardown(
|
||||
};
|
||||
startupDatabase.close = closeDatabaseClients;
|
||||
setSentryRunLinkResolver(createSentryRunLinkResolver(db));
|
||||
await initializeAdapterSettings(db as any);
|
||||
|
||||
// A claimed warm-pool stack may restart while its provider environment still
|
||||
// names the pool host. Restore the signed, durable identity before Better
|
||||
|
||||
@@ -19,11 +19,13 @@ import { readFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
import { Router } from "express";
|
||||
import { companies, type Db } from "@paperclipai/db";
|
||||
import { adapterSettingsService } from "../services/adapter-settings.js";
|
||||
import { logActivity, publishActivity, type ActivityPublication } from "../services/activity-log.js";
|
||||
import {
|
||||
listServerAdapters,
|
||||
findServerAdapter,
|
||||
findActiveServerAdapter,
|
||||
listEnabledServerAdapters,
|
||||
registerServerAdapter,
|
||||
resolveExternalAdapterRegistration,
|
||||
unregisterServerAdapter,
|
||||
@@ -37,8 +39,6 @@ import {
|
||||
removeAdapterPlugin,
|
||||
getAdapterPluginByType,
|
||||
getAdapterPluginsDir,
|
||||
getDisabledAdapterTypes,
|
||||
setAdapterDisabled,
|
||||
} from "../services/adapter-plugin-store.js";
|
||||
import type { AdapterPluginRecord } from "../services/adapter-plugin-store.js";
|
||||
import type { ServerAdapterModule, AdapterConfigSchema } from "../adapters/types.js";
|
||||
@@ -51,7 +51,7 @@ import { logger } from "../middleware/logger.js";
|
||||
import { badRequest, forbidden } from "../errors.js";
|
||||
import { isCloudManagedInstance } from "../services/cloud-instance.js";
|
||||
import { getHiddenSettings } from "../services/settings-visibility.js";
|
||||
import { assertBoardOrgAccess, assertCompanyAccess, assertInstanceAdmin } from "./authz.js";
|
||||
import { assertBoardOrgAccess, assertCompanyAccess, assertInstanceAdmin, getActorInfo } from "./authz.js";
|
||||
import { BUILTIN_ADAPTER_TYPES } from "../adapters/builtin-adapter-types.js";
|
||||
import { agentRunnerAvailability, type RunnerTarget } from "../services/agent-runner-selection.js";
|
||||
import type { AgentRunnerAvailability } from "@paperclipai/shared";
|
||||
@@ -209,7 +209,7 @@ function buildAdapterInfo(adapter: ServerAdapterModule, externalRecord: AdapterP
|
||||
loaded: true, // If it's in the registry, it's loaded
|
||||
disabled: disabledSet.has(adapter.type),
|
||||
capabilities: buildAdapterCapabilities(adapter),
|
||||
...agentRunnerAvailability(adapter.type, target),
|
||||
...agentRunnerAvailability(adapter.type, target, [...disabledSet]),
|
||||
...(adapter.acp ? { acp: adapter.acp } : {}),
|
||||
overriddenBuiltin: externalRecord ? BUILTIN_ADAPTER_TYPES.has(adapter.type) : undefined,
|
||||
overridePaused: BUILTIN_ADAPTER_TYPES.has(adapter.type) ? isOverridePaused(adapter.type) : undefined,
|
||||
@@ -265,11 +265,13 @@ function registerWithSessionManagement(adapter: ServerAdapterModule): void {
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function adapterRoutes(options: {
|
||||
db?: Db;
|
||||
getNativeRunnerEnabled?: () => Promise<boolean>;
|
||||
getOpenAiDotEnabled?: () => Promise<boolean>;
|
||||
resolveRunnerTarget?: (companyId: string, environmentId?: string) => Promise<RunnerTarget>;
|
||||
} = {}) {
|
||||
const router = Router();
|
||||
const settings = adapterSettingsService(options.db);
|
||||
|
||||
/**
|
||||
* GET /api/adapters
|
||||
@@ -303,7 +305,7 @@ export function adapterRoutes(options: {
|
||||
const externalRecords = new Map(
|
||||
listAdapterPlugins().map((r) => [r.type, r]),
|
||||
);
|
||||
const disabledSet = new Set(getDisabledAdapterTypes());
|
||||
const disabledSet = new Set(await settings.getDisabledTypes());
|
||||
// One shared implementation, with independent provider rollouts. Explicit
|
||||
// adapter-admin disabling still applies to both choices.
|
||||
|
||||
@@ -477,7 +479,7 @@ export function adapterRoutes(options: {
|
||||
}
|
||||
|
||||
const externalRecord = getAdapterPluginByType(adapterType);
|
||||
const disabledSet = new Set(getDisabledAdapterTypes());
|
||||
const disabledSet = new Set(await settings.getDisabledTypes());
|
||||
res.json(buildAdapterInfo(adapter, externalRecord, disabledSet));
|
||||
});
|
||||
|
||||
@@ -509,7 +511,22 @@ export function adapterRoutes(options: {
|
||||
return;
|
||||
}
|
||||
|
||||
const changed = setAdapterDisabled(adapterType, disabled);
|
||||
const publications: ActivityPublication[] = [];
|
||||
const changed = await settings.setDisabled(adapterType, disabled, async (tx, current) => {
|
||||
const actor = getActorInfo(req);
|
||||
const companyIds = await tx.select({ id: companies.id }).from(companies);
|
||||
for (const company of companyIds) {
|
||||
await logActivity(tx as Db, {
|
||||
companyId: company.id,
|
||||
...actor,
|
||||
action: "instance.adapter.updated",
|
||||
entityType: "instance_settings",
|
||||
entityId: "adapter-settings:v1",
|
||||
details: { adapterType, disabled, revision: current.revision },
|
||||
}, publications);
|
||||
}
|
||||
});
|
||||
for (const publication of publications) publishActivity(publication);
|
||||
|
||||
if (changed) {
|
||||
logger.info({ type: adapterType, disabled }, "Adapter enabled/disabled");
|
||||
|
||||
@@ -154,7 +154,7 @@ import type {
|
||||
} from "@paperclipai/adapter-utils";
|
||||
import { evaluateCodexCredentialReadiness } from "@paperclipai/adapter-codex-local/server";
|
||||
import type { AdapterAuthSignal, AdapterAuthSignalResponse, CodexAccountBindingClaim } from "@paperclipai/shared";
|
||||
import { getDisabledAdapterTypes } from "../services/adapter-plugin-store.js";
|
||||
import { adapterSettingsService } from "../services/adapter-settings.js";
|
||||
import { skillVersionSelectionMap } from "../services/runtime-skill-selections.js";
|
||||
import { isFixedClaudeOAuthBinding, secretService } from "../services/secrets.js";
|
||||
import { authorizationDeniedDetails } from "../services/authorization.js";
|
||||
@@ -1859,15 +1859,6 @@ export function agentRoutes(
|
||||
* Adapter validation for the paths that CHOOSE a harness for a new agent
|
||||
* (hire + create), as opposed to the paths that operate on an existing one.
|
||||
*
|
||||
* A disabled adapter is one this instance cannot run — most often because a
|
||||
* declarative registry (PAPERCLIP_ADAPTERS) curated it out, which
|
||||
* reconcileAdapterAvailability turns into a disabled type at boot. Registered
|
||||
* but disabled still passes assertKnownAdapterType, so an agent could be
|
||||
* created on it and then fail EVERY run at lease time with
|
||||
* `Adapter "..." is not in the configured adapter registry` — an error that
|
||||
* arrives minutes later, in a run log, with no way back to the choice that
|
||||
* caused it. Refuse at selection time instead, and name what can be chosen.
|
||||
*
|
||||
* Existing agents on a now-disabled adapter are deliberately untouched
|
||||
* (listEnabledServerAdapters documents the same rule: hidden from selection,
|
||||
* still functional for agents that already use them).
|
||||
@@ -1902,7 +1893,7 @@ export function agentRoutes(
|
||||
);
|
||||
}
|
||||
}
|
||||
const disabled = new Set(getDisabledAdapterTypes());
|
||||
const disabled = new Set(await adapterSettingsService(db).getDisabledTypes());
|
||||
if (!disabled.has(adapterType)) return adapterType;
|
||||
const available = listServerAdapters()
|
||||
.map((a) => a.type)
|
||||
|
||||
@@ -32,8 +32,7 @@ describe("reconcileAdapterAvailability", () => {
|
||||
]);
|
||||
expect(result.enabled).toEqual(["opencode_local"]);
|
||||
expect(result.disabled.sort()).toEqual(["claude_local", "pi_local"]);
|
||||
expect(setAdapterDisabled).toHaveBeenCalledWith("claude_local", true);
|
||||
expect(setAdapterDisabled).toHaveBeenCalledWith("opencode_local", false);
|
||||
expect(setAdapterDisabled).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("throws when a declared adapter has no installed implementation", () => {
|
||||
|
||||
@@ -1,73 +1,8 @@
|
||||
/**
|
||||
* Declarative adapter-registry bootstrap.
|
||||
*
|
||||
* One source (`PAPERCLIP_ADAPTERS` inline JSON, or `PAPERCLIP_ADAPTERS_FILE` a
|
||||
* path to a JSON file) feeds two consumers:
|
||||
* 1. Availability: reconcile the file-backed disabled-set so the picker shows
|
||||
* exactly the declared, enabled set (runs for any instance).
|
||||
* 2. k8s runtime: the same registry rides on the Kubernetes environment config
|
||||
* (see execution-policy-bootstrap) so the plugin resolves runtime
|
||||
* image/envKeys/allowFqdns/probe/defaultEnv from it.
|
||||
*
|
||||
* Parsing is pure + fails loud on malformed/invalid config, mirroring
|
||||
* execution-policy-bootstrap.
|
||||
*/
|
||||
import fs from "node:fs";
|
||||
import { adapterRegistrySchema, type AdapterRegistryEntryParsed } from "@paperclipai/shared";
|
||||
import type { AdapterRegistryEntryParsed } from "@paperclipai/shared";
|
||||
import { logger } from "../middleware/logger.js";
|
||||
import { listServerAdapters } from "../adapters/registry.js";
|
||||
import { setAdapterDisabled } from "./adapter-plugin-store.js";
|
||||
export { parseAdapterRegistryEnv, type AdapterRegistryEnv } from "./adapter-registry-config.js";
|
||||
|
||||
export type AdapterRegistryEnv = Record<string, string | undefined>;
|
||||
|
||||
/**
|
||||
* Parse the declarative registry from env. Returns null when unconfigured
|
||||
* (built-in defaults; local/OSS unchanged). Throws on malformed/invalid input.
|
||||
*/
|
||||
export function parseAdapterRegistryEnv(
|
||||
env: AdapterRegistryEnv = process.env,
|
||||
): AdapterRegistryEntryParsed[] | null {
|
||||
const inline = env.PAPERCLIP_ADAPTERS?.trim();
|
||||
const filePath = env.PAPERCLIP_ADAPTERS_FILE?.trim();
|
||||
if (!inline && !filePath) return null;
|
||||
|
||||
let rawText: string;
|
||||
if (inline) {
|
||||
rawText = inline;
|
||||
} else {
|
||||
try {
|
||||
rawText = fs.readFileSync(filePath as string, "utf-8");
|
||||
} catch (err) {
|
||||
throw new Error(
|
||||
`PAPERCLIP_ADAPTERS_FILE could not be read at "${filePath}": ${(err as Error).message}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(rawText);
|
||||
} catch (err) {
|
||||
throw new Error(`PAPERCLIP_ADAPTERS must be valid JSON: ${(err as Error).message}`);
|
||||
}
|
||||
|
||||
const result = adapterRegistrySchema.safeParse(parsed);
|
||||
if (!result.success) {
|
||||
throw new Error(
|
||||
`PAPERCLIP_ADAPTERS failed validation: ${result.error.issues
|
||||
.map((i) => `${i.path.join(".")}: ${i.message}`)
|
||||
.join("; ")}`,
|
||||
);
|
||||
}
|
||||
return result.data;
|
||||
}
|
||||
|
||||
/**
|
||||
* Reconcile availability: every known server adapter NOT enabled in the declared
|
||||
* registry is disabled; declared+enabled ones are enabled. Throws if a declared
|
||||
* adapterType has no installed adapter (cannot offer a harness with no
|
||||
* implementation). No-op when `registry` is null.
|
||||
*/
|
||||
export function reconcileAdapterAvailability(
|
||||
registry: AdapterRegistryEntryParsed[] | null,
|
||||
): { enabled: string[]; disabled: string[] } {
|
||||
@@ -88,10 +23,9 @@ export function reconcileAdapterAvailability(
|
||||
for (const type of knownTypes) {
|
||||
const entry = declared.get(type);
|
||||
const shouldEnable = entry !== undefined && entry.enabled !== false;
|
||||
setAdapterDisabled(type, !shouldEnable);
|
||||
(shouldEnable ? enabled : disabled).push(type);
|
||||
}
|
||||
|
||||
logger.info({ enabled, disabled }, "reconciled adapter availability from PAPERCLIP_ADAPTERS");
|
||||
logger.info({ enabled, disabled }, "validated adapter availability from PAPERCLIP_ADAPTERS");
|
||||
return { enabled, disabled };
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
import fs from "node:fs";
|
||||
import { adapterRegistrySchema, type AdapterRegistryEntryParsed } from "@paperclipai/shared";
|
||||
|
||||
export type AdapterRegistryEnv = Record<string, string | undefined>;
|
||||
|
||||
let cached: { inline: string | undefined; filePath: string | undefined; registry: AdapterRegistryEntryParsed[] } | undefined;
|
||||
|
||||
export function parseAdapterRegistryEnv(
|
||||
env: AdapterRegistryEnv = process.env,
|
||||
): AdapterRegistryEntryParsed[] | null {
|
||||
const inline = env.PAPERCLIP_ADAPTERS?.trim();
|
||||
const filePath = env.PAPERCLIP_ADAPTERS_FILE?.trim();
|
||||
if (!inline && !filePath) return null;
|
||||
if (cached && cached.inline === inline && cached.filePath === filePath) return structuredClone(cached.registry);
|
||||
|
||||
let rawText: string;
|
||||
if (inline) {
|
||||
rawText = inline;
|
||||
} else {
|
||||
try {
|
||||
rawText = fs.readFileSync(filePath as string, "utf-8");
|
||||
} catch (err) {
|
||||
throw new Error(
|
||||
`PAPERCLIP_ADAPTERS_FILE could not be read at "${filePath}": ${(err as Error).message}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(rawText);
|
||||
} catch (err) {
|
||||
throw new Error(`PAPERCLIP_ADAPTERS must be valid JSON: ${(err as Error).message}`);
|
||||
}
|
||||
|
||||
const result = adapterRegistrySchema.safeParse(parsed);
|
||||
if (!result.success) {
|
||||
throw new Error(
|
||||
`PAPERCLIP_ADAPTERS failed validation: ${result.error.issues
|
||||
.map((i) => `${i.path.join(".")}: ${i.message}`)
|
||||
.join("; ")}`,
|
||||
);
|
||||
}
|
||||
cached = { inline, filePath, registry: result.data };
|
||||
return structuredClone(result.data);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
vi.mock("../adapters/registry.js", () => ({ listServerAdapters: () => [{ type: "codex_local" }, { type: "claude_local" }] }));
|
||||
import { adapterSettingsService } from "./adapter-settings.js";
|
||||
import { getDisabledAdapterTypes, setAdapterDisabled } from "./adapter-plugin-store.js";
|
||||
import { parseAdapterRegistryEnv } from "./adapter-registry-config.js";
|
||||
|
||||
describe("local adapter settings compatibility", () => {
|
||||
let directory: string;
|
||||
beforeEach(() => {
|
||||
directory = fs.mkdtempSync(path.join(os.tmpdir(), "adapter-settings-local-"));
|
||||
vi.stubEnv("PAPERCLIP_HOME", directory);
|
||||
vi.stubEnv("PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", undefined);
|
||||
vi.stubEnv("PAPERCLIP_MANAGED_CONFIG", undefined);
|
||||
vi.stubEnv("PAPERCLIP_ADAPTERS", undefined);
|
||||
vi.stubEnv("PAPERCLIP_ADAPTERS_FILE", undefined);
|
||||
});
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
vi.unstubAllEnvs();
|
||||
fs.rmSync(directory, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("reads and writes the existing file without a database", async () => {
|
||||
fs.writeFileSync(path.join(directory, "adapter-settings.json"), JSON.stringify({ disabledTypes: ["claude_local"] }));
|
||||
const service = adapterSettingsService(undefined);
|
||||
expect(await service.getDisabledTypes()).toEqual(["claude_local"]);
|
||||
expect(await service.setDisabled("codex_local", true)).toBe(true);
|
||||
expect(getDisabledAdapterTypes()).toEqual(["claude_local", "codex_local"]);
|
||||
expect(setAdapterDisabled("claude_local", false)).toBe(true);
|
||||
expect(await service.getDisabledTypes()).toEqual(["codex_local"]);
|
||||
expect(JSON.parse(fs.readFileSync(path.join(directory, "adapter-settings.json"), "utf8"))).toEqual({ disabledTypes: ["codex_local"] });
|
||||
});
|
||||
|
||||
it("loads an operator file once and leaves the saved file intact", async () => {
|
||||
setAdapterDisabled("codex_local", true);
|
||||
const file = path.join(directory, "operator-adapters.json");
|
||||
fs.writeFileSync(file, JSON.stringify([{ adapterType: "codex_local", enabled: true }]));
|
||||
const env = { PAPERCLIP_ADAPTERS_FILE: file };
|
||||
parseAdapterRegistryEnv(env);
|
||||
const read = vi.spyOn(fs, "readFileSync");
|
||||
const service = adapterSettingsService(undefined, { runtimeEnv: env });
|
||||
expect(await service.getDisabledTypes()).toEqual(["claude_local"]);
|
||||
expect(await service.getDisabledTypes()).toEqual(["claude_local"]);
|
||||
expect(read).not.toHaveBeenCalled();
|
||||
expect(getDisabledAdapterTypes()).toEqual(["codex_local"]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,215 @@
|
||||
import express from "express";
|
||||
import request from "supertest";
|
||||
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { activityLog, applyPendingMigrations, companies, createDb, instanceSettings, type Db } from "@paperclipai/db";
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../__tests__/helpers/embedded-postgres.js";
|
||||
|
||||
const legacy = vi.hoisted(() => ({ get: vi.fn(() => [] as string[]), set: vi.fn() }));
|
||||
vi.mock("./adapter-plugin-store.js", () => ({
|
||||
getDisabledAdapterTypes: legacy.get,
|
||||
setAdapterDisabled: legacy.set,
|
||||
listAdapterPlugins: () => [],
|
||||
getAdapterPluginByType: () => undefined,
|
||||
}));
|
||||
vi.mock("../adapters/registry.js", () => ({
|
||||
listServerAdapters: () => [{ type: "codex_local" }, { type: "claude_local" }, { type: "paperclip_runner" }],
|
||||
findServerAdapter: (type: string) => ({ type }),
|
||||
findActiveServerAdapter: (type: string) => ({ type }),
|
||||
isOverridePaused: () => false,
|
||||
hasActiveAdapterOverride: () => false,
|
||||
waitForExternalAdapters: async () => {},
|
||||
}));
|
||||
vi.mock("../adapters/plugin-loader.js", () => ({}));
|
||||
import { adapterSettingsService, initializeAdapterSettings } from "./adapter-settings.js";
|
||||
import { adapterRoutes } from "../routes/adapters.js";
|
||||
import { errorHandler } from "../middleware/error-handler.js";
|
||||
import { resolveNewAgentRunnerForCompany } from "./agent-runner-selection.js";
|
||||
|
||||
const externalDatabaseUrl = process.env.PAPERCLIP_ADAPTER_SETTINGS_TEST_DATABASE_URL;
|
||||
const support = externalDatabaseUrl ? { supported: true } : await getEmbeddedPostgresTestSupport();
|
||||
(support.supported ? describe : describe.skip)("durable managed adapter settings", () => {
|
||||
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
|
||||
let db: Db;
|
||||
let peer: Db;
|
||||
let directory: string;
|
||||
const managed = { runtimeEnv: { PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN: "synthetic-test-token" } };
|
||||
const service = () => adapterSettingsService(db, managed);
|
||||
|
||||
beforeAll(async () => {
|
||||
database = externalDatabaseUrl
|
||||
? { connectionString: externalDatabaseUrl, cleanup: async () => {} }
|
||||
: await startEmbeddedPostgresTestDatabase("adapter-settings-");
|
||||
if (externalDatabaseUrl) await applyPendingMigrations(externalDatabaseUrl);
|
||||
db = createDb(database.connectionString);
|
||||
peer = createDb(database.connectionString);
|
||||
await db.insert(companies).values({ name: "Adapter settings test", issuePrefix: "AST" });
|
||||
}, 90_000);
|
||||
afterAll(async () => {
|
||||
await Promise.all([db?.$client.end({ timeout: 0 }), peer?.$client.end({ timeout: 0 })]);
|
||||
await database?.cleanup();
|
||||
}, 60_000);
|
||||
beforeEach(async () => {
|
||||
directory = await fs.mkdtemp(path.join(os.tmpdir(), "adapter-settings-import-"));
|
||||
await db.delete(instanceSettings).where(eq(instanceSettings.singletonKey, "adapter-settings:v1"));
|
||||
legacy.get.mockClear();
|
||||
legacy.set.mockClear();
|
||||
vi.stubEnv("PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", "synthetic-test-token");
|
||||
});
|
||||
afterEach(async () => {
|
||||
vi.unstubAllEnvs();
|
||||
await fs.rm(directory, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
function app() {
|
||||
const app = express();
|
||||
app.use(express.json());
|
||||
app.use((req, _res, next) => {
|
||||
req.actor = { type: "board", source: "local_implicit", userId: "local-board", isInstanceAdmin: true };
|
||||
next();
|
||||
});
|
||||
app.use(adapterRoutes({ db }));
|
||||
app.use(errorHandler);
|
||||
return app;
|
||||
}
|
||||
|
||||
it("persists across independent connections and reconstructed stores without file reads", async () => {
|
||||
await expect(service().getDisabledTypes()).rejects.toThrow("not been initialized");
|
||||
await service().seedIfAbsent({ disabledTypes: [] });
|
||||
expect(await service().setDisabled("codex_local", true)).toBe(true);
|
||||
expect(await adapterSettingsService(peer, managed).getDisabledTypes()).toEqual(["codex_local"]);
|
||||
await db.$client.end({ timeout: 0 });
|
||||
db = createDb(database.connectionString);
|
||||
expect(await service().inspectStored()).toEqual({ version: 1, revision: 1, disabledTypes: ["codex_local"] });
|
||||
expect(legacy.get).not.toHaveBeenCalled();
|
||||
expect(legacy.set).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("preserves simultaneous first writes and concurrent removals from other connections", async () => {
|
||||
const writers = [service(), adapterSettingsService(peer, managed)];
|
||||
await Promise.all(Array.from({ length: 12 }, (_, index) => writers[index % 2].setDisabled(`adapter_${index}`, true)));
|
||||
expect((await service().inspectStored())?.disabledTypes).toHaveLength(12);
|
||||
await Promise.all([
|
||||
writers[0].setDisabled("adapter_0", false),
|
||||
writers[1].setDisabled("adapter_1", false),
|
||||
writers[0].setDisabled("extra", true),
|
||||
]);
|
||||
const stored = await service().inspectStored();
|
||||
expect(stored?.revision).toBe(15);
|
||||
expect(stored?.disabledTypes).not.toContain("adapter_0");
|
||||
expect(stored?.disabledTypes).not.toContain("adapter_1");
|
||||
expect(stored?.disabledTypes).toContain("extra");
|
||||
expect(await writers[1].setDisabled("extra", true)).toBe(false);
|
||||
expect((await service().inspectStored())?.revision).toBe(15);
|
||||
});
|
||||
|
||||
it("imports legacy choices once before admission and ignores stale files after the DB record exists", async () => {
|
||||
const runtimeEnv = { ...managed.runtimeEnv, PAPERCLIP_HOME: directory };
|
||||
const file = path.join(directory, "adapter-settings.json");
|
||||
await fs.writeFile(file, JSON.stringify({ disabledTypes: ["codex_local", "future_adapter"] }));
|
||||
const results = await Promise.all([
|
||||
initializeAdapterSettings(db, { runtimeEnv }),
|
||||
initializeAdapterSettings(peer, { runtimeEnv }),
|
||||
]);
|
||||
expect(results.filter((result) => result?.seeded)).toHaveLength(1);
|
||||
expect(await service().getDisabledTypes()).toEqual(["codex_local", "future_adapter"]);
|
||||
await service().setDisabled("codex_local", false);
|
||||
await fs.writeFile(file, "invalid stale JSON");
|
||||
expect((await initializeAdapterSettings(peer, { runtimeEnv }))?.seeded).toBe(false);
|
||||
expect(await service().getDisabledTypes()).toEqual(["future_adapter"]);
|
||||
});
|
||||
|
||||
it("uses defaults only for an absent legacy file and rejects malformed or oversized input", async () => {
|
||||
const runtimeEnv = { ...managed.runtimeEnv, PAPERCLIP_HOME: directory };
|
||||
const file = path.join(directory, "adapter-settings.json");
|
||||
await fs.writeFile(file, JSON.stringify({ disabledTypes: [42] }));
|
||||
await expect(initializeAdapterSettings(db, { runtimeEnv })).rejects.toThrow("Persisted adapter settings are invalid");
|
||||
expect(await service().inspectStored()).toBeNull();
|
||||
await fs.writeFile(file, Buffer.alloc(65_537));
|
||||
await expect(initializeAdapterSettings(db, { runtimeEnv })).rejects.toThrow("size limit");
|
||||
expect(await service().inspectStored()).toBeNull();
|
||||
await fs.unlink(file);
|
||||
expect((await initializeAdapterSettings(db, { runtimeEnv }))?.current.disabledTypes).toEqual([]);
|
||||
expect(await service().getDisabledTypes()).toEqual([]);
|
||||
});
|
||||
|
||||
it("composes atomic import and receipts with the caller transaction", async () => {
|
||||
await expect(db.transaction(async (tx) => {
|
||||
expect(await service().seedIfAbsent({ disabledTypes: ["codex_local"] }, tx)).toEqual({
|
||||
seeded: true, current: { version: 1, revision: 0, disabledTypes: ["codex_local"] },
|
||||
});
|
||||
throw new Error("receipt failed");
|
||||
})).rejects.toThrow("receipt failed");
|
||||
expect(await service().inspectStored()).toBeNull();
|
||||
await Promise.all([
|
||||
service().seedIfAbsent({ disabledTypes: ["codex_local"] }),
|
||||
adapterSettingsService(peer, managed).seedIfAbsent({ disabledTypes: ["claude_local"] }),
|
||||
]).then((results) => {
|
||||
expect(results.filter((result) => result.seeded)).toHaveLength(1);
|
||||
expect(results[0].current).toEqual(results[1].current);
|
||||
});
|
||||
});
|
||||
|
||||
it("applies operator restrictions without rewriting saved choices", async () => {
|
||||
await service().setDisabled("codex_local", true);
|
||||
const env = { ...managed.runtimeEnv, PAPERCLIP_ADAPTERS: JSON.stringify([{ adapterType: "codex_local", enabled: true }]) };
|
||||
const overlay = adapterSettingsService(peer, { runtimeEnv: env });
|
||||
expect(await overlay.getDisabledTypes()).toEqual(["claude_local", "paperclip_runner"]);
|
||||
expect(await overlay.inspectStored()).toEqual(await service().inspectStored());
|
||||
await expect(overlay.setDisabled("claude_local", false)).rejects.toThrow("hosting operator");
|
||||
await expect(overlay.setDisabled("codex_local", true)).rejects.toThrow("hosting operator");
|
||||
expect(await overlay.getDisabledTypes()).toEqual(["claude_local", "paperclip_runner"]);
|
||||
expect(await service().getDisabledTypes()).toEqual(["codex_local"]);
|
||||
});
|
||||
|
||||
it("refreshes the disabled harness at each new-agent admission", async () => {
|
||||
await service().seedIfAbsent({ disabledTypes: [] });
|
||||
const input = { adapterType: "codex_local", runner: "legacy" as const };
|
||||
expect((await resolveNewAgentRunnerForCompany(db, "company", input)).adapterType).toBe("codex_local");
|
||||
await adapterSettingsService(peer, managed).setDisabled("codex_local", true);
|
||||
await expect(resolveNewAgentRunnerForCompany(db, "company", input)).rejects.toMatchObject({ status: 422, details: { code: "agent_runner_unavailable" } });
|
||||
await adapterSettingsService(peer, managed).setDisabled("codex_local", false);
|
||||
expect((await resolveNewAgentRunnerForCompany(db, "company", input)).adapterType).toBe("codex_local");
|
||||
});
|
||||
|
||||
it("rolls back the setting when its audit write fails", async () => {
|
||||
await expect(service().setDisabled("codex_local", true, async () => { throw new Error("audit failed"); })).rejects.toThrow("audit failed");
|
||||
expect(await service().inspectStored()).toBeNull();
|
||||
await expect(adapterSettingsService(peer, managed).getDisabledTypes()).rejects.toThrow("not been initialized");
|
||||
});
|
||||
|
||||
it("publishes catalog changes across stores and records the instance mutation", async () => {
|
||||
const res = await request(app()).patch("/adapters/codex_local").send({ disabled: true });
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body).toEqual({ type: "codex_local", disabled: true, changed: true });
|
||||
const catalog = await request(app()).get("/adapters");
|
||||
expect(catalog.status).toBe(200);
|
||||
expect(catalog.body.find((entry: { type: string }) => entry.type === "codex_local").disabled).toBe(true);
|
||||
const [audit] = await peer.select().from(activityLog).where(eq(activityLog.action, "instance.adapter.updated"));
|
||||
expect(audit.details).toMatchObject({ adapterType: "codex_local", disabled: true, revision: 1 });
|
||||
});
|
||||
|
||||
it("returns failure when PostgreSQL rejects the update and keeps the durable choice", async () => {
|
||||
await service().seedIfAbsent({ disabledTypes: [] });
|
||||
await db.execute(sql`CREATE FUNCTION reject_adapter_settings() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN RAISE EXCEPTION 'test adapter settings failure'; END $$`);
|
||||
await db.execute(sql`CREATE TRIGGER reject_adapter_settings BEFORE UPDATE ON instance_settings FOR EACH ROW EXECUTE FUNCTION reject_adapter_settings()`);
|
||||
try {
|
||||
const res = await request(app()).patch("/adapters/codex_local").send({ disabled: true });
|
||||
expect(res.status).toBe(500);
|
||||
expect(await adapterSettingsService(peer, managed).getDisabledTypes()).toEqual([]);
|
||||
} finally {
|
||||
await db.execute(sql`DROP TRIGGER reject_adapter_settings ON instance_settings`);
|
||||
await db.execute(sql`DROP FUNCTION reject_adapter_settings()`);
|
||||
}
|
||||
});
|
||||
|
||||
it("fails closed on malformed stored settings and unavailable database", async () => {
|
||||
await db.insert(instanceSettings).values({ singletonKey: "adapter-settings:v1", general: { version: 2, disabledTypes: [] } });
|
||||
await expect(service().getDisabledTypes()).rejects.toThrow("Persisted adapter settings are invalid");
|
||||
await expect(adapterSettingsService(undefined, managed).getDisabledTypes()).rejects.toThrow("database is required");
|
||||
expect(legacy.get).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,126 @@
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { instanceSettings } from "@paperclipai/db";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { open } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { z } from "zod";
|
||||
import { listServerAdapters } from "../adapters/registry.js";
|
||||
import { forbidden } from "../errors.js";
|
||||
import { resolvePaperclipHomeDir } from "../home-paths.js";
|
||||
import { getDisabledAdapterTypes, setAdapterDisabled } from "./adapter-plugin-store.js";
|
||||
import { parseAdapterRegistryEnv } from "./adapter-registry-config.js";
|
||||
import { isCloudManagedInstance } from "./cloud-instance.js";
|
||||
|
||||
const SINGLETON_KEY = "adapter-settings:v1";
|
||||
type Transaction = Parameters<Parameters<Db["transaction"]>[0]>[0];
|
||||
export type AdapterSettingsDb = Pick<Db | Transaction, "select" | "insert" | "update" | "transaction">;
|
||||
export type StoredAdapterSettings = { version: 1; revision: number; disabledTypes: string[] };
|
||||
const storedSettingsSchema = z.object({
|
||||
version: z.literal(1),
|
||||
revision: z.number().int().nonnegative(),
|
||||
disabledTypes: z.array(z.string().refine((type) => type.trim().length > 0)),
|
||||
});
|
||||
|
||||
function parseStored(value: Record<string, unknown>): StoredAdapterSettings {
|
||||
const parsed = storedSettingsSchema.safeParse(value);
|
||||
if (!parsed.success) throw new Error("Persisted adapter settings are invalid");
|
||||
return { ...parsed.data, disabledTypes: [...new Set(parsed.data.disabledTypes)].sort() };
|
||||
}
|
||||
|
||||
export function effectiveDisabledAdapterTypes(saved: string[], knownTypes: string[] | (() => string[]), env: Record<string, string | undefined> = process.env): string[] {
|
||||
const registry = parseAdapterRegistryEnv(env);
|
||||
if (!registry) return [...saved];
|
||||
const enabled = new Set(registry.filter((entry) => entry.enabled !== false).map((entry) => entry.adapterType));
|
||||
const types = typeof knownTypes === "function" ? knownTypes() : knownTypes;
|
||||
return [...new Set(types.filter((type) => !enabled.has(type)))].sort();
|
||||
}
|
||||
|
||||
export function adapterSettingsService(db: AdapterSettingsDb | undefined, options: { runtimeEnv?: Record<string, string | undefined> } = {}) {
|
||||
const env = options.runtimeEnv ?? process.env;
|
||||
const managed = isCloudManagedInstance(env);
|
||||
|
||||
function requireDb(): AdapterSettingsDb {
|
||||
if (!db) throw new Error("A database is required for managed adapter settings");
|
||||
return db;
|
||||
}
|
||||
|
||||
async function inspectStored(handle: AdapterSettingsDb = requireDb()): Promise<StoredAdapterSettings | null> {
|
||||
const [row] = await handle.select({ general: instanceSettings.general }).from(instanceSettings)
|
||||
.where(eq(instanceSettings.singletonKey, SINGLETON_KEY)).limit(1);
|
||||
return row ? parseStored(row.general) : null;
|
||||
}
|
||||
|
||||
async function seedIfAbsent(value: { disabledTypes: string[] }, handle: AdapterSettingsDb = requireDb()) {
|
||||
const snapshot = parseStored({ version: 1, revision: 0, disabledTypes: value.disabledTypes });
|
||||
const inserted = await handle.insert(instanceSettings).values({ singletonKey: SINGLETON_KEY, general: snapshot })
|
||||
.onConflictDoNothing({ target: instanceSettings.singletonKey }).returning({ id: instanceSettings.id });
|
||||
const current = await inspectStored(handle);
|
||||
if (!current) throw new Error("Adapter settings seed was not persisted");
|
||||
return { seeded: inserted.length > 0, current };
|
||||
}
|
||||
|
||||
return {
|
||||
inspectStored,
|
||||
seedIfAbsent,
|
||||
async getDisabledTypes(): Promise<string[]> {
|
||||
const current = managed ? await inspectStored() : null;
|
||||
if (managed && !current) throw new Error("Managed adapter settings have not been initialized");
|
||||
const stored = managed ? current!.disabledTypes : getDisabledAdapterTypes();
|
||||
return effectiveDisabledAdapterTypes(stored, () => listServerAdapters().map((adapter) => adapter.type), env);
|
||||
},
|
||||
async setDisabled(type: string, disabled: boolean, afterChange?: (handle: AdapterSettingsDb, current: StoredAdapterSettings) => Promise<void>): Promise<boolean> {
|
||||
if (!type.trim()) throw new Error("Adapter type is required");
|
||||
const registry = parseAdapterRegistryEnv(env);
|
||||
if (registry && disabled === registry.some((entry) => entry.adapterType === type && entry.enabled !== false)) {
|
||||
throw forbidden("Adapter availability is managed by the hosting operator", { code: "settings_operator_managed" });
|
||||
}
|
||||
if (!managed) {
|
||||
const changed = setAdapterDisabled(type, disabled);
|
||||
if (changed && db) await afterChange?.(db, { version: 1, revision: 0, disabledTypes: getDisabledAdapterTypes() });
|
||||
return changed;
|
||||
}
|
||||
return requireDb().transaction(async (tx) => {
|
||||
await seedIfAbsent({ disabledTypes: [] }, tx);
|
||||
const [row] = await tx.select().from(instanceSettings).where(eq(instanceSettings.singletonKey, SINGLETON_KEY)).for("update");
|
||||
if (!row) throw new Error("Adapter settings are unavailable");
|
||||
const current = parseStored(row.general);
|
||||
const types = new Set(current.disabledTypes);
|
||||
if (types.has(type) === disabled) return false;
|
||||
if (disabled) types.add(type); else types.delete(type);
|
||||
const next: StoredAdapterSettings = { version: 1, revision: current.revision + 1, disabledTypes: [...types].sort() };
|
||||
await tx.update(instanceSettings).set({ general: next, updatedAt: new Date() }).where(eq(instanceSettings.id, row.id));
|
||||
await afterChange?.(tx, next);
|
||||
return true;
|
||||
});
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export type AdapterSettingsService = Pick<ReturnType<typeof adapterSettingsService>, "getDisabledTypes" | "setDisabled">;
|
||||
|
||||
export async function initializeAdapterSettings(db: AdapterSettingsDb, options: { runtimeEnv?: Record<string, string | undefined> } = {}) {
|
||||
const env = options.runtimeEnv ?? process.env;
|
||||
if (!isCloudManagedInstance(env)) return null;
|
||||
const service = adapterSettingsService(db, options);
|
||||
const current = await service.inspectStored();
|
||||
if (current) return { seeded: false, current };
|
||||
const filePath = path.join(resolvePaperclipHomeDir(env.PAPERCLIP_HOME), "adapter-settings.json");
|
||||
let disabledTypes: string[] = [];
|
||||
try {
|
||||
const file = await open(filePath, "r");
|
||||
try {
|
||||
const stat = await file.stat();
|
||||
if (!stat.isFile() || stat.size > 65_536) throw new Error("Legacy adapter settings exceed the size limit or are not a regular file");
|
||||
const buffer = Buffer.alloc(65_537);
|
||||
const { bytesRead } = await file.read(buffer, 0, buffer.length, 0);
|
||||
if (bytesRead > 65_536) throw new Error("Legacy adapter settings exceed the size limit");
|
||||
const value = JSON.parse(buffer.subarray(0, bytesRead).toString("utf8"));
|
||||
disabledTypes = parseStored({ version: 1, revision: 0, disabledTypes: value?.disabledTypes }).disabledTypes;
|
||||
} finally {
|
||||
await file.close();
|
||||
}
|
||||
} catch (error) {
|
||||
if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error;
|
||||
}
|
||||
return service.seedIfAbsent({ disabledTypes });
|
||||
}
|
||||
@@ -3,6 +3,10 @@ import { agentHarnessType, agentRunner, paperclipRunnerProfileForHarness, paperc
|
||||
const state = vi.hoisted(() => ({ disabled: [] as string[], overridden: new Set<string>(), settings: {} as Record<string, unknown>, getEnvironment: vi.fn(), bindings: vi.fn(), activeTemplate: vi.fn().mockResolvedValue(null), secretPaths: vi.fn().mockResolvedValue(new Set(["apiKey"])), managedEnvironment: vi.fn(), kubernetesEnvironment: vi.fn(), ensureKubernetesEnvironment: vi.fn(), ssh: vi.fn(), resolveEnvironment: vi.fn(), sandboxQualification: vi.fn(), pluginByKey: vi.fn() }));
|
||||
vi.mock("../adapters/registry.js", () => ({ findActiveServerAdapter: (type: string) => ({ type }), hasActiveAdapterOverride: (type: string) => state.overridden.has(type), waitForExternalAdapters: async () => {} }));
|
||||
vi.mock("./adapter-plugin-store.js", () => ({ getDisabledAdapterTypes: () => state.disabled }));
|
||||
vi.mock("./adapter-settings.js", async (original) => ({
|
||||
...await original(),
|
||||
adapterSettingsService: () => ({ getDisabledTypes: async () => state.disabled }),
|
||||
}));
|
||||
vi.mock("./instance-settings.js", () => ({ instanceSettingsService: () => ({ get: async () => state.settings }) }));
|
||||
vi.mock("./environments.js", () => ({ environmentService: () => ({ getById: state.getEnvironment, listBoundCompanyIds: state.bindings, findManagedSandboxEnvironment: state.managedEnvironment, findKubernetesEnvironment: state.kubernetesEnvironment, ensureKubernetesEnvironment: state.ensureKubernetesEnvironment }) }));
|
||||
vi.mock("./environment-custom-images.js", () => ({ environmentCustomImageService: () => ({ getActiveTemplate: state.activeTemplate }) }));
|
||||
@@ -351,7 +355,7 @@ describe("server-owned Codex runner selection", () => {
|
||||
state.settings = { general: { executionMode: "kubernetes" } };
|
||||
const selected = await resolveAgentRunnerTargetForCompany({} as never, "company");
|
||||
expect(selected).toEqual({ driver: "sandbox" });
|
||||
expect(agentRunnerAvailability("codex_local", selected)).toEqual({ supportedRunners: ["legacy"], defaultRunner: "legacy" });
|
||||
expect(agentRunnerAvailability("codex_local", selected, state.disabled)).toEqual({ supportedRunners: ["legacy"], defaultRunner: "legacy" });
|
||||
expect(state.kubernetesEnvironment).toHaveBeenCalledWith("company");
|
||||
expect(state.ensureKubernetesEnvironment).not.toHaveBeenCalled();
|
||||
expect(state.resolveEnvironment).not.toHaveBeenCalled();
|
||||
@@ -477,14 +481,14 @@ describe("server-owned Codex runner selection", () => {
|
||||
config: { provider: "daytona", snapshot: "replacement-snapshot" }, metadata: { managedByPaperclip: true } });
|
||||
state.resolveEnvironment.mockRejectedValue(new Error("Saved provider secret is unavailable."));
|
||||
const selected = await resolveAgentRunnerTargetForCompany({} as never, "company", "sandbox");
|
||||
expect(agentRunnerAvailability(adapterType, selected).defaultRunner).toBe("paperclip");
|
||||
expect(agentRunnerAvailability(adapterType, selected, state.disabled).defaultRunner).toBe("paperclip");
|
||||
expect(await resolveNewAgentRunnerForCompany({} as never, "company", {
|
||||
adapterType, defaultEnvironmentId: "sandbox", adapterConfig: { model: adapterType === "opencode_local" ? "provider/model" : undefined },
|
||||
})).toMatchObject({ adapterType: "paperclip_runner" });
|
||||
expect(state.sandboxQualification).not.toHaveBeenCalled();
|
||||
expect(state.resolveEnvironment).not.toHaveBeenCalled();
|
||||
state.disabled.push("paperclip_runner");
|
||||
expect(agentRunnerAvailability(adapterType, selected).defaultRunner).toBe("legacy");
|
||||
expect(agentRunnerAvailability(adapterType, selected, state.disabled).defaultRunner).toBe("legacy");
|
||||
});
|
||||
it.each(["self-hosted", "custom-environment", "captured-template", "stale-marker"])("keeps qualification for a %s sandbox", async kind => {
|
||||
vi.stubEnv("PAPERCLIP_MANAGED_CONFIG", kind === "self-hosted" ? undefined : JSON.stringify({ ...managedCloudConfig,
|
||||
@@ -495,7 +499,7 @@ describe("server-owned Codex runner selection", () => {
|
||||
metadata: { managedByPaperclip: kind !== "custom-environment" } });
|
||||
state.activeTemplate.mockResolvedValue(kind === "captured-template" ? { id: "template", provider: "daytona", templateRef: "custom-snapshot" } : null);
|
||||
const selected = await resolveAgentRunnerTargetForCompany({} as never, "company", "sandbox");
|
||||
expect(agentRunnerAvailability("codex_local", selected).defaultRunner).toBe("legacy");
|
||||
expect(agentRunnerAvailability("codex_local", selected, state.disabled).defaultRunner).toBe("legacy");
|
||||
expect(state.sandboxQualification).toHaveBeenCalled();
|
||||
});
|
||||
it("ignores a stale captured image and preserves a matching custom image without resolving credentials", async () => {
|
||||
@@ -507,17 +511,17 @@ describe("server-owned Codex runner selection", () => {
|
||||
sourceEnvironmentConfigFingerprint: fingerprint({ ...config, snapshot: "previous-snapshot" }, { excludePaths: ["apiKey"] }) };
|
||||
state.activeTemplate.mockResolvedValue(template);
|
||||
const current = await resolveAgentRunnerTargetForCompany({} as never, "company", "sandbox");
|
||||
expect(agentRunnerAvailability("codex_local", current).defaultRunner).toBe("paperclip");
|
||||
expect(agentRunnerAvailability("codex_local", current, state.disabled).defaultRunner).toBe("paperclip");
|
||||
expect(await resolveNewAgentRunnerForCompany({} as never, "company", { adapterType: "codex_local", runner: "paperclip", defaultEnvironmentId: "sandbox" }))
|
||||
.toMatchObject({ adapterType: "paperclip_runner" });
|
||||
expect(state.sandboxQualification).not.toHaveBeenCalled();
|
||||
state.activeTemplate.mockResolvedValue({ ...template, sourceEnvironmentConfigFingerprint: fingerprint(config, { excludePaths: ["apiKey"] }) });
|
||||
const custom = await resolveAgentRunnerTargetForCompany({} as never, "company", "sandbox");
|
||||
expect(agentRunnerAvailability("codex_local", custom).defaultRunner).toBe("legacy");
|
||||
expect(agentRunnerAvailability("codex_local", custom, state.disabled).defaultRunner).toBe("legacy");
|
||||
expect(state.sandboxQualification).toHaveBeenCalled();
|
||||
state.getEnvironment.mockResolvedValue({ id: "sandbox", driver: "sandbox", status: "active", config: { ...config, timeoutMs: "invalid" }, metadata: { managedByPaperclip: true } });
|
||||
const invalid = await resolveAgentRunnerTargetForCompany({} as never, "company", "sandbox");
|
||||
expect(agentRunnerAvailability("codex_local", invalid).defaultRunner).toBe("legacy");
|
||||
expect(agentRunnerAvailability("codex_local", invalid, state.disabled).defaultRunner).toBe("legacy");
|
||||
expect(state.resolveEnvironment).not.toHaveBeenCalled();
|
||||
});
|
||||
it("checks company access before trusting a managed Cloud image", async () => {
|
||||
|
||||
@@ -5,6 +5,9 @@ import { resolveClaudeModel } from "@paperclipai/adapter-claude-local";
|
||||
import { DEFAULT_GROK_LOCAL_MODEL } from "@paperclipai/adapter-grok-local";
|
||||
import { findActiveServerAdapter, hasActiveAdapterOverride, waitForExternalAdapters } from "../adapters/registry.js";
|
||||
import { getDisabledAdapterTypes } from "./adapter-plugin-store.js";
|
||||
import { adapterSettingsService, effectiveDisabledAdapterTypes } from "./adapter-settings.js";
|
||||
import { listServerAdapters } from "../adapters/registry.js";
|
||||
import { isCloudManagedInstance } from "./cloud-instance.js";
|
||||
import { claudeRunnerUnsupportedSettings, PaperclipRunnerProviderProfileError, resolvePaperclipRunnerNativeProviderInput } from "./native-runtime/provider-profile.js";
|
||||
import { forbidden, unprocessable } from "../errors.js";
|
||||
import { runSshCommand } from "@paperclipai/adapter-utils/ssh";
|
||||
@@ -24,6 +27,7 @@ export type RunnerTarget = {
|
||||
export const AGENT_RUNNER_SSH_PLATFORM_COMMAND = "uname -s; uname -m; if paperclip_glibc_version=$(getconf GNU_LIBC_VERSION 2>/dev/null) && [ -n \"$paperclip_glibc_version\" ]; then printf 'glibc\\n%s\\n' \"$paperclip_glibc_version\"; else case \"$(ldd --version 2>&1)\" in *musl*) printf 'musl\\n' ;; *) printf 'unknown\\n' ;; esac; fi";
|
||||
export interface AgentRunnerSelection {
|
||||
adapterType?: string | null;
|
||||
disabledTypes?: string[];
|
||||
adapterConfig?: Record<string, unknown>;
|
||||
runner?: AgentRunnerChoice;
|
||||
target?: RunnerTarget;
|
||||
@@ -32,8 +36,13 @@ export interface AgentRunnerSelection {
|
||||
onConfigurationWarnings?: (warnings: AgentConfigurationWarning[]) => void;
|
||||
}
|
||||
|
||||
function runnerAdaptersAvailable(harness: string, driver: string): boolean {
|
||||
const disabled = new Set(getDisabledAdapterTypes());
|
||||
function localDisabledTypes(): string[] {
|
||||
if (isCloudManagedInstance()) throw new Error("Managed runner selection requires a fresh adapter settings snapshot");
|
||||
return effectiveDisabledAdapterTypes(getDisabledAdapterTypes(), () => listServerAdapters().map((adapter) => adapter.type));
|
||||
}
|
||||
|
||||
function runnerAdaptersAvailable(harness: string, driver: string, disabledTypes: string[]): boolean {
|
||||
const disabled = new Set(disabledTypes);
|
||||
return !!paperclipRunnerProfileForHarness(harness)
|
||||
&& !disabled.has(harness) && !disabled.has("paperclip_runner")
|
||||
&& !!findActiveServerAdapter(harness) && !!findActiveServerAdapter("paperclip_runner")
|
||||
@@ -41,13 +50,13 @@ function runnerAdaptersAvailable(harness: string, driver: string): boolean {
|
||||
&& isEnvironmentDriverSupportedForAdapter("paperclip_runner", driver);
|
||||
}
|
||||
|
||||
export function agentRunnerAvailability(harness: string, target: RunnerTarget = localAgentRunnerTarget()): AgentRunnerAvailability {
|
||||
export function agentRunnerAvailability(harness: string, target: RunnerTarget = localAgentRunnerTarget(), disabledTypes: string[] = localDisabledTypes()): AgentRunnerAvailability {
|
||||
if (target.runnerTasks || target.managedCloudImage) {
|
||||
return runnerAdaptersAvailable(harness, target.driver)
|
||||
return runnerAdaptersAvailable(harness, target.driver, disabledTypes)
|
||||
? { supportedRunners: target.runnerTasks ? ["paperclip"] : ["paperclip", "legacy"], defaultRunner: "paperclip" }
|
||||
: { supportedRunners: ["legacy"], defaultRunner: "legacy" };
|
||||
}
|
||||
const supported = runnerAdaptersAvailable(harness, target.driver)
|
||||
const supported = runnerAdaptersAvailable(harness, target.driver, disabledTypes)
|
||||
&& (target.driver !== "sandbox" || Boolean(target.platform && target.architecture
|
||||
&& (target.platform !== "linux" || (target.libc === "glibc" && target.glibcVersion))))
|
||||
&& (["claude_local", "grok_local"].includes(harness)
|
||||
@@ -166,11 +175,12 @@ export function resolveNewAgentRunner(input: AgentRunnerSelection): { adapterTyp
|
||||
function resolveAgentRunner(input: AgentRunnerSelection): { adapterType: string; adapterConfig: Record<string, unknown> } {
|
||||
const adapterType = input.adapterType ?? "process";
|
||||
let config = { ...input.adapterConfig };
|
||||
const disabledTypes = input.disabledTypes ?? localDisabledTypes();
|
||||
const harness = agentHarnessType(adapterType, config);
|
||||
const native = adapterType === "paperclip_runner";
|
||||
if (paperclipRunnerProfileForHarness(adapterType) && getDisabledAdapterTypes().includes(adapterType)) throw unprocessable(`${adapterType === "grok_local" ? "Grok Build" : adapterType === "pi_local" ? "Pi" : adapterType === "opencode_local" ? "OpenCode" : adapterType === "claude_local" ? "Claude" : "Codex"} is disabled on this instance. Enable its adapter or choose another harness.`, { code: "agent_runner_unavailable" });
|
||||
if (paperclipRunnerProfileForHarness(adapterType) && disabledTypes.includes(adapterType)) throw unprocessable(`${adapterType === "grok_local" ? "Grok Build" : adapterType === "pi_local" ? "Pi" : adapterType === "opencode_local" ? "OpenCode" : adapterType === "claude_local" ? "Claude" : "Codex"} is disabled on this instance. Enable its adapter or choose another harness.`, { code: "agent_runner_unavailable" });
|
||||
if (native && (input.runner === undefined || input.runner === "paperclip" || (harness !== "codex_local" && input.runner !== "legacy"))) return { adapterType, adapterConfig: normalizeLegacyRunnerProvider(config) };
|
||||
const availability = agentRunnerAvailability(harness, input.target);
|
||||
const availability = agentRunnerAvailability(harness, input.target, disabledTypes);
|
||||
const choice = input.runner ?? "auto";
|
||||
const useNative = choice === "paperclip" || (choice === "auto" && availability.defaultRunner === "paperclip");
|
||||
if (!useNative) {
|
||||
@@ -318,13 +328,14 @@ function resolveAgentRunner(input: AgentRunnerSelection): { adapterType: string;
|
||||
}
|
||||
|
||||
export async function resolveNewAgentRunnerForCompany(db: Db, companyId: string, input: AgentRunnerSelection & { defaultEnvironmentId?: string | null }) {
|
||||
input = { ...input, disabledTypes: await adapterSettingsService(db).getDisabledTypes() };
|
||||
if (!paperclipRunnerProfileForHarness(agentHarnessType(input.adapterType ?? "process", input.adapterConfig))
|
||||
|| input.runner === "legacy" || (input.adapterType === "paperclip_runner" && (input.runner === undefined || input.runner === "paperclip"))) return resolveNewAgentRunner(input);
|
||||
const target = await resolveAgentRunnerTargetForCompany(db, companyId, input.defaultEnvironmentId);
|
||||
const target = await resolveAgentRunnerTargetForCompany(db, companyId, input.defaultEnvironmentId, input.disabledTypes);
|
||||
return resolveNewAgentRunner({ ...input, target });
|
||||
}
|
||||
|
||||
export async function resolveAgentRunnerTargetForCompany(db: Db, companyId: string, defaultEnvironmentId?: string | null): Promise<RunnerTarget> {
|
||||
export async function resolveAgentRunnerTargetForCompany(db: Db, companyId: string, defaultEnvironmentId?: string | null, disabledTypes?: string[]): Promise<RunnerTarget> {
|
||||
await waitForExternalAdapters();
|
||||
const { environment, driver } = await resolveAgentRunnerEnvironmentForCompany(db, companyId, defaultEnvironmentId);
|
||||
let target: RunnerTarget = driver === "local" ? localAgentRunnerTarget()
|
||||
@@ -356,7 +367,9 @@ export async function resolveAgentRunnerTargetForCompany(db: Db, companyId: stri
|
||||
const qualified = await readSandboxPlatformQualification(db, environment, companyId);
|
||||
if (qualified) target = { driver, ...qualified };
|
||||
}
|
||||
if (driver === "ssh" && environment && ["codex_local", "claude_local", "opencode_local", "pi_local", "grok_local"].some(harness => runnerAdaptersAvailable(harness, driver))) {
|
||||
const disabledForSsh = driver === "ssh" && environment
|
||||
? disabledTypes ?? await adapterSettingsService(db).getDisabledTypes() : [];
|
||||
if (driver === "ssh" && environment && ["codex_local", "claude_local", "opencode_local", "pi_local", "grok_local"].some(harness => runnerAdaptersAvailable(harness, driver, disabledForSsh))) {
|
||||
const parsed = await resolveEnvironmentDriverConfigForRuntime(db, companyId, environment);
|
||||
if (parsed.driver !== "ssh") throw unprocessable("The selected SSH environment is unavailable.");
|
||||
let output: string;
|
||||
|
||||
Reference in new issue
Block a user