mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-30 05:16:50 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The board tests an agent's selected runtime and AI account. > - Cloud prepares and verifies the managed Runner runtime before execution. > - Its remote ACPX preflight reports that deferred step as a warning. > - The test keeps the warning after the selected account passes its hello probe. > - The board then asks the user to fix a healthy connection. > - This change makes that specific managed step informational after account verification. ## Linked Issues or Issue Description **What happened?** A Cloud agent's Claude connection test passes with Legacy. With Runner, the same account completes the Claude hello probe but shows "Connection needs attention." The only warning says that remote runtime preparation must occur before launch. **Expected behavior** A verified managed connection passes when its only warning concerns runtime preparation that Paperclip performs before execution. Failed authentication and actionable runtime checks retain their results. **Steps to reproduce** 1. Select a working Claude subscription on a Cloud-managed sandbox. 2. Select Runner and run the connection test. 3. Observe a successful Claude hello probe and a warning result. Related: #17961 covers managed OpenRouter model verification. #15880 adds persistent computer runtime interfaces. Neither changes this Cloud connection-test status. ## What Changed - Recognize the deferred ACPX runtime check after a provider hello or live API-key verification succeeds. - Report that check as informational for Cloud-managed sandbox connections. - Preserve every other warning and error, including unexplained warning results. - Add service regressions and document the test/launch boundary. ## Verification - Red: the healthy managed-Claude case returned `warn` after a successful hello. The regression fails before the change and passes after it. - **116 tests pass** across connection verdicts, environment-test HTTP routes and adapter registration. Scoped server TypeScript and the server build pass, including the native Runner build. - Negative cases cover rejected credentials, missing hello receipts, missing runtime packages, actionable or unexplained warnings, self-hosted instances, SSH targets and unexplained provider warning results. API-key success requires live provider re-verification. - Operator evidence confirms that the reported test lease was released with cleanup successful. The agent's saved adapter is Legacy. No saved account, agent or environment setting was changed. - A clean local install hit a frozen-lockfile `patchedDependencies` configuration mismatch. The scoped checks reuse installed workspace dependencies. The final commit has passing hosted CI, including clean-install typecheck, tests and build, and Apex 5/5 with no unresolved findings. - A cold HTTP-suite module load reached the 30-second setup limit while other checks ran. All 54 HTTP cases pass on retry, and the full focused run passes 116 tests without changing assertions or time limits. - No staging deployment is claimed. This change classifies the connection-test result; it does not claim that a native Runner hello turn was performed. ## Risks - This status change is limited to a verified Cloud-managed sandbox ACPX connection and the exact deferred check code. The provider's successful receipt is required. - Runtime package preparation and verification still execute before agent work. A connection test does not replace those launch checks. - There is no schema, permission or credential change. Local, SSH and plugin task probes keep their existing verdicts. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection, tool use and code execution. The session does not expose its exact serving model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
628 lines
31 KiB
TypeScript
628 lines
31 KiB
TypeScript
import { join } from "node:path";
|
|
import { probePluginRunnerEnvironment } from "./native-runtime/plugin-runner-probe.js";
|
|
import { registerPluginTaskCredentialHandoff } from "./plugin-task-credential-writeback.js";
|
|
import { randomUUID } from "node:crypto";
|
|
import { agents, companies, type Db } from "@paperclipai/db";
|
|
import { and, eq } from "drizzle-orm";
|
|
import type { AdapterExecutionTarget } from "@paperclipai/adapter-utils/execution-target";
|
|
import type { AdapterEnvironmentCheck, AdapterEnvironmentTestContext, AdapterEnvironmentTestResult } from "@paperclipai/adapter-utils";
|
|
import { parseObject } from "@paperclipai/adapter-utils/server-utils";
|
|
import { aiRoutingHarness, ADAPTER_AUTH_MISSING_CHECK_CODE, AI_CONNECTION_CAPABILITIES, type AiConnectionBinding } from "@paperclipai/shared";
|
|
import { requireServerAdapter } from "../adapters/index.js";
|
|
import { forbidden, HttpError, unprocessable } from "../errors.js";
|
|
import { isCloudManagedInstance } from "./cloud-instance.js";
|
|
import { instanceSettingsService } from "./instance-settings.js";
|
|
import { environmentService } from "./environments.js";
|
|
import { environmentRuntimeService } from "./environment-runtime.js";
|
|
import { resolveEnvironmentExecutionTarget } from "./environment-execution-target.js";
|
|
import { prepareManagedAiRuntime, assertManagedAiProjectAuth } from "./ai-connection-runtime.js";
|
|
import { aiConnectionService } from "./ai-connections.js";
|
|
import { validateAiApiKey } from "./ai-api-key-test.js";
|
|
import { probeCopilotExecutionTarget } from "./copilot-connection-probe.js";
|
|
import type { PluginWorkerManager } from "./plugin-worker-manager.js";
|
|
import { assertEnvironmentSelectionForCompany } from "./environment-selection.js";
|
|
import { pluginEnvironmentRequiresAgent } from "./plugin-environment-driver.js";
|
|
const asRecord = (value: unknown): Record<string, unknown> | null => value && typeof value === "object" && !Array.isArray(value) ? value as Record<string, unknown> : null;
|
|
function finishManagedConnectionTest(adapterType: string, context: AdapterEnvironmentTestContext, result: AdapterEnvironmentTestResult): AdapterEnvironmentTestResult {
|
|
const target = context.executionTarget;
|
|
if (result.status === "fail" || adapterType !== "paperclip_runner" || context.config.provider !== "acpx"
|
|
|| !isCloudManagedInstance() || target?.kind !== "remote" || target.transport !== "sandbox" || target.runnerTask
|
|
|| !result.checks.some(check => /hello_probe_(passed|succeeded)$/.test(check.code) || check.code === "ai_connection_api_key_reverified")
|
|
|| !result.checks.some(check => check.code === "acpx_remote_runtime_unverified" && check.level === "warn")) return result;
|
|
const checks = result.checks.map(check => check.code === "acpx_remote_runtime_unverified" && check.level === "warn"
|
|
? { ...check, code: "acpx_remote_runtime_managed", level: "info" as const,
|
|
message: "The remote platform is supported. Paperclip manages Runner preparation and verifies its runtime before execution." }
|
|
: check);
|
|
return { ...result, checks, status: checks.some(check => check.level === "error") ? "fail" : checks.some(check => check.level === "warn") ? "warn" : "pass" };
|
|
}
|
|
|
|
export function agentEnvironmentTestService(db: Db, pluginWorkerManager?: PluginWorkerManager) {
|
|
const instanceSettings = instanceSettingsService(db);
|
|
const environmentsSvc = environmentService(db);
|
|
const environmentRuntime = environmentRuntimeService(db, { pluginWorkerManager });
|
|
// A null agent override inherits the instance default, just like dispatch.
|
|
// Resolve this before secrets or probes so a default remote environment can
|
|
// never accidentally validate the account on the control-plane host.
|
|
async function resolveAdapterTestEnvironmentId(companyId: string, environmentId: string | null | undefined, copilot = false) {
|
|
if (copilot && (await instanceSettings.getGeneral()).executionMode === "kubernetes") {
|
|
const kubernetes = await environmentsSvc.findKubernetesEnvironment(companyId);
|
|
if (!kubernetes) throw unprocessable("The Kubernetes execution environment is unavailable.", { code: "copilot_environment_unavailable" });
|
|
return kubernetes.id;
|
|
}
|
|
if (environmentId) return environmentId;
|
|
const settings = await instanceSettings.get();
|
|
if (settings.defaultEnvironmentId) return settings.defaultEnvironmentId;
|
|
if ((await instanceSettings.getExperimental()).enableManagedSandboxOnly === true) {
|
|
const managed = await environmentsSvc.findManagedSandboxEnvironment(companyId);
|
|
if (!managed) {
|
|
throw unprocessable("The managed sandbox is unavailable. Restore Paperclip Computer and retry.", {
|
|
code: "managed_sandbox_unavailable",
|
|
});
|
|
}
|
|
return managed.id;
|
|
}
|
|
return null;
|
|
}
|
|
|
|
/**
|
|
* Resolve the execution target the adapter should run its test probes against.
|
|
*
|
|
* - No environmentId / local environment → returns a local target so the
|
|
* adapter probes the Paperclip host (legacy behavior).
|
|
* - SSH environment → builds an SSH execution target from the environment
|
|
* config so the adapter probes the remote box. No lease is required:
|
|
* the SSH spec is fully derived from the saved environment config.
|
|
* - Sandbox / plugin environments → acquires an ad-hoc lease, realizes the
|
|
* workspace, and resolves a sandbox execution target wired to the runtime
|
|
* so the adapter probe runs inside the sandbox the same way a heartbeat
|
|
* would. The returned `release` callback rolls the lease back when the
|
|
* route is done.
|
|
*
|
|
* The caller MUST always invoke `release()` (typically in a `finally` block).
|
|
*/
|
|
async function resolveAdapterTestExecutionContext(input: {
|
|
agentId?: string;
|
|
companyId: string;
|
|
adapterType: string;
|
|
environmentId: string | null;
|
|
}): Promise<{
|
|
executionTarget: AdapterExecutionTarget | null;
|
|
environmentName: string | null;
|
|
fallbackChecks: AdapterEnvironmentCheck[];
|
|
sandboxIdentityCheck?: AdapterEnvironmentCheck | null;
|
|
release: (status?: "released" | "failed") => Promise<void>;
|
|
}> {
|
|
const noopRelease = async () => {};
|
|
|
|
if (!input.environmentId) {
|
|
return {
|
|
executionTarget: null,
|
|
environmentName: null,
|
|
fallbackChecks: [],
|
|
release: noopRelease,
|
|
};
|
|
}
|
|
|
|
const requestedEnvironment = await environmentsSvc.getById(input.environmentId);
|
|
if (!requestedEnvironment) {
|
|
return {
|
|
executionTarget: null,
|
|
environmentName: null,
|
|
fallbackChecks: [
|
|
{
|
|
code: "environment_not_found",
|
|
level: "warn",
|
|
message: "Selected environment was not found. The test did not run.",
|
|
},
|
|
],
|
|
release: noopRelease,
|
|
};
|
|
}
|
|
|
|
// Managed-sandbox-only policy: redirect a Test that would run on the local
|
|
// host onto the platform-managed sandbox, the same as a real run does
|
|
// (resolveExecutionWorkspaceEnvironmentId in heartbeat). Without this
|
|
// redirect the Test probes the local host while the run executes in the
|
|
// managed sandbox, so a passing Test validates the wrong execution target.
|
|
// With no active managed sandbox the Test fails closed — never local.
|
|
let environment = requestedEnvironment;
|
|
if (requestedEnvironment.driver === "local") {
|
|
const managedSandboxOnly =
|
|
(await instanceSettings.getExperimental()).enableManagedSandboxOnly === true;
|
|
if (managedSandboxOnly) {
|
|
const managedSandboxEnvironment = await environmentsSvc.findManagedSandboxEnvironment(
|
|
input.companyId,
|
|
);
|
|
if (!managedSandboxEnvironment) {
|
|
return {
|
|
executionTarget: null,
|
|
environmentName: requestedEnvironment.name,
|
|
fallbackChecks: [
|
|
{
|
|
code: "managed_sandbox_unavailable",
|
|
level: "error",
|
|
message:
|
|
"This instance runs agents only in its platform-managed sandbox, but no active managed sandbox environment exists. The test did not run.",
|
|
hint: "Restore the managed sandbox environment, then test again.",
|
|
},
|
|
],
|
|
release: noopRelease,
|
|
};
|
|
}
|
|
environment = managedSandboxEnvironment;
|
|
}
|
|
}
|
|
|
|
if (environment.driver === "local") {
|
|
return {
|
|
executionTarget: null,
|
|
environmentName: environment.name,
|
|
fallbackChecks: [],
|
|
release: noopRelease,
|
|
};
|
|
}
|
|
|
|
if (environment.driver === "ssh") {
|
|
try {
|
|
const target = await resolveEnvironmentExecutionTarget({
|
|
db,
|
|
companyId: input.companyId,
|
|
adapterType: input.adapterType,
|
|
environment: {
|
|
id: environment.id,
|
|
driver: environment.driver,
|
|
config: environment.config ?? null,
|
|
},
|
|
leaseMetadata: null,
|
|
});
|
|
if (target) {
|
|
return {
|
|
executionTarget: target,
|
|
environmentName: environment.name,
|
|
fallbackChecks: [],
|
|
release: noopRelease,
|
|
};
|
|
}
|
|
return {
|
|
executionTarget: null,
|
|
environmentName: environment.name,
|
|
fallbackChecks: [
|
|
{
|
|
code: "environment_target_unavailable",
|
|
level: "warn",
|
|
message:
|
|
`Could not resolve an execution target for environment "${environment.name}". The test did not run.`,
|
|
},
|
|
],
|
|
release: noopRelease,
|
|
};
|
|
} catch (err) {
|
|
return {
|
|
executionTarget: null,
|
|
environmentName: environment.name,
|
|
fallbackChecks: [
|
|
{
|
|
code: "environment_target_failed",
|
|
level: "warn",
|
|
message:
|
|
`Could not connect to environment "${environment.name}" to run the test.`,
|
|
detail: err instanceof Error ? err.message : String(err),
|
|
},
|
|
],
|
|
release: noopRelease,
|
|
};
|
|
}
|
|
}
|
|
|
|
// sandbox / plugin / other remote drivers: spin up an ad-hoc lease, realize
|
|
// the workspace inside the box, and run the same probe SSH uses against
|
|
// a sandbox execution target wired to the environment runtime.
|
|
//
|
|
// We pass `heartbeatRunId: null` because there's no heartbeat run for an
|
|
// operator-initiated `Test` invocation — the leases table FKs heartbeat
|
|
// run id to heartbeat_runs.id, and we don't want to manufacture a fake
|
|
// run row. Cleanup goes through the driver's `releaseRunLease` directly
|
|
// (by lease record), since the batch helper queries by heartbeatRunId.
|
|
//
|
|
// Sandbox tests boot a fresh throwaway sandbox (never resume a retained
|
|
// agent lease) and archive it on release instead of deleting it, so the
|
|
// operator can inspect the exact sandbox from the provider dashboard while
|
|
// provider-side expiry reaps it later.
|
|
if (environment.driver === "plugin" && await pluginEnvironmentRequiresAgent(db, environment)) {
|
|
const [agent] = input.agentId ? await db.select({ state: agents.lifecycleState, pauseReason: agents.pauseReason,
|
|
requiresApproval: companies.requireBoardApprovalForNewAgents }).from(agents)
|
|
.innerJoin(companies, eq(companies.id, agents.companyId))
|
|
.where(and(eq(agents.id, input.agentId), eq(agents.companyId, input.companyId))) : [];
|
|
if (agent?.state === "paused" && agent.pauseReason === "setup" && agent.requiresApproval) return {
|
|
executionTarget: null, environmentName: environment.name, fallbackChecks: [{
|
|
code: "environment_agent_approval_required", level: "warn",
|
|
message: "The selected account will be verified on the agent computer after hire approval.",
|
|
}], release: noopRelease,
|
|
};
|
|
if (!agent || ["paused", "preparing"].includes(agent.state)) return {
|
|
executionTarget: null, environmentName: environment.name, fallbackChecks: [{
|
|
code: !input.agentId ? "environment_agent_required" : "environment_agent_preparing", level: "warn",
|
|
message: !input.agentId ? "This environment needs a saved agent before its connection can be tested."
|
|
: "The agent computer is being prepared. Test the connection when preparation finishes.",
|
|
}], release: noopRelease,
|
|
};
|
|
}
|
|
const testEnvironment = environment.driver === "sandbox"
|
|
? {
|
|
...environment,
|
|
config: {
|
|
...(environment.config ?? {}),
|
|
reuseLease: false,
|
|
archiveOnRelease: true,
|
|
},
|
|
}
|
|
: environment;
|
|
let leaseRecord: Awaited<ReturnType<typeof environmentRuntime.acquireRunLease>>;
|
|
try {
|
|
leaseRecord = await environmentRuntime.acquireRunLease({
|
|
companyId: input.companyId,
|
|
agentId: input.agentId,
|
|
adapterType: input.adapterType,
|
|
...(environment.driver === "plugin" && input.adapterType === "paperclip_runner"
|
|
? { requestedExpiresAt: new Date(Date.now() + 180_000) } : {}),
|
|
environment: testEnvironment,
|
|
configuredEnvironment: environment,
|
|
issueId: null,
|
|
heartbeatRunId: null,
|
|
persistedExecutionWorkspace: null,
|
|
// Re-check the company binding atomically at lease time. The route
|
|
// guard already rejected a foreign environment, but the binding could
|
|
// change between the guard check and the lease acquire. This closes
|
|
// that check-to-lease race so a foreign sandbox never gets a lease.
|
|
assertCompanyBinding: true,
|
|
// Apply the active custom-image template so the Test boots with the
|
|
// operator's captured sandbox customizations and prepared image state,
|
|
// matching what real agent runs use. Without this the test would
|
|
// silently fall back to the base image.
|
|
applyCustomImageTemplate: true,
|
|
});
|
|
} catch (err) {
|
|
return {
|
|
executionTarget: null,
|
|
environmentName: environment.name,
|
|
fallbackChecks: [
|
|
{
|
|
code: "environment_lease_acquire_failed",
|
|
level: "error",
|
|
message: `Could not acquire a lease for environment "${environment.name}".`,
|
|
detail: err instanceof Error ? err.message : String(err),
|
|
hint: "Check the environment's provider credentials and quota.",
|
|
},
|
|
],
|
|
release: noopRelease,
|
|
};
|
|
}
|
|
|
|
const driver = environmentRuntime.getDriver(environment.driver);
|
|
const releaseLease = async (status: "released" | "failed" = "released") => {
|
|
try {
|
|
if (driver) {
|
|
await driver.releaseRunLease({
|
|
environment: testEnvironment,
|
|
lease: leaseRecord.lease,
|
|
status,
|
|
});
|
|
} else {
|
|
await environmentsSvc.releaseLease(leaseRecord.lease.id, status);
|
|
}
|
|
} catch (err) {
|
|
// Cleanup failures must not mask the test result.
|
|
// eslint-disable-next-line no-console
|
|
console.warn(
|
|
`[adapter-test] Failed to release lease ${leaseRecord.lease.id}: ${err instanceof Error ? err.message : String(err)}`,
|
|
);
|
|
}
|
|
};
|
|
|
|
let realizedCwd: string | null = null;
|
|
try {
|
|
if (environment.driver === "plugin" && input.adapterType === "paperclip_runner") {
|
|
if (!await environmentRuntime.supportsTasks({ lease: leaseRecord.lease })) throw new Error("Plugin environment does not provide native Runner task execution");
|
|
const updated = await environmentsSvc.updateLeaseMetadata(leaseRecord.lease.id, {
|
|
...leaseRecord.lease.metadata, runnerTaskProbe: {
|
|
runId: randomUUID(), sessionId: randomUUID(), expiresAtMs: Date.now() + 120_000,
|
|
},
|
|
});
|
|
if (!updated) throw new Error("Plugin Runner setup lease disappeared");
|
|
leaseRecord.lease = updated;
|
|
} else {
|
|
const realized = await environmentRuntime.realizeWorkspace({
|
|
environment: testEnvironment,
|
|
lease: leaseRecord.lease,
|
|
// No host workspace to copy for a Test invocation; sandbox/plugin
|
|
// realize implementations use the lease metadata's remoteCwd to
|
|
// create the working directory inside the box.
|
|
workspace: {},
|
|
});
|
|
realizedCwd =
|
|
typeof realized.cwd === "string" && realized.cwd.trim().length > 0
|
|
? realized.cwd.trim()
|
|
: null;
|
|
}
|
|
} catch (err) {
|
|
await releaseLease("failed");
|
|
return {
|
|
executionTarget: null,
|
|
environmentName: environment.name,
|
|
fallbackChecks: [
|
|
{
|
|
code: "environment_workspace_realize_failed",
|
|
level: "error",
|
|
message: `Could not realize a workspace inside "${environment.name}".`,
|
|
detail: err instanceof Error ? err.message : String(err),
|
|
},
|
|
],
|
|
release: noopRelease,
|
|
};
|
|
}
|
|
|
|
let target: AdapterExecutionTarget | null;
|
|
try {
|
|
// Prefer the cwd the realize step returned; fall back to lease metadata.
|
|
const leaseMetadataForTarget: Record<string, unknown> | null =
|
|
realizedCwd
|
|
? { ...(leaseRecord.lease.metadata ?? {}), remoteCwd: realizedCwd }
|
|
: (leaseRecord.lease.metadata as Record<string, unknown> | null) ?? null;
|
|
|
|
target = await resolveEnvironmentExecutionTarget({
|
|
db,
|
|
companyId: input.companyId,
|
|
adapterType: input.adapterType,
|
|
environment: {
|
|
id: testEnvironment.id,
|
|
driver: testEnvironment.driver,
|
|
config: testEnvironment.config ?? null,
|
|
},
|
|
leaseId: leaseRecord.lease.id,
|
|
leaseMetadata: leaseMetadataForTarget,
|
|
lease: leaseRecord.lease,
|
|
environmentRuntime,
|
|
});
|
|
} catch (err) {
|
|
await releaseLease("failed");
|
|
return {
|
|
executionTarget: null,
|
|
environmentName: environment.name,
|
|
fallbackChecks: [
|
|
{
|
|
code: "environment_target_failed",
|
|
level: "error",
|
|
message: `Could not resolve an execution target for "${environment.name}".`,
|
|
detail: err instanceof Error ? err.message : String(err),
|
|
},
|
|
],
|
|
release: noopRelease,
|
|
};
|
|
}
|
|
|
|
if (!target) {
|
|
await releaseLease("failed");
|
|
return {
|
|
executionTarget: null,
|
|
environmentName: environment.name,
|
|
fallbackChecks: [
|
|
{
|
|
code: "environment_target_unsupported",
|
|
level: "warn",
|
|
message:
|
|
`Adapter "${input.adapterType}" is not allowed in "${environment.name}" environments.`,
|
|
},
|
|
],
|
|
release: noopRelease,
|
|
};
|
|
}
|
|
|
|
return {
|
|
executionTarget: target,
|
|
environmentName: environment.name,
|
|
fallbackChecks: [],
|
|
sandboxIdentityCheck: buildSandboxIdentityCheck({
|
|
environmentName: environment.name,
|
|
lease: leaseRecord.lease,
|
|
}),
|
|
release: releaseLease,
|
|
};
|
|
}
|
|
|
|
function readMetadataString(metadata: Record<string, unknown>, keys: string[]): string | null {
|
|
for (const key of keys) {
|
|
const value = metadata[key];
|
|
if (typeof value === "string" && value.trim().length > 0) return value.trim();
|
|
}
|
|
return null;
|
|
}
|
|
|
|
function buildSandboxIdentityCheck(input: {
|
|
environmentName: string;
|
|
lease: {
|
|
id: string;
|
|
provider?: string | null;
|
|
providerLeaseId?: string | null;
|
|
metadata?: Record<string, unknown> | null;
|
|
};
|
|
}): AdapterEnvironmentCheck {
|
|
const metadata = input.lease.metadata ?? {};
|
|
const provider = input.lease.provider ?? readMetadataString(metadata, ["provider"]);
|
|
const sandboxId = readMetadataString(metadata, ["sandboxId", "sandboxID", "sandbox_id", "id"]);
|
|
const sandboxName = readMetadataString(metadata, ["sandboxName", "sandbox_name", "name"]);
|
|
const snapshotRef = readMetadataString(metadata, [
|
|
"snapshot",
|
|
"snapshotId",
|
|
"snapshotID",
|
|
"snapshotRef",
|
|
"snapshot_ref",
|
|
"templateRef",
|
|
"template_ref",
|
|
"templateId",
|
|
"templateID",
|
|
"image",
|
|
"imageId",
|
|
"imageID",
|
|
"imageRef",
|
|
"image_ref",
|
|
]);
|
|
const templateKind = readMetadataString(metadata, [
|
|
"templateKind",
|
|
"template_kind",
|
|
"templateRefKind",
|
|
"template_ref_kind",
|
|
]);
|
|
const detailParts = [
|
|
`paperclipLeaseId=${input.lease.id}`,
|
|
input.lease.providerLeaseId ? `providerLeaseId=${input.lease.providerLeaseId}` : null,
|
|
provider ? `provider=${provider}` : null,
|
|
sandboxId ? `sandboxId=${sandboxId}` : null,
|
|
sandboxName ? `sandboxName=${sandboxName}` : null,
|
|
snapshotRef ? `${templateKind ? `${templateKind}Ref` : "snapshotOrTemplateRef"}=${snapshotRef}` : null,
|
|
].filter((part): part is string => Boolean(part));
|
|
|
|
return {
|
|
code: "sandbox_test_identity",
|
|
level: "info",
|
|
message: `Environment test identity for "${input.environmentName}".`,
|
|
detail: detailParts.join("; "),
|
|
hint: "Use these provider-neutral IDs when comparing model-test output with provider logs or refreshed environment snapshots.",
|
|
};
|
|
}
|
|
|
|
|
|
// The environment drivers the adapter Test route accepts. A local, SSH, or
|
|
// sandbox environment can host a probe; a plugin environment cannot.
|
|
const ADAPTER_TEST_ALLOWED_ENVIRONMENT_DRIVERS = ["local", "ssh", "sandbox"];
|
|
|
|
// The fail-closed tenant-binding guard for the adapter Test route. A caller
|
|
// may name any instance environment by id, so the route must reject an
|
|
// environment that binds to another company before it resolves secrets,
|
|
// merges env, resolves the target, leases a sandbox, or runs the adapter
|
|
// test. The guard checks the company binding BEFORE it validates the status
|
|
// or the driver, so it never reveals the status or the driver of a foreign
|
|
// environment. A same-company or an instance-global environment then gets the
|
|
// shared driver and status validation.
|
|
async function assertAdapterTestEnvironmentForCompany(
|
|
companyId: string,
|
|
environmentId: string,
|
|
adapterType?: string,
|
|
): Promise<void> {
|
|
if (!await assertAdapterTestEnvironmentBinding(companyId, environmentId)) return;
|
|
await assertEnvironmentSelectionForCompany(environmentsSvc, companyId, environmentId, {
|
|
allowedDrivers: adapterType === "paperclip_runner" ? [...ADAPTER_TEST_ALLOWED_ENVIRONMENT_DRIVERS, "plugin"] : ADAPTER_TEST_ALLOWED_ENVIRONMENT_DRIVERS,
|
|
});
|
|
}
|
|
|
|
// The company-binding half of the guard. A route that must resolve the
|
|
// runner before it knows the adapter type runs this first, then runs the
|
|
// full guard with the resolved adapter type. Returns the environment, or
|
|
// null for a missing one, which leaks no tenant state; the execution-context
|
|
// resolver surfaces the existing environment_not_found check.
|
|
async function assertAdapterTestEnvironmentBinding(companyId: string, environmentId: string) {
|
|
const environment = await environmentsSvc.getById(environmentId);
|
|
if (!environment) return null;
|
|
const boundCompanyIds = await environmentsSvc.listBoundCompanyIds(environmentId);
|
|
if (boundCompanyIds.length > 0 && !boundCompanyIds.includes(companyId)) {
|
|
throw forbidden("The selected environment belongs to another company.", {
|
|
code: "environment_company_mismatch",
|
|
});
|
|
}
|
|
return environment;
|
|
}
|
|
|
|
async function testManagedEnvironment(adapterType: string, context: Parameters<ReturnType<typeof requireServerAdapter>["testEnvironment"]>[0], binding: AiConnectionBinding, managed: Awaited<ReturnType<typeof prepareManagedAiRuntime>>, agentId?: string) {
|
|
const startedAt = new Date();
|
|
const target = context.executionTarget;
|
|
if (managed.credentialWriteback && target?.kind === "remote" && target.transport === "sandbox" && target.runnerTask && target.leaseId) {
|
|
await registerPluginTaskCredentialHandoff(db, { companyId: context.companyId, leaseId: target.leaseId }, managed.sessionIdentity, managed.credentialWriteback);
|
|
}
|
|
const result = await probeManagedEnvironment(adapterType, context, binding, managed);
|
|
// A provider rejection invalidates the tested credential generation. A
|
|
// missing CLI, unavailable environment, or other runtime error does not.
|
|
if (result.status === "fail" && result.checks.some(check =>
|
|
check.code === ADAPTER_AUTH_MISSING_CHECK_CODE || /_hello_probe_auth_required$/.test(check.code)
|
|
|| check.code === "ai_connection_api_key_rejected" || check.code === "COPILOT_AUTH_REQUIRED",
|
|
)) {
|
|
await aiConnectionService(db).markAuthenticationFailed({
|
|
companyId: context.companyId, agentId, runStartedAt: startedAt,
|
|
attribution: { ...managed.attribution, identity: managed.identity },
|
|
});
|
|
}
|
|
return result;
|
|
}
|
|
|
|
async function probeManagedEnvironment(adapterType: string, context: Parameters<ReturnType<typeof requireServerAdapter>["testEnvironment"]>[0], binding: AiConnectionBinding, managed: Awaited<ReturnType<typeof prepareManagedAiRuntime>>) {
|
|
if (binding.provider === "github") {
|
|
try {
|
|
const token = parseObject(context.config.env).COPILOT_GITHUB_TOKEN;
|
|
if (typeof token !== "string" || !token) throw unprocessable("Select a saved Copilot token.", { code: "COPILOT_AUTH_REQUIRED" });
|
|
const model = typeof context.config.model === "string" ? context.config.model.trim() : null;
|
|
if (!model || ["auto", "default"].includes(model.toLowerCase())) throw unprocessable("Select an available Copilot model.", { code: "COPILOT_MODEL_UNAVAILABLE" });
|
|
await probeCopilotExecutionTarget(token, context.executionTarget, model);
|
|
return { adapterType, status: "pass" as const, testedAt: new Date().toISOString(), checks: [{ code: "copilot_metadata_verified", level: "info" as const, message: "The verified Copilot runtime authenticated this account and accepted the selected model. No model prompt was sent." }] };
|
|
} catch (error) {
|
|
return { adapterType, status: "fail" as const, testedAt: new Date().toISOString(), checks: [{ code: error instanceof HttpError ? String(asRecord(error.details)?.code ?? "COPILOT_REQUEST_FAILED") : "COPILOT_REQUEST_FAILED", level: "error" as const, message: error instanceof HttpError ? error.message : "Copilot metadata verification failed." }] };
|
|
}
|
|
}
|
|
await assertManagedAiProjectAuth(context.config, binding.provider, context.executionTarget);
|
|
const target = context.executionTarget;
|
|
const result = adapterType === "paperclip_runner" && target?.kind === "remote" && target.transport === "sandbox" && target.runnerTask
|
|
? await probePluginRunnerEnvironment(context, { identity: managed.identity, accountIdentity: managed.sessionIdentity,
|
|
home: managed.home ? join(managed.home, "provider") : undefined })
|
|
: await requireServerAdapter(adapterType).testEnvironment(context);
|
|
if (result.status === "fail") return result;
|
|
// The resolved method, not binding.method — on a responsible_user binding
|
|
// that field is wire-compat only and the default connection decides.
|
|
if (adapterType === "paperclip_runner" && context.executionTarget?.kind === "remote"
|
|
&& context.executionTarget.transport === "sandbox" && context.executionTarget.runnerTask) return result;
|
|
const resolvedMethod = (context.config as { managedAiConnection?: { method?: string } }).managedAiConnection?.method;
|
|
// An api_key account does not take the CLI hello probe below: a key
|
|
// travels as an env var any engine understands, and the engine's own test
|
|
// above judged whether this runtime can execute with it — the ACP lane
|
|
// deliberately runs no hello probe when a key is configured. Demanding one
|
|
// anyway forced the CLI lane, whose probe needs a provider CLI on PATH,
|
|
// and a clean install has none: that walled off onboarding's API-key path
|
|
// on exactly the machines the release smoke exists to guard. The account
|
|
// is still verified live here — the same provider-endpoint check the save
|
|
// performed — so a key revoked since its save fails adoption rather than
|
|
// producing an agent that cannot authenticate at runtime. The hello-probe
|
|
// requirement stays for subscriptions: a stored login is a file layout
|
|
// only a provider CLI reads, so proving the runtime lane can consume it
|
|
// takes a real hello turn.
|
|
const nativePiOpenRouter = adapterType === "paperclip_runner" && context.config.provider === "acpx"
|
|
&& context.config.acpxAgent === "pi" && binding.provider === "openrouter"
|
|
&& asRecord(context.config.managedAiRouting)?.kind === "openrouter";
|
|
if (resolvedMethod === "api_key" && (!context.config.managedAiRouting || nativePiOpenRouter)) {
|
|
const envKey = AI_CONNECTION_CAPABILITIES[binding.provider].methods.api_key?.envKey;
|
|
const key = envKey ? parseObject(context.config.env)[envKey] : undefined;
|
|
try {
|
|
if (typeof key !== "string" || !key) throw unprocessable("The selected account's API key was not available to verify.");
|
|
await validateAiApiKey(binding.provider, key);
|
|
result.checks.push({ code: "ai_connection_api_key_reverified", level: "info", message: "The provider verified this API key for adoption." });
|
|
} catch (error) {
|
|
result.status = "fail";
|
|
const rejected = error instanceof HttpError && asRecord(error.details)?.code === "ai_connection_api_key_rejected";
|
|
result.checks.push({ code: rejected ? "ai_connection_api_key_rejected" : "ai_connection_verification_failed", level: "error", message: error instanceof HttpError ? error.message : "Could not verify the account. Try again." });
|
|
}
|
|
return finishManagedConnectionTest(adapterType, context, result);
|
|
}
|
|
let providerTestPassed = false;
|
|
if (!result.checks.some(check => check.code.includes("hello_probe"))) {
|
|
const providerAdapter = context.config.managedAiRouting ? aiRoutingHarness(adapterType, context.config.provider, context.config.acpxAgent) : { anthropic: "claude_local", openai: "codex_local", openrouter: "opencode_local", xai: "grok_local", google: "gemini_local" }[binding.provider];
|
|
const probe = await requireServerAdapter(providerAdapter).testEnvironment({ ...context, adapterType: providerAdapter, config: { ...context.config, engine: "cli" } });
|
|
providerTestPassed = probe.status === "pass";
|
|
result.checks.push(...probe.checks);
|
|
result.status = probe.status === "fail" ? "fail" : result.status === "warn" || probe.status === "warn" ? "warn" : "pass";
|
|
}
|
|
if (!result.checks.some(check => /hello_probe_(passed|succeeded)$/.test(check.code))) {
|
|
result.status = "fail";
|
|
result.checks.push({ code: "ai_connection_validation_incomplete", level: "error", message: "The selected account has not completed a provider hello test. Retry before adopting it." });
|
|
}
|
|
return providerTestPassed ? finishManagedConnectionTest(adapterType, context, result) : result;
|
|
}
|
|
|
|
|
|
return { resolveAdapterTestEnvironmentId, resolveAdapterTestExecutionContext, assertAdapterTestEnvironmentForCompany, assertAdapterTestEnvironmentBinding, testManagedEnvironment };
|
|
}
|