Files
PaperClipAI/server/src/services/agent-environment-test.ts
T
Devin FoleyandPaperclip 6102f8e82c fix: pass healthy managed Runner connection tests (#17965)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The board tests an agent's selected runtime and AI account.
> - Cloud prepares and verifies the managed Runner runtime before
execution.
> - Its remote ACPX preflight reports that deferred step as a warning.
> - The test keeps the warning after the selected account passes its
hello probe.
> - The board then asks the user to fix a healthy connection.
> - This change makes that specific managed step informational after
account verification.

## Linked Issues or Issue Description

**What happened?**

A Cloud agent's Claude connection test passes with Legacy. With Runner,
the same account completes the Claude hello probe but shows "Connection
needs attention." The only warning says that remote runtime preparation
must occur before launch.

**Expected behavior**

A verified managed connection passes when its only warning concerns
runtime preparation that Paperclip performs before execution. Failed
authentication and actionable runtime checks retain their results.

**Steps to reproduce**

1. Select a working Claude subscription on a Cloud-managed sandbox.
2. Select Runner and run the connection test.
3. Observe a successful Claude hello probe and a warning result.

Related: #17961 covers managed OpenRouter model verification. #15880
adds persistent computer runtime interfaces. Neither changes this Cloud
connection-test status.

## What Changed

- Recognize the deferred ACPX runtime check after a provider hello or
live API-key verification succeeds.
- Report that check as informational for Cloud-managed sandbox
connections.
- Preserve every other warning and error, including unexplained warning
results.
- Add service regressions and document the test/launch boundary.

## Verification

- Red: the healthy managed-Claude case returned `warn` after a
successful hello. The regression fails before the change and passes
after it.
- **116 tests pass** across connection verdicts, environment-test HTTP
routes and adapter registration. Scoped server TypeScript and the server
build pass, including the native Runner build.
- Negative cases cover rejected credentials, missing hello receipts,
missing runtime packages, actionable or unexplained warnings,
self-hosted instances, SSH targets and unexplained provider warning
results. API-key success requires live provider re-verification.
- Operator evidence confirms that the reported test lease was released
with cleanup successful. The agent's saved adapter is Legacy. No saved
account, agent or environment setting was changed.
- A clean local install hit a frozen-lockfile `patchedDependencies`
configuration mismatch. The scoped checks reuse installed workspace
dependencies. The final commit has passing hosted CI, including
clean-install typecheck, tests and build, and Apex 5/5 with no
unresolved findings.
- A cold HTTP-suite module load reached the 30-second setup limit while
other checks ran. All 54 HTTP cases pass on retry, and the full focused
run passes 116 tests without changing assertions or time limits.
- No staging deployment is claimed. This change classifies the
connection-test result; it does not claim that a native Runner hello
turn was performed.

## Risks

- This status change is limited to a verified Cloud-managed sandbox ACPX
connection and the exact deferred check code. The provider's successful
receipt is required.
- Runtime package preparation and verification still execute before
agent work. A connection test does not replace those launch checks.
- There is no schema, permission or credential change. Local, SSH and
plugin task probes keep their existing verdicts.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, repository inspection, tool
use and code execution. The session does not expose its exact serving
model ID or context window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 20:34:10 -07:00

628 lines
31 KiB
TypeScript

import { join } from "node:path";
import { probePluginRunnerEnvironment } from "./native-runtime/plugin-runner-probe.js";
import { registerPluginTaskCredentialHandoff } from "./plugin-task-credential-writeback.js";
import { randomUUID } from "node:crypto";
import { agents, companies, type Db } from "@paperclipai/db";
import { and, eq } from "drizzle-orm";
import type { AdapterExecutionTarget } from "@paperclipai/adapter-utils/execution-target";
import type { AdapterEnvironmentCheck, AdapterEnvironmentTestContext, AdapterEnvironmentTestResult } from "@paperclipai/adapter-utils";
import { parseObject } from "@paperclipai/adapter-utils/server-utils";
import { aiRoutingHarness, ADAPTER_AUTH_MISSING_CHECK_CODE, AI_CONNECTION_CAPABILITIES, type AiConnectionBinding } from "@paperclipai/shared";
import { requireServerAdapter } from "../adapters/index.js";
import { forbidden, HttpError, unprocessable } from "../errors.js";
import { isCloudManagedInstance } from "./cloud-instance.js";
import { instanceSettingsService } from "./instance-settings.js";
import { environmentService } from "./environments.js";
import { environmentRuntimeService } from "./environment-runtime.js";
import { resolveEnvironmentExecutionTarget } from "./environment-execution-target.js";
import { prepareManagedAiRuntime, assertManagedAiProjectAuth } from "./ai-connection-runtime.js";
import { aiConnectionService } from "./ai-connections.js";
import { validateAiApiKey } from "./ai-api-key-test.js";
import { probeCopilotExecutionTarget } from "./copilot-connection-probe.js";
import type { PluginWorkerManager } from "./plugin-worker-manager.js";
import { assertEnvironmentSelectionForCompany } from "./environment-selection.js";
import { pluginEnvironmentRequiresAgent } from "./plugin-environment-driver.js";
const asRecord = (value: unknown): Record<string, unknown> | null => value && typeof value === "object" && !Array.isArray(value) ? value as Record<string, unknown> : null;
function finishManagedConnectionTest(adapterType: string, context: AdapterEnvironmentTestContext, result: AdapterEnvironmentTestResult): AdapterEnvironmentTestResult {
const target = context.executionTarget;
if (result.status === "fail" || adapterType !== "paperclip_runner" || context.config.provider !== "acpx"
|| !isCloudManagedInstance() || target?.kind !== "remote" || target.transport !== "sandbox" || target.runnerTask
|| !result.checks.some(check => /hello_probe_(passed|succeeded)$/.test(check.code) || check.code === "ai_connection_api_key_reverified")
|| !result.checks.some(check => check.code === "acpx_remote_runtime_unverified" && check.level === "warn")) return result;
const checks = result.checks.map(check => check.code === "acpx_remote_runtime_unverified" && check.level === "warn"
? { ...check, code: "acpx_remote_runtime_managed", level: "info" as const,
message: "The remote platform is supported. Paperclip manages Runner preparation and verifies its runtime before execution." }
: check);
return { ...result, checks, status: checks.some(check => check.level === "error") ? "fail" : checks.some(check => check.level === "warn") ? "warn" : "pass" };
}
export function agentEnvironmentTestService(db: Db, pluginWorkerManager?: PluginWorkerManager) {
const instanceSettings = instanceSettingsService(db);
const environmentsSvc = environmentService(db);
const environmentRuntime = environmentRuntimeService(db, { pluginWorkerManager });
// A null agent override inherits the instance default, just like dispatch.
// Resolve this before secrets or probes so a default remote environment can
// never accidentally validate the account on the control-plane host.
async function resolveAdapterTestEnvironmentId(companyId: string, environmentId: string | null | undefined, copilot = false) {
if (copilot && (await instanceSettings.getGeneral()).executionMode === "kubernetes") {
const kubernetes = await environmentsSvc.findKubernetesEnvironment(companyId);
if (!kubernetes) throw unprocessable("The Kubernetes execution environment is unavailable.", { code: "copilot_environment_unavailable" });
return kubernetes.id;
}
if (environmentId) return environmentId;
const settings = await instanceSettings.get();
if (settings.defaultEnvironmentId) return settings.defaultEnvironmentId;
if ((await instanceSettings.getExperimental()).enableManagedSandboxOnly === true) {
const managed = await environmentsSvc.findManagedSandboxEnvironment(companyId);
if (!managed) {
throw unprocessable("The managed sandbox is unavailable. Restore Paperclip Computer and retry.", {
code: "managed_sandbox_unavailable",
});
}
return managed.id;
}
return null;
}
/**
* Resolve the execution target the adapter should run its test probes against.
*
* - No environmentId / local environment → returns a local target so the
* adapter probes the Paperclip host (legacy behavior).
* - SSH environment → builds an SSH execution target from the environment
* config so the adapter probes the remote box. No lease is required:
* the SSH spec is fully derived from the saved environment config.
* - Sandbox / plugin environments → acquires an ad-hoc lease, realizes the
* workspace, and resolves a sandbox execution target wired to the runtime
* so the adapter probe runs inside the sandbox the same way a heartbeat
* would. The returned `release` callback rolls the lease back when the
* route is done.
*
* The caller MUST always invoke `release()` (typically in a `finally` block).
*/
async function resolveAdapterTestExecutionContext(input: {
agentId?: string;
companyId: string;
adapterType: string;
environmentId: string | null;
}): Promise<{
executionTarget: AdapterExecutionTarget | null;
environmentName: string | null;
fallbackChecks: AdapterEnvironmentCheck[];
sandboxIdentityCheck?: AdapterEnvironmentCheck | null;
release: (status?: "released" | "failed") => Promise<void>;
}> {
const noopRelease = async () => {};
if (!input.environmentId) {
return {
executionTarget: null,
environmentName: null,
fallbackChecks: [],
release: noopRelease,
};
}
const requestedEnvironment = await environmentsSvc.getById(input.environmentId);
if (!requestedEnvironment) {
return {
executionTarget: null,
environmentName: null,
fallbackChecks: [
{
code: "environment_not_found",
level: "warn",
message: "Selected environment was not found. The test did not run.",
},
],
release: noopRelease,
};
}
// Managed-sandbox-only policy: redirect a Test that would run on the local
// host onto the platform-managed sandbox, the same as a real run does
// (resolveExecutionWorkspaceEnvironmentId in heartbeat). Without this
// redirect the Test probes the local host while the run executes in the
// managed sandbox, so a passing Test validates the wrong execution target.
// With no active managed sandbox the Test fails closed — never local.
let environment = requestedEnvironment;
if (requestedEnvironment.driver === "local") {
const managedSandboxOnly =
(await instanceSettings.getExperimental()).enableManagedSandboxOnly === true;
if (managedSandboxOnly) {
const managedSandboxEnvironment = await environmentsSvc.findManagedSandboxEnvironment(
input.companyId,
);
if (!managedSandboxEnvironment) {
return {
executionTarget: null,
environmentName: requestedEnvironment.name,
fallbackChecks: [
{
code: "managed_sandbox_unavailable",
level: "error",
message:
"This instance runs agents only in its platform-managed sandbox, but no active managed sandbox environment exists. The test did not run.",
hint: "Restore the managed sandbox environment, then test again.",
},
],
release: noopRelease,
};
}
environment = managedSandboxEnvironment;
}
}
if (environment.driver === "local") {
return {
executionTarget: null,
environmentName: environment.name,
fallbackChecks: [],
release: noopRelease,
};
}
if (environment.driver === "ssh") {
try {
const target = await resolveEnvironmentExecutionTarget({
db,
companyId: input.companyId,
adapterType: input.adapterType,
environment: {
id: environment.id,
driver: environment.driver,
config: environment.config ?? null,
},
leaseMetadata: null,
});
if (target) {
return {
executionTarget: target,
environmentName: environment.name,
fallbackChecks: [],
release: noopRelease,
};
}
return {
executionTarget: null,
environmentName: environment.name,
fallbackChecks: [
{
code: "environment_target_unavailable",
level: "warn",
message:
`Could not resolve an execution target for environment "${environment.name}". The test did not run.`,
},
],
release: noopRelease,
};
} catch (err) {
return {
executionTarget: null,
environmentName: environment.name,
fallbackChecks: [
{
code: "environment_target_failed",
level: "warn",
message:
`Could not connect to environment "${environment.name}" to run the test.`,
detail: err instanceof Error ? err.message : String(err),
},
],
release: noopRelease,
};
}
}
// sandbox / plugin / other remote drivers: spin up an ad-hoc lease, realize
// the workspace inside the box, and run the same probe SSH uses against
// a sandbox execution target wired to the environment runtime.
//
// We pass `heartbeatRunId: null` because there's no heartbeat run for an
// operator-initiated `Test` invocation — the leases table FKs heartbeat
// run id to heartbeat_runs.id, and we don't want to manufacture a fake
// run row. Cleanup goes through the driver's `releaseRunLease` directly
// (by lease record), since the batch helper queries by heartbeatRunId.
//
// Sandbox tests boot a fresh throwaway sandbox (never resume a retained
// agent lease) and archive it on release instead of deleting it, so the
// operator can inspect the exact sandbox from the provider dashboard while
// provider-side expiry reaps it later.
if (environment.driver === "plugin" && await pluginEnvironmentRequiresAgent(db, environment)) {
const [agent] = input.agentId ? await db.select({ state: agents.lifecycleState, pauseReason: agents.pauseReason,
requiresApproval: companies.requireBoardApprovalForNewAgents }).from(agents)
.innerJoin(companies, eq(companies.id, agents.companyId))
.where(and(eq(agents.id, input.agentId), eq(agents.companyId, input.companyId))) : [];
if (agent?.state === "paused" && agent.pauseReason === "setup" && agent.requiresApproval) return {
executionTarget: null, environmentName: environment.name, fallbackChecks: [{
code: "environment_agent_approval_required", level: "warn",
message: "The selected account will be verified on the agent computer after hire approval.",
}], release: noopRelease,
};
if (!agent || ["paused", "preparing"].includes(agent.state)) return {
executionTarget: null, environmentName: environment.name, fallbackChecks: [{
code: !input.agentId ? "environment_agent_required" : "environment_agent_preparing", level: "warn",
message: !input.agentId ? "This environment needs a saved agent before its connection can be tested."
: "The agent computer is being prepared. Test the connection when preparation finishes.",
}], release: noopRelease,
};
}
const testEnvironment = environment.driver === "sandbox"
? {
...environment,
config: {
...(environment.config ?? {}),
reuseLease: false,
archiveOnRelease: true,
},
}
: environment;
let leaseRecord: Awaited<ReturnType<typeof environmentRuntime.acquireRunLease>>;
try {
leaseRecord = await environmentRuntime.acquireRunLease({
companyId: input.companyId,
agentId: input.agentId,
adapterType: input.adapterType,
...(environment.driver === "plugin" && input.adapterType === "paperclip_runner"
? { requestedExpiresAt: new Date(Date.now() + 180_000) } : {}),
environment: testEnvironment,
configuredEnvironment: environment,
issueId: null,
heartbeatRunId: null,
persistedExecutionWorkspace: null,
// Re-check the company binding atomically at lease time. The route
// guard already rejected a foreign environment, but the binding could
// change between the guard check and the lease acquire. This closes
// that check-to-lease race so a foreign sandbox never gets a lease.
assertCompanyBinding: true,
// Apply the active custom-image template so the Test boots with the
// operator's captured sandbox customizations and prepared image state,
// matching what real agent runs use. Without this the test would
// silently fall back to the base image.
applyCustomImageTemplate: true,
});
} catch (err) {
return {
executionTarget: null,
environmentName: environment.name,
fallbackChecks: [
{
code: "environment_lease_acquire_failed",
level: "error",
message: `Could not acquire a lease for environment "${environment.name}".`,
detail: err instanceof Error ? err.message : String(err),
hint: "Check the environment's provider credentials and quota.",
},
],
release: noopRelease,
};
}
const driver = environmentRuntime.getDriver(environment.driver);
const releaseLease = async (status: "released" | "failed" = "released") => {
try {
if (driver) {
await driver.releaseRunLease({
environment: testEnvironment,
lease: leaseRecord.lease,
status,
});
} else {
await environmentsSvc.releaseLease(leaseRecord.lease.id, status);
}
} catch (err) {
// Cleanup failures must not mask the test result.
// eslint-disable-next-line no-console
console.warn(
`[adapter-test] Failed to release lease ${leaseRecord.lease.id}: ${err instanceof Error ? err.message : String(err)}`,
);
}
};
let realizedCwd: string | null = null;
try {
if (environment.driver === "plugin" && input.adapterType === "paperclip_runner") {
if (!await environmentRuntime.supportsTasks({ lease: leaseRecord.lease })) throw new Error("Plugin environment does not provide native Runner task execution");
const updated = await environmentsSvc.updateLeaseMetadata(leaseRecord.lease.id, {
...leaseRecord.lease.metadata, runnerTaskProbe: {
runId: randomUUID(), sessionId: randomUUID(), expiresAtMs: Date.now() + 120_000,
},
});
if (!updated) throw new Error("Plugin Runner setup lease disappeared");
leaseRecord.lease = updated;
} else {
const realized = await environmentRuntime.realizeWorkspace({
environment: testEnvironment,
lease: leaseRecord.lease,
// No host workspace to copy for a Test invocation; sandbox/plugin
// realize implementations use the lease metadata's remoteCwd to
// create the working directory inside the box.
workspace: {},
});
realizedCwd =
typeof realized.cwd === "string" && realized.cwd.trim().length > 0
? realized.cwd.trim()
: null;
}
} catch (err) {
await releaseLease("failed");
return {
executionTarget: null,
environmentName: environment.name,
fallbackChecks: [
{
code: "environment_workspace_realize_failed",
level: "error",
message: `Could not realize a workspace inside "${environment.name}".`,
detail: err instanceof Error ? err.message : String(err),
},
],
release: noopRelease,
};
}
let target: AdapterExecutionTarget | null;
try {
// Prefer the cwd the realize step returned; fall back to lease metadata.
const leaseMetadataForTarget: Record<string, unknown> | null =
realizedCwd
? { ...(leaseRecord.lease.metadata ?? {}), remoteCwd: realizedCwd }
: (leaseRecord.lease.metadata as Record<string, unknown> | null) ?? null;
target = await resolveEnvironmentExecutionTarget({
db,
companyId: input.companyId,
adapterType: input.adapterType,
environment: {
id: testEnvironment.id,
driver: testEnvironment.driver,
config: testEnvironment.config ?? null,
},
leaseId: leaseRecord.lease.id,
leaseMetadata: leaseMetadataForTarget,
lease: leaseRecord.lease,
environmentRuntime,
});
} catch (err) {
await releaseLease("failed");
return {
executionTarget: null,
environmentName: environment.name,
fallbackChecks: [
{
code: "environment_target_failed",
level: "error",
message: `Could not resolve an execution target for "${environment.name}".`,
detail: err instanceof Error ? err.message : String(err),
},
],
release: noopRelease,
};
}
if (!target) {
await releaseLease("failed");
return {
executionTarget: null,
environmentName: environment.name,
fallbackChecks: [
{
code: "environment_target_unsupported",
level: "warn",
message:
`Adapter "${input.adapterType}" is not allowed in "${environment.name}" environments.`,
},
],
release: noopRelease,
};
}
return {
executionTarget: target,
environmentName: environment.name,
fallbackChecks: [],
sandboxIdentityCheck: buildSandboxIdentityCheck({
environmentName: environment.name,
lease: leaseRecord.lease,
}),
release: releaseLease,
};
}
function readMetadataString(metadata: Record<string, unknown>, keys: string[]): string | null {
for (const key of keys) {
const value = metadata[key];
if (typeof value === "string" && value.trim().length > 0) return value.trim();
}
return null;
}
function buildSandboxIdentityCheck(input: {
environmentName: string;
lease: {
id: string;
provider?: string | null;
providerLeaseId?: string | null;
metadata?: Record<string, unknown> | null;
};
}): AdapterEnvironmentCheck {
const metadata = input.lease.metadata ?? {};
const provider = input.lease.provider ?? readMetadataString(metadata, ["provider"]);
const sandboxId = readMetadataString(metadata, ["sandboxId", "sandboxID", "sandbox_id", "id"]);
const sandboxName = readMetadataString(metadata, ["sandboxName", "sandbox_name", "name"]);
const snapshotRef = readMetadataString(metadata, [
"snapshot",
"snapshotId",
"snapshotID",
"snapshotRef",
"snapshot_ref",
"templateRef",
"template_ref",
"templateId",
"templateID",
"image",
"imageId",
"imageID",
"imageRef",
"image_ref",
]);
const templateKind = readMetadataString(metadata, [
"templateKind",
"template_kind",
"templateRefKind",
"template_ref_kind",
]);
const detailParts = [
`paperclipLeaseId=${input.lease.id}`,
input.lease.providerLeaseId ? `providerLeaseId=${input.lease.providerLeaseId}` : null,
provider ? `provider=${provider}` : null,
sandboxId ? `sandboxId=${sandboxId}` : null,
sandboxName ? `sandboxName=${sandboxName}` : null,
snapshotRef ? `${templateKind ? `${templateKind}Ref` : "snapshotOrTemplateRef"}=${snapshotRef}` : null,
].filter((part): part is string => Boolean(part));
return {
code: "sandbox_test_identity",
level: "info",
message: `Environment test identity for "${input.environmentName}".`,
detail: detailParts.join("; "),
hint: "Use these provider-neutral IDs when comparing model-test output with provider logs or refreshed environment snapshots.",
};
}
// The environment drivers the adapter Test route accepts. A local, SSH, or
// sandbox environment can host a probe; a plugin environment cannot.
const ADAPTER_TEST_ALLOWED_ENVIRONMENT_DRIVERS = ["local", "ssh", "sandbox"];
// The fail-closed tenant-binding guard for the adapter Test route. A caller
// may name any instance environment by id, so the route must reject an
// environment that binds to another company before it resolves secrets,
// merges env, resolves the target, leases a sandbox, or runs the adapter
// test. The guard checks the company binding BEFORE it validates the status
// or the driver, so it never reveals the status or the driver of a foreign
// environment. A same-company or an instance-global environment then gets the
// shared driver and status validation.
async function assertAdapterTestEnvironmentForCompany(
companyId: string,
environmentId: string,
adapterType?: string,
): Promise<void> {
if (!await assertAdapterTestEnvironmentBinding(companyId, environmentId)) return;
await assertEnvironmentSelectionForCompany(environmentsSvc, companyId, environmentId, {
allowedDrivers: adapterType === "paperclip_runner" ? [...ADAPTER_TEST_ALLOWED_ENVIRONMENT_DRIVERS, "plugin"] : ADAPTER_TEST_ALLOWED_ENVIRONMENT_DRIVERS,
});
}
// The company-binding half of the guard. A route that must resolve the
// runner before it knows the adapter type runs this first, then runs the
// full guard with the resolved adapter type. Returns the environment, or
// null for a missing one, which leaks no tenant state; the execution-context
// resolver surfaces the existing environment_not_found check.
async function assertAdapterTestEnvironmentBinding(companyId: string, environmentId: string) {
const environment = await environmentsSvc.getById(environmentId);
if (!environment) return null;
const boundCompanyIds = await environmentsSvc.listBoundCompanyIds(environmentId);
if (boundCompanyIds.length > 0 && !boundCompanyIds.includes(companyId)) {
throw forbidden("The selected environment belongs to another company.", {
code: "environment_company_mismatch",
});
}
return environment;
}
async function testManagedEnvironment(adapterType: string, context: Parameters<ReturnType<typeof requireServerAdapter>["testEnvironment"]>[0], binding: AiConnectionBinding, managed: Awaited<ReturnType<typeof prepareManagedAiRuntime>>, agentId?: string) {
const startedAt = new Date();
const target = context.executionTarget;
if (managed.credentialWriteback && target?.kind === "remote" && target.transport === "sandbox" && target.runnerTask && target.leaseId) {
await registerPluginTaskCredentialHandoff(db, { companyId: context.companyId, leaseId: target.leaseId }, managed.sessionIdentity, managed.credentialWriteback);
}
const result = await probeManagedEnvironment(adapterType, context, binding, managed);
// A provider rejection invalidates the tested credential generation. A
// missing CLI, unavailable environment, or other runtime error does not.
if (result.status === "fail" && result.checks.some(check =>
check.code === ADAPTER_AUTH_MISSING_CHECK_CODE || /_hello_probe_auth_required$/.test(check.code)
|| check.code === "ai_connection_api_key_rejected" || check.code === "COPILOT_AUTH_REQUIRED",
)) {
await aiConnectionService(db).markAuthenticationFailed({
companyId: context.companyId, agentId, runStartedAt: startedAt,
attribution: { ...managed.attribution, identity: managed.identity },
});
}
return result;
}
async function probeManagedEnvironment(adapterType: string, context: Parameters<ReturnType<typeof requireServerAdapter>["testEnvironment"]>[0], binding: AiConnectionBinding, managed: Awaited<ReturnType<typeof prepareManagedAiRuntime>>) {
if (binding.provider === "github") {
try {
const token = parseObject(context.config.env).COPILOT_GITHUB_TOKEN;
if (typeof token !== "string" || !token) throw unprocessable("Select a saved Copilot token.", { code: "COPILOT_AUTH_REQUIRED" });
const model = typeof context.config.model === "string" ? context.config.model.trim() : null;
if (!model || ["auto", "default"].includes(model.toLowerCase())) throw unprocessable("Select an available Copilot model.", { code: "COPILOT_MODEL_UNAVAILABLE" });
await probeCopilotExecutionTarget(token, context.executionTarget, model);
return { adapterType, status: "pass" as const, testedAt: new Date().toISOString(), checks: [{ code: "copilot_metadata_verified", level: "info" as const, message: "The verified Copilot runtime authenticated this account and accepted the selected model. No model prompt was sent." }] };
} catch (error) {
return { adapterType, status: "fail" as const, testedAt: new Date().toISOString(), checks: [{ code: error instanceof HttpError ? String(asRecord(error.details)?.code ?? "COPILOT_REQUEST_FAILED") : "COPILOT_REQUEST_FAILED", level: "error" as const, message: error instanceof HttpError ? error.message : "Copilot metadata verification failed." }] };
}
}
await assertManagedAiProjectAuth(context.config, binding.provider, context.executionTarget);
const target = context.executionTarget;
const result = adapterType === "paperclip_runner" && target?.kind === "remote" && target.transport === "sandbox" && target.runnerTask
? await probePluginRunnerEnvironment(context, { identity: managed.identity, accountIdentity: managed.sessionIdentity,
home: managed.home ? join(managed.home, "provider") : undefined })
: await requireServerAdapter(adapterType).testEnvironment(context);
if (result.status === "fail") return result;
// The resolved method, not binding.method — on a responsible_user binding
// that field is wire-compat only and the default connection decides.
if (adapterType === "paperclip_runner" && context.executionTarget?.kind === "remote"
&& context.executionTarget.transport === "sandbox" && context.executionTarget.runnerTask) return result;
const resolvedMethod = (context.config as { managedAiConnection?: { method?: string } }).managedAiConnection?.method;
// An api_key account does not take the CLI hello probe below: a key
// travels as an env var any engine understands, and the engine's own test
// above judged whether this runtime can execute with it — the ACP lane
// deliberately runs no hello probe when a key is configured. Demanding one
// anyway forced the CLI lane, whose probe needs a provider CLI on PATH,
// and a clean install has none: that walled off onboarding's API-key path
// on exactly the machines the release smoke exists to guard. The account
// is still verified live here — the same provider-endpoint check the save
// performed — so a key revoked since its save fails adoption rather than
// producing an agent that cannot authenticate at runtime. The hello-probe
// requirement stays for subscriptions: a stored login is a file layout
// only a provider CLI reads, so proving the runtime lane can consume it
// takes a real hello turn.
const nativePiOpenRouter = adapterType === "paperclip_runner" && context.config.provider === "acpx"
&& context.config.acpxAgent === "pi" && binding.provider === "openrouter"
&& asRecord(context.config.managedAiRouting)?.kind === "openrouter";
if (resolvedMethod === "api_key" && (!context.config.managedAiRouting || nativePiOpenRouter)) {
const envKey = AI_CONNECTION_CAPABILITIES[binding.provider].methods.api_key?.envKey;
const key = envKey ? parseObject(context.config.env)[envKey] : undefined;
try {
if (typeof key !== "string" || !key) throw unprocessable("The selected account's API key was not available to verify.");
await validateAiApiKey(binding.provider, key);
result.checks.push({ code: "ai_connection_api_key_reverified", level: "info", message: "The provider verified this API key for adoption." });
} catch (error) {
result.status = "fail";
const rejected = error instanceof HttpError && asRecord(error.details)?.code === "ai_connection_api_key_rejected";
result.checks.push({ code: rejected ? "ai_connection_api_key_rejected" : "ai_connection_verification_failed", level: "error", message: error instanceof HttpError ? error.message : "Could not verify the account. Try again." });
}
return finishManagedConnectionTest(adapterType, context, result);
}
let providerTestPassed = false;
if (!result.checks.some(check => check.code.includes("hello_probe"))) {
const providerAdapter = context.config.managedAiRouting ? aiRoutingHarness(adapterType, context.config.provider, context.config.acpxAgent) : { anthropic: "claude_local", openai: "codex_local", openrouter: "opencode_local", xai: "grok_local", google: "gemini_local" }[binding.provider];
const probe = await requireServerAdapter(providerAdapter).testEnvironment({ ...context, adapterType: providerAdapter, config: { ...context.config, engine: "cli" } });
providerTestPassed = probe.status === "pass";
result.checks.push(...probe.checks);
result.status = probe.status === "fail" ? "fail" : result.status === "warn" || probe.status === "warn" ? "warn" : "pass";
}
if (!result.checks.some(check => /hello_probe_(passed|succeeded)$/.test(check.code))) {
result.status = "fail";
result.checks.push({ code: "ai_connection_validation_incomplete", level: "error", message: "The selected account has not completed a provider hello test. Retry before adopting it." });
}
return providerTestPassed ? finishManagedConnectionTest(adapterType, context, result) : result;
}
return { resolveAdapterTestEnvironmentId, resolveAdapterTestExecutionContext, assertAdapterTestEnvironmentForCompany, assertAdapterTestEnvironmentBinding, testManagedEnvironment };
}