mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-28 04:02:18 +01:00
perf(runner): reuse compatible protected provider packs (#17749)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native Runner starts provider runtimes in remote execution environments. > - Sandbox images already contain qualified provider packs. > - Startup required the image pack to match the entire current app release manifest. > - An unrelated app commit could therefore cause pack verification, transfer, and installation again. > - This pull request selects protected packs by their provider interface and platform. > - The benefit is no extra pack probe, directory scan, version command, link, or upload on the compatible image path. ## Linked Issues or Issue Description **What happened?** Starting a remote task can reject a compatible image pack because its source revision or unrelated provider differs from the controller. Pack replacement added about 16 seconds in the investigated startup. Startup also repeated full content checks. **Expected behavior** Use an already qualified, protected pack when its selected provider interface and target platform are compatible. Keep the selected artifact identity for process launch and recovery. **Steps to reproduce** 1. Install a qualified provider pack in a sandbox image. 2. Start a controller built from another app commit. 3. Start a remote OpenCode task. The old complete-manifest comparison rejects the image pack even when the provider interface still works. Related repairs: #16493 reduced transfer cost. #17512 retained exact artifact authority during reattachment. This change preserves that recovery rule and changes fresh-launch pack selection. ## What Changed - Add explicit OpenCode and ACPX interface revisions to produced pack manifests. App Git SHAs and dependency pins remain provenance. - Read bounded protected-installation metadata in the existing final runner metadata query. Accept only root-owned paths that the execution user cannot write, including ancestors and manifest. Reject writable aliases. - Select and prepare the pack in server-owned environment setup before constructing the existing transport. Bind provider launch paths and digests from the selected installation. Save that identity for exact reattachment. - Keep full content verification for uploads, writable installations, and legacy packs. Reuse successful verification within one startup. Keep older attached runners compatible with the optional metadata request. - Seal image packs and document the one-time image refresh that enables reuse across later compatible releases. Keep selected ACPX agent qualification checks. ## Verification - Red/green regression: the old code rejects a protected pack from another app revision with `manifest mismatch`; the fixed path selects it without executing verification commands. - Native startup regression: select the installed pack's own Node digest with one existing runner metadata request and no pack verification or version commands. - Targeted pack, artifact, recovery, execution-driver, and Rust metadata tests pass. Incompatible interfaces/platforms, malformed identities, writable roots, path aliases, and exact reattachment are covered. - Final revision `91dc1676ff67bdbe5866c8586daac10240530cca`: workspace typecheck and build pass. All 53 current-head checks are successful, with no pending or failed checks. Greptile is 5/5 with no open findings. - The broad local `pnpm test:run` attempt reported a workspace-runtime conflict test failure; its isolated rerun passed. The broad run was stopped after that failure. Full CI now passes, including the server test shards. This is not a claim that the full local run passed. - Linux filesystem evidence: the production metadata functions read a real root-owned preinstalled pack as a non-root user in 16 microseconds median and 20 microseconds p95 across 100 warm reads. This measures metadata admission, not model or hosted response latency. - All 910 affected server tests and 57 Copilot attestation tests pass locally on the final implementation. Full CI covers the unchanged Runner transport. No provider qualification hash was refreshed. - Built the actual Linux daemon at `416e7ec7425d0d3565b4a5e1808002d650a60414` and ran its metadata command as UID 65534. The protected installation returned its real manifest with `immutable: true`; writable alias and missing-pack cases returned `immutable: false`. This is metadata/permission evidence, not provider execution. - Hosted rollout and a hosted end-to-end latency measurement have not run. ## Risks - Existing images need one rebuild to publish the compatibility declaration. Packs without it keep the legacy content-verification path. - Fast admission relies on image publication producing a qualified protected tree. Root execution and task-writable installations retain full verification. - Bump an interface revision when required runner/pack behavior changes incompatibly. Deploy a new image to roll out compatible runtime fixes or upgrades. - This removes provider-pack preparation delay. Other runner startup, context, model, and finalization costs remain separate. ## Model Used OpenAI Codex, configured model `gpt-6.1-sol`, reasoning effort `xhigh`, with tool use and code execution. The context-window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed
1 parent
f1922d95e1
commit
9ae4c25e7e
20 files changed
+570
-72
No files matched your search
@@ -100,6 +100,30 @@ describe("remote native artifact preparation", () => {
|
||||
expect(f.requests.some(request => request.command === "npm")).toBe(false);
|
||||
});
|
||||
|
||||
it("reads installed pack metadata in the existing final runner probe", async () => {
|
||||
const f = await fixture();
|
||||
const installation = { root: "/opt/paperclip-runner/provider-pack", manifest: { digest: "selected" }, immutable: true };
|
||||
await writeFile(f.runnerSource, `#!/bin/sh\n[ "$1" = --build-metadata ] || exit 17\nprintf '%s\\n' '${JSON.stringify({ ...metadata, providerPacks: [installation, { ...installation, root: "/unrequested" }] })}'\n`, { mode: 0o700 });
|
||||
const artifacts = createRemoteNativeArtifactPreparation({ ...f.options, providerPackRoots: [installation.root] });
|
||||
await artifacts.prepare("listen_ws");
|
||||
expect(artifacts.providerPackInventory()).toEqual([installation]);
|
||||
const probes = f.requests.filter(request => request.command === f.runnerTarget && request.args?.[0] === "--build-metadata");
|
||||
expect(probes).toHaveLength(1);
|
||||
expect(probes[0].args).toEqual(["--build-metadata", "--provider-pack-root", installation.root]);
|
||||
expect(f.requests.some(request => request.args?.includes("--version") && request.command.includes("provider-pack"))).toBe(false);
|
||||
});
|
||||
|
||||
it("reattaches an older runner without the optional pack metadata argument", async () => {
|
||||
const f = await fixture();
|
||||
await writeFile(f.runnerSource, `#!/bin/sh\n[ "$#" = 1 ] && [ "$1" = --build-metadata ] || exit 17\nprintf '%s\\n' '${JSON.stringify(metadata)}'\n`, { mode: 0o700 });
|
||||
const artifacts = createRemoteNativeArtifactPreparation({ ...f.options, providerPackRoots: ["/opt/paperclip-runner/provider-pack"] });
|
||||
await artifacts.verifyRemoteRunner("listen_ws", f.runnerSource, true);
|
||||
expect(artifacts.providerPackInventory()).toEqual([]);
|
||||
expect(f.requests.map(request => request.args)).toEqual([
|
||||
["--build-metadata", "--provider-pack-root", "/opt/paperclip-runner/provider-pack"], ["--build-metadata"],
|
||||
]);
|
||||
});
|
||||
|
||||
it("keeps providers without concurrent sync support serial", async () => {
|
||||
const f = await fixture({ concurrent: false });
|
||||
await createRemoteNativeArtifactPreparation(f.options).prepare("listen_ws");
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import type { AdapterExecutionTarget } from "@paperclipai/adapter-utils/execution-target";
|
||||
import type { RunnerIngressEndpoint } from "@paperclipai/adapter-utils/runner-connectivity";
|
||||
import { createNativeRunnerExecutionDriver } from "./native-runner-execution-driver.js";
|
||||
import { prepareNativeRemoteRunner, createNativeRunnerExecutionDriver } from "./native-runner-execution-driver.js";
|
||||
|
||||
const transport = vi.hoisted(() => ({
|
||||
register: vi.fn(),
|
||||
@@ -181,3 +181,12 @@ describe("shared Runner execution driver", () => {
|
||||
await expect(connection.ready!()).rejects.toThrow("process_identity_invalid");
|
||||
});
|
||||
});
|
||||
|
||||
it("validates connectivity before resolving provider artifacts without acquiring an ingress endpoint", async () => {
|
||||
const prepare = vi.fn(async () => {});
|
||||
await expect(prepareNativeRemoteRunner({ target: ssh, runId: "run", prepareRemote: prepare })).rejects.toThrow("runner_transport_ineligible");
|
||||
expect(prepare).not.toHaveBeenCalled();
|
||||
await prepareNativeRemoteRunner({ target: sandbox(), runnerIngressAuthorized: true, runId: "run", prepareRemote: prepare });
|
||||
expect(prepare).toHaveBeenCalledExactlyOnceWith("listen_ws");
|
||||
expect(transport.connect).not.toHaveBeenCalled();
|
||||
});
|
||||
@@ -59,6 +59,30 @@ export function resolveRemoteRunnerTransportMode(input: {
|
||||
return requiredMode;
|
||||
}
|
||||
|
||||
type RemotePreparation = Pick<NativeRunnerExecutionDriverInput,
|
||||
"target" | "runnerIngressAuthorized" | "runnerPublicUrl" | "runnerCaBundlePath" | "prepareRemote"> & { runId: string };
|
||||
|
||||
async function resolveOutboundTransport(input: RemotePreparation): Promise<PaperclipRunnerTransport> {
|
||||
const transport = await resolvePaperclipRunnerTransport({ target: input.target, runId: input.runId,
|
||||
localConnectUrl: "ws://127.0.0.1/unused", runnerPublicUrl: input.runnerPublicUrl,
|
||||
runnerCaBundlePath: input.runnerCaBundlePath, runnerIngressAuthorized: input.runnerIngressAuthorized === true });
|
||||
if (transport.mode === "direct_outbound" && transport.caBundlePath && !existsSync(transport.caBundlePath)) {
|
||||
throw new Error("runner_direct_wss_failed: configured runner CA bundle is unavailable");
|
||||
}
|
||||
return transport;
|
||||
}
|
||||
|
||||
/** Resolve artifacts before provider-profile binding. Endpoint acquisition stays
|
||||
* in connection registration; outbound eligibility is checked before staging.
|
||||
*/
|
||||
export async function prepareNativeRemoteRunner(input: RemotePreparation) {
|
||||
const mode = resolveRemoteRunnerTransportMode({ target: input.target,
|
||||
runnerIngressAuthorized: input.runnerIngressAuthorized === true });
|
||||
if (mode === "dial_wss") await resolveOutboundTransport(input);
|
||||
if (!input.prepareRemote) throw new Error("runner_remote_preparation_unavailable");
|
||||
await input.prepareRemote(mode);
|
||||
}
|
||||
|
||||
/** Built-in environment lifecycle. PRP commands and authentication are shared;
|
||||
* only environment preparation, process ownership, and connection routing vary.
|
||||
*/
|
||||
@@ -116,28 +140,8 @@ export function createNativeRunnerExecutionDriver(input: NativeRunnerExecutionDr
|
||||
transport = await measureNativeRunnerSpan(
|
||||
input.trace,
|
||||
"runner.transport.resolve",
|
||||
() =>
|
||||
resolvePaperclipRunnerTransport({
|
||||
target: input.target,
|
||||
runId:
|
||||
attachmentIdentity?.runId ??
|
||||
input.binding.runId,
|
||||
localConnectUrl: "ws://127.0.0.1/unused",
|
||||
runnerPublicUrl: input.runnerPublicUrl,
|
||||
runnerCaBundlePath: input.runnerCaBundlePath,
|
||||
runnerIngressAuthorized:
|
||||
input.runnerIngressAuthorized === true,
|
||||
}),
|
||||
() => resolveOutboundTransport({ ...input, runId: attachmentIdentity?.runId ?? input.binding.runId }),
|
||||
);
|
||||
if (
|
||||
transport.mode === "direct_outbound" &&
|
||||
transport.caBundlePath &&
|
||||
!existsSync(transport.caBundlePath)
|
||||
) {
|
||||
throw new Error(
|
||||
"runner_direct_wss_failed: configured runner CA bundle is unavailable",
|
||||
);
|
||||
}
|
||||
await measureNativeRunnerSpan(
|
||||
input.trace,
|
||||
"runner.artifact.prepare",
|
||||
|
||||
@@ -1440,6 +1440,7 @@ describe("remote provider pack manifest", () => {
|
||||
opencodeProxy: "dist/cli/opencode-app-server-proxy.cjs", acpxSidecar: "dist/cli/acpx-runtime-sidecar.cjs",
|
||||
}).map(([key, path]) => [key, { path, sha256: digest }])) as Record<"nodeCommand" | "opencodeCommand", { path: string; sha256: string }>;
|
||||
const payload = {
|
||||
compatibility: { opencode: 1, acpx: 1 },
|
||||
target: { platform: "linux", architecture }, runnerSourceRevision: "a".repeat(40),
|
||||
pins: { nodeMinimum: "24.11.0", codex: QUALIFIED_ACPX_PROFILES.codex.agentRuntimeVersion,
|
||||
opencode: "1.18.34", acpx: "0.13.1", grok: QUALIFIED_ACPX_PROFILES.grok.agentRuntimeVersion,
|
||||
@@ -1449,6 +1450,11 @@ describe("remote provider pack manifest", () => {
|
||||
distDigest: digest, artifacts,
|
||||
bridgeDigest: "sha256:" + createHash("sha256").update(digest + "\n" + digest + "\n" + digest).digest("hex"),
|
||||
};
|
||||
if (provider === "cursor") Object.assign(payload, { providers: { cursor: {
|
||||
version: QUALIFIED_ACPX_PROFILES.cursor.agentServerVersion, profileDigest: QUALIFIED_ACPX_PROFILES.cursor.commandDigest,
|
||||
closureDigest: `sha256:${CURSOR_DISTRIBUTION_PINS["linux-x64"].closureSha256}`, qualification: "qualified",
|
||||
path: "provider-assets/cursor/linux-x64", sha256: digest,
|
||||
} } });
|
||||
const manifest = { schema: "paperclip-runner/remote-provider-pack/v1", payload,
|
||||
digest: "sha256:" + createHash("sha256").update(canonical(payload)).digest("hex") };
|
||||
state.bundledPackManifestPath = join(root, "provider-pack.json");
|
||||
@@ -1463,11 +1469,22 @@ describe("remote provider pack manifest", () => {
|
||||
workspace: { ...execution.workspace, cwd: root },
|
||||
session: { ...execution.session, normalizedSessionId: `bundled-${provider}-${randomUUID()}`, driverKind: provider === "opencode" ? "opencode_server" : "acpx_runtime" },
|
||||
} as unknown as NativeExecutionInputV1;
|
||||
const syncIn = vi.fn(async () => undefined);
|
||||
const execute = vi.fn(async (command: { command: string; args?: string[] }) => {
|
||||
if (command.args?.[0] === "--build-metadata") throw new Error("reached-staged-Linux-daemon-verification");
|
||||
return { exitCode: 0, signal: null, timedOut: false, stdout: "", stderr: "" };
|
||||
});
|
||||
const syncIn = vi.fn(async (_operations: { files: { kind: string }[] }[]) => undefined);
|
||||
const installed = structuredClone(manifest);
|
||||
if (provider === "opencode") {
|
||||
installed.payload.runnerSourceRevision = "c".repeat(40);
|
||||
installed.payload.artifacts.nodeCommand.sha256 = "sha256:" + "d".repeat(64);
|
||||
installed.digest = "sha256:" + createHash("sha256").update(canonical(installed.payload)).digest("hex");
|
||||
}
|
||||
const execute = vi.fn(async (command: { command: string; args?: string[] }) => ({
|
||||
exitCode: 0, signal: null, timedOut: false, stderr: "",
|
||||
stdout: command.args?.[0] === "--build-metadata" ? JSON.stringify({
|
||||
schema: "paperclip-runner/runnerd-build-metadata/v1", binaryName: "paperclip-runnerd",
|
||||
packageName: "@paperclipai/paperclip-runner", binaryContractVersion: 2,
|
||||
durableSessionCapabilities: ["unlimited_runtime", "connection_lease_renewal"], prpTransportModes: ["listen_ws"],
|
||||
providerPacks: [{ root: "/opt/paperclip-runner/provider-pack", manifest: installed, immutable: true }],
|
||||
}) : "",
|
||||
}));
|
||||
try {
|
||||
await createRunnerdBackend({ db: leaseDb(remoteExecution), execution: remoteExecution,
|
||||
runnerInstanceId: "bundled-image-runner", runnerIngressAuthorized: true,
|
||||
@@ -1491,12 +1508,17 @@ describe("remote provider pack manifest", () => {
|
||||
const { bundledRemoteProviderPackManifestPath } = await import("../../vendor/paperclip-runner/index.js");
|
||||
expect(bundledRemoteProviderPackManifestPath).toHaveBeenCalledWith(undefined, `linux-${architecture}`);
|
||||
}
|
||||
await expect(transport.controlPlaneRegistration({})).rejects.toThrow("reached-staged-Linux-daemon-verification");
|
||||
expect(syncIn).toHaveBeenCalledWith([expect.objectContaining({ files: [expect.objectContaining({
|
||||
sourcePath: state.bundledRunnerBinary, targetPath: "/workspace/.paperclip-runtime/paperclip-runner/bin/paperclip-runnerd",
|
||||
})] })]);
|
||||
expect(execute.mock.calls.some(([call]) => call.args?.[1] === "uname -s; uname -m")).toBe(false);
|
||||
expect(existsSync(join(transport.stateDirectory!, "provider-pack-authority.json"))).toBe(false);
|
||||
expect(transport).toMatchObject({ providerPackAuthorityDigest: installed.digest,
|
||||
providerNodeCommand: "/opt/paperclip-runner/provider-pack/node_modules/node/bin/node",
|
||||
providerNodeCommandSha256: installed.payload.artifacts.nodeCommand.sha256 });
|
||||
expect(execute.mock.calls.filter(([call]) => call.args?.[0] === "--build-metadata")).toHaveLength(1);
|
||||
expect(execute.mock.calls.some(([call]) => call.args?.[0] === "-e" || call.args?.[0] === "--version")).toBe(false);
|
||||
expect(syncIn.mock.calls.every(([operations]) => operations.every(operation => operation.files.every(file => file.kind === "file")))).toBe(true);
|
||||
if (provider === "claude-x64") {
|
||||
const identity = transport.prpIdentity!;
|
||||
const artifact = { version: "0.0.0", digest: "sha256:" + createHash("sha256").update(daemon).digest("hex") };
|
||||
@@ -1514,24 +1536,30 @@ describe("remote provider pack manifest", () => {
|
||||
await writeFile(state.bundledPackManifestPath!, JSON.stringify(next));
|
||||
await writeFile(state.bundledRunnerBinary!, Buffer.concat([daemon, Buffer.from("new release")]));
|
||||
const recoveryExecute = vi.fn(async (request: { args: string[] }) => {
|
||||
if (request.args[0] === "--build-metadata") return { exitCode: 0, timedOut: false, stderr: "", stdout: JSON.stringify({
|
||||
schema: "paperclip-runner/runnerd-build-metadata/v1", binaryName: "paperclip-runnerd",
|
||||
packageName: "@paperclipai/paperclip-runner", binaryContractVersion: 2,
|
||||
durableSessionCapabilities: ["unlimited_runtime", "connection_lease_renewal"], prpTransportModes: ["listen_ws"],
|
||||
providerPacks: [{ root: "/opt/paperclip-runner/provider-pack", manifest, immutable: true }],
|
||||
}) };
|
||||
if (request.args[0] === "-e" || request.args[0] === "--version") return { exitCode: 0, timedOut: false, stderr: "", stdout: request.args[0] === "--version" ? "1.18.34" : "" };
|
||||
const script = request.args[1];
|
||||
const stdout = script.includes("base64") ? Buffer.from(JSON.stringify({ ...identity, lifecycle: "running" })).toString("base64")
|
||||
: script.includes("cat --") ? `nonce\n123\n2026-09-19T10:00:00.000Z\n${identity.runnerInstanceId}\nlinux:abcd-1234:100\n` : "";
|
||||
return { stdout, stderr: "", exitCode: 0, timedOut: false };
|
||||
});
|
||||
await createRunnerdBackend({ db: leaseDb(remoteExecution), execution: remoteExecution,
|
||||
runnerInstanceId: identity.runnerInstanceId,
|
||||
runnerInstanceId: identity.runnerInstanceId, runnerIngressAuthorized: true,
|
||||
runnerExecutionTarget: { kind: "remote", transport: "sandbox", remoteCwd: "/workspace",
|
||||
environmentId: "environment", leaseId: "lease", providerKey: "daytona", runner: { execute: recoveryExecute },
|
||||
environmentId: "environment", leaseId: "lease", providerKey: "daytona", effectiveCapabilities: { runnerWebSocketIngress: true }, runner: { execute: recoveryExecute },
|
||||
sandboxLeaseAcquisition: { outcome: "resumed", providerLeaseId: "sandbox" } } as never,
|
||||
restartRecovery: { kind: "reattach_remote_runner", runId: remoteExecution.binding.runId,
|
||||
leaseOwner: "controller", controllerGeneration: 2, providerAttempt: 1, restartKind: "graceful", recoveryRequestId: null,
|
||||
remote: { providerLeaseId: "sandbox", remoteCwd: "/workspace" } },
|
||||
});
|
||||
state.createBackend.mock.calls.at(-1)![1].codexTransportFactory!();
|
||||
expect(state.createTransport.mock.calls.at(-1)![0]).toMatchObject({
|
||||
providerPackAuthorityDigest: manifest.digest, runnerArtifact: artifact, adoptExistingRunner: { pid: 123 },
|
||||
});
|
||||
const recovered = state.createTransport.mock.calls.at(-1)![0];
|
||||
expect(recovered).toMatchObject({ providerPackAuthorityDigest: manifest.digest, runnerArtifact: artifact, adoptExistingRunner: { pid: 123 } });
|
||||
}
|
||||
} finally {
|
||||
state.bundledPackManifestPath = null; state.bundledRunnerBinary = null;
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { providerPackWorkspaceId, recordAdmittedRemoteProviderPack, resolveRemoteProviderPackAuthority } from "./remote-provider-pack-authority.js";
|
||||
import { verifyRemoteProviderPack as verifyRemoteProviderPackIdentity } from "./remote-provider-pack-verification.js";
|
||||
import { type ProviderPackInstallation, verifyRemoteProviderPack as verifyRemoteProviderPackIdentity } from "./remote-provider-pack-verification.js";
|
||||
import { writeEnvironmentCleanup } from "../environment-cleanup-notifications.js";
|
||||
import { DELIVERY_QUEUES } from "../delivery-work-notifications.js";
|
||||
import { writeExecutionControl } from "../execution-control-notifications.js";
|
||||
@@ -23,7 +23,7 @@ import { isCloudManagedInstance } from "../cloud-instance.js";
|
||||
import { dotRunnerBroker } from "../dot-runner-broker.js";
|
||||
import { CURSOR_DISTRIBUTION_PINS, QUALIFIED_ACPX_PROFILES, QUALIFIED_ACPX_VERSION } from "../../vendor/paperclip-runner/index.js";
|
||||
import { isProviderMode } from "../../vendor/paperclip-runner/index.js";
|
||||
import { bundledRemoteProviderPackManifestPath, bundledRemoteRunnerBinary, runnerBinaryTarget } from "../../vendor/paperclip-runner/index.js";
|
||||
import { PROVIDER_PACK_COMPATIBILITY, bundledRemoteProviderPackManifestPath, bundledRemoteRunnerBinary, runnerBinaryTarget } from "../../vendor/paperclip-runner/index.js";
|
||||
import { nativeRetryCancellationEligible, rethrowNativeCancellationLockConflict, assertCancellationRequest, cancellationIntentId as callerCancellationIntentId, cancellationRequestId } from "./native-cancellation-request.js";
|
||||
import { readNativePlanWait } from "./native-plan-wait.js";
|
||||
import { nativeProviderLifecycle } from "./provider-lifecycle.js";
|
||||
@@ -133,7 +133,7 @@ import { createNativeSshCommandRunner } from "./native-ssh-command-runner.js";
|
||||
import { discoverRemoteExecutable, parseRemoteExecutableCandidate } from "./codex-model-fallback.js";
|
||||
export { parseRemoteExecutableCandidate } from "./codex-model-fallback.js";
|
||||
import type { CommandManagedRuntimeRunner } from "@paperclipai/adapter-utils/command-managed-runtime";
|
||||
import { createNativeRunnerExecutionDriver, resolveRemoteRunnerTransportMode } from "./native-runner-execution-driver.js";
|
||||
import { prepareNativeRemoteRunner, createNativeRunnerExecutionDriver, resolveRemoteRunnerTransportMode } from "./native-runner-execution-driver.js";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import {
|
||||
and,
|
||||
@@ -10316,6 +10316,7 @@ type RemoteProviderPackManifest = {
|
||||
schema: typeof REMOTE_PROVIDER_PACK_SCHEMA;
|
||||
digest: string;
|
||||
payload: {
|
||||
compatibility?: { opencode: number; acpx: number };
|
||||
pins: typeof REMOTE_PROVIDER_PACK_PINS;
|
||||
target: { platform: string; architecture: string };
|
||||
runnerSourceRevision: string;
|
||||
@@ -10427,6 +10428,7 @@ function readRemoteProviderPackIdentity(packRoot: string, verifyControllerFiles:
|
||||
if (
|
||||
manifest?.schema !== REMOTE_PROVIDER_PACK_SCHEMA ||
|
||||
!payload ||
|
||||
(payload.compatibility !== undefined && canonicalJson(payload.compatibility) !== canonicalJson(PROVIDER_PACK_COMPATIBILITY)) ||
|
||||
canonicalJson(payload.pins) !== canonicalJson(REMOTE_PROVIDER_PACK_PINS) ||
|
||||
canonicalJson(payload.acpxProfileDigests) !==
|
||||
canonicalJson(REMOTE_PROVIDER_PACK_PROFILE_DIGESTS) ||
|
||||
@@ -12141,7 +12143,9 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
|| !lstatSync(configuredProviderPackRoot).isDirectory()) {
|
||||
throw new Error("runner_remote_provider_artifact_incompatible: configure PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH with the build-owned provider pack");
|
||||
}
|
||||
return readRemoteProviderPackManifest(configuredProviderPackRoot);
|
||||
// Selection reads identity only. Content verification belongs to the
|
||||
// install path; a compatible image pack needs no controller tree scan.
|
||||
return readRemoteProviderPackIdentity(configuredProviderPackRoot, false);
|
||||
})();
|
||||
return currentProviderPackManifest;
|
||||
};
|
||||
@@ -12261,6 +12265,7 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
: Promise.resolve();
|
||||
|
||||
const verifyRemoteProviderPack = async (packRoot: string) => {
|
||||
if (providerPackVerified && packRoot === activeRemoteProviderPackRoot) return;
|
||||
if (!remoteTarget || !remoteCommandRunner || !expectedProviderPackManifest)
|
||||
return;
|
||||
await verifyRemoteProviderPackIdentity({ runner: remoteCommandRunner,
|
||||
@@ -12293,6 +12298,8 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
model: input.execution.provider.kind === "codex" ? input.execution.provider.model : null,
|
||||
reuseRetainedRunner: sandboxLeaseAcquisition?.outcome === "resumed", targetPlatformArtifactVerified: remotePiCompanion !== null || useBundledRemoteImageAssets || targetMatchedCodexArtifact || remoteClaude,
|
||||
trace: input.trace, onLog: input.onLog,
|
||||
providerPackRoots: requiresRemoteProviderPack && stagedRemoteProviderPackRoot
|
||||
? [stagedRemoteProviderPackRoot, "/opt/paperclip-runner/provider-pack"] : undefined,
|
||||
})
|
||||
: null;
|
||||
const verifyRemoteRunner = async (mode: "dial_wss" | "listen_ws", executable?: string | null) => {
|
||||
@@ -12300,6 +12307,33 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
};
|
||||
const verifyRemoteCodex = async () => { await remoteNativeArtifacts?.verifyRemoteCodex(); };
|
||||
|
||||
// A custom operator-selected pack is an explicit runtime selection. The
|
||||
// standard app-image path is the default transport source, not an upgrade request.
|
||||
const explicitProviderPackSelection = Boolean(input.runnerRemoteProviderPackPath?.trim()
|
||||
&& input.runnerRemoteProviderPackPath.trim() !== "/opt/paperclip-runner/provider-pack");
|
||||
const useInstalledProviderPack = async () => {
|
||||
const expected = expectedProviderPackManifest;
|
||||
if (!expected || !remoteTarget || !remoteCommandRunner) return false;
|
||||
for (const installation of remoteNativeArtifacts?.providerPackInventory() ?? []) {
|
||||
try {
|
||||
const manifest = await measureNativeRunnerSpan(input.trace, "provider_pack.compatibility", () => verifyRemoteProviderPackIdentity({
|
||||
runner: remoteCommandRunner, cwd: remoteTarget.remoteCwd, packRoot: installation.root,
|
||||
manifest: expected, installation,
|
||||
provider: input.execution.provider as { kind: "opencode" } | { kind: "acpx"; agent: string },
|
||||
requireExactIdentity: reattachProviderPack || explicitProviderPackSelection,
|
||||
}));
|
||||
if (!reattachProviderPack) providerPackAuthority!.select(manifest);
|
||||
expectedProviderPackManifest = manifest;
|
||||
activeRemoteProviderPackRoot = installation.root;
|
||||
providerPackVerified = true;
|
||||
return true;
|
||||
} catch (error) {
|
||||
if (error instanceof Error && error.name === "AbortError") throw error;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
};
|
||||
|
||||
const prepareRemoteRunner = async (
|
||||
requiredMode: "dial_wss" | "listen_ws",
|
||||
) => {
|
||||
@@ -12312,7 +12346,11 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
return;
|
||||
selectedRemoteMode = requiredMode;
|
||||
if (input.restartRecovery?.kind === "reattach_remote_runner" || input.remoteWarmRecovery) {
|
||||
await verifyRemoteRunner(requiredMode);
|
||||
await remoteNativeArtifacts?.verifyRemoteRunner(requiredMode, undefined, true);
|
||||
if (requiresRemoteProviderPack && await useInstalledProviderPack()) {
|
||||
remotePrepared = true;
|
||||
return;
|
||||
}
|
||||
if (requiresRemoteProviderPack && stagedRemoteProviderPackRoot) {
|
||||
await verifyRemoteProviderPack(stagedRemoteProviderPackRoot);
|
||||
activeRemoteProviderPackRoot = stagedRemoteProviderPackRoot;
|
||||
@@ -12321,6 +12359,10 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
return;
|
||||
}
|
||||
await remoteNativeArtifacts?.prepare(requiredMode);
|
||||
if (requiresRemoteProviderPack && await useInstalledProviderPack()) {
|
||||
remotePrepared = true;
|
||||
return;
|
||||
}
|
||||
if (
|
||||
requiresRemoteProviderPack &&
|
||||
expectedProviderPackManifest &&
|
||||
@@ -12383,8 +12425,10 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
"runner_remote_provider_artifact_incompatible: this remote transport cannot stage a provider pack; preinstall the exact manifest-matched pack",
|
||||
);
|
||||
}
|
||||
const archive = await measureNativeRunnerSpan(input.trace, "provider_pack.archive.verify", () =>
|
||||
readProviderPackArchive(configuredProviderPackRoot, expectedProviderPackManifest!.digest),
|
||||
const archive = await measureNativeRunnerSpan(input.trace, "provider_pack.archive.verify", () => {
|
||||
readRemoteProviderPackManifest(configuredProviderPackRoot);
|
||||
return readProviderPackArchive(configuredProviderPackRoot, expectedProviderPackManifest!.digest);
|
||||
},
|
||||
);
|
||||
const escapedPackRoot = stagedRemoteProviderPackRoot.replaceAll(
|
||||
"'",
|
||||
@@ -13365,6 +13409,11 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
if (!current) throw new Error("native_session_tool_authority_unavailable");
|
||||
return current.execute(call);
|
||||
};
|
||||
if (requiresRemoteProviderPack) {
|
||||
await prepareNativeRemoteRunner({ target, runnerIngressAuthorized: input.runnerIngressAuthorized === true,
|
||||
runnerPublicUrl: input.runnerPublicUrl, runnerCaBundlePath: input.runnerCaBundlePath,
|
||||
runId: input.execution.binding.runId, prepareRemote: prepareRemoteRunner });
|
||||
}
|
||||
const backend = createNativeSessionBackend(runnerExecution, {
|
||||
runnerInstanceId:
|
||||
retainedTransition?.runnerInstanceId ?? input.runnerInstanceId,
|
||||
@@ -13476,10 +13525,10 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
...(expectedProviderPackManifest && stagedRemoteProviderPackRoot
|
||||
...(expectedProviderPackManifest && activeRemoteProviderPackRoot
|
||||
? {
|
||||
providerNodeCommand: posix.join(
|
||||
stagedRemoteProviderPackRoot,
|
||||
activeRemoteProviderPackRoot,
|
||||
expectedProviderPackManifest.payload.artifacts.nodeCommand.path,
|
||||
),
|
||||
providerNodeCommandSha256:
|
||||
@@ -13487,7 +13536,7 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
.sha256,
|
||||
providerPackAuthorityDigest: expectedProviderPackManifest.digest,
|
||||
opencodeCommand: posix.join(
|
||||
stagedRemoteProviderPackRoot,
|
||||
activeRemoteProviderPackRoot,
|
||||
expectedProviderPackManifest.payload.artifacts
|
||||
.opencodeExecutable.path,
|
||||
),
|
||||
@@ -13495,7 +13544,7 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
expectedProviderPackManifest.payload.artifacts
|
||||
.opencodeExecutable.sha256,
|
||||
opencodeProxyPath: posix.join(
|
||||
stagedRemoteProviderPackRoot,
|
||||
activeRemoteProviderPackRoot,
|
||||
expectedProviderPackManifest.payload.artifacts.opencodeProxy
|
||||
.path,
|
||||
),
|
||||
@@ -13503,7 +13552,7 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
expectedProviderPackManifest.payload.artifacts.opencodeProxy
|
||||
.sha256,
|
||||
acpxSidecarPath: posix.join(
|
||||
stagedRemoteProviderPackRoot,
|
||||
activeRemoteProviderPackRoot,
|
||||
expectedProviderPackManifest.payload.artifacts.acpxSidecar.path,
|
||||
),
|
||||
acpxSidecarSha256:
|
||||
@@ -13768,6 +13817,7 @@ export function createRemoteNativeArtifactPreparation(input: {
|
||||
model?: string | null;
|
||||
reuseRetainedRunner?: boolean;
|
||||
targetPlatformArtifactVerified?: boolean;
|
||||
providerPackRoots?: readonly string[];
|
||||
trace?: NativeRunTrace;
|
||||
onLog?: (stream: "stdout" | "stderr", chunk: string) => Promise<void>;
|
||||
}) {
|
||||
@@ -13778,18 +13828,25 @@ export function createRemoteNativeArtifactPreparation(input: {
|
||||
const explicitRemoteCodex = input.runnerRemoteCodexPath?.trim() || null;
|
||||
const remoteCodexNpmSpec = input.runnerRemoteCodexNpmSpec?.trim() || null;
|
||||
if (explicitRemoteCodex && remoteCodexNpmSpec) throw new Error("runner_remote_codex_source_conflict");
|
||||
let providerPackInventory: ProviderPackInstallation[] = [];
|
||||
const verifyRemoteRunner = async (
|
||||
requiredMode: "dial_wss" | "listen_ws",
|
||||
executable = remoteBinary,
|
||||
inspectProviderPacks = false,
|
||||
) => {
|
||||
if (!remoteTarget || !remoteCommandRunner || !executable) return;
|
||||
const metadataResult = await remoteCommandRunner.execute({
|
||||
let metadataResult = await remoteCommandRunner.execute({
|
||||
command: executable,
|
||||
args: ["--build-metadata"],
|
||||
args: ["--build-metadata", ...(inspectProviderPacks ? (input.providerPackRoots ?? []).flatMap(root => ["--provider-pack-root", root]) : [])],
|
||||
cwd: remoteTarget.remoteCwd,
|
||||
bypassSession: true,
|
||||
timeoutMs: 30_000,
|
||||
});
|
||||
if (inspectProviderPacks && input.providerPackRoots?.length && metadataResult.exitCode !== null
|
||||
&& metadataResult.exitCode !== 0 && !metadataResult.timedOut && !metadataResult.signal) {
|
||||
metadataResult = await remoteCommandRunner.execute({ command: executable, args: ["--build-metadata"],
|
||||
cwd: remoteTarget.remoteCwd, bypassSession: true, timeoutMs: 30_000 });
|
||||
}
|
||||
if (metadataResult.exitCode !== 0 || metadataResult.timedOut) {
|
||||
throw new Error("runner_remote_artifact_verification_failed");
|
||||
}
|
||||
@@ -13802,6 +13859,12 @@ export function createRemoteNativeArtifactPreparation(input: {
|
||||
});
|
||||
}
|
||||
assertRemoteRunnerBuildMetadata(metadata, requiredMode);
|
||||
if (inspectProviderPacks) {
|
||||
providerPackInventory = Array.isArray(metadata.providerPacks)
|
||||
? metadata.providerPacks.filter((pack): pack is ProviderPackInstallation =>
|
||||
typeof pack?.root === "string" && input.providerPackRoots?.includes(pack.root) === true
|
||||
&& typeof pack.immutable === "boolean") : [];
|
||||
}
|
||||
};
|
||||
|
||||
const reportedCodexVersions = new Set<string | null>();
|
||||
@@ -13982,7 +14045,7 @@ export function createRemoteNativeArtifactPreparation(input: {
|
||||
});
|
||||
}
|
||||
await measureNativeRunnerSpan(input.trace, "runner.artifact.verify", () =>
|
||||
verifyRemoteRunner(requiredMode),
|
||||
verifyRemoteRunner(requiredMode, undefined, true),
|
||||
);
|
||||
};
|
||||
const prepareHarness = async () => {
|
||||
@@ -14097,5 +14160,5 @@ export function createRemoteNativeArtifactPreparation(input: {
|
||||
await prepareHarness();
|
||||
}
|
||||
};
|
||||
return { prepare, verifyRemoteRunner, verifyRemoteCodex };
|
||||
return { prepare, verifyRemoteRunner, verifyRemoteCodex, providerPackInventory: () => providerPackInventory };
|
||||
}
|
||||
@@ -119,3 +119,14 @@ it("retains the original projectless lease across warm run placeholders but reje
|
||||
const changed = { ...binding, workspaceId: providerPackWorkspaceId("project-workspace", "run-2", "original-lease") };
|
||||
expect(() => resolveRemoteProviderPackAuthority({ ...input, binding: changed, reattach: true })).toThrow("does not match this execution");
|
||||
});
|
||||
|
||||
it("persists the compatible installation selected at launch, then pins it on reattachment", () => {
|
||||
const { input } = fixture();
|
||||
const authority = resolveRemoteProviderPackAuthority(input);
|
||||
const installed = manifest("b");
|
||||
authority.select(installed);
|
||||
authority.verified();
|
||||
const attached = resolveRemoteProviderPackAuthority({ ...input, reattach: true });
|
||||
expect(attached.manifest).toEqual(installed);
|
||||
expect(() => attached.select(manifest("c"))).toThrow("cannot replace an attached pack authority");
|
||||
});
|
||||
@@ -33,7 +33,7 @@ export function resolveRemoteProviderPackAuthority<T extends ProviderPackIdentit
|
||||
reattach: boolean;
|
||||
current: () => T;
|
||||
runnerArtifact: () => RunnerArtifactIdentity;
|
||||
}): { manifest: T; runnerArtifact: RunnerArtifactIdentity; verified: () => void } {
|
||||
}): { manifest: T; runnerArtifact: RunnerArtifactIdentity; select: (manifest: T) => void; verified: () => void } {
|
||||
const root = lstatSync(input.stateDirectory);
|
||||
if (!root.isDirectory() || root.isSymbolicLink()) {
|
||||
throw new Error("runner_remote_provider_artifact_incompatible: provider pack authority directory is unsafe");
|
||||
@@ -68,11 +68,22 @@ export function resolveRemoteProviderPackAuthority<T extends ProviderPackIdentit
|
||||
assertManifest(manifest);
|
||||
runnerArtifact ??= input.runnerArtifact();
|
||||
const selectedRunner = structuredClone(runnerArtifact);
|
||||
const selected = structuredClone(manifest);
|
||||
const receipt = JSON.stringify({ schema, binding: input.binding, manifest: selected, runnerArtifact: selectedRunner });
|
||||
if (Buffer.byteLength(receipt) > maximumBytes) throw new Error("runner_remote_provider_artifact_incompatible: provider pack authority exceeds its size limit");
|
||||
let selected = structuredClone(manifest);
|
||||
const serialize = (manifest: T) => {
|
||||
const receipt = JSON.stringify({ schema, binding: input.binding, manifest, runnerArtifact: selectedRunner });
|
||||
if (Buffer.byteLength(receipt) > maximumBytes) throw new Error("runner_remote_provider_artifact_incompatible: provider pack authority exceeds its size limit");
|
||||
return receipt;
|
||||
};
|
||||
let receipt = serialize(selected);
|
||||
return {
|
||||
manifest: selected,
|
||||
get manifest() { return selected; },
|
||||
select: (manifest: T) => {
|
||||
if (input.reattach) throw new Error("runner_remote_provider_artifact_incompatible: cannot replace an attached pack authority");
|
||||
assertManifest(manifest);
|
||||
const next = structuredClone(manifest);
|
||||
receipt = serialize(next);
|
||||
selected = next;
|
||||
},
|
||||
runnerArtifact: selectedRunner,
|
||||
verified: () => {
|
||||
const temporary = `${path}.${randomUUID()}.tmp`;
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { execFile } from "node:child_process";
|
||||
import { mkdir, mkdtemp, rm, symlink, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { dirname, join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
import type { CommandManagedRuntimeRunner } from "@paperclipai/adapter-utils/command-managed-runtime";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { canonicalProviderPackJson, verifyRemoteProviderPack, type ProviderPackIdentity } from "./remote-provider-pack-verification.js";
|
||||
|
||||
const sha = (text: string) => `sha256:${createHash("sha256").update(text).digest("hex")}`;
|
||||
const manifest = (revision: string): ProviderPackIdentity => {
|
||||
const payload = {
|
||||
compatibility: { opencode: 1, acpx: 1 },
|
||||
pins: { opencode: "1.18.34", nodeMinimum: "24.11.0" },
|
||||
target: { platform: "linux", architecture: "x64" },
|
||||
runnerSourceRevision: revision.repeat(40),
|
||||
artifacts: Object.fromEntries(Object.entries({
|
||||
nodeCommand: "node_modules/node/bin/node", opencodeCommand: "node_modules/.bin/opencode",
|
||||
productionLock: "pnpm-lock.yaml", opencodeExecutable: "node_modules/opencode-ai/bin/opencode.exe",
|
||||
opencodeProxy: "dist/cli/opencode-app-server-proxy.cjs", acpxSidecar: "dist/cli/acpx-runtime-sidecar.cjs",
|
||||
grokLauncher: "dist/providers/grok/launcher.cjs",
|
||||
}).map(([name, path]) => [name, { path, sha256: sha(`${revision}-${name}`) }])) as ProviderPackIdentity["payload"]["artifacts"],
|
||||
};
|
||||
return { schema: "paperclip-runner/remote-provider-pack/v1", payload, digest: sha(canonicalProviderPackJson(payload)) };
|
||||
};
|
||||
|
||||
describe("installed provider pack compatibility", () => {
|
||||
it("reuses a protected pack across app revisions without executing verification commands", async () => {
|
||||
const installed = manifest("a");
|
||||
const current = structuredClone(installed);
|
||||
current.payload.runnerSourceRevision = "b".repeat(40);
|
||||
current.digest = sha(canonicalProviderPackJson(current.payload));
|
||||
const root = await mkdtemp(join(tmpdir(), "installed-pack-compatibility-"));
|
||||
await mkdir(dirname(join(root, installed.payload.artifacts.nodeCommand.path)), { recursive: true });
|
||||
await symlink(process.execPath, join(root, installed.payload.artifacts.nodeCommand.path));
|
||||
await writeFile(join(root, "provider-pack.json"), JSON.stringify(installed));
|
||||
const execute = vi.fn(async (request: Parameters<CommandManagedRuntimeRunner["execute"]>[0]) => {
|
||||
try {
|
||||
const result = await promisify(execFile)(request.command, request.args ?? [], { cwd: root, timeout: 2_000 });
|
||||
return { ...result, exitCode: 0, signal: null, timedOut: false, pid: null, startedAt: new Date().toISOString() };
|
||||
} catch (error) {
|
||||
const failed = error as Error & { code: number; stdout: string; stderr: string };
|
||||
if (typeof failed.code !== "number") throw error;
|
||||
return { stdout: failed.stdout, stderr: failed.stderr, exitCode: failed.code, signal: null, timedOut: false, pid: null, startedAt: new Date().toISOString() };
|
||||
}
|
||||
});
|
||||
const input = { runner: { execute }, cwd: root, packRoot: root, manifest: current };
|
||||
try {
|
||||
// Exercise the old complete-manifest comparison in a real Node process.
|
||||
// Only provenance changed; provider bytes and interface are identical.
|
||||
await expect(verifyRemoteProviderPack(input)).rejects.toThrow("manifest mismatch");
|
||||
expect(execute).toHaveBeenCalledOnce();
|
||||
execute.mockClear();
|
||||
const selected = await verifyRemoteProviderPack({ ...input,
|
||||
installation: { root, manifest: installed, immutable: true }, provider: { kind: "opencode" } });
|
||||
expect(selected).toEqual(installed);
|
||||
expect(execute).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
const reseal = (pack: ProviderPackIdentity) => ({ ...pack, digest: sha(canonicalProviderPackJson(pack.payload)) });
|
||||
const check = (installed: ProviderPackIdentity, options: { immutable?: boolean; requireExactIdentity?: boolean } = {}) => {
|
||||
const execute = vi.fn();
|
||||
return verifyRemoteProviderPack({ runner: { execute }, cwd: "/workspace", packRoot: "/opt/paperclip-runner/provider-pack",
|
||||
manifest: manifest("b"), installation: { root: "/opt/paperclip-runner/provider-pack", manifest: installed, immutable: options.immutable ?? true },
|
||||
provider: { kind: "opencode" }, requireExactIdentity: options.requireExactIdentity });
|
||||
};
|
||||
it.each(["interface", "platform", "schema", "artifact_path", "digest", "legacy"])("rejects an incompatible %s", async field => {
|
||||
let pack = manifest("a");
|
||||
if (field === "interface") pack.payload.compatibility!.opencode += 1;
|
||||
if (field === "legacy") delete pack.payload.compatibility;
|
||||
if (field === "platform") pack.payload.target.architecture = "arm64";
|
||||
if (field === "schema") pack.schema = "unknown";
|
||||
if (field === "artifact_path") pack.payload.artifacts.nodeCommand.path = "../node";
|
||||
pack = reseal(pack);
|
||||
if (field === "digest") pack.digest = sha("corrupt");
|
||||
await expect(check(pack)).rejects.toThrow("runner_remote_provider_artifact_incompatible");
|
||||
});
|
||||
it("keeps writable installations at the content-verification boundary", async () => {
|
||||
await expect(check(manifest("b"), { immutable: false })).rejects.toThrow("installation is incompatible");
|
||||
});
|
||||
it("does not replace an admitted pack during reattachment", async () => {
|
||||
await expect(check(manifest("a"), { requireExactIdentity: true })).rejects.toThrow("incompatible");
|
||||
await expect(check(manifest("b"), { requireExactIdentity: true })).resolves.toEqual(manifest("b"));
|
||||
});
|
||||
it("ignores unrelated provider and dependency changes for OpenCode", async () => {
|
||||
const pack = manifest("a");
|
||||
pack.payload.compatibility!.acpx += 1;
|
||||
pack.payload.pins.acpx = "different-version";
|
||||
pack.payload.acpxProfileDigests = { codex: sha("different-codex") };
|
||||
await expect(check(reseal(pack))).resolves.toEqual(reseal(pack));
|
||||
});
|
||||
it("checks only the selected ACPX profile", async () => {
|
||||
const current = manifest("b"), installed = manifest("a");
|
||||
current.payload.acpxProfileDigests = { claude: sha("claude"), codex: sha("codex") };
|
||||
installed.payload.acpxProfileDigests = { claude: sha("claude"), codex: sha("other-codex") };
|
||||
const input = { runner: { execute: vi.fn() }, cwd: "/workspace", packRoot: "/opt/paperclip-runner/provider-pack",
|
||||
manifest: reseal(current), installation: { root: "/opt/paperclip-runner/provider-pack", manifest: reseal(installed), immutable: true },
|
||||
provider: { kind: "acpx" as const, agent: "claude" } };
|
||||
await expect(verifyRemoteProviderPack(input)).resolves.toEqual(reseal(installed));
|
||||
await expect(verifyRemoteProviderPack({ ...input, provider: { kind: "acpx", agent: "codex" } })).rejects.toThrow("profile is incompatible");
|
||||
});
|
||||
|
||||
});
|
||||
@@ -1,3 +1,4 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { posix } from "node:path";
|
||||
import type { CommandManagedRuntimeRunner } from "@paperclipai/adapter-utils/command-managed-runtime";
|
||||
import { runRemoteProviderVerification } from "./remote-provider-pack.js";
|
||||
@@ -13,6 +14,10 @@ export type ProviderPackIdentity = {
|
||||
schema: string;
|
||||
digest: string;
|
||||
payload: {
|
||||
compatibility?: { opencode: number; acpx: number };
|
||||
acpxProfileDigests?: Record<string, string>;
|
||||
providers?: Record<string, unknown>;
|
||||
candidateProviders?: Record<string, unknown>;
|
||||
pins: { opencode: string; [key: string]: string | null };
|
||||
target: { platform: string; architecture: string };
|
||||
runnerSourceRevision: string;
|
||||
@@ -20,12 +25,66 @@ export type ProviderPackIdentity = {
|
||||
};
|
||||
};
|
||||
|
||||
export async function verifyRemoteProviderPack({ runner, cwd, packRoot, manifest }: {
|
||||
export type ProviderPackInstallation = { root: string; manifest: unknown; immutable: boolean };
|
||||
export type PackProvider = { kind: "opencode" } | { kind: "acpx"; agent: string };
|
||||
|
||||
/** Admission of a build-qualified, OS-protected installation. Source revisions,
|
||||
* dependency pins and unrelated provider changes are provenance, not an interface.
|
||||
* The metadata comes from the trusted runner, never from a provider tool result.
|
||||
*/
|
||||
function compatibleInstallation<T extends ProviderPackIdentity>(
|
||||
installation: ProviderPackInstallation, expected: T, packRoot: string,
|
||||
provider: PackProvider, requireExactIdentity: boolean,
|
||||
): T {
|
||||
const actual = installation.manifest as T | null;
|
||||
const payload = actual?.payload;
|
||||
const contract = expected.payload.compatibility?.[provider.kind];
|
||||
if (!installation.immutable || installation.root !== packRoot || !payload
|
||||
|| actual.schema !== expected.schema
|
||||
|| actual.digest !== `sha256:${createHash("sha256").update(canonicalProviderPackJson(payload)).digest("hex")}`
|
||||
|| canonicalProviderPackJson(payload.target) !== canonicalProviderPackJson(expected.payload.target)
|
||||
|| (requireExactIdentity ? actual.digest !== expected.digest
|
||||
: !Number.isSafeInteger(contract) || contract! < 1 || payload.compatibility?.[provider.kind] !== contract)
|
||||
|| typeof payload.pins?.opencode !== "string") {
|
||||
throw new Error("runner_remote_provider_artifact_incompatible: installed provider pack interface or installation is incompatible");
|
||||
}
|
||||
// Launch profiles and recovery receipts bind the selected installation's own
|
||||
// bytes, never the current controller's artifact hashes.
|
||||
for (const [name, expectedArtifact] of Object.entries(expected.payload.artifacts)) {
|
||||
const artifact = payload.artifacts?.[name];
|
||||
if (artifact?.path !== expectedArtifact.path || !/^sha256:[a-f0-9]{64}$/.test(artifact.sha256)) {
|
||||
throw new Error("runner_remote_provider_artifact_incompatible: installed provider pack artifact identity is invalid");
|
||||
}
|
||||
}
|
||||
if (provider.kind === "acpx") {
|
||||
const agent = provider.agent;
|
||||
const currentProfile = expected.payload.acpxProfileDigests?.[agent];
|
||||
const selected = expected.payload.providers?.[agent] ?? expected.payload.candidateProviders?.[agent];
|
||||
const installed = payload.providers?.[agent] ?? payload.candidateProviders?.[agent];
|
||||
// ACPX's existing qualified launch profiles remain authoritative for the
|
||||
// selected agent. Other providers in the same pack do not invalidate it.
|
||||
if (currentProfile ? payload.acpxProfileDigests?.[agent] !== currentProfile
|
||||
: !selected || !installed || ["version", "profileDigest", "closureDigest", "qualification", "path"].some(key =>
|
||||
(installed as Record<string, unknown>)[key] !== (selected as Record<string, unknown>)[key])) {
|
||||
throw new Error("runner_remote_provider_artifact_incompatible: selected ACPX provider profile is incompatible");
|
||||
}
|
||||
}
|
||||
return structuredClone(actual);
|
||||
}
|
||||
|
||||
export async function verifyRemoteProviderPack<T extends ProviderPackIdentity>({ runner, cwd, packRoot, manifest, installation, provider, requireExactIdentity = false }: {
|
||||
runner: CommandManagedRuntimeRunner;
|
||||
cwd: string;
|
||||
packRoot: string;
|
||||
manifest: ProviderPackIdentity;
|
||||
}): Promise<void> {
|
||||
manifest: T;
|
||||
installation?: ProviderPackInstallation;
|
||||
provider?: PackProvider;
|
||||
requireExactIdentity?: boolean;
|
||||
}): Promise<T> {
|
||||
if (installation) {
|
||||
if (!provider) throw new Error("runner_remote_provider_artifact_incompatible: provider interface is required");
|
||||
return compatibleInstallation(installation, manifest, packRoot, provider, requireExactIdentity);
|
||||
}
|
||||
const expected = Buffer.from(
|
||||
canonicalProviderPackJson(manifest),
|
||||
"utf8",
|
||||
@@ -82,4 +141,5 @@ export async function verifyRemoteProviderPack({ runner, cwd, packRoot, manifest
|
||||
"runner_remote_provider_artifact_incompatible: OpenCode version mismatch",
|
||||
);
|
||||
}
|
||||
return structuredClone(manifest);
|
||||
}
|
||||
+1
@@ -166,6 +166,7 @@ export const completeTerminatedLocalNativeSessionCleanup = runner.completeTermin
|
||||
export const probeAcpxClaudeInstallation = runner.probeAcpxClaudeInstallation;
|
||||
export const probeAcpxGrokInstallation = runner.probeAcpxGrokInstallation;
|
||||
export const probeAcpxPiInstallation = runner.probeAcpxPiInstallation;
|
||||
export const PROVIDER_PACK_COMPATIBILITY = runner.PROVIDER_PACK_COMPATIBILITY;
|
||||
export const bundledRemoteProviderPackManifestPath = runner.bundledRemoteProviderPackManifestPath;
|
||||
export const bundledRemoteRunnerBinary = runner.bundledRemoteRunnerBinary;
|
||||
|
||||
|
||||
Reference in new issue
Block a user