perf(runner): reuse compatible protected provider packs (#17749)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native Runner starts provider runtimes in remote execution
environments.
> - Sandbox images already contain qualified provider packs.
> - Startup required the image pack to match the entire current app
release manifest.
> - An unrelated app commit could therefore cause pack verification,
transfer, and installation again.
> - This pull request selects protected packs by their provider
interface and platform.
> - The benefit is no extra pack probe, directory scan, version command,
link, or upload on the compatible image path.

## Linked Issues or Issue Description

**What happened?**

Starting a remote task can reject a compatible image pack because its
source revision or unrelated provider differs from the controller. Pack
replacement added about 16 seconds in the investigated startup. Startup
also repeated full content checks.

**Expected behavior**

Use an already qualified, protected pack when its selected provider
interface and target platform are compatible. Keep the selected artifact
identity for process launch and recovery.

**Steps to reproduce**

1. Install a qualified provider pack in a sandbox image.
2. Start a controller built from another app commit.
3. Start a remote OpenCode task. The old complete-manifest comparison
rejects the image pack even when the provider interface still works.

Related repairs: #16493 reduced transfer cost. #17512 retained exact
artifact authority during reattachment. This change preserves that
recovery rule and changes fresh-launch pack selection.

## What Changed

- Add explicit OpenCode and ACPX interface revisions to produced pack
manifests. App Git SHAs and dependency pins remain provenance.
- Read bounded protected-installation metadata in the existing final
runner metadata query. Accept only root-owned paths that the execution
user cannot write, including ancestors and manifest. Reject writable
aliases.
- Select and prepare the pack in server-owned environment setup before
constructing the existing transport. Bind provider launch paths and
digests from the selected installation. Save that identity for exact
reattachment.
- Keep full content verification for uploads, writable installations,
and legacy packs. Reuse successful verification within one startup. Keep
older attached runners compatible with the optional metadata request.
- Seal image packs and document the one-time image refresh that enables
reuse across later compatible releases. Keep selected ACPX agent
qualification checks.

## Verification

- Red/green regression: the old code rejects a protected pack from
another app revision with `manifest mismatch`; the fixed path selects it
without executing verification commands.
- Native startup regression: select the installed pack's own Node digest
with one existing runner metadata request and no pack verification or
version commands.
- Targeted pack, artifact, recovery, execution-driver, and Rust metadata
tests pass. Incompatible interfaces/platforms, malformed identities,
writable roots, path aliases, and exact reattachment are covered.
- Final revision `91dc1676ff67bdbe5866c8586daac10240530cca`: workspace
typecheck and build pass. All 53 current-head checks are successful,
with no pending or failed checks. Greptile is 5/5 with no open findings.
- The broad local `pnpm test:run` attempt reported a workspace-runtime
conflict test failure; its isolated rerun passed. The broad run was
stopped after that failure. Full CI now passes, including the server
test shards. This is not a claim that the full local run passed.
- Linux filesystem evidence: the production metadata functions read a
real root-owned preinstalled pack as a non-root user in 16 microseconds
median and 20 microseconds p95 across 100 warm reads. This measures
metadata admission, not model or hosted response latency.
- All 910 affected server tests and 57 Copilot attestation tests pass
locally on the final implementation. Full CI covers the unchanged Runner
transport. No provider qualification hash was refreshed.
- Built the actual Linux daemon at
`416e7ec7425d0d3565b4a5e1808002d650a60414` and ran its metadata command
as UID 65534. The protected installation returned its real manifest with
`immutable: true`; writable alias and missing-pack cases returned
`immutable: false`. This is metadata/permission evidence, not provider
execution.
- Hosted rollout and a hosted end-to-end latency measurement have not
run.

## Risks

- Existing images need one rebuild to publish the compatibility
declaration. Packs without it keep the legacy content-verification path.
- Fast admission relies on image publication producing a qualified
protected tree. Root execution and task-writable installations retain
full verification.
- Bump an interface revision when required runner/pack behavior changes
incompatibly. Deploy a new image to roll out compatible runtime fixes or
upgrades.
- This removes provider-pack preparation delay. Other runner startup,
context, model, and finalization costs remain separate.

## Model Used

OpenAI Codex, configured model `gpt-6.1-sol`, reasoning effort `xhigh`,
with tool use and code execution. The context-window size is not exposed
in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-10-26 14:43:12 -05:00
1 parent f1922d95e1
commit 9ae4c25e7e
20 files changed
+570 -72

No files matched your search

@@ -1440,6 +1440,7 @@ describe("remote provider pack manifest", () => {
opencodeProxy: "dist/cli/opencode-app-server-proxy.cjs", acpxSidecar: "dist/cli/acpx-runtime-sidecar.cjs",
}).map(([key, path]) => [key, { path, sha256: digest }])) as Record<"nodeCommand" | "opencodeCommand", { path: string; sha256: string }>;
const payload = {
compatibility: { opencode: 1, acpx: 1 },
target: { platform: "linux", architecture }, runnerSourceRevision: "a".repeat(40),
pins: { nodeMinimum: "24.11.0", codex: QUALIFIED_ACPX_PROFILES.codex.agentRuntimeVersion,
opencode: "1.18.34", acpx: "0.13.1", grok: QUALIFIED_ACPX_PROFILES.grok.agentRuntimeVersion,
@@ -1449,6 +1450,11 @@ describe("remote provider pack manifest", () => {
distDigest: digest, artifacts,
bridgeDigest: "sha256:" + createHash("sha256").update(digest + "\n" + digest + "\n" + digest).digest("hex"),
};
if (provider === "cursor") Object.assign(payload, { providers: { cursor: {
version: QUALIFIED_ACPX_PROFILES.cursor.agentServerVersion, profileDigest: QUALIFIED_ACPX_PROFILES.cursor.commandDigest,
closureDigest: `sha256:${CURSOR_DISTRIBUTION_PINS["linux-x64"].closureSha256}`, qualification: "qualified",
path: "provider-assets/cursor/linux-x64", sha256: digest,
} } });
const manifest = { schema: "paperclip-runner/remote-provider-pack/v1", payload,
digest: "sha256:" + createHash("sha256").update(canonical(payload)).digest("hex") };
state.bundledPackManifestPath = join(root, "provider-pack.json");
@@ -1463,11 +1469,22 @@ describe("remote provider pack manifest", () => {
workspace: { ...execution.workspace, cwd: root },
session: { ...execution.session, normalizedSessionId: `bundled-${provider}-${randomUUID()}`, driverKind: provider === "opencode" ? "opencode_server" : "acpx_runtime" },
} as unknown as NativeExecutionInputV1;
const syncIn = vi.fn(async () => undefined);
const execute = vi.fn(async (command: { command: string; args?: string[] }) => {
if (command.args?.[0] === "--build-metadata") throw new Error("reached-staged-Linux-daemon-verification");
return { exitCode: 0, signal: null, timedOut: false, stdout: "", stderr: "" };
});
const syncIn = vi.fn(async (_operations: { files: { kind: string }[] }[]) => undefined);
const installed = structuredClone(manifest);
if (provider === "opencode") {
installed.payload.runnerSourceRevision = "c".repeat(40);
installed.payload.artifacts.nodeCommand.sha256 = "sha256:" + "d".repeat(64);
installed.digest = "sha256:" + createHash("sha256").update(canonical(installed.payload)).digest("hex");
}
const execute = vi.fn(async (command: { command: string; args?: string[] }) => ({
exitCode: 0, signal: null, timedOut: false, stderr: "",
stdout: command.args?.[0] === "--build-metadata" ? JSON.stringify({
schema: "paperclip-runner/runnerd-build-metadata/v1", binaryName: "paperclip-runnerd",
packageName: "@paperclipai/paperclip-runner", binaryContractVersion: 2,
durableSessionCapabilities: ["unlimited_runtime", "connection_lease_renewal"], prpTransportModes: ["listen_ws"],
providerPacks: [{ root: "/opt/paperclip-runner/provider-pack", manifest: installed, immutable: true }],
}) : "",
}));
try {
await createRunnerdBackend({ db: leaseDb(remoteExecution), execution: remoteExecution,
runnerInstanceId: "bundled-image-runner", runnerIngressAuthorized: true,
@@ -1491,12 +1508,17 @@ describe("remote provider pack manifest", () => {
const { bundledRemoteProviderPackManifestPath } = await import("../../vendor/paperclip-runner/index.js");
expect(bundledRemoteProviderPackManifestPath).toHaveBeenCalledWith(undefined, `linux-${architecture}`);
}
await expect(transport.controlPlaneRegistration({})).rejects.toThrow("reached-staged-Linux-daemon-verification");
expect(syncIn).toHaveBeenCalledWith([expect.objectContaining({ files: [expect.objectContaining({
sourcePath: state.bundledRunnerBinary, targetPath: "/workspace/.paperclip-runtime/paperclip-runner/bin/paperclip-runnerd",
})] })]);
expect(execute.mock.calls.some(([call]) => call.args?.[1] === "uname -s; uname -m")).toBe(false);
expect(existsSync(join(transport.stateDirectory!, "provider-pack-authority.json"))).toBe(false);
expect(transport).toMatchObject({ providerPackAuthorityDigest: installed.digest,
providerNodeCommand: "/opt/paperclip-runner/provider-pack/node_modules/node/bin/node",
providerNodeCommandSha256: installed.payload.artifacts.nodeCommand.sha256 });
expect(execute.mock.calls.filter(([call]) => call.args?.[0] === "--build-metadata")).toHaveLength(1);
expect(execute.mock.calls.some(([call]) => call.args?.[0] === "-e" || call.args?.[0] === "--version")).toBe(false);
expect(syncIn.mock.calls.every(([operations]) => operations.every(operation => operation.files.every(file => file.kind === "file")))).toBe(true);
if (provider === "claude-x64") {
const identity = transport.prpIdentity!;
const artifact = { version: "0.0.0", digest: "sha256:" + createHash("sha256").update(daemon).digest("hex") };
@@ -1514,24 +1536,30 @@ describe("remote provider pack manifest", () => {
await writeFile(state.bundledPackManifestPath!, JSON.stringify(next));
await writeFile(state.bundledRunnerBinary!, Buffer.concat([daemon, Buffer.from("new release")]));
const recoveryExecute = vi.fn(async (request: { args: string[] }) => {
if (request.args[0] === "--build-metadata") return { exitCode: 0, timedOut: false, stderr: "", stdout: JSON.stringify({
schema: "paperclip-runner/runnerd-build-metadata/v1", binaryName: "paperclip-runnerd",
packageName: "@paperclipai/paperclip-runner", binaryContractVersion: 2,
durableSessionCapabilities: ["unlimited_runtime", "connection_lease_renewal"], prpTransportModes: ["listen_ws"],
providerPacks: [{ root: "/opt/paperclip-runner/provider-pack", manifest, immutable: true }],
}) };
if (request.args[0] === "-e" || request.args[0] === "--version") return { exitCode: 0, timedOut: false, stderr: "", stdout: request.args[0] === "--version" ? "1.18.34" : "" };
const script = request.args[1];
const stdout = script.includes("base64") ? Buffer.from(JSON.stringify({ ...identity, lifecycle: "running" })).toString("base64")
: script.includes("cat --") ? `nonce\n123\n2026-09-19T10:00:00.000Z\n${identity.runnerInstanceId}\nlinux:abcd-1234:100\n` : "";
return { stdout, stderr: "", exitCode: 0, timedOut: false };
});
await createRunnerdBackend({ db: leaseDb(remoteExecution), execution: remoteExecution,
runnerInstanceId: identity.runnerInstanceId,
runnerInstanceId: identity.runnerInstanceId, runnerIngressAuthorized: true,
runnerExecutionTarget: { kind: "remote", transport: "sandbox", remoteCwd: "/workspace",
environmentId: "environment", leaseId: "lease", providerKey: "daytona", runner: { execute: recoveryExecute },
environmentId: "environment", leaseId: "lease", providerKey: "daytona", effectiveCapabilities: { runnerWebSocketIngress: true }, runner: { execute: recoveryExecute },
sandboxLeaseAcquisition: { outcome: "resumed", providerLeaseId: "sandbox" } } as never,
restartRecovery: { kind: "reattach_remote_runner", runId: remoteExecution.binding.runId,
leaseOwner: "controller", controllerGeneration: 2, providerAttempt: 1, restartKind: "graceful", recoveryRequestId: null,
remote: { providerLeaseId: "sandbox", remoteCwd: "/workspace" } },
});
state.createBackend.mock.calls.at(-1)![1].codexTransportFactory!();
expect(state.createTransport.mock.calls.at(-1)![0]).toMatchObject({
providerPackAuthorityDigest: manifest.digest, runnerArtifact: artifact, adoptExistingRunner: { pid: 123 },
});
const recovered = state.createTransport.mock.calls.at(-1)![0];
expect(recovered).toMatchObject({ providerPackAuthorityDigest: manifest.digest, runnerArtifact: artifact, adoptExistingRunner: { pid: 123 } });
}
} finally {
state.bundledPackManifestPath = null; state.bundledRunnerBinary = null;