Files
PaperClipAI/server/src/services/native-runtime/remote-provider-pack-verification.test.ts
T
Nicky LeachandClaude Fable 5.2 2dcfdabc79 feat(runner): let a provider pack record the Runner image it was built in (#21228)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Runner is the part that executes agent work: a daemon
(`paperclip-runnerd`) and a provider pack of agent tools
> - A provider pack has a manifest, `provider-pack.json`. It records the
commit the pack was built from as `payload.runnerSourceRevision`
> - The next pull requests build the Runner alone as an image that has
an identity of its own: a digest of the Runner's inputs
> - A pack from that image must be able to say which image it came from.
The commit does not say that, because many commits have the same inputs
> - This pull request adds one optional field to the pack manifest for
that digest
> - The benefit is that a reader of a pack can find the exact image that
built it. Packs that are built today do not change

## Linked Issues or Issue Description

Refs #21204 (the Runner image, part 2 of this stack). No issue exists
for this change.

**Problem or motivation**

The provider pack manifest records a source revision. It has no place
for the identity of a Runner image. The Runner image in #21204 is named
by a digest of its inputs, and one image serves every commit that has
those inputs. Without a field in the manifest, a pack cannot name the
image that built it.

**Proposed solution**

Add the optional field `payload.runnerInputsDigest`. The pack builder
writes it only when the environment variable
`PAPERCLIP_RUNNER_INPUTS_DIGEST` is set. The value must be 64 lowercase
hexadecimal characters. The pack builder fails for another value.
Nothing sets the variable in this pull request.

**Alternatives considered**

- Put the digest in a file beside the pack. Then the pack digest does
not cover it, and a copy of the pack loses it.
- Reuse `runnerSourceRevision`. That field is a commit, and code
compares it with other commits.

**Roadmap alignment**

This is build infrastructure. It adds no product feature and does not
overlap an item in `ROADMAP.md`.

This is part 1 of 4. Each part is on top of the one before it:

1. `runner/pack-inputs-digest`: an optional field in the provider pack
manifest (this pull request)
2. `runner/runner-image-build` (#21204): build and check the Runner
image, with no publication
3. `runner/runner-image-layers`: layers by change rate, and the slim
variant
4. `runner/runner-image-publish`: publish both variants from `master`

## What Changed

- `packages/paperclip-runner/scripts/build-provider-pack.mjs`: reads
`PAPERCLIP_RUNNER_INPUTS_DIGEST`. When it is set, the script checks its
form and writes `payload.runnerInputsDigest` after
`runnerSourceRevision`. When it is not set, the manifest is the same as
before, byte for byte.
-
`server/src/services/native-runtime/remote-provider-pack-verification.ts`:
the manifest type has the optional field `runnerInputsDigest`. No check
reads it.
-
`server/src/services/native-runtime/remote-provider-pack-verification.test.ts`:
one test. A sealed manifest with the field passes verification. A
manifest whose field was changed after it was sealed fails, because the
pack digest covers the field.

The change is 20 added lines in 3 files.

## Verification

- `cd server && node node_modules/vitest/vitest.mjs run
src/services/native-runtime/remote-provider-pack-verification.test.ts`:
19 tests pass.
- Part 2 builds a pack with the variable set and a pack without it, on
`linux/arm64`. The first has the field and the second does not.

Never run:

- The full test suite and the full typecheck. CI must supply them.

## Risks

- Low risk. The field is optional and additive, and nothing sets the
variable yet.
- The pack digest covers the payload. A pack that has the field
therefore has another digest than the same pack without it. No pack has
the field until part 2 builds one.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`), Anthropic, used through Claude
Code with tool use (shell, file edits, local Docker builds).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 16:58:14 -08:00

186 lines
13 KiB
TypeScript

import { createHash } from "node:crypto";
import { execFile } from "node:child_process";
import { mkdir, mkdtemp, rm, symlink, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { dirname, join } from "node:path";
import { promisify } from "node:util";
import { readFileSync } from "node:fs";
import type { CommandManagedRuntimeRunner } from "@paperclipai/adapter-utils/command-managed-runtime";
import { describe, expect, it, vi } from "vitest";
import { canonicalProviderPackJson, verifyRemoteProviderPack, type ProviderPackIdentity } from "./remote-provider-pack-verification.js";
const sha = (text: string) => `sha256:${createHash("sha256").update(text).digest("hex")}`;
const manifest = (revision: string): ProviderPackIdentity => {
const payload = {
compatibility: { opencode: 1, acpx: 1 },
contractDigest: sha("contracts"),
pins: { opencode: "1.18.34", nodeMinimum: "24.11.0" },
target: { platform: "linux", architecture: "x64" },
runnerSourceRevision: revision.repeat(40),
artifacts: Object.fromEntries(Object.entries({
nodeCommand: "node_modules/node/bin/node", opencodeCommand: "node_modules/.bin/opencode",
productionLock: "pnpm-lock.yaml", opencodeExecutable: "node_modules/opencode-ai/bin/opencode.exe",
opencodeProxy: "dist/cli/opencode-app-server-proxy.cjs", acpxSidecar: "dist/cli/acpx-runtime-sidecar.cjs",
grokLauncher: "dist/providers/grok/launcher.cjs",
}).map(([name, path]) => [name, { path, sha256: sha(`${revision}-${name}`) }])) as ProviderPackIdentity["payload"]["artifacts"],
};
return { schema: "paperclip-runner/remote-provider-pack/v1", payload, digest: sha(canonicalProviderPackJson(payload)) };
};
describe("installed provider pack compatibility", () => {
it("reuses a protected pack across app revisions without executing verification commands", async () => {
const installed = manifest("a");
const current = structuredClone(installed);
current.payload.runnerSourceRevision = "b".repeat(40);
current.digest = sha(canonicalProviderPackJson(current.payload));
const root = await mkdtemp(join(tmpdir(), "installed-pack-compatibility-"));
await mkdir(dirname(join(root, installed.payload.artifacts.nodeCommand.path)), { recursive: true });
await symlink(process.execPath, join(root, installed.payload.artifacts.nodeCommand.path));
await writeFile(join(root, "provider-pack.json"), JSON.stringify(installed));
const execute = vi.fn(async (request: Parameters<CommandManagedRuntimeRunner["execute"]>[0]) => {
try {
const result = await promisify(execFile)(request.command, request.args ?? [], { cwd: root, timeout: 2_000 });
return { ...result, exitCode: 0, signal: null, timedOut: false, pid: null, startedAt: new Date().toISOString() };
} catch (error) {
const failed = error as Error & { code: number; stdout: string; stderr: string };
if (typeof failed.code !== "number") throw error;
return { stdout: failed.stdout, stderr: failed.stderr, exitCode: failed.code, signal: null, timedOut: false, pid: null, startedAt: new Date().toISOString() };
}
});
const input = { runner: { execute }, cwd: root, packRoot: root, manifest: current };
try {
// Exercise the old complete-manifest comparison in a real Node process.
// Only provenance changed; provider bytes and interface are identical.
await expect(verifyRemoteProviderPack(input)).rejects.toThrow("manifest mismatch");
expect(execute).toHaveBeenCalledOnce();
execute.mockClear();
const selected = await verifyRemoteProviderPack({ ...input,
installation: { root, manifest: installed, immutable: true }, provider: { kind: "opencode" } });
expect(selected).toEqual(installed);
expect(execute).not.toHaveBeenCalled();
} finally {
await rm(root, { recursive: true, force: true });
}
});
const reseal = (pack: ProviderPackIdentity) => ({ ...pack, digest: sha(canonicalProviderPackJson(pack.payload)) });
const check = (installed: ProviderPackIdentity, options: { immutable?: boolean; requireExactIdentity?: boolean; requireCurrentOpenCodeProxy?: boolean } = {}) => {
const execute = vi.fn();
return verifyRemoteProviderPack({ runner: { execute }, cwd: "/workspace", packRoot: "/opt/paperclip-runner/provider-pack",
manifest: manifest("b"), installation: { root: "/opt/paperclip-runner/provider-pack", manifest: installed, immutable: options.immutable ?? true },
provider: { kind: "opencode" }, requireExactIdentity: options.requireExactIdentity,
requireCurrentOpenCodeProxy: options.requireCurrentOpenCodeProxy });
};
it.each(["interface", "contracts", "platform", "schema", "artifact_path", "digest", "legacy", "legacy_contracts"])("rejects an incompatible %s", async field => {
let pack = manifest("a");
if (field === "interface") pack.payload.compatibility!.opencode += 1;
if (field === "contracts") pack.payload.contractDigest = sha("other-contracts");
if (field === "legacy") delete pack.payload.compatibility;
if (field === "legacy_contracts") delete pack.payload.contractDigest;
if (field === "platform") pack.payload.target.architecture = "arm64";
if (field === "schema") pack.schema = "unknown";
if (field === "artifact_path") pack.payload.artifacts.nodeCommand.path = "../node";
pack = reseal(pack);
if (field === "digest") pack.digest = sha("corrupt");
await expect(check(pack)).rejects.toThrow("runner_remote_provider_artifact_incompatible");
});
it("does not reuse image packs when the controller's bundled pack records no contract digest", async () => {
const current = manifest("b");
delete current.payload.contractDigest;
const installed = manifest("a");
await expect(verifyRemoteProviderPack({ runner: { execute: vi.fn() }, cwd: "/workspace", packRoot: "/opt/paperclip-runner/provider-pack",
manifest: reseal(current), installation: { root: "/opt/paperclip-runner/provider-pack", manifest: installed, immutable: true },
provider: { kind: "opencode" } })).rejects.toThrow("interface or installation is incompatible");
});
it("keeps writable installations at the content-verification boundary", async () => {
await expect(check(manifest("b"), { immutable: false })).rejects.toThrow("installation is incompatible");
});
it("does not replace an admitted pack during reattachment", async () => {
await expect(check(manifest("a"), { requireExactIdentity: true })).rejects.toThrow("incompatible");
await expect(check(manifest("b"), { requireExactIdentity: true })).resolves.toEqual(manifest("b"));
});
it("requires the current proxy for requested OpenCode diagnostics", async () => {
const installed = manifest("a");
await expect(check(installed)).resolves.toEqual(installed);
await expect(check(installed, { requireCurrentOpenCodeProxy: true }))
.rejects.toThrow("requested OpenCode diagnostics");
installed.payload.artifacts.opencodeProxy = manifest("b").payload.artifacts.opencodeProxy;
await expect(check(reseal(installed), { requireCurrentOpenCodeProxy: true }))
.resolves.toEqual(reseal(installed));
});
it("ignores unrelated provider and dependency changes for OpenCode", async () => {
const pack = manifest("a");
pack.payload.compatibility!.acpx += 1;
pack.payload.pins.acpx = "different-version";
pack.payload.acpxProfileDigests = { codex: sha("different-codex") };
await expect(check(reseal(pack))).resolves.toEqual(reseal(pack));
});
it("treats the Runner image inputs digest as provenance", async () => {
// A pack built in the Runner image records which image it came from. The
// controller's own pack may record another image, or none.
const pack = manifest("a");
pack.payload.runnerInputsDigest = "c".repeat(64);
await expect(check(reseal(pack))).resolves.toEqual(reseal(pack));
// The field is part of the sealed payload, so it cannot change unnoticed.
await expect(check({ ...reseal(pack), payload: { ...pack.payload, runnerInputsDigest: "d".repeat(64) } }))
.rejects.toThrow("runner_remote_provider_artifact_incompatible");
});
it("checks only the selected ACPX profile", async () => {
const current = manifest("b"), installed = manifest("a");
current.payload.acpxProfileDigests = { claude: sha("claude"), codex: sha("codex") };
installed.payload.acpxProfileDigests = { claude: sha("claude"), codex: sha("other-codex") };
const input = { runner: { execute: vi.fn() }, cwd: "/workspace", packRoot: "/opt/paperclip-runner/provider-pack",
manifest: reseal(current), installation: { root: "/opt/paperclip-runner/provider-pack", manifest: reseal(installed), immutable: true },
provider: { kind: "acpx" as const, agent: "claude" } };
await expect(verifyRemoteProviderPack(input)).resolves.toEqual(reseal(installed));
await expect(verifyRemoteProviderPack({ ...input, provider: { kind: "acpx", agent: "codex" } })).rejects.toThrow("profile is incompatible");
});
it("rejects the published Grok pack from before native effort admission", async () => {
const profiles = JSON.parse(readFileSync(new URL("../../../../packages/paperclip-runner/acpx-profiles.json", import.meta.url), "utf8"));
const current = manifest("b"), installed = manifest("a");
// Published builtin Grok profile 1 predates grokEffort and grokDisableWebSearch
// in session.open. Its launcher bytes equal today's, so its release digest
// matches; the pack predates contract digests, so that gate refuses it.
const published = "sha256:f0b698395a3704ed2ffaf84ea19bdb20c36c8a0a70b7c629c7b6ffe144e59e55";
expect(profiles.profiles.grok.commandDigest).toBe(published);
installed.payload.acpxProfileDigests = { grok: published };
delete installed.payload.contractDigest;
current.payload.acpxProfileDigests = { grok: profiles.profiles.grok.commandDigest };
const execute = vi.fn();
await expect(verifyRemoteProviderPack({ runner: { execute }, cwd: "/workspace", packRoot: "/opt/paperclip-runner/provider-pack",
manifest: reseal(current), installation: { root: "/opt/paperclip-runner/provider-pack", manifest: reseal(installed), immutable: true },
provider: { kind: "acpx", agent: "grok" } })).rejects.toThrow("interface or installation is incompatible");
expect(execute).not.toHaveBeenCalled();
});
it("rejects an installed pack with another Grok release digest", async () => {
const profiles = JSON.parse(readFileSync(new URL("../../../../packages/paperclip-runner/acpx-profiles.json", import.meta.url), "utf8"));
const current = manifest("b"), installed = manifest("a");
// The last source-bound Grok profile digest.
installed.payload.acpxProfileDigests = { grok: "sha256:5205c2197b94cfbd3e62fbeea027f2e80d092f736ca52656dda2f7044ba9a5ca" };
current.payload.acpxProfileDigests = { grok: profiles.profiles.grok.commandDigest };
const execute = vi.fn();
await expect(verifyRemoteProviderPack({ runner: { execute }, cwd: "/workspace", packRoot: "/opt/paperclip-runner/provider-pack",
manifest: reseal(current), installation: { root: "/opt/paperclip-runner/provider-pack", manifest: reseal(installed), immutable: true },
provider: { kind: "acpx", agent: "grok" } })).rejects.toThrow("selected ACPX provider profile is incompatible");
expect(execute).not.toHaveBeenCalled();
});
it("reuses the current Grok profile across source-only changes and retains exact historical recovery authority", async () => {
const profiles = JSON.parse(readFileSync(new URL("../../../../packages/paperclip-runner/acpx-profiles.json", import.meta.url), "utf8"));
const current = manifest("b"), installed = manifest("a");
for (const pack of [current, installed]) pack.payload.acpxProfileDigests = { grok: profiles.profiles.grok.commandDigest };
const execute = vi.fn();
const input = { runner: { execute }, cwd: "/workspace", packRoot: "/opt/paperclip-runner/provider-pack",
manifest: reseal(current), installation: { root: "/opt/paperclip-runner/provider-pack", manifest: reseal(installed), immutable: true },
provider: { kind: "acpx" as const, agent: "grok" } };
await expect(verifyRemoteProviderPack(input)).resolves.toEqual(reseal(installed));
const historical = manifest("a");
historical.payload.acpxProfileDigests = { grok: "sha256:5205c2197b94cfbd3e62fbeea027f2e80d092f736ca52656dda2f7044ba9a5ca" };
await expect(verifyRemoteProviderPack({ ...input, manifest: reseal(historical),
installation: { ...input.installation, manifest: reseal(historical) }, requireExactIdentity: true })).resolves.toEqual(reseal(historical));
expect(execute).not.toHaveBeenCalled();
});
});