mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-03 01:27:09 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Remote native runs use a provider pack. The controller reuses a protected pack from the sandbox image when the pack is compatible. > - #17749 gates this reuse on a hand-maintained interface revision in `provider-pack-contract.json`. > - Later changes altered the runner/pack boundary (#17922, #18188), but nobody bumped the revision. A stale Claude or Codex image pack can still be reused across a sidecar contract change. This is the failure class of #17972. > - A manual revision has a "forgot to bump" failure mode. Source-byte hashes change on every refactor, so they are too brittle. > - This pull request declares the sidecar `session.open` contract in its JSON schema and computes a contract digest from the JSON contracts and Rust shapes. > - Image-pack reuse now requires an equal digest. > - The benefit is that an incidental change keeps image packs reusable, and a real contract change retires them automatically. ## Linked Issues or Issue Description Refs #17749, #17972, #17922, #18188. **What happened?** The controller reused an image provider pack when `compatibility.acpx` and the selected agent's profile digest matched. For Claude and Codex, that profile digest binds only the upstream executable. The ACPX interface revision stayed at 1 after later sidecar contract changes. A pack built before those changes passed the reuse check. **Expected behavior** The controller reuses an image pack only when the pack uses the same runner/pack contracts as the controller's bundled pack. Comments, formatting and refactors do not change that result. **Steps to reproduce** 1. Build an image pack from a commit before a sidecar input change, such as a new `session.open` field. 2. Run a controller from a later commit that sends the new field, with the same interface revision and Claude version. 3. Before this fix, `verifyRemoteProviderPack` accepts the old image pack. ## What Changed - Declare the sidecar `session.open` parameter set as a closed `sessionOpenParams` definition in `acpx-sidecar.schema.json`. - `generate-acpx-sidecar-contract.mjs` now generates `GENERATED_ACPX_SIDECAR_OPEN_PARAMS` from it. - `parseOpenParams` uses the generated list instead of an inline `Set`. The accepted set is unchanged. - This is the surface of #17972. The schema is now its source of truth. - Declare the nested `session.open` contracts that the sidecar parses strictly: - `protocol/provider-schemas/runtime-context.schema.json` closes every runtime context object. `parseNativeRuntimeContext` uses the field lists that `scripts/generate-runtime-context-contract.mjs` generates from it. - The sidecar schema declares the closed `providerPolicy` object, and `parseProviderPolicy` uses its generated field list. - The accepted fields do not change. - Add `packages/paperclip-runner/scripts/contracts/contract-fingerprints.mjs`. It fingerprints these contracts: - the PRP and sidecar JSON schemas, with annotation text removed - the generated catalogs in `generated/` - the Rust serde shapes, `json!` payload keys, match literals and env keys in `runner-core/src`, without test-only modules - the reserved env key lists in `configured_environment.rs` - Read only the trees that every pack build context contains. Fail when a registered source is missing, so two build contexts cannot silently produce different digests. - Use only Node built-ins. The registry adds no dependency. - Write `contractDigest` as a top-level payload field in `build-provider-pack.mjs`. The manifest digest algorithm and the existing field layout do not change. - Require an equal `contractDigest` for image-pack reuse in `remote-provider-pack-verification.ts`. A pack without a digest is not reused. The controller uploads its bundled pack, as it does today for any incompatible pack. - Re-attest the source-hashed profiles, because this PR changes their bound sidecar sources: - Grok becomes profile 15 and now also binds the generated allowlist, so a Grok field change in the schema still invalidates the Grok identity. - Copilot becomes profile 88, and also binds the generated runtime context fields. - Earlier profiles stay decodable. - Add the registry script and `generated/` to the Daytona image content inputs. - Document the digest and the schema-first rule in `docker/daytona-runner/README.md` and `doc/DEVELOPING.md`. ## Verification - `node --test scripts/contracts/contract-fingerprints.test.mjs` (in `packages/paperclip-runner`): 4 tests pass. - These changes keep the digest: - schema description text - schema reformatting - renaming `parseOpenParams` - Rust comments, `#[serde(skip)]` fields and `#[cfg(test)]` modules - edits to the `#[path]` test module `tests/provider_api_validation.rs` - These changes alter the digest: - a new `session.open` schema field - a changed constraint on the `remainingWork[].description` property - a new reserved env key - a new sidecar env key - a new checkpoint field - A build context without a registered source fails. - `node --test test/acpx-sidecar-contract.test.mjs`: 13 tests pass. A new case rejects an unknown `session.open` parameter and accepts each declared parameter. - `node --test scripts/grok-profile-identity.test.mjs`: 4 tests pass. A change to the Grok field in the generated allowlist invalidates the Grok identity. - `generate-acpx-profiles.mjs --check`, `generate-acpx-sidecar-contract.mjs --check` and `generate-protocol-manifest.mjs --check` pass. - Runner vitest suites pass: 470 passed, 1 skipped. They cover the sidecar, sidecar input and protocol, Copilot and Pi installation, profile compatibility, native execution, recovery identity, the backend factory and the environment. - Server tests: pack verification, executor and artifact preparation pass (1,004 tests). - `tsc --noEmit` passes for `packages/paperclip-runner` and `server`. - `tests/runner-e2e/daytona-image.test.ts`: 10 of 11 tests pass. The failing test also fails on `master`. It expects a literal `profile-v19-identity.json` line in `.dockerignore`, which uses a `profile-v*` glob. - Not run locally: - A complete `build-provider-pack.mjs` run. The builder rejects the local Homebrew Node because it is not portable after relocation. - Rust tests and `acpx-controller-recovery.test.ts`, because `cargo` is not installed. - The full `pnpm -r typecheck`, `pnpm test:run` and `pnpm build`. ## Risks - Only JSON contracts and Rust shapes are part of the digest. The remaining TypeScript-only definitions are the `paperclip_finish` and `paperclip_block` tool schemas in `completion-result.ts` and the TypeScript env key lists. They are not covered until they move into a schema. The nested `expectedIdentity` and `tools` values ignore unknown fields, so a change there cannot cause a startup rejection. - A change there can let a stale image pack be reused. The sidecar then rejects the input with its strict parser error. - Today, Claude and Codex image packs have no contract check at all. - The digest covers all registered contracts for the whole pack. A contract change for one provider kind also retires image packs for the other. Each replacement costs about 16 s per remote start until the image is refreshed. - Existing image packs have no digest. After deploy, the controller uploads its bundled pack until the image is rebuilt. - The Grok and Copilot re-attestations start fresh sessions for those agents once after deploy, as each profile bump does today. - The Daytona image content ID formula changes, so the next image build does not reuse a cached image. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used Anthropic Claude Opus 5.5 (`claude-opus-5-5`) through Claude Code in the desktop app, with repository tools, shell execution and research subagents. The session does not expose the context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
157 lines
9.7 KiB
TypeScript
157 lines
9.7 KiB
TypeScript
import { createHash } from "node:crypto";
|
|
import { posix } from "node:path";
|
|
import type { CommandManagedRuntimeRunner } from "@paperclipai/adapter-utils/command-managed-runtime";
|
|
import { runRemoteProviderVerification } from "./remote-provider-pack.js";
|
|
|
|
export function canonicalProviderPackJson(value: unknown): string {
|
|
if (Array.isArray(value)) return `[${value.map(canonicalProviderPackJson).join(",")}]`;
|
|
if (value && typeof value === "object") return `{${Object.entries(value).sort(([a], [b]) => a.localeCompare(b))
|
|
.map(([key, entry]) => `${JSON.stringify(key)}:${canonicalProviderPackJson(entry)}`).join(",")}}`;
|
|
return JSON.stringify(value) ?? "null";
|
|
}
|
|
|
|
export type ProviderPackIdentity = {
|
|
schema: string;
|
|
digest: string;
|
|
payload: {
|
|
compatibility?: { opencode: number; acpx: number };
|
|
contractDigest?: string;
|
|
acpxProfileDigests?: Record<string, string>;
|
|
providers?: Record<string, unknown>;
|
|
candidateProviders?: Record<string, unknown>;
|
|
pins: { opencode: string; [key: string]: string | null };
|
|
target: { platform: string; architecture: string };
|
|
runnerSourceRevision: string;
|
|
artifacts: { nodeCommand: { path: string; sha256: string }; opencodeCommand: { path: string; sha256: string }; [key: string]: { path: string; sha256: string } };
|
|
};
|
|
};
|
|
|
|
export type ProviderPackInstallation = { root: string; manifest: unknown; immutable: boolean };
|
|
export type PackProvider = { kind: "opencode" } | { kind: "acpx"; agent: string };
|
|
|
|
/** Admission of a build-qualified, OS-protected installation. Source revisions,
|
|
* dependency pins and unrelated provider changes are provenance, not an interface.
|
|
* The interface is the hand-maintained compatibility revision plus the pack's
|
|
* computed contract digest; a pack without a digest predates that check and
|
|
* is replaced rather than reused.
|
|
* The metadata comes from the trusted runner, never from a provider tool result.
|
|
*/
|
|
function compatibleInstallation<T extends ProviderPackIdentity>(
|
|
installation: ProviderPackInstallation, expected: T, packRoot: string,
|
|
provider: PackProvider, requireExactIdentity: boolean, requireCurrentOpenCodeProxy: boolean,
|
|
): T {
|
|
const actual = installation.manifest as T | null;
|
|
const payload = actual?.payload;
|
|
const contract = expected.payload.compatibility?.[provider.kind];
|
|
if (!installation.immutable || installation.root !== packRoot || !payload
|
|
|| actual.schema !== expected.schema
|
|
|| actual.digest !== `sha256:${createHash("sha256").update(canonicalProviderPackJson(payload)).digest("hex")}`
|
|
|| canonicalProviderPackJson(payload.target) !== canonicalProviderPackJson(expected.payload.target)
|
|
|| (requireExactIdentity ? actual.digest !== expected.digest
|
|
: !Number.isSafeInteger(contract) || contract! < 1 || payload.compatibility?.[provider.kind] !== contract
|
|
|| !/^sha256:[a-f0-9]{64}$/.test(expected.payload.contractDigest ?? "")
|
|
|| payload.contractDigest !== expected.payload.contractDigest)
|
|
|| typeof payload.pins?.opencode !== "string") {
|
|
throw new Error("runner_remote_provider_artifact_incompatible: installed provider pack interface or installation is incompatible");
|
|
}
|
|
// Launch profiles and recovery receipts bind the selected installation's own
|
|
// bytes, never the current controller's artifact hashes.
|
|
for (const [name, expectedArtifact] of Object.entries(expected.payload.artifacts)) {
|
|
const artifact = payload.artifacts?.[name];
|
|
if (artifact?.path !== expectedArtifact.path || !/^sha256:[a-f0-9]{64}$/.test(artifact.sha256)) {
|
|
throw new Error("runner_remote_provider_artifact_incompatible: installed provider pack artifact identity is invalid");
|
|
}
|
|
}
|
|
if (provider.kind === "opencode" && requireCurrentOpenCodeProxy
|
|
&& payload.artifacts.opencodeProxy?.sha256 !== expected.payload.artifacts.opencodeProxy?.sha256) {
|
|
throw new Error("runner_remote_provider_artifact_incompatible: installed proxy does not match the requested OpenCode diagnostics");
|
|
}
|
|
if (provider.kind === "acpx") {
|
|
const agent = provider.agent;
|
|
const currentProfile = expected.payload.acpxProfileDigests?.[agent];
|
|
const selected = expected.payload.providers?.[agent] ?? expected.payload.candidateProviders?.[agent];
|
|
const installed = payload.providers?.[agent] ?? payload.candidateProviders?.[agent];
|
|
// ACPX's existing qualified launch profiles remain authoritative for the
|
|
// selected agent. Other providers in the same pack do not invalidate it.
|
|
if (currentProfile ? payload.acpxProfileDigests?.[agent] !== currentProfile
|
|
: !selected || !installed || ["version", "profileDigest", "closureDigest", "qualification", "path"].some(key =>
|
|
(installed as Record<string, unknown>)[key] !== (selected as Record<string, unknown>)[key])) {
|
|
throw new Error("runner_remote_provider_artifact_incompatible: selected ACPX provider profile is incompatible");
|
|
}
|
|
}
|
|
return structuredClone(actual);
|
|
}
|
|
|
|
export async function verifyRemoteProviderPack<T extends ProviderPackIdentity>({ runner, cwd, packRoot, manifest, installation, provider, requireExactIdentity = false, requireCurrentOpenCodeProxy = false }: {
|
|
runner: CommandManagedRuntimeRunner;
|
|
cwd: string;
|
|
packRoot: string;
|
|
manifest: T;
|
|
installation?: ProviderPackInstallation;
|
|
provider?: PackProvider;
|
|
requireExactIdentity?: boolean;
|
|
requireCurrentOpenCodeProxy?: boolean;
|
|
}): Promise<T> {
|
|
if (installation) {
|
|
if (!provider) throw new Error("runner_remote_provider_artifact_incompatible: provider interface is required");
|
|
return compatibleInstallation(installation, manifest, packRoot, provider, requireExactIdentity, requireCurrentOpenCodeProxy);
|
|
}
|
|
const expected = Buffer.from(
|
|
canonicalProviderPackJson(manifest),
|
|
"utf8",
|
|
).toString("base64");
|
|
const providerNodeCommand = posix.join(
|
|
packRoot,
|
|
manifest.payload.artifacts.nodeCommand.path,
|
|
);
|
|
const verifyScript = [
|
|
"const fs=require('node:fs')",
|
|
"const crypto=require('node:crypto')",
|
|
"const path=require('node:path')",
|
|
"const root=process.argv[1]",
|
|
"const expected=Buffer.from(process.argv[2],'base64').toString('utf8')",
|
|
"const actual=fs.readFileSync(path.join(root,'provider-pack.json'),'utf8').trim()",
|
|
"const canonical=(v)=>Array.isArray(v)?'['+v.map(canonical).join(',')+']':v&&typeof v==='object'?'{'+Object.keys(v).sort().map(k=>JSON.stringify(k)+':'+canonical(v[k])).join(',')+'}':JSON.stringify(v)",
|
|
"const manifest=JSON.parse(actual)",
|
|
"if(canonical(manifest)!==expected)throw new Error('manifest mismatch')",
|
|
"const hash=(p)=>'sha256:'+crypto.createHash('sha256').update(fs.readFileSync(path.join(root,p))).digest('hex')",
|
|
"const tree=(treeRoot)=>{const digest=crypto.createHash('sha256');const visit=(directory,prefix='')=>{for(const entry of fs.readdirSync(directory,{withFileTypes:true}).sort((a,b)=>a.name.localeCompare(b.name))){const relative=prefix?prefix+'/'+entry.name:entry.name;const absolute=path.join(directory,entry.name);if(entry.isDirectory()){digest.update('directory\\0'+relative+'\\n');visit(absolute,relative)}else if(entry.isFile()){digest.update('file\\0'+relative+'\\0'+'sha256:'+crypto.createHash('sha256').update(fs.readFileSync(absolute)).digest('hex')+'\\n')}else if(entry.isSymbolicLink()){digest.update('symlink\\0'+relative+'\\0'+fs.readlinkSync(absolute)+'\\n')}else throw new Error('unsupported dist entry '+relative)}};visit(treeRoot);return 'sha256:'+digest.digest('hex')}",
|
|
"for(const name of ['nodeCommand','productionLock','opencodeCommand','opencodeExecutable','opencodeProxy','acpxSidecar','grokLauncher']){const artifact=manifest.payload.artifacts[name];if(hash(artifact.path)!==artifact.sha256)throw new Error(name+' digest mismatch')}",
|
|
"if(tree(path.join(root,'dist'))!==manifest.payload.distDigest)throw new Error('dist tree digest mismatch')",
|
|
"for(const candidate of Object.values({...manifest.payload.providers,...manifest.payload.candidateProviders})){if(tree(path.join(root,candidate.path))!==candidate.sha256)throw new Error('candidate asset tree digest mismatch')}",
|
|
"const version=process.versions.node.split('.').map(Number)",
|
|
"const minimum=manifest.payload.pins.nodeMinimum.split('.').map(Number)",
|
|
"if(version[0]<minimum[0]||(version[0]===minimum[0]&&(version[1]<minimum[1]||(version[1]===minimum[1]&&version[2]<minimum[2]))))throw new Error('Node version incompatible')",
|
|
"if(process.platform!==manifest.payload.target.platform||process.arch!==manifest.payload.target.architecture)throw new Error('provider pack target mismatch')",
|
|
"const packageVersion=(pkg)=>JSON.parse(fs.readFileSync(path.join(root,'node_modules',...pkg.split('/'),'package.json'),'utf8')).version",
|
|
"const expectedPackages={acpx:manifest.payload.pins.acpx,'@agentclientprotocol/claude-agent-acp':manifest.payload.pins.claudeAcp,'@agentclientprotocol/codex-acp':manifest.payload.pins.codexAcp,'opencode-ai':manifest.payload.pins.opencode}",
|
|
"for(const [pkg,version] of Object.entries(expectedPackages))if(packageVersion(pkg)!==version)throw new Error(pkg+' version mismatch')",
|
|
].join(";");
|
|
await runRemoteProviderVerification(() => runner.execute({
|
|
command: providerNodeCommand,
|
|
args: ["-e", verifyScript, packRoot, expected],
|
|
cwd: cwd,
|
|
bypassSession: true,
|
|
timeoutMs: 30_000,
|
|
}), "provider pack verification");
|
|
const opencodeCommand = posix.join(
|
|
packRoot,
|
|
manifest.payload.artifacts.opencodeCommand.path,
|
|
);
|
|
const opencodeVersion = await runRemoteProviderVerification(() => runner.execute({
|
|
command: opencodeCommand,
|
|
args: ["--version"],
|
|
cwd: cwd,
|
|
bypassSession: true,
|
|
timeoutMs: 30_000,
|
|
}), "OpenCode version verification");
|
|
if (
|
|
opencodeVersion.stdout.trim() !== manifest.payload.pins.opencode
|
|
) {
|
|
throw new Error(
|
|
"runner_remote_provider_artifact_incompatible: OpenCode version mismatch",
|
|
);
|
|
}
|
|
return structuredClone(manifest);
|
|
}
|