mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-07 07:55:47 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Remote native runs use a provider pack. The controller reuses a protected pack from the sandbox image when the pack is compatible. > - #17749 gates this reuse on a hand-maintained interface revision in `provider-pack-contract.json`. > - Later changes altered the runner/pack boundary (#17922, #18188), but nobody bumped the revision. A stale Claude or Codex image pack can still be reused across a sidecar contract change. This is the failure class of #17972. > - A manual revision has a "forgot to bump" failure mode. Source-byte hashes change on every refactor, so they are too brittle. > - This pull request declares the sidecar `session.open` contract in its JSON schema and computes a contract digest from the JSON contracts and Rust shapes. > - Image-pack reuse now requires an equal digest. > - The benefit is that an incidental change keeps image packs reusable, and a real contract change retires them automatically. ## Linked Issues or Issue Description Refs #17749, #17972, #17922, #18188. **What happened?** The controller reused an image provider pack when `compatibility.acpx` and the selected agent's profile digest matched. For Claude and Codex, that profile digest binds only the upstream executable. The ACPX interface revision stayed at 1 after later sidecar contract changes. A pack built before those changes passed the reuse check. **Expected behavior** The controller reuses an image pack only when the pack uses the same runner/pack contracts as the controller's bundled pack. Comments, formatting and refactors do not change that result. **Steps to reproduce** 1. Build an image pack from a commit before a sidecar input change, such as a new `session.open` field. 2. Run a controller from a later commit that sends the new field, with the same interface revision and Claude version. 3. Before this fix, `verifyRemoteProviderPack` accepts the old image pack. ## What Changed - Declare the sidecar `session.open` parameter set as a closed `sessionOpenParams` definition in `acpx-sidecar.schema.json`. - `generate-acpx-sidecar-contract.mjs` now generates `GENERATED_ACPX_SIDECAR_OPEN_PARAMS` from it. - `parseOpenParams` uses the generated list instead of an inline `Set`. The accepted set is unchanged. - This is the surface of #17972. The schema is now its source of truth. - Declare the nested `session.open` contracts that the sidecar parses strictly: - `protocol/provider-schemas/runtime-context.schema.json` closes every runtime context object. `parseNativeRuntimeContext` uses the field lists that `scripts/generate-runtime-context-contract.mjs` generates from it. - The sidecar schema declares the closed `providerPolicy` object, and `parseProviderPolicy` uses its generated field list. - The accepted fields do not change. - Add `packages/paperclip-runner/scripts/contracts/contract-fingerprints.mjs`. It fingerprints these contracts: - the PRP and sidecar JSON schemas, with annotation text removed - the generated catalogs in `generated/` - the Rust serde shapes, `json!` payload keys, match literals and env keys in `runner-core/src`, without test-only modules - the reserved env key lists in `configured_environment.rs` - Read only the trees that every pack build context contains. Fail when a registered source is missing, so two build contexts cannot silently produce different digests. - Use only Node built-ins. The registry adds no dependency. - Write `contractDigest` as a top-level payload field in `build-provider-pack.mjs`. The manifest digest algorithm and the existing field layout do not change. - Require an equal `contractDigest` for image-pack reuse in `remote-provider-pack-verification.ts`. A pack without a digest is not reused. The controller uploads its bundled pack, as it does today for any incompatible pack. - Re-attest the source-hashed profiles, because this PR changes their bound sidecar sources: - Grok becomes profile 15 and now also binds the generated allowlist, so a Grok field change in the schema still invalidates the Grok identity. - Copilot becomes profile 88, and also binds the generated runtime context fields. - Earlier profiles stay decodable. - Add the registry script and `generated/` to the Daytona image content inputs. - Document the digest and the schema-first rule in `docker/daytona-runner/README.md` and `doc/DEVELOPING.md`. ## Verification - `node --test scripts/contracts/contract-fingerprints.test.mjs` (in `packages/paperclip-runner`): 4 tests pass. - These changes keep the digest: - schema description text - schema reformatting - renaming `parseOpenParams` - Rust comments, `#[serde(skip)]` fields and `#[cfg(test)]` modules - edits to the `#[path]` test module `tests/provider_api_validation.rs` - These changes alter the digest: - a new `session.open` schema field - a changed constraint on the `remainingWork[].description` property - a new reserved env key - a new sidecar env key - a new checkpoint field - A build context without a registered source fails. - `node --test test/acpx-sidecar-contract.test.mjs`: 13 tests pass. A new case rejects an unknown `session.open` parameter and accepts each declared parameter. - `node --test scripts/grok-profile-identity.test.mjs`: 4 tests pass. A change to the Grok field in the generated allowlist invalidates the Grok identity. - `generate-acpx-profiles.mjs --check`, `generate-acpx-sidecar-contract.mjs --check` and `generate-protocol-manifest.mjs --check` pass. - Runner vitest suites pass: 470 passed, 1 skipped. They cover the sidecar, sidecar input and protocol, Copilot and Pi installation, profile compatibility, native execution, recovery identity, the backend factory and the environment. - Server tests: pack verification, executor and artifact preparation pass (1,004 tests). - `tsc --noEmit` passes for `packages/paperclip-runner` and `server`. - `tests/runner-e2e/daytona-image.test.ts`: 10 of 11 tests pass. The failing test also fails on `master`. It expects a literal `profile-v19-identity.json` line in `.dockerignore`, which uses a `profile-v*` glob. - Not run locally: - A complete `build-provider-pack.mjs` run. The builder rejects the local Homebrew Node because it is not portable after relocation. - Rust tests and `acpx-controller-recovery.test.ts`, because `cargo` is not installed. - The full `pnpm -r typecheck`, `pnpm test:run` and `pnpm build`. ## Risks - Only JSON contracts and Rust shapes are part of the digest. The remaining TypeScript-only definitions are the `paperclip_finish` and `paperclip_block` tool schemas in `completion-result.ts` and the TypeScript env key lists. They are not covered until they move into a schema. The nested `expectedIdentity` and `tools` values ignore unknown fields, so a change there cannot cause a startup rejection. - A change there can let a stale image pack be reused. The sidecar then rejects the input with its strict parser error. - Today, Claude and Codex image packs have no contract check at all. - The digest covers all registered contracts for the whole pack. A contract change for one provider kind also retires image packs for the other. Each replacement costs about 16 s per remote start until the image is refreshed. - Existing image packs have no digest. After deploy, the controller uploads its bundled pack until the image is rebuilt. - The Grok and Copilot re-attestations start fresh sessions for those agents once after deploy, as each profile bump does today. - The Daytona image content ID formula changes, so the next image build does not reuse a cached image. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used Anthropic Claude Opus 5.5 (`claude-opus-5-5`) through Claude Code in the desktop app, with repository tools, shell execution and research subagents. The session does not expose the context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
240 lines
13 KiB
TypeScript
240 lines
13 KiB
TypeScript
import { createHash } from "node:crypto";
|
|
import {
|
|
PAPERCLIP_AGENT_DIRECTORY_PROMPT,
|
|
PAPERCLIP_EDITABLE_INSTRUCTIONS_PROMPT,
|
|
PAPERCLIP_EXECUTION_PROMPT,
|
|
PAPERCLIP_INSTRUCTION_SIBLING_ROOT_PROMPT,
|
|
} from "./runtime-context-prompts.js";
|
|
import { GENERATED_RUNTIME_CONTEXT_FIELDS } from "./generated-runtime-context-contract.js";
|
|
|
|
export { PAPERCLIP_EXECUTION_PROMPT, PAPERCLIP_EXECUTION_PROMPT_REVISION } from "./runtime-context-prompts.js";
|
|
|
|
export const NATIVE_RUNTIME_ASSET_SCHEMA = "paperclip.runtime-asset.v1" as const;
|
|
|
|
export interface NativeRuntimeAssetReference {
|
|
schema: typeof NATIVE_RUNTIME_ASSET_SCHEMA;
|
|
digest: string;
|
|
manifestDigest: string;
|
|
rootPath: string;
|
|
fileCount: number;
|
|
totalBytes: number;
|
|
}
|
|
|
|
export interface NativeRuntimeContextSnapshot {
|
|
prompt: { revision: string; text: string; digest: string };
|
|
instructions: {
|
|
entryPath: string;
|
|
bundle: NativeRuntimeAssetReference;
|
|
/** Server-registered writable copy; excluded from the pinned prompt digest. */
|
|
workingCopy?: { rootPath: string; entryPath: string; kind?: "agent_files" };
|
|
};
|
|
skills: Array<{ key: string; runtimeName: string; versionId: string | null; bundle: NativeRuntimeAssetReference }>;
|
|
mcp: { assignmentSetId: string; digest: string; bindingId: string | null };
|
|
connectionInstructions?: { text: string; digest: string };
|
|
aggregateDigest: string;
|
|
}
|
|
|
|
/** Authenticated current-run grants; immutable session policy is verified separately. */
|
|
export interface NativeRunRuntimeGrant {
|
|
runtimeContext: NativeRuntimeContextSnapshot | null;
|
|
instructions: string;
|
|
}
|
|
|
|
export class NativeRuntimeContextError extends Error {
|
|
readonly code = "native_runtime_context_invalid" as const;
|
|
constructor(message: string) { super(message); this.name = "NativeRuntimeContextError"; }
|
|
}
|
|
|
|
const sha256 = (value: string) => createHash("sha256").update(value).digest("hex");
|
|
const object = (value: unknown, path: string): Record<string, unknown> => {
|
|
if (!value || typeof value !== "object" || Array.isArray(value)) throw new NativeRuntimeContextError(`${path} must be an object`);
|
|
return value as Record<string, unknown>;
|
|
};
|
|
const exact = (value: Record<string, unknown>, keys: readonly string[], path: string) => {
|
|
const allowed = new Set(keys);
|
|
const unknown = Object.keys(value).find((key) => !allowed.has(key));
|
|
if (unknown) throw new NativeRuntimeContextError(`${path} contains unknown field ${unknown}`);
|
|
};
|
|
const text = (value: unknown, path: string) => {
|
|
if (typeof value !== "string" || !value.trim()) throw new NativeRuntimeContextError(`${path} must be a non-empty string`);
|
|
return value;
|
|
};
|
|
const digest = (value: unknown, path: string) => {
|
|
const result = text(value, path);
|
|
if (!/^[a-f0-9]{64}$/.test(result)) throw new NativeRuntimeContextError(`${path} must be a sha256 digest`);
|
|
return result;
|
|
};
|
|
const integer = (value: unknown, path: string) => {
|
|
if (!Number.isSafeInteger(value) || Number(value) < 0) throw new NativeRuntimeContextError(`${path} must be a non-negative integer`);
|
|
return Number(value);
|
|
};
|
|
const safeRelativePath = (value: unknown, path: string) => {
|
|
const result = text(value, path).replaceAll("\\", "/");
|
|
if (result.startsWith("/") || result.includes("\0") || result.split("/").some((part) => !part || part === "." || part === "..")) {
|
|
throw new NativeRuntimeContextError(`${path} must stay within its bundle root`);
|
|
}
|
|
return result;
|
|
};
|
|
|
|
function parseAsset(value: unknown, path: string): NativeRuntimeAssetReference {
|
|
const asset = object(value, path);
|
|
exact(asset, GENERATED_RUNTIME_CONTEXT_FIELDS.asset, path);
|
|
if (asset.schema !== NATIVE_RUNTIME_ASSET_SCHEMA) throw new NativeRuntimeContextError(`${path}.schema is unsupported`);
|
|
return {
|
|
schema: NATIVE_RUNTIME_ASSET_SCHEMA,
|
|
digest: digest(asset.digest, `${path}.digest`),
|
|
manifestDigest: digest(asset.manifestDigest, `${path}.manifestDigest`),
|
|
rootPath: text(asset.rootPath, `${path}.rootPath`),
|
|
fileCount: integer(asset.fileCount, `${path}.fileCount`),
|
|
totalBytes: integer(asset.totalBytes, `${path}.totalBytes`),
|
|
};
|
|
}
|
|
|
|
function aggregatePayload(value: Omit<NativeRuntimeContextSnapshot, "aggregateDigest">) {
|
|
return {
|
|
prompt: value.prompt,
|
|
instructions: { entryPath: value.instructions.entryPath, bundleDigest: value.instructions.bundle.digest },
|
|
skills: [...value.skills].sort((a, b) => a.key.localeCompare(b.key)).map((skill) => ({
|
|
key: skill.key, runtimeName: skill.runtimeName, versionId: skill.versionId, bundleDigest: skill.bundle.digest,
|
|
})),
|
|
// The binding is deliberately run-scoped. Compatibility is determined by the
|
|
// assigned access set so a fresh capability can be rebound without forcing a
|
|
// provider-session rotation when policy has not changed.
|
|
mcp: { assignmentSetId: value.mcp.assignmentSetId, digest: value.mcp.digest },
|
|
...(value.connectionInstructions ? { connectionInstructions: value.connectionInstructions } : {}),
|
|
};
|
|
}
|
|
|
|
export function canonicalNativeRuntimeContextDigest(value: Omit<NativeRuntimeContextSnapshot, "aggregateDigest">): string {
|
|
return sha256(JSON.stringify(aggregatePayload(value)));
|
|
}
|
|
|
|
export function nativeRuntimePromptDigest(): string { return sha256(PAPERCLIP_EXECUTION_PROMPT); }
|
|
|
|
export function parseNativeRuntimeContext(value: unknown): NativeRuntimeContextSnapshot {
|
|
const context = object(value, "input.runtimeContext");
|
|
exact(context, GENERATED_RUNTIME_CONTEXT_FIELDS.runtimeContext, "input.runtimeContext");
|
|
const prompt = object(context.prompt, "input.runtimeContext.prompt");
|
|
exact(prompt, GENERATED_RUNTIME_CONTEXT_FIELDS.prompt, "input.runtimeContext.prompt");
|
|
// New runs use the current constants. Recovery uses the immutable saved
|
|
// snapshot; its revision is metadata, not a lookup in this release's source.
|
|
const promptRevision = text(prompt.revision, "input.runtimeContext.prompt.revision");
|
|
const promptText = text(prompt.text, "input.runtimeContext.prompt.text");
|
|
const promptDigest = digest(prompt.digest, "input.runtimeContext.prompt.digest");
|
|
if (sha256(promptText) !== promptDigest) {
|
|
throw new NativeRuntimeContextError("input.runtimeContext.prompt.digest does not match prompt text");
|
|
}
|
|
const instructions = object(context.instructions, "input.runtimeContext.instructions");
|
|
exact(instructions, GENERATED_RUNTIME_CONTEXT_FIELDS.instructions, "input.runtimeContext.instructions");
|
|
const workingCopy = instructions.workingCopy === undefined ? undefined : object(instructions.workingCopy, "input.runtimeContext.instructions.workingCopy");
|
|
if (workingCopy) exact(workingCopy, GENERATED_RUNTIME_CONTEXT_FIELDS.workingCopy, "input.runtimeContext.instructions.workingCopy");
|
|
if (workingCopy?.kind !== undefined && workingCopy.kind !== "agent_files") throw new NativeRuntimeContextError("Unknown agent file contract");
|
|
if (!Array.isArray(context.skills)) throw new NativeRuntimeContextError("input.runtimeContext.skills must be an array");
|
|
const skills = context.skills.map((value, index) => {
|
|
const skill = object(value, `input.runtimeContext.skills[${index}]`);
|
|
exact(skill, GENERATED_RUNTIME_CONTEXT_FIELDS.skill, `input.runtimeContext.skills[${index}]`);
|
|
return {
|
|
key: text(skill.key, `input.runtimeContext.skills[${index}].key`),
|
|
runtimeName: safeRelativePath(skill.runtimeName, `input.runtimeContext.skills[${index}].runtimeName`),
|
|
versionId: skill.versionId === null ? null : text(skill.versionId, `input.runtimeContext.skills[${index}].versionId`),
|
|
bundle: parseAsset(skill.bundle, `input.runtimeContext.skills[${index}].bundle`),
|
|
};
|
|
});
|
|
if (new Set(skills.flatMap((skill) => [skill.key, `runtime:${skill.runtimeName}`])).size !== skills.length * 2) {
|
|
throw new NativeRuntimeContextError("input.runtimeContext.skills contains duplicate identities");
|
|
}
|
|
const mcp = object(context.mcp, "input.runtimeContext.mcp");
|
|
exact(mcp, GENERATED_RUNTIME_CONTEXT_FIELDS.mcp, "input.runtimeContext.mcp");
|
|
let connectionInstructions: NativeRuntimeContextSnapshot["connectionInstructions"];
|
|
if (context.connectionInstructions !== undefined) {
|
|
const block = object(context.connectionInstructions, "input.runtimeContext.connectionInstructions");
|
|
exact(block, GENERATED_RUNTIME_CONTEXT_FIELDS.connectionInstructions, "input.runtimeContext.connectionInstructions");
|
|
const content = text(block.text, "input.runtimeContext.connectionInstructions.text");
|
|
const contentDigest = digest(block.digest, "input.runtimeContext.connectionInstructions.digest");
|
|
if (sha256(content) !== contentDigest) throw new NativeRuntimeContextError("Connection instruction digest does not match text");
|
|
connectionInstructions = { text: content, digest: contentDigest };
|
|
}
|
|
const parsed = {
|
|
prompt: { revision: promptRevision, text: promptText, digest: promptDigest },
|
|
instructions: {
|
|
entryPath: safeRelativePath(instructions.entryPath, "input.runtimeContext.instructions.entryPath"),
|
|
bundle: parseAsset(instructions.bundle, "input.runtimeContext.instructions.bundle"),
|
|
...(workingCopy ? { workingCopy: {
|
|
...(workingCopy.kind === "agent_files" ? { kind: "agent_files" as const } : {}),
|
|
rootPath: text(workingCopy.rootPath, "input.runtimeContext.instructions.workingCopy.rootPath"),
|
|
entryPath: safeRelativePath(workingCopy.entryPath, "input.runtimeContext.instructions.workingCopy.entryPath"),
|
|
} } : {}),
|
|
},
|
|
skills,
|
|
mcp: {
|
|
assignmentSetId: text(mcp.assignmentSetId, "input.runtimeContext.mcp.assignmentSetId"),
|
|
digest: digest(mcp.digest, "input.runtimeContext.mcp.digest"),
|
|
bindingId: mcp.bindingId === null ? null : text(mcp.bindingId, "input.runtimeContext.mcp.bindingId"),
|
|
},
|
|
...(connectionInstructions ? { connectionInstructions } : {}),
|
|
} satisfies Omit<NativeRuntimeContextSnapshot, "aggregateDigest">;
|
|
const aggregateDigest = digest(context.aggregateDigest, "input.runtimeContext.aggregateDigest");
|
|
if (aggregateDigest !== canonicalNativeRuntimeContextDigest(parsed)) {
|
|
throw new NativeRuntimeContextError("input.runtimeContext.aggregateDigest does not match the canonical context");
|
|
}
|
|
return { ...parsed, aggregateDigest };
|
|
}
|
|
|
|
function renderInstructionPaths(prompt: string, paths: { rootPath: string; entryPath: string }): string {
|
|
return prompt.replace(/\{(rootPath|entryPath)\}/g, (_placeholder, key: "rootPath" | "entryPath") => paths[key]);
|
|
}
|
|
|
|
export function composeNativeSystemInstructions(context: NativeRuntimeContextSnapshot, entryContent: string): string {
|
|
return [
|
|
context.prompt.text,
|
|
entryContent.trim(),
|
|
context.connectionInstructions?.text,
|
|
context.instructions.workingCopy?.kind === "agent_files"
|
|
? renderInstructionPaths(PAPERCLIP_AGENT_DIRECTORY_PROMPT, context.instructions.workingCopy)
|
|
: context.instructions.workingCopy
|
|
? renderInstructionPaths(PAPERCLIP_EDITABLE_INSTRUCTIONS_PROMPT, context.instructions.workingCopy)
|
|
: null,
|
|
// Keep this canonical suffix intact for provider-specific asset remapping.
|
|
PAPERCLIP_INSTRUCTION_SIBLING_ROOT_PROMPT + context.instructions.bundle.rootPath,
|
|
].filter(Boolean).join("\n\n");
|
|
}
|
|
|
|
export function retargetNativeSystemInstructions(
|
|
instructions: string,
|
|
prior: NativeRuntimeContextSnapshot,
|
|
current: NativeRuntimeContextSnapshot | null,
|
|
): string {
|
|
const priorPrefix = prior.prompt.text;
|
|
const priorSuffix = composeNativeSystemInstructions(prior, "").slice(priorPrefix.length);
|
|
// Legacy callers can supply opaque system instructions. Only the composer's
|
|
// exact framing identifies a trusted asset block; never rewrite their text.
|
|
if (!instructions.startsWith(priorPrefix) || !instructions.endsWith(priorSuffix)) {
|
|
return instructions;
|
|
}
|
|
const custom = instructions.slice(priorPrefix.length, -priorSuffix.length);
|
|
if (custom && !custom.startsWith("\n\n")) return instructions;
|
|
// Keep custom entry bytes intact, including historical path examples or an
|
|
// identical paragraph quoted inside the entry. Only replace the final block.
|
|
return (current?.prompt.text ?? priorPrefix) + custom + (current
|
|
? composeNativeSystemInstructions(current, "").slice(current.prompt.text.length)
|
|
: "");
|
|
}
|
|
|
|
/** Explicit per-turn selection; availability alone never invokes a skill. */
|
|
export interface NativeSkillInput {
|
|
type: "skill";
|
|
name: string;
|
|
path: string;
|
|
}
|
|
|
|
|
|
/** Select only explicit task references from assigned names, never comments. */
|
|
export function explicitTaskSkillNames(description: string | null, assignedNames: readonly string[]): string[] {
|
|
if (!description) return [];
|
|
const names = new Set(Array.from(
|
|
description.matchAll(/(?:^|[\s(`])[$/]([a-zA-Z0-9_-]+)(?=$|[\s)`,.;:!?])/g),
|
|
(match) => match[1],
|
|
));
|
|
return [...new Set(assignedNames)].filter((name) => names.has(name));
|
|
}
|