Bind candidate diagnostics and recovery to explicit runner policy

Preserve Pi pricing receipt provenance, require evaluation opt-in, and record qualification boundaries.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip committed 2026-09-28 11:55:09 -05:00
1 parent 84336a1b7d
commit 588dd00af1
18 files changed
+302 -45

No files matched your search

@@ -0,0 +1,150 @@
# Rich ACP integration and qualification report
Updated: 2026-09-28. Base: `c65fc9e3c81c41aafe421aa90a00514b84343285`.
Status: implementation and deterministic verification in progress; all three new
profiles remain **pending qualification**. No paid inference or Daytona resources
have been used. This report does not certify a provider from its ACP listing.
The [harness priorities report](https://pages.paperclip.ing/2026-09-25-harness-priorities/)
recommends Cursor and Copilot, followed by Pi, using the existing qualified ACPX
path. Codex app-server is the richness benchmark. The legacy Cursor and Pi
adapters are outside this change.
## Branches and evidence ownership
| Branch | Deliverable |
| --- | --- |
| `codex/runner-rich-acp` | Shared ACPX extension boundary, durable permissions, canonical display events, provider pack infrastructure, configuration and UI |
| `codex/runner-cursor-acp` | Cursor native distribution, questions/plans, child activity, policy admission, wire fixtures |
| `codex/runner-copilot-acp` | Copilot native distribution, event inventory/projections, permission and settlement probes |
| `codex/runner-pi-acp` | Patched wrapper, owned extension, MCP/tools, permissions/input, portable dependency closure |
Each provider branch depends on the foundation. They remain separate worktrees
and review units. Source reports on the provider branches are
`doc/architecture/runner-cursor-capabilities.md`,
`doc/architecture/runner-copilot-capabilities.md`, and
`doc/architecture/runner-pi-capabilities.md`. Those reports retain versioned
source references, per-field dispositions, fixture paths, and narrower claims.
The Copilot inventory enumerates all 150 pinned native event types.
## Capability matrix
“Candidate” means implemented or observed in deterministic tests. It does not
mean the required paid local and Daytona product cases have passed. “Not exposed”
means the pinned interface was inspected; “unverified” is a separate finding.
Codex's row is the existing app-server integration, not the Codex ACP bridge.
| Capability | Codex app-server benchmark | Cursor ACP candidate | Copilot ACP candidate | Pi ACP candidate |
| --- | --- | --- | --- | --- |
| Exact model | Selected and reported model | Explicit ID required; authenticated discovery unverified | Explicit ID required; offline ACP status omits model metadata; admission fails if exact model cannot be proved | Exact `openrouter/deepseek/deepseek-v4-flash-0731`; paid model response unverified |
| Text and tools | Typed thread/turn/item events | Standard ACP updates; child activity kept separate | Standard ACP plus opt-in native session events | Wrapper text/tool updates and owned tool gates |
| Active steering | Dedicated `turn/steer` | Concurrent prompt replaces/cancels, so it is not steering | Concurrent prompt replaces/cancels, so it is not steering | Owned `pi/steer` requires handshake, exact active turn and acknowledgment |
| Queued follow-up | Product continuation controls | Controller can schedule a later prompt; native queue not established | Native pending-message activity exists; no qualified ACP queue responder | Owned `pi/follow_up`, separately named and ordered |
| Cancellation | Typed interrupt and process lifecycle | ACP cancel; paid command cleanup pending | ACP cancel; paid background settlement pending | Native abort; wrapper waits for `agent_settled` and treats provider errors as failure |
| Session continuity | Read/load/history/fork and durable identity | Session load/list observed; authenticated replay pending; fork absent in tested methods | Session load plus native history events; live recovery pending | Private Pi JSONL mapping/load; unresolved UI promises cannot survive provider death |
| Questions | Typed input requests and response correlation | `cursor/ask_question`, option identity and multiple selection preserved | Native ask-user capability exists, but pinned ACP does not wire its responder; do not display a false answerable form | `select`, `confirm`, `input`, `editor` through typed form elicitation |
| Permissions | Durable typed approvals | Standard ACP permission options; only supported decisions shown | Standard ACP; offline deny-before-marker proof; session decision scope inspected | Native pre-tool gate; allow once, exact-operation session grant, deny; paths rechecked after wait |
| Plans | Typed plan and collaboration mode | `cursor/create_plan` includes full plan and revision-bound accept/reject/cancel; todo activity separate | Native plan events displayed; native plan-decision callback not exposed in ACP | No native structured plan event; authenticated Paperclip planning tools available |
| Authenticated tools | Runner bridge and governed operations | ACP HTTP MCP binding; paid semantic cases pending | ACP HTTP MCP binding; paid semantic cases pending | Owned extension registers exact bound MCP tools; no ambient servers |
| Delegation | Typed agent roles and lifecycle | Opt-in subagent lifecycle, nested ownership and bounded child activity; never parent transcript flattening | Native delegation/session events projected with role/model/agent provenance | No built-in ACP delegation protocol; arbitrary extensions are excluded |
| Files/diffs | Typed file changes and artifact references | Standard tool changes plus validated image references | File/workspace events retained; provider session files are not silently treated as task files | Native read/write/edit diffs; semantic artifact tools |
| Images/artifacts | Typed references and registered work products | Existing contained files only, provenance, `registered:false` | Contained task references only; external/session-store paths become descriptive notices | Image/resource tool blocks preserved by wrapper; dedicated artifact channel absent |
| Usage | Per-request receipt and model context | Native usage completeness unverified | ACP/native usage retained without inventing a billing charge | Assistant-message token receipts; dollar cost is a catalog pricing estimate, never authoritative billing |
| Config/model changes | Typed configurable controls | Known modes/model interfaces researched; runtime policy cannot be changed by a display event | Model/reasoning/mode options exist; runtime policy remains authoritative | Exact qualified model; arbitrary slash commands/config/extensions disabled |
| Reconnect/restart | Durable controller replay and qualified provider restoration | Live-process pending input only until exact restoration is proved | Same; history alone is not approval restoration | Same; wrapper explicitly advertises live-process-only pending-input recovery |
## Shared event and interaction contract
The active connection, wire session, normalized turn and original request identity
bind every extension callback. The allowlist is per provider. An omitted wire
session can only acquire the verified active connection's session; an explicit
mismatch is rejected. Retired streams and inactive turns cannot emit new activity.
Requests enter durable runtime state before the UI presents them. The response
must match an outstanding request and an offered action or valid typed answer.
The sidecar validates and accepts delivery before Rust settles its durable record.
Standard ACP does not acknowledge application of a permission reply; a lost
transport acknowledgment is not proof of exactly-once external effects. A
replacement provider process cannot inherit an old approval promise. No tool
mutation or approval is automatically replayed into a replacement.
Full plan documents have a bounded 100,000-character description and a 196 KiB
question-set envelope. Oversized plans fail rather than approve an unseen suffix.
Display redaction remains visible. The rich event channel has exact canonical
schemas and a bounded envelope. It cannot create terminal outcomes, dispatch a
semantic tool, register an artifact, synchronize a durable plan, or supply source
authority. Notices retain useful bounded fields and provenance in expandable UI
details. Provider references remain unregistered until a control-plane operation
registers them.
Permission labels are derived from offered option kinds. An unknown or duplicate
option is rejected. A provider's “always” decision is not relabeled “this session”
unless the pinned implementation proves that scope. Restrictive execution policy
is separate from automatic approval and company governance. Fresh permission
configuration defaults to full auto; it never overrides read-only task policy.
## Distribution and isolation
Cursor pins `2026.09.26-dd393fe`; Copilot pins `1.0.88`; Pi pins wrapper `0.0.33`,
runtime `0.84.2`, portable Node and its full npm lock. Native distribution hashes
cover macOS ARM64, macOS x64 and Linux x64. Source-owned closure pins remain
separate from profile declaration digests. Native admission reads held files,
verifies every admitted byte, creates a private immutable snapshot and retains
the existing process guardian. A manifest cannot supply its own trusted pin.
Candidate credentials are only read from explicitly bound run environments:
Cursor `CURSOR_API_KEY`/`CURSOR_AUTH_TOKEN`, Copilot `COPILOT_GITHUB_TOKEN`, Pi
`OPENROUTER_API_KEY`. Ambient GitHub login variables, provider configuration,
extensions and MCP discovery do not establish authority. Homes/config/cache are
private. Updates are disabled. Pi launches only its owned extension and assigned
skills, with a startup sentinel before a prompt can run.
Build candidate packs explicitly with `--candidate-providers=<name>` on that
provider's branch. Local and remote pack verification includes the complete
candidate asset tree. The corresponding Daytona build argument is documented in
`docker/daytona-runner/README.md`. Candidate packaging never promotes a profile.
ACP is not an OS sandbox. Cursor still reads project MCP/hooks beyond its
`--disable-project-configs` flag; the adapter rejects known ambient execution
configuration. Mutations during a run and remote team hooks still require
qualification. Pi's tool policy supplements the execution boundary; arbitrary
shell commands and filesystem races require the host boundary. These are
qualification gates, not claims that a JavaScript path check confines a shell.
## Explicit gaps and follow-ups
| Priority | Exposed but unused, partial, or unverified | Reason and next proof |
| --- | --- | --- |
| P0 | All paid product cases on local and Daytona | Bound provider/Daytona credentials and independently inspectable spend are absent. Prove setup, semantic tools, edit+validation, inputs, cancellation, warm continuation and restart. |
| P0 | Copilot native ask-user and plan-decision callbacks | Pinned ACP does not install native responders. Prove no blocking request is exposed, or add a qualified responder/wrapper; never swallow the request. |
| P0 | Copilot denial and detached background settlement | Offline deny/attached-shell tests pass only their narrow fixtures. Exercise affected native tools and detached commands against real service; failing release stays unqualified. |
| P0 | Exact Cursor/Copilot model reporting | Authentication/entitlement required; no guessed default and no fallback. Fail admission when effective model is unverified. |
| P0 | Cursor project and remote hooks; native shell boundaries | Configuration flags do not cover every native source. Demonstrate policy cannot be bypassed before production qualification. |
| P1 | Native Cursor/Copilot active steering and queues | ACP prompt replacement is not steering; native SDK capabilities may be richer. Require a dedicated bound method plus acknowledgment before advertising. |
| P1 | Child tool media/diff/raw payloads | Bounded delegation summaries preserve lifecycle and identity. Large nested payloads need a child-owned canonical item model; current omission is a visible notice and provider report entry. |
| P1 | Copilot session-store files and export/artifact URIs | Provider paths are not task-workspace paths. Add a separately authorized export flow with validated bytes and provenance; do not resolve arbitrary URLs or auto-register. |
| P1 | Complete usage/billing provenance | Missing cache fields remain unknown. Pi price estimates are displayed separately. Budget qualification requires actual spend coverage, not an estimate presented as a bill. |
| P1 | Fork/history/model/mode controls not exposed by Paperclip | Research documents the native and ACP methods separately. Add governance-aware controls and durable lineage before enabling them. |
| P1 | Exact pending-request restoration after process death | Session transcript restoration does not restore callbacks. Expire unresolved requests unless a provider proves exact restoration. |
| P2 | Remaining Copilot native diagnostic/config/account events | The provider inventory records every event and field, its projection or reason for omission. Preserve bounded useful context; avoid credentials, raw environment or unbounded blobs. |
| P2 | Pi native extension surfaces and unsupported slash commands | Arbitrary extensions/templates/themes may execute ambient code. Only reviewed runner-owned capabilities are admitted; native plan/fork/goals are not fabricated. |
## Qualification ledger
Combined ceiling: **$100**, including retries and infrastructure. Cursor allocation:
$25; Copilot: $25; Pi: $25; coordinated diagnosis reserve: $25. Used: **$0**.
No model requests or Daytona leases have been started. Registry/binary downloads,
loopback fake-model tests and uncredentialed initialization are deterministic
engineering evidence, not paid qualification.
Before each paid batch, record source SHA, executable and closure/profile digests,
exact model, OS/architecture or Daytona image, selected cases, prior spend,
maximum batch spend and authoritative billing coverage. Stop before the shared
ceiling. Missing spend coverage blocks a run rather than treating unknown cost as
zero. Retain screenshots and wire evidence without credentials. Never convert a
provider to supported solely because a test suite or packaging check passed.
Verification commands and final results are recorded with the prerequisite and
provider PRs. The full handoff requires runner checks, token gates, recursive
typecheck, `pnpm test:run`, and `pnpm build`. Until that evidence is recorded, this
report is an implementation report rather than a PR-ready certification.
+30
View File
@@ -0,0 +1,30 @@
# Cursor, Copilot and Pi through rich ACP
Approved implementation plan, 2026-09-28. Shared base:
`c65fc9e3c81c41aafe421aa90a00514b84343285`.
Keep the existing ACPX runner transport and Codex app-server richness benchmark.
Implement a foundation prerequisite, then three separate provider branches and
PRs. Do not merge automatically. Preserve legacy Cursor/Pi adapters.
1. Investigate each pinned harness, its native API, ACP adapter, observed wire,
runner normalization and UI surface. Inventory every meaningful event field.
2. Extend shared profile declarations, durable permissions/input, extension
dispatch, steering/queue distinctions, artifact provenance and bounded display
activity. Preserve existing replay and exact warm-session identities.
3. Pin complete provider distributions for macOS ARM64/x64 and Linux x64. Use
verified launches, isolated credentials/configuration and build-time assets.
4. Implement Cursor native ACP and its question/plan/subagent extensions;
Copilot ACP authentication/events/permission and settlement handling; Pi
wrapper patch and owned extension with authenticated tools and input policy.
5. Run deterministic, recovery, isolation and richness conformance. Qualify each
provider locally and in Daytona with product E2E evidence before advertising
support. Use a combined $100 ceiling: $25 each plus $25 diagnosis reserve.
6. Run required full checks, prepare one foundation and three dependent PRs,
retain evidence and deliver the comparative capability report. List every
unused/partial capability with reason and prioritized follow-up.
Implementation and qualification status is maintained in
[the capability report](../architecture/runner-rich-acp-capabilities.md).
Missing credentials or spend verification leave qualification pending; they do
not authorize a weaker supported profile or an implicit model fallback.
+18 -8
View File
@@ -54,7 +54,11 @@ leave the board showing an active run after the provider turn has already ended.
The package also builds `paperclip-runner-acpx-sidecar`. This bounded v2
stdin/stdout bridge admits the pinned Claude and Codex ACPX profiles. It
validates the exact model, session identity, tool catalog, structured input,
and terminal settlement at the process boundary. Pi remains unavailable.
and terminal settlement at the process boundary. Cursor, Copilot and Pi have
separate candidate branches and remain unavailable in production until local
and Daytona qualification passes. Their verified distributions are build-owned;
no candidate accepts an arbitrary executable. See
[the rich ACP capability report](../../doc/architecture/runner-rich-acp-capabilities.md).
Native Claude skill assignments travel in the runtime-context snapshot through
runnerd to the ACPX sidecar. After acquiring the provider lifetime lease, the
@@ -83,12 +87,14 @@ do not change workspace isolation or grant credentials or connection access.
task tools; `approve-reads` allows assigned reads; `deny-all` rejects requests.
None of these restrictive modes is the default.
This runtime has no interactive permission handler. An operation that still
requires approval stops the turn with `approval_required`. The server marks the
task blocked, exposes the permission action to the operator, and disables
automatic retry. The operator must review the operation and the agent's
permission setting before retrying. Company access checks still run when each
Paperclip tool executes.
Restrictive profiles route supported permission decisions through durable runtime
requests and the existing task interaction controls. Requests are persisted
before presentation; answers are checked against the offered decisions and
acknowledged by the sidecar before settlement. Unknown, stale and duplicate
responses fail. Missing provider decision support remains a blocked disposition,
not implicit approval. Company access checks still run for each Paperclip tool.
Provider death expires pending promises; approvals are never replayed into a
replacement process.
Runnerd selects only qualified provider profiles. Claude Managed and AWS
AgentCore receive immutable company-profile snapshots with explicit retention,
@@ -265,7 +271,11 @@ pnpm --filter @paperclipai/paperclip-runner report:runner-chaos-evals
`report:runner-live-evals` is a paid, provider-backed command. Native Codex
requires `OPENAI_API_KEY`; ACPX Claude requires
`ANTHROPIC_API_KEY`; OpenCode candidates require `OPENROUTER_API_KEY`. The live
matrix admits no Pi profile and does not persist credential values. Set
matrix remains qualified-only and does not persist credential values. Candidate
qualification uses `eval-session --candidate-profile <pi|cursor|copilot>` with an
explicit model and a separately materialized pinned candidate pack. This option
is a constructor-bound diagnostic opt-in; session JSON cannot enable a candidate.
Missing credentials or unverifiable spend block paid qualification. Set
`PAPERCLIP_EVAL_MAX_CAMPAIGN_COST_USD` to a positive finite number to bound
additional scheduling after the observed campaign total reaches that value:
+1 -1
View File
@@ -74,7 +74,7 @@
"typecheck:browser": "tsc -p tsconfig.browser.json --noEmit",
"test": "pnpm run test:typescript && pnpm run test:rust",
"test:typescript": "pnpm run test:typescript:prep && vitest run",
"test:typescript:prep": "pnpm run ensure:eval-build-deps && pnpm run build:rust && node --test test/protocol-contract.test.mjs test/acpx-sidecar-contract.test.mjs test/acpx-codex-package-contract.test.mjs test/acpx-rich-extensions-package-contract.test.mjs scripts/aws-agentcore-provisioning.test.mjs scripts/build-verified-provider-entrypoints.test.mjs scripts/local-provider-smoke-environment.test.mjs scripts/materialize-opencode-binary.test.mjs",
"test:typescript:prep": "pnpm run ensure:eval-build-deps && pnpm run build:rust && node --test test/protocol-contract.test.mjs test/acpx-sidecar-contract.test.mjs test/acpx-codex-package-contract.test.mjs test/acpx-rich-extensions-package-contract.test.mjs test/acpx-pi-receipt-package-contract.test.mjs test/acpx-response-delivery-package-contract.test.mjs scripts/candidate-provider-pack.test.mjs scripts/aws-agentcore-provisioning.test.mjs scripts/build-verified-provider-entrypoints.test.mjs scripts/local-provider-smoke-environment.test.mjs scripts/materialize-opencode-binary.test.mjs",
"test:typescript:vitest": "node ./scripts/run-pr-vitest-lane.mjs",
"test:rust": "cargo test --release --manifest-path runner/Cargo.toml --locked --workspace",
"test:codex": "cargo test --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --test codex_provider",
@@ -13,7 +13,7 @@ import {
export interface CodexAcpxNativeSessionBackendOptions extends Omit<
CodexAcpxDriverOptions,
"model" | "permissionMode" | "systemInstructions"
"model" | "permissionMode" | "systemInstructions" | "providerPolicy"
> {}
export type AcpxNativeSessionBackendOptions =
@@ -30,11 +30,6 @@ export function createAcpxNativeSessionBackend(
if (input.provider.kind !== "acpx") {
throw new Error("ACPX backend requires provider kind acpx");
}
if (input.provider.agent === "pi") {
throw new Error(
"Pi ACPX backend is unavailable until descriptor-confined verified launch is implemented",
);
}
const qualifiedProfile = resolveQualifiedAcpxProfile(
input.provider.agent,
input.provider.model,
@@ -73,6 +68,7 @@ export function createAcpxNativeSessionBackend(
model: input.provider.model,
permissionMode: input.provider.permissionMode ?? "approve-reads",
systemInstructions,
providerPolicy: { readOnly: "executionMode" in input && input.executionMode === "plan" },
}),
);
}
@@ -487,12 +487,9 @@ describe("native backend factory", () => {
},
);
it("rejects Pi before constructing an ACPX backend", () => {
expect(() =>
createNativeSessionBackend(acpxExecution("pi"), {
acpxRuntimeDirectory: "/runtime",
}),
).toThrow("descriptor-confined verified launch");
it("constructs a Pi candidate descriptor while runtime installation remains separately gated", async () => {
const backend = createNativeSessionBackend(acpxExecution("pi"), { acpxRuntimeDirectory: "/runtime" });
await expect(backend.descriptor()).resolves.toMatchObject({ name: "acpx_runtime", version: "0.13.1" });
});
it("rejects a Codex ACPX snapshot that drifts from its qualified profile", () => {
@@ -77,11 +77,6 @@ export function createNativeSessionBackend(
});
}
if (input.provider.kind === "acpx") {
if (input.provider.agent === "pi") {
throw new Error(
"Native ACPX backend for pi is unavailable until descriptor-confined verified launch is implemented",
);
}
if (!options.acpxRuntimeDirectory?.trim()) {
throw new Error("ACPX backend requires an instance runtime directory");
}
@@ -11,3 +11,8 @@ export async function verifyAcpxProfileInstallation(profile: QualifiedAcpxProfil
/** Provider policy admission is repeated immediately before each process launch. */
export async function assertAcpxProfileWorkspace(_agent: QualifiedAcpxAgent, _workspace: string): Promise<void> {}
/** Optional provider-specific classification; never changes whether admission succeeded. */
export function classifyAcpxProfileError(_agent: QualifiedAcpxAgent, _error: unknown): Error | null {
return null;
}
@@ -76,6 +76,21 @@ describe("ACPX recovery identity", () => {
).not.toThrow();
});
it("rejects restoration across task execution-policy changes", async () => {
const fixture = await recoveryFixture();
const readonly = await createAcpxRecoveryBinding({ ...fixture.input, providerPolicy: { readOnly: true } });
const writable = await createAcpxRecoveryBinding({ ...fixture.input, providerPolicy: { readOnly: false } });
const record = createAcpxIdentityRecord(fixture.expected, readonly);
expect(readonly.profileSessionKey).not.toBe(writable.profileSessionKey);
expect(readonly.profileDigest).not.toBe(writable.profileDigest);
expect(() => verifyExpectedAcpxIdentity(fixture.expected, writable, record)).toThrow("persisted runtime record");
const changedRoots = await createAcpxRecoveryBinding({ ...fixture.input,
providerPolicy: { readOnly: true, protectedPaths: ["/another-protected-root"] } });
expect(changedRoots.profileSessionKey).not.toBe(readonly.profileSessionKey);
await expect(createAcpxRecoveryBinding({ ...fixture.input,
providerPolicy: { readOnly: "yes" as never } })).rejects.toThrow("valid task execution policy");
});
it("uses collision-resistant roots and policy-bound provider keys", async () => {
const fixture = await recoveryFixture();
const otherSession = await createAcpxRecoveryBinding({
@@ -43,8 +43,10 @@ export async function createAcpxRecoveryBinding(input: {
profile: QualifiedAcpxProfile;
requestedModel: string;
permissionMode: NativeAcpxPermissionMode;
providerPolicy?: { readOnly: boolean; readRoots?: readonly string[]; protectedPaths?: readonly string[] };
}): Promise<AcpxRecoveryBinding> {
validateIdentity(input.normalizedSessionId, "normalized session");
if (input.providerPolicy !== undefined && typeof input.providerPolicy.readOnly !== "boolean") throw new Error("ACPX recovery requires a valid task execution policy");
if (input.requestedModel !== input.profile.qualificationModel) {
throw new Error("ACPX recovery requested an unqualified model");
}
@@ -72,6 +74,11 @@ export async function createAcpxRecoveryBinding(input: {
qualificationModel: input.profile.qualificationModel,
reportedModelId: input.profile.reportedModelId,
permissionPolicy: input.profile.permissionPolicy,
...(input.providerPolicy === undefined ? {} : { executionPolicy: {
readOnly: input.providerPolicy.readOnly,
readRoots: input.providerPolicy.readRoots ?? [],
protectedPaths: input.providerPolicy.protectedPaths ?? [],
} }),
}),
);
const profileSessionKey = digest(
@@ -1,5 +1,5 @@
import { dirname, join } from "node:path";
import { assertAcpxProfileWorkspace, verifyAcpxProfileInstallation } from "./profile-installation.js";
import { assertAcpxProfileWorkspace, classifyAcpxProfileError, verifyAcpxProfileInstallation } from "./profile-installation.js";
import { createAcpxRuntimeSkillLease } from "./runtime-skill-lease.js";
import { claudeNativeSkillPrompt } from "./native-skill-prompt.js";
import { nativeMcpLaunchBinding } from "../native-mcp.js";
@@ -328,6 +328,8 @@ export class AcpxRuntimeHost {
profile,
requestedModel: options.model,
permissionMode: options.permissionMode,
...(["cursor", "copilot", "pi"].includes(options.agent) && options.providerPolicy !== undefined
? { providerPolicy: options.providerPolicy } : {}),
}),
dependencies.retainAdmissionCleanup,
);
@@ -605,6 +607,7 @@ export class AcpxRuntimeHost {
: [],
});
} catch (error) {
error = classifyAcpxProfileError(options.agent, error) ?? error;
const cleanup = cleanupRuntimeResources(
runtime,
toolBridge,
@@ -55,9 +55,9 @@ describe("qualified ACPX usage", () => {
expect(qualifiedAcpxUsageBreakdown("claude", null)).toBeNull();
});
it("does not infer accounting semantics for Pi or an uninitialized agent", () => {
it("does not double count reasoning for the exact pinned Pi model and preserves unknown profiles", () => {
const usage = { inputTokens: 12, outputTokens: 30, thoughtTokens: 20 };
expect(qualifiedAcpxUsageBreakdown("pi", usage)).toEqual(usage);
expect(qualifiedAcpxUsageBreakdown("pi", usage)).toEqual({ ...usage, thoughtTokens: 0 });
expect(qualifiedAcpxUsageBreakdown(null, usage)).toEqual(usage);
});
});
@@ -157,4 +157,16 @@ describe("Pi prompt accounting authority", () => {
expect(acpxUsageEstimateNotice(usage, "turn:pricing")).toBeNull();
}
});
it("preserves compaction receipt provenance without treating estimates as billing", () => {
const usage = persistedAcpxTurnUsage({}, { ...after, requestTokenUsage: {
message: { input_tokens: 12, paperclip_pi: {
provenance: "assistant_message_and_compaction_receipts", cost_usd: 0.25,
} },
} }, "turn", "pi")!;
expect(usage.cost).toBeUndefined();
expect(usage.usageProvenance).toBe("pi_assistant_message_and_compaction_receipts");
expect(acpxUsageEstimateNotice(usage, "turn:pricing")?.payload).toMatchObject({
details: expect.arrayContaining([{ name: "Usage source", value: "Assistant message and compaction receipts for this prompt" }]),
});
});
});
@@ -14,9 +14,10 @@ export function qualifiedAcpxUsageBreakdown(
): unknown {
if (value === null || value === undefined) return value;
const breakdown = record(value);
if (agent !== "claude" && agent !== "codex") return breakdown;
if (agent !== "claude" && agent !== "codex" && agent !== "pi") return breakdown;
// Claude SDK aggregate output and Codex ACP toPromptUsage.outputTokens both
// INCLUDE reasoning. PRP folds thought into output, so its additive component
// INCLUDE reasoning. The exact pinned Pi OpenRouter model uses
// openai-completions, where completion_tokens also includes reasoning. PRP folds thought into output, so its additive component
// is zero here, not the provider's diagnostic reasoning-token subset.
return {
...breakdown,
@@ -50,7 +51,8 @@ export function persistedAcpxTurnUsage(
if (added.length !== 1) return null;
const usage = record(receipts[added[0]!]);
const piReceipt = agent === "pi" ? record(usage.paperclip_pi) : {};
const piReceiptVerified = piReceipt.provenance === "assistant_message_receipts";
const piReceiptVerified = piReceipt.provenance === "assistant_message_receipts"
|| piReceipt.provenance === "assistant_message_and_compaction_receipts";
const estimate = piReceiptVerified && typeof piReceipt.cost_usd === "number"
&& Number.isFinite(piReceipt.cost_usd) && piReceipt.cost_usd >= 0 ? piReceipt.cost_usd : undefined;
return {
@@ -60,7 +62,7 @@ export function persistedAcpxTurnUsage(
// Pi calculates cost from catalog prices, not billing receipts. Never feed
// this estimate into the authoritative/cumulative provider spend channel.
cost: agent === "pi" ? undefined : current.usageCost,
...(piReceiptVerified ? { usageProvenance: "pi_assistant_message_receipts" } : {}),
...(piReceiptVerified ? { usageProvenance: `pi_${piReceipt.provenance}` } : {}),
...(estimate === undefined ? {} : { pricingEstimateUsd: estimate }),
breakdown: {
inputTokens: usage.input_tokens,
@@ -75,7 +77,8 @@ export function persistedAcpxTurnUsage(
/** Preserve the estimate for inspection while keeping billing authority separate. */
export function acpxUsageEstimateNotice(usage: Record<string, unknown>, itemId: string): CanonicalProviderEvent | null {
if (usage.usageProvenance !== "pi_assistant_message_receipts"
if ((usage.usageProvenance !== "pi_assistant_message_receipts"
&& usage.usageProvenance !== "pi_assistant_message_and_compaction_receipts")
|| typeof usage.pricingEstimateUsd !== "number" || !Number.isFinite(usage.pricingEstimateUsd)
|| usage.pricingEstimateUsd < 0) return null;
return {
@@ -86,7 +89,8 @@ export function acpxUsageEstimateNotice(usage: Record<string, unknown>, itemId:
recoverable: true, userActionable: false,
summary: `Pi estimates this turn at $${usage.pricingEstimateUsd.toFixed(6)} from its model prices. Billing cost is unverified.`,
details: [{ name: "Cost source", value: "Pi model catalog pricing estimate" },
{ name: "Usage source", value: "Assistant message receipts for this prompt" },
{ name: "Usage source", value: usage.usageProvenance === "pi_assistant_message_and_compaction_receipts"
? "Assistant message and compaction receipts for this prompt" : "Assistant message receipts for this prompt" },
{ name: "Estimated USD", value: String(usage.pricingEstimateUsd) }],
},
};
@@ -1,4 +1,7 @@
import { realpathSync } from "node:fs";
import { mkdtempSync, realpathSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterAll } from "vitest";
import { fileURLToPath } from "node:url";
import {
@@ -45,7 +48,10 @@ import {
type CodexTraceInterpretation,
} from "./app-server-transport.js";
export const WORKSPACE = realpathSync.native(process.cwd());
// Tests run in managed worktrees beneath sensitive Codex state. Use a real
// disposable workspace instead of weakening production host-path admission.
export const WORKSPACE = realpathSync.native(mkdtempSync(join(tmpdir(), "paperclip-codex-test-workspace-")));
afterAll(() => rmSync(WORKSPACE, { recursive: true, force: true }));
export class TestQueue<T> implements AsyncIterable<T> {
values: T[] = [];
@@ -25,6 +25,17 @@ import { DurableCapabilityLiveSessionStore } from "./durable-live-session-store.
import { defaultCapabilityRunnerdBinary } from "./runnerd-codex-transport.js";
import { captureTurnRejection } from "../../test/capture-turn-rejection.js";
it.each(["pi", "cursor", "copilot"] as const)("requires separately bound evaluation opt-in for %s", async (acpxAgent) => {
const service = new CapabilityLiveSessionService();
await expect(service.create({ provider: "acpx", acpxAgent, requestedModel: "explicit-model" }))
.rejects.toThrow("explicit evaluation opt-in");
const mismatched = new CapabilityLiveSessionService({ transportOptions: {
acpxCandidateProfile: acpxAgent === "pi" ? "cursor" : "pi",
} });
await expect(mismatched.create({ provider: "acpx", acpxAgent, requestedModel: "explicit-model" }))
.rejects.toThrow("explicit evaluation opt-in");
});
class AsyncNotifications implements AsyncIterable<CodexRpcNotification> {
#values: CodexRpcNotification[] = [];
#waiters: Array<(value: IteratorResult<CodexRpcNotification>) => void> = [];
@@ -628,7 +628,7 @@ export function assertCapabilityLiveSessionSnapshot(
}
if (provider === "acpx") {
const agent = config.acpxAgent;
if (agent !== "pi" && agent !== "claude" && agent !== "codex") {
if (agent !== "pi" && agent !== "claude" && agent !== "codex" && agent !== "cursor" && agent !== "copilot") {
throw new Error("capability_live_checkpoint_corrupt: invalid config.acpxAgent");
}
const expected = resolveQualifiedAcpxProfile(agent, text(config.requestedModel));
@@ -861,8 +861,10 @@ export class CapabilityLiveSessionService {
}
async create(input: CreateCapabilityLiveSessionInput = {}): Promise<CapabilityLiveSession> {
if (input.provider === "acpx" && input.acpxAgent === "pi") {
throw new Error("The Pi ACPX profile is not available");
if (input.provider === "acpx" && input.acpxAgent !== undefined
&& ["pi", "cursor", "copilot"].includes(input.acpxAgent)
&& this.#transportOptions.acpxCandidateProfile !== input.acpxAgent) {
throw new Error("The candidate ACPX profile requires explicit evaluation opt-in");
}
if (input.provider === "claude_managed" && !input.managedProfile) {
throw new Error("Claude Managed live sessions require a qualified managed profile");
@@ -1647,6 +1647,10 @@ it("uses file-backed AWS workload identity without forwarding access keys or Pap
});
it.each([
{ agent: "cursor" as const, allowed: ["CURSOR_API_KEY", "CURSOR_AUTH_TOKEN"],
denied: ["COPILOT_GITHUB_TOKEN", "GH_TOKEN", "GITHUB_TOKEN", "OPENROUTER_API_KEY", "ANTHROPIC_API_KEY", "OPENAI_API_KEY"] },
{ agent: "copilot" as const, allowed: ["COPILOT_GITHUB_TOKEN"],
denied: ["CURSOR_API_KEY", "CURSOR_AUTH_TOKEN", "GH_TOKEN", "GITHUB_TOKEN", "OPENROUTER_API_KEY", "ANTHROPIC_API_KEY", "OPENAI_API_KEY"] },
{
agent: "pi" as const,
allowed: ["OPENROUTER_API_KEY"],
@@ -1682,6 +1686,8 @@ it.each([
"passes only $agent ACPX credentials and the durable runtime binding",
({ agent, allowed, denied }) => {
const credentialEnvironment: Record<string, string> = {
CURSOR_API_KEY: "cursor-key-canary", CURSOR_AUTH_TOKEN: "cursor-token-canary",
COPILOT_GITHUB_TOKEN: "copilot-canary", GH_TOKEN: "ambient-gh-canary", GITHUB_TOKEN: "ambient-github-canary",
OPENROUTER_API_KEY: "openrouter-canary",
ANTHROPIC_API_KEY: "anthropic-canary",
CLAUDE_CODE_OAUTH_TOKEN: "claude-oauth-canary",
@@ -1078,6 +1078,8 @@ export interface CapabilityRunnerdCodexTransportOptions {
provider?: "codex" | "opencode" | "claude_managed" | "aws_agentcore" | "acpx";
opencodePermissionMode?: NativeOpenCodePermissionMode;
acpxAgent?: QualifiedAcpxAgent;
/** Explicit evaluation-only candidate selection, never derived from persisted session input. */
acpxCandidateProfile?: "pi" | "cursor" | "copilot";
acpxPermissionMode?: NativeAcpxPermissionMode;
acpxPermissionModePinned?: boolean;
acpxSidecarPath?: string;
@@ -3100,8 +3102,7 @@ export function createCapabilityRunnerdProviderEnvironment(input: {
const providerPackageAuthority = acpxProviderPackageAuthority(sidecarPath);
// This is the trusted runner/sidecar boundary. The provider sandbox still
// uses createSanitizedAcpxSpawnInput and does not inherit gateway tokens.
const assignedGateway = input.options.acpxAgent === "pi"
? null : nativeMcpLaunchBinding(input.options.environment ?? {});
const assignedGateway = nativeMcpLaunchBinding(input.options.environment ?? {});
return {
...(assignedGateway ? {
PAPERCLIP_NATIVE_MCP_TOKEN: assignedGateway.token,
@@ -3351,8 +3352,10 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
if (options.adoptExistingRunner && !options.stateDirectory?.trim()) {
throw new Error("native_adopted_runner_state_directory_required");
}
if (options.provider === "acpx" && options.acpxAgent === "pi") {
throw new Error("The Pi ACPX profile is not available");
if (options.provider === "acpx" && options.acpxAgent !== undefined
&& ["pi", "cursor", "copilot"].includes(options.acpxAgent)
&& options.acpxCandidateProfile !== options.acpxAgent) {
throw new Error("The candidate ACPX profile requires explicit evaluation opt-in");
}
this.#failureSignal = new Promise<never>((_resolve, reject) => {
this.#rejectFailureSignal = reject;
@@ -3466,6 +3469,7 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
? params.expectedTurnId
: this.#turnId;
if (!text.trim()) throw new Error("turn/steer requires a message");
if (params.mode !== undefined && params.mode !== "steer" && params.mode !== "follow_up") throw new Error("turn/steer mode is invalid");
if (expectedTurnId !== this.#turnId)
throw new Error("turn/steer named a stale turn");
const correlationId =
@@ -3478,6 +3482,7 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
text,
turnId: this.#durableTurnId,
providerTurnId: expectedTurnId,
...(params.mode === "follow_up" ? { mode: "follow_up" } : {}),
...(correlationId ? { correlationId } : {}),
},
correlationId,
@@ -4524,6 +4529,7 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
runId: identity.runId,
cwd: String(params.cwd ?? tmpdir()),
instructions: baseInstructions,
providerPolicy: { readOnly: params.permissions === "paperclip-runner-workspace-read-only" },
permissionMode: resolveRunnerdAcpxPermissionMode(
this.options.acpxPermissionMode,
),
@@ -6185,7 +6191,9 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
if (
descriptor.agent === "pi" ||
descriptor.agent === "claude" ||
descriptor.agent === "codex"
descriptor.agent === "codex" ||
descriptor.agent === "cursor" ||
descriptor.agent === "copilot"
)
this.#evidence.acpxAgent = descriptor.agent;
if (typeof descriptor.agentServerVersion === "string")