fix: scope Pi remote fixture setup approval to its published read

This commit is contained in:
Dotta committed 2026-10-04 20:50:07 -05:00
1 parent 585b43d3ab
commit 9cb5ac2c41
12 files changed
+308 -23

No files matched your search

@@ -43,6 +43,46 @@ Check remaining credit and that BYOK usage stays zero before and after each
attempt. API key deltas are provisional billing observations. Pi model-catalog
prices are estimates. Unpriced usage must stay unpriced in the ledger and UI.
## Accepted corrections — 2026-10-04
The operator accepts the scoped Product E2E fixture corrections and authorizes
necessary corrected attempts within the existing $100 total campaign limit.
The dedicated Pi key retains its $5 lifetime cap. Automatic paid retries,
fallback keys, new models, GitHub Actions edits and lockfile commits remain
excluded. The previously rejected SDK-corrected attempt remains undispatched
and preserved; the new authorization applies to a separate explicit phase.
The canonical-closure fixture correction is committed at `585b43d3a`. All 14
native Linux fault tests pass, including ownership/pidfd and malformed metadata
calibration. That head has 53 successful CI checks, two expected skips and no
unresolved root review findings. These free checks do not qualify provider death.
A fresh owned Linux controller passes normal public installation, normal Pi
setup and admission in 7.207 seconds. Installed startup passes health/UI with
zero companies and cleanup. Complete audits pass 101,226 consumer files and
16,356 plugin files. All 13 Daytona browser cases collect, and the pinned SDK
constructor passes without credentials or provider calls. The two expired
controllers remain absent. The rejected 30 GiB allocation created no sandbox;
the prepared controller uses the account's 10 GiB limit and a finite lifetime.
The restrictive fixture correction approves only the exact published setup-file
read through the public operator API after observer arming. It retains the
completed read, resolution and unchanged file hash. All tested write permissions,
Stop/steering/denial assertions and retirement/no-effect proofs remain required.
Controls definitions advance to 7 and native definitions to 5. Prior attempt
fingerprints and grades remain unchanged. All 110 targeted native Linux tests
pass, including positive Stop/steering settlement and negative setup approvals.
The local pure oracle tests pass; the local ownership flows still fail their
unchanged file-watch completeness gate. Local Product typecheck is blocked by
stale dependency declarations in the existing linked build. Fresh CI must verify
the committed head. Live proof on the new definitions is still required.
The dedicated key's latest observed usage is $0.2144 with $4.7856 remaining and
zero BYOK usage. These are provisional key observations. The browser account
session has expired, so paid dispatch awaits a fresh all-provider BYOK settings
check after sign-in. Budget approval is already sufficient. No paid provider
attempt occurs during this resumed preparation.
## Qualification snapshot — 2026-10-03
The current ledger has 21 passes, ten failed cells, two unattempted cells and
@@ -146,8 +186,9 @@ Remaining corrections and evidence:
All 13 unchanged Daytona cases collect, and the SDK constructor passes without
credentials or provider calls. Automatic approval review rejects a further
agent-files attempt because the proposed retry guard exceeds the bounded
correction authorization. That attempt is held pending human approval; no
key is read or remote work dispatched by the rejected action.
correction authorization. That rejected attempt stays held and unchanged; no
key was read or remote work dispatched by that action. The October 4 human
authorization now permits a separate corrected attempt after fresh guards.
- Daytona provider-death reaches a real unanswered native question, then fails
its exact-child fault observer receipt. Canonical independent retirement and
cleanup pass; provider-loss and stale-answer assertions are not reached.
@@ -162,10 +203,10 @@ Remaining corrections and evidence:
The image's closure is valid; the fixture's hash representation needs repair.
The original observer error masks its
specific internal cause; this diagnosis does not regrade that failure.
An unapplied private correction passes 14 free Linux tests: nine exact-child
The accepted canonical-closure correction passes 14 free Linux tests: nine exact-child
ownership/pidfd calibrations and five actual-metadata, formatting and negative
pin regressions. A fixture correction remains held for workflow-scope
clarification; those free tests do not qualify the failed paid case.
pin regressions. The operator accepts the scoped fixture correction on October 4; those free
tests do not qualify the failed paid case.
- The unrestricted Daytona file-edit first attempt edits and publishes the
correct file and passes cleanup, but performs extra native shell executions.
It fails the unchanged exactly-one validation-execution gate. Both platform
@@ -188,9 +229,9 @@ Remaining corrections and evidence:
the store. Installed graph identities and all original evidence are preserved.
- A free counterexample with the unchanged controls oracle proves that an
approved bootstrap read followed by the pending write creates two permissions
and is rejected. An external approval alone cannot repair the gate. Scoped
Product E2E fixture/oracle changes require clarifying the user prohibition on
workflow edits; GitHub Actions and production permissions need no change.
and is rejected. An external approval alone cannot repair the gate. The operator
now accepts scoped Product E2E fixture/oracle corrections. GitHub Actions and
production permissions need no change.
- The previously unexecuted workspace-A group passes 7,176 tests and fails one
save-navigation assertion. The API update is already observed, but navigation
follows asynchronous query invalidation. Waiting for the same form-close
+14
View File
@@ -421,6 +421,20 @@ identity mapping. Pi does not emit Cursor/Copilot diagnostic notices; those
notices are never synthesized. Earlier native reads can provide orientation;
other native operations cannot substitute for the observed write.
On Daytona, the operator first publishes the setup instruction file after the
owned observer is armed. If Pi delegates that native read, the fixture approves
only its exact request through the public API with `accept` (allow once). The
read must name the published random setup file, complete successfully in the
same native run/turn/session/source, and leave the file hash unchanged. The
fixture retains the request, resolution, completed read and both observer
snapshots. Only the two hash-bound setup permission records are excluded from
the write-permission count. All native read rows remain in the oracle. Another
permission, edit, shell command, foreign path or incomplete read cannot receive
this exemption. The tested write remains unanswered until Stop or browser Deny.
The production permission policy and all write/retirement assertions stay in
place. `pi-controls` version 7 and `pi-native` version 5 record this correction;
older attempt fingerprints and grades remain historical evidence.
Native tool arguments can arrive after the start event. An earlier null target
is allowed only until the same execution first supplies the exact expected
path. Missing targets, conflicting paths, another execution's path, or a later
+4
View File
@@ -1480,6 +1480,10 @@ file persistence uses two fresh task runs and independent byte checks; restricti
denial uses one run and requires both a failed tool receipt and no file effect.
`human-permission-denial` additionally requires the exact browser Decline response,
delivered native denial and independent file/process observation through retirement.
The restrictive Daytona fixtures approve only the exact operator-published
setup-file read after observer arming. They retain its public resolution and
completed native read before testing the separate write permission. This setup
approval does not change production policy or answer the tested write.
See [the fixture contract](FIXTURES.md#pi-native-boundaries) for the exact oracles
and the limits of reconnect evidence.
+3 -2
View File
@@ -1090,8 +1090,9 @@ export const runnerSuites: readonly RunnerSuiteFixture[] = [
description: "Pending native-write Stop and browser-originated same-turn steering, with exact control receipts and independent retirement/no-effect evidence.",
groups: ["native"], profiles: extendedHarnessProfiles.filter(profile => profile.qualificationCandidate === "pi"),
environments: runnerEnvironments, tasks: piControlTasks, expectedMatrixSize: 4,
definitionMetadata: { version: 6, qualification: "pending", scheduling: "explicit-only", profileVersion: QUALIFIED_ACPX_PROFILES.pi.agentProfileVersion,
definitionMetadata: { version: 7, qualification: "pending", scheduling: "explicit-only", profileVersion: QUALIFIED_ACPX_PROFILES.pi.agentProfileVersion,
nativeArguments: "streamed-until-exact-target",
remoteBootstrapApproval: "exact-published-native-read-public-accept-once-v1",
controlPlaneSettlement: "required-scoped-result-and-terminal-after-runner",
steeringComment: "exact-browser-submitted-markdown",
steeringDispatch: "require-public-api-acceptance",
@@ -1112,7 +1113,7 @@ export const runnerSuites: readonly RunnerSuiteFixture[] = [
groups: ["native"], profiles: extendedHarnessProfiles.filter(profile => profile.qualificationCandidate === "pi"),
environments: runnerEnvironments, tasks: piNativeTasks, expectedMatrixSize: 10,
excludedExecutionIds: ["pi-native.runner-acpx-pi.daytona.restrictive-denial", "pi-native.runner-acpx-pi.local.native-pending-provider-death"],
definitionMetadata: { version: 4, qualification: "pending", scheduling: "explicit-only", profileVersion: QUALIFIED_ACPX_PROFILES.pi.agentProfileVersion, providerDeath: "daytona-exact-pi-child-pidfd-production-expiry", remoteDenyAll: "unsupported-native-bootstrap-read-is-denied", remoteEvidence: "owned-lease-sealed-observer", pendingControllerRestart: "same-live-native-request" },
definitionMetadata: { version: 5, qualification: "pending", scheduling: "explicit-only", profileVersion: QUALIFIED_ACPX_PROFILES.pi.agentProfileVersion, providerDeath: "daytona-exact-pi-child-pidfd-production-expiry", remoteBootstrapApproval: "exact-published-native-read-public-accept-once-v1", remoteDenyAll: "unsupported-native-bootstrap-read-is-denied", remoteEvidence: "owned-lease-sealed-observer", pendingControllerRestart: "same-live-native-request" },
},
{
id: "native-active-stop", label: "Stop an unanswered native permission", manualOnly: true,
@@ -0,0 +1,76 @@
import { createHash } from "node:crypto";
import { describe, expect, it } from "vitest";
import { canonicalJson } from "../../packages/shared/src/portability-hash.js";
import { approvePiBootstrapRead, withoutApprovedPiBootstrapRequests, type PiBootstrapApproval } from "./pi-bootstrap-permission.js";
import { observePiControlPending, readPiStopSettlement, readPiSteeringSettlement } from "./pi-controls-evidence.js";
import { piCancellationId, piControlCaller, piControlFixture } from "./pi-controls-test-fixture.js";
const actionFile = `.paperclip-eval-action-${"a".repeat(36)}.txt`;
const digest = (value: unknown) => `sha256:${createHash("sha256").update(canonicalJson(value)).digest("hex")}`;
function fixture() {
const f = piControlFixture(), write = structuredClone(f.events);
f.events.length = 0;
const read = { schema: "paperclip.tool.execution.v1", transport: "builtin", executionId: "setup-tool", name: "read", operation: "read", target: actionFile, status: "running" };
const request = { ...f.request, requestId: "setup-request", itemId: "setup-item", details: { toolCallId: "setup-tool" }, choices: [{ key: "accept", label: "Allow once" }, { key: "decline", label: "Deny" }] };
f.append("tool.execution.started", read); f.append("runtime_request.created", { request });
const finish = () => {
f.append("runtime_request.resolved", { requestId: request.requestId, turnId: "turn", action: "accept" });
f.append("tool.execution.completed", { ...read, status: "completed" });
for (const row of write) f.append(row.eventType, row.payload.prpEvent.payload);
};
const proof = (): PiBootstrapApproval => ({ schema: "paperclip.e2e.pi-bootstrap-read-approval.v1", companyId: f.scope.companyId, issueId: f.scope.issueId, runId: f.scope.runId, actionFile, publishedSha256: `sha256:${"b".repeat(64)}`,
requestId: "setup-request", toolCallId: "setup-tool", turnId: "turn", normalizedSessionId: "session", sourceInstanceId: "source",
createdRowSha256: digest(f.events[1]), resolvedRowSha256: digest(f.events[2]), completedRowSha256: digest(f.events[3]) });
return { ...f, read, finish, proof };
}
describe("Pi operator setup read", () => {
it("approves only the published owned file through the exact public request", async () => {
const f = fixture(), saved = new Map<string, unknown>(); let posts = 0;
const result = await approvePiBootstrapRead({
api: { post: async (path: string, data: unknown) => { expect(path).toBe("/api/heartbeat-runs/run/runtime-requests/setup-request/resolve"); expect(data).toEqual({ turnId: "turn", requestKind: "permission_approval", resolution: { action: "accept" } }); posts++; f.finish(); } } as any,
fixture: { actionFile, binding: { companyId: "company", runId: "run" }, snapshot: async () => ({ complete: true, setup: { path: actionFile, published: true, sha256: `sha256:${"b".repeat(64)}` } }) } as any,
companyId: "company", issueId: "issue", runId: "run", deadlineAt: Date.now() + 2000, load: async () => f.state(), evidence: async (name, value) => { saved.set(name, value); },
});
expect(posts).toBe(1); expect(result).toEqual(f.proof()); expect(saved.has("pi-bootstrap-read-before-approval.json")).toBe(true);
const pending = observePiControlPending({ ...f.state(), scope: f.scope, bootstrapApproval: result });
expect(pending.requestId).toBe("request"); expect(pending.bootstrapApproval).toEqual(result);
});
it.each(["unpublished", "foreign-binding", "other-file", "shell"])("does not approve %s setup", async failure => {
const f = fixture(); let posts = 0;
if (failure === "other-file") f.events[0]!.payload.prpEvent.payload.target = "private.txt";
if (failure === "shell") { f.events[0]!.payload.prpEvent.payload.name = "bash"; f.events[0]!.payload.prpEvent.payload.operation = "execute"; }
await expect(approvePiBootstrapRead({ api: { post: async () => { posts++; } } as any,
fixture: { actionFile, binding: { companyId: failure === "foreign-binding" ? "other" : "company", runId: "run" }, snapshot: async () => ({ complete: true, setup: { path: actionFile, published: failure !== "unpublished", sha256: `sha256:${"b".repeat(64)}` } }) } as any,
companyId: "company", issueId: "issue", runId: "run", deadlineAt: Date.now() + 2000, load: async () => f.state(), evidence: async () => {},
})).rejects.toThrow("Pi bootstrap"); expect(posts).toBe(0);
});
it.each(["stop", "steering"])("retains the original %s assertions after a proven setup read", mode => {
const f = fixture(); f.finish(); const proof = f.proof();
expect(() => observePiControlPending({ ...f.state(), scope: f.scope })).toThrow("one native permission");
const pending = observePiControlPending({ ...f.state(), scope: f.scope, bootstrapApproval: proof });
if (mode === "stop") { f.cancel(); expect(readPiStopSettlement({ ...f.state(), pending, caller: piControlCaller, cancellationRequestId: piCancellationId, dispatchMonotonicNs: (BigInt(pending.observedMonotonicNs) + 1n).toString() }).normalCompletionAccepted).toBe(false); }
else { f.steer();
// The wrapper's setup finish is separate from the original write finish.
f.append("runtime_request.resolved", { requestId: "request", turnId: "turn", action: "decline" });
f.append("tool.execution.completed", { ...f.tool, status: "failed", output: "Pi operation was denied or cancelled" });
f.append("turn.completed", { status: "completed", error: null }); f.run.status = "succeeded"; f.issue.status = "done";
f.append("run.result.accepted", { result: { schema: "paperclip.run_result.v1", reportedWorkDisposition: "done", summary: f.marker } }, { sourceKind: "control_plane", sourceInstanceId: "source:control", sourceSeq: 1, sourceEventId: "source:control:run:1" });
f.append("run.terminal", { schema: "paperclip.prp.terminal.v1", runTerminalState: "succeeded", turnTerminalState: "completed", reportedWorkDisposition: "done" }, { sourceKind: "control_plane", sourceInstanceId: "source:control", sourceSeq: 2, sourceEventId: "source:control:run:2" });
expect(readPiSteeringSettlement({ ...f.state(), pending, commentId: f.commentId, queueId: f.queueId, marker: f.marker, finalMessage: f.marker }).marker).toBe(f.marker);
}
});
it.each(["changed-receipt", "wrong-path", "failed-read", "extra-permission", "reordered", "allow-always"])("rejects %s before excluding setup records", failure => {
const f = fixture(); f.finish(); const proof = f.proof();
if (failure === "changed-receipt") proof.resolvedRowSha256 = `sha256:${"0".repeat(64)}`;
if (failure === "wrong-path") f.events[0]!.payload.prpEvent.payload.target = "other.txt";
if (failure === "failed-read") f.events[3]!.payload.prpEvent.payload.status = "failed";
if (failure === "extra-permission") f.append("runtime_request.created", { request: { ...f.request, requestId: "extra" } });
if (failure === "reordered") f.events[3]!.seq = 1;
if (failure === "allow-always") f.events[2]!.payload.prpEvent.payload.action = "accept_always";
expect(() => observePiControlPending({ ...f.state(), scope: f.scope, bootstrapApproval: proof })).toThrow();
});
it("does not discard any native lifecycle row", () => { const f = fixture(); f.finish(); const rows = withoutApprovedPiBootstrapRequests(f.events, f.proof()); expect(rows).toHaveLength(f.events.length - 2); expect(rows.filter((row: any) => row.eventType.startsWith("tool.execution."))).toHaveLength(3); });
});
+134
View File
@@ -0,0 +1,134 @@
import { createHash } from "node:crypto";
import { canonicalJson } from "../../packages/shared/src/portability-hash.js";
import { pollUntil, type RunnerApi } from "./api.js";
import { bootstrapReadExecutionId } from "./native-bootstrap-read-proof.js";
import { isValidNativePrpEnvelope } from "./native-event-envelope.js";
import { piPermissionRequests } from "./pi-native-evidence.js";
import type { RemoteNativeFixture, RemoteNativeSnapshot } from "./remote-native-fixtures.js";
type Row = Record<string, any>;
const hash = (row: unknown) => `sha256:${createHash("sha256").update(canonicalJson(row)).digest("hex")}`;
const requireProof = (ok: unknown) => { if (!ok) throw new Error("Pi bootstrap: exact published read approval is unproven"); };
export interface PiBootstrapApproval {
schema: "paperclip.e2e.pi-bootstrap-read-approval.v1";
companyId: string; issueId: string; runId: string; actionFile: string; publishedSha256: string;
requestId: string; toolCallId: string; turnId: string; normalizedSessionId: string; sourceInstanceId: string;
createdRowSha256: string; resolvedRowSha256: string; completedRowSha256: string;
}
export interface PiBootstrapState { run: Row; issue: Row; events: readonly unknown[] }
function readLifecycle(events: readonly unknown[], input: Pick<PiBootstrapApproval, "companyId" | "runId" | "actionFile" | "requestId">) {
requireProof(events.length > 0 && events.length <= 20_000 && /^\.paperclip-eval-action-[a-f0-9]{36}\.txt$/u.test(input.actionFile));
const seqs = new Set<number>(), sourceIds = new Set<string>(), lastSource = new Map<string, number>();
for (const row of events as readonly Row[]) {
const e = row.payload?.prpEvent; if (e === undefined) continue;
requireProof(e && isValidNativePrpEnvelope(e, row.protocolSchemaVersion) && row.companyId === input.companyId
&& row.runId === input.runId && e.runId === input.runId && e.eventType === row.eventType
&& ["runner", "control_plane"].includes(e.sourceKind) && typeof e.sourceInstanceId === "string" && e.sourceInstanceId.length > 0
&& Number.isSafeInteger(row.seq) && row.seq > 0 && !seqs.has(row.seq)
&& Number.isSafeInteger(e.sourceSeq) && e.sourceSeq > (lastSource.get(e.sourceInstanceId) ?? 0)
&& e.sourceEventId === `${e.sourceInstanceId}:${input.runId}:${e.sourceSeq}` && !sourceIds.has(e.sourceEventId)
&& Number.isFinite(Date.parse(e.emittedAt)));
seqs.add(row.seq); sourceIds.add(e.sourceEventId); lastSource.set(e.sourceInstanceId, e.sourceSeq);
}
const created = piPermissionRequests(events, input.runId).filter(x => x.request.requestId === input.requestId);
requireProof(created.length === 1); const card = created[0]!;
requireProof(isValidNativePrpEnvelope(card.event, card.row.protocolSchemaVersion) && card.row.companyId === input.companyId && card.request.requestKind === "permission_approval"
&& typeof card.event.turnId === "string" && card.event.turnId.length > 0
&& typeof card.event.normalizedSessionId === "string" && card.event.normalizedSessionId.length > 0
&& card.request.choices.filter((choice: Row) => choice.key === "accept").length === 1);
const executionId = bootstrapReadExecutionId(card.request.details.toolCallId);
const rows = (events as readonly Row[]).filter(row => row.payload?.prpEvent?.payload?.executionId === executionId);
requireProof(rows.length > 0 && rows.length <= 2048);
let known = false, lastSeq = 0;
for (const row of rows) {
const event = row.payload.prpEvent, p = event.payload;
requireProof(isValidNativePrpEnvelope(event, row.protocolSchemaVersion) && row.companyId === input.companyId && row.runId === input.runId && event.runId === input.runId
&& row.protocolSchemaVersion === 1 && event.schema === "paperclip.prp.event.v1" && event.schemaVersion === 1
&& event.sourceKind === "runner" && event.turnId === card.event.turnId && event.normalizedSessionId === card.event.normalizedSessionId
&& event.sourceInstanceId === card.event.sourceInstanceId && event.eventType === row.eventType
&& Number.isSafeInteger(row.seq) && row.seq > lastSeq
&& p.schema === "paperclip.tool.execution.v1" && p.transport === "builtin" && p.name === "read" && p.operation === "read"
&& ["tool.execution.started", "tool.execution.progressed", "tool.execution.completed"].includes(row.eventType)
&& (p.target === input.actionFile || (!known && p.target === null && row.eventType !== "tool.execution.completed")));
known ||= p.target === input.actionFile; lastSeq = row.seq;
}
requireProof(known && rows[0]!.eventType === "tool.execution.started"
&& rows.filter(row => row.eventType === "tool.execution.started").length === 1
&& rows.filter(row => row.eventType === "tool.execution.completed").length <= 1
&& rows.every((row, index) => row.eventType === "tool.execution.completed"
? index === rows.length - 1 && row.payload.prpEvent.payload.status === "completed"
: row.payload.prpEvent.payload.status === "running"));
return { card, rows, executionId };
}
/** Remove only the two hash-bound setup permission records. All native read
* lifecycle rows and every tested write/permission remain in the oracle. */
export function withoutApprovedPiBootstrapRequests(events: readonly unknown[], proof?: PiBootstrapApproval): unknown[] {
if (!proof) return [...events];
requireProof(proof.schema === "paperclip.e2e.pi-bootstrap-read-approval.v1" && /^sha256:[a-f0-9]{64}$/u.test(proof.publishedSha256));
const { card, rows } = readLifecycle(events, proof);
requireProof(card.request.details.toolCallId === proof.toolCallId && card.event.turnId === proof.turnId
&& card.event.normalizedSessionId === proof.normalizedSessionId && card.event.sourceInstanceId === proof.sourceInstanceId
&& hash(card.row) === proof.createdRowSha256);
const closures = (events as readonly Row[]).filter(row => ["runtime_request.resolved", "runtime_request.cancelled", "runtime_request.expired"].includes(row.eventType)
&& row.payload?.prpEvent?.payload?.requestId === proof.requestId);
requireProof(closures.length === 1); const closed = closures[0]!, e = closed.payload.prpEvent;
requireProof(isValidNativePrpEnvelope(e, closed.protocolSchemaVersion) && closed.companyId === proof.companyId && closed.runId === proof.runId && closed.protocolSchemaVersion === 1
&& e.schema === "paperclip.prp.event.v1" && e.schemaVersion === 1 && e.sourceKind === "runner" && e.runId === proof.runId
&& e.eventType === "runtime_request.resolved" && closed.eventType === e.eventType && e.sourceInstanceId === proof.sourceInstanceId
&& e.turnId === proof.turnId && e.normalizedSessionId === proof.normalizedSessionId && e.payload.turnId === proof.turnId
&& e.payload.action === "accept" && closed.seq > card.row.seq && hash(closed) === proof.resolvedRowSha256);
const completed = rows.at(-1)!;
requireProof(completed.eventType === "tool.execution.completed" && completed.seq > closed.seq && hash(completed) === proof.completedRowSha256);
const tested = piPermissionRequests(events, proof.runId).filter(x => x.request.requestId !== proof.requestId);
requireProof(tested.every(x => x.row.seq > completed.seq && x.event.turnId === proof.turnId
&& x.event.normalizedSessionId === proof.normalizedSessionId && x.event.sourceInstanceId === proof.sourceInstanceId));
return events.filter(row => row !== card.row && row !== closed);
}
/** The operator may approve one setup read after publication and observer
* arming. Production permission policy and the tested write are untouched. */
export async function approvePiBootstrapRead(input: {
api: RunnerApi; fixture: { actionFile: string; binding: Pick<RemoteNativeFixture["binding"], "companyId" | "runId">; snapshot(label: string): Promise<Pick<RemoteNativeSnapshot, "complete" | "setup">> }; companyId: string; issueId: string; runId: string;
deadlineAt: number; load(): Promise<PiBootstrapState>; evidence(name: string, value: unknown): Promise<void>;
}): Promise<PiBootstrapApproval | undefined> {
const actionFile = input.fixture.actionFile;
requireProof(/^\.paperclip-eval-action-[a-f0-9]{36}\.txt$/u.test(actionFile)
&& input.fixture.binding.companyId === input.companyId && input.fixture.binding.runId === input.runId);
const pending = await pollUntil({ label: "Pi published bootstrap read", deadlineAt: input.deadlineAt, intervalMs: 100,
load: input.load, reject: state => ["failed", "timed_out", "cancelled", "succeeded"].includes(state.run.status) ? "Pi bootstrap run settled" : undefined,
accept: state => piPermissionRequests(state.events, input.runId).length > 0 });
requireProof(pending.run.status === "running" && pending.run.runtimeMode === "native" && pending.issue.status === "in_progress" && pending.run.id === input.runId && pending.run.companyId === input.companyId && pending.run.nativeIssueId === input.issueId
&& pending.issue.id === input.issueId && pending.issue.companyId === input.companyId);
const requests = piPermissionRequests(pending.events, input.runId); requireProof(requests.length === 1);
const card = requests[0]!, executionId = bootstrapReadExecutionId(card.request.details.toolCallId);
const native = (pending.events as readonly Row[]).filter(row => row.payload?.prpEvent?.payload?.executionId === executionId);
// A read permitted without delegation can already have led to the tested
// write. There is no setup permission to remove in that existing path.
if (native.some(row => row.payload.prpEvent.payload.operation === "edit")) return undefined;
const checked = readLifecycle(pending.events, { companyId: input.companyId, runId: input.runId, actionFile, requestId: card.request.requestId });
requireProof(!checked.rows.some(row => row.eventType === "tool.execution.completed"));
const snapshot = await input.fixture.snapshot("bootstrap-read-published");
requireProof(snapshot.complete && snapshot.setup.path === actionFile && snapshot.setup.published
&& /^sha256:[a-f0-9]{64}$/u.test(snapshot.setup.sha256 ?? ""));
await input.evidence("pi-bootstrap-read-before-approval.json", { binding: input.fixture.binding, snapshot, request: card.row, native: checked.rows });
await input.api.post(`/api/heartbeat-runs/${input.runId}/runtime-requests/${encodeURIComponent(card.request.requestId)}/resolve`,
{ turnId: card.event.turnId, requestKind: "permission_approval", resolution: { action: "accept" } });
const finished = await pollUntil({ label: "Pi exact bootstrap read completion", deadlineAt: input.deadlineAt, intervalMs: 100, load: input.load,
reject: state => ["failed", "timed_out", "cancelled", "succeeded"].includes(state.run.status) ? "Pi bootstrap run settled" : undefined,
accept: state => (state.events as readonly Row[]).some(row => row.eventType === "tool.execution.completed"
&& row.payload?.prpEvent?.payload?.executionId === executionId) });
const closure = (finished.events as readonly Row[]).find(row => row.eventType === "runtime_request.resolved" && row.payload?.prpEvent?.payload?.requestId === card.request.requestId);
const completed = (finished.events as readonly Row[]).find(row => row.eventType === "tool.execution.completed" && row.payload?.prpEvent?.payload?.executionId === executionId);
requireProof(closure && completed);
const after = await input.fixture.snapshot("bootstrap-read-completed");
requireProof(after.complete && after.setup.path === actionFile && after.setup.published && after.setup.sha256 === snapshot.setup.sha256);
await input.evidence("pi-bootstrap-read-completed.json", { binding: input.fixture.binding, snapshot: after });
const proof: PiBootstrapApproval = { schema: "paperclip.e2e.pi-bootstrap-read-approval.v1", companyId: input.companyId, issueId: input.issueId, runId: input.runId,
actionFile, publishedSha256: snapshot.setup.sha256!, requestId: card.request.requestId, toolCallId: card.request.details.toolCallId,
turnId: card.event.turnId, normalizedSessionId: card.event.normalizedSessionId, sourceInstanceId: card.event.sourceInstanceId,
createdRowSha256: hash(card.row), resolvedRowSha256: hash(closure), completedRowSha256: hash(completed) };
withoutApprovedPiBootstrapRequests(finished.events, proof);
await input.evidence("pi-bootstrap-read-approval.json", proof); return proof;
}
@@ -189,8 +189,8 @@ describe("Pi controls catalog admission", () => {
// provider death, pending restart and the strict file oracle; no runtime admission is promoted.
const pi = runnerMatrix.find(c => c.profile.qualificationCandidate === "pi")!.profile;
expect(pi.modelQualification?.qualificationId).toBe("pi:0.0.33:1.0.0:openrouter");
expect(runnerSuites.find(s => s.id === "pi-native")!.definitionMetadata).toMatchObject({ version: 4, profileVersion: 14 });
expect(runnerSuites.find(s => s.id === "pi-controls")!.definitionMetadata).toMatchObject({ version: 6, profileVersion: 14,
expect(runnerSuites.find(s => s.id === "pi-native")!.definitionMetadata).toMatchObject({ version: 5, profileVersion: 14 });
expect(runnerSuites.find(s => s.id === "pi-controls")!.definitionMetadata).toMatchObject({ version: 7, profileVersion: 14,
controlPlaneSettlement: "required-scoped-result-and-terminal-after-runner" });
expect(runnerSuites.find(s => s.id === "extended-harnesses")!.definitionMetadata).toMatchObject({ version: 2 });
for (const cell of runnerMatrix.filter(cell => cell.profile.qualificationCandidate === "pi")) {
@@ -199,7 +199,7 @@ describe("Pi controls catalog admission", () => {
}
const hashes = Object.fromEntries(runnerSuites.filter(s => ["pi-native", "native-active-stop", "extended-harnesses", "rich-acp-warm-continuity"].includes(s.id)).map(s => [s.id, suiteDefinitionHash(s)]));
expect(hashes).toEqual({
"pi-native": "00f1ec9adc5980c6d147fbc8a4992ff76fb2327ad8d8dc1fc8516a69d4131bd1",
"pi-native": "bbfcb15e83082b11046ab72ed31acf40bae268a4f13897f88214b1d42f7f23f2",
"native-active-stop": "99682b2b106d816a011834fae5a944ed7729958893709d5b83a19b6f595e7e4d",
"rich-acp-warm-continuity": "036c0faebc2f6eee5cd22ea38887c9c22650a83561fd08ee83473a565b11bb00",
"extended-harnesses": "9814841e571cb8bb1dc5188a8577245896e0ce8c851294ac5dea9e3d42689db6",
+10 -7
View File
@@ -2,14 +2,15 @@ import { createHash } from "node:crypto";
import { canonicalJson } from "../../packages/shared/src/portability-hash.js";
import { hasAcpxNativeOrigin } from "./acpx-native-origin.js";
import { bootstrapReadExecutionId } from "./native-bootstrap-read-proof.js";
import { withoutApprovedPiBootstrapRequests, type PiBootstrapApproval } from "./pi-bootstrap-permission.js";
import { isValidNativePrpEnvelope } from "./native-event-envelope.js";
import type { ActiveStopCaller } from "./native-active-stop-evidence.js";
type Row = Record<string, any>;
export interface PiControlScope { companyId: string; issueId: string; runId: string; target: string }
export interface PiControlState { run: Row; issue: Row; events: readonly unknown[] }
export interface PiControlState { run: Row; issue: Row; events: readonly unknown[]; bootstrapApproval?: PiBootstrapApproval }
export interface PiControlPending {
schema: "paperclip.e2e.pi-control-pending.v1"; scope: PiControlScope; requestId: string; toolCallId: string; executionId: string;
schema: "paperclip.e2e.pi-control-pending.v1"; bootstrapApproval?: PiBootstrapApproval; scope: PiControlScope; requestId: string; toolCallId: string; executionId: string;
turnId: string; normalizedSessionId: string; sourceInstanceId: string; itemId: string;
requestSourceSeq: number; startedSourceSeq: number; requestRowSha256: string; startedRowSha256: string; observedMonotonicNs: string;
}
@@ -23,7 +24,9 @@ const controlSettlementTypes = new Set(["run.result.accepted", "run.terminal"]);
/** Consume the actual Product projection. Pi does not emit Cursor/Copilot native
* diagnostic notices; never manufacture those to reuse another provider's oracle. */
function origin(events: readonly unknown[], scope: PiControlScope) {
function origin(events: readonly unknown[], scope: PiControlScope, bootstrapApproval?: PiBootstrapApproval) {
if (bootstrapApproval) requireProof(bootstrapApproval.companyId === scope.companyId && bootstrapApproval.issueId === scope.issueId && bootstrapApproval.runId === scope.runId, "foreign bootstrap approval");
const gradingRows = new Set(withoutApprovedPiBootstrapRequests(events, bootstrapApproval));
requireProof(Object.values(scope).every(id) && events.length > 0 && events.length <= 20_000, "bounded exact scope required");
const projected = events.map(rec).filter(row => rec(row.payload).prpEvent !== undefined).map(row => ({ row, event: rec(rec(row.payload).prpEvent) })).sort((a, b) => a.row.seq - b.row.seq);
const seqs = new Set<number>(), sourceIds = new Set<string>(), last = new Map<string, number>();
@@ -35,7 +38,7 @@ function origin(events: readonly unknown[], scope: PiControlScope) {
&& e.sourceEventId === `${e.sourceInstanceId}:${e.runId}:${e.sourceSeq}` && !sourceIds.has(e.sourceEventId), "foreign, duplicate or reordered event");
seqs.add(row.seq); sourceIds.add(e.sourceEventId); last.set(e.sourceInstanceId, e.sourceSeq);
}
const rows = projected.filter(x => x.event.sourceKind === "runner");
const rows = projected.filter(x => x.event.sourceKind === "runner" && gradingRows.has(x.row));
const control = projected.filter(x => x.event.sourceKind === "control_plane");
const created = rows.filter(x => x.event.eventType === "runtime_request.created");
requireProof(created.length === 1, "one native permission required");
@@ -80,20 +83,20 @@ function origin(events: readonly unknown[], scope: PiControlScope) {
return { rows, control, card, request, started: started[0]!, executionId, stream };
}
export function observePiControlPending(input: PiControlState & { scope: PiControlScope }): PiControlPending {
const { run, issue, scope } = input, p = origin(input.events, scope);
const { run, issue, scope } = input, p = origin(input.events, scope, input.bootstrapApproval);
requireProof(run.id === scope.runId && run.companyId === scope.companyId && run.nativeIssueId === scope.issueId && run.status === "running" && run.runtimeMode === "native"
&& issue.id === scope.issueId && issue.companyId === scope.companyId && issue.status === "in_progress"
&& run.resultJson?.startupCancellation == null && run.resultJson?.nativeCancellation == null, "run is not fresh active work");
requireProof(!p.rows.some(x => terminalTypes.has(x.event.eventType) || closureTypes.has(x.event.eventType)
|| (x.event.eventType === "tool.execution.completed" && x.event.payload.executionId === p.executionId)), "permission already answered or provider settled");
return { schema: "paperclip.e2e.pi-control-pending.v1", scope, requestId: p.request.requestId, toolCallId: p.request.details.toolCallId, executionId: p.executionId,
return { schema: "paperclip.e2e.pi-control-pending.v1", ...(input.bootstrapApproval ? { bootstrapApproval: input.bootstrapApproval } : {}), scope, requestId: p.request.requestId, toolCallId: p.request.details.toolCallId, executionId: p.executionId,
turnId: p.card.event.turnId, normalizedSessionId: p.card.event.normalizedSessionId, sourceInstanceId: p.card.event.sourceInstanceId, itemId: p.request.itemId,
requestSourceSeq: p.card.event.sourceSeq, startedSourceSeq: p.started.event.sourceSeq, requestRowSha256: hash(p.card.row), startedRowSha256: hash(p.started.row), observedMonotonicNs: process.hrtime.bigint().toString() };
}
function retained(input: PiControlState & { pending: PiControlPending }) {
const b = input.pending;
requireProof(b.schema === "paperclip.e2e.pi-control-pending.v1", "pending receipt missing");
const p = origin(input.events, b.scope);
const p = origin(input.events, b.scope, b.bootstrapApproval);
requireProof(hash(p.card.row) === b.requestRowSha256 && hash(p.started.row) === b.startedRowSha256 && p.card.event.sourceSeq === b.requestSourceSeq && p.started.event.sourceSeq === b.startedSourceSeq
&& p.request.requestId === b.requestId && p.request.details.toolCallId === b.toolCallId && p.executionId === b.executionId && p.request.itemId === b.itemId
&& p.card.event.turnId === b.turnId && p.card.event.normalizedSessionId === b.normalizedSessionId && p.card.event.sourceInstanceId === b.sourceInstanceId, "retained pending identity changed");
@@ -8,6 +8,8 @@ import { runPiControlsFlow } from "./pi-controls-flow.js";
import { persistedFinalRunMessage } from "./matchers.js";
import { readPiSteeringSettlement } from "./pi-controls-evidence.js";
vi.mock("./pi-bootstrap-permission.js", async importOriginal => ({ ...await importOriginal<typeof import("./pi-bootstrap-permission.js")>(), approvePiBootstrapRead: async () => undefined }));
const harness = vi.hoisted(() => ({ target: "", prompt: "", message: "", mutation: false, incomplete: false }));
vi.mock("./user-actions.js", () => ({
createTaskThroughUi: async (input: { prompt: string }) => { harness.prompt = input.prompt; },
+5 -1
View File
@@ -8,6 +8,7 @@ import { createDeniedTargetFixture, exists, observeRunProcesses } from "./copilo
import { assertCopilotRemoteRetirement, copilotRemoteDeniedSample, prepareCopilotRemoteAction, type CopilotRemoteBootstrap, type CopilotRemoteFixture, type CopilotRemoteSnapshot } from "./copilot-protection-evidence.js";
import { assertActiveStopRetirement, readActiveStopRemoteRetirement, readActiveStopCaller, type ActiveStopCaller, type ActiveStopRemoteObservation } from "./native-active-stop-evidence.js";
import { assertSamePiPending, observePiControlPending, readPiStopSettlement, readPiSteeringAcknowledgement, readPiSteeringSettlement, type PiControlPending, type PiControlScope, type PiControlState } from "./pi-controls-evidence.js";
import { approvePiBootstrapRead, type PiBootstrapApproval } from "./pi-bootstrap-permission.js";
import { piNativeFinish } from "./pi-native-cases.js";
import type { LiveFixtureValues } from "./live-fixtures.js";
import type { MatrixExecution } from "./types.js";
@@ -51,6 +52,7 @@ export async function runPiControlsFlow(input: {
|| !["pending-permission-stop", "same-turn-steering"].includes(execution.task.id)) throw new Error("Unknown Pi control case");
const stopCase = execution.task.id === "pending-permission-stop", remote = execution.environment.id === "daytona";
if ((!remote && execution.environment.id !== "local") || (remote && !input.remoteBootstrap)) throw new Error("Pi controls require an isolated admitted environment");
let bootstrapApproval: PiBootstrapApproval | undefined;
const checks: Check[] = []; let issue: Row = {}, runs: Row[] = [], events: Row[] = [];
const check = (id: string, passed: boolean, detail: string) => { checks.push({ id, passed, detail }); expect(passed, detail).toBe(true); };
const name = `pi-control-${nonce}.txt`, local = remote ? undefined : await createDeniedTargetFixture(input.workspacePath, name), target = local?.targetRelativePath ?? name;
@@ -72,7 +74,7 @@ export async function runPiControlsFlow(input: {
runs = await Promise.all(list.map(run => api.get<Row>(`/api/heartbeat-runs/${run.id}`)));
if (runs.length > 1) throw new Error("Stopped waiting for Pi controls: extra provider run");
events = runs[0] ? await collectRunEvents<Row>((afterSeq, limit) => api.get(`/api/heartbeat-runs/${runs[0]!.id}/events?afterSeq=${afterSeq}&limit=${limit}`)) : [];
observeProcesses(); input.observe(issue, runs); return { run: runs[0] ?? {}, issue, events };
observeProcesses(); input.observe(issue, runs); return { run: runs[0] ?? {}, issue, events, bootstrapApproval };
};
const scope = (): PiControlScope => ({ companyId: fixtures.company.id, issueId: issue.id, runId: runs[0]!.id, target });
const readPresentation = async (state: PiControlState) => {
@@ -151,6 +153,8 @@ export async function runPiControlsFlow(input: {
await input.evidence("pi-control-before-request-remote.json", baseline); return prepared.prompt;
} });
if (bound !== fixture) throw new Error("Pi remote action identity changed");
bootstrapApproval = await approvePiBootstrapRead({ api, fixture: bound, companyId: fixtures.company.id, issueId: issue.id, runId: runs[0]!.id,
deadlineAt: input.deadlineAt, load, evidence: input.evidence });
}
await pollUntil({ label: "Pi unanswered native write", deadlineAt: input.deadlineAt, intervalMs: 200,
load: async () => { const state = await load(); if (["failed", "timed_out", "cancelled", "succeeded"].includes(state.run.status)) throw new Error("Stopped waiting for Pi unanswered native write: no active pending permission"); return state; },
+6 -2
View File
@@ -13,6 +13,7 @@ import type { LiveFixtureValues } from "./live-fixtures.js";
import type { MatrixExecution } from "./types.js";
import type { RemoteNativeFixture, RemoteNativeSnapshot } from "./remote-native-fixtures.js";
import { approvePiBootstrapRead, withoutApprovedPiBootstrapRequests, type PiBootstrapApproval } from "./pi-bootstrap-permission.js";
import { runPiPendingProviderDeath, PI_DEATH_MARKER } from "./pi-native-provider-death-flow.js";
import { runPiPendingControllerRestart } from "./pi-native-restart-flow.js";
@@ -38,6 +39,7 @@ export async function runPiNativeFlow(input: {
if (!["local", "daytona"].includes(execution.environment.id) || execution.profile.qualificationCandidate !== "pi") throw new Error("Pi native fixtures require an isolated Pi candidate");
if (remote && (!input.remoteBootstrap || !input.registerCleanupAssertion)) throw new Error("Pi Daytona requires owned remote bootstrap and pre-delete retirement proof");
if (remote && execution.task.id === "restrictive-denial") throw new Error("Pi deny-all cannot read the native action-file bootstrap; remote auto-denial remains unsupported");
let bootstrapApproval: PiBootstrapApproval | undefined;
let currentRemote: RemoteNativeFixture | undefined, currentBaseline: RemoteNativeSnapshot | undefined;
let remoteSequence = 0;
async function finishRemote(label: string) {
@@ -202,10 +204,12 @@ export async function runPiNativeFlow(input: {
});
if (!remote) check("human-target-initially-absent", await absent(path), "Independent target is absent before provider work");
await create(execution.task.buildTitle(nonce), localTarget ? bindDeniedTargetPrompt(execution.task.buildPrompt(nonce), "pi-human-denied.txt", target) : execution.task.buildPrompt(nonce), { targets: [target] });
if (remote) bootstrapApproval = await approvePiBootstrapRead({ api, fixture: currentRemote!, companyId: fixtures.company.id, issueId: issue.id, runId: runs[0]!.id,
deadlineAt: input.deadlineAt, load: async () => { await load(); return { run: runs[0]!, issue, events: await events(runs[0]!.id) }; }, evidence: input.evidence });
if (remote) check("human-target-initially-absent", currentBaseline?.targets[target]?.absent === true && currentBaseline.targets[target]!.complete, "Remote watcher was armed before action publication with an absent target");
const pending = await pollUntil({ label: "Pi native browser permission", deadlineAt: input.deadlineAt, load: async () => { const state = await load(); processes = observe(); return { ...state, events: state.runs.length === 1 ? await events(state.runs[0]!.id) : [] }; }, reject: rejectFailure,
accept: state => state.runs.length === 1 && piPermissionRequests(state.events, state.runs[0]!.id).length === 1 });
const native = piPermissionRequests(pending.events, pending.runs[0]!.id)[0]!;
accept: state => state.runs.length === 1 && piPermissionRequests(withoutApprovedPiBootstrapRequests(state.events, bootstrapApproval), state.runs[0]!.id).length === 1 });
const native = piPermissionRequests(withoutApprovedPiBootstrapRequests(pending.events, bootstrapApproval), pending.runs[0]!.id)[0]!;
const identity = { runId: pending.runs[0]!.id, turnId: native.event.turnId, requestId: native.request.requestId, toolCallId: native.request.details.toolCallId, target };
check("human-target-pending-absent", remote ? (await currentRemote!.snapshot("permission-pending")).targets[target]?.absent === true : await absent(path), "Pending native write has no file effect");
await page.reload();
@@ -4,6 +4,8 @@ import { join } from "node:path";
import { expect, it, vi } from "vitest";
import { piNativeTasks } from "./pi-native-cases.js";
import { runPiNativeFlow } from "./pi-native-flow.js";
vi.mock("./pi-bootstrap-permission.js", async importOriginal => ({ ...await importOriginal<typeof import("./pi-bootstrap-permission.js")>(), approvePiBootstrapRead: async () => undefined }));
const proof = vi.hoisted(() => ({ mutation: false, incomplete: false, live: false, target: "pi-human-denied.txt", prompt: "" }));
vi.mock("./user-actions.js", () => ({ createTaskThroughUi: vi.fn(async (input: { prompt: string }) => { proof.prompt = input.prompt; }) }));
vi.mock("./copilot-local-fixtures.js", async importOriginal => {