From b012b3aebeaf1edf72fd857a374f6be7c990529b Mon Sep 17 00:00:00 2001 From: Dotta Date: Mon, 5 Oct 2026 17:56:58 -0500 Subject: [PATCH] fix(runner): synchronize Pi profile 15 native admission --- .../2026-10-02-pi-production-readiness.md | 22 ++++++++++---- .../runner-core/src/acpx_provider_backend.rs | 29 +++++++++++++++++-- .../tests/native_provider_backend.rs | 2 +- .../live/fixtures/fake-pi-warm-sidecar.cjs | 2 +- .../runner-e2e/pi-native-remote-flow.test.ts | 7 ++++- 5 files changed, 51 insertions(+), 11 deletions(-) diff --git a/doc/plans/2026-10-02-pi-production-readiness.md b/doc/plans/2026-10-02-pi-production-readiness.md index ee2598c26f9..d2b8305189e 100644 --- a/doc/plans/2026-10-02-pi-production-readiness.md +++ b/doc/plans/2026-10-02-pi-production-readiness.md @@ -76,6 +76,16 @@ installation on all three platforms remains required. The historical profile fixtures remain intact. Profile 14 results cannot qualify profile 15. Pi 1.0.0, pi-acp 0.0.33, ACPX 0.13.1, Node 24.21.0, the model and native low do not change. +The first normal profile-15 ARM/Linux admission probes reject the descriptor +before any prompt: Rust still binds profile 14 while TypeScript sends profile +15. Those failed artifact proofs remain failed; the owned Linux admission +sandbox is deleted. The correction synchronizes Rust admission and its current +fixtures, adds a published-identity regression that fails before the correction, +and retains explicit rejection of profile 14. The remote-memory unit fixture +now parses the fenced JSON and independently checks all 33 bytes. Full build +and recursive typecheck pass at the prior source; new artifacts and all live +qualification remain required after this binding correction. + ## Frozen target - Pi: `@earendil-works/pi-coding-agent@1.0.0`. @@ -95,11 +105,11 @@ pi-acp 0.0.33, ACPX 0.13.1, Node 24.21.0, the model and native low do not change | --- | --- | --- | | Runtime identity and admission | Exact runtime/profile/model; verified effective thinking; startup below the 60-second admission limit; unsupported configuration fails before a prompt | Frozen shipping source `0bd040093` passes normal public CLI/server installation, Pi setup and credential-free profile-14 admission on ARM Mac (7.921 s), Intel Mac (31.692 s), and native Linux in Daytona (8.699 s). No prompt, credentials or binary override is used. The local Rosetta handshake failures remain failed evidence. | | Live lifecycle | Pending native question survives controller restart in the original process; one creation and resolution; stale answer after provider death rejected; Stop retires owned processes; three warm turns keep process identity | Frozen shipping source `0bd040093` passes local Stop, steering, controller restart, native questions and warm continuity. Daytona warm continuity, human denial, corrected Stop and first steering pass. Native questions, native controller-restart and provider-death remain failed. Prior failed attempts remain unchanged. | -| Product workflows | All 26 explicit Pi Product E2E cells (13 local, 13 Daytona); screenshots, public state, independent artifacts, terminal and cleanup evidence | All 26 cells are attempted at frozen shipping source `0bd040093`: 20 pass and six remain failed. Local has 11 passes and two failures: agent-files and file-edit. Daytona has nine passes and four failures: native-questions, native controller-restart, agent-files and provider-death. The separately corrected Daytona file-edit attempt passes all seven matchers and canonical cleanup on extended definitions 3; its original failure remains failed. Human denial passes nine matchers and cleanup on native definitions 5. Corrected Stop passes five matchers; first steering passes eight; both pass cleanup on controls definitions 8. No cells remain unattempted or running. Original grades and source fingerprints are preserved. | -| Runner protocol | Pi roster passes through the native packaged runner and authenticated mock control plane; lifecycle/denial/control cases remain distinct from Product tests | All seven cases are attempted at `0bd040093`: five pass; context-before-action retains its 120-second terminal timeout, and finish-task retains use of the advisory report without the required task mutation. One context/document workflow retry passes only after verified protection against the recorded host sleep; its original failure stays unchanged. The seven `b148b73ea` passes remain historical. | +| Product workflows | All 26 explicit Pi Product E2E cells (13 local, 13 Daytona); screenshots, public state, independent artifacts, terminal and cleanup evidence | Historical frozen runtime `0bd040093`: **21/26 pass** (local 12/13, Daytona 9/13). Remaining failures: local and Daytona agent-files, Daytona native questions, native controller restart and provider death. Corrected local/Daytona file-edit, human denial, Stop and steering pass with their source-bound fingerprints. All original grades remain retained. The changed shipping candidate has no paid qualification yet. | +| Runner protocol | Pi roster passes through the native packaged runner and authenticated mock control plane; lifecycle/denial/control cases remain distinct from Product tests | Historical frozen runtime `0bd040093`: **7/7 pass**, including corrected context-before-action and finish-task prompts at private definitions `7b112ffe3`, unchanged graders and zero automatic retries. Earlier failures stay failed. The changed shipping candidate requires a fresh Runner qualification. | | Installed distribution | Public CLI/server tars on ARM Mac, Intel Mac and Linux; normal Pi setup; exact Linux companion and immutable Daytona image imported without binary override | Public CLI/server installation and normal Pi setup pass on all three platforms at original execution source `0bd040093`. The installer correction at `fcef1eae9` changes CLI and provisioner bytes; fresh normal public artifact/install proof must bind the updated source. Old artifact proofs keep their original labels. Public immutable image `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:342f1fd5cb8cabfa2242f38f4f686608b28b6536aa879c54b0cb0da6fba9ef47` imports into native Linux Daytona and passes closed admission. Temporary sandbox cleanup passes. Image jobs 37102904889, 37106313185 and 37110683910 are terminal/cancelled. | | Governance and spend | Company isolation, human-only permission, duplicate/stale answers, Stop and budget hard stop; pricing estimates never become claimed bills | 58 focused governance/cost/session invariants pass at `f5f57e380`. Live pending-permission Stop passes at `03dd6ef93`. The qualification key cap is not proof of Paperclip budget enforcement. | -| Integration and rollout | Review each prerequisite; final-head typecheck, tests, build and CI pass; no unresolved review; exact artifacts; rollback recorded | Last verified readiness head `146f578c0` has 53 successful CI checks, two skips, current-head Greptile 5/5 and no unresolved root threads, including full build and typecheck. The new installer integration must receive its own completed current-head CI, review and updated public artifact/install proof. All 116 targeted native Linux fixture tests and 13 browser-case collections pass at `04f639eeb`; the new descriptor correction separately passes all 17 free native Linux fault tests. Full typecheck, complete test and build evidence remains recorded below; original failed commands remain failed. Frozen Pi execution inputs remain equivalent to `0bd040093`; the explicit installer has three changed executable inputs. Prerequisite #14921 passes every test/build/typecheck check but its valid admission review blocks merge. The retained SDK and setup network findings receive scoped corrections with focused verification; their new heads require fresh CI and review. Production stays held for eight failed gates and prerequisite disposition. No merge or release. | +| Integration and rollout | Review each prerequisite; final-head typecheck, tests, build and CI pass; no unresolved review; exact artifacts; rollback recorded | Last verified readiness head `146f578c0` has 53 successful CI checks, two skips, current-head Greptile 5/5 and no unresolved root threads, including full build and typecheck. The new installer integration must receive its own completed current-head CI, review and updated public artifact/install proof. All 116 targeted native Linux fixture tests and 13 browser-case collections pass at `04f639eeb`; the new descriptor correction separately passes all 17 free native Linux fault tests. Full typecheck, complete test and build evidence remains recorded below; original failed commands remain failed. Frozen Pi execution inputs remain equivalent to `0bd040093`; the explicit installer has three changed executable inputs. Prerequisite #14921 passes every test/build/typecheck check but its valid admission review blocks merge. The retained SDK and setup network findings receive scoped corrections with focused verification; their new heads require fresh CI and review. Production stays held for five remaining historical Product failures, fresh shipping-candidate qualification and prerequisite disposition. No merge or release. | ## Bounded execution @@ -115,10 +125,10 @@ Check remaining credit and that BYOK usage stays zero before and after each attempt. API key deltas are provisional billing observations. Pi model-catalog prices are estimates. Unpriced usage must stay unpriced in the ledger and UI. -## Current qualification snapshot — 2026-10-05 +## Historical qualification snapshot — 2026-10-05, before corrected passes -This snapshot supersedes the older counts below. All 33 required cells are -attempted: **25 pass and eight fail**, with none running or unattempted. +This retained snapshot predates the completed corrections reported above. +At that point all 33 required cells were attempted: **25 pass and eight fail**, with none running or unattempted. Product has 20 passes and six failures; Runner has five passes and two failures. Remaining failures are local agent-files and file-edit; Daytona native questions, agent-files, native controller restart and provider death; and Runner diff --git a/packages/paperclip-runner/runner/crates/runner-core/src/acpx_provider_backend.rs b/packages/paperclip-runner/runner/crates/runner-core/src/acpx_provider_backend.rs index f3ef5b04284..540d0ce720c 100644 --- a/packages/paperclip-runner/runner/crates/runner-core/src/acpx_provider_backend.rs +++ b/packages/paperclip-runner/runner/crates/runner-core/src/acpx_provider_backend.rs @@ -209,7 +209,7 @@ impl AcpxProviderDescriptor { "0.0.33", Some("@earendil-works/pi-coding-agent"), Some("1.0.0"), - "sha256:f35145437eeb355ed37bc5a4fa93d7ede561d9c45daf311979b46890b808ddd4", + "sha256:790f8b954be995ef63aef0ebdb0e06215e4c0d1416d40d605b33966a3d6ba053", ), "cursor" => ( self.model.as_str(), @@ -3049,7 +3049,7 @@ mod tests { "pi", "pi-acp", "0.0.33", - "sha256:f35145437eeb355ed37bc5a4fa93d7ede561d9c45daf311979b46890b808ddd4", + "sha256:790f8b954be995ef63aef0ebdb0e06215e4c0d1416d40d605b33966a3d6ba053", Some("@earendil-works/pi-coding-agent"), Some("1.0.0"), "openrouter/deepseek/deepseek-v4-flash-0731", @@ -3255,6 +3255,31 @@ mod tests { } } + #[test] + fn pi_profile_matches_published_identity_and_rejects_profile14() { + let published: Value = serde_json::from_str(include_str!( + "../../../../test-fixtures/pi-acp/profile-v15-identity.json" + )) + .unwrap(); + let mut value = descriptor("codex"); + value["agent"] = json!("pi"); + value["model"] = json!("openrouter/deepseek/deepseek-v4-flash-0731"); + value["agentServerPackage"] = json!("pi-acp"); + value["agentServerVersion"] = json!("0.0.33"); + value["agentRuntimePackage"] = json!("@earendil-works/pi-coding-agent"); + value["agentRuntimeVersion"] = json!("1.0.0"); + value["piThinkingLevel"] = json!("low"); + value["providerPolicy"] = json!({"readOnly": true}); + value["commandDigest"] = published["commandDigest"].clone(); + let current: AcpxProviderDescriptor = serde_json::from_value(value.clone()).unwrap(); + current.validate(&context()).unwrap(); + + value["commandDigest"] = + json!("sha256:f35145437eeb355ed37bc5a4fa93d7ede561d9c45daf311979b46890b808ddd4"); + let prior: AcpxProviderDescriptor = serde_json::from_value(value).unwrap(); + assert!(prior.validate(&context()).is_err()); + } + #[test] fn rejects_pi_with_another_profile_identity_before_process_launch() { let mut pi = descriptor("codex"); diff --git a/packages/paperclip-runner/runner/crates/runner-core/tests/native_provider_backend.rs b/packages/paperclip-runner/runner/crates/runner-core/tests/native_provider_backend.rs index a42562a031a..0d64b3ccd16 100644 --- a/packages/paperclip-runner/runner/crates/runner-core/tests/native_provider_backend.rs +++ b/packages/paperclip-runner/runner/crates/runner-core/tests/native_provider_backend.rs @@ -17,7 +17,7 @@ use sha2::{Digest, Sha256}; const CODEX_ACPX_DIGEST: &str = "sha256:c4538599d1ab767db5dff50934f13bb5ba313a59d9c4a83e993fac4617ea63d3"; const PI_ACPX_DIGEST: &str = - "sha256:f35145437eeb355ed37bc5a4fa93d7ede561d9c45daf311979b46890b808ddd4"; + "sha256:790f8b954be995ef63aef0ebdb0e06215e4c0d1416d40d605b33966a3d6ba053"; fn temporary_directory(label: &str) -> PathBuf { let nonce = SystemTime::now() diff --git a/packages/paperclip-runner/src/live/fixtures/fake-pi-warm-sidecar.cjs b/packages/paperclip-runner/src/live/fixtures/fake-pi-warm-sidecar.cjs index c19b76d2cf3..d905a6d7d06 100644 --- a/packages/paperclip-runner/src/live/fixtures/fake-pi-warm-sidecar.cjs +++ b/packages/paperclip-runner/src/live/fixtures/fake-pi-warm-sidecar.cjs @@ -27,7 +27,7 @@ async function handle({ id, command, params = {} }) { identity = params.expectedIdentity ?? { kind: "acpx", normalizedSessionId: params.normalizedSessionId, acpxRecordId: "record-warm", backendSessionId: "backend-warm", agentSessionId: "agent-warm", - profileDigest: "sha256:f35145437eeb355ed37bc5a4fa93d7ede561d9c45daf311979b46890b808ddd4", + profileDigest: "sha256:790f8b954be995ef63aef0ebdb0e06215e4c0d1416d40d605b33966a3d6ba053", workspaceDigest: `sha256:${"2".repeat(64)}`, requestedModel: params.model, effectiveModel: params.model, permissionMode: params.permissionMode, piThinkingLevel: params.piThinkingLevel, providerLifetimeFenceCandidates: [61001, 61002, 61003], diff --git a/tests/runner-e2e/pi-native-remote-flow.test.ts b/tests/runner-e2e/pi-native-remote-flow.test.ts index d824f99d264..577089fe486 100644 --- a/tests/runner-e2e/pi-native-remote-flow.test.ts +++ b/tests/runner-e2e/pi-native-remote-flow.test.ts @@ -40,7 +40,12 @@ it("rebinds both remote runs, saves managed bytes, and reads sealed sandbox byte }; const action = await input.actionPrompt(fixture); expect(armed).toBe(true); if (ordinal === 1) { - expect(action).toContain(fixture.outsideTarget); const literal = /Write exactly (".*?") to memory\/pi-native.txt/.exec(action)?.[1]; expect(literal).toBeDefined(); personal = JSON.parse(literal!); expect(personal).toMatch(/^[a-f0-9]{32}\n$/); + expect(action).toContain(fixture.outsideTarget); + const content = /```json\n(.*?)\n```/s.exec(action)?.[1]; + expect(content).toBeDefined(); + personal = JSON.parse(content!).content; + expect(personal).toMatch(/^[a-f0-9]{32}\n$/); + expect(Buffer.byteLength(personal, "utf8")).toBe(33); } else { expect(action).not.toContain(personal.trim()); expect(input.targets).toEqual(["pi-agent-memory-proof.txt"]); } issues.at(-1).status = "done"; runs.at(-1).status = "succeeded"; captures.push(fixture.binding); return fixture; } };