mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 08:22:20 +02:00
Retain Pi local hello pass and model admission proof
Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip committed
1 parent
dd78df1ef8
commit
06e3bae33b
7 files changed
+479
-46
No files matched your search
@@ -1,11 +1,11 @@
|
||||
# Pi rich ACP runtime
|
||||
|
||||
Status: implementation candidate, 2026-09-28. Deterministic protocol tests and an
|
||||
uncredentialed admission probe pass. One paid local Runner turn reached semantic
|
||||
tools but timed out; the latest Product attempt stopped before a prompt because
|
||||
verified snapshot preparation exceeded the startup deadline.
|
||||
Local and Linux x64 Daytona qualification remain pending; this document does not promote the candidate to a
|
||||
qualified production runtime.
|
||||
Status: implementation candidate, 2026-09-28. The local Product hello-complete
|
||||
journey passes with real Pi, semantic MCP completion, terminal tokens and measured
|
||||
billing. Deterministic protocol and immutable launch checks also pass. The earlier
|
||||
Runner protocol timeout and Product infrastructure failures remain retained.
|
||||
The wider local and Linux x64 Daytona matrix remains pending; this document does
|
||||
not promote the candidate to a qualified production runtime.
|
||||
|
||||
The runner pins `pi-acp@0.0.33` and
|
||||
`@earendil-works/pi-coding-agent@0.84.2`. The candidate model is
|
||||
@@ -242,19 +242,22 @@ helper discards failed-attempt usage, so any retry or missing compaction receipt
|
||||
invalidates complete-turn token/cost totals instead of inventing complete coverage.
|
||||
|
||||
The retained [macOS ARM64 provider-pack admission proof](../../packages/paperclip-runner/test-fixtures/pi-acp/offline-provider-pack-proof.darwin-arm64.json)
|
||||
records source `9f2d0420ed9a398e3bacd1d7e996614f3296ac56`, including the
|
||||
merged Grok support, candidate identity and completion-contract fixes. Its manifest
|
||||
digest is `sha256:2a6f457205743487cdd35ed3634532028e6e0200001f70e087d0e084fc85d452`.
|
||||
Both clean dependency resolutions matched the reviewed Docker lock digest
|
||||
`9c163adf40fe1c6a65abda6207bd9f5acfc8196bc94f81348b8bf1b2769e050e`.
|
||||
records source `dd78df1ef8b279c30c710c9b7a7f9fda22e321d6`, after foundation
|
||||
`f063fbf2b`, including bounded immutable snapshot copying. Its manifest digest is
|
||||
`sha256:f75d3de7814276508f3706edb6e4510ce1dcecbc3e8fa674991fd554e5a75273`.
|
||||
Both independent clean dependency resolutions matched
|
||||
`2c46a68811ba1d504a41b6f4d3f642bc93f4a1c615097754c9c6486881dba8e6`.
|
||||
The generic installation registry and immutable snapshot passed initialization
|
||||
and rejected the missing bound credential before any model prompt. Current-source
|
||||
verification passed the TypeScript and release Runner builds, 97 Vitest checks
|
||||
verification passed the TypeScript and release Runner builds, 107 Vitest checks
|
||||
(with one optional installation case skipped), and 22 package/receipt/materializer
|
||||
checks using the newly built distribution. An initial package-test invocation
|
||||
omitted the isolated package locations; the corrected explicit pack invocation
|
||||
passed all 22 checks. These checks do not constitute full repository verification
|
||||
or authenticated qualification.
|
||||
checks using the newly built distribution. These checks do not constitute full
|
||||
repository verification or authenticated qualification.
|
||||
|
||||
The [9f2d0420e proof](../../packages/paperclip-runner/test-fixtures/pi-acp/offline-provider-pack-proof.darwin-arm64.9f2d0420e.json)
|
||||
retains the earlier source and pack that reproduced the startup timeout. Its 97
|
||||
Vitest checks and 22 package checks, including the corrected explicit package-path
|
||||
invocation, remain historical evidence.
|
||||
|
||||
The [a6167ce3a proof](../../packages/paperclip-runner/test-fixtures/pi-acp/offline-provider-pack-proof.darwin-arm64.a6167ce3a.json)
|
||||
and [58511d79d proof](../../packages/paperclip-runner/test-fixtures/pi-acp/offline-provider-pack-proof.darwin-arm64.58511d79d.json)
|
||||
@@ -321,16 +324,18 @@ every field in Pi's native event stream. These are explicit follow-ups:
|
||||
## Authenticated local qualification progress
|
||||
|
||||
The [sanitized attempt ledger](../../packages/paperclip-runner/test-fixtures/pi-acp/paid-qualification-progress.darwin-arm64.json)
|
||||
retains all eight attempts, including seven failures before a model prompt. One Runner protocol turn reached the exact
|
||||
model and successfully called `get_task_context`, `get_task_history`,
|
||||
retains all ten attempts, including seven failures before a model prompt. One
|
||||
Runner protocol turn reached the exact model and successfully called `get_task_context`, `get_task_history`,
|
||||
`list_documents`, and `read_document`. The canonical case permits those extra
|
||||
orientation reads, but also requires a completed turn; it timed out after 120s
|
||||
without terminal usage. No successful protocol or Product journey is claimed.
|
||||
without terminal usage. The later Product hello journey passes; the canonical
|
||||
Runner protocol case has not yet passed.
|
||||
|
||||
Exclusive OpenRouter key billing increased by **$0.005748807**, confirmed again
|
||||
after diagnosis; all other attempts had two post-attempt observations with zero
|
||||
delta. This is measured provider spend, separate from Pi's stale catalog pricing
|
||||
estimate. A missing terminal receipt remains unknown, not free execution. Each
|
||||
Exclusive OpenRouter key billing increased by **$0.006755083**, including
|
||||
$0.000351509 from the sleep-interrupted Product attempt and $0.000654767
|
||||
from the successful hello journey. The seven pre-prompt failures had two post-attempt
|
||||
observations with zero delta. This is measured provider spend, separate from
|
||||
Pi's stale catalog pricing estimate. A missing terminal receipt remains unknown, not free execution. Each
|
||||
attempt had no automatic retries, a $2 reservation and a $0.50 billing watchdog;
|
||||
the Product supervisor also imposed a 300s outer deadline.
|
||||
|
||||
@@ -377,9 +382,62 @@ its immutable snapshot, before any Pi process starts. Shared fix `2a30219f1`
|
||||
copies at most eight files and 32 MiB per batch, with larger admitted files alone.
|
||||
It retains every file identity, digest and confinement check and drains all copy
|
||||
promises before cleanup. The same tree then copied in 5.851 seconds; ten security
|
||||
and concurrency tests and both Runner TypeScript checks passed. This measurement
|
||||
is not yet proof of successful end-to-end startup after the fix; the pack and
|
||||
paid attempt above intentionally retain their earlier source identity.
|
||||
and concurrency tests and both Runner TypeScript checks passed. The rebuilt
|
||||
`dd78df1e` pack then settled the full credential-free Runner
|
||||
`session.open` request in 10.087 seconds (10.198 including close), within the
|
||||
unchanged deadline. It returned terminal `session_ensure_failed`; the outer PRP
|
||||
response omits the underlying credential error. The independent immutable ACP
|
||||
probe from the same pack asserted the missing-credential error directly. This
|
||||
is timely rejection, not authenticated session success. The earlier timeout
|
||||
attempt retains its original source identity.
|
||||
|
||||
The next Product attempt (`hello05`, source `dd78df1e`) reached an authenticated
|
||||
session in 11.631 seconds, submitted a turn and received `turn.accepted`. It then
|
||||
crossed a confirmed macOS Maintenance Sleep of 888 seconds. The retained browser
|
||||
trace ends without a semantic-tool or terminal event; completion and usage are
|
||||
unknown. The canonical result remains failed/`secret_leak`: the evidence scanner
|
||||
could not inspect an incomplete ZIP (`unzip` exit 9), rather than detecting a
|
||||
credential-value match. Browser cleanup also failed after wake; a later process
|
||||
check found no remaining process from this exact attempt. Both the canonical
|
||||
921.632-second wall duration and supervisor 72.168-second monotonic duration are
|
||||
retained. This result proves startup admission, not a completed Product journey.
|
||||
The explicit retry adds an idle-sleep assertion and a wall-clock supervisor bound
|
||||
without changing the provider timeout, canonical oracle or evidence scanner.
|
||||
|
||||
The explicit [hello06 Product proof](../../packages/paperclip-runner/test-fixtures/pi-acp/product-hello-proof.darwin-arm64.json)
|
||||
passes all six canonical matchers, including the exact response once, task `done`,
|
||||
run `succeeded`, native runtime and local environment. Pi executes
|
||||
`mcp__paperclip__paperclip_finish`, then reports a completed turn and succeeded run.
|
||||
The case took 32.668 seconds; cleanup and evidence inspection passed. Its receipt
|
||||
contains 26,867 input tokens, 283 output tokens and zero cached input tokens.
|
||||
The catalog notice estimates $0.00384062; measured key billing is $0.000654767.
|
||||
Product correctly marks dollar coverage `unpriced` and incomplete. These values
|
||||
are not interchangeable.
|
||||
|
||||
The separate [authenticated model-selection proof](../../packages/paperclip-runner/test-fixtures/pi-acp/model-admission-proof.darwin-arm64.json)
|
||||
sends no prompt and measures zero key billing delta. It reads the actual ACP
|
||||
model response, selects the declared model, and observes
|
||||
`openrouter/deepseek/deepseek-v4-flash-0731` after Pi's native RPC state refresh.
|
||||
The Product path enforces this comparison before prompting, but its retained
|
||||
`session.started` model metadata says `unknown`; final usage carries the configured
|
||||
identity. The separate proof closes model-selection evidence without pretending
|
||||
it came from that Product transcript.
|
||||
|
||||
The current pack, model and hello proof JSON files retain Git tree hashes for
|
||||
Runner source, scripts, Rust, protocol, patches, server source and Product E2E.
|
||||
The pack runtime stays pinned to `dd78df1e` when later commits change only these
|
||||
reports and fixtures; an execution-source change requires a fresh build.
|
||||
|
||||
| Execution source path | Git tree at `dd78df1e` |
|
||||
| --- | --- |
|
||||
| `packages/paperclip-runner/src` | `a8123cb4396c77ce0127d5ccf49d3584b63a9a58` |
|
||||
| `packages/paperclip-runner/scripts` | `9ede97769bb3f2bbfa8e02be220b5ce06e438a99` |
|
||||
| `packages/paperclip-runner/runner` | `2cc84e150aecbd5faf2c8bcecc4b6352c24dbca8` |
|
||||
| `packages/paperclip-runner/protocol` | `1d50458fb572d0c75f9a398f419209c52c23ba6f` |
|
||||
| `patches` | `1e2f846b8b2dda202b79ee83baeb0860a3b9ab64` |
|
||||
| `server/src` | `b267adc1e0ad36955b68314829cfd4256ab88d37` |
|
||||
| `tests/runner-e2e` | `5ff4b8a85a14e3e005e3e240c145899c0bc742b5` |
|
||||
|
||||
|
||||
Production admission remains qualification-pending until the live matrix and
|
||||
spend receipt checks pass.
|
||||
+41
@@ -0,0 +1,41 @@
|
||||
{
|
||||
"schema": "paperclip.pi-model-admission-proof.v1",
|
||||
"recordedAt": "2026-09-28T20:42:44.289Z",
|
||||
"sourceRevision": "dd78df1ef8b279c30c710c9b7a7f9fda22e321d6",
|
||||
"providerPackDigest": "sha256:f75d3de7814276508f3706edb6e4510ce1dcecbc3e8fa674991fd554e5a75273",
|
||||
"profileDigest": "sha256:0f687e38cb3c607a01fab80f03e19bbbf5cb53a8afb1fc40d71f9ab54551cff1",
|
||||
"closureDigest": "sha256:b30047dce9c4c25e1fbc88a1aa828690a6a11ebdc3f6fcc083875986eb2d9585",
|
||||
"environment": {
|
||||
"platform": "darwin",
|
||||
"architecture": "arm64",
|
||||
"node": "24.19.0"
|
||||
},
|
||||
"declaredModel": "openrouter/deepseek/deepseek-v4-flash-0731",
|
||||
"initialReportedCurrentModelId": "openrouter/moonshotai/kimi-k2.6",
|
||||
"selectedReportedCurrentModelId": "openrouter/deepseek/deepseek-v4-flash-0731",
|
||||
"provenance": "Actual ACP session/new models.currentModelId and session/set_config_option configOptions.currentValue. The pinned wrapper builds both from Pi native RPC get_state after model selection.",
|
||||
"credentials": "explicit authorized OpenRouter binding; value omitted",
|
||||
"rpcMethods": [
|
||||
"initialize",
|
||||
"session/new",
|
||||
"session/set_config_option"
|
||||
],
|
||||
"promptRequests": 0,
|
||||
"paidProviderCalls": 0,
|
||||
"billingDeltaUsd": 0,
|
||||
"qualificationStatus": "pending",
|
||||
"limits": [
|
||||
"This proof verifies authenticated model selection without prompting.",
|
||||
"The separate Product hello pass uses the same pinned runtime and mandatory model-verification admission gate; this metadata-only probe is not its transcript.",
|
||||
"Native tool, permission, question, steering delivery and remote behavior remain separate qualifications."
|
||||
],
|
||||
"executionTrees": {
|
||||
"packages/paperclip-runner/src": "a8123cb4396c77ce0127d5ccf49d3584b63a9a58",
|
||||
"packages/paperclip-runner/scripts": "9ede97769bb3f2bbfa8e02be220b5ce06e438a99",
|
||||
"packages/paperclip-runner/runner": "2cc84e150aecbd5faf2c8bcecc4b6352c24dbca8",
|
||||
"packages/paperclip-runner/protocol": "1d50458fb572d0c75f9a398f419209c52c23ba6f",
|
||||
"patches": "1e2f846b8b2dda202b79ee83baeb0860a3b9ab64",
|
||||
"server/src": "b267adc1e0ad36955b68314829cfd4256ab88d37",
|
||||
"tests/runner-e2e": "5ff4b8a85a14e3e005e3e240c145899c0bc742b5"
|
||||
}
|
||||
}
|
||||
+83
@@ -0,0 +1,83 @@
|
||||
{
|
||||
"schema": "paperclip.pi-provider-pack-offline-proof.v1",
|
||||
"recordedAt": "2026-09-28T20:02:30.796Z",
|
||||
"classification": "offline_candidate_admission",
|
||||
"source": {
|
||||
"runnerRevision": "9f2d0420ed9a398e3bacd1d7e996614f3296ac56",
|
||||
"distDigest": "sha256:3fb488c2a156f8ee1feab94f98985662da393bfeb516a19150638d6b45ec55cd",
|
||||
"bridgeDigest": "sha256:f6c56eac02b7962f100add6c5ce2daeeae4ab2c70aaa39b046b8bb1fc584e175",
|
||||
"productionLockDigest": "sha256:e0c928a494f90ddad3c00791e83f09315ee8c82df2a0418a809dcf93649a8ab3",
|
||||
"note": "Built from merged foundation c3b7e9ecd plus the Pi provider branch, including native candidate identity and visible completion-contract fixes. This probe executes the deployed generic registry and immutable snapshot without credentials or prompts. It predates shared copy-performance fix 2a30219f1; the public paid ledger separately retains the startup failure from this source."
|
||||
},
|
||||
"environment": {
|
||||
"platform": "darwin",
|
||||
"architecture": "arm64",
|
||||
"nodeVersion": "24.19.0",
|
||||
"execution": "local macOS ARM64; immutable guarded provider snapshot",
|
||||
"packageManager": "pnpm@9.15.4"
|
||||
},
|
||||
"candidate": {
|
||||
"agent": "pi",
|
||||
"qualification": "pending",
|
||||
"agentProfileVersion": 2,
|
||||
"declaredModel": "openrouter/deepseek/deepseek-v4-flash-0731",
|
||||
"reportedModelId": null,
|
||||
"modelVerification": "not_performed_authentication_missing",
|
||||
"wrapperVersion": "0.0.33",
|
||||
"runtimeVersion": "0.84.2",
|
||||
"profileDigest": "sha256:0f687e38cb3c607a01fab80f03e19bbbf5cb53a8afb1fc40d71f9ab54551cff1",
|
||||
"closureDigest": "sha256:b30047dce9c4c25e1fbc88a1aa828690a6a11ebdc3f6fcc083875986eb2d9585",
|
||||
"assetTreeDigest": "sha256:5c046265335d4a6924efcd207f837d5aa0e0fc3baa0bf6ea7059afe8d6ab3939",
|
||||
"distributionManifestDigest": "sha256:406345b3b75d8aee399da886fbdf8566aaaa44ff29fe3462da21f0c29901aecd",
|
||||
"wrapperDigest": "sha256:2eb201c3a931177490a8e18f4a8392f00cbcf21847e9cc171de528da5e253935",
|
||||
"helperDigest": "sha256:4333d284ff06ab1195e49630041f0b03a486a92fc678100525ad107e0982dba1",
|
||||
"extensionDigest": "sha256:0a2ca138d6d87a85498bd982ee64543ea8b14f2ecceded96cfc27b0836cc4254"
|
||||
},
|
||||
"providerPack": {
|
||||
"manifestDigest": "sha256:2a6f457205743487cdd35ed3634532028e6e0200001f70e087d0e084fc85d452",
|
||||
"portableNodeDigest": "sha256:27db838bb204ef7c21df2931f5656e4c8fb32e6e947f363a402b49714d32b5b1",
|
||||
"piNodeDigest": "sha256:27db838bb204ef7c21df2931f5656e4c8fb32e6e947f363a402b49714d32b5b1",
|
||||
"acpxSidecarDigest": "sha256:aa71619ea0f5e08d78c13bbca1812038d4ab6fee8550177be210b5afeb26d01a"
|
||||
},
|
||||
"observed": {
|
||||
"closedSnapshotLaunch": true,
|
||||
"acpInitialize": "passed",
|
||||
"authMethods": [],
|
||||
"capabilities": {
|
||||
"version": 1,
|
||||
"steering": true,
|
||||
"queuedFollowUp": true,
|
||||
"questions": [
|
||||
"select",
|
||||
"confirm",
|
||||
"input",
|
||||
"editor"
|
||||
],
|
||||
"nativePermissions": true,
|
||||
"promptUsage": true,
|
||||
"nativePlan": false,
|
||||
"pendingRequestRecovery": "live-process-only"
|
||||
},
|
||||
"sessionNew": "rejected_authentication_missing",
|
||||
"expectedError": "Bind the configured OpenRouter API credential to this Paperclip runtime.",
|
||||
"missingCredentialRejected": true,
|
||||
"capabilityProvenance": "Read and asserted in full from this pack launch.",
|
||||
"rpcMethods": [
|
||||
"initialize",
|
||||
"session/new"
|
||||
]
|
||||
},
|
||||
"usage": {
|
||||
"providerCalls": 0,
|
||||
"promptRequests": 0,
|
||||
"paidProviderCalls": 0,
|
||||
"credentials": "none",
|
||||
"spentUsd": 0
|
||||
},
|
||||
"limitations": [
|
||||
"No provider authentication, model prompt, or reported currentModelId was verified.",
|
||||
"Native tool execution, permissions, questions, and semantic MCP calls were not exercised by this pack-launch probe.",
|
||||
"Linux x64, macOS x64, and Daytona execution were not exercised.",
|
||||
"Production qualification remains pending."
|
||||
]
|
||||
}
|
||||
+16
-7
@@ -1,13 +1,22 @@
|
||||
{
|
||||
"schema": "paperclip.pi-provider-pack-offline-proof.v1",
|
||||
"recordedAt": "2026-09-28T20:02:30.796Z",
|
||||
"recordedAt": "2026-09-28T20:17:59.387Z",
|
||||
"classification": "offline_candidate_admission",
|
||||
"source": {
|
||||
"runnerRevision": "9f2d0420ed9a398e3bacd1d7e996614f3296ac56",
|
||||
"distDigest": "sha256:3fb488c2a156f8ee1feab94f98985662da393bfeb516a19150638d6b45ec55cd",
|
||||
"bridgeDigest": "sha256:f6c56eac02b7962f100add6c5ce2daeeae4ab2c70aaa39b046b8bb1fc584e175",
|
||||
"runnerRevision": "dd78df1ef8b279c30c710c9b7a7f9fda22e321d6",
|
||||
"distDigest": "sha256:2df3f6f348001bd1dcfc1c0a151533ebbc00295bc37518f83c68a73d3637e3f6",
|
||||
"bridgeDigest": "sha256:a852df2c032b5f353db29012c397aa6070891116a699a459a94f044e3890f057",
|
||||
"productionLockDigest": "sha256:e0c928a494f90ddad3c00791e83f09315ee8c82df2a0418a809dcf93649a8ab3",
|
||||
"note": "Built from merged foundation c3b7e9ecd plus the Pi provider branch, including native candidate identity and visible completion-contract fixes. This probe executes the deployed generic registry and immutable snapshot without credentials or prompts. It predates shared copy-performance fix 2a30219f1; the public paid ledger separately retains the startup failure from this source."
|
||||
"note": "Built from Pi source dd78df1e after foundation f063fbf2b, including bounded immutable snapshot copy, explicit candidate admission, exact completion instructions and paused-cost coverage. This probe executed the deployed generic installation registry and immutable snapshot. Every capability field was asserted from its actual initialize response. No prompt was sent.",
|
||||
"executionTrees": {
|
||||
"packages/paperclip-runner/src": "a8123cb4396c77ce0127d5ccf49d3584b63a9a58",
|
||||
"packages/paperclip-runner/scripts": "9ede97769bb3f2bbfa8e02be220b5ce06e438a99",
|
||||
"packages/paperclip-runner/runner": "2cc84e150aecbd5faf2c8bcecc4b6352c24dbca8",
|
||||
"packages/paperclip-runner/protocol": "1d50458fb572d0c75f9a398f419209c52c23ba6f",
|
||||
"patches": "1e2f846b8b2dda202b79ee83baeb0860a3b9ab64",
|
||||
"server/src": "b267adc1e0ad36955b68314829cfd4256ab88d37",
|
||||
"tests/runner-e2e": "5ff4b8a85a14e3e005e3e240c145899c0bc742b5"
|
||||
}
|
||||
},
|
||||
"environment": {
|
||||
"platform": "darwin",
|
||||
@@ -34,10 +43,10 @@
|
||||
"extensionDigest": "sha256:0a2ca138d6d87a85498bd982ee64543ea8b14f2ecceded96cfc27b0836cc4254"
|
||||
},
|
||||
"providerPack": {
|
||||
"manifestDigest": "sha256:2a6f457205743487cdd35ed3634532028e6e0200001f70e087d0e084fc85d452",
|
||||
"manifestDigest": "sha256:f75d3de7814276508f3706edb6e4510ce1dcecbc3e8fa674991fd554e5a75273",
|
||||
"portableNodeDigest": "sha256:27db838bb204ef7c21df2931f5656e4c8fb32e6e947f363a402b49714d32b5b1",
|
||||
"piNodeDigest": "sha256:27db838bb204ef7c21df2931f5656e4c8fb32e6e947f363a402b49714d32b5b1",
|
||||
"acpxSidecarDigest": "sha256:aa71619ea0f5e08d78c13bbca1812038d4ab6fee8550177be210b5afeb26d01a"
|
||||
"acpxSidecarDigest": "sha256:f51286992633bc54b6b169c255d3620146dfa975b24f22aae69b73994a234aa9"
|
||||
},
|
||||
"observed": {
|
||||
"closedSnapshotLaunch": true,
|
||||
|
||||
+112
-11
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"schema": "paperclip.pi-paid-qualification-progress.v1",
|
||||
"recordedAt": "2026-09-28T20:04:32.924765+00:00",
|
||||
"recordedAt": "2026-09-28T20:39:38.907308+00:00",
|
||||
"qualificationStatus": "pending",
|
||||
"environment": {
|
||||
"platform": "darwin",
|
||||
@@ -17,7 +17,7 @@
|
||||
"budget": {
|
||||
"authorizedMaximumUsd": 25,
|
||||
"reservedPerBatchUsd": 2,
|
||||
"measuredSpendUsd": 0.005748807,
|
||||
"measuredSpendUsd": 0.006755083,
|
||||
"measurement": "exclusive OpenRouter key billing delta; model catalog estimates are not provider billing"
|
||||
},
|
||||
"attempts": [
|
||||
@@ -259,6 +259,101 @@
|
||||
"outerDeadlineSeconds": 300,
|
||||
"configuredActiveTimeoutSeconds": 120
|
||||
}
|
||||
},
|
||||
{
|
||||
"attemptId": "pi-product-hello-05",
|
||||
"family": "product-e2e",
|
||||
"caseId": "extended-harnesses.runner-acpx-pi.local.hello-complete",
|
||||
"startedAt": "2026-09-28T20:19:17.958518+00:00",
|
||||
"finishedAt": "2026-09-28T20:35:14.566352+00:00",
|
||||
"durationSeconds": 72.168,
|
||||
"durationClock": "monotonic; host sleep does not advance this clock on this machine",
|
||||
"canonicalWallDurationMs": 921632,
|
||||
"sourceRevision": "dd78df1ef8b279c30c710c9b7a7f9fda22e321d6",
|
||||
"providerPackDigest": "sha256:f75d3de7814276508f3706edb6e4510ce1dcecbc3e8fa674991fd554e5a75273",
|
||||
"runnerdDigest": "sha256:f9aad049d57a2073ce4ec96b6577266afdba5b99db75bb8fc21825a5b7281e39",
|
||||
"profileDigest": "sha256:0f687e38cb3c607a01fab80f03e19bbbf5cb53a8afb1fc40d71f9ab54551cff1",
|
||||
"exitCode": 1,
|
||||
"retainedMachineDisposition": "secret_leak",
|
||||
"retainedInnerDisposition": "candidate_failure",
|
||||
"retainedStatus": "failed",
|
||||
"cleanup": "failed",
|
||||
"diagnosis": "Authenticated session and provider turn admission succeeded. The host then entered Maintenance Sleep for888s. After wake, the canonical issue/run wait timed out, browser/API cleanup failed, and the scanner rejected an uninspectable incomplete ZIP. The scanner reported no credential-value match; its fail-closed secret_leak classification is retained. No terminal or semantic-tool event was observed before capture ended.",
|
||||
"submittedProviderTurns": 1,
|
||||
"providerRequestCount": null,
|
||||
"billingDeltaUsd": 0.000351509,
|
||||
"billingCoverage": "exclusive OpenRouter key usage delta; two spaced post-attempt observations plus later unchanged reconciliation",
|
||||
"terminalUsage": null,
|
||||
"retainedCostCoverage": "unavailable",
|
||||
"automaticRetries": 0,
|
||||
"supervision": {
|
||||
"reservationUsd": 2,
|
||||
"billingWatchdogUsd": 0.5,
|
||||
"outerDeadlineSeconds": 300,
|
||||
"configuredActiveTimeoutSeconds": 120,
|
||||
"limitation": "Host sleep suspended the monotonic supervisor and billing polling; no assertion prevented idle sleep for this attempt."
|
||||
},
|
||||
"evidence": {
|
||||
"authenticatedSessionStarted": true,
|
||||
"sessionStartedAfterRunMs": 11631,
|
||||
"turnSubmitted": true,
|
||||
"turnAccepted": true,
|
||||
"uniqueRetainedEvents": 32,
|
||||
"semanticToolEvents": 0,
|
||||
"terminalEvents": 0,
|
||||
"reportedExactModelVerified": false,
|
||||
"partialModelMetadata": "unknown/openrouter",
|
||||
"pmsetSleepSeconds": 888,
|
||||
"billingWallObservationGapSeconds": 887.495,
|
||||
"zipScanReason": "zip could not be inspected: unzip exited with code9",
|
||||
"credentialValueMatchDetected": false,
|
||||
"exactAttemptProcessesRemaining": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"attemptId": "pi-product-hello-06",
|
||||
"family": "product-e2e",
|
||||
"caseId": "extended-harnesses.runner-acpx-pi.local.hello-complete",
|
||||
"startedAt": "2026-09-28T20:38:49.368264+00:00",
|
||||
"finishedAt": "2026-09-28T20:39:38.907308+00:00",
|
||||
"durationSeconds": 49.539,
|
||||
"wallDurationSeconds": 49.539,
|
||||
"sourceRevision": "dd78df1ef8b279c30c710c9b7a7f9fda22e321d6",
|
||||
"providerPackDigest": "sha256:f75d3de7814276508f3706edb6e4510ce1dcecbc3e8fa674991fd554e5a75273",
|
||||
"runnerdDigest": "sha256:f9aad049d57a2073ce4ec96b6577266afdba5b99db75bb8fc21825a5b7281e39",
|
||||
"profileDigest": "sha256:0f687e38cb3c607a01fab80f03e19bbbf5cb53a8afb1fc40d71f9ab54551cff1",
|
||||
"exitCode": 0,
|
||||
"retainedMachineDisposition": null,
|
||||
"retainedStatus": "passed",
|
||||
"cleanup": "passed",
|
||||
"diagnosis": "All six canonical hello matchers pass. Authenticated Pi executes paperclip_finish over semantic MCP; the provider turn completes, task becomes done, and native run succeeds. Sleep assertion and dual-clock supervision avoid the prior environmental interruption. USD remains unpriced; independent measured billing delta is separate.",
|
||||
"submittedProviderTurns": 1,
|
||||
"providerRequestCount": null,
|
||||
"billingDeltaUsd": 0.000654767,
|
||||
"billingCoverage": "exclusive OpenRouter key delta, two spaced post-attempt observations",
|
||||
"terminalUsage": {
|
||||
"inputTokens": 26867,
|
||||
"outputTokens": 283,
|
||||
"cachedInputTokens": 0,
|
||||
"usageSource": "per_run",
|
||||
"costStatus": "unpriced",
|
||||
"billingType": "unknown"
|
||||
},
|
||||
"retainedCostCoverage": "unpriced",
|
||||
"automaticRetries": 0,
|
||||
"supervision": {
|
||||
"reservationUsd": 2,
|
||||
"billingWatchdogUsd": 0.5,
|
||||
"activeTimeoutSeconds": 120,
|
||||
"outerTimeoutSeconds": 300,
|
||||
"outerClocks": [
|
||||
"wall",
|
||||
"monotonic"
|
||||
],
|
||||
"sleepAssertion": "caffeinate -i",
|
||||
"automaticRetries": 0
|
||||
},
|
||||
"evidence": "product-hello-proof.darwin-arm64.json"
|
||||
}
|
||||
],
|
||||
"observedSemanticOperations": {
|
||||
@@ -297,18 +392,24 @@
|
||||
"boundary": "Advisory native report does not mutate mock state or settle provider turn; stale revision, wrong turn, conflicting report and unauthorized create_task rejected"
|
||||
},
|
||||
"currentPiSource": {
|
||||
"source": "9f2d0420ed9a398e3bacd1d7e996614f3296ac56",
|
||||
"source": "dd78df1ef8b279c30c710c9b7a7f9fda22e321d6",
|
||||
"typescriptBuild": "passed",
|
||||
"releaseRunnerBuild": "passed",
|
||||
"vitestPassed": 97,
|
||||
"vitestPassed": 107,
|
||||
"vitestSkipped": 1,
|
||||
"installedPackageChecksPassed": 22,
|
||||
"packageCheckInvocation": "The initial command lacked isolated Pi package locations and failed module resolution; all 22 passed after selecting the freshly built pack packages explicitly.",
|
||||
"deployedPackCredentialFreeProbe": "passed",
|
||||
"fullCredentialFreeRunnerStartup": "terminal rejection within30s deadline; not authenticated success",
|
||||
"cleanDependencyResolution": "two independent runs from tracked manifest/config/patch inputs with pnpm9.15.4 --resolution-only --ignore-scripts --no-frozen-lockfile",
|
||||
"cleanDependencyLockSha256": "9c163adf40fe1c6a65abda6207bd9f5acfc8196bc94f81348b8bf1b2769e050e"
|
||||
"cleanDependencyLockSha256": "2c46a68811ba1d504a41b6f4d3f642bc93f4a1c615097754c9c6486881dba8e6"
|
||||
},
|
||||
"startupDiagnosis": "startup-diagnostic.darwin-arm64.json"
|
||||
"startupDiagnosis": "startup-diagnostic.darwin-arm64.json",
|
||||
"authenticatedModelNoPrompt": {
|
||||
"evidence": "model-admission-proof.darwin-arm64.json",
|
||||
"actualSelectedCurrentModelId": "openrouter/deepseek/deepseek-v4-flash-0731",
|
||||
"promptRequests": 0,
|
||||
"billingDeltaUsd": 0
|
||||
}
|
||||
},
|
||||
"publicationBoundary": {
|
||||
"hiddenReasoningIncluded": false,
|
||||
@@ -317,10 +418,10 @@
|
||||
"credentialsIncluded": false
|
||||
},
|
||||
"remaining": [
|
||||
"Successful protocol terminal settlement and complete usage receipt",
|
||||
"Successful Product E2E verified native backend admission and hello completion",
|
||||
"File edit, typed question, plan and restart journeys",
|
||||
"Native permissions and explicit steering acceptance/delivery",
|
||||
"Successful canonical Runner protocol completion",
|
||||
"File edit, typed question, plan and restart Product journeys",
|
||||
"Native permissions and active steering/follow-up delivery",
|
||||
"Complete price/billing coverage, separate from catalog estimates",
|
||||
"Linux x64 Daytona qualification"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,126 @@
|
||||
{
|
||||
"schema": "paperclip.pi-product-qualification-proof.v1",
|
||||
"attemptId": "pi-product-hello-06",
|
||||
"sourceRevision": "dd78df1ef8b279c30c710c9b7a7f9fda22e321d6",
|
||||
"providerPackDigest": "sha256:f75d3de7814276508f3706edb6e4510ce1dcecbc3e8fa674991fd554e5a75273",
|
||||
"runnerdDigest": "sha256:f9aad049d57a2073ce4ec96b6577266afdba5b99db75bb8fc21825a5b7281e39",
|
||||
"profileDigest": "sha256:0f687e38cb3c607a01fab80f03e19bbbf5cb53a8afb1fc40d71f9ab54551cff1",
|
||||
"executionTrees": {
|
||||
"packages/paperclip-runner/src": "a8123cb4396c77ce0127d5ccf49d3584b63a9a58",
|
||||
"packages/paperclip-runner/scripts": "9ede97769bb3f2bbfa8e02be220b5ce06e438a99",
|
||||
"packages/paperclip-runner/runner": "2cc84e150aecbd5faf2c8bcecc4b6352c24dbca8",
|
||||
"packages/paperclip-runner/protocol": "1d50458fb572d0c75f9a398f419209c52c23ba6f",
|
||||
"patches": "1e2f846b8b2dda202b79ee83baeb0860a3b9ab64",
|
||||
"server/src": "b267adc1e0ad36955b68314829cfd4256ab88d37",
|
||||
"tests/runner-e2e": "5ff4b8a85a14e3e005e3e240c145899c0bc742b5"
|
||||
},
|
||||
"environment": {
|
||||
"platform": "darwin",
|
||||
"architecture": "arm64",
|
||||
"execution": "local"
|
||||
},
|
||||
"caseId": "extended-harnesses.runner-acpx-pi.local.hello-complete",
|
||||
"status": "passed",
|
||||
"failureClass": null,
|
||||
"canonicalDurationMs": 32668,
|
||||
"supervisedDurationSeconds": 49.539,
|
||||
"supervisedWallDurationSeconds": 49.539,
|
||||
"agentRunDurationMs": 25621,
|
||||
"configuredModel": "openrouter/deepseek/deepseek-v4-flash-0731",
|
||||
"biller": "openrouter",
|
||||
"runtimeMode": "native",
|
||||
"matchers": [
|
||||
{
|
||||
"kind": "message_exact",
|
||||
"passed": true,
|
||||
"detail": "matched"
|
||||
},
|
||||
{
|
||||
"kind": "message_occurrences",
|
||||
"passed": true,
|
||||
"detail": "matched"
|
||||
},
|
||||
{
|
||||
"kind": "issue_status",
|
||||
"passed": true,
|
||||
"detail": "matched"
|
||||
},
|
||||
{
|
||||
"kind": "run_status",
|
||||
"passed": true,
|
||||
"detail": "matched"
|
||||
},
|
||||
{
|
||||
"kind": "runtime_mode",
|
||||
"passed": true,
|
||||
"detail": "matched"
|
||||
},
|
||||
{
|
||||
"kind": "environment",
|
||||
"passed": true,
|
||||
"detail": "matched"
|
||||
}
|
||||
],
|
||||
"semanticTools": [
|
||||
{
|
||||
"name": "mcp__paperclip__paperclip_finish",
|
||||
"result": "completed"
|
||||
}
|
||||
],
|
||||
"terminal": {
|
||||
"schema": "paperclip.prp.terminal.v1",
|
||||
"runTerminalState": "succeeded",
|
||||
"turnTerminalState": "completed",
|
||||
"reportedWorkDisposition": "done"
|
||||
},
|
||||
"usage": {
|
||||
"inputTokens": 26867,
|
||||
"outputTokens": 283,
|
||||
"cachedInputTokens": 0,
|
||||
"usageSource": "per_run",
|
||||
"costStatus": "unpriced",
|
||||
"billingType": "unknown"
|
||||
},
|
||||
"cost": {
|
||||
"providerReportedUsd": null,
|
||||
"catalogEstimateUsd": 0.00384062,
|
||||
"catalogEstimateProvenance": "Pi assistant message receipts priced by its model catalog; estimate notice explicitly marks billing unverified",
|
||||
"measuredExclusiveKeyDeltaUsd": 0.000654767,
|
||||
"canonicalBillingComplete": false,
|
||||
"billingCoverage": "two spaced post-attempt OpenRouter key usage observations"
|
||||
},
|
||||
"cleanup": "passed",
|
||||
"evidenceScan": {
|
||||
"leaks": [],
|
||||
"missing": []
|
||||
},
|
||||
"sourceModelVerification": {
|
||||
"mandatoryAdmissionGate": "requireVerifiedAcpxModel compares actual ACP currentModelId before accepting a prompt",
|
||||
"retainedProductModelMetadata": "unknown/openrouter at session.started; final usage carries configured exact model",
|
||||
"separateAuthenticatedNoPromptProof": "model-admission-proof.darwin-arm64.json"
|
||||
},
|
||||
"qualificationStatus": "pending",
|
||||
"supervision": {
|
||||
"reservationUsd": 2,
|
||||
"billingWatchdogUsd": 0.5,
|
||||
"activeTimeoutSeconds": 120,
|
||||
"outerTimeoutSeconds": 300,
|
||||
"outerClocks": [
|
||||
"wall",
|
||||
"monotonic"
|
||||
],
|
||||
"sleepAssertion": "caffeinate -i",
|
||||
"automaticRetries": 0
|
||||
},
|
||||
"publicationBoundary": {
|
||||
"credentialsIncluded": false,
|
||||
"privateReasoningIncluded": false,
|
||||
"rawTranscriptsIncluded": false,
|
||||
"providerSessionIdsIncluded": false
|
||||
},
|
||||
"limitations": [
|
||||
"This is one local hello-complete journey, not full provider qualification.",
|
||||
"Dollar usage remains unpriced in Product; independent key-delta billing is retained separately.",
|
||||
"File, question, plan, restart, active steering and Daytona are not proven by this case."
|
||||
]
|
||||
}
|
||||
+17
-2
@@ -63,8 +63,23 @@
|
||||
]
|
||||
},
|
||||
"limits": [
|
||||
"The post-fix timing measures actual immutable snapshot creation, not a full authenticated Runner session.",
|
||||
"The full post-fix credential-free Runner request settles within the unchanged deadline; this is not authenticated-session success.",
|
||||
"No paid prompt or provider request was issued by either diagnostic.",
|
||||
"Private process identifiers, credentials and raw traces are excluded."
|
||||
]
|
||||
],
|
||||
"postFixCredentialFreeRunnerProbe": {
|
||||
"sourceRevision": "dd78df1ef8b279c30c710c9b7a7f9fda22e321d6",
|
||||
"providerPackDigest": "sha256:f75d3de7814276508f3706edb6e4510ce1dcecbc3e8fa674991fd554e5a75273",
|
||||
"runnerdDigest": "sha256:f9aad049d57a2073ce4ec96b6577266afdba5b99db75bb8fc21825a5b7281e39",
|
||||
"requestSettledMs": 10087,
|
||||
"includingCloseMs": 10198,
|
||||
"sessionOpenDeadlineMs": 30000,
|
||||
"result": "terminal session_ensure_failed rejection",
|
||||
"timedOut": false,
|
||||
"underlyingErrorVisibility": "The outer PRP response does not preserve the missing-credential message. The independent immutable ACP launch from this exact pack asserted that message at session/new.",
|
||||
"authenticatedSessionSuccess": false,
|
||||
"credentials": "none",
|
||||
"promptCalls": 0,
|
||||
"providerCalls": 0
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user