mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 08:22:20 +02:00
docs: record host recovery and authenticated ACP qualification
Retain failed attempts, new local and Daytona passes, current runtime identities, and remaining production gates. Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
f939d369e6
commit
07369cc801
4 files changed
+361
-38
No files matched your search
@@ -22,7 +22,15 @@ v7 has paid Product hello passes on local source `807feaecf` and Daytona source
|
||||
The [qualification checkpoint](runner-rich-acp-qualification-2026-09-29.json)
|
||||
retains exact profile, runtime, image and billing identities. Remaining Product
|
||||
journeys, native interactions and platform coverage are still unqualified; the
|
||||
prepared `bd4cc29c3` local campaign has not launched. The canonical
|
||||
`bd4cc29c3` local campaign passed hello and stopped at native questions: all
|
||||
15 behavioral checks passed, but the required API evidence snapshot was absent.
|
||||
The failed result is retained and the six later cells did not launch. A committed
|
||||
harness-only fix captures the complete final API evidence; its paid retry passed
|
||||
all 15 native-question checks. The following question-continuation case reached
|
||||
Done but failed the exact final-message assertion: earlier pre-tool narration
|
||||
was aggregated into the final item. Three runs settled across these two cells,
|
||||
all owned processes exited, and five later cells did not launch. The native
|
||||
message attribution defect remains open. The canonical
|
||||
[v7 declaration](../../packages/paperclip-runner/test-fixtures/pi-acp/profile-v7-identity.json)
|
||||
binds the exact wrapper, helper, extension and three target closures.
|
||||
|
||||
@@ -611,8 +619,10 @@ same pinned PostgreSQL binary: `semget(..., 17, 03600)` fails with `No space lef
|
||||
on device` when only 16 semaphores are free. It exits in 0.557 seconds, removes
|
||||
its temporary directory, leaves no child process, and leaves IPC totals unchanged.
|
||||
The original Product bootstrap stderr remains unavailable. No unrelated database
|
||||
or IPC object was changed. Further paid
|
||||
qualification is held until a supported database resource path is available.
|
||||
or IPC object was changed during that diagnosis. The later user-approved
|
||||
fixed-snapshot host cleanup removed 4,892 stale semaphore sets and skipped 11
|
||||
whose recorded creator PID was present. Startup and graceful shutdown now pass;
|
||||
the current checkpoint above supersedes this historical qualification hold.
|
||||
|
||||
|
||||
The [sanitized attempt ledger](../../packages/paperclip-runner/test-fixtures/pi-acp/paid-qualification-progress.darwin-arm64.json)
|
||||
|
||||
@@ -17,13 +17,15 @@ bound MCP names to provider-safe aliases, and rejects malformed native UI
|
||||
requests explicitly. Its four native question forms use an owned tool.
|
||||
|
||||
Credential-free tests cover the new policy boundaries, real pinned native
|
||||
processes, clean runtime builds and Product E2E assertions. Pi v7 has passed its
|
||||
paid local hello case. Copilot v4 has passed local hello, question continuation,
|
||||
plan approval and controller-restart recovery on the integrated `bd4cc29c3`
|
||||
runtime; file-edit qualification failed during session startup. Pi's corrected Daytona hello also passed. Its first Daytona attempt created and
|
||||
retired one sandbox but failed before inference because the launcher omitted the
|
||||
build-owned remote artifact paths. Remaining
|
||||
authenticated cases, platform coverage and Daytona Product E2E are still required.
|
||||
processes, clean runtime builds and Product E2E assertions. On integrated source
|
||||
`bd4cc29c3`, Cursor v4 has passed local hello, and Copilot v4 has passed all five
|
||||
basic local journeys: hello, semantic questions, semantic plans, controller
|
||||
restart and file edit/validation. Pi v7 has passed local hello on `807feaecf` and
|
||||
Daytona hello on `bd4cc29c3`. Copilot also passed Daytona hello. Pi’s repaired
|
||||
native-question evidence case passed 15/15; the following continuation case
|
||||
exposed final-message aggregation of earlier narration.
|
||||
These passes retain earlier failures and do not qualify native interactions,
|
||||
all required platforms or the remaining remote journeys.
|
||||
|
||||
The [harness priorities report](https://pages.paperclip.ing/2026-09-25-harness-priorities/)
|
||||
recommends Cursor and Copilot, followed by Pi, using the existing qualified ACPX
|
||||
@@ -46,10 +48,10 @@ candidate scorecards from historical runtime policies.
|
||||
|
||||
| Current candidate | Latest observed result | Remaining work |
|
||||
| --- | --- | --- |
|
||||
| Cursor v4, source `2e0b0fec0` | All three closures verify. Native instruction acknowledgement is enforced on new/load and automatic reload. Bounded todo projection preserves all IDs/statuses and marks shortened details. Exact-head Apex reports 5/5 | Authenticated current-profile Product tests, remaining platform and Daytona proof; paid tests await the account spend-limit decision |
|
||||
| Copilot v4, source `2042ca14c` | Fresh/load model-request capture proves native instructions reach the executable. Lifetime ownership prevents instruction-write races. Native session notices preserve bounded fields without inventing originating-turn attribution. Exact-head Apex reports 5/5 | The continuation fix is paid-tested on `bd4cc29c3`: hello, semantic question/answer, plan approval and controller restart passed (six matchers each). File editing failed during `session.open`; diagnose before retrying. Remote and remaining platform qualification are open |
|
||||
| Pi v7, sources `807feaecf` and `bd4cc29c3` | Local hello on `807feaecf` and Daytona hello on `bd4cc29c3` each passed all six matchers with one provider run, zero retries and complete owned cleanup. Delayed local billing was $0.000646464; remote model plus sandbox analytics stabilized at $0.0053185282. Apex reports 5/5 on the reviewed provider head | The eight-case local `bd4cc29c3` campaign remains held at the host database blocker. Remaining remote, native-interaction and platform cases plus final-source checks are required. Historical receipts retain their original retry policies; new supervisors explicitly enforce zero retries |
|
||||
| All three, Linux/Daytona | The immutable `bd4cc29c3` image and actual server artifact admission pass. Pi hello attempt 02 passed all six matchers, followed by complete owned cleanup and stable delayed billing; failed attempt 01 remains retained | Resolve host database startup pressure, then qualify the remaining remote cases; Copilot's first remote attempt never reached fixture creation |
|
||||
| Cursor v4, source `2e0b0fec0`; paid runtime `bd4cc29c3` | All three closures verify; native instructions and bounded todos are implemented. Exact-head Apex reports 5/5. Current-profile local hello passed six matchers with one authenticated run, complete process cleanup and unchanged semaphore count | The question-resume case failed reopening its provider session; the following three cases were not launched. A rebuilt `627451ecf` admission fix is being paid-tested. Native callbacks, restrictive permissions, other platforms and Daytona remain unqualified. The approved account on-demand cap is $25 |
|
||||
| Copilot v4, source `2042ca14c`; paid runtime `bd4cc29c3` | Native instruction delivery and lifetime ownership have deterministic proof. Exact-head Apex reports 5/5. All five basic local journeys passed; file edit/validation passed seven matchers in one run with complete cleanup | Daytona hello also passed six matchers with complete cleanup. Native permissions, attached background-command settlement, remaining remote workflows and other platforms remain open. The original session startup timeout is retained without a proven root cause |
|
||||
| Pi v7, sources `807feaecf` and `bd4cc29c3` | Local hello on `807feaecf` and Daytona hello on `bd4cc29c3` passed six matchers each with zero retries and complete owned cleanup. Remote model plus sandbox analytics stabilized at $0.0053185282. Apex reports 5/5 on the reviewed provider head | The `bd4cc29c3` campaign passed hello, then failed packaging native-question evidence despite 15 passing behavioral checks. The harness-only repair subsequently passed all 15 native checks. The next continuation case failed final-message aggregation; five later cells did not launch. Remote/platform evidence remains incomplete |
|
||||
| All three, Linux/Daytona | The immutable `bd4cc29c3` image and actual server artifact admission pass. Pi hello attempt 02 passed; failed attempt 01 remains retained | The host database resource blocker is repaired. Copilot’s retry passed six matchers and its exact sandbox remained absent for the full 360-second observation. Remaining remote cases are unqualified |
|
||||
|
||||
Cursor and Copilot ignore generic ACP `_meta.systemPrompt`; setting `AGENT_HOME`
|
||||
in the process environment does not make it part of model instructions. The v4
|
||||
@@ -150,14 +152,60 @@ was absent throughout the full cleanup window and all owned processes retired.
|
||||
Three delayed reads stabilized at $0.000630644 OpenRouter usage plus $0.0046878842
|
||||
sandbox analytics ($0.0053185282 combined, provisional provider accounting).
|
||||
|
||||
Further live tests are paused because a credential-free reproduction confirmed
|
||||
host semaphore exhaustion during embedded PostgreSQL bootstrap (`semget` returned
|
||||
`ENOSPC`). Copilot's diagnostic local and first Daytona attempts failed before
|
||||
provider execution. The Daytona invocation never reached fixture creation or a
|
||||
cloud-write path; its machine cleanup flag remains `unresolved_scope_not_captured`
|
||||
and is retained separately from that source-order assessment. This host failure
|
||||
does not explain the earlier Copilot provider-session startup timeout. The
|
||||
remaining Pi local/native campaign is prepared but has not launched.
|
||||
Host semaphore exhaustion was confirmed by a credential-free PostgreSQL
|
||||
bootstrap reproduction (`semget` returned `ENOSPC`). With the user's approval,
|
||||
a fixed-snapshot cleanup removed 4,892 stale sets (83,164 semaphores) and skipped
|
||||
11 whose recorded creator PID was present. Startup and graceful shutdown now
|
||||
pass. Private qualification supervisors were also repaired to let the owned
|
||||
server shut down PostgreSQL before terminating launcher wrappers. Paid Cursor
|
||||
hello and Copilot file runs each returned to the 255-set baseline without a
|
||||
forced PostgreSQL kill; no provider runtime bytes changed.
|
||||
|
||||
Cursor hello attempt 01 stopped in the private supervisor before provider
|
||||
launch. A process-inspection race was reproduced and repaired; attempt 02 passed
|
||||
six matchers with one run. Its matching account usage row is labeled Free (24K
|
||||
tokens), while native token and dollar receipts remain unavailable. Copilot's
|
||||
new file diagnostic passed seven matchers, exact file bytes and validation in
|
||||
one run; its included-credit counter moved from 11 to 12 with additional usage
|
||||
disabled. An initial workspace-only citation was rejected, then the model
|
||||
registered its deliverable and completed in the same run. This is not a retry.
|
||||
|
||||
Earlier Copilot local and first Daytona failures remain preserved. The remote
|
||||
invocation never reached fixture creation or a cloud-write path; its machine
|
||||
cleanup flag remains `unresolved_scope_not_captured`, separately from that
|
||||
source-order assessment. Host exhaustion does not explain the earlier provider
|
||||
session timeout. The Pi local/native campaign stopped at its second cell: hello passed, and all
|
||||
15 native-question behavioral checks passed, but a required API snapshot was
|
||||
missing. The failed Product result remains retained; the six later cells did
|
||||
not launch. A minimal harness-only snapshot repair and seven focused regressions
|
||||
are committed. The two calls cost $0.004005496 by delayed exclusive-key delta.
|
||||
Cursor’s subsequent question-resume test also stopped its batch on a provider
|
||||
reopen failure; all 56 owned processes exited and semaphore counts returned to
|
||||
baseline. Its matching account row is Free (24.7K tokens). A credential-free reproduction shows that a fresh ACPX manager can return a
|
||||
saved Cursor record without loading the native provider. The committed repair
|
||||
forces an exact-model control to load that session and acknowledge current
|
||||
instructions within the existing admission deadline, then renews the consumed
|
||||
launch lease. Missing or incorrect acknowledgements still block prompting.
|
||||
The original private sidecar stack is unavailable; this is corroborating
|
||||
reproduction evidence. The fix passed 93 focused tests and package typecheck;
|
||||
the rebuilt `627451ecf` runtime is now in a bounded paid continuation batch.
|
||||
Later cases from the failed batch were not launched.
|
||||
|
||||
Copilot Daytona hello attempt 02 passed six matchers with one run. All 45 owned
|
||||
processes retired and the exact owned sandbox remained absent throughout the
|
||||
360-second cleanup observation. Three delayed sandbox analytics reads stabilized
|
||||
at $0.0039682144; this remains provisional infrastructure accounting, separate
|
||||
from unavailable native per-run dollars and the account’s unchanged included-credit
|
||||
counter. Additional Copilot usage remains disabled.
|
||||
|
||||
Pi’s native-question retry on unchanged `bd4cc29c3` runtime and separate
|
||||
`43c99f044` eval source passed 15/15 with complete required evidence. The next
|
||||
question continuation reached Done with two succeeded runs, but failed one of
|
||||
six matchers because the final item included earlier pre-tool narration. Retained
|
||||
post-tool text deltas alone match the expected final marker. Both cells fully
|
||||
settled, all 96 owned process identities retired across the two cells, and delayed
|
||||
exclusive-key charges totaled $0.005707808. Five later cells did not launch;
|
||||
the final-message defect remains under investigation.
|
||||
|
||||
GitHub billing coverage was also audited: public standard-runner and self-hosted
|
||||
GitHub minute charges do not add a per-minute charge, but external fleet costs
|
||||
|
||||
@@ -1,16 +1,16 @@
|
||||
{
|
||||
"schema": "paperclip.rich-acp-qualification-checkpoint.v1",
|
||||
"updatedAt": "2026-09-29T18:26:17.961300+00:00",
|
||||
"updatedAt": "2026-09-29T20:58:14.579734+00:00",
|
||||
"qualificationStatus": "pending",
|
||||
"runtimeSource": "bd4cc29c3017ed5e1484423e842fd48e3b2f49f3",
|
||||
"modelReceiptPolicy": "Native per-run USD remains null/unpriced when absent. Included subscription credits, exclusive API-key charges and sandbox analytics are separate evidence. Unknown costs are never assumed zero.",
|
||||
"evidenceScope": "Sanitized checkpoint of retained private receipts; historical reports retain earlier attempts. This file is not an executable conformance result or production qualification certificate.",
|
||||
"currentProfilePaidCoverage": {
|
||||
"cursor4": "No current-profile paid Product receipt. Historical local successes used Cursor2. Account spend-limit decision pending.",
|
||||
"copilot4": "bd4 local hello, semantic question, semantic plan and controller-restart passed. File startup timeout unresolved; two diagnostic attempts failed before provider launch because private DB initialization failed. No successful Daytona Product run.",
|
||||
"pi7": "807 local hello and bd4 Daytona hello passed. Current bd4 local8 campaign is reserved but not launched."
|
||||
"cursor4": "bd4 local hello passed. Cold continuation failed; corroborating offline reproduction led to source627451ecf session-load repair. The rebuilt four-cell paid batch is running, with no result yet. Native callbacks, restrictive permissions, Daytona and remaining platforms are open.",
|
||||
"copilot4": "bd4 all five basic local journeys and Daytona hello passed. Native permission denial and attached background settlement Product cases plus remaining remote/platform coverage are open. The original session.open timeout remains unexplained.",
|
||||
"pi7": "Local and Daytona hello passed. After preserving an earlier missing-evidence failure, the harness-only repair on runtimebd4/eval43c99f0 produced a complete native-question pass15/15. The next question-continuation cell failed5/6: final-message aggregation included pre-tool narration. All three runs settled and owned cleanup passed; five later cells were not launched."
|
||||
},
|
||||
"hostBlocker": "Credential-free exact initdb reproduced semget ENOSPC; kernel semaphore pool nearly exhausted. No global IPC changes performed. Paid launches paused.",
|
||||
"hostBlocker": "Resolved for live testing: user-approved fixed-snapshot cleanup removed 4,892 stale PostgreSQL semaphore sets, skipping 11 whose creator PID was present. Credential-free startup and role-aware shutdown passed; subsequent paid Cursor and Copilot runs restored the 255-set baseline. No broad follow-up IPC deletion is authorized.",
|
||||
"ciCostCoverage": "Public standard GitHub-hosted minutes and self-hosted GitHub runner charges are zero; external self-hosted fleet infrastructure costs and private evals CI incremental charges remain unmetered.",
|
||||
"attempts": [
|
||||
{
|
||||
@@ -255,7 +255,11 @@
|
||||
{
|
||||
"id": "pi-v7-local-bd4-campaign-20260929-01",
|
||||
"provider": "pi",
|
||||
"status": "reserved_waiting_for_copilot_product_shutdown",
|
||||
"status": "failed_second_cell_evidence_packaging",
|
||||
"sourceRevision": "bd4cc29c3017ed5e1484423e842fd48e3b2f49f3",
|
||||
"profileVersion": 7,
|
||||
"profileDigest": "sha256:fec5b1448f4135710887133a9192747c476a825a56c255b52b17b1780ccb3761",
|
||||
"model": "openrouter/deepseek/deepseek-v4-flash-0731",
|
||||
"cells": [
|
||||
"extended-harnesses.runner-acpx-pi.local.hello-complete",
|
||||
"pi-native.runner-acpx-pi.local.native-questions",
|
||||
@@ -266,13 +270,243 @@
|
||||
"pi-native.runner-acpx-pi.local.agent-files-fresh-run",
|
||||
"pi-native.runner-acpx-pi.local.restrictive-denial"
|
||||
],
|
||||
"maxRetries": 0,
|
||||
"createdAt": "2026-09-29T18:23:16.046245+00:00",
|
||||
"actualCostUsd": 0.004005496,
|
||||
"reconciledAt": "2026-09-29T20:31:33.209288+00:00",
|
||||
"costCoverage": "Exclusive OpenRouter key delayed180s delta: hello .000637804 plus nativequestions .003367692; native perrunUSD remains unpriced.",
|
||||
"attemptCount": 2,
|
||||
"productRunCount": 2,
|
||||
"ownedProcessesRetired": true,
|
||||
"postRunAuthorityPassed": true,
|
||||
"observedAutomaticRetries": 0,
|
||||
"completedCells": 2,
|
||||
"unlaunchedCells": 6,
|
||||
"failure": "Native questions completed15/15 behavioral assertions but required snapshots/api-state.json missing; Product outcome failed and campaign stopped. Hello passed6/6.",
|
||||
"privateEvidenceName": "campaign.json",
|
||||
"privateEvidenceSha256": "e095503ab4b32eaabf34db807d41454fcd3417796bbbcdd9033167092c984383"
|
||||
},
|
||||
{
|
||||
"provider": "copilot",
|
||||
"sourceRevision": "bd4cc29c3017ed5e1484423e842fd48e3b2f49f3",
|
||||
"profileVersion": 4,
|
||||
"profileDigest": "sha256:a119e436b4ad13ee70e1f58bedad4a0f20761b5fd982b752981f0cb1bc65f797",
|
||||
"providerPackDigest": "sha256:c19135b9175fffe9b18c06ca735ba1610e6cb2e36305c760d763c1f41fa67d81",
|
||||
"model": "gpt-5.6-luna",
|
||||
"cell": "extended-harnesses.runner-acpx-copilot.local.file-edit-validate",
|
||||
"id": "copilot-v4-local-file-20260929-02",
|
||||
"status": "passed",
|
||||
"actualCostUsd": null,
|
||||
"createdAt": "2026-09-29T20:07:12.133150+00:00",
|
||||
"maxRetries": 0,
|
||||
"attemptCount": 1,
|
||||
"includedCreditDelta": 1,
|
||||
"ownedProcessesRetired": true,
|
||||
"ownedProcessCount": 55,
|
||||
"postRunAuthorityPassed": true,
|
||||
"costCoverage": "GitHub includedcredit11->12; additionalusage remainsdisabled$0/$0; nativeperrunUSDunknown",
|
||||
"reconciledAt": "2026-09-29T20:18:07.543888+00:00",
|
||||
"productRunCount": 1,
|
||||
"matcherPassCount": 7,
|
||||
"matcherCount": 7,
|
||||
"privateEvidenceName": "review-receipt.private.json",
|
||||
"privateEvidenceSha256": "c2699ecf375181f9722652535f54db6712117a3b4e0708d0c44e6500034edcf1"
|
||||
},
|
||||
{
|
||||
"provider": "cursor",
|
||||
"profileVersion": 4,
|
||||
"profileDigest": "sha256:b1440d559ebc4eef5c7a582f1c81fc153270cfbafa1731a8ee76d83713bdf61b",
|
||||
"sourceRevision": "bd4cc29c3017ed5e1484423e842fd48e3b2f49f3",
|
||||
"model": "gpt-5.6-luna[context=272k,reasoning=medium,fast=false]",
|
||||
"id": "cursor-v4-local-hello-20260929-01",
|
||||
"status": "failed_supervisor_before_provider",
|
||||
"actualCostUsd": null,
|
||||
"createdAt": "2026-09-29T20:08:34.490882+00:00",
|
||||
"cell": "extended-harnesses.runner-acpx-cursor.local.hello-complete",
|
||||
"maxRetries": 0,
|
||||
"attemptCount": 1,
|
||||
"productRunCount": 0,
|
||||
"ownedProcessesRetired": true,
|
||||
"failure": "Process monitor CalledProcessError during PG startup; agent is confirming ps process-exit race. No Product run/browser/provider reached; root awaits complete failure proof before assigning actual cost0.",
|
||||
"reconciledAt": "2026-09-29T20:10:39.989926+00:00",
|
||||
"privateEvidenceName": "cursor-v4-bd4-hello-01"
|
||||
},
|
||||
{
|
||||
"provider": "cursor",
|
||||
"profileVersion": 4,
|
||||
"profileDigest": "sha256:b1440d559ebc4eef5c7a582f1c81fc153270cfbafa1731a8ee76d83713bdf61b",
|
||||
"sourceRevision": "bd4cc29c3017ed5e1484423e842fd48e3b2f49f3",
|
||||
"model": "gpt-5.6-luna[context=272k,reasoning=medium,fast=false]",
|
||||
"id": "cursor-v4-local-hello-20260929-02",
|
||||
"status": "passed",
|
||||
"actualCostUsd": null,
|
||||
"createdAt": "2026-09-29T20:13:46.584978+00:00",
|
||||
"cell": "extended-harnesses.runner-acpx-cursor.local.hello-complete",
|
||||
"maxRetries": 0,
|
||||
"attemptCount": 1,
|
||||
"productRunCount": 1,
|
||||
"matcherPassCount": 6,
|
||||
"matcherCount": 6,
|
||||
"ownedProcessesRetired": true,
|
||||
"ownedProcessCount": 47,
|
||||
"postRunAuthorityPassed": true,
|
||||
"reconciledAt": "2026-09-29T20:18:07.543888+00:00",
|
||||
"accountObservedChargeUsd": 0,
|
||||
"costCoverage": "Matching24Kmodelrow is explicitlyFree in Cursor account; native token/cost receipt is unavailable. Do not fabricate native priced usage.",
|
||||
"privateEvidenceName": "analysis.private.json",
|
||||
"privateEvidenceSha256": "e558d1dbde30f49fa09f0519de34e2eddd65694ce3ab8aadeb71b79040de249c"
|
||||
},
|
||||
{
|
||||
"id": "cursor-v4-local-remaining-20260929-01",
|
||||
"provider": "cursor",
|
||||
"status": "failed_first_cell_continuation",
|
||||
"cells": [
|
||||
"extended-harnesses.runner-acpx-cursor.local.question-resume-complete",
|
||||
"extended-harnesses.runner-acpx-cursor.local.plan-approve-complete",
|
||||
"extended-harnesses.runner-acpx-cursor.local.structured-question-restart-resume",
|
||||
"extended-harnesses.runner-acpx-cursor.local.file-edit-validate"
|
||||
],
|
||||
"sourceRevision": "bd4cc29c3017ed5e1484423e842fd48e3b2f49f3",
|
||||
"profileVersion": 4,
|
||||
"profileDigest": "sha256:b1440d559ebc4eef5c7a582f1c81fc153270cfbafa1731a8ee76d83713bdf61b",
|
||||
"model": "gpt-5.6-luna[context=272k,reasoning=medium,fast=false]",
|
||||
"createdAt": "2026-09-29T20:28:30.857363+00:00",
|
||||
"maxRetries": 0,
|
||||
"actualCostUsd": null,
|
||||
"attemptCount": 1,
|
||||
"completedCells": 1,
|
||||
"unlaunchedCells": 3,
|
||||
"ownedProcessesRetired": true,
|
||||
"ownedProcessCount": 56,
|
||||
"postRunAuthorityPassed": true,
|
||||
"failure": "Question continuation run.attach failed: ACPX sidecar session.open rejected; cause under investigation. Subsequent3cells not launched.",
|
||||
"reconciledAt": "2026-09-29T20:33:09.782788+00:00",
|
||||
"accountObservedChargeUsd": 0,
|
||||
"costCoverage": "Matching20:29UTC gpt5.6-luna-medium accountrow is Free24.7Ktokens; accountonDemand0. Native perrunUSD unavailable, not fabricatedzero."
|
||||
},
|
||||
{
|
||||
"id": "copilot-v4-daytona-hello-20260929-02",
|
||||
"provider": "copilot",
|
||||
"status": "passed",
|
||||
"cell": "extended-harnesses.runner-acpx-copilot.daytona.hello-complete",
|
||||
"sourceRevision": "bd4cc29c3017ed5e1484423e842fd48e3b2f49f3",
|
||||
"profileVersion": 4,
|
||||
"profileDigest": "sha256:a119e436b4ad13ee70e1f58bedad4a0f20761b5fd982b752981f0cb1bc65f797",
|
||||
"model": "gpt-5.6-luna",
|
||||
"image": "ghcr.io/paperclipai/paperclip-daytona-runner@sha256:bff4c3f291087a0eeae37e4c20dd51857b92833eaf73ba3aca4157f37de1e109",
|
||||
"createdAt": "2026-09-29T20:31:06.298515+00:00",
|
||||
"maxRetries": 0,
|
||||
"actualCostUsd": null,
|
||||
"reconciledAt": "2026-09-29T20:43:18.535492+00:00",
|
||||
"attemptCount": 1,
|
||||
"productRunCount": 1,
|
||||
"providerInferenceTurns": 1,
|
||||
"matcherPassCount": 6,
|
||||
"matcherCount": 6,
|
||||
"ownedProcessesRetired": true,
|
||||
"ownedProcessCount": 45,
|
||||
"ownedSandboxCount": 1,
|
||||
"cleanupStatus": "observed_absent_for_360_seconds",
|
||||
"postRunAuthorityPassed": true,
|
||||
"observedAutomaticRetries": 0,
|
||||
"nativeUsd": null,
|
||||
"costCoverage": "Exact owned sandbox delayed analytics$.0039682144. Copilot nativeUSDunknown; subscriptioncounter12->12 and additionalusage disabled0/0; accountcounter may lag. Source-bound720s infraupperbound$.053496 met. Runtimeestimate is separate from billing.",
|
||||
"daytonaObservedCostUsd": 0.0039682144,
|
||||
"costFinality": "Provisional sandbox-specific analytics stable across three delayed reads; invoice finality unknown.",
|
||||
"privateEvidenceName": "review-receipt.private.json",
|
||||
"privateEvidenceSha256": "2be810d807d12e35c237feb0b60d71b74e27727336f4c6a904a6dbbc0bfd2bab"
|
||||
},
|
||||
{
|
||||
"id": "pi-v7-local-bd4-campaign-20260929-02",
|
||||
"provider": "pi",
|
||||
"status": "failed_second_cell_final_message_projection",
|
||||
"createdAt": "2026-09-29T20:44:04.035168+00:00",
|
||||
"sourceRevision": "43c99f04403c0e1e5353b41b41dcab8fdade6a2f",
|
||||
"runtimeSourceRevision": "bd4cc29c3017ed5e1484423e842fd48e3b2f49f3",
|
||||
"evalSourceRevision": "43c99f04403c0e1e5353b41b41dcab8fdade6a2f",
|
||||
"evalPatchCommit": "7e7563ca8b8f351b74b922f8c06aee612dfc84fa",
|
||||
"evalPatchSha256": "18119aef75b5f5a13aa10c1c9c7f7a761b13257dd67d395ec575842663cd8266",
|
||||
"runtimeBuildPerformed": false,
|
||||
"cells": [
|
||||
"native-questions",
|
||||
"question-resume-complete",
|
||||
"plan-approve-complete",
|
||||
"structured-question-restart-resume",
|
||||
"file-edit-validate",
|
||||
"agent-files-fresh-run",
|
||||
"restrictive-denial"
|
||||
],
|
||||
"expectedProductRuns": 11,
|
||||
"model": "openrouter/deepseek/deepseek-v4-flash-0731",
|
||||
"profileVersion": 7,
|
||||
"profileDigest": "sha256:fec5b1448f4135710887133a9192747c476a825a56c255b52b17b1780ccb3761",
|
||||
"model": "openrouter/deepseek/deepseek-v4-flash-0731",
|
||||
"createdAt": "2026-09-29T18:23:16.046245+00:00",
|
||||
"actualCostUsd": null,
|
||||
"maxRetries": 0
|
||||
"providerPackDigest": "sha256:c19135b9175fffe9b18c06ca735ba1610e6cb2e36305c760d763c1f41fa67d81",
|
||||
"maxRetries": 0,
|
||||
"actualCostUsd": 0.005707808,
|
||||
"completedCells": 2,
|
||||
"unlaunchedCells": 5,
|
||||
"productRunCount": 3,
|
||||
"attemptCount": 1,
|
||||
"inferenceKeyDeltaUsd": 0.005707808,
|
||||
"ownedProcessesRetired": true,
|
||||
"postRunAuthorityPassed": true,
|
||||
"reconciledAt": "2026-09-29T20:56:40.420584+00:00",
|
||||
"failure": "Question continuation reached Done with two succeeded native runs, but final item included earlier pre-tool narration; exact final response matcher failed. Retained deltas after the last tool concatenate to the exact expected marker. Five later cells were not launched.",
|
||||
"costCoverage": "Exclusive OpenRouter key delta for two cells after 0/30/90/180s settlement; native per-run USD remains unpriced.",
|
||||
"settledCells": [
|
||||
{
|
||||
"cell": "native-questions",
|
||||
"status": "passed",
|
||||
"matcherPassCount": 15,
|
||||
"matcherCount": 15,
|
||||
"productRunCount": 1,
|
||||
"exclusiveKeyDeltaUsd": 0.003543848,
|
||||
"ownedProcessesRetired": true,
|
||||
"ownedProcessCount": 44,
|
||||
"postRunAuthorityPassed": true
|
||||
},
|
||||
{
|
||||
"cell": "question-resume-complete",
|
||||
"status": "candidate_failure",
|
||||
"matcherPassCount": 5,
|
||||
"matcherCount": 6,
|
||||
"productRunCount": 2,
|
||||
"exclusiveKeyDeltaUsd": 0.00216396,
|
||||
"ownedProcessesRetired": true,
|
||||
"ownedProcessCount": 52,
|
||||
"postRunAuthorityPassed": true
|
||||
}
|
||||
],
|
||||
"privateEvidenceName": "campaign.json",
|
||||
"privateEvidenceSha256": "d0bb6e7045dad5ba44162d66dd3ec6db6cd31e9fbd4c88e926e99b05b4e06ad9"
|
||||
},
|
||||
{
|
||||
"id": "cursor-v4-local-627-remaining-20260929-01",
|
||||
"provider": "cursor",
|
||||
"status": "authorized",
|
||||
"createdAt": "2026-09-29T20:56:40.420584+00:00",
|
||||
"sourceRevision": "627451ecf15641504a485c6f868497177bd1bce8",
|
||||
"profileVersion": 4,
|
||||
"profileDigest": "sha256:b1440d559ebc4eef5c7a582f1c81fc153270cfbafa1731a8ee76d83713bdf61b",
|
||||
"model": "gpt-5.6-luna[context=272k,reasoning=medium,fast=false]",
|
||||
"cells": [
|
||||
"extended-harnesses.runner-acpx-cursor.local.question-resume-complete",
|
||||
"extended-harnesses.runner-acpx-cursor.local.plan-approve-complete",
|
||||
"extended-harnesses.runner-acpx-cursor.local.structured-question-restart-resume",
|
||||
"extended-harnesses.runner-acpx-cursor.local.file-edit-validate"
|
||||
],
|
||||
"expectedProductRuns": 7,
|
||||
"maxRetries": 0,
|
||||
"actualCostUsd": null
|
||||
}
|
||||
]
|
||||
],
|
||||
"hostRecovery": {
|
||||
"removedSets": 4892,
|
||||
"removedSemaphores": 83164,
|
||||
"skippedSets": 11,
|
||||
"cleanupResultSha256": "105ad0e67c74078f8d3b62784ba08151712646c07289f93e7907fd0f306410ca",
|
||||
"startupAndShutdownProofSha256": "0f9e42e295d87cbf45731be089f65bc9cd6d1a26e185ab1847f3a7d2d0291fbe",
|
||||
"currentSupervisorHelperSha256": "c3c801ef7dc3ef969384dd9bb6d822fe6ab21025d40a4702abfdd9ee392d1148",
|
||||
"scope": "The credential-free startup proof used an earlier role-aware helper revision. Current-helper cleanup is additionally proven by the paid Cursor hello and Copilot file runs. Private qualification supervisors changed; frozen runtime bytes did not."
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"schema": "paperclip.rich-acp-validation-checkpoint.v1",
|
||||
"updatedAt": "2026-09-29T18:52:19.212615+00:00",
|
||||
"updatedAt": "2026-09-29T20:58:14.579734+00:00",
|
||||
"qualificationStatus": "pending",
|
||||
"runtimeSource": "bd4cc29c3017ed5e1484423e842fd48e3b2f49f3",
|
||||
"validationSourcePatches": [
|
||||
@@ -147,9 +147,40 @@
|
||||
"otherStageReceiptSha256": "024be9b0a264f97d46ecaa724e72fed17ffd3c6380303dbfdc6a4596db2d5f56"
|
||||
},
|
||||
"qualificationBlockers": [
|
||||
"Host PostgreSQL semaphore exhaustion; stale-object cleanup awaits user authorization.",
|
||||
"Cursor account spending-limit decision pending.",
|
||||
"Remaining authenticated local/native/Daytona cases and platform coverage; Copilot file startup timeout unresolved.",
|
||||
"Complete infrastructure cost coverage and current final-head CI/review remain required."
|
||||
"Remaining native/local/Daytona workflows and macOS x64 execution coverage; Pi final-message aggregation failure is under repair.",
|
||||
"A complete green repository test run is still missing; the retained full run failed and has not been repeated after host recovery.",
|
||||
"Shared prerequisite CI has a failed server job; its bounded retry was canceled. Current final-head CI/review remains required.",
|
||||
"Complete infrastructure cost coverage remains unavailable; native per-run USD is unpriced for the subscription providers."
|
||||
],
|
||||
"hostRecovery": {
|
||||
"status": "complete",
|
||||
"removedStaleSets": 4892,
|
||||
"skippedBecauseCreatorPidPresent": 11,
|
||||
"startupAndGracefulShutdown": "passed",
|
||||
"currentPaidCleanup": "Cursor hello and Copilot file each returned to the 255-set baseline",
|
||||
"frozenRuntimeModified": false
|
||||
},
|
||||
"sharedPrerequisiteCi": {
|
||||
"recordedAt": "2026-09-29T20:26:11.895912+00:00",
|
||||
"sourceRevision": "11da146c3c39aea5605fdfac31e619fd8024227b",
|
||||
"jobId": 109571559222,
|
||||
"jobConclusion": "failure",
|
||||
"testStepConclusion": "cancelled",
|
||||
"classification": "runner_shutdown_during_test_step",
|
||||
"evidence": "19:09:17Z: The runner has received a shutdown signal; 19:09:29Z: The operation was canceled.",
|
||||
"logSha256": "cb927b9d77810c5c38f9d5ff22adf3f1daa8f5d9bc35f967c330eb8d2ee6b8cf",
|
||||
"newTestFailureEstablished": false,
|
||||
"additionalRetryRequested": false
|
||||
},
|
||||
"sourcePatchVerification": [
|
||||
{
|
||||
"source": "627451ecf15641504a485c6f868497177bd1bce8",
|
||||
"scope": "Cursor cold-session admission repair and prior shared integration",
|
||||
"focusedTests": 94,
|
||||
"runnerTypecheck": "passed",
|
||||
"sidecarAndProviderPackBuild": "passed",
|
||||
"daemonBuild": "passed; bytes unchanged",
|
||||
"paidQualification": "four-cell batch running; not yet passed"
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in new issue
Block a user