Record isolated two-case initial qualification and terminal reconciliation

2026-11-05 03:41:32 +01:00 · 2026-10-27 05:39:51 -05:00
1 parent 770f1c30e1
commit b602e342da
@@ -6,14 +6,14 @@ Updated: 2026-10-27. This is the single progress record for [the saved goal](202

 - **Frozen inventory:244 scenarios /35areas /191capabilities;196 eligible passes required.** Inventory digest `9dc9cf6a5426cf1b56bc30a3571851f233f9dd5e93a1292eaee414d33b52df77`. Original queries, contexts and outcomes remain unchanged; blocked, partial and unrun cases stay in the denominator.
 - **All244 callbacks authored and authoring-resolved** at closed source `b65cce861b5667f2424bdfacbdfd610fa697ba60`:238 without declared gaps and6 concrete executable blocker probes,0 absent/unfinished (242 lazy definitions +2 legacy). Grok, named ChatGPT and private GitHub after-grant callbacks are integrated. This is authored coverage, not live acceptance.
- **72 scenarios processed /73 completed measurements:10 initial passes and63 failures;0 running at the closed checkpoint;0 final eligible passes.** Actual retained native usage:28,018,825 tokens; subscription dollars remain unpriced. The latest30-case batch finished2passed/28failed; all30 have verified cleanup and exact Docker removal. Closed receipt:`tests/runner-e2e/results/chat-catalog-campaign/closed-initial-checkpoint-72-73.json`. Two older cleanup uncertainties remain unresolved; original measurement bytes are preserved.
+- **74 scenarios processed /75 completed measurements:11 initial passes and64 failures;0 running.** Actual retained native usage:28,615,045 tokens; subscription dollars remain unpriced. The concurrent qualification pair produced1pass/1failure with successful independent cleanup. Archive-company's selected identity still matches current source; its pass remains provisional until the final closed-source checkpoint. The other10 initial passes require their already-recorded selected-closure revalidation. Two older cleanup uncertainties remain unresolved; original measurement bytes are preserved.
 - **Single coherent final-B retry set remains held.** Goal-tree exhausted two measurements. All original outcomes, evidence and measured digests remain unchanged. Actual current selected identity review is retained in `closed-pass-identity-review-b65.json`; changed selected execution/grader closures require reviewed revalidation within the same max-two budget. Unrelated Git revisions and unchanged failed behavior do not justify reruns.
- **The30-case efbb batch completed; a two-case concurrent initial qualification is authorized** from immutable `c167a54162fadffe665b4d35a7a1308c6233b75d`:companies.archive-company and connection-setup.choose-provider, each1native/1source-chat human turn/300s; two one-case children, each maxParallel1/retries0. Exact source/fingerprints, qualified Docker/native,40 archived helper/shared controls plus parent prepare/negative/atomic controls passed. Prior launcher/writer exited and exact last container absence was independently verified. Sole-writer qualifying metadata is prepared before the parent lock; stop after the pair for review. No B rerun is included.
+- **Two-case concurrency qualification completed; further admissions held for manager review.** Immutable `c167a54162fadffe665b4d35a7a1308c6233b75d`:companies.archive-company passed and connection-setup.choose-provider failed its actual reply check; each1native/1source human. Worker overlap25.479s; separate roots, ports53206/53205, containers and PostgreSQL system identifiers. Both containers and exact-root-labeled volumes are absent, both roots removed, and coordinator/child PIDs absent. A parent-only duplicate Playwright import prevented the passing completion from initially recording; isolated-process replay confirmed unchanged selected identity and finalized the existing result without rerun/reservation. Original parent error/stop preserved and owner lock retained. Receipt:`tests/runner-e2e/results/chat-catalog-campaign/two-lane-qualification-closed-c167.json`;5 offline replay controls passed. Heavy broad source verification is running separately before further timed initials.
 - **Infrastructure is qualified:**owned local Docker PostgreSQL18.6 at the pinned installed image, normal onboarding/migrations/API and final persisted-state scan passed with zero model/native/scenario work; exact resources removed. No further infrastructure health is needed for the unchanged lane. The proposed host SysV cleanup was never performed and is no longer needed. Source/build/native bytes remain bound to production baseline `8373d7867963674ff960d2345d0d28086f799bc7`; no core candidate is used as benchmark proof.
- **Separate core PRs remain OPEN/unmerged, auto-merge off.** Doctor [#17847](https://github.com/paperclipai/paperclip/pull/17847), head `a042fb4a28f2e1574972798ed80de2e616daf9fc`, and invite logging [#17848](https://github.com/paperclipai/paperclip/pull/17848), head `98a2432f88c33afacff2e014bd8c30630b26b339`, each have55 successful checks. Budget [#17857](https://github.com/paperclipai/paperclip/pull/17857), head `e5682d22fa8f7c0dd01ddb5e9cc42d941db6fe43`, and approval routing [#17858](https://github.com/paperclipai/paperclip/pull/17858), head `b60aaf19343389faf67580fb1ff7a561983bb4f5`, have CI/review pending at the latest manager checkpoint. None changes measured baseline8373. No eval PR or merge is eligible.
+- **Separate core PRs remain OPEN/unmerged, auto-merge off.** Doctor [#17847](https://github.com/paperclipai/paperclip/pull/17847), head `a042fb4a28f2e1574972798ed80de2e616daf9fc`, and invite logging [#17848](https://github.com/paperclipai/paperclip/pull/17848), head `98a2432f88c33afacff2e014bd8c30630b26b339`, each have55 successful checks. Budget [#17857](https://github.com/paperclipai/paperclip/pull/17857), head `b6cd49d4b6926fae61a2c68a31ce7c659b69eaa6`, and approval routing [#17858](https://github.com/paperclipai/paperclip/pull/17858), head `b60aaf19343389faf67580fb1ff7a561983bb4f5`, have CI/review pending at the latest manager checkpoint. None changes measured baseline8373. No eval PR or merge is eligible.
 - **Genuine missing inputs:**AgentMail/GitHub read-only discovery was authorized and completed; designated self-mail/recipient and repository choices remain pending. Standard provider-key source references exist, but validity and disposable remote/operator targets remain unqualified. No provider send, remote setup or account discovery is implied by authoring coverage.

-The efbb coordinator exited0 after30 retained measurements; no old launcher/writer remains. The authoritative ledger is the current manifest; immutable snapshot selection summaries are historical. The remaining initial-only draft covers172 IDs after this batch, including concrete missing-resource probes; its known two alias fixtures are now corrected in151488bf8c before future admission.
+The efbb coordinator exited0 after30 retained measurements; no old launcher/writer remains. The authoritative ledger is the current manifest; immutable snapshot selection summaries are historical. The remaining initial-only draft originally covered172 IDs after efbb; the qualification pair consumed two, leaving170 unattempted IDs. It includes concrete missing-resource probes; its known two alias fixtures are corrected in151488bf8c before future admission.

 ## Current measured continuation and held corrections

@@ -25,7 +25,7 @@ Necessary-question's retained observation contains no saved task, document or re

 ## Measured evidence — all failures retained

-All results below use local native Codex subscription profile `runner-codex-subscription`, model `gpt-5.6-sol`, max parallelism one and zero automatic retries. Retained paths are under ignored `tests/runner-e2e/results/`. The launcher numbers each fresh invocation `attempt-1`; the ledger separately preserves governed campaign ordinals.
+All results below use local native Codex subscription profile `runner-codex-subscription`, model `gpt-5.6-sol`, one case per launcher and zero automatic retries. The first72 scenarios were sequential; the separately recorded qualification pair used two concurrent isolated launchers. Retained paths are under ignored `tests/runner-e2e/results/`. The launcher numbers each fresh invocation `attempt-1`; the ledger separately preserves governed campaign ordinals.

 | Scenario / campaign ordinal | Source | Retained campaign | Outcome and supported cause | Cleanup / accounted tokens |
 | --- | --- | --- | --- | --- |
@@ -3075,7 +3075,61 @@
      "readiness": "implemented",
      "executionId": "chat-features.runner-codex-subscription.local.companies.archive-company",
      "blocker": null,
-      "attempts": [],
+      "attempts": [
+        {
+          "ordinal": 1,
+          "outcome": "passed",
+          "executionId": "chat-features.runner-codex-subscription.local.companies.archive-company",
+          "startedAt": "2026-10-27T10:29:19.521Z",
+          "finishedAt": "2026-10-27T10:31:23.533Z",
+          "sourceRevision": "c167a54162fadffe665b4d35a7a1308c6233b75d",
+          "workingTreeDigest": "6251a7763d81986af3d43a142a52c702da1bde2b57cee5aef2606e4418ccc698",
+          "definitionDigest": "e45461119cd11cbc7d9f3659a35bbd7a6c1ac72d5c5b2e4f187f97210ac8c7c7",
+          "suiteDefinitionDigest": "c0a8d311ea4dc7616bc8cef7a815457e5c013bd491d2ac6510f62500d3d0ac25",
+          "graderDigest": "e45461119cd11cbc7d9f3659a35bbd7a6c1ac72d5c5b2e4f187f97210ac8c7c7",
+          "model": "gpt-5.6-sol",
+          "profile": "runner-codex-subscription",
+          "environment": "local",
+          "automaticRetries": 0,
+          "correction": null,
+          "classification": "provisional_initial_pass_pending_final_closure",
+          "observedFailure": null,
+          "evidencePaths": [
+            "tests/runner-e2e/results/chat-catalog-campaign/two-lane-initial-pair-c167a54162.json",
+            "tests/runner-e2e/results/two-lane-initial-c167a54162fa-b95423a10d79997385db/chat-features/runner-codex-subscription/local/companies.archive-company/attempt-1/result.json",
+            "tests/runner-e2e/results/chat-catalog-campaign/active-companies.archive-company.json",
+            "tests/runner-e2e/results/chat-catalog-campaign/initial-pass-identity-review-companies.archive-company.json",
+            "tests/runner-e2e/results/chat-catalog-campaign/operations/two-lane-exact-absence-c167.json",
+            "tests/runner-e2e/results/chat-catalog-campaign/operations/two-lane-archive-replayed-completion.json"
+          ],
+          "usage": {
+            "costUsd": null,
+            "tokens": 541825,
+            "coverage": "Actual heartbeat usage receipts, including cache; subscription cost unpriced unless complete billing; fixture costs excluded."
+          },
+          "cleanup": "passed",
+          "corePrDependencies": [],
+          "databaseLane": {
+            "lane": "owned-docker-postgres",
+            "image": "postgres@sha256:74935e72241653ca55e0414067e6d8763aceb8a810eb51b452253ec3dcfc4336",
+            "imageId": "sha256:74935e72241653ca55e0414067e6d8763aceb8a810eb51b452253ec3dcfc4336",
+            "version": "18.6-1.pgdg13+2"
+          },
+          "productIdentity": {
+            "kind": "source-build",
+            "productionSourceRevision": "8373d7867963674ff960d2345d0d28086f799bc7",
+            "snapshotSourceRevision": "c167a54162fadffe665b4d35a7a1308c6233b75d",
+            "nativeBinarySha256": "9d8af6b85b836a03092ade2273be6f29f5c2f6ec67ee891d83ee9693ba39d3b4",
+            "workspaceBuildDigest": "222e10a8a2a20d3015ec4470b952e2dd9b927dfe9390c4a5ed1e19d740c0e35b",
+            "receiptPaths": [
+              "tests/runner-e2e/results/chat-catalog-campaign/snapshot-two-lane-c167a54162.json",
+              "tests/runner-e2e/results/chat-catalog-campaign/snapshot-10d0ce63610.json",
+              "tests/runner-e2e/results/chat-catalog-campaign/snapshot-0fdcb539a8.json"
+            ],
+            "coreCandidate": false
+          }
+        }
+      ],
      "qualifyingDefinitionDigest": "e45461119cd11cbc7d9f3659a35bbd7a6c1ac72d5c5b2e4f187f97210ac8c7c7",
      "qualifyingGraderDigest": "e45461119cd11cbc7d9f3659a35bbd7a6c1ac72d5c5b2e4f187f97210ac8c7c7",
      "productPolicy": {
@@ -3405,7 +3459,58 @@
      "readiness": "implemented",
      "executionId": "chat-features.runner-codex-subscription.local.connection-setup.choose-provider",
      "blocker": null,
-      "attempts": [],
+      "attempts": [
+        {
+          "ordinal": 1,
+          "outcome": "failed",
+          "executionId": "chat-features.runner-codex-subscription.local.connection-setup.choose-provider",
+          "startedAt": "2026-10-27T10:29:19.551Z",
+          "finishedAt": "2026-10-27T10:30:11.290Z",
+          "sourceRevision": "c167a54162fadffe665b4d35a7a1308c6233b75d",
+          "workingTreeDigest": "c7fd9a3a9f33cdc907f3e8c61d5f40eeb107cc8f856e01552745cd438e4e27d7",
+          "definitionDigest": "2650b5969af7a28ce555ba5ec92dfb6cf3eb3c21dfce0b8ab99be9c41e2ff772",
+          "suiteDefinitionDigest": "c0a8d311ea4dc7616bc8cef7a815457e5c013bd491d2ac6510f62500d3d0ac25",
+          "graderDigest": "2650b5969af7a28ce555ba5ec92dfb6cf3eb3c21dfce0b8ab99be9c41e2ff772",
+          "model": "gpt-5.6-sol",
+          "profile": "runner-codex-subscription",
+          "environment": "local",
+          "automaticRetries": 0,
+          "correction": null,
+          "classification": "candidate_failure",
+          "observedFailure": "Actual reply does not explain the advertised Notion choices and unavailable prerequisite",
+          "evidencePaths": [
+            "tests/runner-e2e/results/chat-catalog-campaign/two-lane-initial-pair-c167a54162.json",
+            "tests/runner-e2e/results/two-lane-initial-c167a54162fa-41b15d773be40166eeb1/chat-features/runner-codex-subscription/local/connection-setup.choose-provider/attempt-1/result.json",
+            "tests/runner-e2e/results/chat-catalog-campaign/active-connection-setup.choose-provider.json"
+          ],
+          "usage": {
+            "costUsd": null,
+            "tokens": 54395,
+            "coverage": "Actual heartbeat usage receipts, including cache; subscription cost unpriced unless complete billing; fixture costs excluded."
+          },
+          "cleanup": "passed",
+          "corePrDependencies": [],
+          "databaseLane": {
+            "lane": "owned-docker-postgres",
+            "image": "postgres@sha256:74935e72241653ca55e0414067e6d8763aceb8a810eb51b452253ec3dcfc4336",
+            "imageId": "sha256:74935e72241653ca55e0414067e6d8763aceb8a810eb51b452253ec3dcfc4336",
+            "version": "18.6-1.pgdg13+2"
+          },
+          "productIdentity": {
+            "kind": "source-build",
+            "productionSourceRevision": "8373d7867963674ff960d2345d0d28086f799bc7",
+            "snapshotSourceRevision": "c167a54162fadffe665b4d35a7a1308c6233b75d",
+            "nativeBinarySha256": "9d8af6b85b836a03092ade2273be6f29f5c2f6ec67ee891d83ee9693ba39d3b4",
+            "workspaceBuildDigest": "222e10a8a2a20d3015ec4470b952e2dd9b927dfe9390c4a5ed1e19d740c0e35b",
+            "receiptPaths": [
+              "tests/runner-e2e/results/chat-catalog-campaign/snapshot-two-lane-c167a54162.json",
+              "tests/runner-e2e/results/chat-catalog-campaign/snapshot-10d0ce63610.json",
+              "tests/runner-e2e/results/chat-catalog-campaign/snapshot-0fdcb539a8.json"
+            ],
+            "coreCandidate": false
+          }
+        }
+      ],
      "qualifyingDefinitionDigest": "2650b5969af7a28ce555ba5ec92dfb6cf3eb3c21dfce0b8ab99be9c41e2ff772",
      "qualifyingGraderDigest": "2650b5969af7a28ce555ba5ec92dfb6cf3eb3c21dfce0b8ab99be9c41e2ff772",
      "productPolicy": {