Retain broken-link fixture and diagnostic privacy failure

doc/plans/2026-10-26-chat-catalog-eval-progress.md

@@ -7,8 +7,8 @@ Updated: 2026-10-27. This is the single progress record for [the saved goal](202
 - **244 scenarios / 35 areas / 191 capabilities**, frozen inventory digest `9dc9cf6a5426cf1b56bc30a3571851f233f9dd5e93a1292eaee414d33b52df77`. All original queries, contexts and expected outcomes remain unchanged. **196 passes required**; blocked, partial, unsupported and unrun remain in the denominator.
 - **229 authored: 220 without declared coverage gaps, 9 partial, 15 unimplemented** at exact committed-source metadata checkpoint `32b472dfa5640987e199c2e0aa21b2c7c02555c6`. There are 227 concrete lazy descriptors plus two legacy task cases. Actual callbacks, bounds, policies and source dependencies were reconciled from this one Git archive; newer area work remains pending. These are authoring counts, not acceptance.
 - Five partial definitions contain concrete executable supported-surface/context blocker probes: two access grants, subscription provenance, skill-use authority/context and genuine prior-day routine history. Four have unfinished required branches: the Grok origin integration closure, named ChatGPT setup, team installer failures and private-repository after-grant completion. **225 authoring-resolved; 4 unfinished; 15 absent.** Only actual reserved execution can process a concrete blocker; offline probes/calibrations do not establish processing. Incomplete conditions cannot pass.
-- **Six scenarios attempted; seven actual measurements; zero first-attempt passes and zero final passes (0/244 for both rates).** Goal-tree is exhausted after two failures. Alignment and purpose each retain one failed initial attempt and one allowed correction/rerun remaining.
-- **No live launcher or provider process is running.** Comment-with-reference ordinal 1 failed during normal startup doctor from immutable `f73390f925edcddd3af977e337adf16d5818fb41`, before browser/native/model admission: doctor requires a persisted PostgreSQL connection string despite the server supporting `DATABASE_URL`. Exact Docker resources report removed; overall cleanup is failed/unconfirmed and private recovery state is retained. Admissions stopped again. Broken-link and edit-profile remain unreserved. Both project and comment initial failures are preserved; all final retries remain held for coherent B.
+- **Seven scenarios attempted; eight actual measurements; zero first-attempt passes and zero final passes (0/244 for both rates).** Goal-tree is exhausted after two failures. Alignment and purpose each retain one failed initial attempt and one allowed correction/rerun remaining.
+- **No live launcher or provider process is running.** Broken-link ordinal 1 failed from qualified immutable6be after a deduplicated fixture alias caused the distractor reread to return404. Packaging additionally caught/redacted a real fixture invitation token in the product HTTP request log and withheld the affected blob. Cleanup passed/Docker removed; admissions stopped for this shared privacy issue. Profile and the next12 approved initials remain unreserved. All original failures and final-B holds remain unchanged.
 - Current recovery: semaphore cleanup approval is still pending and no set has been removed. Read-only discovery verified an already running local Docker Linux/arm64 engine; an explicit owned PostgreSQL18.6 lane is being qualified. Its one infrastructure-only health check at `772c29dad9` failed before container start because Docker Desktop represents the exact host password bind with `/host_mnt`; no Paperclip, native or model work began. The initial failed cleanup receipt is preserved; separate authorized cleanup verified exact container/volume absence and removed only its password. The separately authorized corrected infrastructure-only health at `325af67e3d` passed PostgreSQL18.6, normal onboarding/migrations/API and zero work; exact owned resources were removed with no cleanup failures. The Docker lane is qualified; host IPC cleanup is no longer required. AgentMail/GitHub read-only discovery is authorized/completed; designated self-email and repository choices remain pending.
 - Production baseline `8373d7867963674ff960d2345d0d28086f799bc7`; branch `codex/chat-catalog-evals`. PR #17794 merged via `2ee36a4`. No eval/core PR has been created. No merge is eligible.
 
@@ -253,3 +253,13 @@ Commit `1fc30a0ed1d3c2d3770b0e70d94710f0b3310589` restores the product-required
 Immutable infrastructure qualification proposal `6beacfe0a94e87b9b834f0f1d46d42e8ec7aec42` overlays only ten owned correction paths on failed snapshot f733. Archived E2E typecheck exited 0 and 120 focused controls across five files passed. Production baseline8373/lock, copied workspace artifacts, native binary and 46 audited dependency links remain unchanged. The health helper now executes the same final persisted-home secret scanner and exact custody policy, recording `sameFinalPersistedStateSecretScan` explicitly. One infrastructure-only health qualification has been proposed to the manager; no new health/canonical launch or reservation occurred while authoring.
 
 Append-only read-only cleanup evidence `results/chat-catalog-campaign/comment-hZhllt-readonly-cleanup.json` verifies the retained comment root is owned0700, exact recorded Docker container and volume are absent, four known observed PIDs are absent, and no current command references that exact root. This limited census does not establish a complete descendant history; original cleanup failed/unconfirmed remains unchanged and no resource was removed. The CLI doctor’s file-only DSN check versus the server’s supported `DATABASE_URL` override is recorded as a candidate core consistency issue for a later separately qualified PR, with no production edit here.
+
+### Qualified custody health and broken-link initial
+
+The one authorized infrastructure health from6be passed in19.656s. Retained `docker-infrastructure-health-2026-10-27T07-32-21-104Z.json` confirms PostgreSQL18.6, API/normal migrations, `sameFinalPersistedStateSecretScan:true`, zero companies/agents/native/model/scenarios, cleanupFailures[] and exact owned container status removed. No canonical attempt was charged for infrastructure qualification.
+
+Broken-link ordinal1 then ran as `catalog-initial-2026-10-27T07-35-44-803Z-navigation-preferences-broken-link`, one actual native run and323,491 tokens, subscription dollars unpriced. It failed after the bounded poll; original result and usage remain unchanged. Retained setup proves `stale.id === similar.id`: production’s recent-open-title dedup returned the first task for the second same-title create. Deleting the old task therefore also deleted the fixture alias intended as the separate distractor. The expected404 status helper worked; the later unchecked `similarAfter` read caused the timeout. Browser receipts still show the genuine missing URL and private-resource nondisclosure. This is a fixture defect, not model navigation failure.
+
+Separately, diagnostic privacy evidence shows the registered setup human invitation token in the ordinary product HTTP URL `POST /invites/[REDACTED]/accept202`. The stream/report safeguards redacted both text sinks and omitted the affected blob; no raw credential value was inspected for diagnosis. All admissions stopped on this shared privacy finding despite cleanup passing and exact Docker status removed. Profile plus the approved12 initial list remain unreserved, with no rerun or final-B measurement. Bounded diagnosis: `broken-link-6be-diagnosis.json`.
+
+The separately authorized registered-artifact transport fix `cbb6593441` uses the existing authenticated `getAttachmentContent` path, retaining attachment origin/size/hash gates.33 focused controls and scoped tsc passed; no executed archive was modified. Selection-summary provenance sidecar explicitly retains historical snapshot4/5 counts versus current ledger7/8 and the previously acknowledged external-fixture source overlap.

tests/runner-e2e/chat-campaign-manifest.json

@@ -3682,9 +3682,68 @@
       "readiness": "implemented",
       "executionId": "chat-features.runner-codex-subscription.local.navigation-preferences.broken-link",
       "blocker": null,
-      "attempts": [],
-      "qualifyingDefinitionDigest": null,
-      "qualifyingGraderDigest": null,
+      "attempts": [
+        {
+          "ordinal": 1,
+          "outcome": "failed",
+          "executionId": "chat-features.runner-codex-subscription.local.navigation-preferences.broken-link",
+          "startedAt": "2026-10-27T07:35:44.803Z",
+          "finishedAt": "2026-10-27T07:40:48.275Z",
+          "sourceRevision": "6beacfe0a94e87b9b834f0f1d46d42e8ec7aec42",
+          "workingTreeDigest": "2a18d79fd9b9a9d9e72e8ae512c837477246cd8cd3c7bd95f20c282147668e4f",
+          "definitionDigest": "0a5ee01627586a70777f18d6ebc2e5fec840e639f8da05ef4963321416cbf110",
+          "suiteDefinitionDigest": "87540a2990ac4e8af4ec918da2550547cc10e18fc0ec557941d427080951430a",
+          "graderDigest": "0a5ee01627586a70777f18d6ebc2e5fec840e639f8da05ef4963321416cbf110",
+          "model": "gpt-5.6-sol",
+          "profile": "runner-codex-subscription",
+          "environment": "local",
+          "automaticRetries": 0,
+          "correction": null,
+          "classification": "eval_fixture_deduplicated_distractor_and_private_invitation_url_logging",
+          "observedFailure": "Timed out waiting for navigation-preferences.broken-link: GET /api/issues/d31f5db9-1b2e-407e-87ef-2c348a892fb7 returned 404; body withheld; Reporter privacy violation: redacted server.log, omitted blob-report/report.zip, redacted playwright process stdout/stderr",
+          "evidencePaths": [
+            "tests/runner-e2e/results/chat-catalog-campaign/initial-selection-6beacfe0a9.json",
+            "tests/runner-e2e/results/catalog-initial-2026-10-27T07-35-44-803Z-navigation-preferences-broken-link/chat-features/runner-codex-subscription/local/navigation-preferences.broken-link/attempt-1/result.json",
+            "tests/runner-e2e/results/chat-catalog-campaign/active-navigation-preferences.broken-link.json"
+          ],
+          "usage": {
+            "costUsd": null,
+            "tokens": 323491,
+            "coverage": "Actual heartbeat usage receipts, including cache; subscription cost unpriced unless complete billing; fixture costs excluded."
+          },
+          "cleanup": "passed",
+          "corePrDependencies": [],
+          "databaseLane": {
+            "lane": "owned-docker-postgres",
+            "image": "postgres@sha256:74935e72241653ca55e0414067e6d8763aceb8a810eb51b452253ec3dcfc4336",
+            "imageId": "sha256:74935e72241653ca55e0414067e6d8763aceb8a810eb51b452253ec3dcfc4336",
+            "version": "18.6-1.pgdg13+2"
+          },
+          "productIdentity": {
+            "kind": "source-build",
+            "productionSourceRevision": "8373d7867963674ff960d2345d0d28086f799bc7",
+            "snapshotSourceRevision": "6beacfe0a94e87b9b834f0f1d46d42e8ec7aec42",
+            "nativeBinarySha256": "9d8af6b85b836a03092ade2273be6f29f5c2f6ec67ee891d83ee9693ba39d3b4",
+            "workspaceBuildDigest": "222e10a8a2a20d3015ec4470b952e2dd9b927dfe9390c4a5ed1e19d740c0e35b",
+            "receiptPaths": [
+              "tests/runner-e2e/results/chat-catalog-campaign/snapshot-6beacfe0a9.json",
+              "tests/runner-e2e/results/chat-catalog-campaign/snapshot-10d0ce63610.json",
+              "tests/runner-e2e/results/chat-catalog-campaign/snapshot-0fdcb539a8.json"
+            ],
+            "coreCandidate": false
+          },
+          "diagnosisHistory": [
+            {
+              "at": "2026-10-27T07:46:43.984946+00:00",
+              "classification": "candidate_failure",
+              "detail": "Original result class preserved. Saved stale and similar fixture IDs were identical because normal public same-title task creation deduplicated; later similarAfter GET queried the deleted alias. Separate retained diagnostics show invitation-token URL in product request log, redacted/withheld by report safeguards.",
+              "supersededBy": "eval_fixture_deduplicated_distractor_and_private_invitation_url_logging"
+            }
+          ]
+        }
+      ],
+      "qualifyingDefinitionDigest": "0a5ee01627586a70777f18d6ebc2e5fec840e639f8da05ef4963321416cbf110",
+      "qualifyingGraderDigest": "0a5ee01627586a70777f18d6ebc2e5fec840e639f8da05ef4963321416cbf110",
       "productPolicy": {
         "lane": "source-build",
         "sourceBaselineRevision": "8373d7867963674ff960d2345d0d28086f799bc7"