Record remote host refusal without consuming case initials

doc/plans/2026-10-26-chat-catalog-eval-goal.md

@@ -8,7 +8,7 @@ All new local eval admissions are stopped. Already-active local children may fin
 
 All future eval execution must run in GitHub Actions on the existing AWS spot runners, using the new Paperclip runner **OpenCode** provider through **OpenRouter**, model **`deepseek/deepseek-v4.1-flash-1017`**. The remote campaign requires a separately named execution/product/database/model identity and separately reported rate across all244 scenarios. Do not carry local passes into the remote rate, and do not execute obsolete local source merely to preserve the previous queue.
 
-Carry the audited coherent harness corrections into the remote source. Preserve prior ordinal histories. The manager approved one new remote initial measurement for every scenario and at most one coherent correction measurement per scenario. The representative remote smoke counts as its scenario’s remote initial. Preserve all prior local ordinals separately; this is not an unlimited repair or unchanged rerun policy. Production/core fixes remain isolated and unmerged unless separately authorized.
+Carry the audited coherent harness corrections into the remote source. Preserve prior ordinal histories. The manager approved one new remote initial measurement for every scenario and at most one coherent correction measurement per scenario. The representative remote smoke counts as its scenario’s remote initial. A host qualification refusal before evaluation is a retained infrastructure operation, not a case measurement or consumed initial ordinal; the first such operation40519322850 left all244 remote initials unrun. Preserve all prior local ordinals separately; this is not an unlimited repair or unchanged rerun policy. Production/core fixes remain isolated and unmerged unless separately authorized.
 
 Immediate deliverables are an inspectable safe results page, an exact closed local checkpoint, and evidence-based accounting of elapsed wall time, stages reached, harness/product/model/prerequisite outcomes, actual usage and operational duplicate/reconciliation history. The original goal below remains historical context where superseded by this section.
 

doc/plans/2026-10-26-chat-catalog-eval-progress.md

@@ -4,6 +4,7 @@ Updated: 2026-10-27. This is the single progress record for [the saved goal](202
 
 ## Current state
 
+- **First remote operation:**[Actions run40519322850](https://github.com/paperclipai/paperclip/actions/runs/40519322850), exact target6175603, was refused by the actual-host gate because AWS supplied On-Demand instead of Spot. Authenticated artifact12740329812 and its SHA256-bound host receipt prove evaluation was not admitted:0canonical/model/native work and0case ordinals consumed. All244 remote initials remain unrun. Separately, trusted reporter dependency installation failed with `ERR_PNPM_LOCKFILE_CONFIG_MISMATCH`; this is not a case/model outcome. No redispatch or retry is authorized by this record. Remote operation evidence is under `results/chat-catalog-remote-opencode/operation-40519322850.json`; the historical local ledger is unchanged.
 - **User steering supersedes the local queue:**all new local admissions are stopped. All admitted children have finished and exact owned absence is verified; no remaining76/22 local initials, local B or new health launches. Future execution is GitHub Actions on existing AWS spot runners, using the new Paperclip runner OpenCode provider via OpenRouter model `deepseek/deepseek-v4.1-flash-1017`. Local histories remain preserved; their passes cannot qualify the separately named remote lane. The manager approved a new remote initial for every scenario plus at most one coherent correction measurement, with distinct history and no erased local ordinals.
 - **Readable results:**`tests/runner-e2e/results/chat-catalog-campaign/interim-results.html` now shows all244 searchable outcomes plus frozen public canonical query/context/intended outcome, conservative stage proofs, edited diagnoses, correction commits, evidence references, elapsed gaps and six open/unmerged core PRs. The local checkpoint is closed. Proofs establish105 saved canonical queries,102 exact canonical-native admissions,14 returned feature grades and110 observation-grade snapshots; unproven stages stay unknown. Eighteen measurements have confirmed zero native IDs, two lack bound run-count proof; zero native alone is not a setup classification. Recorded reservation span is15.539h; union of completed attempt wall intervals4.719h (not model time), sum5.879h (overlaps counted). Two large no-measurement gaps total8h26m and are being source-attributed separately.
 

tests/runner-e2e/chat-remote-campaign.test.ts

@@ -1,6 +1,6 @@
 import { createHash } from "node:crypto";
 import { afterEach, describe, expect, it, vi } from "vitest";
-import { applyRemoteCell, validateRemoteCell, remoteCellUsage, type RemoteArtifactDocument, type RemoteCampaign } from "./chat-remote-campaign.js";
+import { applyRemoteCell, validateRemoteCell, remoteCellUsage, validateRemoteHostRefusal, type RemoteArtifactDocument, type RemoteCampaign } from "./chat-remote-campaign.js";
 import { DATABASE_LANE_ENV, POSTGRES_ARTIFACT_ENV, POSTGRES_IMAGE_ID_ENV, POSTGRES_AMD64_IMAGE, POSTGRES_AMD64_IMAGE_IDS, POSTGRES_VERSION } from "./docker-postgres.js";
 import { CHAT_OPENCODE_MODEL } from "./chat-runtime-profile.js";
 import type { CampaignAttempt } from "./chat-campaign.js";
@@ -37,6 +37,14 @@ describe("remote Actions cell import", () => {
     const foreign = f.input(); foreign.result.path = "../result.json"; expect(() => validateRemoteCell(foreign)).toThrow("artifact path");
     const rerun = f.input(); rerun.provenance.runAttempt = 2; expect(() => validateRemoteCell(rerun)).toThrow("job rerun");
   });
+  it("imports actual On-Demand refusal only as an operation with zero case ordinals", () => {
+    const f = fixture(); Object.assign(f.host, { status: "blocked", lifecycle: "on-demand", reason: "on-demand-or-unknown-host-refused-before-evaluation", instanceId: "i-12345678", imageId: "ami-12345678" });
+    delete f.host.postgres; delete f.host.dockerEngineSha256;
+    const operation = validateRemoteHostRefusal(f.input());
+    expect(operation.caseMeasurements).toBe(0); expect(operation.caseOrdinalsConsumed).toBe(0); expect(operation.initialIntentConsumed).toBe(false);
+    f.host.evaluationAdmitted = true; expect(() => validateRemoteHostRefusal(f.input())).toThrow("On-Demand refusal");
+    f.host.evaluationAdmitted = false; f.host.lifecycle = "spot"; expect(() => validateRemoteHostRefusal(f.input())).toThrow("On-Demand refusal");
+  });
   it("keeps unknown billing unknown and reported OpenRouter billing separate", () => {
     const f = fixture(); expect(remoteCellUsage(f.result)).toEqual({ costUsd: null, tokens: null, coverage: "unavailable" });
     f.result.billing = { llm: { totalTokens: 123, reportedCostUsd: 0.02, costStatus: "reported" } };

tests/runner-e2e/chat-remote-campaign.ts

@@ -71,6 +71,34 @@ export function validateRemoteCell(input: {
     resultPath: input.result.path, resultSha256: input.result.sha256, hostSha256: input.host.sha256, definitionSha256: input.definition.sha256,
     result, definitionDigest: definition.definitionDigest, graderDigest: definition.graderDigest, databaseLane };
 }
+/** A pre-attempt qualification refusal is an operation, never a case reservation.
+ * Only the source-qualified On-Demand branch proves this exact zero-admission stage. */
+export function validateRemoteHostRefusal(input: {
+  intent: RemoteCellIntent; provenance: RemoteArtifactProvenance; host: RemoteArtifactDocument;
+}) {
+  const { intent, provenance: p } = input, host = document(input.host);
+  requireProof(p.repository === "paperclipai/paperclip" && p.workflow === "runner-full-stack-e2e.yml" && p.workflowRef === "refs/heads/master"
+    && /^\d+$/.test(p.runId) && p.runAttempt === 1 && /^\d+$/.test(p.artifactId) && hex(p.archiveSha256), "unverified qualification artifact");
+  requireProof(/^[a-f0-9]{40}$/.test(intent.sourceRevision) && p.targetSourceRevision === intent.sourceRevision
+    && host.sourceRevision === intent.sourceRevision && host.workflowRunId === p.runId && host.workflowAttempt === p.runAttempt
+    && host.workflowRef === "paperclipai/paperclip/.github/workflows/runner-full-stack-e2e.yml@refs/heads/master", "qualification source/run mismatch");
+  requireProof(host.schema === "paperclip.chat-remote-host/v1" && host.status === "blocked" && host.scope === "before-evaluation"
+    && host.evaluationAdmitted === false && host.automaticRetries === 0 && host.lifecycle === "on-demand"
+    && host.reason === "on-demand-or-unknown-host-refused-before-evaluation" && !host.postgres && !host.dockerEngineSha256
+    && host.region === "us-east-1" && host.architecture === "x86_64" && /^i-[a-f0-9]{8,17}$/.test(host.instanceId)
+    && /^ami-[a-f0-9]{8,17}$/.test(host.imageId), "not the proven pre-evaluation On-Demand refusal");
+  requireProof(Array.isArray(host.executionIds) && host.executionIds.length === intent.selected.length
+    && new Set(host.executionIds).size === host.executionIds.length
+    && intent.selected.every(row => host.executionIds.includes(row.executionId)), "qualification selection mismatch");
+  return { schema: "paperclip.chat-remote-operation/v1", campaignId: intent.campaignId, runId: p.runId,
+    sourceRevision: intent.sourceRevision, artifactId: p.artifactId, artifactSha256: p.archiveSha256,
+    hostReceiptPath: input.host.path, hostReceiptSha256: input.host.sha256,
+    outcome: "infrastructure-blocked-before-evaluation", reason: host.reason,
+    evaluationAdmitted: false, canonicalQueries: 0, nativeRuns: 0, providerCalls: 0,
+    caseMeasurements: 0, caseOrdinalsConsumed: 0, initialIntentConsumed: false,
+    caseExecutionIds: host.executionIds as string[], actualHost: { lifecycle: "on-demand", region: host.region, architecture: host.architecture },
+    reporterConclusionIsIndependent: true } as const;
+}
 export function remoteCellUsage(result: RunnerE2EResult): CampaignAttempt["usage"] {
   const usage = result.billing?.llm;
   return { tokens: usage && Number.isFinite(usage.totalTokens) ? usage.totalTokens : null,