Bind new remote intents to user-authorized AWS fleet fallback

doc/plans/2026-10-26-chat-catalog-eval-goal.md

@@ -6,7 +6,7 @@ Date: 2026-10-26
 
 All new local eval admissions are stopped. Already-active local children may finish their existing work and cleanup; no further local initial, final-B, infrastructure-health or provider qualification launches are authorized. Preserve the complete local measurement/ordinal histories and original evidence as a historical lane.
 
-All future eval execution must run in GitHub Actions on the existing AWS spot runners, using the new Paperclip runner **OpenCode** provider through **OpenRouter**, model **`deepseek/deepseek-v4.1-flash-1017`**. The remote campaign requires a separately named execution/product/database/model identity and separately reported rate across all244 scenarios. Do not carry local passes into the remote rate, and do not execute obsolete local source merely to preserve the previous queue.
+All future eval execution must run in GitHub Actions on the existing AWS fleet, allowing its normal Spot or On-Demand fallback as directly authorized by the user on2026-10-27 at15:21UTC, using the new Paperclip runner **OpenCode** provider through **OpenRouter**, model **`deepseek/deepseek-v4.1-flash-1017`**. Record and verify the actual AWS host lifecycle; unknown hosts still fail closed. The earlier strict-Spot intents and refused operations remain unchanged and do not become retroactive case admissions. No local eval execution is authorized. The remote campaign requires a separately named execution/product/database/model identity and separately reported rate across all244 scenarios. Do not carry local passes into the remote rate, and do not execute obsolete local source merely to preserve the previous queue.
 
 Carry the audited coherent harness corrections into the remote source. Preserve prior ordinal histories. The manager approved one new remote initial measurement for every scenario and at most one coherent correction measurement per scenario. The representative remote smoke counts as its scenario’s remote initial. A host qualification refusal before evaluation is a retained infrastructure operation, not a case measurement or consumed initial ordinal; the first such operation40519322850 left all244 remote initials unrun. Preserve all prior local ordinals separately; this is not an unlimited repair or unchanged rerun policy. Production/core fixes remain isolated and unmerged unless separately authorized.
 

doc/plans/2026-10-26-chat-catalog-eval-progress.md

@@ -4,6 +4,7 @@ Updated: 2026-10-27. This is the single progress record for [the saved goal](202
 
 ## Current state
 
+- **New user host authorization15:21UTC:**future GitHub Actions execution may use the normal AWS fleet Spot or On-Demand fallback, with actual lifecycle retained and unknown hosts rejected. This supersedes the strict-Spot interpretation only for an explicitly new source/intent. Old617/941 intents and the two pre-evaluation refusals remain unchanged; all244 case initials are still unconsumed. No local execution is permitted.
 - **Second remote qualification operation:**[run40524193545](https://github.com/paperclipai/paperclip/actions/runs/40524193545), target6175603, proved actual Spot/x86_64 then failed host database-image qualification (`owned-linux-amd64-postgres-image-unqualified`) before evaluation. Authenticated artifact12743322992 and exact extracted bytes are verified; the underlying Docker/image cause is not yet established. Operation1 remains separate. Both have0canonical/native/provider work and consume0case ordinals; all244 remote initials remain unrun. Receipt:`results/chat-catalog-remote-opencode/operation-40524193545.json`. Separate reporter PR17888 head822f171177 remains open/unmerged with CI pending at the manager checkpoint; no reporter outcome is inferred from this host receipt.
 - **First remote operation:**[Actions run40519322850](https://github.com/paperclipai/paperclip/actions/runs/40519322850), exact target6175603, was refused by the actual-host gate because AWS supplied On-Demand instead of Spot. Authenticated artifact12740329812 and its SHA256-bound host receipt prove evaluation was not admitted:0canonical/model/native work and0case ordinals consumed. All244 remote initials remain unrun. Separately, trusted reporter dependency installation failed with `ERR_PNPM_LOCKFILE_CONFIG_MISMATCH`; this is not a case/model outcome. No redispatch or retry is authorized by this record. Remote operation evidence is under `results/chat-catalog-remote-opencode/operation-40519322850.json`; the historical local ledger is unchanged.
 - **User steering supersedes the local queue:**all new local admissions are stopped. All admitted children have finished and exact owned absence is verified; no remaining76/22 local initials, local B or new health launches. Future execution is GitHub Actions on existing AWS spot runners, using the new Paperclip runner OpenCode provider via OpenRouter model `deepseek/deepseek-v4.1-flash-1017`. Local histories remain preserved; their passes cannot qualify the separately named remote lane. The manager approved a new remote initial for every scenario plus at most one coherent correction measurement, with distinct history and no erased local ordinals.

tests/runner-e2e/chat-remote-campaign.test.ts

@@ -37,6 +37,17 @@ describe("remote Actions cell import", () => {
     const foreign = f.input(); foreign.result.path = "../result.json"; expect(() => validateRemoteCell(foreign)).toThrow("artifact path");
     const rerun = f.input(); rerun.provenance.runAttempt = 2; expect(() => validateRemoteCell(rerun)).toThrow("job rerun");
   });
+  it("accepts proven normal-fleet fallback only under a new explicit intent and matching receipt", () => {
+    const f = fixture(); f.host.hostPolicy = "aws-fleet-spot-or-on-demand"; f.host.lifecycle = "on-demand"; f.host.spotVerified = false;
+    expect(() => validateRemoteCell(f.input())).toThrow("intent policy");
+    const allowed = () => { const input = f.input(); return { ...input, intent: { ...input.intent, hostPolicy: "aws-fleet-spot-or-on-demand" as const } }; };
+    expect(validateRemoteCell(allowed())).toMatchObject({ hostLifecycle: "on-demand", hostPolicy: "aws-fleet-spot-or-on-demand" });
+    f.host.spotVerified = true; expect(() => validateRemoteCell(allowed())).toThrow("intent policy");
+    f.host.spotVerified = false; f.host.lifecycle = "unknown"; expect(() => validateRemoteCell(allowed())).toThrow("intent policy");
+    f.host.lifecycle = "on-demand"; delete f.host.hostPolicy; expect(() => validateRemoteCell(allowed())).toThrow("intent policy");
+    f.host.hostPolicy = "aws-fleet-spot-or-on-demand"; f.host.lifecycle = "spot"; f.host.spotVerified = true;
+    expect(validateRemoteCell(allowed()).hostLifecycle).toBe("spot");
+  });
   it("imports actual On-Demand refusal only as an operation with zero case ordinals", () => {
     const f = fixture(); Object.assign(f.host, { status: "blocked", lifecycle: "on-demand", reason: "on-demand-or-unknown-host-refused-before-evaluation", instanceId: "i-12345678", imageId: "ami-12345678" });
     delete f.host.postgres; delete f.host.dockerEngineSha256;

tests/runner-e2e/chat-remote-campaign.ts

@@ -9,6 +9,8 @@ const hex = (value: unknown): value is string => typeof value === "string" && /^
 function requireProof(condition: unknown, message: string): asserts condition { if (!condition) throw new Error(`Remote import: ${message}`); }
 export interface RemoteCellIntent {
   campaignId: string; sourceRevision: string;
+  /** Absent on preserved strict-Spot intents. Only new explicit user-authorized sources opt in. */
+  hostPolicy?: "aws-fleet-spot-or-on-demand";
   selected: Array<{ scenarioId: string; executionId: string; remoteOrdinal: 1 | 2 }>;
 }
 /** Obtained by the coordinator through authenticated Actions run/artifact APIs, not from
@@ -25,6 +27,7 @@ export interface ValidatedRemoteCell {
   scenarioId: string; ordinal: 1 | 2; sourceRevision: string;
   resultPath: string; resultSha256: string; hostSha256: string; definitionSha256: string;
   result: RunnerE2EResult; definitionDigest: string; graderDigest: string;
+  hostLifecycle: "spot" | "on-demand"; hostPolicy: "spot-only" | "aws-fleet-spot-or-on-demand";
   databaseLane: DatabaseFingerprintIdentity;
 }
 function document(value: RemoteArtifactDocument): Record<string, any> {
@@ -55,8 +58,12 @@ export function validateRemoteCell(input: {
   requireProof(result.source?.sha === intent.sourceRevision && result.source.workflowRunUrl === `https://github.com/paperclipai/paperclip/actions/runs/${p.runId}`, "result source/run mismatch");
   requireProof(!!result.accountingIdentity?.campaignId && /^[a-f0-9]{32}$/.test(result.accountingIdentity?.attemptId ?? ""), "missing actual framework accounting identity");
   requireProof(Number.isFinite(Date.parse(result.startedAt)) && Date.parse(result.finishedAt) >= Date.parse(result.startedAt) && Number.isFinite(result.durationMs) && result.durationMs >= 0, "invalid actual timing");
+  const normalFleet = intent.hostPolicy === "aws-fleet-spot-or-on-demand";
+  const allowedHost = normalFleet ? host.hostPolicy === intent.hostPolicy && ["spot", "on-demand"].includes(host.lifecycle)
+    && host.spotVerified === (host.lifecycle === "spot") : host.lifecycle === "spot" && host.spotVerified === true;
+  requireProof(intent.hostPolicy === undefined || normalFleet, "unknown host policy");
   requireProof(host.schema === "paperclip.chat-remote-host/v1" && host.status === "qualified-host-only" && host.scope === "before-evaluation" && host.evaluationAdmitted === false
-    && host.spotVerified === true && host.lifecycle === "spot" && host.architecture === "x86_64" && host.region === "us-east-1", "host not qualified actual Spot");
+    && allowedHost && host.architecture === "x86_64" && host.region === "us-east-1", "host not qualified for explicit intent policy");
   requireProof(host.workflowRef === "paperclipai/paperclip/.github/workflows/runner-full-stack-e2e.yml@refs/heads/master"
     && host.workflowRunId === p.runId && host.workflowAttempt === p.runAttempt && host.sourceRevision === intent.sourceRevision
     && Array.isArray(host.executionIds) && host.executionIds.includes(result.executionId) && host.automaticRetries === 0 && hex(host.dockerEngineSha256), "host/result provenance mismatch");
@@ -69,7 +76,8 @@ export function validateRemoteCell(input: {
     measurementKey: `${result.accountingIdentity!.campaignId}/${result.accountingIdentity!.attemptId}/${result.executionId}`,
     artifactSha256: p.archiveSha256, scenarioId: selection.scenarioId, ordinal: selection.remoteOrdinal, sourceRevision: intent.sourceRevision,
     resultPath: input.result.path, resultSha256: input.result.sha256, hostSha256: input.host.sha256, definitionSha256: input.definition.sha256,
-    result, definitionDigest: definition.definitionDigest, graderDigest: definition.graderDigest, databaseLane };
+    result, definitionDigest: definition.definitionDigest, graderDigest: definition.graderDigest, databaseLane,
+    hostLifecycle: host.lifecycle, hostPolicy: normalFleet ? "aws-fleet-spot-or-on-demand" : "spot-only" };
 }
 /** A pre-attempt qualification refusal is an operation, never a case reservation.
  * Only these two source-qualified host-gate branches prove this zero-admission stage. */
@@ -106,7 +114,7 @@ export function remoteCellUsage(result: RunnerE2EResult): CampaignAttempt["usage
     costUsd: usage && ["reported", "partial"].includes(usage.costStatus) && Number.isFinite(usage.reportedCostUsd) ? usage.reportedCostUsd : null,
     coverage: usage?.costStatus ?? "unavailable" };
 }
-export interface RemoteImportRecord { importKey: string; measurementKey: string; resultSha256: string; artifactSha256: string; ordinal: 1 | 2; scenarioId: string }
+export interface RemoteImportRecord { importKey: string; measurementKey: string; resultSha256: string; artifactSha256: string; ordinal: 1 | 2; scenarioId: string; hostLifecycle: "spot" | "on-demand"; hostPolicy: ValidatedRemoteCell["hostPolicy"] }
 export type RemoteCampaign = ChatCampaign & { remoteImports?: RemoteImportRecord[] };
 /** Pure transaction: caller is the sole writer and atomically persists this returned
  * existing campaign ledger. A failed gate never partially reserves an ordinal.
@@ -134,6 +142,6 @@ export function applyRemoteCell(campaign: RemoteCampaign, cell: ValidatedRemoteC
   requireProof(scenario, "scenario absent from frozen campaign");
   reserveCampaignAttempt(scenario, structuredClone(attempt));
   completeCampaignAttempt(scenario, structuredClone(completion));
-  next.remoteImports = [...records, { importKey: cell.importKey, measurementKey: cell.measurementKey, resultSha256: cell.resultSha256, artifactSha256: cell.artifactSha256, ordinal: cell.ordinal, scenarioId: cell.scenarioId }];
+  next.remoteImports = [...records, { importKey: cell.importKey, measurementKey: cell.measurementKey, resultSha256: cell.resultSha256, artifactSha256: cell.artifactSha256, ordinal: cell.ordinal, scenarioId: cell.scenarioId, hostLifecycle: cell.hostLifecycle, hostPolicy: cell.hostPolicy }];
   return { campaign: next, duplicate: false };
 }