test(evals): enforce exact-source stock harness prerequisites

Run credential-free gates before live admission and verify retained evidence in direct browser execution. Include evaluated instruction sources in suite revisions and calibrate stale/missing evidence rejection.

Co-Authored-By: Paperclip <noreply@paperclip.ing>

doc/plans/2026-10-02-stock-harness-paperclip-checklist.md

@@ -3,8 +3,9 @@
 Created: 2026-10-02. Status: item 1 merged for native Codex app-server in
 [PR #14920](https://github.com/paperclipai/paperclip/pull/14920). Item 2's default
 hire manual is reduced to identity only, and common legacy startup/resume
-instructions are reduced locally. Additional carriers and native instructions
-remain open.
+instructions are in [PR #14948](https://github.com/paperclipai/paperclip/pull/14948).
+GitHub live qualification and a matched prior-instruction comparison are in progress.
+Additional carriers and native instructions remain open.
 
 Goal: keep the agent's stock harness behavior and add only what it needs to work
 with Paperclip. Apply this across legacy adapters, the new Runner, and their
@@ -118,7 +119,8 @@ and existing-test update.
   specialized wake contracts, custom templates, skills, and auth.
 - [ ] **2.2 Review additional legacy carriers.** Reduce Hermes local/gateway
   wrappers, review Pi system delivery, and check OpenClaw fresh-wake framing.
-  Preserve transport facts and user configuration.
+  Preserve transport facts and user configuration. Dotta deferred this item on
+  2026-10-02 for a later revisit; it remains open.
 - [ ] **2.3 Reduce native Runner instructions and constraints.** Improve
   discoverable tool documentation first, then shorten fixed guidance and
   consolidate completion rules. Verify prompt revisions, digests, and session
@@ -140,6 +142,16 @@ custom `promptTemplate`, and `bootstrapPromptTemplate` mechanics are unchanged.
 Hermes's own wrappers and Pi's system carrier remain follow-up 2.2; native
 fixed instructions and full-turn constraints remain follow-up 2.3.
 
+The later read-only 2.2 audit found another OpenClaw-owned HTTP identity,
+checkout/status, delegation, plan-approval and task-discovery wrapper in
+`buildWakeText`. Its short conversation branch is selected when optional task
+Markdown is present, including on ordinary task dispatch. Existing dispatch
+coverage does not assert the absence of conversation waiting language. This is
+a framing concern to verify, not a measured failure. Pi already uses additive
+`--append-system-prompt` delivery and suppresses the duplicate user-prompt copy;
+its next step is a delivery audit with minimal changes. These findings are
+deferred with 2.2 and are not implemented in PR #14948.
+
 The new defaults apply when prompts are assembled after deployment. Existing
 provider sessions can retain earlier startup instructions in their history
 until reset; no active session or saved custom template is rewritten here.

tests/runner-e2e/STOCK-HARNESS.md

@@ -20,8 +20,17 @@ including the Rust test. It writes JSON reports, the source SHA, a working-sourc
 fingerprint, selected checks, test counts, and `providerCalls: 0` under
 `results/stock-harness-preflight-<UTC>/`. Missing reports or required skipped
 assertions fail. Unrelated native tests filtered by the name selector remain
-explicitly skipped; they are not counted as executed coverage. Run this gate
-before qualifying live cells; the live launcher does not run Cargo implicitly.
+explicitly skipped; they are not counted as executed coverage. The live launcher
+runs the prerequisites automatically before loading local credentials or starting
+an isolated provider instance. Its subprocess receives only allowlisted toolchain
+and operating-system variables. Disposable GitHub runners may resolve Cargo
+dependencies; local prerequisites retain offline Cargo execution.
+
+The direct Playwright path also requires the retained prerequisite receipt. It
+verifies the exact checkout SHA, evaluated-source fingerprint, all requested
+Vitest reports and required assertions, and the Rust result before creating a
+company. Missing, stale, partial, or failed prerequisites cannot qualify a cell.
+Oracle/admission calibration is itself included in the prerequisite gate.
 
 ## Live matrix
 
@@ -77,8 +86,9 @@ existing secret sanitizer; screenshots remain the original captured pixels.
 
 The oracle's identity is independent of the implementation constant, so changing
 the shipped manual cannot silently change the expected result. The suite
-definition digest incorporates the fixture, journeys, grader, and execution
-integration sources. Positive and plausible-negative support tests exercise
+definition digest incorporates the evaluated default manual and shared prompt
+implementation, fixture, journeys, grader, prerequisite, and execution integration
+sources. Positive and plausible-negative support tests exercise
 missing/malformed receipts, manual regrowth, removed startup/resume procedures,
 absent connection guidance, and budget drift. Existing calibrated lifecycle
 graders still own task/chat success.
@@ -108,3 +118,41 @@ need a provider-session reset to restore a previously replaced vendor base.
 Private Runner protocol definitions remain separate: they use mock control-plane
 operations and cannot substitute for this public hiring and assembled-prompt
 coverage.
+
+## GitHub qualification and matched comparison
+
+The instruction reductions and coverage are under review in
+[PR #14948](https://github.com/paperclipai/paperclip/pull/14948).
+The first diagnostic native Codex skill cell
+[passed on GitHub](https://github.com/paperclipai/paperclip/actions/runs/37034213743)
+at `a63437069de58d22ee5adbcb6a6202c007dcf037`. All seven independent skill/task
+checks passed; the provider run lasted 34.745 seconds and the cell 56.660 seconds.
+Its tiny public hire bundle and budget receipts passed, and cleanup passed.
+This diagnostic predates enforced prerequisite admission and the expanded source
+digest, so it is retained separately from the final qualification matrix.
+
+Dispatch the trusted workflow from `master`, with `target_branch` naming the
+same-repository candidate and an exact cell selector first. The workflow resolves
+that target once to an immutable SHA. Never dispatch target-controlled workflow
+definitions with protected credentials. Protected environments, scoped provider
+keys, frozen target dependencies, report sanitization, publication, and existing
+bounded retry/cleanup policies retain their existing owners.
+
+A temporary `codex/stock-harness-previous-instructions` branch compares the same
+24 cells with the previous default manual and shared startup/resume prompts.
+It holds merged native Codex fix #14920 constant. Only those two production
+instruction sources differ. Its explicit historical structural oracle expects
+the old manual and records old generic procedures; the candidate's reduction
+assertions remain mandatory. Historical tests verify that prior contract.
+The independent skill/context/chat journeys, behavioral graders, fixtures,
+models, effort, tools, permissions, and credentials are identical. The retained
+comparison manifest records their hashes and the restored instruction revision.
+
+One full campaign per variant initially expects 48 provider turns, plus any
+existing bounded automatic retries; the earlier one-cell diagnostic remains
+separate. Compare behavioral results by profile and journey, with missing
+evidence unqualified. Keep structural instruction differences separate from task
+success. Report every attempt, failure attribution, provider timing, token usage,
+reported costs and unknown spend. A reported zero subtotal is not proof of zero
+provider spending. This small single-trial matrix cannot establish general coding
+quality or broad performance equivalence, and does not compare #14920 before/after.

tests/runner-e2e/launch.ts

@@ -50,6 +50,7 @@ import {
   type RunnerE2EResult,
 } from "./types.js";
 import { assertRunnerE2EPrerequisites } from "./prerequisites.js";
+import { prepareStockHarnessPreflight, STOCK_PREFLIGHT_ENV } from "./stock-harness-admission.js";
 import {
   reapNewDetachedDarwinSharedMemory,
   snapshotDarwinSharedMemory,
@@ -1098,6 +1099,9 @@ async function main() {
   // Keep admission before local-env loading and credential checks. Pending
   // profiles remain discoverable, but cannot reach a provider.
   assertRunnerE2EPrerequisites(executions);
+  if (executions.some(execution => execution.suite.id === "stock-harness")) {
+    process.env[STOCK_PREFLIGHT_ENV] = prepareStockHarnessPreflight();
+  }
 
   await loadLocalEnvironment(process.env);
   const missingCredentials = [

tests/runner-e2e/runner.spec.ts

@@ -15,6 +15,7 @@ import { runEverydayFlow } from "./everyday-flow.js";
 import { gradeTaskTitle } from "./task-titles.js";
 import { runContextIntegrityFlow } from "./context-integrity-flow.js";
 import { captureStockHarness, gradeStockHarness, gradeStockHire } from "./stock-harness.js";
+import { verifyStockHarnessPreflight, STOCK_PREFLIGHT_ENV } from "./stock-harness-admission.js";
 import { createTaskThroughUi, submitTaskReply } from "./user-actions.js";
 
 import { runFirstTaskFlow, setupFirstTaskFixtures } from "./first-task-flow.js";
@@ -815,6 +816,10 @@ for (const execution of executions) {
     });
 
     try {
+      if (execution.suite.id === "stock-harness") {
+        const receipt = verifyStockHarnessPreflight(process.env[STOCK_PREFLIGHT_ENV]);
+        await writeSanitizedJson(snapshotsDir, "stock-harness-preflight.json", receipt, secrets);
+      }
       const experimental = await api.patch<{
         enableNativeRunner: boolean;
       }>("/api/instance/settings/experimental", {

tests/runner-e2e/stock-harness-admission.test.ts

@@ -0,0 +1,43 @@
+import { beforeEach, describe, expect, it, vi } from "vitest";
+import { spawnSync } from "node:child_process";
+import { prepareStockHarnessPreflight, stockPreflightEnvironment, verifyStockHarnessPreflight } from "./stock-harness-admission.js";
+import { CREDENTIAL_NAMES } from "./types.js";
+
+vi.mock("node:child_process", () => ({ spawnSync: vi.fn() }));
+const spawn = vi.mocked(spawnSync);
+beforeEach(() => { vi.resetAllMocks(); vi.unstubAllEnvs(); });
+
+describe("stock harness credential-free admission", () => {
+  it("allows toolchain paths and excludes every present or future credential", () => {
+    const source = { PATH: "/bin", HOME: "/home/fixture", CARGO_HOME: "/cargo", CI: "true",
+      GH_TOKEN: "secret", FUTURE_PROVIDER_API_KEY: "secret", ...Object.fromEntries(CREDENTIAL_NAMES.map(name => [name, "secret"])) };
+    expect(stockPreflightEnvironment(source)).toEqual({ PATH: "/bin", HOME: "/home/fixture", CARGO_HOME: "/cargo", CI: "true" });
+  });
+  it("rejects missing receipts without invoking a process", () => {
+    expect(() => verifyStockHarnessPreflight(undefined)).toThrow("no prerequisite receipt");
+    expect(spawn).not.toHaveBeenCalled();
+  });
+  it("fails before provider execution when the prerequisite subprocess fails", () => {
+    spawn.mockReturnValue({ status: 1 } as ReturnType<typeof spawnSync>);
+    expect(() => prepareStockHarnessPreflight()).toThrow("before provider execution");
+    expect(spawn).toHaveBeenCalledTimes(1);
+  });
+  it("verifies retained evidence after a passing prerequisite subprocess", () => {
+    spawn.mockReturnValueOnce({ status: 0 } as ReturnType<typeof spawnSync>);
+    spawn.mockReturnValueOnce({ status: 0, stdout: '{"passed":true}' } as ReturnType<typeof spawnSync>);
+    const receipt = prepareStockHarnessPreflight();
+    expect(receipt).toMatch(/stock-harness-preflight-.*\/preflight.json$/);
+    expect(spawn.mock.calls[1]?.[1]).toContain(`--verify=${receipt}`);
+  });
+  it("allows Cargo dependency resolution only on the disposable GitHub runner", () => {
+    vi.stubEnv("GITHUB_ACTIONS", "true");
+    spawn.mockReturnValueOnce({ status: 0 } as ReturnType<typeof spawnSync>);
+    spawn.mockReturnValueOnce({ status: 0, stdout: '{}' } as ReturnType<typeof spawnSync>);
+    prepareStockHarnessPreflight();
+    expect(spawn.mock.calls[0]?.[1]).toContain("--allow-rust-network");
+  });
+  it("refuses failed receipt verification", () => {
+    spawn.mockReturnValue({ status: 1, stderr: "stale source" } as ReturnType<typeof spawnSync>);
+    expect(() => verifyStockHarnessPreflight("/fixture/preflight.json")).toThrow("stale source");
+  });
+});

tests/runner-e2e/stock-harness-admission.ts

@@ -0,0 +1,36 @@
+import { spawnSync } from "node:child_process";
+import { randomUUID } from "node:crypto";
+import path from "node:path";
+
+const root = path.resolve(import.meta.dirname, "../..");
+export const STOCK_PREFLIGHT_ENV = "PAPERCLIP_RUNNER_E2E_STOCK_PREFLIGHT";
+
+// An allowlist keeps every provider credential, ambient auth override, and GH
+// token out of the prerequisite subprocess, including secrets added later.
+export function stockPreflightEnvironment(source: NodeJS.ProcessEnv): NodeJS.ProcessEnv {
+  return Object.fromEntries([
+    "PATH", "HOME", "TMPDIR", "TMP", "TEMP", "SYSTEMROOT", "LANG", "LC_ALL",
+    "CARGO_HOME", "RUSTUP_HOME", "CI", "GITHUB_ACTIONS",
+  ].flatMap(name => source[name] === undefined ? [] : [[name, source[name]]]));
+}
+
+export function verifyStockHarnessPreflight(receiptPath: string | undefined) {
+  if (!receiptPath) throw new Error("Stock harness has no prerequisite receipt; use the live launcher.");
+  const run = spawnSync(process.execPath, [path.join(root, "tests/runner-e2e/stock-harness-checks.mjs"), `--verify=${receiptPath}`], {
+    cwd: root, env: stockPreflightEnvironment(process.env), encoding: "utf8", timeout: 30_000,
+  });
+  if (run.status !== 0) throw new Error(`Stock harness prerequisite verification failed: ${run.stderr || run.error?.message || run.status}`);
+  return JSON.parse(run.stdout) as Record<string, unknown>;
+}
+
+export function prepareStockHarnessPreflight() {
+  const output = path.join(root, "tests/runner-e2e/results", `stock-harness-preflight-${randomUUID()}`);
+  const receipt = path.join(output, "preflight.json");
+  const run = spawnSync(process.execPath, [path.join(root, "tests/runner-e2e/stock-harness-checks.mjs"),
+    `--output-dir=${output}`, ...(process.env.GITHUB_ACTIONS === "true" ? ["--allow-rust-network"] : [])], {
+    cwd: root, env: stockPreflightEnvironment(process.env), stdio: "inherit", timeout: 50 * 60_000,
+  });
+  if (run.status !== 0) throw new Error(`Stock harness prerequisites failed before provider execution. Retained receipt: ${receipt}`);
+  verifyStockHarnessPreflight(receipt);
+  return receipt;
+}

tests/runner-e2e/stock-harness-checks.mjs

@@ -38,6 +38,13 @@ export const stockHarnessGates = [
   { id: "SH-3-hermes", name: "Hermes shared-prompt delivery", cwd: "packages/adapters/hermes",
     files: ["src/server/prompt-rendering.test.ts"],
     required: ["renders standard assignment wake with task authority", "renders scoped planning wake authority"] },
+  { id: "SH-eval", name: "Independent oracle and qualification admission", cwd: ".",
+    config: "tests/runner-e2e/vitest.config.ts",
+    files: ["tests/runner-e2e/stock-harness.test.ts", "tests/runner-e2e/stock-harness-checks.test.mjs",
+      "tests/runner-e2e/stock-harness-admission.test.ts", "tests/runner-e2e/stock-harness-digest.test.ts"],
+    required: ["rejects old SHA before providers", "allows toolchain paths and excludes every present or future credential",
+      "changes when the evaluated server/src/onboarding-assets/default/AGENTS.md changes",
+      "changes when the evaluated packages/adapter-utils/src/server-utils.ts changes"] },
 ];
 
 export function gradeGate(gate, report, exitCode) {
@@ -54,37 +61,12 @@ export function gradeGate(gate, report, exitCode) {
     failedTests: report?.numFailedTests ?? 0, pendingTests: report?.numPendingTests ?? 0 };
 }
 
-export function main(args = process.argv.slice(2)) {
-  if (args.includes("--list")) {
-    console.log(JSON.stringify({ gates: stockHarnessGates, rust: "runtime_instructions_are_additive_for_codex_on_start_and_resume", live: "not invoked" }, null, 2));
-    return;
-  }
-  if (args.length) throw new Error("Use --list or no arguments; this command never runs paid providers.");
-  const output = join(root, "tests/runner-e2e/results", `stock-harness-preflight-${new Date().toISOString().replaceAll(":", "-")}`);
-  mkdirSync(output, { recursive: true });
-  const results = [];
-  for (const gate of stockHarnessGates) {
-    console.log(`Checking ${gate.id}: ${gate.name}`);
-    const file = join(output, `${gate.id}.json`);
-    const run = spawnSync(process.execPath, [join(root, "node_modules/vitest/vitest.mjs"), "run", ...gate.files,
-      ...(gate.testPattern ? ["--testNamePattern", gate.testPattern] : []),
-      "--reporter=default", "--reporter=json", `--outputFile.json=${file}`],
-    { cwd: resolve(root, gate.cwd), stdio: "inherit", timeout: 10 * 60_000 });
-    let report;
-    try { report = JSON.parse(readFileSync(file, "utf8")); } catch { /* missing evidence fails closed below */ }
-    results.push(gradeGate(gate, report, run.status));
-  }
-  const rust = spawnSync("cargo", ["test", "--offline", "-p", "paperclip-runner-core", "--test", "codex_provider",
-    "runtime_instructions_are_additive_for_codex_on_start_and_resume", "--", "--exact"],
-  { cwd: join(root, "packages/paperclip-runner/runner"), encoding: "utf8", timeout: 10 * 60_000 });
-  const rustOutput = `${rust.stdout ?? ""}\n${rust.stderr ?? ""}`;
-  writeFileSync(join(output, "rust.txt"), rustOutput);
-  results.push({ id: "SH-1-rust", passed: rust.status === 0 && /test runtime_instructions_are_additive_for_codex_on_start_and_resume \.\.\. ok/.test(rustOutput), exitCode: rust.status });
-  const git = spawnSync("git", ["rev-parse", "HEAD"], { cwd: root, encoding: "utf8" });
+export function sourceFingerprint() {
   const hash = createHash("sha256");
   const sources = new Set([
     ...stockHarnessGates.flatMap(gate => gate.files.map(file => join(gate.cwd, file))),
     "tests/runner-e2e/stock-harness.ts", "tests/runner-e2e/stock-harness-checks.mjs", "tests/runner-e2e/catalog.ts",
+    "tests/runner-e2e/stock-harness-admission.ts", "tests/runner-e2e/launch.ts", "tests/runner-e2e/runner.spec.ts",
     "packages/adapter-utils/src/server-utils.ts", "server/src/onboarding-assets/default/AGENTS.md", "server/src/routes/agents.ts",
     "packages/paperclip-runner/src/drivers/codex/codex-app-server-driver-impl.ts",
     "packages/paperclip-runner/src/live/runnerd-codex-transport.ts",
@@ -98,9 +80,76 @@ export function main(args = process.argv.slice(2)) {
     try { hash.update(readFileSync(join(root, source))); }
     catch { sourceErrors.push(source); }
   }
+  return { fingerprint: hash.digest("hex"), sourceErrors };
+}
+
+export function assertPreflightReceipt(report, current) {
+  const expected = [...stockHarnessGates.map(gate => gate.id), "SH-1-rust"];
+  if (report?.schema !== "paperclip.stock-harness-preflight.v1" || report.passed !== true ||
+      report.providerCalls !== 0 || report.sourceSha !== current.sha ||
+      report.sourceFingerprint !== current.fingerprint || report.sourceErrors?.length !== 0 ||
+      !Array.isArray(report.gates) || report.gates.length !== expected.length ||
+      expected.some(id => report.gates.filter(gate => gate.id === id && gate.passed === true && gate.exitCode === 0).length !== 1)) {
+    throw new Error("Stock harness requires passing prerequisites for this exact source SHA and fingerprint.");
+  }
+  return report;
+}
+
+export function main(args = process.argv.slice(2)) {
+  if (args.includes("--list")) {
+    console.log(JSON.stringify({ gates: stockHarnessGates, rust: "runtime_instructions_are_additive_for_codex_on_start_and_resume", live: "not invoked" }, null, 2));
+    return;
+  }
+  if (args.some(arg => !arg.startsWith("--output-dir=") && !arg.startsWith("--verify=") && arg !== "--allow-rust-network"))
+    throw new Error("Use --list, --output-dir=<path>, --verify=<receipt>, or --allow-rust-network; never paid providers.");
+  const git = spawnSync("git", ["rev-parse", "HEAD"], { cwd: root, encoding: "utf8" });
+  const verify = args.find(arg => arg.startsWith("--verify="))?.slice("--verify=".length);
+  if (verify) {
+    const source = sourceFingerprint();
+    if (git.status !== 0 || source.sourceErrors.length) throw new Error("Cannot verify stock harness source provenance.");
+    const report = assertPreflightReceipt(JSON.parse(readFileSync(verify, "utf8")), {
+      sha: git.stdout.trim(), fingerprint: source.fingerprint,
+    });
+    const output = resolve(verify, "..");
+    for (const gate of stockHarnessGates) {
+      const grade = gradeGate(gate, JSON.parse(readFileSync(join(output, `${gate.id}.json`), "utf8")), 0);
+      if (!grade.passed) throw new Error(`Missing or failed retained prerequisite assertions: ${gate.id}`);
+    }
+    if (!/test runtime_instructions_are_additive_for_codex_on_start_and_resume \.\.\. ok/.test(readFileSync(join(output, "rust.txt"), "utf8")))
+      throw new Error("Missing passing retained Rust prerequisite.");
+    console.log(JSON.stringify(report));
+    return report;
+  }
+  const output = args.find(arg => arg.startsWith("--output-dir="))?.slice("--output-dir=".length) ??
+    join(root, "tests/runner-e2e/results", `stock-harness-preflight-${new Date().toISOString().replaceAll(":", "-")}`);
+  mkdirSync(output, { recursive: true });
+  const env = Object.fromEntries([
+    "PATH", "HOME", "TMPDIR", "TMP", "TEMP", "SYSTEMROOT", "LANG", "LC_ALL",
+    "CARGO_HOME", "RUSTUP_HOME", "CI", "GITHUB_ACTIONS",
+  ].flatMap(name => process.env[name] === undefined ? [] : [[name, process.env[name]]]));
+  const results = [];
+  for (const gate of stockHarnessGates) {
+    console.log(`Checking ${gate.id}: ${gate.name}`);
+    const file = join(output, `${gate.id}.json`);
+    const run = spawnSync(process.execPath, [join(root, "node_modules/vitest/vitest.mjs"), "run", ...gate.files,
+      ...(gate.config ? ["--config", gate.config] : []),
+      ...(gate.testPattern ? ["--testNamePattern", gate.testPattern] : []),
+      "--reporter=default", "--reporter=json", `--outputFile.json=${file}`],
+    { cwd: resolve(root, gate.cwd), env, stdio: "inherit", timeout: 10 * 60_000 });
+    let report;
+    try { report = JSON.parse(readFileSync(file, "utf8")); } catch { /* missing evidence fails closed below */ }
+    results.push(gradeGate(gate, report, run.status));
+  }
+  const rust = spawnSync("cargo", ["test", ...(args.includes("--allow-rust-network") ? [] : ["--offline"]), "-p", "paperclip-runner-core", "--test", "codex_provider",
+    "runtime_instructions_are_additive_for_codex_on_start_and_resume", "--", "--exact"],
+  { cwd: join(root, "packages/paperclip-runner/runner"), env, encoding: "utf8", timeout: 10 * 60_000 });
+  const rustOutput = `${rust.stdout ?? ""}\n${rust.stderr ?? ""}`;
+  writeFileSync(join(output, "rust.txt"), rustOutput);
+  results.push({ id: "SH-1-rust", passed: rust.status === 0 && /test runtime_instructions_are_additive_for_codex_on_start_and_resume \.\.\. ok/.test(rustOutput), exitCode: rust.status });
+  const { fingerprint, sourceErrors } = sourceFingerprint();
   const report = { schema: "paperclip.stock-harness-preflight.v1", sourceSha: git.stdout?.trim() || null,
-    sourceFingerprint: hash.digest("hex"), measuredAt: new Date().toISOString(), providerCalls: 0,
-    live: "not_run", passed: sourceErrors.length === 0 && results.every(row => row.passed), sourceErrors, gates: results };
+    sourceFingerprint: fingerprint, measuredAt: new Date().toISOString(), providerCalls: 0,
+    live: "not_run", passed: git.status === 0 && sourceErrors.length === 0 && results.every(row => row.passed), sourceErrors, gates: results };
   writeFileSync(join(output, "preflight.json"), JSON.stringify(report, null, 2) + "\n");
   console.log(`Stock harness prerequisite evidence: ${output}/preflight.json`);
   if (!report.passed) process.exitCode = 1;

tests/runner-e2e/stock-harness-checks.test.mjs

@@ -1,5 +1,5 @@
 import { describe, expect, it } from "vitest";
-import { gradeGate, stockHarnessGates } from "./stock-harness-checks.mjs";
+import { assertPreflightReceipt, gradeGate, stockHarnessGates } from "./stock-harness-checks.mjs";
 
 const gate = { id: "SH-test", name: "Boundary", files: ["boundary.test.ts"], required: ["must preserve instructions"] };
 const report = () => ({ testResults: [{ name: "/repo/boundary.test.ts", status: "passed",
@@ -7,7 +7,7 @@ const report = () => ({ testResults: [{ name: "/repo/boundary.test.ts", status:
 numTotalTests: 1, numPassedTests: 1, numFailedTests: 0, numPendingTests: 0 });
 describe("stock harness prerequisite coverage", () => {
   it("maps every implemented change to an executable gate", () => {
-    expect(stockHarnessGates.map(gate => gate.id)).toEqual(["SH-1", "SH-2", "SH-3", "SH-3-hermes"]);
+    expect(stockHarnessGates.map(gate => gate.id)).toEqual(["SH-1", "SH-2", "SH-3", "SH-3-hermes", "SH-eval"]);
     expect(stockHarnessGates.every(gate => gate.files.length > 0 && gate.required.length > 0)).toBe(true);
   });
   it("accepts an executed passing boundary", () => expect(gradeGate(gate, report(), 0).passed).toBe(true));
@@ -38,3 +38,23 @@ describe("stock harness prerequisite coverage", () => {
     expect(stockHarnessGates.find(gate => gate.id === "SH-3-hermes").cwd).toBe("packages/adapters/hermes");
   });
 });
+
+describe("stock harness prerequisite admission", () => {
+  const current = { sha: "a".repeat(40), fingerprint: "b".repeat(64) };
+  const receipt = () => ({ schema: "paperclip.stock-harness-preflight.v1", passed: true,
+    providerCalls: 0, sourceSha: current.sha, sourceFingerprint: current.fingerprint,
+    sourceErrors: [], gates: [...stockHarnessGates.map(g => g.id), "SH-1-rust"].map(id => ({ id, passed: true, exitCode: 0 })) });
+  it("admits the same passing source revision", () => expect(assertPreflightReceipt(receipt(), current).passed).toBe(true));
+  it.each([
+    ["old SHA", r => { r.sourceSha = "c".repeat(40); }],
+    ["changed source", r => { r.sourceFingerprint = "d".repeat(64); }],
+    ["failed boundary", r => { r.gates[0].passed = false; }],
+    ["nonzero exit", r => { r.gates[0].exitCode = 1; }],
+    ["missing Rust", r => { r.gates.pop(); }],
+    ["duplicate boundary", r => { r.gates[1] = r.gates[0]; }],
+    ["provider calls", r => { r.providerCalls = 1; }],
+    ["missing source", r => { r.sourceErrors.push("missing.ts"); }],
+  ])("rejects %s before providers", (_name, mutate) => {
+    const r = receipt(); mutate(r); expect(() => assertPreflightReceipt(r, current)).toThrow("exact source SHA and fingerprint");
+  });
+});

tests/runner-e2e/stock-harness-digest.test.ts

@@ -0,0 +1,15 @@
+import { describe, expect, it, vi } from "vitest";
+import { readFileSync } from "node:fs";
+import { stockHarnessSourceDigest } from "./stock-harness.js";
+
+vi.mock("node:fs", async importOriginal => ({ ...await importOriginal<typeof import("node:fs")>(), readFileSync: vi.fn() }));
+
+describe("stock harness instruction revision", () => {
+  it.each(["server/src/onboarding-assets/default/AGENTS.md", "packages/adapter-utils/src/server-utils.ts"])(
+    "changes when the evaluated %s changes", source => {
+      vi.mocked(readFileSync).mockImplementation(() => Buffer.from("unchanged"));
+      const original = stockHarnessSourceDigest();
+      vi.mocked(readFileSync).mockImplementation(file => Buffer.from(String(file).endsWith(source) ? "changed instructions" : "unchanged"));
+      expect(stockHarnessSourceDigest()).not.toBe(original);
+    });
+});

tests/runner-e2e/stock-harness.ts

@@ -110,6 +110,9 @@ export function stockHarnessSourceDigest() {
   for (const source of [
     "stock-harness.ts", "context-integrity-cases.ts", "context-integrity-scoring.ts",
     "context-integrity-flow.ts", "chat-cases.ts", "chat-flow.ts", "live-fixtures.ts", "runner.spec.ts",
+    "stock-harness-checks.mjs", "stock-harness-admission.ts",
+    "../../server/src/onboarding-assets/default/AGENTS.md",
+    "../../packages/adapter-utils/src/server-utils.ts",
   ]) hash.update(source).update(readFileSync(new URL(source, import.meta.url)));
   return hash.digest("hex");
 }