test(evals): admit matched ACP instruction variants with single attempts

Pin exact historical and reduced manual bytes, keep source-variant structural observations separate from behavioral outcomes, and include ACP delivery/persistence and retry sources in qualification fingerprints. Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-12 08:19:54 +02:00 · 2026-10-03 07:03:57 -05:00
1 parent b54a456d40
commit da529b7ec4
@@ -7,6 +7,19 @@ bundle. Their passing results therefore did not qualify a hire with the tiny
 production default. The new suite omits that fixture bundle and lets the public
 agent-creation route materialize the shipped default.

+The 2026-10-03 legacy ACP Claude repair compares only the original
+`assigned-skill-explicit-invocation` cell on matched current-master sources.
+Both variants carry identical bounded skill-description/staged-path delivery
+and env-free session persistence. Only the historical versus reduced default
+manual/shared prompts and their declared structural unit expectations differ.
+The public bundle/procedure observations derive from exact admitted source
+bytes (eight-word identity or the independently pinned 4,249-byte historical
+manual), never a branch/environment label. Candidate absence assertions stay
+intact. Independent document/task and exact-credential guards are unchanged.
+All stock tasks enforce `single_attempt` in the hosted launcher; automatic
+product disposition recovery still counts as actual usage. This is not a new
+24-cell campaign or qualification of the frozen native-completion context.
+
 ## Coverage contract

 | Change | Required deterministic evidence | Product E2E evidence |
@@ -1235,12 +1235,14 @@ export const runnerSuites: readonly RunnerSuiteFixture[] = [
    profiles: [...contextIntegrityProfiles.filter(profile => !pendingContextIntegrityProfiles.includes(profile)),
      runnerProfiles.find(profile => profile.id === "legacy-opencode")!].map(productionDefaultHireProfile),
    environments: [localEnvironment],
-    tasks: [...contextIntegrityTasks, chatTasks.find(task => task.id === "continuity-restart")!, paperclipDocumentTask],
+    tasks: [...contextIntegrityTasks, chatTasks.find(task => task.id === "continuity-restart")!, paperclipDocumentTask]
+      .map(task => ({ ...task, automaticRetryPolicy: "single_attempt" as const })),
    expectedMatrixSize: 26,
    excludedExecutionIds: ["legacy-codex", "legacy-acp-codex", "legacy-acp-claude", "runner-codex", "runner-acpx-claude", "runner-opencode"]
      .map(profile => `stock-harness.${profile}.local.${paperclipDocumentTask.id}`),
    definitionMetadata: {
      version: 2, instructions: "production-default-hire", scheduling: "explicit-only",
+      automaticRetryPolicy: "single_attempt", maximumAttemptsPerCell: 1,
      sourceDigest: stockHarnessSourceDigest(),
      operationalSkillSources: stockHarnessSkillSources(),
      grading: "public-default-bundle-and-delivered-prompts-plus-independent-lifecycle-oracles",
@@ -0,0 +1,24 @@
+You are an agent at Paperclip company.
+
+## Execution Contract
+
+- Start actionable work in the same heartbeat. Do not stop at a plan unless the issue explicitly asks for planning.
+- Keep the work moving until it is done. If you need QA to review it, ask them. If you need your boss to review it, ask them.
+- Leave durable progress in task comments, documents, or work products, then update the issue to a clear final disposition before you exit.
+- When your work produces a user-inspectable deliverable file, follow the Paperclip skill's "Generated Artifacts and Work Products" workflow before final disposition. Use `skills/paperclip/scripts/paperclip-upload-artifact.sh` when working in this repo, create/update an artifact work product when the file is the deliverable, and link the uploaded attachment in the final comment. Do not rely on local filesystem paths as the only access path. If an important file intentionally remains workspace-only, create/update a work product with `metadata.resourceRef.kind: "workspace_file"` and a workspace-relative path, then name that work product and path in the final comment. Treat browse/search as a fallback for recovering workspace files, not the preferred deliverable path.
+- When your work produces or updates an operator-facing engineering output, create/update the matching work product: `pull_request` for opened PRs, `preview_url` for published previews, `runtime_service` for managed preview/dev services, `commit` for notable pushed commits, and `branch` when the branch itself is the handoff. A comment is not a substitute for the work product access path.
+- Comments, documents, screenshots, work products, and `Remaining` bullets are evidence, not valid liveness paths by themselves.
+- Final disposition checklist: mark `done` when complete and verified; use `in_review` only with a real reviewer, approval, interaction, or monitor path; use `blocked` only with first-class blockers or a named unblock owner/action; create delegated follow-up issues with blockers when another agent owns the next step; keep `in_progress` only when a live continuation path exists.
+- Use child issues for parallel or long delegated work instead of polling agents, sessions, or processes.
+- Create child issues directly when you know what needs to be done. If the board/user needs to choose suggested tasks, answer structured questions, or confirm a proposal first, create an issue-thread interaction on the current issue with `POST /api/issues/{issueId}/interactions` using `kind: "suggest_tasks"`, `kind: "ask_user_questions"`, or `kind: "request_confirmation"`.
+- Use `request_confirmation` instead of asking for yes/no decisions in markdown. Before presenting a plan for review, you MUST complete this publish contract:
+  1. `PUT /issues/{id}/documents/plan` with `{ format: 'markdown', body, changeSummary }`.
+  2. Re-`GET /documents/plan`, assert it returns `200`, and capture its `latestRevisionId`.
+  3. Only then create `request_confirmation` with `target={ type: 'issue_document', key: 'plan', revisionId: latestRevisionId }` and `idempotencyKey=confirmation:{issueId}:plan:{revisionId}`.
+  4. Wait for acceptance before creating implementation subtasks.
+  Never present a plan only in a thread comment or through `ask_user_questions`; comments are supporting context and questions are for gathering input, not plan review.
+- `ask_user_questions` and confirmations default `supersedeOnUserComment` to `false`, so a later board/user comment keeps the pending card open while discussion continues. Set it to `true` when a new comment should replace the pending request. If you wake up from a superseding comment, revise the artifact, question set, or proposal and create a fresh interaction if input is still needed.
+- For human input, save a pending question/confirmation interaction and set `in_review`; prose alone does not create a waiting path. Use `blockedByIssueIds` for issue dependencies. An agent may set an `unblockDescriptor` only for itself (`owner: { "agentId": "<your-agent-id>" }` plus `action`), not for the board/user or another agent.
+- Respect budget, pause/cancel, approval gates, and company boundaries.
+
+Do not let work sit here. You must always update your task with a comment.
@@ -5,6 +5,7 @@ import { spawnSync } from "node:child_process";
 import { createHash } from "node:crypto";
 import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
 import { join, resolve } from "node:path";
+import { readStockInstructionVariant } from "./stock-harness-instruction-variant.mjs";

 const root = resolve(import.meta.dirname, "../..");
 export const stockHarnessGates = [
@@ -27,12 +28,15 @@ export const stockHarnessGates = [
    "packages/adapter-utils/src/server-utils.test.ts",
    "packages/adapter-utils/src/prompt-sections.test.ts",
    "packages/adapter-utils/src/acpx-engine/execute.test.ts",
+    "packages/adapter-utils/src/acpx-engine/ephemeral-session-environment.test.ts",
    "packages/adapters/pi-local/src/server/execute.remote.test.ts",
    "packages/adapters/opencode-local/src/server/execute.test.ts",
    "packages/adapters/cursor-cloud/src/server/execute.test.ts",
    "server/src/__tests__/codex-local-execute.test.ts",
  ], required: ["keeps task and chat defaults to identity and connection guidance",
-    "does not restore generic procedures on resume or with the legacy opt-in"] },
+    "does not restore generic procedures on resume or with the legacy opt-in",
+    "integrates the env-free store", "advertises folded routing metadata", "bounds routing descriptions",
+    "loads an old env-bearing file with rotated credentials", "drops a skill that fails to materialize"] },
  // Hermes is not in the root Vitest project list. Run its package config so
  // the requested file cannot silently disappear from discovery.
  { id: "SH-3-hermes", name: "Hermes shared-prompt delivery", cwd: "packages/adapters/hermes",
@@ -42,7 +46,7 @@ export const stockHarnessGates = [
    config: "tests/runner-e2e/vitest.config.ts",
    files: ["tests/runner-e2e/stock-harness-manifest.test.ts", "tests/runner-e2e/paperclip-document.test.ts", "tests/runner-e2e/stock-harness.test.ts", "tests/runner-e2e/stock-harness-checks.test.mjs",
      "tests/runner-e2e/stock-harness-admission.test.ts", "tests/runner-e2e/stock-harness-digest.test.ts",
-      "tests/runner-e2e/select-rerun-artifacts.test.ts"],
+      "tests/runner-e2e/select-rerun-artifacts.test.ts", "tests/runner-e2e/stock-harness-instruction-variant.test.mjs", "tests/runner-e2e/automatic-retry.test.ts"],
    required: ["requires generated capability manifests for the current skill sources", "the shipped recipe delivers the current", "the shipped recipe supports an unnumbered issue", "rejects old SHA before providers", "allows toolchain paths and excludes every present or future credential",
      "changes when the evaluated server/src/onboarding-assets/default/AGENTS.md changes",
      "changes when the evaluated packages/adapter-utils/src/server-utils.ts changes",
@@ -50,6 +54,19 @@ export const stockHarnessGates = [
      "retains credential-free prerequisites inside the exact campaign root"] },
 ];

+export function stockHarnessGatesForVariant(variant) {
+  if (variant !== "reduced" && variant !== "historical") throw new Error("Unknown stock instruction variant.");
+  return stockHarnessGates.map(gate => variant !== "historical" ? gate : {
+    ...gate,
+    required: gate.required.map(name => name === "materializes minimal default instructions for non-CEO agents with no prompt template"
+      ? "materializes the bundled default instruction set for non-CEO agents with no prompt template"
+      : name === "keeps task and chat defaults to identity and connection guidance"
+        ? "keeps the default local-agent prompt action-oriented"
+        : name === "does not restore generic procedures on resume or with the legacy opt-in"
+          ? "adds the execution contract to resume delta prompts and opted-in fresh prompts" : name),
+  });
+}
+
 export function gradeGate(gate, report, exitCode) {
  const assertions = (report?.testResults ?? []).flatMap(file => file.assertionResults ?? []);
  const requirements = (gate.required ?? []).map(name => ({ name,
@@ -70,6 +87,9 @@ export function sourceFingerprint() {
    ...stockHarnessGates.flatMap(gate => gate.files.map(file => join(gate.cwd, file))),
    "tests/runner-e2e/stock-harness.ts", "tests/runner-e2e/stock-harness-checks.mjs", "tests/runner-e2e/catalog.ts",
    "tests/runner-e2e/stock-harness-admission.ts", "tests/runner-e2e/launch.ts", "tests/runner-e2e/runner.spec.ts",
+    "tests/runner-e2e/stock-harness-instruction-variant.mjs", "tests/runner-e2e/stock-harness-instruction-variant.d.mts", "tests/runner-e2e/fixtures/stock-harness/historical-default-agents.md",
+    "tests/runner-e2e/automatic-retry.ts", "tests/runner-e2e/types.ts",
+    "packages/adapter-utils/src/acpx-engine/execute.ts", "packages/adapter-utils/src/acpx-engine/ephemeral-session-environment.ts",
    "tests/runner-e2e/context-integrity-cases.ts", "tests/runner-e2e/context-integrity-flow.ts", "tests/runner-e2e/context-integrity-scoring.ts",
    "tests/runner-e2e/stock-harness-manifest.ts", "packages/paperclip-runner/scripts/generate-capability-contract.mjs",
    "packages/paperclip-runner/spec/capability/source-contract.json",
@@ -105,6 +125,7 @@ export function sourceFingerprint() {
 }

 export function assertPreflightReceipt(report, current) {
+  const instructionVariant = readStockInstructionVariant();
  const expected = [...stockHarnessGates.map(gate => gate.id), "SH-1-rust"];
  if (report?.schema !== "paperclip.stock-harness-preflight.v3" || report.passed !== true ||
      report.setup?.passed !== true || report.setup?.exitCode !== 0 ||
@@ -113,6 +134,7 @@ export function assertPreflightReceipt(report, current) {
      report.setup?.fakeCodexSha256 !== current.fakeCodexSha256 ||
      report.providerCalls !== 0 || report.sourceSha !== current.sha ||
      report.sourceFingerprint !== current.fingerprint || report.sourceErrors?.length !== 0 ||
+      report.instructionVariant?.variant !== instructionVariant.variant || report.instructionVariant?.sha256 !== instructionVariant.sha256 ||
      !Array.isArray(report.gates) || report.gates.length !== expected.length ||
      expected.some(id => report.gates.filter(gate => gate.id === id && gate.passed === true && gate.exitCode === 0).length !== 1)) {
    throw new Error("Stock harness requires passing prerequisites for this exact source SHA and fingerprint.");
@@ -121,8 +143,11 @@ export function assertPreflightReceipt(report, current) {
 }

 export function main(args = process.argv.slice(2)) {
+  const { variant, sha256 } = readStockInstructionVariant();
+  const instructionVariant = { variant, sha256 };
+  const gates = stockHarnessGatesForVariant(variant);
  if (args.includes("--list")) {
-    console.log(JSON.stringify({ gates: stockHarnessGates, rust: "runtime_instructions_are_additive_for_codex_on_start_and_resume", live: "not invoked" }, null, 2));
+    console.log(JSON.stringify({ gates, instructionVariant, rust: "runtime_instructions_are_additive_for_codex_on_start_and_resume", live: "not invoked" }, null, 2));
    return;
  }
  if (args.some(arg => !arg.startsWith("--output-dir=") && !arg.startsWith("--verify=") && arg !== "--allow-rust-network"))
@@ -140,7 +165,7 @@ export function main(args = process.argv.slice(2)) {
        "packages/paperclip-runner/runner/target/debug/fake-codex-app-server"))).digest("hex"),
    });
    const output = resolve(verify, "..");
-    for (const gate of stockHarnessGates) {
+    for (const gate of gates) {
      const grade = gradeGate(gate, JSON.parse(readFileSync(join(output, `${gate.id}.json`), "utf8")), 0);
      if (!grade.passed) throw new Error(`Missing or failed retained prerequisite assertions: ${gate.id}`);
    }
@@ -176,7 +201,7 @@ export function main(args = process.argv.slice(2)) {
  if (!setup.passed) {
    const source = sourceFingerprint();
    writeFileSync(join(output, "preflight.json"), JSON.stringify({
-      schema: "paperclip.stock-harness-preflight.v3", sourceSha: git.stdout?.trim() || null,
+      schema: "paperclip.stock-harness-preflight.v3", instructionVariant, sourceSha: git.stdout?.trim() || null,
      sourceFingerprint: source.fingerprint, measuredAt: new Date().toISOString(), providerCalls: 0,
      live: "not_run", passed: false, sourceErrors: source.sourceErrors, setup, gates: [],
    }, null, 2) + "\n");
@@ -189,7 +214,7 @@ export function main(args = process.argv.slice(2)) {
  setup.fakeCodexSha256 = createHash("sha256").update(readFileSync(join(root,
    "packages/paperclip-runner/runner/target/debug/fake-codex-app-server"))).digest("hex");
  const results = [];
-  for (const gate of stockHarnessGates) {
+  for (const gate of gates) {
    console.log(`Checking ${gate.id}: ${gate.name}`);
    const file = join(output, `${gate.id}.json`);
    const run = spawnSync(process.execPath, [join(root, "node_modules/vitest/vitest.mjs"), "run", ...gate.files,
@@ -210,7 +235,7 @@ export function main(args = process.argv.slice(2)) {
  writeFileSync(join(output, "rust.txt"), rustOutput);
  results.push({ id: "SH-1-rust", passed: rust.status === 0 && /test runtime_instructions_are_additive_for_codex_on_start_and_resume \.\.\. ok/.test(rustOutput), exitCode: rust.status });
  const { fingerprint, sourceErrors } = sourceFingerprint();
-  const report = { schema: "paperclip.stock-harness-preflight.v3", sourceSha: git.stdout?.trim() || null,
+  const report = { schema: "paperclip.stock-harness-preflight.v3", instructionVariant, sourceSha: git.stdout?.trim() || null,
    sourceFingerprint: fingerprint, measuredAt: new Date().toISOString(), providerCalls: 0,
    live: "not_run", passed: git.status === 0 && sourceErrors.length === 0 && results.every(row => row.passed), sourceErrors, setup, gates: results };
  writeFileSync(join(output, "preflight.json"), JSON.stringify(report, null, 2) + "\n");
@@ -1,5 +1,6 @@
 import { describe, expect, it } from "vitest";
-import { assertPreflightReceipt, gradeGate, stockHarnessGates } from "./stock-harness-checks.mjs";
+import { assertPreflightReceipt, gradeGate, stockHarnessGates, stockHarnessGatesForVariant } from "./stock-harness-checks.mjs";
+import { readStockInstructionVariant } from "./stock-harness-instruction-variant.mjs";

 const gate = { id: "SH-test", name: "Boundary", files: ["boundary.test.ts"], required: ["must preserve instructions"] };
 const report = () => ({ testResults: [{ name: "/repo/boundary.test.ts", status: "passed",
@@ -11,6 +12,15 @@ describe("stock harness prerequisite coverage", () => {
    expect(stockHarnessGates.every(gate => gate.files.length > 0 && gate.required.length > 0)).toBe(true);
  });
  it("accepts an executed passing boundary", () => expect(gradeGate(gate, report(), 0).passed).toBe(true));
+  it("preserves candidate assertions and admits only declared historical structural alternatives", () => {
+    expect(stockHarnessGatesForVariant("reduced")).toEqual(stockHarnessGates);
+    const historical = stockHarnessGatesForVariant("historical");
+    expect(historical.map(value => value.files)).toEqual(stockHarnessGates.map(value => value.files));
+    expect(historical.find(value => value.id === "SH-2").required).toContain("materializes the bundled default instruction set for non-CEO agents with no prompt template");
+    expect(historical.find(value => value.id === "SH-3").required).toContain("adds the execution contract to resume delta prompts and opted-in fresh prompts");
+    expect(historical.find(value => value.id === "SH-3").required).toContain("integrates the env-free store");
+    expect(() => stockHarnessGatesForVariant("other")).toThrow("Unknown");
+  });
  it.each([null, undefined, { testResults: [] }])("rejects unavailable evidence %s", (report) => {
    expect(gradeGate(gate, report, 0).passed).toBe(false);
  });
@@ -42,6 +52,7 @@ describe("stock harness prerequisite coverage", () => {
 describe("stock harness prerequisite admission", () => {
  const current = { sha: "a".repeat(40), fingerprint: "b".repeat(64), runnerdSha256: "c".repeat(64), fakeCodexSha256: "e".repeat(64) };
  const receipt = () => ({ schema: "paperclip.stock-harness-preflight.v3", passed: true,
+    instructionVariant: (({ variant, sha256 }) => ({ variant, sha256 }))(readStockInstructionVariant()),
    setup: { passed: true, exitCode: 0, sdkExitCode: 0, runnerdExitCode: 0, runnerdSha256: current.runnerdSha256,
      fakeCodexSha256: current.fakeCodexSha256 },
    providerCalls: 0, sourceSha: current.sha, sourceFingerprint: current.fingerprint,
@@ -50,6 +61,8 @@ describe("stock harness prerequisite admission", () => {
  it.each([
    ["old SHA", r => { r.sourceSha = "c".repeat(40); }],
    ["changed source", r => { r.sourceFingerprint = "d".repeat(64); }],
+    ["wrong instruction variant", r => { r.instructionVariant.variant = "other"; }],
+    ["wrong manual hash", r => { r.instructionVariant.sha256 = "f".repeat(64); }],
    ["failed boundary", r => { r.gates[0].passed = false; }],
    ["nonzero exit", r => { r.gates[0].exitCode = 1; }],
    ["missing Rust", r => { r.gates.pop(); }],
@@ -5,7 +5,7 @@ import { stockHarnessSourceDigest, stockHarnessSkillSources } from "./stock-harn
 vi.mock("node:fs", async importOriginal => ({ ...await importOriginal<typeof import("node:fs")>(), readFileSync: vi.fn() }));

 describe("stock harness instruction revision", () => {
-  it.each(["server/src/onboarding-assets/default/AGENTS.md", "packages/adapter-utils/src/server-utils.ts", "packages/shared/src/connection-intent-guidance.ts", "skills/paperclip/SKILL.md", "skills/paperclip/references/issue-documents.md", "packages/paperclip-runner/generated/capability/capabilities.yaml", "packages/paperclip-runner/spec/capability/capabilities.yaml", "tests/runner-e2e/stock-harness-manifest.ts"])(
+  it.each(["server/src/onboarding-assets/default/AGENTS.md", "packages/adapter-utils/src/server-utils.ts", "packages/shared/src/connection-intent-guidance.ts", "skills/paperclip/SKILL.md", "skills/paperclip/references/issue-documents.md", "packages/paperclip-runner/generated/capability/capabilities.yaml", "packages/paperclip-runner/spec/capability/capabilities.yaml", "tests/runner-e2e/stock-harness-manifest.ts", "packages/adapter-utils/src/acpx-engine/execute.ts", "packages/adapter-utils/src/acpx-engine/ephemeral-session-environment.ts", "tests/runner-e2e/stock-harness-instruction-variant.mjs", "tests/runner-e2e/automatic-retry.ts", "tests/runner-e2e/catalog.ts"])(
    "changes when the evaluated %s changes", source => {
      vi.mocked(readFileSync).mockImplementation(() => Buffer.from("unchanged"));
      const original = stockHarnessSourceDigest();
@@ -0,0 +1,7 @@
+export interface StockInstructionVariant {
+  variant: "reduced" | "historical";
+  sha256: string;
+  content: string;
+}
+export function classifyStockInstructionManual(content: unknown): StockInstructionVariant;
+export function readStockInstructionVariant(): StockInstructionVariant;
@@ -0,0 +1,24 @@
+import { createHash } from "node:crypto";
+import { readFileSync } from "node:fs";
+
+const reduced = "You are an agent in a Paperclip company.\n";
+const historicalSha256 = "e4d2375d722602cd744403292d99f6e6c9b7e8014d9cdfa6f9811ace03428e5f";
+
+/**
+ * @param {unknown} content
+ * @returns {{ variant: 'reduced' | 'historical', sha256: string, content: string }}
+ */
+export function classifyStockInstructionManual(content) {
+  if (typeof content !== "string") throw new Error("Stock harness instruction manual must be exact UTF-8 text.");
+  const sha256 = createHash("sha256").update(content, "utf8").digest("hex");
+  if (content === reduced) return { variant: "reduced", sha256, content };
+  if (Buffer.byteLength(content, "utf8") === 4249 && sha256 === historicalSha256)
+    return { variant: "historical", sha256, content };
+  throw new Error("Stock harness instruction manual is not an admitted reduced or historical source.");
+}
+
+export function readStockInstructionVariant() {
+  return classifyStockInstructionManual(readFileSync(new URL(
+    "../../server/src/onboarding-assets/default/AGENTS.md", import.meta.url,
+  ), "utf8"));
+}
@@ -0,0 +1,48 @@
+import { createHash } from "node:crypto";
+import { readFileSync } from "node:fs";
+import { afterEach, describe, expect, it, vi } from "vitest";
+import { classifyStockInstructionManual, readStockInstructionVariant } from "./stock-harness-instruction-variant.mjs";
+
+const reduced = "You are an agent in a Paperclip company.\n";
+const historicalBytes = readFileSync(new URL("./fixtures/stock-harness/historical-default-agents.md", import.meta.url));
+const historical = historicalBytes.toString("utf8");
+const historicalSha256 = "e4d2375d722602cd744403292d99f6e6c9b7e8014d9cdfa6f9811ace03428e5f";
+
+afterEach(() => vi.unstubAllEnvs());
+describe("closed stock harness instruction variants", () => {
+  it("admits only the exact reduced manual text", () => {
+    expect(classifyStockInstructionManual(reduced)).toEqual({ variant: "reduced", content: reduced,
+      sha256: createHash("sha256").update(reduced).digest("hex") });
+  });
+  it("admits the independently pinned 4249-byte historical manual", () => {
+    expect(historicalBytes.length).toBe(4249);
+    expect(createHash("sha256").update(historicalBytes).digest("hex")).toBe(historicalSha256);
+    expect(classifyStockInstructionManual(historical)).toEqual({ variant: "historical", sha256: historicalSha256, content: historical });
+  });
+  it.each([
+    ["missing newline", reduced.trimEnd()],
+    ["appended reduced instructions", `${reduced}Follow additional procedures.\n`],
+    ["altered reduced wording", reduced.replace("agent", "coder")],
+    ["CRLF reduced manual", reduced.replace("\n", "\r\n")],
+    ["leading byte order mark", `\uFEFF${reduced}`],
+    ["appended historical instructions", `${historical}Extra instructions.\n`],
+    ["same-size altered historical content", `X${historical.slice(1)}`],
+    ["unknown manual", "Historical instructions"],
+    ["empty text", ""],
+  ])("rejects %s", (_name, content) => {
+    expect(() => classifyStockInstructionManual(content)).toThrow("not an admitted");
+  });
+  it.each([undefined, null, 42, {}, [], Buffer.from(reduced)])("rejects malformed non-string content %#", content => {
+    expect(() => classifyStockInstructionManual(content)).toThrow("must be exact UTF-8 text");
+  });
+  it("reads and classifies the exact current repository source without selecting by environment or branch labels", () => {
+    const source = readFileSync(new URL("../../server/src/onboarding-assets/default/AGENTS.md", import.meta.url), "utf8");
+    const expected = classifyStockInstructionManual(source);
+    for (const label of ["historical", "reduced", "unknown"]) {
+      vi.stubEnv("PAPERCLIP_STOCK_HARNESS_VARIANT", label);
+      vi.stubEnv("PAPERCLIP_RUNNER_E2E_SOURCE_REF", `refs/heads/${label}`);
+      vi.stubEnv("GITHUB_HEAD_REF", label);
+      expect(readStockInstructionVariant()).toEqual(expected);
+    }
+  });
+});
@@ -1,6 +1,12 @@
 import { describe, expect, it } from "vitest";
 import { runnerMatrix, runnerProfiles, suiteDefinitionHash } from "./catalog.js";
-import { captureStockHarness, gradeStockHarness, gradeStockHire, STOCK_HIRE_IDENTITY, type StockHarnessEvidence } from "./stock-harness.js";
+import { captureStockHarness, gradeStockHarness as gradeSourceHarness, gradeStockHire as gradeSourceHire, STOCK_HIRE_IDENTITY, type StockHarnessEvidence } from "./stock-harness.js";
+import { readFileSync } from "node:fs";
+import { classifyStockInstructionManual, readStockInstructionVariant } from "./stock-harness-instruction-variant.mjs";
+
+const reduced = classifyStockInstructionManual(STOCK_HIRE_IDENTITY);
+const gradeStockHire = (evidence: Parameters<typeof gradeSourceHire>[0]) => gradeSourceHire(evidence, reduced);
+const gradeStockHarness = (evidence: StockHarnessEvidence) => gradeSourceHarness(evidence, reduced);

 function recording(generation: "legacy" | "native" = "legacy"): StockHarnessEvidence {
  return {
@@ -28,6 +34,8 @@ describe("stock harness Product E2E", () => {
      "assigned-skill-paperclip-document",
    ]));
    expect(cells.every(row => row.suite.manualOnly && row.environment.id === "local")).toBe(true);
+    expect(cells.every(row => row.task.automaticRetryPolicy === "single_attempt")).toBe(true);
+    expect(cells[0]!.suite.definitionMetadata).toMatchObject({ automaticRetryPolicy: "single_attempt", maximumAttemptsPerCell: 1 });
    expect(cells.reduce((turns, row) => turns + row.task.expectedRunCount, 0)).toBe(50);
    expect(cells.filter(row => row.task.id === "assigned-skill-paperclip-document").map(row => row.profile.id).sort()).toEqual(["legacy-claude", "legacy-opencode"]);
    expect(cells[0]!.suite.definitionMetadata?.sourceDigest).toMatch(/^[a-f0-9]{64}$/);
@@ -61,6 +69,21 @@ describe("stock harness Product E2E", () => {
    expect(gradeStockHire(hire)).toContainEqual(expect.objectContaining({ id: "default-hire-bundle", passed: false }));
  });

+  it("keeps historical instruction observations separate from the unchanged behavioral oracle", () => {
+    const historical = classifyStockInstructionManual(readFileSync(new URL("fixtures/stock-harness/historical-default-agents.md", import.meta.url), "utf8"));
+    const evidence = recording();
+    evidence.bundle.files[0]!.content = historical.content;
+    evidence.invocations[0]!.prompt += "\nExecution contract:";
+    expect(gradeSourceHarness(evidence, historical).every(check => check.passed)).toBe(true);
+    expect(gradeSourceHarness(evidence, historical)).toContainEqual(expect.objectContaining({ id: "historical-generic-procedures-observed", passed: true }));
+    expect(gradeSourceHarness(evidence, reduced)).toContainEqual(expect.objectContaining({ id: "default-hire-bundle", passed: false }));
+    evidence.invocations[0]!.prompt = "You are agent agent. Connection tools: connections_search";
+    expect(gradeSourceHarness(evidence, historical)).toContainEqual(expect.objectContaining({ id: "historical-generic-procedures-observed", passed: false }));
+    const current = readStockInstructionVariant();
+    evidence.bundle.files[0]!.content = current.content;
+    expect(gradeSourceHire(evidence).every(check => check.passed)).toBe(true);
+  });
+
  it.each([
    ["manual-regrew", (e: StockHarnessEvidence) => { e.bundle.files[0]!.content += "Always comment and delegate."; }, "default-hire-bundle"],
    ["extra-file", (e: StockHarnessEvidence) => { e.bundle.files.push({ path: "HEARTBEAT.md", content: "Do everything again." }); }, "default-hire-bundle"],
@@ -1,6 +1,7 @@
 import { createHash } from "node:crypto";
 import { readFileSync } from "node:fs";
 import type { RunnerProfileFixture } from "./types.js";
+import { readStockInstructionVariant } from "./stock-harness-instruction-variant.mjs";

 // These are an independent product contract, not imports of the implementation
 // constants. Otherwise a larger shipped manual could silently update the oracle.
@@ -48,20 +49,23 @@ export interface StockHarnessEvidence {
  runIds: string[];
 }

-export function gradeStockHire(evidence: Pick<StockHarnessEvidence, "bundle" | "budgets">) {
+export function gradeStockHire(evidence: Pick<StockHarnessEvidence, "bundle" | "budgets">,
+  instructionVariant = readStockInstructionVariant()) {
  const checks: Array<{ id: string; passed: boolean; detail: string }> = [];
  const check = (id: string, passed: boolean, detail: string) => checks.push({ id, passed, detail });
  check("budget-hard-stops", evidence.budgets.companyMonthlyCents === 1_000 && evidence.budgets.agentMonthlyCents === 1_000,
    "Public company and agent records must both retain the 1,000-cent monthly hard stops.");
  check("default-hire-bundle", evidence.bundle.entryFile === "AGENTS.md" &&
    evidence.bundle.files.length === 1 && evidence.bundle.files[0]?.path === "AGENTS.md" &&
-    evidence.bundle.files[0]?.content === STOCK_HIRE_IDENTITY,
-  "The public managed bundle must contain only the eight-word shipped identity, without an injected QA manual.");
+    evidence.bundle.files[0]?.content === instructionVariant.content,
+  instructionVariant.variant === "reduced"
+    ? "The public managed bundle must contain only the eight-word shipped identity, without an injected QA manual."
+    : "The comparison bundle must exactly match the independently pinned historical manual; this is structural evidence, not a task outcome.");
  return checks;
 }

-export function gradeStockHarness(evidence: StockHarnessEvidence) {
-  const checks = gradeStockHire(evidence);
+export function gradeStockHarness(evidence: StockHarnessEvidence, instructionVariant = readStockInstructionVariant()) {
+  const checks = gradeStockHire(evidence, instructionVariant);
  const check = (id: string, passed: boolean, detail: string) => checks.push({ id, passed, detail });
  check("provider-runs-present", evidence.runIds.length > 0 && evidence.runIds.every(Boolean),
    "The lifecycle oracle must reach actual provider runs; missing runs cannot pass instruction delivery.");
@@ -70,9 +74,15 @@ export function gradeStockHarness(evidence: StockHarnessEvidence) {
    check("invocation-evidence-complete", evidence.runIds.length > 0 && evidence.runIds.every(runId =>
      prompts.some(row => row.runId === runId)),
    "Every legacy run must have a public adapter.invoke event with its actual nonempty prompt.");
-    check("generic-procedures-absent", prompts.length > 0 && prompts.every(row =>
-      REMOVED_PROCEDURES.every(procedure => !String(row.prompt).includes(procedure))),
-    "Neither startup nor continuation may reintroduce the removed generic manual.");
+    if (instructionVariant.variant === "reduced") {
+      check("generic-procedures-absent", prompts.length > 0 && prompts.every(row =>
+        REMOVED_PROCEDURES.every(procedure => !String(row.prompt).includes(procedure))),
+      "Neither startup nor continuation may reintroduce the removed generic manual.");
+    } else {
+      check("historical-generic-procedures-observed", prompts.length > 0 && prompts.some(row =>
+        REMOVED_PROCEDURES.some(procedure => String(row.prompt).includes(procedure))),
+      "Observe the historical instruction carrier separately from the identical task outcome oracle.");
+    }
    const fresh = prompts.filter(row => Number(row.promptMetrics?.heartbeatPromptChars) > 0);
    check("fresh-default-delivered", fresh.length > 0 && fresh.every(row =>
      String(row.prompt).includes(STOCK_TEMPLATE_IDENTITY) && String(row.prompt).includes("Connection tools:") &&
@@ -124,6 +134,11 @@ export function stockHarnessSourceDigest() {
    "stock-harness.ts", "context-integrity-cases.ts", "context-integrity-scoring.ts",
    "context-integrity-flow.ts", "chat-cases.ts", "chat-flow.ts", "live-fixtures.ts", "runner.spec.ts",
    "stock-harness-checks.mjs", "stock-harness-admission.ts", "stock-harness-manifest.ts",
+    "stock-harness-instruction-variant.mjs", "stock-harness-instruction-variant.d.mts", "stock-harness-instruction-variant.test.mjs",
+    "fixtures/stock-harness/historical-default-agents.md", "automatic-retry.ts", "automatic-retry.test.ts", "types.ts", "catalog.ts", "launch.ts",
+    "../../packages/adapter-utils/src/acpx-engine/execute.ts",
+    "../../packages/adapter-utils/src/acpx-engine/ephemeral-session-environment.ts",
+    "../../packages/adapter-utils/src/acpx-engine/ephemeral-session-environment.test.ts",
    "../../packages/paperclip-runner/scripts/generate-capability-contract.mjs",
    "../../packages/paperclip-runner/scripts/check-capability-inventory.mjs",
    "../../packages/paperclip-runner/scripts/lib/capability-inventory.mjs",