fix(evals): name why a private evidence transfer refused, without its text (#19908)

## Thinking Path

- Paid chat evals retain their evidence through a trusted transfer
(`tests/runner-e2e/manual-private-transfer.ts`) that runs from master,
separate from the target checkout.
- When the transfer refuses an attempt, it prints one fixed line,
"Restricted manual evidence operation failed; no public fallback", and
keeps nothing. The closed diagnostic on the target side cannot see this
step either.
- So a refused attempt leaves no evidence and no reason.
`runs-adapters.inspect-failed-run` lost three attempts in a row this
way, and `goals.create-goal-tree` lost one; nobody could tell which
check refused.
- This change keeps the privacy rule (no message text in public logs)
but names the refusal with a closed code.

## Linked Issues or Issue Description

**What happened**

A paid chat-eval attempt whose private evidence the trusted transfer
refused printed only "Restricted manual evidence operation failed; no
public fallback". The receipt showed `attemptedSourceBytes: 0` and
`transfer.status: failed_or_unknown`, and no evidence was kept, so the
reason was unknowable. `runs-adapters.inspect-failed-run` lost three
consecutive attempts this way.

**Expected behavior**

A refused transfer still keeps message text private, but names which
check refused it with a closed code, so the attempt can be diagnosed.

**Steps to reproduce**

1. Dispatch `runner-full-stack-e2e.yml` manually for a chat cell whose
private evidence fails transfer validation.
2. Open the "Retain complete approved manual originals" step log: it
shows the single fixed failure line with no reason.

## What Changed

- New `tests/runner-e2e/private-refusal-code.ts`:
`privateRefusalCode(error)` returns a slug of the message only when it
is a fixed literal `new Error("...")` in the trusted evidence modules
(transfer, handoff, retention, report assets, history, framework report,
history budget, evidence, redaction). Otherwise it returns `fs-<code>`
for Node error codes, `error-type-<Name>` for typed errors,
`non-error-thrown`, or `dynamic`. Message text is never returned.
- `manual-private-transfer.ts`: the failure line now includes `(refusal:
<code>)`.
- Tests: fixed literals are named; dynamic, filesystem, typed and
non-error throws never leak their text.

## Verification

- `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts
tests/runner-e2e/private-refusal-code.test.ts
tests/runner-e2e/manual-private-transfer.test.ts`: 15 passed.
- `npx tsc --noEmit -p tests/runner-e2e`: clean.
- The three files the transfer requires to match between trusted and
target checkouts (`manual-private-handoff.ts`,
`manual-private-retention.ts`, `manual-private-report-assets.ts`) are
unchanged.

## Risks

- The public log line gains a code. Codes are either fixed literal
messages (already public in this repository's source), Node error codes,
or error type names. Dynamic messages, which can contain paths, are
reported only as `dynamic`.
- Reading the module sources once at failure time adds a small file read
on the error path only.

## Model Used

Anthropic Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code, with
tool use (shell, file editing).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

tests/runner-e2e/manual-private-transfer.ts

@@ -13,6 +13,7 @@ import { privateArtifactKey, retainPrivateTransfer, transferPrivateArtifact, ver
 import { authorizeManualPrivateEvidence, MANUAL_PRIVATE_PROTOCOL, manualCellKind, manualPrivateSessionPolicy } from "./manual-private-retention.js";
 import { shippedChatEvidenceAuthority } from "./testops-chat-admission.js";
 import { manualDispatchEvidenceAuthority } from "./manual-dispatch-authority.js";
+import { privateRefusalCode } from "./private-refusal-code.js";
 
 const hash = (bytes: string | Buffer) => createHash("sha256").update(bytes).digest("hex");
 function request(env: NodeJS.ProcessEnv) {
@@ -233,5 +234,6 @@ async function main() {
   else throw new Error("Invalid private evidence command");
 }
 if (process.argv[1] && import.meta.url === pathToFileURL(path.resolve(process.argv[1])).href) {
-  main().catch(() => { process.stderr.write("Restricted manual evidence operation failed; no public fallback\n"); process.exitCode = 1; });
+  // Name only a closed refusal code (never message text) so a refused attempt is diagnosable.
+  main().catch(error => { process.stderr.write(`Restricted manual evidence operation failed (refusal: ${privateRefusalCode(error)}); no public fallback\n`); process.exitCode = 1; });
 }

tests/runner-e2e/private-refusal-code.test.ts

@@ -0,0 +1,41 @@
+import { readFileSync } from "node:fs";
+import { expect, it } from "vitest";
+import { fixedRefusalMessages, privateRefusalCode, PRIVATE_REFUSAL_MODULES } from "./private-refusal-code.js";
+
+it("names only fixed literal refusals from the trusted evidence modules", () => {
+  const fixed = fixedRefusalMessages();
+  expect(fixed.has("Contaminated private evidence cannot be uploaded")).toBe(true);
+  expect(fixed.has("Original evidence byte accounting changed")).toBe(true);
+  expect(privateRefusalCode(new Error("Contaminated private evidence cannot be uploaded"))).toBe("contaminated-private-evidence-cannot-be-uploaded");
+});
+
+it("never returns the text of a dynamic or system message", () => {
+  const leaked = "/home/runner/_work/private/secret-token-abc";
+  expect(privateRefusalCode(new Error(`Secret leak in ${leaked}`))).toBe("dynamic");
+  expect(privateRefusalCode(Object.assign(new Error(`ENOENT: no such file, open '${leaked}'`), { code: "ENOENT" }))).toBe("fs-ENOENT");
+  expect(privateRefusalCode(new TypeError(`bad ${leaked}`))).toBe("error-type-TypeError");
+  expect(privateRefusalCode(leaked)).toBe("non-error-thrown");
+  for (const value of [new Error(`x ${leaked}`), new TypeError(leaked), leaked]) expect(privateRefusalCode(value)).not.toContain("secret");
+});
+
+it("names refusals from the dispatch-authority and admission modules the transfer calls", () => {
+  const fixed = fixedRefusalMessages();
+  for (const file of ["manual-dispatch-authority.ts", "testops-chat-admission.ts"]) {
+    const literals = [...readFileSync(new URL(`./${file}`, import.meta.url), "utf8").matchAll(/new Error\("([^"$`\\]+)"\)/g)].map(match => match[1]!);
+    expect(literals.length, file).toBeGreaterThan(0);
+    for (const message of literals) expect(privateRefusalCode(new Error(message), fixed), message).not.toBe("dynamic");
+  }
+});
+
+it("covers every local module the trusted transfer imports that has fixed refusals", () => {
+  const transfer = readFileSync(new URL("./manual-private-transfer.ts", import.meta.url), "utf8");
+  const imported = [...transfer.matchAll(/from "\.\/([a-z0-9-]+)\.js"/g)].map(match => `${match[1]}.ts`);
+  for (const file of imported.filter(name => name !== "private-refusal-code.ts")) {
+    const source = readFileSync(new URL(`./${file}`, import.meta.url), "utf8");
+    if (/new Error\("[^"$`\\]+"\)/.test(source)) expect(PRIVATE_REFUSAL_MODULES as readonly string[], file).toContain(file);
+  }
+});
+it("never throws while naming a refusal", () => {
+  const broken = new Proxy(new Set<string>(), { get() { throw new Error("read failed /private/path"); } });
+  expect(privateRefusalCode(new Error("x"), broken)).toBe("unnamed");
+});

tests/runner-e2e/private-refusal-code.ts

@@ -0,0 +1,36 @@
+import { readFileSync } from "node:fs";
+
+/** Trusted evidence modules whose fixed refusal messages may be named publicly. */
+export const PRIVATE_REFUSAL_MODULES = [
+  "manual-private-transfer.ts", "manual-private-handoff.ts", "manual-private-retention.ts", "manual-private-report-assets.ts",
+  "private-history.ts", "private-framework-report.ts", "history-budget.ts", "evidence.ts", "redaction.ts",
+  "manual-dispatch-authority.ts", "testops-chat-admission.ts",
+] as const;
+
+let known: ReadonlySet<string> | undefined;
+/** Literal `new Error("...")` messages in the trusted modules; dynamic messages are never included. */
+export function fixedRefusalMessages(): ReadonlySet<string> {
+  known ??= new Set(PRIVATE_REFUSAL_MODULES.flatMap(file => {
+    const source = readFileSync(new URL(`./${file}`, import.meta.url), "utf8");
+    return [...source.matchAll(/new Error\("([^"$`\\]+)"\)/g)].map(match => match[1]!);
+  }));
+  return known;
+}
+
+/**
+ * A closed, public-safe name for why a private evidence operation refused. Only a fixed literal
+ * message from the trusted modules is named (as a slug); otherwise a Node error code, an error type,
+ * or "dynamic". The message text of anything else is never returned.
+ */
+export function privateRefusalCode(error: unknown, fixed?: ReadonlySet<string>): string {
+  // Naming must never prevent the fixed failure line: a module read failure degrades to "unnamed".
+  try { return refusalCode(error, fixed ?? fixedRefusalMessages()); } catch { return "unnamed"; }
+}
+function refusalCode(error: unknown, fixed: ReadonlySet<string>): string {
+  if (!(error instanceof Error)) return "non-error-thrown";
+  if (fixed.has(error.message)) return error.message.toLowerCase().replace(/[^a-z0-9]+/g, "-").replace(/^-|-$/g, "").slice(0, 96);
+  const code = (error as NodeJS.ErrnoException).code;
+  if (typeof code === "string" && /^(?:E[A-Z]{2,15}|ERR_[A-Z0-9_]{2,48})$/.test(code)) return `fs-${code}`;
+  if (error.name !== "Error" && /^[A-Z][A-Za-z]{2,40}$/.test(error.name)) return `error-type-${error.name}`;
+  return "dynamic";
+}