feat(evals): allow reviewed bounded provisional accounting (#17365)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Product E2E campaigns include model, setup, storage and
observability costs.
> - Some billing windows close after the tests and reports finish.
> - Requiring settled receipts for retained storage can block a good
campaign for weeks.
> - A reservation alone does not prove that a pending charge has an
upper bound.
> - This change permits provisional completion only with reviewed
evidence for the exact cost configuration.
> - Reports preserve unknown amounts and holds until later
reconciliation.

## Linked Issues or Issue Description

Refs: #17360, #17354

**What existing behavior does this improve?**

The Product E2E scheduled accounting gate and its retained cost reports.

**Current behavior**

Every pending receipt fails finalization. A completed test can wait
through the retained-storage billing window. The journal has dollar
reservations but no explicit qualification of their bounds.

**Proposed behavior**

Permit provisional completion when all pending charges have reviewed
full-lifecycle bounds, no requests remain in flight, no receipts
conflict, and known costs plus held amounts fit the campaign limit. Keep
all existing holds and receipt-finality rules. Missing evidence still
fails.

**Reason and benefit**

An operator can distinguish delayed but bounded billing from unbounded
or missing costs. The campaign can finish without claiming that its
invoice is settled.

**Breaking changes**

The journal accepts optional bound metadata and an invalidation record.
New reports add an optional bounded-pending count. Current readers
retain old journals and reports without granting them bounds. Older code
must not read new bound-bearing journals. The existing daily/weekly
enable gates and budget amounts stay unchanged.

**Additional context**

This implements the operator-approved provisional-cost policy for the
existing eval harness. It does not change product budgets or the
agent-eval roadmap. A targeted GitHub search found no duplicate open PR.

## What Changed

- Accounting owns a narrow evidence contract: a reviewed audit hash,
exact configuration hash and full-lifecycle scope.
- Gateway reservations retain that evidence before dispatch. Changed
limits or rates fail before dispatch. Unexpected routing, model identity
or excessive usage invalidates a bound durably.
- Operating allowances can carry separately reviewed bounds for each
cost component, including its full storage lifetime. No existing
allowance becomes qualified by default.
- Reports and Braintrust experiment metadata distinguish provisional
accounting, complete receipts, blocked accounting and unavailable
evidence. Public output omits private audit hashes.
- Scheduled finalization allows qualified provisional costs and
explicitly rejects spending overruns, including overruns with complete
receipts.
- Add regressions for legacy data, restarts, late reconciliation,
conflicts, malformed evidence, routing uncertainty and exceeded limits.
Document the evidence and migration boundaries.

## Verification

- Full offline eval-support suite: 4,062 Vitest tests passed, two
existing tests skipped; 137 Node tests passed.
- `pnpm test:e2e:runner:typecheck`: passed.
- `git diff --check`: passed.
- Read the retained hosted qualification journal with the new policy.
All six legacy pending charges remain unqualified. Known cost
`$0.421245200`, held `$19.327680000`, and committed `$19.748925200`
remain unchanged. The journal's SHA-256 is unchanged. The original
failed workflow remains failed.
- No paid models, network provider calls, receipt imports or reservation
releases were used for this verification.
- Rebased onto master `4d90634f74`; 142 targeted tests and typecheck
passed.
- Review reproduced a lost OpenAI input-total limit check when cache
details were missing (1 failed, 47 passed). Fix
`5fef9ba7c149951a51eae468614a4b89f0b3d6db` preserves unpriced provider
usage bounds. Then 178 focused tests, full offline support and typecheck
passed.
- Current-head hosted CI passed:
https://github.com/paperclipai/paperclip/actions/runs/40020746585
(attempt 2). Typecheck, build, all general/Runner/serialized tests and
browser gates are green.
- CI attempt 1 hit a PostgreSQL deadlock in unchanged chat-test cleanup;
its skipped flag restoration caused a second assertion failure. One
failed-jobs-only retry passed. Original failure evidence is preserved in
attempt 1. No paid eval campaign was rerun.
- Greptile reviewed exact head
`5fef9ba7c149951a51eae468614a4b89f0b3d6db` at 5/5; its one finding is
fixed and resolved. The optional Paperclip review is paused and supplied
no assessment.

## Risks

- Evidence hashes bind an audit to a configuration; they do not prove
the audit is true. Operators must review real provider limits, all price
tiers, full resource lifetimes and enforceable infrastructure limits
before configuring bounds.
- Provisional completion is not invoice settlement. Later billing can
reveal overruns, which must be reconciled and inspected. The ledger
cannot stop external billing.
- Existing pilot journals, configured allowances and price cards have no
new bound approval. This PR alone does not qualify or enable recurring
paid work.
- The gateway cannot qualify a canceled Braintrust request without route
confirmation. It preserves that unknown hold.
- No new dependency, secret, repository variable, workflow, schedule or
cloud resource changes.

## Model Used

OpenAI Codex, GPT-6 family. The exact deployment ID and context-window
size are not exposed in this session. Used reasoning, repository tools
and code execution. No paid evaluation model calls were made for this
change.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

tests/runner-e2e/OPERATIONS-COVERAGE.md

@@ -133,7 +133,10 @@ the existing $249/month fee while including new usage and capacity charges.
 Scheduled admission now reserves the declared CI, storage, observability and
 subscription allowances inside the same $20/$100 journal limit. Missing or stale
 allowances stop scheduling before the AWS test runner starts. Missing receipts
-keep their reservations held and accounting incomplete. This is a local admission
+keep their reservations held. Explicit reviewed full-lifecycle bounds may allow
+provisional completion while billing remains pending; missing bounds, conflicts
+and overspending still fail. Legacy reservations gain no qualification from the
+policy change. This is a local admission
 control, not a provider billing cap. Actual resource bounds, cost collection and
 reviewed allowances still need qualification before recurrence is enabled; see
 [the allowance contract](OPERATIONS.md#recurring-campaigns).

tests/runner-e2e/OPERATIONS.md

@@ -173,6 +173,78 @@ Private request receipts retain transport IDs, response IDs, usage and cache TTL
 route/cache/failover evidence, and final accounting status without request bodies
 or credentials. Raw provider responses remain only in the normal private evidence.
 
+## Provisional completion and later reconciliation
+
+Recurring campaigns may finish with provisional accounting when every pending
+charge has an explicit reviewed bound, all requests have stopped, no receipts
+conflict, and known costs plus retained holds fit the campaign cap. Product,
+cleanup, feedback, trace and ingestion requirements still apply. Reports and
+Braintrust experiment metadata label accounting as `provisional`,
+`receipts_complete`, `blocked`, or `unavailable`. Receipt completeness is separate
+from invoice reconciliation. The original failed campaign remains failed; a
+later report or policy does not change its original result.
+
+The optional `provisionalBound` on a manifest model or operating component is:
+
+```json
+{
+  "version": 1,
+  "scope": "full_lifecycle",
+  "evidenceSha256": "SHA256_OF_RETAINED_REVIEWED_AUDIT",
+  "configurationSha256": "SHA256_OF_EXACT_REVIEWED_CONFIGURATION"
+}
+```
+
+These placeholders are invalid until replaced with actual lowercase SHA-256
+values. Use `costBoundConfigurationSha256` from `cost-bound-evidence.ts` for
+stable key-sorted JSON hashing. The model configuration contains `provider`,
+`mode` (the manifest route), `price`, `maximumInputTokens`,
+`maximumOutputTokens`, and `reservation` (the nine-decimal value from
+`evalRequestReservation`). An operating configuration contains allowance
+`reference`, `runner`, `verifiedAt`, `expiresAt`, and a `component` object with
+its `component`, `maxUsd`, and `reference`. Changing any of those values
+invalidates the approval hash. Evidence metadata itself is excluded from this
+configuration hash.
+
+An operator must retain and review the audit before configuring its hash.
+Hashes bind evidence and configuration; they do not prove the audit is true.
+A model audit must establish the provider's full context ceiling, all billable
+token prices and tiers, standard inference, and the route's billing behavior.
+The gateway enforces output limits and supported request shapes. An operating
+audit must cover setup, failed attempts, build time, publication, transfer,
+observability and the full retained-storage lifecycle, including uncertainty in
+resource deletion and invoice arrival. It must name the enforceable resource
+limits and rates that keep the whole component below `maxUsd`. A current-day
+usage estimate or an arbitrary contingency amount is insufficient. A zero bound
+also needs evidence. Keep raw audits private; public reports expose only counts
+and the existing journal checkpoint.
+
+The ledger retains bound evidence before model dispatch. Unexpected or missing
+Braintrust route confirmation, failover, a returned model mismatch, or observed
+usage above a declared limit disqualifies that request's bound. Provider usage
+normalization retains a separate input/output lower bound even when missing or
+contradictory cache counters prevent pricing. Malformed reported counts also
+disqualify a bound. In-flight
+requests, conflicts, pending charges above their bound, and campaign overruns
+still block completion. Gateway cancellation before route confirmation remains
+unqualified even if it retained an admission reservation. Later receipts remain
+the way to resolve it.
+
+Provisional completion never releases a reservation or manufactures a zero
+charge. A missing receipt stays unknown and holds its full bound, less any
+known partial amount, until a valid final receipt supersedes it. The existing
+receipt importer still requires complete billing/resource windows before it
+marks a receipt final. Later bills can reveal an overrun; this local journal
+cannot stop external billing. Reconcile the same retained journal after its
+billing windows close, regenerate its report, and inspect any overruns before
+admitting more work. Existing subscription costs remain separate from the
+incremental daily/weekly budgets.
+
+Old journals and reports remain readable. They gain no bound evidence by
+upgrade or by this policy; unresolved legacy charges still block. Existing
+configured allowances and price cards are not automatically qualified by the
+new optional field. Recurrence remains subject to its existing enable gates.
+
 ## Manifest
 
 A manifest contains no credentials. This structural example uses fixture prices;
@@ -420,8 +492,9 @@ lifecycles. They do not occupy model concurrency, but they count toward the
 finite charge-record limit. Resuming cannot change or drop them. Until matched
 cost observations arrive, the report shows them as unknown and keeps their
 allowance held; even a zero allowance does not claim a known zero charge.
-The existing finalizer reports incomplete accounting while these receipts are
-missing. Do not dismiss that finding to enable recurrence.
+The finalizer permits provisional completion only when each pending charge has
+explicit reviewed bound evidence as described below. Missing or unqualified
+bounds remain incomplete accounting; a reservation alone does not qualify.
 
 Reconcile a component through its existing request ID with provenance-bound
 `infrastructure` or `invoice` observations. Partial observations keep the
@@ -587,8 +660,8 @@ explicit, unbounded qualification path and does not inherit these allowances.
 
 After execution, deterministic review dismisses only exact synthetic feedback
 bodies corroborated by all feedback-persistence graders. Novel feedback remains
-`needs_investigation`; no extra model call is made. Missing accounting, unresolved
-receipts, feedback collection/submission failures, novel feedback, export failure
+`needs_investigation`; no extra model call is made. Missing accounting, unbounded
+pending charges, conflicts, spending overruns, feedback collection/submission failures, novel feedback, export failure
 and incomplete experiment import or indexing fail the job with bounded diagnostic codes.
 Successful unchanged runs produce artifacts without posting messages. GitHub's
 workflow failure notifications remain the operator's notification channel.

tests/runner-e2e/braintrust-experiment.ts

@@ -1,3 +1,4 @@
+import { evalAccountingStatus } from "./operations-report.js";
 import { createHash } from "node:crypto";
 import { buildCampaignAccounting } from "./attempt-accounting.js";
 import { exportBraintrustAttemptParent, type EvalBraintrustConfig } from "./braintrust.js";
@@ -98,6 +99,7 @@ export function buildBraintrustExperiment(projectId: string, campaign: RunnerE2E
     missing_attempts: accounting.missingAttempts,
     request_ledger: operations ? { journal_hash: operations.journalHash, known_usd: operations.knownUsd,
       additional_held_usd: operations.heldUsd, committed_usd: operations.committedUsd,
+      accounting_status: evalAccountingStatus(operations), bounded_unresolved_requests: operations.boundedUnresolvedRequests ?? 0,
       requests: operations.requests, unresolved_requests: operations.unresolvedRequests, breakdown: operations.breakdown } : null,
     feedback_review: campaign.feedbackReview ?? null,
     accounting_note: "Ledger amounts are separate observations, not additive trace metrics. Held reservations are not charges. Requests outside the gateway and infrastructure require separate accounting." };

tests/runner-e2e/cost-bound-evidence.ts

@@ -0,0 +1,45 @@
+import { createHash } from "node:crypto";
+
+export interface EvalCostBoundEvidence {
+  version: 1;
+  scope: "full_lifecycle";
+  evidenceSha256: string;
+  configurationSha256: string;
+}
+export interface EvalReservationBound {
+  basis: "model_limits" | "operating_allowance";
+  evidence: EvalCostBoundEvidence;
+  calculationSha256: string;
+}
+const hash = (value: unknown) => typeof value === "string" && /^[a-f0-9]{64}$/.test(value);
+
+export function validateCostBoundEvidence(value: EvalCostBoundEvidence): void {
+  if (!value || typeof value !== "object" || Array.isArray(value)
+      || Object.keys(value).some(key => !["version", "scope", "evidenceSha256", "configurationSha256"].includes(key))
+      || value.version !== 1 || value.scope !== "full_lifecycle" || !hash(value.evidenceSha256) || !hash(value.configurationSha256)) {
+    throw new Error("Invalid reviewed cost bound evidence");
+  }
+}
+
+export function costBoundConfigurationSha256(calculation: unknown): string {
+  return createHash("sha256").update(JSON.stringify(calculation, (_key, value) =>
+    value && typeof value === "object" && !Array.isArray(value)
+      ? Object.fromEntries(Object.keys(value).sort().map(key => [key, value[key]])) : value)).digest("hex");
+}
+
+export function reservationBound(basis: EvalReservationBound["basis"], evidence: EvalCostBoundEvidence, calculation: unknown): EvalReservationBound {
+  validateCostBoundEvidence(evidence);
+  const calculationSha256 = costBoundConfigurationSha256(calculation);
+  if (calculationSha256 !== evidence.configurationSha256) throw new Error("Reviewed cost bound configuration changed");
+  return { basis, evidence: structuredClone(evidence), calculationSha256 };
+}
+
+export function validateReservationBound(value: EvalReservationBound, infrastructure: boolean): void {
+  if (!value || typeof value !== "object" || Array.isArray(value)
+      || Object.keys(value).some(key => !["basis", "evidence", "calculationSha256"].includes(key))
+      || value.basis !== (infrastructure ? "operating_allowance" : "model_limits") || !hash(value.calculationSha256)) {
+    throw new Error("Invalid reservation bound");
+  }
+  validateCostBoundEvidence(value.evidence);
+  if (value.calculationSha256 !== value.evidence.configurationSha256) throw new Error("Reviewed cost bound configuration changed");
+}

tests/runner-e2e/cost-ledger.ts

@@ -1,9 +1,10 @@
+import { validateReservationBound, type EvalReservationBound } from "./cost-bound-evidence.js";
 import { createHash } from "node:crypto";
 import { constants } from "node:fs";
 import { mkdir, open, readFile, rm, type FileHandle } from "node:fs/promises";
 import path from "node:path";
 import { usdToUnits } from "../../packages/shared/src/money.js";
-import { validateOperatingOverhead, type EvalOperatingOverhead } from "./operating-overhead.js";
+import { operatingReservationBound, validateOperatingOverhead, type EvalOperatingOverhead } from "./operating-overhead.js";
 
 export interface EvalSpendLimits {
   campaignId: string;
@@ -46,11 +47,12 @@ type Event =
   | { type: "campaign"; limits: EvalSpendLimits }
   | ({ type: "attempt_policy" } & EvalLegacyAttemptAudit)
   | { type: "attempt"; executionId: string; attemptId: string }
-  | { type: "reserve"; requestId: string; identity: EvalChargeIdentity; maxUsd: string }
+  | { type: "reserve"; requestId: string; identity: EvalChargeIdentity; maxUsd: string; bound?: EvalReservationBound }
+  | { type: "invalidate_bound"; requestId: string }
   | { type: "observe"; observation: EvalCostObservation }
   | { type: "finish"; requestId: string; outcome: "succeeded" | "failed" | "cancelled" };
 interface Line { seq: number; at: string; previous: string; event: Event; hash: string }
-interface RequestState { identity: EvalChargeIdentity; reserved: bigint; observations: EvalCostObservation[]; finished: boolean }
+interface RequestState { identity: EvalChargeIdentity; reserved: bigint; observations: EvalCostObservation[]; finished: boolean; bound?: EvalReservationBound; boundInvalidated?: boolean }
 const usd = (units: bigint) => `${units / 1_000_000_000n}.${String(units % 1_000_000_000n).padStart(9, "0")}`;
 export const validEvalLedgerId = (value: unknown): value is string => typeof value === "string" && /^[A-Za-z0-9][A-Za-z0-9_.:/-]{0,511}$/.test(value);
 const validId = validEvalLedgerId;
@@ -89,6 +91,7 @@ export class EvalSpendState {
       for (const component of this.limits.overhead.components) this.requests.set(`overhead:${component.component}`, {
         identity: { executionId: "campaign-overhead", attemptId: component.component, phase: "infrastructure" },
         reserved: amount(component.maxUsd), observations: [], finished: true,
+        ...(component.provisionalBound ? { bound: operatingReservationBound(this.limits.overhead, component) } : {}),
       });
     }
   }
@@ -105,7 +108,7 @@ export class EvalSpendState {
       resolved: finalized.length > 0 && amounts.size === 1 };
   }
   snapshot() {
-    let known = 0n, held = 0n, committed = 0n, inflight = 0, unknown = 0, conflicts = 0;
+    let known = 0n, held = 0n, committed = 0n, inflight = 0, unknown = 0, conflicts = 0, boundedUnresolved = 0;
     const breakdown: Record<"reported" | "estimated" | "allocated" | "reconciled", bigint> = { reported: 0n, estimated: 0n, allocated: 0n, reconciled: 0n };
     const requests = [...this.requests].map(([requestId, request]) => {
       const receipt = this.authoritative(request);
@@ -113,6 +116,9 @@ export class EvalSpendState {
       if (receipt.conflict) conflicts++;
       if (!receipt.resolved) unknown++;
       const cost = receipt.cost ?? 0n;
+      const pendingBounded = !receipt.resolved && !receipt.conflict && request.finished && !!request.bound
+        && !request.boundInvalidated && cost <= request.reserved;
+      if (pendingBounded) boundedUnresolved++;
       known += cost;
       if (receipt.classification !== "unknown") breakdown[receipt.classification] += cost;
       const commitment = !request.finished || !receipt.resolved
@@ -120,10 +126,10 @@ export class EvalSpendState {
       committed += commitment;
       held += commitment - cost;
       return { requestId, ...request.identity, reservedUsd: usd(request.reserved), costUsd: receipt.cost === null ? null : usd(cost),
-        classification: receipt.classification, conflict: receipt.conflict, receiptComplete: receipt.resolved, finished: request.finished };
+        classification: receipt.classification, conflict: receipt.conflict, receiptComplete: receipt.resolved, finished: request.finished, pendingBounded };
     });
     return { campaignId: this.limits.campaignId, limitUsd: usd(amount(this.limits.maxUsd)), knownUsd: usd(known), heldUsd: usd(held), committedUsd: usd(committed),
-      overBudget: committed > amount(this.limits.maxUsd), unknownRequests: unknown, conflictingRequests: conflicts, inflight,
+      overBudget: committed > amount(this.limits.maxUsd), unknownRequests: unknown, boundedUnresolvedRequests: boundedUnresolved, conflictingRequests: conflicts, inflight,
       breakdown: Object.fromEntries(Object.entries(breakdown).map(([key, value]) => [key, usd(value)])), requests,
       admittedAttempts: [...this.attempts].map(([attemptId, executionId]) => ({ executionId, attemptId })) };
   }
@@ -156,6 +162,8 @@ export class EvalSpendState {
       if (now >= Date.parse(this.limits.deadlineAt)) throw new EvalAdmissionError("Eval campaign deadline reached");
       if ([...this.attempts.values()].filter(id => id === event.executionId).length >= this.limits.maxAttemptsPerExecution) throw new EvalAdmissionError("Eval campaign attempt cap reached");
     } else if (event.type === "reserve") {
+      fields(event, ["type", "requestId", "identity", "maxUsd", "bound"]);
+      if (event.bound !== undefined) validateReservationBound(event.bound, event.identity.phase === "infrastructure");
       fields(event.identity, ["executionId", "attemptId", "phase", "runId"]);
       if (!validId(event.requestId) || !validId(event.identity.executionId) || !validId(event.identity.attemptId)
           || event.identity.runId !== undefined && !validId(event.identity.runId)
@@ -189,6 +197,9 @@ export class EvalSpendState {
         const request = this.providerRequests.get(`${o.provider}:${o.providerRequestId}`);
         if (request && request !== o.requestId) throw new Error("Provider request belongs to another reservation; reconcile identities before import");
       }
+    } else if (event.type === "invalidate_bound") {
+      fields(event, ["type", "requestId"]);
+      if (!this.requests.get(event.requestId)?.bound) throw new Error("Cannot invalidate an absent reservation bound");
     } else if (event.type === "finish") {
       if (!this.requests.has(event.requestId) || !["succeeded", "failed", "cancelled"].includes(event.outcome)) throw new Error("Invalid request completion");
     } else throw new Error("Unknown ledger event");
@@ -198,14 +209,15 @@ export class EvalSpendState {
       this.limits.maxAttemptsPerExecution = event.maxAttemptsPerExecution;
       for (const attempt of event.priorAttempts) this.attempts.set(attempt.attemptId, attempt.executionId);
     } else if (event.type === "attempt") this.attempts.set(event.attemptId, event.executionId);
-    else if (event.type === "reserve") this.requests.set(event.requestId, { identity: event.identity, reserved: amount(event.maxUsd), observations: [], finished: false });
+    else if (event.type === "reserve") this.requests.set(event.requestId, { identity: event.identity, reserved: amount(event.maxUsd), observations: [], finished: false, ...(event.bound ? { bound: event.bound } : {}) });
     else if (event.type === "observe") {
       const o = event.observation;
       if (this.observations.has(o.observationId)) return;
       this.observations.set(o.observationId, o);
       this.requests.get(o.requestId)!.observations.push(o);
       if (o.providerRequestId) this.providerRequests.set(`${o.provider}:${o.providerRequestId}`, o.requestId);
-    } else if (event.type === "finish") this.requests.get(event.requestId)!.finished = true;
+    } else if (event.type === "invalidate_bound") this.requests.get(event.requestId)!.boundInvalidated = true;
+    else if (event.type === "finish") this.requests.get(event.requestId)!.finished = true;
   }
 }
 
@@ -273,7 +285,8 @@ export async function openEvalCostLedger(file: string, limits: EvalSpendLimits)
     return {
       adoptLegacyAttemptPolicy: (audit: EvalLegacyAttemptAudit) => append({ ...structuredClone(audit), type: "attempt_policy" }),
       admitAttempt: (executionId: string, attemptId: string) => append({ type: "attempt", executionId, attemptId }),
-      reserve: (requestId: string, identity: EvalChargeIdentity, maxUsd: string) => append({ type: "reserve", requestId, identity: structuredClone(identity), maxUsd }),
+      reserve: (requestId: string, identity: EvalChargeIdentity, maxUsd: string, bound?: EvalReservationBound) => append({ type: "reserve", requestId, identity: structuredClone(identity), maxUsd, ...(bound ? { bound: structuredClone(bound) } : {}) }),
+      invalidateBound: (requestId: string) => append({ type: "invalidate_bound", requestId }),
       observe: (observation: EvalCostObservation) => append({ type: "observe", observation: structuredClone(observation) }),
       finish: (requestId: string, outcome: "succeeded" | "failed" | "cancelled") => append({ type: "finish", requestId, outcome }),
       snapshot: () => state.snapshot(),

tests/runner-e2e/eval-gateway.test.ts

@@ -1,3 +1,4 @@
+import { costBoundConfigurationSha256 } from "./cost-bound-evidence.js";
 import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
 import { tmpdir } from "node:os";
 import path from "node:path";
@@ -355,3 +356,48 @@ it("refuses missing or unreviewed attribution checkpoints before request dispatc
   await writeFile(phasePath, JSON.stringify({ version: 1, ...identity, providerResponse: "private" }));
   await expect(readEvalAttemptPhase(phasePath, identity)).rejects.toThrow("Invalid eval phase");
 });
+
+const provisionalBound = { version: 1 as const, scope: "full_lifecycle" as const, evidenceSha256: "a".repeat(64),
+  configurationSha256: costBoundConfigurationSha256({ provider: route.provider, mode: route.mode, price: route.price,
+    maximumInputTokens: route.maximumInputTokens, maximumOutputTokens: route.maximumOutputTokens, reservation: evalRequestReservation(route) }) };
+it("retains a reviewed model bound through a partial response and durable reload without releasing the hold", async () => {
+  const f = await fixture();
+  const receipt = await forwardEvalRequest({ ...f, route: { ...route, provisionalBound }, transport: async () => {
+    expect((await readEvalCostLedger(f.file)).state.requests.values().next().value?.bound?.evidence).toEqual(provisionalBound);
+    return Response.json({ object: "response", status: "in_progress", model: "fixture" },
+      { headers: { "x-bt-used-endpoint": route.endpointName! } });
+  } });
+  expect(receipt).toMatchObject({ costUsd: null, final: false });
+  expect((await readEvalCostLedger(f.file)).state.snapshot()).toMatchObject({ boundedUnresolvedRequests: 1, heldUsd: "0.003000000" });
+});
+it.each(["no-headers", "route", "failover", "model", "input-limit", "output-limit"])("disqualifies a model bound on %s evidence", async reason => {
+  const f = await fixture();
+  await forwardEvalRequest({ ...f, route: { ...route, provisionalBound }, transport: async () => {
+    if (reason === "no-headers") throw new Error("No response headers");
+    return Response.json({ object: "response", status: "in_progress", model: reason === "model" ? "other" : "fixture",
+      usage: { input_tokens: reason === "input-limit" ? 1001 : 10, input_tokens_details: { cache_write_tokens: 0, cached_tokens: 0 },
+        output_tokens: reason === "output-limit" ? 101 : 0 } },
+      { headers: { "x-bt-used-endpoint": reason === "route" ? "OTHER" : route.endpointName!,
+        ...(reason === "failover" ? { "x-bt-failover-to": "OTHER" } : {}) } });
+  } });
+  const replay = (await readEvalCostLedger(f.file)).state;
+  expect(replay.snapshot()).toMatchObject({ boundedUnresolvedRequests: 0, unknownRequests: 1, inflight: 0 });
+  expect([...replay.requests.values()][0]!.boundInvalidated).toBe(true);
+});
+
+it("refuses a changed model limit or rate with a stale reviewed bound before dispatch", async () => {
+  const f = await fixture(), transport = vi.fn<typeof fetch>();
+  for (const changed of [{ ...route, maximumInputTokens: 500 }, { ...route, price: { ...route.price, inputUsdPerMillion: "20" } }]) {
+    await expect(forwardEvalRequest({ ...f, route: { ...changed, provisionalBound }, transport })).rejects.toThrow("configuration changed");
+  }
+  expect(transport).not.toHaveBeenCalled();
+  expect(f.ledger.snapshot().requests).toHaveLength(0);
+});
+
+it("disqualifies excessive reported input even when missing cache details prevent pricing", async () => {
+  const f = await fixture();
+  await forwardEvalRequest({ ...f, route: { ...route, provisionalBound }, transport: async () => Response.json({
+    object: "response", status: "completed", model: "fixture", usage: { input_tokens: 1001, output_tokens: 1 },
+  }, { headers: { "x-bt-used-endpoint": route.endpointName! } }) });
+  expect((await readEvalCostLedger(f.file)).state.snapshot()).toMatchObject({ boundedUnresolvedRequests: 0, unknownRequests: 1, heldUsd: "0.003000000" });
+});

tests/runner-e2e/eval-gateway.ts

@@ -1,3 +1,4 @@
+import { reservationBound, validateCostBoundEvidence, type EvalCostBoundEvidence } from "./cost-bound-evidence.js";
 import { randomBytes, randomUUID, timingSafeEqual } from "node:crypto";
 import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
 import { once } from "node:events";
@@ -17,6 +18,7 @@ export interface EvalGatewayRoute {
   maximumInputTokens: number;
   maximumOutputTokens: number;
   timeoutMs: number;
+  provisionalBound?: EvalCostBoundEvidence;
 }
 export interface EvalGatewayReceipt {
   requestId: string; status?: number; route: "direct" | "braintrust";
@@ -32,6 +34,7 @@ const row = (value: unknown): Record<string, unknown> => value && typeof value =
 class EvalRequestError extends Error {}
 
 export function evalRequestReservation(route: EvalGatewayRoute): string {
+  if (route.provisionalBound !== undefined) validateCostBoundEvidence(route.provisionalBound);
   if (![route.maximumInputTokens, route.maximumOutputTokens, route.timeoutMs].every(n => Number.isSafeInteger(n) && n > 0)
       || route.timeoutMs > 300_000 || route.provider !== route.price.provider || !validEvalLedgerId(route.price.version) || !validEvalLedgerId(route.price.model)) throw new Error("Invalid eval gateway limits or price identity");
   const parse = (rate: string) => {
@@ -50,7 +53,16 @@ export function evalRequestReservation(route: EvalGatewayRoute): string {
     const output = parse(tier.outputUsdPerMillion); if (output > maximumOutputRate) maximumOutputRate = output;
   }
   const numerator = BigInt(route.maximumInputTokens) * maximumInputRate + BigInt(route.maximumOutputTokens) * maximumOutputRate;
-  return money((numerator + 999_999n) / 1_000_000n);
+  const reservation = money((numerator + 999_999n) / 1_000_000n);
+  modelReservationBound(route, reservation);
+  return reservation;
+}
+
+function modelReservationBound(route: EvalGatewayRoute, reservation: string) {
+  return route.provisionalBound ? reservationBound("model_limits", route.provisionalBound, {
+    provider: route.provider, mode: route.mode, price: route.price, maximumInputTokens: route.maximumInputTokens,
+    maximumOutputTokens: route.maximumOutputTokens, reservation,
+  }) : undefined;
 }
 
 function boundedPayload(value: unknown, route: EvalGatewayRoute) {
@@ -115,7 +127,8 @@ export async function forwardEvalRequest(input: {
   const receipt: EvalGatewayReceipt = { requestId, route: route.mode, outcome: "failed", usageComplete: false, final: false, costUsd: null,
     ...(route.mode === "braintrust" ? { braintrustParent: braintrustGatewayRequestParent(route.projectId!, requestId, ledger.snapshot().campaignId, input.identity) } : {}) };
   const usage = new ProviderUsageAccumulator(route.provider, body);
-  await ledger.reserve(requestId, input.identity, reservation);
+  const bound = modelReservationBound(route, reservation);
+  await ledger.reserve(requestId, input.identity, reservation, bound);
   const abort = new AbortController();
   const signal = AbortSignal.any([abort.signal, AbortSignal.timeout(Math.max(1, Math.min(route.timeoutMs, input.deadlineAt - Date.now()))), ...(input.signal ? [input.signal] : [])]);
   let response: Response | undefined;
@@ -184,6 +197,12 @@ export async function forwardEvalRequest(input: {
       delete tokens.reasoning;
       receipt.errorCode ??= "invalid_usage";
     }
+    const observedInput = observed.tokenBounds?.input ?? 0;
+    if (bound && (observed.tokenBounds?.valid === false || receipt.errorCode === "invalid_usage" || observed.model !== undefined && observed.model !== route.price.model
+        || !Number.isSafeInteger(observedInput) || observedInput > route.maximumInputTokens
+        || (observed.tokenBounds?.output ?? 0) > route.maximumOutputTokens
+        || route.mode === "braintrust" && (!response || receipt.endpoint !== route.endpointName
+          || receipt.cache === "HIT" || receipt.failoverFrom || receipt.failoverTo))) await ledger.invalidateBound(requestId);
     await input.persist(receipt);
     await ledger.observe({ observationId: `${requestId}:usage`, requestId, source: route.mode === "braintrust" ? "gateway" : "provider",
       provider: route.provider,

tests/runner-e2e/operating-overhead.ts

@@ -1,3 +1,4 @@
+import { reservationBound, type EvalCostBoundEvidence } from "./cost-bound-evidence.js";
 import { usdToUnits } from "../../packages/shared/src/money.js";
 
 export const EVAL_OVERHEAD_COMPONENTS = ["ci_compute", "artifact_storage", "observability", "shared_subscription"] as const;
@@ -7,7 +8,7 @@ export interface EvalOperatingOverhead {
   runner: "github-hosted" | "aws-fleet";
   verifiedAt: string;
   expiresAt: string;
-  components: Array<{ component: typeof EVAL_OVERHEAD_COMPONENTS[number]; maxUsd: string; reference: string }>;
+  components: Array<{ component: typeof EVAL_OVERHEAD_COMPONENTS[number]; maxUsd: string; reference: string; provisionalBound?: EvalCostBoundEvidence }>;
 }
 
 const object = (value: unknown, allowed: string[]) => {
@@ -16,6 +17,14 @@ const object = (value: unknown, allowed: string[]) => {
 };
 const reference = (value: unknown) => typeof value === "string" && /^[A-Za-z0-9][A-Za-z0-9_.:/-]{0,511}$/.test(value);
 
+export function operatingReservationBound(allowance: EvalOperatingOverhead, component: EvalOperatingOverhead["components"][number]) {
+  if (component.provisionalBound === undefined) return undefined;
+  return reservationBound("operating_allowance", component.provisionalBound, {
+    reference: allowance.reference, runner: allowance.runner, verifiedAt: allowance.verifiedAt, expiresAt: allowance.expiresAt,
+    component: { component: component.component, maxUsd: component.maxUsd, reference: component.reference },
+  });
+}
+
 export function validateOperatingOverhead(value: EvalOperatingOverhead, totalUsd: string): bigint {
   object(value, ["version", "reference", "runner", "verifiedAt", "expiresAt", "components"]);
   if (value.version !== 1 || !reference(value.reference) || !["github-hosted", "aws-fleet"].includes(value.runner)
@@ -29,7 +38,8 @@ export function validateOperatingOverhead(value: EvalOperatingOverhead, totalUsd
   const seen = new Set<string>();
   let reserved = 0n;
   for (const component of value.components) {
-    object(component, ["component", "maxUsd", "reference"]);
+    object(component, ["component", "maxUsd", "reference", "provisionalBound"]);
+    if (component.provisionalBound !== undefined) operatingReservationBound(value, component);
     if (!EVAL_OVERHEAD_COMPONENTS.includes(component.component) || seen.has(component.component)
         || !reference(component.reference) || typeof component.maxUsd !== "string"
         || !/^(?:0|[1-9][0-9]*)(?:\.[0-9]{1,9})?$/.test(component.maxUsd)) throw new Error("Invalid operating overhead component");

tests/runner-e2e/operations-report.ts

@@ -17,6 +17,7 @@ interface EvalOperationsAvailableReport {
   committedUsd: string;
   requests: number;
   unresolvedRequests: number;
+  boundedUnresolvedRequests?: number;
   conflictingRequests: number;
   inflight: number;
   breakdown: { reported: string; estimated: string; allocated: string; reconciled: string };
@@ -38,11 +39,15 @@ export function validateEvalOperationsReport(report: EvalOperationsReport): void
   }
   const dollars = (value: unknown): bigint => typeof value === "string" && /^(?:0|[1-9][0-9]*)\.[0-9]{9}$/.test(value) ? usdToUnits(value) : fail();
   const count = (value: unknown) => { if (!Number.isSafeInteger(value) || Number(value) < 0) fail(); };
-  fields(report, ["version", "scope", "ledgerCampaignId", "journalHash", "records", "deadlineAt", "limitUsd", "knownUsd", "heldUsd", "committedUsd", "requests", "unresolvedRequests", "conflictingRequests", "inflight", "breakdown", "attempts"]);
+  fields(report, ["version", "scope", "ledgerCampaignId", "journalHash", "records", "deadlineAt", "limitUsd", "knownUsd", "heldUsd", "committedUsd", "requests", "unresolvedRequests", "boundedUnresolvedRequests", "conflictingRequests", "inflight", "breakdown", "attempts"]);
   if (report.version !== 1 || report.scope !== "shared_gateway_ledger" || !validEvalLedgerId(report.ledgerCampaignId)
       || !/^[a-f0-9]{64}$/.test(report.journalHash) || !isPublicReportTimestamp(report.deadlineAt) || !Array.isArray(report.attempts)) fail();
   [report.records, report.requests, report.unresolvedRequests, report.conflictingRequests, report.inflight].forEach(count);
   if (report.records < 1 || report.unresolvedRequests > report.requests || report.conflictingRequests > report.unresolvedRequests || report.inflight > report.requests) fail();
+  if (report.boundedUnresolvedRequests !== undefined) {
+    count(report.boundedUnresolvedRequests);
+    if (report.boundedUnresolvedRequests > report.unresolvedRequests - report.conflictingRequests) fail();
+  }
   dollars(report.limitUsd);
   const known = dollars(report.knownUsd), held = dollars(report.heldUsd);
   if (known + held !== dollars(report.committedUsd)) fail();
@@ -84,18 +89,27 @@ async function readAvailableEvalOperationsReport(file: string): Promise<EvalOper
   const report: EvalOperationsAvailableReport = { version: 1, scope: "shared_gateway_ledger", ledgerCampaignId: snapshot.campaignId,
     journalHash: loaded.previous, records: loaded.sequence, deadlineAt: new Date(loaded.state.limits.deadlineAt).toISOString(),
     limitUsd: snapshot.limitUsd, knownUsd: snapshot.knownUsd, heldUsd: snapshot.heldUsd, committedUsd: snapshot.committedUsd,
-    requests: snapshot.requests.length, unresolvedRequests: snapshot.unknownRequests, conflictingRequests: snapshot.conflictingRequests,
+    requests: snapshot.requests.length, unresolvedRequests: snapshot.unknownRequests, boundedUnresolvedRequests: snapshot.boundedUnresolvedRequests, conflictingRequests: snapshot.conflictingRequests,
     inflight: snapshot.inflight, breakdown: snapshot.breakdown as EvalOperationsAvailableReport["breakdown"], attempts: [...groups.values()] };
   validateEvalOperationsReport(report);
   return report;
 }
 
+export function evalAccountingStatus(report: EvalOperationsReport): "unavailable" | "blocked" | "provisional" | "receipts_complete" {
+  validateEvalOperationsReport(report);
+  if (report.status === "unavailable") return "unavailable";
+  if (report.conflictingRequests || report.inflight || usdToUnits(report.committedUsd) > usdToUnits(report.limitUsd)
+      || report.unresolvedRequests !== (report.boundedUnresolvedRequests ?? 0)) return "blocked";
+  return report.unresolvedRequests ? "provisional" : "receipts_complete";
+}
+
 export function operationsReportText(report: EvalOperationsReport): string[] {
   if (report.status === "unavailable") return [
     "Request ledger accounting unavailable: the journal could not be read or validated. Request amounts, counts, and held reservations are unknown.",
     "Retained test results are preserved. Recover and validate the original journal before resuming paid work; do not reset the allowance or treat missing amounts as zero.",
   ];
   return [
+    `Accounting status: ${evalAccountingStatus(report)}. ${report.boundedUnresolvedRequests ?? 0} pending charges have reviewed bounds; ${report.unresolvedRequests - (report.boundedUnresolvedRequests ?? 0)} do not. Provisional completion preserves holds and requires later reconciliation; it does not establish invoice completeness.`,
     `Shared request ledger ${report.ledgerCampaignId}: $${report.knownUsd} known subtotal; $${report.heldUsd} additional held reservations; $${report.committedUsd} committed against $${report.limitUsd}.`,
     `Charges: $${report.breakdown.reconciled} reconciled; $${report.breakdown.reported} reported; $${report.breakdown.estimated} estimated; $${report.breakdown.allocated} allocated.`,
     `${report.requests} recorded requests or overhead reservations; ${report.unresolvedRequests} unresolved; ${report.conflictingRequests} conflicting; ${report.inflight} in flight. Deadline ${report.deadlineAt}.`,

tests/runner-e2e/operations.ts

@@ -35,7 +35,7 @@ export function validateEvalOperationsManifest(value: EvalOperationsManifest, ex
   if (value.route === "braintrust" && !/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(value.projectId ?? "")) throw new Error("Braintrust gateway requires an explicit project UUID");
   const models = new Map<string, bigint>();
   for (const model of value.models) {
-    fields(model, ["provider", "price", "maximumInputTokens", "maximumOutputTokens", "timeoutMs"]);
+    fields(model, ["provider", "price", "maximumInputTokens", "maximumOutputTokens", "timeoutMs", "provisionalBound"]);
     const rateFields = ["inputUsdPerMillion", "cachedReadUsdPerMillion", "outputUsdPerMillion", "cacheWrite5mUsdPerMillion", "cacheWrite1hUsdPerMillion", "cacheWrite30mUsdPerMillion", "cacheWrite12hUsdPerMillion"];
     fields(model.price, ["provider", "model", "version", "sourceUrl", "longContext", ...rateFields]);
     if (model.price.longContext) { fields(model.price.longContext, ["aboveInputTokens", "rates"]); fields(model.price.longContext.rates, rateFields); }

tests/runner-e2e/provider-usage.test.ts

@@ -111,3 +111,18 @@ it("prefers provider TTL splits and never repairs malformed or partial provider
     expect(estimateProviderCost(receipt, policyPrice)).toBeNull();
   }
 });
+
+it.each([
+  ["openai", { input_tokens: 1001, output_tokens: 1 }, 1001],
+  ["openai", { input_tokens_details: { cached_tokens: 1001 } }, 1001],
+  ["anthropic", { input_tokens: 1, cache_read_input_tokens: 2, cache_creation_input_tokens: 1000,
+    cache_creation: { ephemeral_5m_input_tokens: 999, ephemeral_1h_input_tokens: 0 } }, 1003],
+  ["anthropic", { input_tokens: 1, cache_creation: { ephemeral_5m_input_tokens: 900, ephemeral_1h_input_tokens: 100 } }, 1001],
+] as const)("retains %s input bounds independently of incomplete or contradictory price counters", (provider, usage, input) => {
+  const receipt = normalizeProviderUsage(provider, { usage });
+  expect(receipt).toMatchObject({ complete: false, tokenBounds: { input, valid: true } });
+  expect(estimateProviderCost(receipt, { ...price, provider })).toBeNull();
+});
+it.each([null, -1, 1.5, "1001", Number.MAX_SAFE_INTEGER + 1])("marks malformed input evidence %j ineligible for a bound", input_tokens => {
+  expect(normalizeProviderUsage("openai", { usage: { input_tokens } }).tokenBounds?.valid).toBe(false);
+});

tests/runner-e2e/provider-usage.ts

@@ -9,6 +9,7 @@ export interface ProviderUsageReceipt {
   provider: "openai" | "anthropic";
   tokens: Tokens;
   complete: boolean;
+  tokenBounds?: { input: number; output: number; valid: boolean };
   providerResponseId?: string;
   model?: string;
   cacheWrite5m?: number;
@@ -38,6 +39,24 @@ function openAiRequestCachePolicy(payload: unknown): { model: string; ttl: "30m"
   return { model: body.model, ttl };
 }
 
+function tokenBounds(provider: "openai" | "anthropic", usage: Row) {
+  const input = row(usage.input_tokens_details), output = row(usage.output_tokens_details);
+  const writes = provider === "openai" ? row(input.cache_write_tokens_details) : row(usage.cache_creation);
+  const rawWrite = provider === "openai" ? input.cache_write_tokens : usage.cache_creation_input_tokens;
+  const rawRead = provider === "openai" ? input.cached_tokens : usage.cache_read_input_tokens;
+  const ttls = provider === "openai" ? [writes.ttl_30m_tokens, writes.ttl_12h_tokens]
+    : [writes.ephemeral_5m_input_tokens, writes.ephemeral_1h_input_tokens];
+  const inputs = [usage.input_tokens, rawRead, rawWrite, ...ttls];
+  const outputs = [usage.output_tokens, ...(provider === "openai" ? [output.reasoning_tokens] : [])];
+  const value = (raw: unknown) => count(raw) ?? 0;
+  const cached = value(rawRead) + Math.max(value(rawWrite), value(ttls[0]) + value(ttls[1]));
+  const inputBound = provider === "openai" ? Math.max(value(usage.input_tokens), cached) : value(usage.input_tokens) + cached;
+  const outputBound = Math.max(...outputs.map(value));
+  return { input: inputBound, output: outputBound,
+    valid: [...inputs, ...outputs].every(raw => raw === undefined || count(raw) !== undefined)
+      && Number.isSafeInteger(inputBound) && Number.isSafeInteger(outputBound) };
+}
+
 export function normalizeProviderUsage(provider: "openai" | "anthropic", payload: unknown): ProviderUsageReceipt {
   const body = row(payload), usage = row(body.usage);
   const output = count(usage.output_tokens);
@@ -65,7 +84,7 @@ export function normalizeProviderUsage(provider: "openai" | "anthropic", payload
   }
   const complete = tokens.input !== undefined && tokens.cachedRead !== undefined && tokens.cachedWrite !== undefined && output !== undefined
     && !(tokens.reasoning !== undefined && tokens.reasoning > output);
-  return { provider, tokens: Object.fromEntries(Object.entries(tokens).filter(([, value]) => value !== undefined)), complete,
+  return { provider, tokenBounds: tokenBounds(provider, usage), tokens: Object.fromEntries(Object.entries(tokens).filter(([, value]) => value !== undefined)), complete,
     ...(typeof body.id === "string" ? { providerResponseId: body.id } : {}),
     ...(typeof body.model === "string" ? { model: body.model } : {}),
     ...(cacheWrite5m !== undefined ? { cacheWrite5m } : {}), ...(cacheWrite1h !== undefined ? { cacheWrite1h } : {}),

tests/runner-e2e/provisional-accounting.test.ts

@@ -0,0 +1,131 @@
+import { mkdtemp, readFile, rm } from "node:fs/promises";
+import { tmpdir } from "node:os";
+import path from "node:path";
+import { afterEach, expect, it } from "vitest";
+import { openEvalCostLedger, type EvalSpendLimits } from "./cost-ledger.js";
+import { costBoundConfigurationSha256, reservationBound, type EvalCostBoundEvidence } from "./cost-bound-evidence.js";
+import { EVAL_OVERHEAD_COMPONENTS } from "./operating-overhead.js";
+import { evalAccountingStatus, readEvalOperationsReport, renderEvalOperationsReport, validateEvalOperationsReport } from "./operations-report.js";
+import { scheduledCampaignFindings } from "./scheduled-finalization.js";
+import type { RunnerE2ECampaign } from "./types.js";
+
+const directories: string[] = [];
+afterEach(async () => { for (const dir of directories.splice(0)) await rm(dir, { recursive: true, force: true }); });
+const evidence: EvalCostBoundEvidence = { version: 1, scope: "full_lifecycle", evidenceSha256: "a".repeat(64), configurationSha256: costBoundConfigurationSha256({ fixture: true }) };
+const identity = { executionId: "cell", attemptId: "one", phase: "agent" as const };
+const bound = reservationBound("model_limits", evidence, { fixture: true });
+async function fixture(overhead = false) {
+  const dir = await mkdtemp(path.join(tmpdir(), "provisional-accounting-")); directories.push(dir);
+  const file = path.join(dir, "ledger.jsonl");
+  const limits: EvalSpendLimits = { campaignId: "fixture", maxUsd: "20", maxRequests: 20, maxConcurrent: 2,
+    deadlineAt: "2099-01-01T00:00:00Z", maxAttemptsPerExecution: 1,
+    ...(overhead ? { overhead: { version: 1 as const, reference: "fixture-audit", runner: "github-hosted" as const,
+      verifiedAt: new Date(Date.now() - 60_000).toISOString(), expiresAt: new Date(Date.now() + 86_400_000).toISOString(),
+      components: EVAL_OVERHEAD_COMPONENTS.map(component => ({ component, maxUsd: component === "shared_subscription" ? "0" : "1",
+        reference: `fixture-${component}`, provisionalBound: evidence })) } } : {}) };
+  if (limits.overhead) for (const component of limits.overhead.components) {
+    component.provisionalBound = { ...evidence, configurationSha256: costBoundConfigurationSha256({
+      reference: limits.overhead.reference, runner: limits.overhead.runner, verifiedAt: limits.overhead.verifiedAt, expiresAt: limits.overhead.expiresAt,
+      component: { component: component.component, maxUsd: component.maxUsd, reference: component.reference },
+    }) };
+  }
+  const ledger = await openEvalCostLedger(file, limits);
+  await ledger.admitAttempt(identity.executionId, identity.attemptId);
+  return { ledger, file, limits };
+}
+function accountingFindings(operations: Awaited<ReturnType<typeof readEvalOperationsReport>>) {
+  return scheduledCampaignFindings({ results: [], expected: [], operations } as unknown as RunnerE2ECampaign)
+    .filter(finding => finding.startsWith("accounting_"));
+}
+
+it("allows reviewed pending operating/model costs without changing holds or finality, including after restart", async () => {
+  const { ledger, file, limits } = await fixture(true);
+  await ledger.reserve("request", identity, "2", bound);
+  await ledger.observe({ observationId: "partial", requestId: "request", source: "provider", provider: "fixture", currency: "USD",
+    route: "direct", classification: "estimated", pricingVersion: "fixture", costUsd: "0.1", final: false });
+  await ledger.finish("request", "cancelled");
+  await ledger.close();
+  const original = await readFile(file, "utf8");
+  const report = await readEvalOperationsReport(file);
+  expect(report).toMatchObject({ unresolvedRequests: 5, boundedUnresolvedRequests: 5, knownUsd: "0.100000000", heldUsd: "4.900000000", committedUsd: "5.000000000" });
+  expect(evalAccountingStatus(report)).toBe("provisional");
+  expect(accountingFindings(report)).toEqual([]);
+  const rendered = renderEvalOperationsReport(report);
+  expect(rendered).toContain("Accounting status: provisional");
+  expect(rendered).toContain("requires later reconciliation");
+  expect(rendered).not.toContain(evidence.evidenceSha256);
+  const resumed = await openEvalCostLedger(file, limits);
+  try { expect(resumed.snapshot()).toMatchObject({ boundedUnresolvedRequests: 5, heldUsd: "4.900000000" }); }
+  finally { await resumed.close(); }
+  expect(await readFile(file, "utf8")).toBe(original);
+});
+
+it("keeps legacy reservations and legacy reports unqualified instead of inferring a bound from dollars", async () => {
+  const { ledger, file } = await fixture();
+  try {
+    await ledger.reserve("request", identity, "2");
+    await ledger.finish("request", "cancelled");
+    const report = await readEvalOperationsReport(file);
+    expect(evalAccountingStatus(report)).toBe("blocked");
+    expect(accountingFindings(report)).toEqual(["accounting_incomplete"]);
+    if (report.status === "unavailable") throw new Error("Missing fixture report");
+    delete report.boundedUnresolvedRequests;
+    validateEvalOperationsReport(report);
+    expect(evalAccountingStatus(report)).toBe("blocked");
+    expect(() => validateEvalOperationsReport({ ...report, boundedUnresolvedRequests: 2 })).toThrow();
+  } finally { await ledger.close(); }
+});
+
+it.each(["inflight", "invalidated", "partial-over-bound", "conflict", "over-budget"])("blocks %s even with reviewed bound evidence", async reason => {
+  const { ledger, file } = await fixture();
+  try {
+    await ledger.reserve("request", identity, "2", bound);
+    if (reason === "invalidated") await ledger.invalidateBound("request");
+    if (reason !== "inflight") await ledger.finish("request", "cancelled");
+    if (["partial-over-bound", "conflict", "over-budget"].includes(reason)) {
+      await ledger.observe({ observationId: "first", requestId: "request", source: "provider", provider: "fixture", currency: "USD", route: "direct",
+        classification: "reported", costUsd: reason === "over-budget" ? "21" : "3", final: reason !== "partial-over-bound" });
+    }
+    if (reason === "conflict") await ledger.observe({ observationId: "second", requestId: "request", source: "provider", provider: "fixture",
+      currency: "USD", route: "direct", classification: "reported", costUsd: "4", final: true });
+    const report = await readEvalOperationsReport(file);
+    expect(evalAccountingStatus(report)).toBe("blocked");
+    expect(accountingFindings(report)).toContain("accounting_incomplete");
+    if (reason === "over-budget") expect(accountingFindings(report)).toContain("accounting_over_budget");
+    expect(report).toMatchObject({ boundedUnresolvedRequests: 0 });
+  } finally { await ledger.close(); }
+});
+
+it("lets a later complete receipt settle the original reservation without treating the provisional hold as a charge", async () => {
+  const { ledger, file } = await fixture();
+  try {
+    await ledger.reserve("request", identity, "2", bound);
+    await ledger.finish("request", "cancelled");
+    expect(evalAccountingStatus(await readEvalOperationsReport(file))).toBe("provisional");
+    await ledger.observe({ observationId: "invoice", requestId: "request", source: "invoice", provider: "fixture", currency: "USD",
+      route: "direct", classification: "reconciled", costUsd: "0.2", final: true });
+    const report = await readEvalOperationsReport(file);
+    expect(evalAccountingStatus(report)).toBe("receipts_complete");
+    expect(report).toMatchObject({ boundedUnresolvedRequests: 0, knownUsd: "0.200000000", heldUsd: "0.000000000" });
+  } finally { await ledger.close(); }
+});
+
+it("rejects malformed or mismatched evidence before creating a reservation", async () => {
+  const { ledger } = await fixture();
+  try {
+    for (const invalid of [ { ...bound, calculationSha256: "missing" }, { ...bound, basis: "operating_allowance" },
+      { ...bound, evidence: { ...evidence, scope: "current_day" } }, { ...bound, evidence: { ...evidence, evidenceSha256: "bad" } },
+      { ...bound, evidence: { ...evidence, privateField: "not-allowed" } } ]) {
+      await expect(ledger.reserve("request", identity, "2", invalid as never)).rejects.toThrow();
+    }
+    expect(ledger.snapshot().requests).toHaveLength(0);
+  } finally { await ledger.close(); }
+});
+
+it("does not accept null as reviewed operating-bound evidence", async () => {
+  const { ledger, limits } = await fixture(true);
+  await ledger.close();
+  limits.overhead!.components[0]!.provisionalBound = null as never;
+  const directory = await mkdtemp(path.join(tmpdir(), "provisional-malformed-")); directories.push(directory);
+  await expect(openEvalCostLedger(path.join(directory, "ledger.jsonl"), limits)).rejects.toThrow("Invalid reviewed cost bound evidence");
+});

tests/runner-e2e/scheduled-finalization.ts

@@ -1,3 +1,5 @@
+import { usdToUnits } from "../../packages/shared/src/money.js";
+import { evalAccountingStatus } from "./operations-report.js";
 import { readFile, writeFile } from "node:fs/promises";
 import path from "node:path";
 import { pathToFileURL } from "node:url";
@@ -44,7 +46,10 @@ export function scheduledCampaignFindings(campaign: RunnerE2ECampaign): string[]
   if (campaign.accounting?.attempts.some(row => row.cleanup !== "passed")) findings.push("cleanup_failure");
   const operations = campaign.operations;
   if (!operations || operations.status === "unavailable") findings.push("accounting_unavailable");
-  else if (operations.unresolvedRequests || operations.conflictingRequests || operations.inflight) findings.push("accounting_incomplete");
+  else {
+    if (evalAccountingStatus(operations) === "blocked") findings.push("accounting_incomplete");
+    if (usdToUnits(operations.committedUsd) > usdToUnits(operations.limitUsd)) findings.push("accounting_over_budget");
+  }
   const feedback = summarizeCampaignFeedback(campaign.accounting?.attempts ?? campaign.results);
   if (feedback.incomplete || feedback.failedSubmissions || feedback.unknownSubmissions) findings.push("feedback_collection_incomplete");
   if (!campaign.feedbackReview || campaign.feedbackReview.decisions.some(row => ["actionable", "needs_investigation"].includes(row.disposition))) findings.push("feedback_needs_review");
@@ -107,6 +112,7 @@ export async function finalizeScheduledCampaign(env: NodeJS.ProcessEnv) {
   if (indexing !== "verified") findings.push("trace_or_experiment_indexing_incomplete");
   await writeFile(path.join(directory, "findings.json"), `${JSON.stringify({ version: 1, findings,
     nativeIndexVerification: indexing, invoiceReconciliation: "pending",
+    accountingStatus: campaign.operations ? evalAccountingStatus(campaign.operations) : "unavailable",
     feedbackReviewMethod: "Only exact synthetic fixture bodies with all corroborating graders are automatically dismissed; all other feedback requires review" }, null, 2)}\n`, { mode: 0o600 });
   for (const finding of findings) console.error(`::error title=Eval operations::${finding}`);
   return findings;