mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-10 01:50:13 +01:00
fix(budgets): bound the accounting hold so one missing receipt cannot wedge a company (#19700)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Pending run accounting holds new company work until it settles (#19497, #19606, #19673) > - A run settles only with closure evidence: a verified local stop, or a terminated remote sandbox. Without that evidence it held the whole company forever. In restart-chaos storm 4, a single run held 35 tasks for hours > - Two more shapes kept the hold alive. A run whose result had already been accepted never counted as closed. A recovery that already owned its held reservation was refused by other tasks' recovering accounting, so concurrent recoveries waited on each other > - This pull request bounds the hold, closes accepted-result runs on a verified stop, and lets an owned recovery finish > - The benefit is that no single missing receipt can wedge a company, while genuinely unknown spend still surfaces under the blocking policy ## Linked Issues or Issue Description Follow-up to #19497, #19606 and #19673. **What happened?** - Storm 4: run `f879895f` timed out in `session.open`. Its local stop was never verified (its process identity is still recorded), recovery ended in `terminal_failure`, and its receipt never became ready. From then on, every new-work wake got 409 "New work must wait while a native run recovers its unfinished accounting". That was 93 refusals, 35 blocked tasks, and no Retry. - Storm 4: run `6d3c76bf` was resuming as the recovery owner with its own held reservation. `reserveRunBudget` refused it because other tasks' runs were recovering. - Wave s7401: run `0a3a849d` failed after its result was accepted, with a verified local stop. It had 10 receipts across 2 sources, all incomplete with 0 tokens. It stayed "completed runs await accounting" because closure only covered coordinators in terminal failure with no result. **What did you expect?** A run without closure evidence holds new work only for a bounded time, then settles visibly. An accepted-result run with a verified stop closes like a terminal failure. A recovery that already owns admitted work can finish. **Steps to reproduce** 1. Restart the control plane during several native turns so that one run's local stop is never verified. 2. Observe the run stay "recovering" and every new wake refused indefinitely. ## What Changed - `run-cost-accounting.ts`: - A coordinator counts as settled when it ended in terminal failure without a result, or when it has an accepted result. The verified-stop and remote-termination closures use that rule, which covers the s7401 shape. - A native run whose provider never started settles at once, like a pre-provider failure, with no charge and its reservation released. Its recovery has settled, no usage receipt was ever journaled (a provider turn journals one as soon as it starts), and its local process never launched or was verified stopped (wave s7403, run 3c1d7b70, `session.open` timeout). - New bound `ACCOUNTING_CLOSURE_TIMEOUT_MS`: the remote runner reconnect grace (7 minutes) plus 3 minutes. A terminal native run with a settled coordinator and no closure evidence settles at the bound: - its known usage is recorded as unpriced (or estimated from the rate card), with provenance evidence "closure evidence never arrived…"; - it is marked `accountingClosureTimedOut`; - a `cost.accounting_closure_timed_out` activity entry names the run. - The reconciler selects these runs, and `readCostAccountingWork` schedules a wake at the bound. - `budget-exposure.ts`: a timed-out closure's final charge is exempt from none of the unpriced-block exemptions. Under the `block` policy it raises the normal budget notice ("recorded usage has no reliable price") instead of a silent hold. - `budgets.ts`, `budget-reservations.ts`, `run-dispatch/adapters/postgres.ts`: when a validated recovery owner continues admitted work, admission and both dispatch rechecks apply only hard stops and pauses. Pending and recovering accounting holds still gate new work. - `doc/SPEC-implementation.md`: documents the bound, the accepted-result closure and the provenance. ## Verification - `closed-native-receipt-accounting.test.ts` (embedded Postgres). All of these fail on master and pass here: - s7401 shape (accepted result, verified stop, all receipts incomplete with 0 tokens): settles and clears admission. - A run without closure evidence: held before the bound with a scheduled wake, then settled with the "closure evidence never arrived" provenance, the `accountingClosureTimedOut` flag and an activity entry. Admission clears under `allow`. - Storm 4 `f879895f` shape (terminal failure, recorded process identity, no stop event): held, then released after the bound. - The blocking policy still blocks on a timed-out closure. - s7403 shape (no receipts, model unknown): a run whose process never launched, or was verified stopped, settles at once with no charge and its reservation released. With an unverified process it waits for the bound. - Storm 4 `6d3c76bf` shape: a recovery owner with its held reservation reserves past another task's recovering accounting, while new work for another task still waits. - Existing suites pass locally: budget-enforcement-scoping, budget-enforcement-worker, native-stop-accounting, cost-data-quality, heartbeat-cost-accounting, run-cost-accounting, budgets-service. Server typecheck passes. - The other storm 4 terminal-failure runs (`2e477ae9`, `e4d2e14d`, `6d3c76bf`, `7d21a44c`, `4bc49df0`, `1d85fbd9`) were already settled by #19673 in that database (`cost_accounting_pending=false`) and were not holding admission. `f879895f` alone held it. ## Risks - After the bound, a run whose provider might still be running settles as unpriced. Under the `block` policy that raises a hard-stop budget notice, by design. Under `allow`, work continues and a late receipt still reconciles. - A recovery owner skips accounting holds, so concurrent recoveries can spend while other tasks' accounting is unknown. Hard stops still apply, consistent with best-effort budget enforcement. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file editing, embedded Postgres tests). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed
1 parent
4612fa9fb4
commit
8a34c1b62c
8 files changed
+344
-31
No files matched your search
@@ -4,11 +4,11 @@ import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest";
|
||||
import {
|
||||
agents, budgetIncidents, budgetPolicies, budgetReservations, companies, costEvents, createDb, environmentLeases, environments,
|
||||
heartbeatRunEvents, heartbeatRuns, issues,
|
||||
nativeRunFinalizations, type Db,
|
||||
activityLog, completionContracts, nativeRunFinalizations, nativeRunResults, type Db,
|
||||
} from "@paperclipai/db";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
|
||||
import { budgetService, flushBudgetEvaluationWork, recheckAdmittedRunBudgetReadOnly } from "../services/budgets.js";
|
||||
import { accountRunCost, reconcileRunCosts } from "../services/run-cost-accounting.js";
|
||||
import { accountRunCost, readCostAccountingWork, reconcileRunCosts } from "../services/run-cost-accounting.js";
|
||||
import { persistUsageReceipt } from "../services/usage-receipts.js";
|
||||
import { recheckRunBudgetReservationInTransaction, reserveRunBudget } from "../services/budget-reservations.js";
|
||||
|
||||
@@ -197,6 +197,57 @@ const support = await getEmbeddedPostgresTestSupport();
|
||||
await expect(reserveRunBudget(db, f.companyId, other, null)).rejects.toThrow(/recovers its unfinished accounting/);
|
||||
});
|
||||
|
||||
// Storm 4 (run 6d3c76bf): after a restart storm several tasks recover at once.
|
||||
// A recovery that already owns its run's held reservation was refused by
|
||||
// another task's recovering accounting, so no recovery could finish.
|
||||
it("lets a recovery that owns its held reservation continue past other tasks' accounting holds", async () => {
|
||||
const f = await fixture({ phase: "observed" });
|
||||
const [otherIssue] = await db.insert(issues).values({ companyId: f.companyId, title: "Recovering task", status: "in_progress",
|
||||
assigneeAgentId: f.otherAgentId }).returning();
|
||||
const recovering = randomUUID(), owner = `recovery:${randomUUID()}`;
|
||||
await db.insert(heartbeatRuns).values({ id: recovering, companyId: f.companyId, agentId: f.otherAgentId, status: "running",
|
||||
runtimeMode: "native", scopeKind: "issue", issueId: otherIssue.id, nativeIssueId: otherIssue.id });
|
||||
await db.insert(nativeRunFinalizations).values({ companyId: f.companyId, issueId: otherIssue.id, runId: recovering,
|
||||
phase: "observed", attempt: 2, leaseOwner: owner, leaseExpiresAt: new Date(Date.now() + 60_000) });
|
||||
await db.insert(budgetReservations).values({ companyId: f.companyId, runId: recovering, agentId: f.otherAgentId,
|
||||
amountCents: "0", state: "held", providerStartedAt: new Date() });
|
||||
expect(await f.block()).toMatchObject({ reason: "New work must wait while a native run recovers its unfinished accounting." });
|
||||
await expect(reserveRunBudget(db, f.companyId, recovering, null, {}, owner)).resolves.toMatchObject({ reused: true });
|
||||
// New work for another task still waits.
|
||||
const fresh = randomUUID();
|
||||
await db.insert(heartbeatRuns).values({ id: fresh, companyId: f.companyId, agentId: f.otherAgentId, status: "queued",
|
||||
runtimeMode: "native", scopeKind: "issue", issueId: otherIssue.id, nativeIssueId: null });
|
||||
await expect(reserveRunBudget(db, f.companyId, fresh, null)).rejects.toThrow(/recovers its unfinished accounting/);
|
||||
});
|
||||
|
||||
it("lets an owned recovery dispatch when other runs' held reservations fill the budget", async () => {
|
||||
const f = await fixture({ phase: "observed" });
|
||||
const [otherIssue] = await db.insert(issues).values({ companyId: f.companyId, title: "Recovering task", status: "in_progress",
|
||||
assigneeAgentId: f.otherAgentId }).returning();
|
||||
const recovering = randomUUID(), busy = randomUUID(), owner = `recovery:${randomUUID()}`;
|
||||
await db.insert(heartbeatRuns).values([
|
||||
{ id: recovering, companyId: f.companyId, agentId: f.otherAgentId, status: "running",
|
||||
runtimeMode: "native", scopeKind: "issue", issueId: otherIssue.id, nativeIssueId: otherIssue.id },
|
||||
{ id: busy, companyId: f.companyId, agentId: f.otherAgentId, status: "running" },
|
||||
]);
|
||||
await db.insert(nativeRunFinalizations).values({ companyId: f.companyId, issueId: otherIssue.id, runId: recovering,
|
||||
phase: "observed", attempt: 2, leaseOwner: owner, leaseExpiresAt: new Date(Date.now() + 60_000) });
|
||||
// Held reservations total 600 of the 500-cent budget.
|
||||
const [own] = await db.insert(budgetReservations).values([
|
||||
{ companyId: f.companyId, runId: recovering, agentId: f.otherAgentId, amountCents: "300", state: "held", providerStartedAt: new Date() },
|
||||
{ companyId: f.companyId, runId: busy, agentId: f.otherAgentId, amountCents: "300", state: "held", providerStartedAt: new Date() },
|
||||
]).returning();
|
||||
await expect(reserveRunBudget(db, f.companyId, recovering, null, {}, owner)).resolves.toMatchObject({ reused: true });
|
||||
const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, recovering));
|
||||
expect(await db.transaction(tx => recheckRunBudgetReservationInTransaction(tx as unknown as typeof db, [],
|
||||
run, own.id, owner))).toBeNull();
|
||||
expect(await recheckAdmittedRunBudgetReadOnly(db, { companyId: f.companyId, agentId: f.otherAgentId, runId: recovering,
|
||||
projectId: null, reservationId: own.id, admittedRecovery: true })).toBeNull();
|
||||
// Without the recovery ownership, the same dispatch is refused.
|
||||
expect(await recheckAdmittedRunBudgetReadOnly(db, { companyId: f.companyId, agentId: f.otherAgentId, runId: recovering,
|
||||
projectId: null, reservationId: own.id })).not.toBeNull();
|
||||
});
|
||||
|
||||
// RC-06e: unpriced charges with no observed usage, or with a known partial
|
||||
// cost, hard-stopped a company at 120 of 2000 cents.
|
||||
it.each([
|
||||
@@ -375,6 +426,156 @@ const support = await getEmbeddedPostgresTestSupport();
|
||||
expect(await f.charges()).toHaveLength(0);
|
||||
});
|
||||
|
||||
// Wave s7401: a local ACPX Claude run was killed after its result was
|
||||
// accepted. Ten receipts across two sources were all incomplete with zero
|
||||
// tokens, its local stop was verified, and the company refused all new work.
|
||||
async function acceptedResultFixture(options: { stopEvidence?: boolean; finishedMinutesAgo?: number } = {}) {
|
||||
const f = await fixture({ stopEvidence: options.stopEvidence });
|
||||
const contractId = randomUUID();
|
||||
await db.insert(completionContracts).values({ id: contractId, companyId: f.companyId, issueId: f.issue.id, revision: 1,
|
||||
schemaVersion: "paperclip.completion-contract.v1", policyVersion: "result-worker-v1", risk: "standard",
|
||||
completionAuthority: "server_arbiter", incompleteCriteriaPolicy: "preserve_non_terminal",
|
||||
contractJson: { revision: "result-worker-v1", objective: "Finish", criteria: [{ id: "objective", requirement: "Complete the task" }] },
|
||||
canonicalSha256: contractId, createdByActorType: "system", createdByActorId: "test" });
|
||||
await db.update(heartbeatRuns).set({ completionContractId: contractId }).where(eq(heartbeatRuns.id, f.run.id));
|
||||
const [accepted] = await db.insert(nativeRunResults).values({ companyId: f.companyId, issueId: f.issue.id, runId: f.run.id,
|
||||
completionContractId: contractId, serverFingerprint: "s7401", schemaStatus: "accepted", resultJson: {},
|
||||
canonicalSha256: "c".repeat(64) }).returning();
|
||||
await db.update(nativeRunFinalizations).set({ phase: "committed", resultId: accepted.id, failureCode: null })
|
||||
.where(eq(nativeRunFinalizations.runId, f.run.id));
|
||||
await db.update(heartbeatRuns).set({ errorCode: "adapter_failed",
|
||||
finishedAt: new Date(Date.now() - (options.finishedMinutesAgo ?? 0) * 60_000),
|
||||
usageJson: sql`${heartbeatRuns.usageJson} || ${JSON.stringify({ accountingReceiptSourceFences: { [randomUUID()]: 5 },
|
||||
accountingReceiptSequence: 5 })}::jsonb` }).where(eq(heartbeatRuns.id, f.run.id));
|
||||
return f;
|
||||
}
|
||||
|
||||
it("closes a result-accepted run's lost receipt once its local stop is verified", async () => {
|
||||
const f = await acceptedResultFixture();
|
||||
expect(await f.block()).toMatchObject({ reason: "Company cannot start work while completed runs await accounting." });
|
||||
expect(await reconcileRunCosts(db)).toMatchObject({ accounted: 1 });
|
||||
await flushBudgetEvaluationWork(db);
|
||||
expect(await f.charges()).toMatchObject([{ costStatus: "unpriced" }]);
|
||||
expect(await f.block()).toBeNull();
|
||||
});
|
||||
|
||||
// RC-06g: with no closure evidence at all, one run must not hold a company's
|
||||
// new work forever.
|
||||
it("holds a run without closure evidence only until the bound, then settles it visibly", async () => {
|
||||
const recent = await acceptedResultFixture({ stopEvidence: false, finishedMinutesAgo: 5 });
|
||||
expect(await reconcileRunCosts(db)).toMatchObject({ accounted: 0 });
|
||||
expect(await recent.block()).not.toBeNull();
|
||||
// The worker schedules a wake for the bound instead of polling.
|
||||
const work = await readCostAccountingWork(db, Date.now());
|
||||
expect(work.nextRunAt).not.toBeNull();
|
||||
expect(work.nextRunAt!).toBeGreaterThan(Date.now());
|
||||
// One cutoff decides both: past the bound the run is actionable now.
|
||||
const later = await readCostAccountingWork(db, work.nextRunAt! + 1);
|
||||
expect(later.pending).toBe(true);
|
||||
|
||||
await db.update(heartbeatRuns).set({ finishedAt: new Date(Date.now() - 11 * 60_000) }).where(eq(heartbeatRuns.id, recent.run.id));
|
||||
expect(await reconcileRunCosts(db)).toMatchObject({ accounted: 1 });
|
||||
await flushBudgetEvaluationWork(db);
|
||||
const [charge] = await recent.charges();
|
||||
expect(charge).toMatchObject({ costStatus: "unpriced" });
|
||||
expect(charge.pricingProvenance).toMatchObject({ evidence: expect.stringContaining("closure evidence never arrived") });
|
||||
expect((await recent.readRun()).usageJson).toMatchObject({ accountingClosureTimedOut: true });
|
||||
// A board-visible notice names the run.
|
||||
expect(await db.select().from(activityLog).where(and(eq(activityLog.companyId, recent.companyId),
|
||||
eq(activityLog.action, "cost.accounting_closure_timed_out"), eq(activityLog.entityId, recent.run.id)))).toHaveLength(1);
|
||||
expect(await recent.block()).toBeNull();
|
||||
});
|
||||
|
||||
// Storm 4 (run f879895f): session.open timed out, the local stop was never
|
||||
// verified (the identity is still recorded), and recovery ended in terminal
|
||||
// failure. That one run held all new company work for hours.
|
||||
it("bounds the hold of a terminal failure whose stop was never verified", async () => {
|
||||
const f = await fixture({ processPid: 15099, stopEvidence: false });
|
||||
await db.update(heartbeatRuns).set({ errorCode: "native_session_interrupted", finishedAt: new Date(Date.now() - 2 * 60_000) })
|
||||
.where(eq(heartbeatRuns.id, f.run.id));
|
||||
expect(await reconcileRunCosts(db)).toMatchObject({ accounted: 0 });
|
||||
expect(await f.block()).toMatchObject({ reason: "New work must wait while a native run recovers its unfinished accounting." });
|
||||
await db.update(heartbeatRuns).set({ finishedAt: new Date(Date.now() - 11 * 60_000) }).where(eq(heartbeatRuns.id, f.run.id));
|
||||
expect(await reconcileRunCosts(db)).toMatchObject({ accounted: 1 });
|
||||
await flushBudgetEvaluationWork(db);
|
||||
expect((await f.readRun()).usageJson).toMatchObject({ accountingClosureTimedOut: true });
|
||||
expect(await f.block()).toBeNull();
|
||||
});
|
||||
|
||||
// Wave s7403 (run 3c1d7b70): the final bootstrap died in session.open. No
|
||||
// usage receipt was ever journaled and the model was never known, yet the
|
||||
// run held company work like a lost receipt.
|
||||
async function neverStartedFixture(processState: "never_launched" | "verified_stop" | "unverified") {
|
||||
const f = await fixture({ processPid: processState === "unverified" ? 15099 : null,
|
||||
stopEvidence: processState === "verified_stop" });
|
||||
await db.update(heartbeatRuns).set({ errorCode: "native_session_interrupted",
|
||||
usageJson: sql`(${heartbeatRuns.usageJson} - 'accountingReceiptOwner' - 'budgetExposure') || '{"accountingReceiptSequence":0,"model":null,"costStatus":null,"provider":null,"biller":null}'::jsonb` })
|
||||
.where(eq(heartbeatRuns.id, f.run.id));
|
||||
if (processState === "never_launched") await db.delete(heartbeatRunEvents).where(eq(heartbeatRunEvents.runId, f.run.id));
|
||||
return f;
|
||||
}
|
||||
|
||||
it.each(["never_launched", "verified_stop"] as const)("settles a run whose provider never started at once (%s)", async processState => {
|
||||
const f = await neverStartedFixture(processState);
|
||||
expect(await reconcileRunCosts(db)).toMatchObject({ accounted: 1 });
|
||||
await flushBudgetEvaluationWork(db);
|
||||
expect((await f.readRun()).costAccountingPending).toBe(false);
|
||||
expect(await f.charges()).toHaveLength(0);
|
||||
expect((await f.reservation()).state).toBe("released");
|
||||
expect(await f.block()).toBeNull();
|
||||
});
|
||||
|
||||
it("keeps waiting for the bound when the provider process was never verified stopped", async () => {
|
||||
const f = await neverStartedFixture("unverified");
|
||||
expect(await reconcileRunCosts(db)).toMatchObject({ accounted: 0 });
|
||||
expect((await f.readRun()).costAccountingPending).toBe(true);
|
||||
});
|
||||
|
||||
it("bounds a succeeded run whose accepted result left its receipt incomplete", async () => {
|
||||
const f = await acceptedResultFixture({ stopEvidence: false, finishedMinutesAgo: 11 });
|
||||
await db.update(heartbeatRuns).set({ status: "succeeded", errorCode: null }).where(eq(heartbeatRuns.id, f.run.id));
|
||||
expect(await reconcileRunCosts(db)).toMatchObject({ accounted: 1 });
|
||||
await flushBudgetEvaluationWork(db);
|
||||
expect((await f.readRun()).usageJson).toMatchObject({ accountingClosureTimedOut: true });
|
||||
expect(await f.block()).toBeNull();
|
||||
});
|
||||
|
||||
it("never schedules a wake for a bound that already passed or for a capture failure", async () => {
|
||||
const past = await acceptedResultFixture({ stopEvidence: false, finishedMinutesAgo: 11 });
|
||||
await db.update(heartbeatRuns).set({ usageJson: sql`${heartbeatRuns.usageJson} || '{"accountingCaptureFailed":true}'::jsonb` })
|
||||
.where(eq(heartbeatRuns.id, past.run.id));
|
||||
expect((await readCostAccountingWork(db, Date.now())).nextRunAt).toBeNull();
|
||||
});
|
||||
|
||||
it("stops blocking once a complete late receipt accounts for a timed-out closure", async () => {
|
||||
const f = await acceptedResultFixture({ stopEvidence: false, finishedMinutesAgo: 11 });
|
||||
await db.update(budgetPolicies).set({ unpricedUsagePolicy: "block" }).where(eq(budgetPolicies.id, f.policy.id));
|
||||
expect(await reconcileRunCosts(db)).toMatchObject({ accounted: 1 });
|
||||
await flushBudgetEvaluationWork(db);
|
||||
expect((await f.block())?.reason).toMatch(/no reliable price|budget hard-stop/);
|
||||
await persistUsageReceipt(db, { schema: "paperclip/accounting-receipt/v1", id: randomUUID(),
|
||||
companyId: f.companyId, runId: f.run.id, adapterType: "paperclip_runner", sourceId: f.sourceId, sequence: 6,
|
||||
receivedAt: new Date().toISOString(),
|
||||
producerEvent: { sourceEventId: randomUUID(), eventType: "turn.completed" },
|
||||
receipt: { complete: true, usageBasis: "per_run", provider: "anthropic", biller: "anthropic",
|
||||
model: "claude-sonnet-5", billingType: "metered_api", costUsd: 0.25, costStatus: "reported",
|
||||
pricingProvenance: { source: "provider_reported", version: "test" },
|
||||
usage: { inputTokens: 1200, cachedInputTokens: 0, outputTokens: 80 } } });
|
||||
await accountRunCost(db, f.run.id);
|
||||
await budgetService(db).evaluateAffectedScopes(f.companyId, f.agentId, null);
|
||||
await flushBudgetEvaluationWork(db);
|
||||
expect((await f.readRun()).usageJson).toMatchObject({ accountingClosureTimedOut: false });
|
||||
expect(await f.block()).toBeNull();
|
||||
});
|
||||
|
||||
it("keeps the blocking policy for a timed-out closure, since the run may still spend", async () => {
|
||||
const f = await acceptedResultFixture({ stopEvidence: false, finishedMinutesAgo: 11 });
|
||||
await db.update(budgetPolicies).set({ unpricedUsagePolicy: "block" }).where(eq(budgetPolicies.id, f.policy.id));
|
||||
expect(await reconcileRunCosts(db)).toMatchObject({ accounted: 1 });
|
||||
await flushBudgetEvaluationWork(db);
|
||||
expect((await f.block())?.reason).toMatch(/no reliable price|budget hard-stop/);
|
||||
});
|
||||
|
||||
it("keeps the debt pending while native recovery can still produce a result", async () => {
|
||||
const f = await fixture({ phase: "observed" });
|
||||
expect(await reconcileRunCosts(db)).toMatchObject({ accounted: 0 });
|
||||
|
||||
Reference in new issue
Block a user