mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-29 04:25:52 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The cost ledger records usage and controls agent budgets. > - Some runs lose the selected model or API billing category before their usage reaches the ledger. > - The dashboard reports missing costs but does not identify the affected runs or the available actions. > - This pull request preserves usage attribution and adds a compact view of cost gaps grouped by agent. > - Operators can inspect runs, open agent settings, and record saved costs without starting new agent work. ## Linked Issues or Issue Description Refs #15634, #11938, #15879. This follows the merged subscription reporting in #15634. The older proposal in #11938 addresses part of ACP model attribution. This change also rejects conflicting models across a turn, fixes credential-backed billing capture, and explains cost gaps in the existing dashboard. The Pi receipt recovery work in #15879 covers a separate adapter recovery path. This PR preserves unresolved receipts and does not recover provider usage or reprice past entries. **What happened?** ACP results use the requested model even when the runtime selects a model. Native and OpenCode API runs can report unknown billing despite a matching API credential. The dashboard shows notices without links to the affected runs. These gaps make API and subscription totals harder to interpret. **Expected behavior** Record the model selected by the runtime. Identify API billing only from a matching credential. Keep costs unresolved when reliable usage or pricing is absent. Show the affected agents and runs. Offer an accounting retry only when saved evidence is ready. **Steps to reproduce** 1. Run an ACP agent with its model left unset, or use a matching API credential with a native or OpenCode agent. 2. Inspect the recorded model and billing category for the run. 3. Open Costs Overview for a period with unpriced usage or terminal runs awaiting receipts. 4. Inspect the notice and try to find the affected run or the relevant agent settings. **Paperclip version or commit** Current master at `657e71584a0`. The patch is rebased on that commit. **Deployment mode** Self-hosted server and local development. The cost detail tests use a disposable PostgreSQL database. ## What Changed - Preserve the runtime-selected ACP model in checkpoints and final results. Conflicting models across one turn remain unpriced. - Share credential-backed API billing checks across native and OpenCode results. Keep unknown billing when the provider and credential do not match. - Add company-scoped cost detail and accounting retry endpoints. Board viewers can read details but cannot retry. Agents and foreign companies are denied. - Limit each detail category to 25 entries. Use one consistent database snapshot and sparse indexes matching the filters and sort order. Add the unknown-billing count to the existing summary query. - Replace the notice with an expandable count line. Group entries alphabetically by agent. Show the saved blob icon, link each name to configuration, and put run and retry links beside the date. - Preserve estimate and unavailable-price labels on Providers and Billers; keep retry confirmation after the resolved notice disappears. - Fetch details only while expanded. Retain the last loaded details after a refresh error. Retry saved accounting only; do not rerun the agent. - Add capture, database, authorization, UI, and Storybook coverage. Update the implementation contract and cost reporting guide. ## Verification - `pnpm -r typecheck` passed after rebase. - `pnpm build` and `pnpm build-storybook` passed after rebase. - `pnpm check:token-gates` and `git diff --check` passed. - 1,086 focused tests passed across 17 files. These cover ACP, native and OpenCode capture, PostgreSQL reports, route authorization, OpenAPI, Costs UI, and existing accounting regressions. - Desktop and mobile browser checks passed during UI review. They cover lazy expansion, alphabetical groups, loaded blob icons, configuration and run links, keyboard retry, and layout overflow. - `pnpm test:run` was attempted with command-scoped `init.defaultBranch=master` for Git fixtures. It was stopped after a native session recovery failure reproduced on unchanged base `657e71584a0` (`archives a damaged prior epoch before completing guarded same-task replacement with real runnerd`). An MCP upload case failed in the broad run but passed in isolation on both the base and PR branch. These local limitations do not affect the 1,086 passing focused tests. - Current-head CI is green, including all unit test lanes, serialized server suites, runner checks, typecheck, build, policy, and eight browser shards. Apex re-review is 5/5 on `56de60effdb`; all four original findings are addressed and every review thread is resolved. - PostgreSQL applies migration `0333` in the integration fixtures. EXPLAIN ANALYZE on the production detail statements uses the new sparse indexes behind 5,000 healthy records in each table. - Ten provider/biller card cases cover missing and estimated prices; the retry test covers row removal and notice unmount. A live Storybook browser check verifies the confirmation and updated count. - Run `pnpm storybook`. Open **Costs / Subscriptions / Cost Data Quality Details** to inspect the production page with representative fixtures. ## Risks - Model and billing metadata change for future runs. Estimates still require complete usage and a supported price. No past cost amount is rewritten. - A missing final receipt remains unresolved for budget checks. The display changes after fifteen minutes; that time limit never authorizes settlement. - Accounting retry is a write action. It retains the existing company checks, receipt fences, idempotency, and audit logging. - Detail reads are bounded and run only after expansion. They poll every 30 seconds while open. No provider probe or new summary query is added. - Migration `0333` adds two partial indexes; it changes no stored cost amounts or table columns. Index creation scans the existing tables and may briefly block writes during deployment. No lockfile or workflow change is included. ## Model Used OpenAI Codex, based on GPT-6, with repository inspection, code editing, browser checks, and test execution. The exact serving model ID, reasoning setting, and context window were not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Paperclip <noreply@paperclip.ing>
138 lines
8.2 KiB
TypeScript
138 lines
8.2 KiB
TypeScript
import { useId, useState } from "react";
|
||
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
|
||
import { ChevronDown, ChevronRight } from "lucide-react";
|
||
import { Link } from "../lib/router";
|
||
import type { CostDataQualityItem, CostDataQualityReason } from "@paperclipai/shared";
|
||
import { costsApi } from "../api/costs";
|
||
import { formatDateTime } from "../lib/utils";
|
||
import { Button } from "./ui/button";
|
||
import { Badge } from "./ui/badge";
|
||
import { AgentAvatar } from "./AgentAvatar";
|
||
import { useToastActions } from "../context/ToastContext";
|
||
|
||
const explanations: Record<CostDataQualityReason, { label: string; description: string }> = {
|
||
unknown_billing: {
|
||
label: "Billing category unknown",
|
||
description: "This run did not identify API or subscription billing. Any recorded charges are included in the inference ledger. For future runs, select an AI connection in the agent’s configuration so Paperclip can identify its billing.",
|
||
},
|
||
missing_model: {
|
||
label: "Model not reported",
|
||
description: "Token usage was recorded without a model, so Paperclip cannot estimate its cost. Select an explicit model in the agent’s configuration for future runs. Changing settings will not reprice this past run.",
|
||
},
|
||
missing_price: {
|
||
label: "Cost unavailable",
|
||
description: "The provider did not supply a reliable price for this usage. There is no action required for this past entry. New API runs with complete usage and a supported model are estimated automatically.",
|
||
},
|
||
waiting_for_receipt: {
|
||
label: "Receiving usage",
|
||
description: "This run has stopped. Paperclip is waiting for its final usage data and will record it automatically. You do not need to rerun the agent.",
|
||
},
|
||
receipt_unavailable: {
|
||
label: "Final usage unavailable",
|
||
description: "This run stopped without final usage data. Its cost remains unresolved for budget checks. Paperclip will record a late receipt if one arrives; rerunning the agent will not recover this usage.",
|
||
},
|
||
retryable: {
|
||
label: "Ready to record",
|
||
description: "Saved usage data is ready to be recorded. Paperclip retries automatically, or you can retry now. This records costs only; it does not rerun the agent.",
|
||
},
|
||
};
|
||
|
||
export function CostDataQualityNotice({ companyId, from, to, reportKey, unpricedCount, pendingCount, unknownBillingCount }: {
|
||
companyId: string; from?: string; to?: string; reportKey: readonly unknown[];
|
||
unpricedCount: number; pendingCount: number; unknownBillingCount: number;
|
||
}) {
|
||
const [open, setOpen] = useState(false);
|
||
const unavailableCount = unpricedCount + pendingCount;
|
||
const hasCostGaps = unavailableCount > 0 || unknownBillingCount > 0;
|
||
const detailsId = useId();
|
||
const client = useQueryClient();
|
||
const { pushToast } = useToastActions();
|
||
const key = [...reportKey, "data-quality"];
|
||
const { data, error, isPending, refetch } = useQuery({
|
||
queryKey: key,
|
||
queryFn: () => costsApi.dataQuality(companyId, from, to),
|
||
enabled: open && hasCostGaps,
|
||
refetchInterval: open && hasCostGaps ? 30_000 : false,
|
||
});
|
||
const retry = useMutation({
|
||
mutationFn: (runId: string) => costsApi.retryAccounting(companyId, runId),
|
||
onSuccess: result => {
|
||
pushToast({
|
||
title: result.accounted ? "Costs recorded." : "No new cost data is available.",
|
||
body: "The agent was not rerun.",
|
||
tone: result.accounted ? "success" : "info",
|
||
});
|
||
client.invalidateQueries({ queryKey: ["costs", companyId] });
|
||
client.invalidateQueries({ queryKey: key });
|
||
client.invalidateQueries({ queryKey: ["cost-subscriptions", companyId] });
|
||
},
|
||
});
|
||
const label = unavailableCount > 0
|
||
? `${unavailableCount} run ${unavailableCount === 1 ? "cost" : "costs"} unavailable`
|
||
: `${unknownBillingCount} billing ${unknownBillingCount === 1 ? "type" : "types"} unknown`;
|
||
|
||
const groups = new Map<string | null, {
|
||
agent: { id?: string; name: string; appearance?: CostDataQualityItem["agentAppearance"] };
|
||
items: { item: CostDataQualityItem; source: "event" | "run" }[];
|
||
}>();
|
||
for (const [source, items] of [["event", data?.events ?? []], ["run", data?.runs ?? []]] as const) {
|
||
for (const item of items) {
|
||
let group = groups.get(item.agentId);
|
||
if (!group) {
|
||
group = { agent: { id: item.agentId ?? undefined, name: item.agentName ?? (item.agentId ? "Unknown agent" : "Paperclip services"), appearance: item.agentAppearance }, items: [] };
|
||
groups.set(item.agentId, group);
|
||
}
|
||
group.items.push({ item, source });
|
||
}
|
||
}
|
||
const sortedGroups = [...groups.values()].sort((a, b) =>
|
||
a.agent.name.localeCompare(b.agent.name, "en", { sensitivity: "base", numeric: true })
|
||
|| (a.agent.id ?? "").localeCompare(b.agent.id ?? ""));
|
||
for (const group of sortedGroups) group.items.sort((a, b) =>
|
||
Date.parse(b.item.occurredAt) - Date.parse(a.item.occurredAt)
|
||
|| `${a.source}:${a.item.id}`.localeCompare(`${b.source}:${b.item.id}`));
|
||
|
||
function itemRow(item: CostDataQualityItem, source: "event" | "run") {
|
||
const explanation = explanations[item.reason];
|
||
return <li key={`${source}:${item.id}`} className="space-y-2 border-b border-border py-3 last:border-0">
|
||
<div className="flex flex-wrap items-center gap-2">
|
||
<Badge variant="outline" className="font-normal text-muted-foreground">{explanation.label}</Badge>
|
||
<div className="flex flex-wrap items-center gap-2 text-xs">
|
||
<time dateTime={item.occurredAt}>{formatDateTime(item.occurredAt)}</time>
|
||
{item.runId && item.agentId && <Link className="text-primary underline underline-offset-4 hover:text-primary/80" to={`/agents/${item.agentId}/runs/${item.runId}`}>View run</Link>}
|
||
{item.reason === "retryable" && item.runId && data?.canRetry && <Button type="button" variant="link" size="sm" className="h-auto p-0 text-xs font-normal underline hover:text-primary/80" disabled={retry.isPending}
|
||
onClick={() => { retry.reset(); retry.mutate(item.runId!); }}>Retry cost recording</Button>}
|
||
</div>
|
||
</div>
|
||
<p>{explanation.description}</p>
|
||
{retry.variables === item.runId && retry.isPending && <p role="status">Recording saved costs…</p>}
|
||
{retry.variables === item.runId && retry.error && <p role="status">Cost recording failed. Try again shortly.</p>}
|
||
</li>;
|
||
}
|
||
|
||
return <div className="space-y-2 text-sm text-muted-foreground">
|
||
<Button type="button" variant="ghost" size="sm" className="h-auto justify-start gap-1 px-0 py-1 font-normal has-[>svg]:px-0" aria-expanded={open} aria-controls={detailsId} onClick={() => setOpen(value => !value)}>
|
||
{open ? <ChevronDown className="size-3" aria-hidden="true" /> : <ChevronRight className="size-3" aria-hidden="true" />}
|
||
{label}
|
||
</Button>
|
||
{open && <div id={detailsId} className="space-y-2">
|
||
{hasCostGaps && isPending && <p role="status">Loading affected runs…</p>}
|
||
{hasCostGaps && error && !data && <div className="flex flex-wrap items-center gap-2"><p role="status">Cost details could not be loaded.</p><Button variant="outline" size="sm" onClick={() => refetch()}>Retry loading</Button></div>}
|
||
{hasCostGaps && data && <>
|
||
{error && <p role="status">Showing the last loaded cost details. Updates will resume automatically.</p>}
|
||
{data.events.length === 0 && data.runs.length === 0 && <p role="status">No cost gaps remain in this period.</p>}
|
||
<ul aria-label="Cost details by agent" className="space-y-4">
|
||
{sortedGroups.map(group => <li key={group.agent.id ?? "services"}>
|
||
<h3 className="flex items-center gap-2 font-medium text-foreground">
|
||
<AgentAvatar agent={group.agent} size={20} />
|
||
{group.agent.id ? <Link className="text-primary underline underline-offset-4 hover:text-primary/80" to={`/agents/${group.agent.id}/configuration`}>{group.agent.name}</Link> : group.agent.name}
|
||
</h3>
|
||
<ul aria-label={`${group.agent.name} cost details`}>{group.items.map(({ item, source }) => itemRow(item, source))}</ul>
|
||
</li>)}
|
||
</ul>
|
||
{(data.hasMoreEvents || data.hasMoreRuns) && <p className="text-xs">Showing the 25 most recent entries in each category. Select a shorter date range to inspect more.</p>}
|
||
</>}
|
||
</div>}
|
||
</div>;
|
||
}
|