Files
PaperClipAI/docs/guides/agent-developer/cost-reporting.md
1b4c803816 fix(costs): preserve usage attribution and explain unavailable run costs (#15948)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The cost ledger records usage and controls agent budgets.
> - Some runs lose the selected model or API billing category before
their usage reaches the ledger.
> - The dashboard reports missing costs but does not identify the
affected runs or the available actions.
> - This pull request preserves usage attribution and adds a compact
view of cost gaps grouped by agent.
> - Operators can inspect runs, open agent settings, and record saved
costs without starting new agent work.

## Linked Issues or Issue Description

Refs #15634, #11938, #15879.

This follows the merged subscription reporting in #15634. The older
proposal in #11938 addresses part of ACP model attribution. This change
also rejects conflicting models across a turn, fixes credential-backed
billing capture, and explains cost gaps in the existing dashboard. The
Pi receipt recovery work in #15879 covers a separate adapter recovery
path. This PR preserves unresolved receipts and does not recover
provider usage or reprice past entries.

**What happened?**

ACP results use the requested model even when the runtime selects a
model. Native and OpenCode API runs can report unknown billing despite a
matching API credential. The dashboard shows notices without links to
the affected runs. These gaps make API and subscription totals harder to
interpret.

**Expected behavior**

Record the model selected by the runtime. Identify API billing only from
a matching credential. Keep costs unresolved when reliable usage or
pricing is absent. Show the affected agents and runs. Offer an
accounting retry only when saved evidence is ready.

**Steps to reproduce**

1. Run an ACP agent with its model left unset, or use a matching API
credential with a native or OpenCode agent.
2. Inspect the recorded model and billing category for the run.
3. Open Costs Overview for a period with unpriced usage or terminal runs
awaiting receipts.
4. Inspect the notice and try to find the affected run or the relevant
agent settings.

**Paperclip version or commit**

Current master at `657e71584a0`. The patch is rebased on that commit.

**Deployment mode**

Self-hosted server and local development. The cost detail tests use a
disposable PostgreSQL database.

## What Changed

- Preserve the runtime-selected ACP model in checkpoints and final
results. Conflicting models across one turn remain unpriced.
- Share credential-backed API billing checks across native and OpenCode
results. Keep unknown billing when the provider and credential do not
match.
- Add company-scoped cost detail and accounting retry endpoints. Board
viewers can read details but cannot retry. Agents and foreign companies
are denied.
- Limit each detail category to 25 entries. Use one consistent database
snapshot and sparse indexes matching the filters and sort order. Add the
unknown-billing count to the existing summary query.
- Replace the notice with an expandable count line. Group entries
alphabetically by agent. Show the saved blob icon, link each name to
configuration, and put run and retry links beside the date.
- Preserve estimate and unavailable-price labels on Providers and
Billers; keep retry confirmation after the resolved notice disappears.
- Fetch details only while expanded. Retain the last loaded details
after a refresh error. Retry saved accounting only; do not rerun the
agent.
- Add capture, database, authorization, UI, and Storybook coverage.
Update the implementation contract and cost reporting guide.

## Verification

- `pnpm -r typecheck` passed after rebase.
- `pnpm build` and `pnpm build-storybook` passed after rebase.
- `pnpm check:token-gates` and `git diff --check` passed.
- 1,086 focused tests passed across 17 files. These cover ACP, native
and OpenCode capture, PostgreSQL reports, route authorization, OpenAPI,
Costs UI, and existing accounting regressions.
- Desktop and mobile browser checks passed during UI review. They cover
lazy expansion, alphabetical groups, loaded blob icons, configuration
and run links, keyboard retry, and layout overflow.
- `pnpm test:run` was attempted with command-scoped
`init.defaultBranch=master` for Git fixtures. It was stopped after a
native session recovery failure reproduced on unchanged base
`657e71584a0` (`archives a damaged prior epoch before completing guarded
same-task replacement with real runnerd`). An MCP upload case failed in
the broad run but passed in isolation on both the base and PR branch.
These local limitations do not affect the 1,086 passing focused tests.
- Current-head CI is green, including all unit test lanes, serialized
server suites, runner checks, typecheck, build, policy, and eight
browser shards. Apex re-review is 5/5 on `56de60effdb`; all four
original findings are addressed and every review thread is resolved.
- PostgreSQL applies migration `0333` in the integration fixtures.
EXPLAIN ANALYZE on the production detail statements uses the new sparse
indexes behind 5,000 healthy records in each table.
- Ten provider/biller card cases cover missing and estimated prices; the
retry test covers row removal and notice unmount. A live Storybook
browser check verifies the confirmation and updated count.
- Run `pnpm storybook`. Open **Costs / Subscriptions / Cost Data Quality
Details** to inspect the production page with representative fixtures.

## Risks

- Model and billing metadata change for future runs. Estimates still
require complete usage and a supported price. No past cost amount is
rewritten.
- A missing final receipt remains unresolved for budget checks. The
display changes after fifteen minutes; that time limit never authorizes
settlement.
- Accounting retry is a write action. It retains the existing company
checks, receipt fences, idempotency, and audit logging.
- Detail reads are bounded and run only after expansion. They poll every
30 seconds while open. No provider probe or new summary query is added.
- Migration `0333` adds two partial indexes; it changes no stored cost
amounts or table columns. Index creation scans the existing tables and
may briefly block writes during deployment. No lockfile or workflow
change is included.

## Model Used

OpenAI Codex, based on GPT-6, with repository inspection, code editing,
browser checks, and test execution. The exact serving model ID,
reasoning setting, and context window were not exposed to this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-11 22:48:29 -07:00

98 lines
4.3 KiB
Markdown

---
title: Cost Reporting
summary: How agents report token costs
---
Agents report their token usage and costs back to Paperclip so the system can track spending and enforce budgets.
## How It Works
Cost reporting happens automatically through adapters. When an agent heartbeat completes, the adapter parses the agent's output to extract:
- **Provider** — which LLM provider was used (e.g. "anthropic", "openai")
- **Model** — which model was used (e.g. "claude-sonnet-4-20250514")
- **Input tokens** — tokens sent to the model
- **Output tokens** — tokens generated by the model
- **Cost** — dollar cost of the invocation (if available from the runtime)
The server records this as a cost event for budget tracking.
Paperclip keeps token usage even when the provider does not report a price. A
complete receipt for a supported direct API model can be priced using a recorded
list-price estimate; estimates are labeled in the Costs dashboard. Missing costs
are never treated as proof of free usage.
## Understanding incomplete cost data
The Costs overview shows a compact line such as **12 run costs unavailable** when
usage is missing a price or final receipt. Expand it with the arrow to see the
affected entries, grouped alphabetically by agent with links to
each run. Unknown billing types use their own label rather than counting as
unavailable costs. Estimates remain labeled on spend cards and tables.
Select an agent's name to open its configuration.
Provider and biller cards also label estimates and unavailable prices. A total
with only unpriced entries shows a dash instead of $0.00. Successful recording
shows a confirmation even when it resolves the last gap.
- **Billing category unknown:** select an AI connection in the agent's
configuration for future runs. Recorded charges remain in the inference ledger.
- **Model not reported:** select an explicit model for future runs. ACP adapters
also retain the provider-selected model when the runtime reports it, including
its default model. A model change during a turn is not priced as one model.
- **Cost unavailable:** the past entry has no reliable price. No cleanup is
required; totals include known charges only.
- **Receiving usage / Final usage unavailable:** the final provider receipt has
not arrived. The label changes after fifteen minutes, but this is a display
distinction only. Paperclip still accepts late receipts and preserves the
accounting debt. Rerunning an agent does not recover its previous receipt.
- **Ready to record:** Paperclip can retry saved accounting data. **Retry cost
recording** records costs only and never starts agent work. Recording is
idempotent, and company viewers cannot request this action.
Changing an agent's configuration does not reprice historical entries. Normal
credits and fixed billing are listed as other billing usage, without an unknown
billing warning. OpenCode API billing can be identified from a nonempty API key
for its selected provider; the provider name alone is insufficient evidence.
Remote OpenCode runs use only keys forwarded to the remote process. Keys present
on the Paperclip server do not establish a remote run's billing category.
Board cost readers can inspect bounded details at
`GET /api/companies/{companyId}/costs/data-quality`, with the same date bounds as
other cost reports. Details contain no provider errors or credentials and load
only when expanded in the UI. Authorized board members can request recording at
`POST /api/companies/{companyId}/costs/retry-accounting` with `{ "runId": "uuid" }`.
## Cost Events API
Cost events can also be reported directly:
```
POST /api/companies/{companyId}/cost-events
{
"agentId": "{agentId}",
"provider": "anthropic",
"model": "claude-sonnet-4-20250514",
"inputTokens": 15000,
"outputTokens": 3000,
"costCents": 12
}
```
## Budget Awareness
Agents should check their budget at the start of each heartbeat:
```
GET /api/agents/me
# Check: spentMonthlyCents vs budgetMonthlyCents
```
If budget utilization is above 80%, focus on critical tasks only. At 100%, the agent is auto-paused.
## Best Practices
- Let the adapter handle cost reporting — don't duplicate it
- Check budget early in the heartbeat to avoid wasted work
- Above 80% utilization, skip low-priority tasks
- If you're running out of budget mid-task, leave a comment and exit gracefully