mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-24 01:41:42 +02:00
DottaandPaperclip
a3e1d915a7
## Thinking Path > - Paperclip manages AI agents and their work. > - Direct Runner evals report usage and costs for each attempt. > - Codex App Server supplies token counts without a provider USD receipt. > - The legacy Runner ledger fills an absent cost with zero. > - The eval result exposed that ledger value as provider-reported cost. > - This fix keeps provider cost unknown and preserves the estimate and budget stop. ## Linked Issues or Issue Description Refs #17009 and #17154. This is a separate direct Runner accounting fix based on master. ## What Changed - Codex direct eval usage now reports null provider USD cost with unavailable provenance. Token counts and the pinned price estimate remain available. - Preserve the original ledger amount as optional `legacyLedgerCostNanodollars`. Keep the original snapshot receipts unchanged. - Apply the existing budget ceiling to the retained ledger amount as well as the token estimate. Correcting a label does not relax admission or settlement checks. - Use the unique operator UUID for Muse test company prefixes, eliminating collisions from the previous 24-bit random suffix. - Document the cost distinction in the Runner/Evals integration contract. Other provider behavior and historical artifacts remain unchanged. ## Verification - Reproduced both zero and nonzero legacy amounts being labelled as provider USD cost before the fix. - The current eval-session contract and entrypoint tests pass (39 tests). The new regression uses the bounded usage consumer and actual campaign cost projection. It checks unknown provider cost, the separate estimate, unchanged raw evidence and the legacy ledger budget stop. - All 15 Runner report, campaign and metrics Node tests pass. Four adjacent CLI entrypoint/runtime tests also pass. - Runner TypeScript validation passes with its standard dependency-build prerequisite; `git diff --check` passes. Both Runner TypeScript configurations also passed during the original accounting verification. - Hosted CI run [39960367674](https://github.com/paperclipai/paperclip/actions/runs/39960367674) passes on `7d84f138044bac58cdced6baa2d0ec3d63767bb8`, including typecheck, build, server tests and browser checks. Greptile and Paperclip Review both report 5/5 on that commit with no actionable findings. - The previous CI failure was a PostgreSQL unique-index collision on `companies_issue_prefix_idx` during Muse fixture setup. Local Muse verification could not reach test execution because embedded PostgreSQL timed out during macOS startup (initially reporting a missing zstd library). The corrected fixture passed in hosted Linux CI. - Verification used no model calls. Full repository checks were performed by hosted CI; local verification was scoped to the changed contracts and fixture. ## Risks Consumers that assumed Codex always had a numeric provider cost must handle null. The contract already permits null for other profiles, and the report consumer retains missing-cost coverage. The legacy amount remains available as unverified evidence and still triggers the existing budget stop. Neither the estimate nor that ledger amount proves an invoice charge. This change does not rewrite old artifacts or qualify other provider routes. ## Model Used OpenAI GPT-6 through Codex, with repository tools and code execution. The exact deployment variant and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>