mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-24 01:41:42 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its Costs page reports API charges and subscription token usage. > - Subscription runs do not show the recurring fee paid for the account. > - Several agents can share one paid seat, so a fee per agent would overstate cost. > - This pull request identifies connected subscription accounts and reports each monthly fee once. > - Users can compare API spend and subscription costs, see uncertainty, and enter their actual price. ## Linked Issues or Issue Description Refs #498 and #3406 for related subscription reporting needs. This change does not add subscription budget enforcement or automatic optimization. Related prior approaches: #10544 and #11104 proposed API-equivalent token costs for subscription work. This PR reports recurring account fees separately from usage charges and budgets. A search found no open PR for monthly account fee reporting. **Subsystem affected** Costs, managed AI connections, shared contracts, database schema, and board UI. **Problem or motivation** An organization can pay for subscriptions even when included runs record no additional dollar charge. The Costs page cannot show that monthly commitment or distinguish a shared paid seat from several separate seats. **Proposed solution** Discover account identity from the managed connection selected for a run. Use exact known plans for labeled monthly list-price estimates. Keep ambiguous plans and unconfirmed identities unpriced until the user supplies a price. Sum current monthly fees separately from date-filtered API charges and token usage. **Alternatives considered** A fee per agent double-counts shared accounts. Token-based API price estimates do not measure the recurring subscription fee. Provider invoices would be more exact but are not available through these account credentials. **Roadmap alignment** Extends the shipped Better Budgeting and Personal & Shared AI Accounts capabilities. The feature and UI were reviewed with the maintainer before this PR. ## What Changed - Add company-scoped subscription accounts, connection bindings, and immutable price revisions in replay-safe migration 0322. - Identify OpenAI workspace/seat and Claude organization/account pairs only after provider authentication. Keep provisional records grant-scoped; unchecked ID-token claims cannot combine accounts or grant billing editors. Keep credentials and raw provider identities out of reports. - Attach the selected subscription to new run receipts. Resolve duplicate aliases without rewriting old monetary receipts. - Add cached background discovery with four provider checks and twenty discovery requests at a time. Accepted requests wait for capacity; excess requests receive HTTP 429 and a UI retry action. Preserve saved prices and observations when a provider check fails. Dashboard polls load current prices only; audit revisions stay in the database. - Support owner-controlled monthly or annual prices, fractional-cent precision, separate currency totals, tracking status, and duplicate linking. Preserve both personal owners and shared-account managers when discovery finds the same paid seat. - Show API spend beside monthly subscription commitment, with separate token totals and clear estimate labels. Account details and fee totals follow billing-owner/shared-audience visibility; company-wide token totals remain available to cost readers. - Use consistent amount and estimate badge formatting for agent, user, project, and model rows. - Add real PostgreSQL tests, route authorization tests, UI tests, interactive Storybook fixtures, and operator documentation. ## Verification - Current head `d7b5c78dadf5033a3c447d83434676ac94687e77` passed all 53 checks, with two checks skipped. This includes the full test matrix, browser tests, build, typecheck, migration policy, canary packaging, and security checks. - Greptile Apex reviewed that exact head at 5/5 with no unresolved findings. - Local `pnpm -r typecheck`, `pnpm build`, `pnpm check:token-gates`, migration safety checks, and `git diff --check` passed. - All 238 focused tests passed across accounting, subscription persistence, authorization, UI, migration replay, and snapshot drift. - The expanded local chat suite passed 1,149 tests. Two cases failed because this workspace found an unrelated ancestor skill directory; both passed unchanged in an isolated checkout. The full local Vitest command was not repeated after the rebase because of this environment issue. CI covered the complete test matrix on the final head. - Migration `0322_reflective_kree.sql` follows master's `0321_acoustic_human_fly.sql`. Its snapshot and journal were regenerated. Master's migration and snapshot are unchanged. The subscription SQL retains its original content and hash. - Disposable PostgreSQL upgrade fixtures applied master's added lifecycle and Slack migrations without changing saved subscription prices or cost receipts. The renamed subscription migration was recognized by hash, retained one history row, and a second migration run was a no-op. - Regression tests cover provider capacity, fractional-cent edits, billing visibility after grant revocation, and fixed-price/credit/unknown usage. The new UI billing-label tests fail against the old wording and pass with the fix. - Browser checks passed for desktop and mobile layouts, shared-account totals, annual price edits, unknown prices, and stale provider data. - A live OpenAI account probe returned HTTP 200 and a verified Pro account identity. The generic Pro plan correctly remains unpriced. - Live Claude discovery identified Max 20x and a $200/month estimate. Two real Claude agent runs were attributed to that same account, with two usage receipts, two agents, one monthly fee, and zero additional usage charges. Two OpenAI connections also deduplicated to one account. - Live Codex execution is not verified: both Paperclip's managed hello test and a direct host `codex exec` fail with a workspace-routing authentication error. The current credential passes the usage API probe. This is separate from the successful account-discovery check. - A local performance fixture with 50,000 usage events and 5,001 saved price revisions produced a 1,589-byte report in 11.5 ms median / 12.2 ms maximum over ten warmed runs. PostgreSQL used the subscription/revision index to read one current price row. These are local measurements, not a production latency guarantee. - [Desktop, account details, and mobile screenshots](https://pages.paperclip.ing/pr-15634-subscription-costs-d5d285e2/) show the production Costs page with synthetic Storybook fixtures. - To inspect the UI, run `pnpm storybook` and open **Costs / Subscriptions**. Edits there change fixture data only. ## Risks - Provider plan identifiers can change. Unknown plans remain unpriced. Automatic prices are estimates before tax; discounts and negotiated prices require a user-supplied amount. - Subscription fees describe the current monthly commitment. They do not follow the selected usage date range and do not consume agent budgets or create finance events. - Monthly fees cover only accounts visible to the viewer; company-wide token totals retain their existing scope. - Legacy or unmanaged usage can remain unattributed. Unconfirmed identities require manual prices and may need explicit duplicate linking after credential changes. - Disconnecting a connection does not prove provider billing stopped. Users must end or exclude tracking explicitly. - Migration 0322 adds tables, a nullable cost-event column, and an index. It is replay-safe, but normal migration locks still apply. ## Model Used OpenAI Codex, based on GPT-6. This session did not expose a more specific backend model ID or context-window size. Capabilities used: code editing, shell execution, database tests, and browser verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
954 lines
29 KiB
TypeScript
954 lines
29 KiB
TypeScript
import express from "express";
|
||
import request from "supertest";
|
||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||
import { afterAll, afterEach, beforeAll } from "vitest";
|
||
import { randomUUID } from "node:crypto";
|
||
import { eq } from "drizzle-orm";
|
||
import {
|
||
createDb,
|
||
companies,
|
||
agents,
|
||
activityLog,
|
||
costEvents,
|
||
financeEvents,
|
||
heartbeatRuns,
|
||
issues,
|
||
projects,
|
||
} from "@paperclipai/db";
|
||
import { costService } from "../services/costs.ts";
|
||
import { financeService } from "../services/finance.ts";
|
||
import {
|
||
getEmbeddedPostgresTestSupport,
|
||
startEmbeddedPostgresTestDatabase,
|
||
} from "./helpers/embedded-postgres.js";
|
||
import { hoistModuleGraph } from "./helpers/hoist-module-graph.js";
|
||
|
||
function makeDb(overrides: Record<string, unknown> = {}) {
|
||
const selectChain = {
|
||
from: vi.fn().mockReturnThis(),
|
||
where: vi.fn().mockReturnThis(),
|
||
leftJoin: vi.fn().mockReturnThis(),
|
||
innerJoin: vi.fn().mockReturnThis(),
|
||
groupBy: vi.fn().mockReturnThis(),
|
||
orderBy: vi.fn().mockReturnThis(),
|
||
limit: vi.fn().mockReturnThis(),
|
||
then: vi.fn().mockResolvedValue([]),
|
||
};
|
||
|
||
const thenableChain = Object.assign(Promise.resolve([]), selectChain);
|
||
|
||
return {
|
||
select: vi.fn().mockReturnValue(thenableChain),
|
||
insert: vi.fn().mockReturnValue({
|
||
values: vi.fn().mockReturnValue({ returning: vi.fn().mockResolvedValue([]) }),
|
||
}),
|
||
update: vi.fn().mockReturnValue({
|
||
set: vi.fn().mockReturnValue({ where: vi.fn().mockResolvedValue([]) }),
|
||
}),
|
||
...overrides,
|
||
};
|
||
}
|
||
|
||
const mockCompanyService = vi.hoisted(() => ({
|
||
getById: vi.fn(),
|
||
update: vi.fn(),
|
||
}));
|
||
const mockAgentService = vi.hoisted(() => ({
|
||
getById: vi.fn(),
|
||
update: vi.fn(),
|
||
}));
|
||
const mockIssueService = vi.hoisted(() => ({
|
||
getById: vi.fn(),
|
||
getByIdentifier: vi.fn(),
|
||
}));
|
||
const mockHeartbeatService = vi.hoisted(() => ({
|
||
cancelBudgetScopeWork: vi.fn().mockResolvedValue(undefined),
|
||
}));
|
||
const mockLogActivity = vi.hoisted(() => vi.fn());
|
||
const mockFetchAllQuotaWindows = vi.hoisted(() => vi.fn());
|
||
const mockCostService = vi.hoisted(() => ({
|
||
createEvent: vi.fn(),
|
||
summary: vi.fn().mockResolvedValue({ spendCents: 0 }),
|
||
byAgent: vi.fn().mockResolvedValue([]),
|
||
byAgentModel: vi.fn().mockResolvedValue([]),
|
||
byProvider: vi.fn().mockResolvedValue([]),
|
||
byBiller: vi.fn().mockResolvedValue([]),
|
||
issueTreeSummary: vi.fn().mockResolvedValue({
|
||
issueId: "issue-1",
|
||
issueCount: 1,
|
||
includeDescendants: true,
|
||
costCents: 0,
|
||
inputTokens: 0,
|
||
cachedInputTokens: 0,
|
||
outputTokens: 0,
|
||
runCount: 0,
|
||
runtimeMs: 0,
|
||
}),
|
||
windowSpend: vi.fn().mockResolvedValue([]),
|
||
byProject: vi.fn().mockResolvedValue([]),
|
||
}));
|
||
const mockFinanceService = vi.hoisted(() => ({
|
||
createEvent: vi.fn(),
|
||
summary: vi.fn().mockResolvedValue({ debitCents: 0, creditCents: 0, netCents: 0, estimatedDebitCents: 0, eventCount: 0 }),
|
||
byBiller: vi.fn().mockResolvedValue([]),
|
||
byKind: vi.fn().mockResolvedValue([]),
|
||
list: vi.fn().mockResolvedValue([]),
|
||
}));
|
||
const mockBudgetService = vi.hoisted(() => ({
|
||
overview: vi.fn().mockResolvedValue({
|
||
companyId: "company-1",
|
||
policies: [],
|
||
activeIncidents: [],
|
||
pausedAgentCount: 0,
|
||
pausedProjectCount: 0,
|
||
pendingApprovalCount: 0,
|
||
}),
|
||
upsertPolicy: vi.fn(),
|
||
resolveIncident: vi.fn(),
|
||
}));
|
||
const mockAccessService = vi.hoisted(() => ({
|
||
decide: vi.fn(),
|
||
}));
|
||
|
||
function registerModuleMocks() {
|
||
vi.doMock("../services/index.js", () => ({
|
||
accessService: () => mockAccessService,
|
||
budgetService: () => mockBudgetService,
|
||
costService: () => mockCostService,
|
||
financeService: () => mockFinanceService,
|
||
companyService: () => mockCompanyService,
|
||
agentService: () => mockAgentService,
|
||
issueService: () => mockIssueService,
|
||
heartbeatService: () => mockHeartbeatService,
|
||
logActivity: mockLogActivity,
|
||
}));
|
||
|
||
vi.doMock("../services/quota-windows.js", () => ({
|
||
fetchAllQuotaWindows: mockFetchAllQuotaWindows,
|
||
}));
|
||
}
|
||
|
||
describe("cost routes", () => {
|
||
const routeModules = hoistModuleGraph(registerModuleMocks, async () => {
|
||
const [costsRouteModule, middlewareModule] = await Promise.all([
|
||
vi.importActual<typeof import("../routes/costs.js")>("../routes/costs.js"),
|
||
vi.importActual<typeof import("../middleware/index.js")>("../middleware/index.js"),
|
||
]);
|
||
return { ...costsRouteModule, errorHandler: middlewareModule.errorHandler };
|
||
});
|
||
|
||
function createApp() {
|
||
const { costRoutes, errorHandler } = routeModules.value;
|
||
const app = express();
|
||
app.use(express.json());
|
||
app.use((req, _res, next) => {
|
||
req.actor = { type: "board", userId: "board-user", source: "local_implicit" };
|
||
next();
|
||
});
|
||
app.use("/api", costRoutes(makeDb() as any));
|
||
app.use(errorHandler);
|
||
return app;
|
||
}
|
||
|
||
function createAppWithActor(actor: any) {
|
||
const { costRoutes, errorHandler } = routeModules.value;
|
||
const app = express();
|
||
app.use(express.json());
|
||
app.use((req, _res, next) => {
|
||
req.actor = actor;
|
||
next();
|
||
});
|
||
app.use("/api", costRoutes(makeDb() as any));
|
||
app.use(errorHandler);
|
||
return app;
|
||
}
|
||
|
||
function loadCostParsers() {
|
||
const { parseCostDateRange, parseCostLimit } = routeModules.value;
|
||
return { parseCostDateRange, parseCostLimit };
|
||
}
|
||
|
||
beforeEach(() => {
|
||
vi.clearAllMocks();
|
||
mockAccessService.decide.mockReset();
|
||
mockAccessService.decide.mockResolvedValue({
|
||
allowed: true,
|
||
action: "company_scope:read",
|
||
reason: "allow_test",
|
||
explanation: "Allowed by test mock.",
|
||
});
|
||
mockCompanyService.update.mockResolvedValue({
|
||
id: "company-1",
|
||
name: "Paperclip",
|
||
budgetMonthlyCents: 100,
|
||
spentMonthlyCents: 0,
|
||
});
|
||
mockAgentService.getById.mockResolvedValue({
|
||
id: "agent-1",
|
||
companyId: "company-1",
|
||
name: "Budget Agent",
|
||
budgetMonthlyCents: 100,
|
||
spentMonthlyCents: 0,
|
||
});
|
||
mockAgentService.update.mockResolvedValue({
|
||
id: "agent-1",
|
||
companyId: "company-1",
|
||
name: "Budget Agent",
|
||
budgetMonthlyCents: 100,
|
||
spentMonthlyCents: 0,
|
||
});
|
||
mockIssueService.getById.mockResolvedValue({
|
||
id: "issue-1",
|
||
companyId: "company-1",
|
||
identifier: "PC1A2-1",
|
||
});
|
||
mockIssueService.getByIdentifier.mockResolvedValue({
|
||
id: "issue-1",
|
||
companyId: "company-1",
|
||
identifier: "PC1A2-1",
|
||
});
|
||
mockBudgetService.upsertPolicy.mockResolvedValue(undefined);
|
||
});
|
||
|
||
it("accepts valid ISO date strings", async () => {
|
||
const { parseCostDateRange } = loadCostParsers();
|
||
expect(parseCostDateRange({
|
||
from: "2026-01-01T00:00:00.000Z",
|
||
to: "2026-01-31T23:59:59.999Z",
|
||
})).toEqual({
|
||
from: new Date("2026-01-01T00:00:00.000Z"),
|
||
to: new Date("2026-01-31T23:59:59.999Z"),
|
||
});
|
||
});
|
||
|
||
it("requires an explicit all-time request and rejects conflicting ranges", () => {
|
||
const { parseCostDateRange } = loadCostParsers();
|
||
expect(parseCostDateRange({})).toBeUndefined();
|
||
expect(parseCostDateRange({ period: "all" })).toEqual({ allTime: true });
|
||
for (const input of [
|
||
{ from: "2026-09-02", to: "2026-09-01" },
|
||
{ from: ["2026-09-01"] },
|
||
{ from: "2026-02-30" },
|
||
{ from: "" },
|
||
{ from: "09/01/2026" },
|
||
{ period: "all", from: "2026-09-01" },
|
||
{ period: "unknown" },
|
||
]) expect(() => parseCostDateRange(input)).toThrow();
|
||
});
|
||
|
||
it("returns 400 for an invalid 'from' date string", async () => {
|
||
const { parseCostDateRange } = loadCostParsers();
|
||
expect(() => parseCostDateRange({ from: "not-a-date" })).toThrow(/invalid 'from' date/i);
|
||
});
|
||
|
||
it("returns 400 for an invalid 'to' date string", async () => {
|
||
const { parseCostDateRange } = loadCostParsers();
|
||
expect(() => parseCostDateRange({ to: "banana" })).toThrow(/invalid 'to' date/i);
|
||
});
|
||
|
||
it("returns finance summary rows for valid requests", async () => {
|
||
const app = createApp();
|
||
const res = await request(app)
|
||
.get("/api/companies/company-1/costs/finance-summary")
|
||
.query({ from: "2026-02-01T00:00:00.000Z", to: "2026-02-28T23:59:59.999Z" });
|
||
expect(res.status).toBe(200);
|
||
expect(res.body).toEqual({
|
||
debitCents: 0,
|
||
creditCents: 0,
|
||
netCents: 0,
|
||
estimatedDebitCents: 0,
|
||
eventCount: 0,
|
||
});
|
||
});
|
||
|
||
it("returns issue subtree cost summaries for issue refs", async () => {
|
||
const app = createApp();
|
||
const res = await request(app).get("/api/issues/pc1a2-1/cost-summary");
|
||
|
||
expect(res.status).toBe(200);
|
||
expect(mockIssueService.getByIdentifier).toHaveBeenCalledWith("PC1A2-1");
|
||
expect(mockCostService.issueTreeSummary).toHaveBeenCalledWith("company-1", "issue-1", {
|
||
excludeRoot: false,
|
||
});
|
||
expect(res.body).toEqual({
|
||
issueId: "issue-1",
|
||
issueCount: 1,
|
||
includeDescendants: true,
|
||
costCents: 0,
|
||
inputTokens: 0,
|
||
cachedInputTokens: 0,
|
||
outputTokens: 0,
|
||
runCount: 0,
|
||
runtimeMs: 0,
|
||
});
|
||
});
|
||
|
||
it("returns 400 for invalid finance event list limits", async () => {
|
||
const { parseCostLimit } = loadCostParsers();
|
||
for (const limit of ["0", "25garbage", "1.5", ["25"], "501"]) expect(() => parseCostLimit({ limit })).toThrow(/invalid 'limit'/i);
|
||
});
|
||
|
||
it("accepts valid finance event list limits", async () => {
|
||
const { parseCostLimit } = loadCostParsers();
|
||
expect(parseCostLimit({ limit: "25" })).toBe(25);
|
||
});
|
||
|
||
it("rejects company budget updates for board users outside the company", async () => {
|
||
const app = createAppWithActor({
|
||
type: "board",
|
||
userId: "board-user",
|
||
source: "session",
|
||
isInstanceAdmin: false,
|
||
companyIds: ["company-2"],
|
||
});
|
||
|
||
const res = await request(app)
|
||
.patch("/api/companies/company-1/budgets")
|
||
.send({ budgetMonthlyCents: 2500 });
|
||
|
||
expect(res.status).toBe(403);
|
||
expect(mockCompanyService.update).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("rejects agent budget updates for board users outside the agent company", async () => {
|
||
const app = createAppWithActor({
|
||
type: "board",
|
||
userId: "board-user",
|
||
source: "session",
|
||
isInstanceAdmin: false,
|
||
companyIds: ["company-2"],
|
||
});
|
||
|
||
const res = await request(app)
|
||
.patch("/api/agents/agent-1/budgets")
|
||
.send({ budgetMonthlyCents: 2500 });
|
||
|
||
expect(res.status).toBe(404);
|
||
expect(res.body.error).toBe("Agent not found");
|
||
expect(mockAgentService.update).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("rejects agent budget updates from the target agent without changing the budget policy", async () => {
|
||
const app = createAppWithActor({
|
||
type: "agent",
|
||
agentId: "agent-1",
|
||
companyId: "company-1",
|
||
runId: "run-1",
|
||
});
|
||
|
||
const res = await request(app)
|
||
.patch("/api/agents/agent-1/budgets")
|
||
.send({ budgetMonthlyCents: 2500 });
|
||
|
||
expect(res.status).toBe(403);
|
||
expect(res.body).toEqual({ error: "Board access required" });
|
||
expect(mockAgentService.update).not.toHaveBeenCalled();
|
||
expect(mockBudgetService.upsertPolicy).not.toHaveBeenCalled();
|
||
expect(mockLogActivity).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("rejects agent budget updates from another same-company agent without changing the budget policy", async () => {
|
||
const app = createAppWithActor({
|
||
type: "agent",
|
||
agentId: "agent-2",
|
||
companyId: "company-1",
|
||
runId: "run-2",
|
||
});
|
||
|
||
const res = await request(app)
|
||
.patch("/api/agents/agent-1/budgets")
|
||
.send({ budgetMonthlyCents: 2500 });
|
||
|
||
expect(res.status).toBe(403);
|
||
expect(res.body).toEqual({ error: "Board access required" });
|
||
expect(mockAgentService.update).not.toHaveBeenCalled();
|
||
expect(mockBudgetService.upsertPolicy).not.toHaveBeenCalled();
|
||
expect(mockLogActivity).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it("allows authorized board users to update an agent budget and budget policy", async () => {
|
||
mockAgentService.getById.mockResolvedValueOnce({ id: "agent-1", companyId: "company-1", name: "Budget Agent", budgetMonthlyCents: 100, spentMonthlyCents: 0 });
|
||
mockAgentService.getById.mockResolvedValueOnce({
|
||
id: "agent-1",
|
||
companyId: "company-1",
|
||
name: "Budget Agent",
|
||
budgetMonthlyCents: 2500,
|
||
spentMonthlyCents: 0,
|
||
});
|
||
const app = createAppWithActor({
|
||
type: "board",
|
||
userId: "board-user",
|
||
source: "session",
|
||
isInstanceAdmin: false,
|
||
companyIds: ["company-1"],
|
||
memberships: [{ companyId: "company-1", status: "active", membershipRole: "admin" }],
|
||
});
|
||
|
||
const res = await request(app)
|
||
.patch("/api/agents/agent-1/budgets")
|
||
.send({ budgetMonthlyCents: 2500 });
|
||
|
||
expect(res.status).toBe(200);
|
||
expect(res.body.budgetMonthlyCents).toBe(2500);
|
||
expect(mockAgentService.update).not.toHaveBeenCalled();
|
||
expect(mockBudgetService.upsertPolicy).toHaveBeenCalledTimes(1);
|
||
expect(mockBudgetService.upsertPolicy).toHaveBeenCalledWith(
|
||
"company-1",
|
||
{
|
||
scopeType: "agent",
|
||
scopeId: "agent-1",
|
||
amount: 2500,
|
||
isActive: true,
|
||
windowKind: "calendar_month_utc",
|
||
},
|
||
"board-user",
|
||
);
|
||
// Policy mutation owns the atomic entity update and activity log.
|
||
expect(mockLogActivity).not.toHaveBeenCalled();
|
||
});
|
||
});
|
||
|
||
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
|
||
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
|
||
|
||
describeEmbeddedPostgres("cost and finance aggregate overflow handling", () => {
|
||
let db!: ReturnType<typeof createDb>;
|
||
let costs!: ReturnType<typeof costService>;
|
||
let finance!: ReturnType<typeof financeService>;
|
||
let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
|
||
|
||
beforeAll(async () => {
|
||
tempDb = await startEmbeddedPostgresTestDatabase("paperclip-costs-service-");
|
||
db = createDb(tempDb.connectionString);
|
||
costs = costService(db);
|
||
finance = financeService(db);
|
||
}, 20_000);
|
||
|
||
afterEach(async () => {
|
||
await db.delete(financeEvents);
|
||
await db.delete(costEvents);
|
||
await db.delete(activityLog);
|
||
await db.delete(heartbeatRuns);
|
||
await db.delete(issues);
|
||
await db.delete(projects);
|
||
await db.delete(agents);
|
||
await db.delete(companies);
|
||
});
|
||
|
||
afterAll(async () => {
|
||
await tempDb?.cleanup();
|
||
});
|
||
|
||
it("persists unpriced token usage without inflating monthly spend", async () => {
|
||
const companyId = randomUUID();
|
||
const agentId = randomUUID();
|
||
|
||
await db.insert(companies).values({
|
||
id: companyId,
|
||
name: "Paperclip",
|
||
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
|
||
requireBoardApprovalForNewAgents: false,
|
||
});
|
||
await db.insert(agents).values({
|
||
id: agentId,
|
||
companyId,
|
||
name: "CLI Agent",
|
||
role: "engineer",
|
||
status: "active",
|
||
adapterType: "codex_local",
|
||
adapterConfig: {},
|
||
runtimeConfig: {},
|
||
permissions: {},
|
||
});
|
||
|
||
const event = await costs.createEvent(companyId, {
|
||
agentId,
|
||
provider: "openai",
|
||
biller: "chatgpt",
|
||
billingType: "subscription_included",
|
||
costStatus: "unpriced",
|
||
model: "gpt-5.6-terra",
|
||
inputTokens: 2_732_577,
|
||
cachedInputTokens: 2_632_998,
|
||
outputTokens: 32_644,
|
||
costCents: 0,
|
||
occurredAt: new Date("2026-07-13T14:22:54.000Z"),
|
||
});
|
||
|
||
expect(event.costStatus).toBe("unpriced");
|
||
expect(event.inputTokens).toBe(2_732_577);
|
||
const [agent] = await db.select().from(agents).where(eq(agents.id, agentId));
|
||
expect(agent?.spentMonthlyCents).toBe(0);
|
||
});
|
||
|
||
it("aggregates cost event sums above int32 without raising Postgres integer overflow", async () => {
|
||
const companyId = randomUUID();
|
||
const agentId = randomUUID();
|
||
const projectId = randomUUID();
|
||
|
||
await db.insert(companies).values({
|
||
id: companyId,
|
||
name: "Paperclip",
|
||
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
|
||
requireBoardApprovalForNewAgents: false,
|
||
});
|
||
await db.insert(agents).values({
|
||
id: agentId,
|
||
companyId,
|
||
name: "Cost Agent",
|
||
role: "engineer",
|
||
status: "active",
|
||
adapterType: "codex_local",
|
||
adapterConfig: {},
|
||
runtimeConfig: {},
|
||
permissions: {},
|
||
});
|
||
await db.insert(projects).values({
|
||
id: projectId,
|
||
companyId,
|
||
name: "Overflow Project",
|
||
status: "active",
|
||
});
|
||
|
||
await db.insert(costEvents).values([
|
||
{
|
||
companyId,
|
||
agentId,
|
||
projectId,
|
||
provider: "openai",
|
||
biller: "openai",
|
||
billingType: "metered_api",
|
||
model: "gpt-5",
|
||
inputTokens: 2_000_000_000,
|
||
cachedInputTokens: 0,
|
||
outputTokens: 200_000_000,
|
||
costCents: 2_000_000_000,
|
||
occurredAt: new Date("2026-04-10T00:00:00.000Z"),
|
||
},
|
||
{
|
||
companyId,
|
||
agentId,
|
||
projectId,
|
||
provider: "openai",
|
||
biller: "openai",
|
||
billingType: "metered_api",
|
||
model: "gpt-5",
|
||
inputTokens: 2_000_000_000,
|
||
cachedInputTokens: 10,
|
||
outputTokens: 200_000_000,
|
||
costCents: 2_000_000_000,
|
||
costStatus: "estimated",
|
||
occurredAt: new Date("2026-04-11T00:00:00.000Z"),
|
||
},
|
||
]);
|
||
|
||
const range = {
|
||
from: new Date("2026-04-01T00:00:00.000Z"),
|
||
to: new Date("2026-04-15T23:59:59.999Z"),
|
||
};
|
||
|
||
const [byAgentRow] = await costs.byAgent(companyId, range);
|
||
const [byProjectRow] = await costs.byProject(companyId, range);
|
||
const [byAgentModelRow] = await costs.byAgentModel(companyId, range);
|
||
|
||
expect(byAgentRow?.costCents).toBe(4_000_000_000);
|
||
// Historical OpenAI input includes these ten cached tokens.
|
||
expect(byAgentRow?.inputTokens).toBe(3_999_999_990);
|
||
expect(byAgentRow!.inputTokens + byAgentRow!.cachedInputTokens).toBe(4_000_000_000);
|
||
expect(byProjectRow?.costCents).toBe(4_000_000_000);
|
||
expect(byProjectRow).toMatchObject({ eventCount: 2, estimatedEventCount: 1 });
|
||
const [firstDayProject] = await costs.byProject(companyId, {
|
||
...range, to: new Date("2026-04-10T23:59:59.999Z"),
|
||
});
|
||
expect(firstDayProject).toMatchObject({ eventCount: 1, estimatedEventCount: 0 });
|
||
expect(byAgentModelRow?.costCents).toBe(4_000_000_000);
|
||
});
|
||
|
||
it("aggregates issue costs across recursive descendants only", async () => {
|
||
const companyId = randomUUID();
|
||
const agentId = randomUUID();
|
||
const rootIssueId = randomUUID();
|
||
const childIssueId = randomUUID();
|
||
const grandchildIssueId = randomUUID();
|
||
const harnessIssueId = randomUUID();
|
||
const siblingIssueId = randomUUID();
|
||
|
||
await db.insert(companies).values({
|
||
id: companyId,
|
||
name: "Paperclip",
|
||
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
|
||
requireBoardApprovalForNewAgents: false,
|
||
});
|
||
await db.insert(agents).values({
|
||
id: agentId,
|
||
companyId,
|
||
name: "Cost Agent",
|
||
role: "engineer",
|
||
status: "active",
|
||
adapterType: "codex_local",
|
||
adapterConfig: {},
|
||
runtimeConfig: {},
|
||
permissions: {},
|
||
});
|
||
await db.insert(issues).values([
|
||
{
|
||
id: rootIssueId,
|
||
companyId,
|
||
title: "Root",
|
||
status: "in_progress",
|
||
priority: "medium",
|
||
issueNumber: 1,
|
||
identifier: "TST-1",
|
||
},
|
||
{
|
||
id: childIssueId,
|
||
companyId,
|
||
parentId: rootIssueId,
|
||
title: "Child",
|
||
status: "done",
|
||
priority: "medium",
|
||
issueNumber: 2,
|
||
identifier: "TST-2",
|
||
},
|
||
{
|
||
id: grandchildIssueId,
|
||
companyId,
|
||
parentId: childIssueId,
|
||
title: "Grandchild",
|
||
status: "done",
|
||
priority: "medium",
|
||
issueNumber: 3,
|
||
identifier: "TST-3",
|
||
},
|
||
{
|
||
id: harnessIssueId,
|
||
companyId,
|
||
parentId: rootIssueId,
|
||
title: "Hidden skill test harness",
|
||
status: "done",
|
||
priority: "medium",
|
||
issueNumber: 5,
|
||
identifier: "TST-5",
|
||
workMode: "skill_test",
|
||
harnessKind: "skill_test",
|
||
},
|
||
{
|
||
id: siblingIssueId,
|
||
companyId,
|
||
title: "Sibling",
|
||
status: "done",
|
||
priority: "medium",
|
||
issueNumber: 4,
|
||
identifier: "TST-4",
|
||
},
|
||
]);
|
||
await db.insert(costEvents).values([
|
||
{
|
||
companyId,
|
||
agentId,
|
||
issueId: rootIssueId,
|
||
provider: "openai",
|
||
biller: "openai",
|
||
billingType: "metered_api",
|
||
model: "gpt-5",
|
||
inputTokens: 10,
|
||
cachedInputTokens: 1,
|
||
outputTokens: 2,
|
||
costCents: 100,
|
||
occurredAt: new Date("2026-04-10T00:00:00.000Z"),
|
||
},
|
||
{
|
||
companyId,
|
||
agentId,
|
||
issueId: childIssueId,
|
||
provider: "openai",
|
||
biller: "openai",
|
||
billingType: "metered_api",
|
||
model: "gpt-5",
|
||
inputTokens: 20,
|
||
cachedInputTokens: 2,
|
||
outputTokens: 4,
|
||
costCents: 200,
|
||
occurredAt: new Date("2026-04-10T00:01:00.000Z"),
|
||
},
|
||
{
|
||
companyId,
|
||
agentId,
|
||
issueId: grandchildIssueId,
|
||
provider: "openai",
|
||
biller: "openai",
|
||
billingType: "metered_api",
|
||
model: "gpt-5",
|
||
inputTokens: 30,
|
||
cachedInputTokens: 3,
|
||
outputTokens: 6,
|
||
costCents: 300,
|
||
occurredAt: new Date("2026-04-10T00:02:00.000Z"),
|
||
},
|
||
{
|
||
companyId,
|
||
agentId,
|
||
issueId: siblingIssueId,
|
||
provider: "openai",
|
||
biller: "openai",
|
||
billingType: "metered_api",
|
||
model: "gpt-5",
|
||
inputTokens: 40,
|
||
cachedInputTokens: 4,
|
||
outputTokens: 8,
|
||
costCents: 400,
|
||
occurredAt: new Date("2026-04-10T00:03:00.000Z"),
|
||
},
|
||
]);
|
||
|
||
const summary = await costs.issueTreeSummary(companyId, rootIssueId);
|
||
|
||
expect(summary).toEqual({
|
||
issueId: rootIssueId,
|
||
issueCount: 3,
|
||
includeDescendants: true,
|
||
costCents: 600,
|
||
costCentsExact: "600.0000000",
|
||
inputTokens: 54,
|
||
cachedInputTokens: 6,
|
||
outputTokens: 12,
|
||
runCount: 0,
|
||
runtimeMs: 0,
|
||
});
|
||
});
|
||
|
||
it("aggregates run wall-clock duration across the recursive issue tree", async () => {
|
||
const companyId = randomUUID();
|
||
const agentId = randomUUID();
|
||
const rootIssueId = randomUUID();
|
||
const childIssueId = randomUUID();
|
||
const grandchildIssueId = randomUUID();
|
||
const harnessIssueId = randomUUID();
|
||
const siblingIssueId = randomUUID();
|
||
|
||
await db.insert(companies).values({
|
||
id: companyId,
|
||
name: "Paperclip",
|
||
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
|
||
requireBoardApprovalForNewAgents: false,
|
||
});
|
||
await db.insert(agents).values({
|
||
id: agentId,
|
||
companyId,
|
||
name: "Run Agent",
|
||
role: "engineer",
|
||
status: "active",
|
||
adapterType: "codex_local",
|
||
adapterConfig: {},
|
||
runtimeConfig: {},
|
||
permissions: {},
|
||
});
|
||
await db.insert(issues).values([
|
||
{
|
||
id: rootIssueId,
|
||
companyId,
|
||
title: "Root",
|
||
status: "in_progress",
|
||
priority: "medium",
|
||
issueNumber: 1,
|
||
identifier: "TST-1",
|
||
},
|
||
{
|
||
id: childIssueId,
|
||
companyId,
|
||
parentId: rootIssueId,
|
||
title: "Child",
|
||
status: "in_progress",
|
||
priority: "medium",
|
||
issueNumber: 2,
|
||
identifier: "TST-2",
|
||
},
|
||
{
|
||
id: grandchildIssueId,
|
||
companyId,
|
||
parentId: childIssueId,
|
||
title: "Grandchild",
|
||
status: "done",
|
||
priority: "medium",
|
||
issueNumber: 3,
|
||
identifier: "TST-3",
|
||
},
|
||
{
|
||
id: siblingIssueId,
|
||
companyId,
|
||
title: "Sibling",
|
||
status: "done",
|
||
priority: "medium",
|
||
issueNumber: 4,
|
||
identifier: "TST-4",
|
||
},
|
||
{
|
||
id: harnessIssueId,
|
||
companyId,
|
||
parentId: rootIssueId,
|
||
title: "Harness child",
|
||
status: "done",
|
||
priority: "medium",
|
||
workMode: "skill_test",
|
||
harnessKind: "skill_test",
|
||
issueNumber: 5,
|
||
identifier: "TST-5",
|
||
},
|
||
]);
|
||
|
||
const linkedViaContextRunId = randomUUID();
|
||
const linkedViaActivityRunId = randomUUID();
|
||
const grandchildRunId = randomUUID();
|
||
const harnessRunId = randomUUID();
|
||
const siblingRunId = randomUUID();
|
||
const livePartialRunId = randomUUID();
|
||
|
||
await db.insert(heartbeatRuns).values([
|
||
// 60s run linked to root via contextSnapshot.issueId
|
||
{
|
||
id: linkedViaContextRunId,
|
||
companyId,
|
||
agentId,
|
||
invocationSource: "on_demand",
|
||
status: "completed",
|
||
startedAt: new Date("2026-04-10T00:00:00.000Z"),
|
||
finishedAt: new Date("2026-04-10T00:01:00.000Z"),
|
||
contextSnapshot: { issueId: rootIssueId },
|
||
},
|
||
// 120s run linked to child via activity_log
|
||
{
|
||
id: linkedViaActivityRunId,
|
||
companyId,
|
||
agentId,
|
||
invocationSource: "on_demand",
|
||
status: "completed",
|
||
startedAt: new Date("2026-04-10T00:05:00.000Z"),
|
||
finishedAt: new Date("2026-04-10T00:07:00.000Z"),
|
||
},
|
||
// 30s run linked to grandchild
|
||
{
|
||
id: grandchildRunId,
|
||
companyId,
|
||
agentId,
|
||
invocationSource: "on_demand",
|
||
status: "completed",
|
||
startedAt: new Date("2026-04-10T00:10:00.000Z"),
|
||
finishedAt: new Date("2026-04-10T00:10:30.000Z"),
|
||
contextSnapshot: { issueId: grandchildIssueId },
|
||
},
|
||
// 45s harness run under root - should be excluded from visible issue tree rollups
|
||
{
|
||
id: harnessRunId,
|
||
companyId,
|
||
agentId,
|
||
invocationSource: "on_demand",
|
||
status: "completed",
|
||
startedAt: new Date("2026-04-10T00:15:00.000Z"),
|
||
finishedAt: new Date("2026-04-10T00:15:45.000Z"),
|
||
contextSnapshot: { issueId: harnessIssueId },
|
||
},
|
||
// sibling run NOT under root – should be excluded
|
||
{
|
||
id: siblingRunId,
|
||
companyId,
|
||
agentId,
|
||
invocationSource: "on_demand",
|
||
status: "completed",
|
||
startedAt: new Date("2026-04-10T00:20:00.000Z"),
|
||
finishedAt: new Date("2026-04-10T00:21:00.000Z"),
|
||
contextSnapshot: { issueId: siblingIssueId },
|
||
},
|
||
// Still-running run on child (no finishedAt) – should contribute (now - startedAt)
|
||
{
|
||
id: livePartialRunId,
|
||
companyId,
|
||
agentId,
|
||
invocationSource: "on_demand",
|
||
status: "running",
|
||
startedAt: new Date(Date.now() - 5_000),
|
||
contextSnapshot: { issueId: childIssueId },
|
||
},
|
||
]);
|
||
|
||
await db.insert(activityLog).values({
|
||
companyId,
|
||
runId: linkedViaActivityRunId,
|
||
actorType: "agent",
|
||
actorId: agentId,
|
||
agentId,
|
||
action: "issue.checked_out",
|
||
entityType: "issue",
|
||
entityId: childIssueId,
|
||
details: {},
|
||
});
|
||
|
||
const summary = await costs.issueTreeSummary(companyId, rootIssueId);
|
||
|
||
expect(summary.issueCount).toBe(3);
|
||
// 3 finished runs in tree (root, child via activity, grandchild) + 1 live run
|
||
expect(summary.runCount).toBe(4);
|
||
// 60s + 120s + 30s = 210s = 210_000ms from finished runs.
|
||
// Live run adds ~5_000ms; allow some slack so the assertion isn't flaky.
|
||
expect(summary.runtimeMs).toBeGreaterThanOrEqual(210_000 + 4_000);
|
||
expect(summary.runtimeMs).toBeLessThan(210_000 + 60_000);
|
||
|
||
// excludeRoot drops the root issue's own runs (the 60s contextSnapshot run)
|
||
// while keeping the child + grandchild runs and any live child run.
|
||
const descendantsOnly = await costs.issueTreeSummary(companyId, rootIssueId, {
|
||
excludeRoot: true,
|
||
});
|
||
expect(descendantsOnly.issueCount).toBe(2);
|
||
expect(descendantsOnly.runCount).toBe(3);
|
||
// 120s + 30s = 150s + ~5s live run
|
||
expect(descendantsOnly.runtimeMs).toBeGreaterThanOrEqual(150_000 + 4_000);
|
||
expect(descendantsOnly.runtimeMs).toBeLessThan(150_000 + 60_000);
|
||
});
|
||
|
||
it("aggregates finance event sums above int32 without raising Postgres integer overflow", async () => {
|
||
const companyId = randomUUID();
|
||
|
||
await db.insert(companies).values({
|
||
id: companyId,
|
||
name: "Paperclip",
|
||
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
|
||
requireBoardApprovalForNewAgents: false,
|
||
});
|
||
|
||
await db.insert(financeEvents).values([
|
||
{
|
||
companyId,
|
||
biller: "openai",
|
||
eventKind: "invoice",
|
||
amountCents: 2_000_000_000,
|
||
currency: "USD",
|
||
direction: "debit",
|
||
estimated: false,
|
||
occurredAt: new Date("2026-04-10T00:00:00.000Z"),
|
||
},
|
||
{
|
||
companyId,
|
||
biller: "openai",
|
||
eventKind: "invoice",
|
||
amountCents: 2_000_000_000,
|
||
currency: "USD",
|
||
direction: "debit",
|
||
estimated: true,
|
||
occurredAt: new Date("2026-04-11T00:00:00.000Z"),
|
||
},
|
||
]);
|
||
|
||
const range = {
|
||
from: new Date("2026-04-01T00:00:00.000Z"),
|
||
to: new Date("2026-04-15T23:59:59.999Z"),
|
||
};
|
||
|
||
const summary = await finance.summary(companyId, range);
|
||
const [byKindRow] = await finance.byKind(companyId, range);
|
||
|
||
expect(summary.debitCents).toBe(4_000_000_000);
|
||
expect(summary.estimatedDebitCents).toBe(2_000_000_000);
|
||
expect(byKindRow?.debitCents).toBe(4_000_000_000);
|
||
expect(byKindRow?.netCents).toBe(4_000_000_000);
|
||
});
|
||
});
|