Files
PaperClipAI/server/src/__tests__/costs-service.test.ts
T
DottaandClaude Opus 5.5 6dc36700b1 fix(costs): mark unpriced usage in the agent, model and project cost reports (#20889)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Cost reports show what agents and projects spent
> - A run can have usage with no known price; Paperclip saves it with
costStatus "unpriced" and costCents 0
> - The summary and the provider, biller and user reports count those
events, but the agent, agent-model and project reports did not
> - An agent that read the by-agent-model report saw a native run at
costCents 0 and told the person it cost $0.00
> - This pull request adds unpricedEventCount to those three reports and
tells agents what it means
> - The benefit is that missing charges are reported as unpriced, not as
free

## Linked Issues or Issue Description

**What happened**
In the chat eval budgets-costs.spend-breakdown, the person asked "How
much did Launch cost this week, broken down by agent?". The week
included a real native run whose usage had no known price (data-quality
reason missing_price). The agent read /costs/by-agent-model and
/costs/by-project. Both returned costCents 0 for that usage and no
unpriced count. The reply showed "1 native run ($0.00)" and gave the
total as complete.

**Expected behavior**
The reports say which rows include unpriced usage. The agent says that
usage is unpriced, marks the total incomplete, and links the record.

**Steps to reproduce**
1. Record one priced and one unpriced metered cost event for an agent in
a project.
2. Call GET /api/companies/{companyId}/costs/by-agent, /by-agent-model
and /by-project.
3. See costCents counts only the priced event, and no field shows the
unpriced event.

## What Changed

- `costService.byAgent`, `byAgentModel` and `byProject` return
`unpricedEventCount` (same filter as the summary: unpriced and not
subscription_included).
- Shared types `CostByAgent`, `CostByAgentModel` and `CostByProject` add
`unpricedEventCount`.
- The OpenAPI catalog entries for the cost reports that count unpriced
usage now say that costCents counts priced usage only, that unpriced
usage must be reported as unpriced (never $0.00), and that
/costs/data-quality lists those runs.
- Tests: embedded-Postgres service test (red on master, green here) and
an OpenAPI description test. A UI test fixture adds the new field.

## Verification

- `npx vitest run src/__tests__/costs-service.test.ts` (19 passed). The
new test fails with the master service.
- `npx vitest run src/__tests__/openapi-routes.test.ts
src/services/native-runtime/runner-api.test.ts` (2146 passed).
- `tsc --noEmit` for packages/shared and server.

## Risks

- Low risk. One added aggregate column per report and an added response
field. The UI does not read it yet.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 13:36:03 -06:00

985 lines
31 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import express from "express";
import request from "supertest";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { afterAll, afterEach, beforeAll } from "vitest";
import { randomUUID } from "node:crypto";
import { eq } from "drizzle-orm";
import {
createDb,
companies,
agents,
activityLog,
costEvents,
financeEvents,
heartbeatRuns,
issues,
projects,
} from "@paperclipai/db";
import { costService } from "../services/costs.ts";
import { financeService } from "../services/finance.ts";
import {
getEmbeddedPostgresTestSupport,
startEmbeddedPostgresTestDatabase,
} from "./helpers/embedded-postgres.js";
import { hoistModuleGraph } from "./helpers/hoist-module-graph.js";
function makeDb(overrides: Record<string, unknown> = {}) {
const selectChain = {
from: vi.fn().mockReturnThis(),
where: vi.fn().mockReturnThis(),
leftJoin: vi.fn().mockReturnThis(),
innerJoin: vi.fn().mockReturnThis(),
groupBy: vi.fn().mockReturnThis(),
orderBy: vi.fn().mockReturnThis(),
limit: vi.fn().mockReturnThis(),
then: vi.fn().mockResolvedValue([]),
};
const thenableChain = Object.assign(Promise.resolve([]), selectChain);
return {
select: vi.fn().mockReturnValue(thenableChain),
insert: vi.fn().mockReturnValue({
values: vi.fn().mockReturnValue({ returning: vi.fn().mockResolvedValue([]) }),
}),
update: vi.fn().mockReturnValue({
set: vi.fn().mockReturnValue({ where: vi.fn().mockResolvedValue([]) }),
}),
...overrides,
};
}
const mockCompanyService = vi.hoisted(() => ({
getById: vi.fn(),
update: vi.fn(),
}));
const mockAgentService = vi.hoisted(() => ({
getById: vi.fn(),
update: vi.fn(),
}));
const mockIssueService = vi.hoisted(() => ({
getById: vi.fn(),
getByIdentifier: vi.fn(),
}));
const mockHeartbeatService = vi.hoisted(() => ({
cancelBudgetScopeWork: vi.fn().mockResolvedValue(undefined),
}));
const mockLogActivity = vi.hoisted(() => vi.fn());
const mockFetchAllQuotaWindows = vi.hoisted(() => vi.fn());
const mockCostService = vi.hoisted(() => ({
createEvent: vi.fn(),
summary: vi.fn().mockResolvedValue({ spendCents: 0 }),
byAgent: vi.fn().mockResolvedValue([]),
byAgentModel: vi.fn().mockResolvedValue([]),
byProvider: vi.fn().mockResolvedValue([]),
byBiller: vi.fn().mockResolvedValue([]),
issueTreeSummary: vi.fn().mockResolvedValue({
issueId: "issue-1",
issueCount: 1,
includeDescendants: true,
costCents: 0,
inputTokens: 0,
cachedInputTokens: 0,
outputTokens: 0,
runCount: 0,
runtimeMs: 0,
}),
windowSpend: vi.fn().mockResolvedValue([]),
byProject: vi.fn().mockResolvedValue([]),
}));
const mockFinanceService = vi.hoisted(() => ({
createEvent: vi.fn(),
summary: vi.fn().mockResolvedValue({ debitCents: 0, creditCents: 0, netCents: 0, estimatedDebitCents: 0, eventCount: 0 }),
byBiller: vi.fn().mockResolvedValue([]),
byKind: vi.fn().mockResolvedValue([]),
list: vi.fn().mockResolvedValue([]),
}));
const mockBudgetService = vi.hoisted(() => ({
overview: vi.fn().mockResolvedValue({
companyId: "company-1",
policies: [],
activeIncidents: [],
pausedAgentCount: 0,
pausedProjectCount: 0,
pendingApprovalCount: 0,
}),
upsertPolicy: vi.fn(),
resolveIncident: vi.fn(),
}));
const mockAccessService = vi.hoisted(() => ({
decide: vi.fn(),
}));
function registerModuleMocks() {
vi.doMock("../services/index.js", () => ({
accessService: () => mockAccessService,
budgetService: () => mockBudgetService,
costService: () => mockCostService,
financeService: () => mockFinanceService,
companyService: () => mockCompanyService,
agentService: () => mockAgentService,
issueService: () => mockIssueService,
heartbeatService: () => mockHeartbeatService,
logActivity: mockLogActivity,
}));
vi.doMock("../services/quota-windows.js", () => ({
fetchAllQuotaWindows: mockFetchAllQuotaWindows,
}));
}
describe("cost routes", () => {
const routeModules = hoistModuleGraph(registerModuleMocks, async () => {
const [costsRouteModule, middlewareModule] = await Promise.all([
vi.importActual<typeof import("../routes/costs.js")>("../routes/costs.js"),
vi.importActual<typeof import("../middleware/index.js")>("../middleware/index.js"),
]);
return { ...costsRouteModule, errorHandler: middlewareModule.errorHandler };
});
function createApp() {
const { costRoutes, errorHandler } = routeModules.value;
const app = express();
app.use(express.json());
app.use((req, _res, next) => {
req.actor = { type: "board", userId: "board-user", source: "local_implicit" };
next();
});
app.use("/api", costRoutes(makeDb() as any));
app.use(errorHandler);
return app;
}
function createAppWithActor(actor: any) {
const { costRoutes, errorHandler } = routeModules.value;
const app = express();
app.use(express.json());
app.use((req, _res, next) => {
req.actor = actor;
next();
});
app.use("/api", costRoutes(makeDb() as any));
app.use(errorHandler);
return app;
}
function loadCostParsers() {
const { parseCostDateRange, parseCostLimit } = routeModules.value;
return { parseCostDateRange, parseCostLimit };
}
beforeEach(() => {
vi.clearAllMocks();
mockAccessService.decide.mockReset();
mockAccessService.decide.mockResolvedValue({
allowed: true,
action: "company_scope:read",
reason: "allow_test",
explanation: "Allowed by test mock.",
});
mockCompanyService.update.mockResolvedValue({
id: "company-1",
name: "Paperclip",
budgetMonthlyCents: 100,
spentMonthlyCents: 0,
});
mockAgentService.getById.mockResolvedValue({
id: "agent-1",
companyId: "company-1",
name: "Budget Agent",
budgetMonthlyCents: 100,
spentMonthlyCents: 0,
});
mockAgentService.update.mockResolvedValue({
id: "agent-1",
companyId: "company-1",
name: "Budget Agent",
budgetMonthlyCents: 100,
spentMonthlyCents: 0,
});
mockIssueService.getById.mockResolvedValue({
id: "issue-1",
companyId: "company-1",
identifier: "PC1A2-1",
});
mockIssueService.getByIdentifier.mockResolvedValue({
id: "issue-1",
companyId: "company-1",
identifier: "PC1A2-1",
});
mockBudgetService.upsertPolicy.mockResolvedValue(undefined);
});
it("accepts valid ISO date strings", async () => {
const { parseCostDateRange } = loadCostParsers();
expect(parseCostDateRange({
from: "2026-01-01T00:00:00.000Z",
to: "2026-01-31T23:59:59.999Z",
})).toEqual({
from: new Date("2026-01-01T00:00:00.000Z"),
to: new Date("2026-01-31T23:59:59.999Z"),
});
});
it("requires an explicit all-time request and rejects conflicting ranges", () => {
const { parseCostDateRange } = loadCostParsers();
expect(parseCostDateRange({})).toBeUndefined();
expect(parseCostDateRange({ period: "all" })).toEqual({ allTime: true });
for (const input of [
{ from: "2026-09-02", to: "2026-09-01" },
{ from: ["2026-09-01"] },
{ from: "2026-02-30" },
{ from: "" },
{ from: "09/01/2026" },
{ period: "all", from: "2026-09-01" },
{ period: "unknown" },
]) expect(() => parseCostDateRange(input)).toThrow();
});
it("returns 400 for an invalid 'from' date string", async () => {
const { parseCostDateRange } = loadCostParsers();
expect(() => parseCostDateRange({ from: "not-a-date" })).toThrow(/invalid 'from' date/i);
});
it("returns 400 for an invalid 'to' date string", async () => {
const { parseCostDateRange } = loadCostParsers();
expect(() => parseCostDateRange({ to: "banana" })).toThrow(/invalid 'to' date/i);
});
it("returns finance summary rows for valid requests", async () => {
const app = createApp();
const res = await request(app)
.get("/api/companies/company-1/costs/finance-summary")
.query({ from: "2026-02-01T00:00:00.000Z", to: "2026-02-28T23:59:59.999Z" });
expect(res.status).toBe(200);
expect(res.body).toEqual({
debitCents: 0,
creditCents: 0,
netCents: 0,
estimatedDebitCents: 0,
eventCount: 0,
});
});
it("returns issue subtree cost summaries for issue refs", async () => {
const app = createApp();
const res = await request(app).get("/api/issues/pc1a2-1/cost-summary");
expect(res.status).toBe(200);
expect(mockIssueService.getByIdentifier).toHaveBeenCalledWith("PC1A2-1");
expect(mockCostService.issueTreeSummary).toHaveBeenCalledWith("company-1", "issue-1", {
excludeRoot: false,
});
expect(res.body).toEqual({
issueId: "issue-1",
issueCount: 1,
includeDescendants: true,
costCents: 0,
inputTokens: 0,
cachedInputTokens: 0,
outputTokens: 0,
runCount: 0,
runtimeMs: 0,
});
});
it("returns 400 for invalid finance event list limits", async () => {
const { parseCostLimit } = loadCostParsers();
for (const limit of ["0", "25garbage", "1.5", ["25"], "501"]) expect(() => parseCostLimit({ limit })).toThrow(/invalid 'limit'/i);
});
it("accepts valid finance event list limits", async () => {
const { parseCostLimit } = loadCostParsers();
expect(parseCostLimit({ limit: "25" })).toBe(25);
});
it("rejects company budget updates for board users outside the company", async () => {
const app = createAppWithActor({
type: "board",
userId: "board-user",
source: "session",
isInstanceAdmin: false,
companyIds: ["company-2"],
});
const res = await request(app)
.patch("/api/companies/company-1/budgets")
.send({ budgetMonthlyCents: 2500 });
expect(res.status).toBe(403);
expect(mockCompanyService.update).not.toHaveBeenCalled();
});
it("rejects agent budget updates for board users outside the agent company", async () => {
const app = createAppWithActor({
type: "board",
userId: "board-user",
source: "session",
isInstanceAdmin: false,
companyIds: ["company-2"],
});
const res = await request(app)
.patch("/api/agents/agent-1/budgets")
.send({ budgetMonthlyCents: 2500 });
expect(res.status).toBe(404);
expect(res.body.error).toBe("Agent not found");
expect(mockAgentService.update).not.toHaveBeenCalled();
});
it("rejects agent budget updates from the target agent without changing the budget policy", async () => {
const app = createAppWithActor({
type: "agent",
agentId: "agent-1",
companyId: "company-1",
runId: "run-1",
});
const res = await request(app)
.patch("/api/agents/agent-1/budgets")
.send({ budgetMonthlyCents: 2500 });
expect(res.status).toBe(403);
expect(res.body).toMatchObject({ error: "Board access required", code: "board_access_required" });
expect(mockAgentService.update).not.toHaveBeenCalled();
expect(mockBudgetService.upsertPolicy).not.toHaveBeenCalled();
expect(mockLogActivity).not.toHaveBeenCalled();
});
it("rejects agent budget updates from another same-company agent without changing the budget policy", async () => {
const app = createAppWithActor({
type: "agent",
agentId: "agent-2",
companyId: "company-1",
runId: "run-2",
});
const res = await request(app)
.patch("/api/agents/agent-1/budgets")
.send({ budgetMonthlyCents: 2500 });
expect(res.status).toBe(403);
expect(res.body).toMatchObject({ error: "Board access required", code: "board_access_required" });
expect(mockAgentService.update).not.toHaveBeenCalled();
expect(mockBudgetService.upsertPolicy).not.toHaveBeenCalled();
expect(mockLogActivity).not.toHaveBeenCalled();
});
it("allows authorized board users to update an agent budget and budget policy", async () => {
mockAgentService.getById.mockResolvedValueOnce({ id: "agent-1", companyId: "company-1", name: "Budget Agent", budgetMonthlyCents: 100, spentMonthlyCents: 0 });
mockAgentService.getById.mockResolvedValueOnce({
id: "agent-1",
companyId: "company-1",
name: "Budget Agent",
budgetMonthlyCents: 2500,
spentMonthlyCents: 0,
});
const app = createAppWithActor({
type: "board",
userId: "board-user",
source: "session",
isInstanceAdmin: false,
companyIds: ["company-1"],
memberships: [{ companyId: "company-1", status: "active", membershipRole: "admin" }],
});
const res = await request(app)
.patch("/api/agents/agent-1/budgets")
.send({ budgetMonthlyCents: 2500 });
expect(res.status).toBe(200);
expect(res.body.budgetMonthlyCents).toBe(2500);
expect(mockAgentService.update).not.toHaveBeenCalled();
expect(mockBudgetService.upsertPolicy).toHaveBeenCalledTimes(1);
expect(mockBudgetService.upsertPolicy).toHaveBeenCalledWith(
"company-1",
{
scopeType: "agent",
scopeId: "agent-1",
amount: 2500,
isActive: true,
windowKind: "calendar_month_utc",
},
"board-user",
);
// Policy mutation owns the atomic entity update and activity log.
expect(mockLogActivity).not.toHaveBeenCalled();
});
});
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
describeEmbeddedPostgres("cost and finance aggregate overflow handling", () => {
let db!: ReturnType<typeof createDb>;
let costs!: ReturnType<typeof costService>;
let finance!: ReturnType<typeof financeService>;
let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
beforeAll(async () => {
tempDb = await startEmbeddedPostgresTestDatabase("paperclip-costs-service-");
db = createDb(tempDb.connectionString);
costs = costService(db);
finance = financeService(db);
}, 20_000);
afterEach(async () => {
await db.delete(financeEvents);
await db.delete(costEvents);
await db.delete(activityLog);
await db.delete(heartbeatRuns);
await db.delete(issues);
await db.delete(projects);
await db.delete(agents);
await db.delete(companies);
});
afterAll(async () => {
await tempDb?.cleanup();
});
it("persists unpriced token usage without inflating monthly spend", async () => {
const companyId = randomUUID();
const agentId = randomUUID();
await db.insert(companies).values({
id: companyId,
name: "Paperclip",
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
requireBoardApprovalForNewAgents: false,
});
await db.insert(agents).values({
id: agentId,
companyId,
name: "CLI Agent",
role: "engineer",
status: "active",
adapterType: "codex_local",
adapterConfig: {},
runtimeConfig: {},
permissions: {},
});
const event = await costs.createEvent(companyId, {
agentId,
provider: "openai",
biller: "chatgpt",
billingType: "subscription_included",
costStatus: "unpriced",
model: "gpt-5.6-terra",
inputTokens: 2_732_577,
cachedInputTokens: 2_632_998,
outputTokens: 32_644,
costCents: 0,
occurredAt: new Date(),
});
expect(event.costStatus).toBe("unpriced");
expect(event.inputTokens).toBe(2_732_577);
expect(await costs.summary(companyId)).toMatchObject({ eventCount: 1, spendCents: 0, spendCentsExact: "0.0000000" });
const [usage] = await costs.byAgent(companyId);
expect(usage).toMatchObject({ agentId, costCents: 0, inputTokens: 2_732_577,
cachedInputTokens: 2_632_998, outputTokens: 32_644 });
});
it("marks metered usage with no known price as unpriced in the agent, model and project reports", async () => {
const companyId = randomUUID();
const agentId = randomUUID();
const projectId = randomUUID();
await db.insert(companies).values({
id: companyId,
name: "Paperclip",
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
requireBoardApprovalForNewAgents: false,
});
await db.insert(agents).values({
id: agentId, companyId, name: "Morgan", role: "engineer", status: "active",
adapterType: "opencode_local", adapterConfig: {}, runtimeConfig: {}, permissions: {},
});
await db.insert(projects).values({ id: projectId, companyId, name: "Launch" });
const base = { agentId, projectId, provider: "deepseek", biller: "openrouter", billingType: "metered_api" as const,
inputTokens: 100, cachedInputTokens: 0, outputTokens: 10, occurredAt: new Date() };
await costs.createEvent(companyId, { ...base, model: "priced-model", costStatus: "reported", costCents: 137 });
await costs.createEvent(companyId, { ...base, model: "unpriced-model", costStatus: "unpriced", costCents: 0 });
const [byAgentRow] = await costs.byAgent(companyId);
expect(byAgentRow).toMatchObject({ agentId, costCents: 137, eventCount: 2, unpricedEventCount: 1 });
const modelRows = await costs.byAgentModel(companyId);
expect(modelRows.find(row => row.model === "unpriced-model")).toMatchObject({ costCents: 0, unpricedEventCount: 1 });
expect(modelRows.find(row => row.model === "priced-model")).toMatchObject({ costCents: 137, unpricedEventCount: 0 });
const [byProjectRow] = await costs.byProject(companyId);
expect(byProjectRow).toMatchObject({ projectId, costCents: 137, eventCount: 2, unpricedEventCount: 1 });
});
it("aggregates cost event sums above int32 without raising Postgres integer overflow", async () => {
const companyId = randomUUID();
const agentId = randomUUID();
const projectId = randomUUID();
await db.insert(companies).values({
id: companyId,
name: "Paperclip",
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
requireBoardApprovalForNewAgents: false,
});
await db.insert(agents).values({
id: agentId,
companyId,
name: "Cost Agent",
role: "engineer",
status: "active",
adapterType: "codex_local",
adapterConfig: {},
runtimeConfig: {},
permissions: {},
});
await db.insert(projects).values({
id: projectId,
companyId,
name: "Overflow Project",
status: "active",
});
await db.insert(costEvents).values([
{
companyId,
agentId,
projectId,
provider: "openai",
biller: "openai",
billingType: "metered_api",
model: "gpt-5",
inputTokens: 2_000_000_000,
cachedInputTokens: 0,
outputTokens: 200_000_000,
costCents: 2_000_000_000,
occurredAt: new Date("2026-04-10T00:00:00.000Z"),
},
{
companyId,
agentId,
projectId,
provider: "openai",
biller: "openai",
billingType: "metered_api",
model: "gpt-5",
inputTokens: 2_000_000_000,
cachedInputTokens: 10,
outputTokens: 200_000_000,
costCents: 2_000_000_000,
costStatus: "estimated",
occurredAt: new Date("2026-04-11T00:00:00.000Z"),
},
]);
const range = {
from: new Date("2026-04-01T00:00:00.000Z"),
to: new Date("2026-04-15T23:59:59.999Z"),
};
const [byAgentRow] = await costs.byAgent(companyId, range);
const [byProjectRow] = await costs.byProject(companyId, range);
const [byAgentModelRow] = await costs.byAgentModel(companyId, range);
expect(byAgentRow?.costCents).toBe(4_000_000_000);
// Historical OpenAI input includes these ten cached tokens.
expect(byAgentRow?.inputTokens).toBe(3_999_999_990);
expect(byAgentRow!.inputTokens + byAgentRow!.cachedInputTokens).toBe(4_000_000_000);
expect(byProjectRow?.costCents).toBe(4_000_000_000);
expect(byProjectRow).toMatchObject({ eventCount: 2, estimatedEventCount: 1 });
const [firstDayProject] = await costs.byProject(companyId, {
...range, to: new Date("2026-04-10T23:59:59.999Z"),
});
expect(firstDayProject).toMatchObject({ eventCount: 1, estimatedEventCount: 0 });
expect(byAgentModelRow?.costCents).toBe(4_000_000_000);
});
it("aggregates issue costs across recursive descendants only", async () => {
const companyId = randomUUID();
const agentId = randomUUID();
const rootIssueId = randomUUID();
const childIssueId = randomUUID();
const grandchildIssueId = randomUUID();
const harnessIssueId = randomUUID();
const siblingIssueId = randomUUID();
await db.insert(companies).values({
id: companyId,
name: "Paperclip",
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
requireBoardApprovalForNewAgents: false,
});
await db.insert(agents).values({
id: agentId,
companyId,
name: "Cost Agent",
role: "engineer",
status: "active",
adapterType: "codex_local",
adapterConfig: {},
runtimeConfig: {},
permissions: {},
});
await db.insert(issues).values([
{
id: rootIssueId,
companyId,
title: "Root",
status: "in_progress",
priority: "medium",
issueNumber: 1,
identifier: "TST-1",
},
{
id: childIssueId,
companyId,
parentId: rootIssueId,
title: "Child",
status: "done",
priority: "medium",
issueNumber: 2,
identifier: "TST-2",
},
{
id: grandchildIssueId,
companyId,
parentId: childIssueId,
title: "Grandchild",
status: "done",
priority: "medium",
issueNumber: 3,
identifier: "TST-3",
},
{
id: harnessIssueId,
companyId,
parentId: rootIssueId,
title: "Hidden skill test harness",
status: "done",
priority: "medium",
issueNumber: 5,
identifier: "TST-5",
workMode: "skill_test",
harnessKind: "skill_test",
},
{
id: siblingIssueId,
companyId,
title: "Sibling",
status: "done",
priority: "medium",
issueNumber: 4,
identifier: "TST-4",
},
]);
await db.insert(costEvents).values([
{
companyId,
agentId,
issueId: rootIssueId,
provider: "openai",
biller: "openai",
billingType: "metered_api",
model: "gpt-5",
inputTokens: 10,
cachedInputTokens: 1,
outputTokens: 2,
costCents: 100,
occurredAt: new Date("2026-04-10T00:00:00.000Z"),
},
{
companyId,
agentId,
issueId: childIssueId,
provider: "openai",
biller: "openai",
billingType: "metered_api",
model: "gpt-5",
inputTokens: 20,
cachedInputTokens: 2,
outputTokens: 4,
costCents: 200,
occurredAt: new Date("2026-04-10T00:01:00.000Z"),
},
{
companyId,
agentId,
issueId: grandchildIssueId,
provider: "openai",
biller: "openai",
billingType: "metered_api",
model: "gpt-5",
inputTokens: 30,
cachedInputTokens: 3,
outputTokens: 6,
costCents: 300,
occurredAt: new Date("2026-04-10T00:02:00.000Z"),
},
{
companyId,
agentId,
issueId: siblingIssueId,
provider: "openai",
biller: "openai",
billingType: "metered_api",
model: "gpt-5",
inputTokens: 40,
cachedInputTokens: 4,
outputTokens: 8,
costCents: 400,
occurredAt: new Date("2026-04-10T00:03:00.000Z"),
},
]);
const summary = await costs.issueTreeSummary(companyId, rootIssueId);
expect(summary).toEqual({
issueId: rootIssueId,
issueCount: 3,
includeDescendants: true,
costCents: 600,
costCentsExact: "600.0000000",
inputTokens: 54,
cachedInputTokens: 6,
outputTokens: 12,
runCount: 0,
runtimeMs: 0,
});
});
it("aggregates run wall-clock duration across the recursive issue tree", async () => {
const companyId = randomUUID();
const agentId = randomUUID();
const rootIssueId = randomUUID();
const childIssueId = randomUUID();
const grandchildIssueId = randomUUID();
const harnessIssueId = randomUUID();
const siblingIssueId = randomUUID();
await db.insert(companies).values({
id: companyId,
name: "Paperclip",
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
requireBoardApprovalForNewAgents: false,
});
await db.insert(agents).values({
id: agentId,
companyId,
name: "Run Agent",
role: "engineer",
status: "active",
adapterType: "codex_local",
adapterConfig: {},
runtimeConfig: {},
permissions: {},
});
await db.insert(issues).values([
{
id: rootIssueId,
companyId,
title: "Root",
status: "in_progress",
priority: "medium",
issueNumber: 1,
identifier: "TST-1",
},
{
id: childIssueId,
companyId,
parentId: rootIssueId,
title: "Child",
status: "in_progress",
priority: "medium",
issueNumber: 2,
identifier: "TST-2",
},
{
id: grandchildIssueId,
companyId,
parentId: childIssueId,
title: "Grandchild",
status: "done",
priority: "medium",
issueNumber: 3,
identifier: "TST-3",
},
{
id: siblingIssueId,
companyId,
title: "Sibling",
status: "done",
priority: "medium",
issueNumber: 4,
identifier: "TST-4",
},
{
id: harnessIssueId,
companyId,
parentId: rootIssueId,
title: "Harness child",
status: "done",
priority: "medium",
workMode: "skill_test",
harnessKind: "skill_test",
issueNumber: 5,
identifier: "TST-5",
},
]);
const linkedViaContextRunId = randomUUID();
const linkedViaActivityRunId = randomUUID();
const grandchildRunId = randomUUID();
const harnessRunId = randomUUID();
const siblingRunId = randomUUID();
const livePartialRunId = randomUUID();
await db.insert(heartbeatRuns).values([
// 60s run linked to root via contextSnapshot.issueId
{
id: linkedViaContextRunId,
companyId,
agentId,
invocationSource: "on_demand",
status: "completed",
startedAt: new Date("2026-04-10T00:00:00.000Z"),
finishedAt: new Date("2026-04-10T00:01:00.000Z"),
contextSnapshot: { issueId: rootIssueId },
},
// 120s run linked to child via activity_log
{
id: linkedViaActivityRunId,
companyId,
agentId,
invocationSource: "on_demand",
status: "completed",
startedAt: new Date("2026-04-10T00:05:00.000Z"),
finishedAt: new Date("2026-04-10T00:07:00.000Z"),
},
// 30s run linked to grandchild
{
id: grandchildRunId,
companyId,
agentId,
invocationSource: "on_demand",
status: "completed",
startedAt: new Date("2026-04-10T00:10:00.000Z"),
finishedAt: new Date("2026-04-10T00:10:30.000Z"),
contextSnapshot: { issueId: grandchildIssueId },
},
// 45s harness run under root - should be excluded from visible issue tree rollups
{
id: harnessRunId,
companyId,
agentId,
invocationSource: "on_demand",
status: "completed",
startedAt: new Date("2026-04-10T00:15:00.000Z"),
finishedAt: new Date("2026-04-10T00:15:45.000Z"),
contextSnapshot: { issueId: harnessIssueId },
},
// sibling run NOT under root – should be excluded
{
id: siblingRunId,
companyId,
agentId,
invocationSource: "on_demand",
status: "completed",
startedAt: new Date("2026-04-10T00:20:00.000Z"),
finishedAt: new Date("2026-04-10T00:21:00.000Z"),
contextSnapshot: { issueId: siblingIssueId },
},
// Still-running run on child (no finishedAt) – should contribute (now - startedAt)
{
id: livePartialRunId,
companyId,
agentId,
invocationSource: "on_demand",
status: "running",
startedAt: new Date(Date.now() - 5_000),
contextSnapshot: { issueId: childIssueId },
},
]);
await db.insert(activityLog).values({
companyId,
runId: linkedViaActivityRunId,
actorType: "agent",
actorId: agentId,
agentId,
action: "issue.checked_out",
entityType: "issue",
entityId: childIssueId,
details: {},
});
const summary = await costs.issueTreeSummary(companyId, rootIssueId);
expect(summary.issueCount).toBe(3);
// 3 finished runs in tree (root, child via activity, grandchild) + 1 live run
expect(summary.runCount).toBe(4);
// 60s + 120s + 30s = 210s = 210_000ms from finished runs.
// Live run adds ~5_000ms; allow some slack so the assertion isn't flaky.
expect(summary.runtimeMs).toBeGreaterThanOrEqual(210_000 + 4_000);
expect(summary.runtimeMs).toBeLessThan(210_000 + 60_000);
// excludeRoot drops the root issue's own runs (the 60s contextSnapshot run)
// while keeping the child + grandchild runs and any live child run.
const descendantsOnly = await costs.issueTreeSummary(companyId, rootIssueId, {
excludeRoot: true,
});
expect(descendantsOnly.issueCount).toBe(2);
expect(descendantsOnly.runCount).toBe(3);
// 120s + 30s = 150s + ~5s live run
expect(descendantsOnly.runtimeMs).toBeGreaterThanOrEqual(150_000 + 4_000);
expect(descendantsOnly.runtimeMs).toBeLessThan(150_000 + 60_000);
});
it("aggregates finance event sums above int32 without raising Postgres integer overflow", async () => {
const companyId = randomUUID();
await db.insert(companies).values({
id: companyId,
name: "Paperclip",
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
requireBoardApprovalForNewAgents: false,
});
await db.insert(financeEvents).values([
{
companyId,
biller: "openai",
eventKind: "invoice",
amountCents: 2_000_000_000,
currency: "USD",
direction: "debit",
estimated: false,
occurredAt: new Date("2026-04-10T00:00:00.000Z"),
},
{
companyId,
biller: "openai",
eventKind: "invoice",
amountCents: 2_000_000_000,
currency: "USD",
direction: "debit",
estimated: true,
occurredAt: new Date("2026-04-11T00:00:00.000Z"),
},
]);
const range = {
from: new Date("2026-04-01T00:00:00.000Z"),
to: new Date("2026-04-15T23:59:59.999Z"),
};
const summary = await finance.summary(companyId, range);
const [byKindRow] = await finance.byKind(companyId, range);
expect(summary.debitCents).toBe(4_000_000_000);
expect(summary.estimatedDebitCents).toBe(2_000_000_000);
expect(byKindRow?.debitCents).toBe(4_000_000_000);
expect(byKindRow?.netCents).toBe(4_000_000_000);
});
});