Files
PaperClipAI/packages/adapters/codex-local/src/server/parse.ts
T
DottaandPaperclip efcce9cc8e fix(adapters): record unpriced CLI usage (#9505)
## Thinking Path

> - Paperclip is the open source control plane people use to manage
AI-agent companies
> - Budgets and spend telemetry are control-plane safety features, not
just reporting
> - Local Codex and Claude adapters can execute through either ACP or
their native CLI engines
> - The ACP lane records usage and reported cost, but CLI JSON output
often reports tokens without a price
> - The CLI lane was either losing per-run usage semantics or coercing
missing cost to zero, making real usage indistinguishable from a
genuinely free run
> - This pull request preserves CLI usage as per-run totals and records
token-bearing runs without a reported price as explicitly unpriced
ledger events
> - The benefit is accurate usage accounting and a visible pricing gap
instead of silently misleading zero-cost telemetry

## Linked Issues or Issue Description

Refs #9471
Refs #9230

**Bug description**

A `codex_local` run using the CLI engine can emit a final
`turn.completed` event with millions of input tokens and tens of
thousands of output tokens while the agent's spend ledger remains
indistinguishable from a true zero-usage, zero-cost run. Claude CLI
output has the same missing-price edge case.

**Expected behavior**

Token-bearing CLI runs should persist their usage. If the adapter
reports a price, the ledger should record it as reported; if the CLI
reports usage but no price, the ledger should explicitly mark the event
as unpriced rather than silently treating missing price data as a
reported `$0` cost.

**Reproduction shape**

1. Configure `codex_local` with `engine: cli`.
2. Run a task that produces a `turn.completed` usage payload.
3. Observe token usage in the run stream.
4. Before this change, missing price data is represented as ordinary
zero-cost spend and the CLI usage basis is not consistently propagated.

## What Changed

- Mark Codex and Claude native CLI usage totals as `per_run` and
propagate that basis through success and failure results.
- Stop coercing missing Claude CLI cost to `0`.
- Add `cost_status` to cost events with `reported` and `unpriced`
values, including an idempotent migration and shared validation/types.
- Persist token-bearing runs without a reported price as `unpriced`
ledger events while retaining zero cents until an authoritative price
exists.
- Add parser, execute-path, heartbeat-accounting, and cost-service
regression coverage for both local CLI adapters.
- Document the cost-status invariant and CLI accounting behavior.

## Verification

- `pnpm exec vitest run
packages/adapters/codex-local/src/server/parse.test.ts
packages/adapters/claude-local/src/server/parse.test.ts
server/src/__tests__/codex-local-execute.test.ts
server/src/__tests__/claude-local-execute.test.ts
server/src/__tests__/heartbeat-cost-accounting.test.ts
server/src/__tests__/costs-service.test.ts` — 6 files / 102 tests
passed.
- `pnpm --filter @paperclipai/shared typecheck`
- `pnpm --filter @paperclipai/db typecheck` — includes migration
numbering and safety checks.
- `pnpm --filter @paperclipai/adapter-codex-local typecheck`
- `pnpm --filter @paperclipai/adapter-claude-local typecheck`
- `pnpm --filter @paperclipai/server typecheck`

## Risks

- Existing cost rows default to `reported`, preserving current
interpretation; only new token-bearing events with absent cost are
marked `unpriced`.
- This change does not invent model pricing. Budget hard stops still
cannot charge an unknown amount, but operators and evals can now
distinguish missing pricing from a genuinely reported zero cost.
- Consumers that enumerate cost-event fields should tolerate the
additive `costStatus` field.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, exact model `gpt-5.3-codex`, with repository tool use
and code execution; default reasoning mode.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-13 20:44:38 -05:00

273 lines
8.3 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import {
asString,
asNumber,
parseObject,
parseJson,
} from "@paperclipai/adapter-utils/server-utils";
const CODEX_TRANSIENT_UPSTREAM_RE =
/(?:we(?:'|’)re\s+currently\s+experiencing\s+high\s+demand|temporary\s+errors|rate[-\s]?limit(?:ed)?|too\s+many\s+requests|\b429\b|server\s+overloaded|service\s+unavailable|try\s+again\s+later)/i;
const CODEX_REMOTE_COMPACTION_RE = /remote\s+compact\s+task/i;
const CODEX_USAGE_LIMIT_RE =
/you(?:'|’)ve hit your usage limit for .+\.\s+switch to another model now,\s+or try again at\s+([^.!\n]+)(?:[.!]|\n|$)/i;
const CODEX_PROVIDER_QUOTA_RE =
/(?:you(?:'|’)ve hit your usage limit|usage limit|model (?:is )?at capacity|at capacity for this model|capacity limit)/i;
export function parseCodexJsonl(stdout: string) {
let sessionId: string | null = null;
let finalMessage: string | null = null;
let errorMessage: string | null = null;
const usage = {
inputTokens: 0,
cachedInputTokens: 0,
outputTokens: 0,
};
for (const rawLine of stdout.split(/\r?\n/)) {
const line = rawLine.trim();
if (!line) continue;
const event = parseJson(line);
if (!event) continue;
const type = asString(event.type, "");
if (type === "thread.started") {
sessionId = asString(event.thread_id, sessionId ?? "") || sessionId;
continue;
}
if (type === "error") {
const msg = asString(event.message, "").trim();
if (msg) errorMessage = msg;
continue;
}
if (type === "item.completed") {
const item = parseObject(event.item);
if (asString(item.type, "") === "agent_message") {
const text = asString(item.text, "");
if (text) finalMessage = text;
}
continue;
}
if (type === "turn.completed") {
const usageObj = parseObject(event.usage);
usage.inputTokens = asNumber(usageObj.input_tokens, usage.inputTokens);
usage.cachedInputTokens = asNumber(usageObj.cached_input_tokens, usage.cachedInputTokens);
usage.outputTokens = asNumber(usageObj.output_tokens, usage.outputTokens);
continue;
}
if (type === "turn.failed") {
const err = parseObject(event.error);
const msg = asString(err.message, "").trim();
if (msg) errorMessage = msg;
}
}
return {
sessionId,
summary: finalMessage?.trim() ?? "",
usage,
usageBasis: "per_run" as const,
errorMessage,
};
}
export function isCodexUnknownSessionError(stdout: string, stderr: string): boolean {
const haystack = `${stdout}\n${stderr}`
.split(/\r?\n/)
.map((line) => line.trim())
.filter(Boolean)
.join("\n");
return /unknown (session|thread)|session .* not found|thread .* not found|conversation .* not found|missing rollout path for thread|state db missing rollout path|state db returned stale rollout path|no rollout found for thread id/i.test(
haystack,
);
}
function buildCodexErrorHaystack(input: {
stdout?: string | null;
stderr?: string | null;
errorMessage?: string | null;
}): string {
return [
input.errorMessage ?? "",
input.stdout ?? "",
input.stderr ?? "",
]
.join("\n")
.split(/\r?\n/)
.map((line) => line.trim())
.filter(Boolean)
.join("\n");
}
function readTimeZoneParts(date: Date, timeZone: string) {
const values = new Map(
new Intl.DateTimeFormat("en-US", {
timeZone,
hourCycle: "h23",
year: "numeric",
month: "2-digit",
day: "2-digit",
hour: "2-digit",
minute: "2-digit",
}).formatToParts(date).map((part) => [part.type, part.value]),
);
return {
year: Number.parseInt(values.get("year") ?? "", 10),
month: Number.parseInt(values.get("month") ?? "", 10),
day: Number.parseInt(values.get("day") ?? "", 10),
hour: Number.parseInt(values.get("hour") ?? "", 10),
minute: Number.parseInt(values.get("minute") ?? "", 10),
};
}
function normalizeResetTimeZone(timeZoneHint: string | null | undefined): string | null {
const normalized = timeZoneHint?.trim();
if (!normalized) return null;
if (/^(?:utc|gmt)$/i.test(normalized)) return "UTC";
try {
new Intl.DateTimeFormat("en-US", { timeZone: normalized }).format(new Date(0));
return normalized;
} catch {
return null;
}
}
function dateFromTimeZoneWallClock(input: {
year: number;
month: number;
day: number;
hour: number;
minute: number;
timeZone: string;
}): Date | null {
let candidate = new Date(Date.UTC(input.year, input.month - 1, input.day, input.hour, input.minute, 0, 0));
const targetUtc = Date.UTC(input.year, input.month - 1, input.day, input.hour, input.minute, 0, 0);
for (let attempt = 0; attempt < 4; attempt += 1) {
const actual = readTimeZoneParts(candidate, input.timeZone);
const actualUtc = Date.UTC(actual.year, actual.month - 1, actual.day, actual.hour, actual.minute, 0, 0);
const offsetMs = targetUtc - actualUtc;
if (offsetMs === 0) break;
candidate = new Date(candidate.getTime() + offsetMs);
}
const verified = readTimeZoneParts(candidate, input.timeZone);
if (
verified.year !== input.year ||
verified.month !== input.month ||
verified.day !== input.day ||
verified.hour !== input.hour ||
verified.minute !== input.minute
) {
return null;
}
return candidate;
}
function nextClockTimeInTimeZone(input: {
now: Date;
hour: number;
minute: number;
timeZoneHint: string;
}): Date | null {
const timeZone = normalizeResetTimeZone(input.timeZoneHint);
if (!timeZone) return null;
const nowParts = readTimeZoneParts(input.now, timeZone);
let retryAt = dateFromTimeZoneWallClock({
year: nowParts.year,
month: nowParts.month,
day: nowParts.day,
hour: input.hour,
minute: input.minute,
timeZone,
});
if (!retryAt) return null;
if (retryAt.getTime() <= input.now.getTime()) {
const nextDay = new Date(Date.UTC(nowParts.year, nowParts.month - 1, nowParts.day + 1, 0, 0, 0, 0));
retryAt = dateFromTimeZoneWallClock({
year: nextDay.getUTCFullYear(),
month: nextDay.getUTCMonth() + 1,
day: nextDay.getUTCDate(),
hour: input.hour,
minute: input.minute,
timeZone,
});
}
return retryAt;
}
function parseLocalClockTime(clockText: string, now: Date): Date | null {
const normalized = clockText.trim();
const match = normalized.match(/^(\d{1,2})(?::(\d{2}))?\s*([ap])\.?\s*m\.?(?:\s*\(([^)]+)\)|\s+([A-Z]{2,5}))?$/i);
if (!match) return null;
const hour12 = Number.parseInt(match[1] ?? "", 10);
const minute = Number.parseInt(match[2] ?? "0", 10);
if (!Number.isInteger(hour12) || hour12 < 1 || hour12 > 12) return null;
if (!Number.isInteger(minute) || minute < 0 || minute > 59) return null;
let hour24 = hour12 % 12;
if ((match[3] ?? "").toLowerCase() === "p") hour24 += 12;
const timeZoneHint = match[4] ?? match[5];
if (timeZoneHint) {
const explicitRetryAt = nextClockTimeInTimeZone({
now,
hour: hour24,
minute,
timeZoneHint,
});
if (explicitRetryAt) return explicitRetryAt;
}
const retryAt = new Date(now);
retryAt.setHours(hour24, minute, 0, 0);
if (retryAt.getTime() <= now.getTime()) {
retryAt.setDate(retryAt.getDate() + 1);
}
return retryAt;
}
export function extractCodexRetryNotBefore(input: {
stdout?: string | null;
stderr?: string | null;
errorMessage?: string | null;
}, now = new Date()): Date | null {
const haystack = buildCodexErrorHaystack(input);
const usageLimitMatch = haystack.match(CODEX_USAGE_LIMIT_RE);
if (!usageLimitMatch) return null;
return parseLocalClockTime(usageLimitMatch[1] ?? "", now);
}
export function isCodexTransientUpstreamError(input: {
stdout?: string | null;
stderr?: string | null;
errorMessage?: string | null;
}): boolean {
const haystack = buildCodexErrorHaystack(input);
if (isCodexProviderQuotaError(input)) return false;
if (!CODEX_TRANSIENT_UPSTREAM_RE.test(haystack)) return false;
// Keep automatic retries scoped to the observed remote-compaction/high-demand
// failure shape.
return CODEX_REMOTE_COMPACTION_RE.test(haystack) || /high\s+demand|temporary\s+errors/i.test(haystack);
}
export function isCodexProviderQuotaError(input: {
stdout?: string | null;
stderr?: string | null;
errorMessage?: string | null;
}): boolean {
const haystack = buildCodexErrorHaystack(input);
return CODEX_PROVIDER_QUOTA_RE.test(haystack) || extractCodexRetryNotBefore(input) != null;
}