mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-18 05:23:39 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its Costs page reports API charges and subscription token usage. > - Subscription runs do not show the recurring fee paid for the account. > - Several agents can share one paid seat, so a fee per agent would overstate cost. > - This pull request identifies connected subscription accounts and reports each monthly fee once. > - Users can compare API spend and subscription costs, see uncertainty, and enter their actual price. ## Linked Issues or Issue Description Refs #498 and #3406 for related subscription reporting needs. This change does not add subscription budget enforcement or automatic optimization. Related prior approaches: #10544 and #11104 proposed API-equivalent token costs for subscription work. This PR reports recurring account fees separately from usage charges and budgets. A search found no open PR for monthly account fee reporting. **Subsystem affected** Costs, managed AI connections, shared contracts, database schema, and board UI. **Problem or motivation** An organization can pay for subscriptions even when included runs record no additional dollar charge. The Costs page cannot show that monthly commitment or distinguish a shared paid seat from several separate seats. **Proposed solution** Discover account identity from the managed connection selected for a run. Use exact known plans for labeled monthly list-price estimates. Keep ambiguous plans and unconfirmed identities unpriced until the user supplies a price. Sum current monthly fees separately from date-filtered API charges and token usage. **Alternatives considered** A fee per agent double-counts shared accounts. Token-based API price estimates do not measure the recurring subscription fee. Provider invoices would be more exact but are not available through these account credentials. **Roadmap alignment** Extends the shipped Better Budgeting and Personal & Shared AI Accounts capabilities. The feature and UI were reviewed with the maintainer before this PR. ## What Changed - Add company-scoped subscription accounts, connection bindings, and immutable price revisions in replay-safe migration 0322. - Identify OpenAI workspace/seat and Claude organization/account pairs only after provider authentication. Keep provisional records grant-scoped; unchecked ID-token claims cannot combine accounts or grant billing editors. Keep credentials and raw provider identities out of reports. - Attach the selected subscription to new run receipts. Resolve duplicate aliases without rewriting old monetary receipts. - Add cached background discovery with four provider checks and twenty discovery requests at a time. Accepted requests wait for capacity; excess requests receive HTTP 429 and a UI retry action. Preserve saved prices and observations when a provider check fails. Dashboard polls load current prices only; audit revisions stay in the database. - Support owner-controlled monthly or annual prices, fractional-cent precision, separate currency totals, tracking status, and duplicate linking. Preserve both personal owners and shared-account managers when discovery finds the same paid seat. - Show API spend beside monthly subscription commitment, with separate token totals and clear estimate labels. Account details and fee totals follow billing-owner/shared-audience visibility; company-wide token totals remain available to cost readers. - Use consistent amount and estimate badge formatting for agent, user, project, and model rows. - Add real PostgreSQL tests, route authorization tests, UI tests, interactive Storybook fixtures, and operator documentation. ## Verification - Current head `d7b5c78dadf5033a3c447d83434676ac94687e77` passed all 53 checks, with two checks skipped. This includes the full test matrix, browser tests, build, typecheck, migration policy, canary packaging, and security checks. - Greptile Apex reviewed that exact head at 5/5 with no unresolved findings. - Local `pnpm -r typecheck`, `pnpm build`, `pnpm check:token-gates`, migration safety checks, and `git diff --check` passed. - All 238 focused tests passed across accounting, subscription persistence, authorization, UI, migration replay, and snapshot drift. - The expanded local chat suite passed 1,149 tests. Two cases failed because this workspace found an unrelated ancestor skill directory; both passed unchanged in an isolated checkout. The full local Vitest command was not repeated after the rebase because of this environment issue. CI covered the complete test matrix on the final head. - Migration `0322_reflective_kree.sql` follows master's `0321_acoustic_human_fly.sql`. Its snapshot and journal were regenerated. Master's migration and snapshot are unchanged. The subscription SQL retains its original content and hash. - Disposable PostgreSQL upgrade fixtures applied master's added lifecycle and Slack migrations without changing saved subscription prices or cost receipts. The renamed subscription migration was recognized by hash, retained one history row, and a second migration run was a no-op. - Regression tests cover provider capacity, fractional-cent edits, billing visibility after grant revocation, and fixed-price/credit/unknown usage. The new UI billing-label tests fail against the old wording and pass with the fix. - Browser checks passed for desktop and mobile layouts, shared-account totals, annual price edits, unknown prices, and stale provider data. - A live OpenAI account probe returned HTTP 200 and a verified Pro account identity. The generic Pro plan correctly remains unpriced. - Live Claude discovery identified Max 20x and a $200/month estimate. Two real Claude agent runs were attributed to that same account, with two usage receipts, two agents, one monthly fee, and zero additional usage charges. Two OpenAI connections also deduplicated to one account. - Live Codex execution is not verified: both Paperclip's managed hello test and a direct host `codex exec` fail with a workspace-routing authentication error. The current credential passes the usage API probe. This is separate from the successful account-discovery check. - A local performance fixture with 50,000 usage events and 5,001 saved price revisions produced a 1,589-byte report in 11.5 ms median / 12.2 ms maximum over ten warmed runs. PostgreSQL used the subscription/revision index to read one current price row. These are local measurements, not a production latency guarantee. - [Desktop, account details, and mobile screenshots](https://pages.paperclip.ing/pr-15634-subscription-costs-d5d285e2/) show the production Costs page with synthetic Storybook fixtures. - To inspect the UI, run `pnpm storybook` and open **Costs / Subscriptions**. Edits there change fixture data only. ## Risks - Provider plan identifiers can change. Unknown plans remain unpriced. Automatic prices are estimates before tax; discounts and negotiated prices require a user-supplied amount. - Subscription fees describe the current monthly commitment. They do not follow the selected usage date range and do not consume agent budgets or create finance events. - Monthly fees cover only accounts visible to the viewer; company-wide token totals retain their existing scope. - Legacy or unmanaged usage can remain unattributed. Unconfirmed identities require manual prices and may need explicit duplicate linking after credential changes. - Disconnecting a connection does not prove provider billing stopped. Users must end or exclude tracking explicitly. - Migration 0322 adds tables, a nullable cost-event column, and an index. It is replay-safe, but normal migration locks still apply. ## Model Used OpenAI Codex, based on GPT-6. This session did not expose a more specific backend model ID or context-window size. Capabilities used: code editing, shell execution, database tests, and browser verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
199 lines
15 KiB
TypeScript
199 lines
15 KiB
TypeScript
import { and, eq, inArray, sql } from "drizzle-orm";
|
|
import { aiSubscriptions, aiSubscriptionPrices, aiSubscriptionConnections, connectionGrants, toolConnections, type Db } from "@paperclipai/db";
|
|
import { aiConnectionMetadataSchema, normalizeCents, subscriptionPlan, type AiProvider, type UpdateSubscriptionPrice } from "@paperclipai/shared";
|
|
import { conflict, forbidden, notFound } from "../errors.js";
|
|
import { logActivity, type ActivityPublication } from "./activity-log.js";
|
|
import { publishAccountingActivities } from "./accounting-transaction.js";
|
|
import { subscriptionCredentialIdentity, subscriptionIdentityKey, type SubscriptionIdentity } from "./subscription-identity.js";
|
|
|
|
type Account = typeof aiSubscriptions.$inferSelect;
|
|
type Price = typeof aiSubscriptionPrices.$inferInsert;
|
|
export interface SubscriptionActor { userId: string; canManage: boolean; readOnly?: boolean }
|
|
export interface SubscriptionConnection {
|
|
companyId: string; connectionId: string; grantId: string; provider: AiProvider;
|
|
name: string; ownerUserId: string | null; credential: string;
|
|
}
|
|
|
|
export function canonicalSubscriptionId(id: string, accounts: Pick<Account, "id" | "mergedIntoId">[] | Map<string, string | null>): string {
|
|
const map = accounts instanceof Map ? accounts : new Map(accounts.map(row => [row.id, row.mergedIntoId]));
|
|
const seen = new Set<string>();
|
|
while (map.get(id)) {
|
|
if (seen.has(id)) throw new Error("Subscription link cycle");
|
|
seen.add(id); id = map.get(id)!;
|
|
}
|
|
return id;
|
|
}
|
|
|
|
function detectedPrice(companyId: string, subscriptionId: string, plan: string | null, revision: number): Price {
|
|
return { companyId, subscriptionId, revision, plan: plan ?? "Plan unknown", amountCents: null, source: "unknown" };
|
|
}
|
|
|
|
function autoPrice(account: Pick<Account, "id" | "companyId" | "provider" | "identityVerified">, plan: string | null, revision: number): Price {
|
|
// A plan hint alone cannot prove whether two rotated credentials are one paid
|
|
// seat. Require a stable identity before adding an automatic fee to totals.
|
|
const catalog = account.identityVerified ? subscriptionPlan(account.provider, plan) : null;
|
|
return { ...detectedPrice(account.companyId, account.id, plan, revision),
|
|
...(catalog ? { plan: catalog.name, amountCents: normalizeCents(catalog.monthlyCents), currency: catalog.currency, source: "catalog" as const, sourceUrl: catalog.sourceUrl } : {}),
|
|
};
|
|
}
|
|
|
|
async function lock(db: Db, companyId: string) {
|
|
// A separate, short lock; no provider requests or credential rotation under it.
|
|
await db.execute(sql`select pg_advisory_xact_lock(hashtext('subscription_costs'), hashtext(${companyId}))`);
|
|
}
|
|
|
|
async function withSubscriptionTransaction<T>(db: Db, companyId: string, work: (tx: Db, publications: ActivityPublication[]) => Promise<T>) {
|
|
const publications: ActivityPublication[] = [];
|
|
const result = await db.transaction(async transaction => {
|
|
const tx = transaction as unknown as Db;
|
|
await lock(tx, companyId);
|
|
return work(tx, publications);
|
|
});
|
|
publishAccountingActivities(companyId, publications);
|
|
return result;
|
|
}
|
|
|
|
type Ownership = Pick<Account, "ownerUserIds" | "shared">;
|
|
function combineOwnership(a: Ownership, b: Ownership): Ownership {
|
|
return { ownerUserIds: [...new Set([...a.ownerUserIds, ...b.ownerUserIds])].sort(), shared: a.shared || b.shared };
|
|
}
|
|
|
|
export function canEditSubscription(account: Ownership, actor: SubscriptionActor) {
|
|
return !actor.readOnly && (account.ownerUserIds.includes(actor.userId) || (account.shared && actor.canManage));
|
|
}
|
|
|
|
async function mergeInTransaction(db: Db, companyId: string, source: Account, target: Account, automatic: boolean) {
|
|
if (source.id === target.id) throw conflict("Choose a different subscription to link");
|
|
if (source.provider !== target.provider || source.mergedIntoId || target.mergedIntoId) throw conflict("Choose two current accounts from the same provider");
|
|
await db.update(aiSubscriptions).set(combineOwnership(source, target)).where(and(eq(aiSubscriptions.id, target.id), eq(aiSubscriptions.companyId, companyId)));
|
|
const prices = await db.select().from(aiSubscriptionPrices).where(and(eq(aiSubscriptionPrices.companyId, companyId), inArray(aiSubscriptionPrices.subscriptionId, [source.id, target.id])));
|
|
const sourcePrice = prices.find(row => row.subscriptionId === source.id && row.revision === source.revision)!;
|
|
const targetPrice = prices.find(row => row.subscriptionId === target.id && row.revision === target.revision)!;
|
|
if (automatic && sourcePrice.source === "user") {
|
|
if (targetPrice.source === "user" && ["amountCents", "currency", "cadence", "status"].some(key => sourcePrice[key as keyof typeof sourcePrice] !== targetPrice[key as keyof typeof targetPrice])) {
|
|
throw conflict("These accounts have different saved prices. Link them after choosing the price to keep.");
|
|
}
|
|
if (targetPrice.source !== "user") {
|
|
await db.insert(aiSubscriptionPrices).values({ ...sourcePrice, id: undefined, subscriptionId: target.id, revision: target.revision + 1, effectiveAt: new Date() });
|
|
await db.update(aiSubscriptions).set({ revision: target.revision + 1 }).where(eq(aiSubscriptions.id, target.id));
|
|
}
|
|
}
|
|
await db.update(aiSubscriptions).set({ mergedIntoId: target.id, revision: source.revision + 1 }).where(and(eq(aiSubscriptions.id, source.id), eq(aiSubscriptions.companyId, companyId)));
|
|
// Preserve each old ID for historical run attribution. Flatten existing links.
|
|
await db.update(aiSubscriptions).set({ mergedIntoId: target.id }).where(and(eq(aiSubscriptions.companyId, companyId), eq(aiSubscriptions.mergedIntoId, source.id)));
|
|
await db.update(aiSubscriptionConnections).set({ subscriptionId: target.id }).where(and(eq(aiSubscriptionConnections.companyId, companyId), eq(aiSubscriptionConnections.subscriptionId, source.id)));
|
|
return target.id;
|
|
}
|
|
|
|
export function subscriptionService(db: Db) {
|
|
async function register(input: SubscriptionConnection, observed?: SubscriptionIdentity) {
|
|
const identity = { ...(observed ?? subscriptionCredentialIdentity(input.companyId, input.provider, input.credential)) };
|
|
// Only a successful provider observation may match another grant's billing
|
|
// identity or add its editors. Provisional rows are grant-scoped, even when
|
|
// two callers supply the same unverified credential or forged ID claims.
|
|
if (!observed?.verified) identity.accountKey = subscriptionIdentityKey(input.companyId, input.provider, ["grant", input.grantId, identity.credentialKey]);
|
|
if (!observed) {
|
|
const [cached] = await db.select().from(aiSubscriptionConnections).where(and(eq(aiSubscriptionConnections.companyId, input.companyId), eq(aiSubscriptionConnections.grantId, input.grantId)));
|
|
if (cached?.credentialKey === identity.credentialKey && cached.connectionId === input.connectionId) return cached.subscriptionId;
|
|
}
|
|
return withSubscriptionTransaction(db, input.companyId, async (tx, publications) => {
|
|
// Company and grant identity are authoritative, even for internal callers.
|
|
const [grant] = await tx.select({ grant: connectionGrants, connection: toolConnections }).from(connectionGrants)
|
|
.innerJoin(toolConnections, and(eq(toolConnections.id, connectionGrants.connectionId), eq(toolConnections.companyId, connectionGrants.companyId)))
|
|
.where(and(eq(connectionGrants.companyId, input.companyId), eq(connectionGrants.id, input.grantId), eq(toolConnections.id, input.connectionId)));
|
|
const metadata = aiConnectionMetadataSchema.safeParse(grant?.connection.config.ai);
|
|
if (!grant || grant.grant.status !== "active" || grant.connection.connectionPurpose !== "ai"
|
|
|| !grant.connection.enabled || grant.connection.status !== "active"
|
|
|| !metadata.success || metadata.data.method !== "subscription" || metadata.data.provider !== input.provider) {
|
|
throw notFound("Subscription connection not found");
|
|
}
|
|
const [binding] = await tx.select().from(aiSubscriptionConnections).where(and(eq(aiSubscriptionConnections.companyId, input.companyId), eq(aiSubscriptionConnections.grantId, input.grantId)));
|
|
const accounts = await tx.select().from(aiSubscriptions).where(eq(aiSubscriptions.companyId, input.companyId));
|
|
if (observed && binding && binding.credentialKey !== identity.credentialKey) return canonicalSubscriptionId(binding.subscriptionId, accounts);
|
|
// A fresh credential-local hint cannot override a verified profile binding.
|
|
if (!observed && binding?.credentialKey === identity.credentialKey) return canonicalSubscriptionId(binding.subscriptionId, accounts);
|
|
const known = accounts.find(row => row.provider === input.provider && row.accountKey === identity.accountKey);
|
|
let account = known ? accounts.find(row => row.id === canonicalSubscriptionId(known.id, accounts))! : undefined;
|
|
const previous = binding?.credentialKey === identity.credentialKey ? accounts.find(row => row.id === canonicalSubscriptionId(binding.subscriptionId, accounts)) : undefined;
|
|
if (!account && previous && observed && !previous.identityVerified && !previous.mergedIntoId) {
|
|
[account] = await tx.update(aiSubscriptions).set({ accountKey: identity.accountKey, identityVerified: identity.verified }).where(eq(aiSubscriptions.id, previous.id)).returning();
|
|
}
|
|
if (!account) {
|
|
[account] = await tx.insert(aiSubscriptions).values({ companyId: input.companyId, provider: input.provider,
|
|
accountKey: identity.accountKey, identityVerified: identity.verified, name: grant.connection.name,
|
|
ownerUserIds: grant.grant.subjectUserId ? [grant.grant.subjectUserId] : [], shared: grant.grant.kind === "organization", detectedPlan: identity.plan,
|
|
}).returning();
|
|
await tx.insert(aiSubscriptionPrices).values(autoPrice(account, identity.plan, 0));
|
|
await logActivity(tx, { companyId: input.companyId, actorType: "system", actorId: "subscription_reporting",
|
|
action: "subscription.discovered", entityType: "ai_subscription", entityId: account.id, details: { provider: input.provider } }, publications);
|
|
}
|
|
// The union is independent of discovery order. Credential sharing does
|
|
// not transfer personal ownership, and personal reuse does not remove
|
|
// managers' authority over a fee already connected to the company.
|
|
const ownership = combineOwnership(account, {
|
|
ownerUserIds: grant.grant.subjectUserId ? [grant.grant.subjectUserId] : [], shared: grant.grant.kind === "organization",
|
|
});
|
|
if (ownership.shared !== account.shared || ownership.ownerUserIds.join() !== account.ownerUserIds.join()) {
|
|
[account] = await tx.update(aiSubscriptions).set(ownership).where(eq(aiSubscriptions.id, account.id)).returning();
|
|
await logActivity(tx, { companyId: input.companyId, actorType: "system", actorId: "subscription_reporting",
|
|
action: "subscription.ownership_updated", entityType: "ai_subscription", entityId: account.id,
|
|
details: { shared: ownership.shared, ownerCount: ownership.ownerUserIds.length } }, publications);
|
|
}
|
|
if (observed && previous && previous.id !== account.id && !previous.identityVerified) {
|
|
await mergeInTransaction(tx, input.companyId, previous, account, true);
|
|
await logActivity(tx, { companyId: input.companyId, actorType: "system", actorId: "subscription_reporting",
|
|
action: "subscription.linked", entityType: "ai_subscription", entityId: previous.id, details: { targetId: account.id, automatic: true } }, publications);
|
|
// Merging can adopt a user price and advance the revision.
|
|
[account] = await tx.select().from(aiSubscriptions).where(eq(aiSubscriptions.id, account.id));
|
|
}
|
|
if (observed) {
|
|
if (identity.plan) {
|
|
const [price] = await tx.select().from(aiSubscriptionPrices).where(and(eq(aiSubscriptionPrices.subscriptionId, account.id), eq(aiSubscriptionPrices.revision, account.revision)));
|
|
const next = autoPrice(account, identity.plan, account.revision + 1);
|
|
if (price?.source !== "user" && (price?.plan !== next.plan || price?.amountCents !== (next.amountCents ?? null) || price?.source !== next.source)) {
|
|
account.revision++;
|
|
await tx.insert(aiSubscriptionPrices).values(next);
|
|
}
|
|
}
|
|
await tx.update(aiSubscriptions).set({ detectedPlan: identity.plan ?? account.detectedPlan, observedAt: identity.plan ? new Date() : account.observedAt,
|
|
lastCheckedAt: new Date(), refreshStatus: identity.plan ? "ok" : "unavailable", revision: account.revision }).where(eq(aiSubscriptions.id, account.id));
|
|
}
|
|
await tx.insert(aiSubscriptionConnections).values({ companyId: input.companyId, connectionId: input.connectionId, grantId: input.grantId,
|
|
credentialKey: identity.credentialKey, subscriptionId: account.id }).onConflictDoUpdate({ target: [aiSubscriptionConnections.companyId, aiSubscriptionConnections.grantId],
|
|
set: { credentialKey: identity.credentialKey, subscriptionId: account.id } });
|
|
return account.id;
|
|
});
|
|
}
|
|
|
|
async function updatePrice(companyId: string, id: string, input: UpdateSubscriptionPrice, actor: SubscriptionActor) {
|
|
return withSubscriptionTransaction(db, companyId, async (tx, publications) => {
|
|
const [account] = await tx.select().from(aiSubscriptions).where(and(eq(aiSubscriptions.companyId, companyId), eq(aiSubscriptions.id, id)));
|
|
if (!account) throw notFound("Subscription not found");
|
|
if (!canEditSubscription(account, actor)) throw forbidden("Only subscription owners or managers of shared accounts can edit prices");
|
|
if (account.mergedIntoId || account.revision !== input.expectedRevision) throw conflict("This subscription changed. Reload it before saving.");
|
|
const revision = account.revision + 1;
|
|
await tx.insert(aiSubscriptionPrices).values({ companyId, subscriptionId: id, revision, plan: input.plan,
|
|
amountCents: input.amountCents == null ? null : normalizeCents(input.amountCents), currency: input.currency,
|
|
cadence: input.cadence, status: input.status, source: "user" });
|
|
await tx.update(aiSubscriptions).set({ revision }).where(eq(aiSubscriptions.id, id));
|
|
await logActivity(tx, { companyId, actorType: "user", actorId: actor.userId, action: "subscription.price_updated",
|
|
entityType: "ai_subscription", entityId: id, details: { revision, status: input.status } }, publications);
|
|
return { id, revision };
|
|
});
|
|
}
|
|
|
|
async function merge(companyId: string, id: string, targetId: string, expectedRevision: number, targetRevision: number, actor: SubscriptionActor) {
|
|
return withSubscriptionTransaction(db, companyId, async (tx, publications) => {
|
|
const rows = await tx.select().from(aiSubscriptions).where(and(eq(aiSubscriptions.companyId, companyId), inArray(aiSubscriptions.id, [id, targetId])));
|
|
const source = rows.find(row => row.id === id), target = rows.find(row => row.id === targetId);
|
|
if (!source || !target) throw notFound("Subscription not found");
|
|
if (!canEditSubscription(source, actor) || !canEditSubscription(target, actor)) throw forbidden("You must be able to edit both subscriptions");
|
|
if (source.revision !== expectedRevision || target.revision !== targetRevision) throw conflict("These subscriptions changed. Reload before linking them.");
|
|
const result = await mergeInTransaction(tx, companyId, source, target, false);
|
|
await logActivity(tx, { companyId, actorType: "user", actorId: actor.userId, action: "subscription.linked", entityType: "ai_subscription", entityId: id, details: { targetId } }, publications);
|
|
return { id: result };
|
|
});
|
|
}
|
|
return { register, updatePrice, merge };
|
|
}
|