mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 00:05:49 +02:00
feat: add company decision models with permission and cost controls (#15473)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Optional product features need small, typed model decisions. > - Each company needs to choose which shared API connection pays for those decisions. > - Calls must retain user and task permissions, budget limits, and cost attribution. > - This pull request adds a managed decision service, setup UI, and request history. > - Features can check availability cheaply and keep their existing behavior when decisions are unavailable. ## Linked Issues or Issue Description **Subsystem affected** Server services, shared contracts, database accounting, Company Settings, and Costs. **Problem or motivation** Paperclip has no common decision-model service. Adding provider calls within each feature would duplicate credential access, permission checks, and billing rules. **Proposed solution** Let a connection manager configure one company decision model. Support OpenAI Decisions and Jev through OpenRouter. Provide a fixed setup test and metadata-only history. Default company-sponsored background decisions to on during setup, and preserve a saved off setting. **Alternatives considered** Per-feature credentials would duplicate existing connection management. Personal overrides and provider fallback chains add permission and billing complexity; they remain deferred. **Roadmap alignment** Reviewed ROADMAP.md and searched open PRs. This extends existing connection access and budget accounting. Product features that call the service remain outside this change. No matching decision-model service PR was found. ## What Changed - Add company settings, an internal `decisionModelService`, local availability checks, and trusted human, agent/run, and system contexts. - Pin Vercel AI SDK provider dependencies and adapt boolean, choice, and ordered-score decisions for both providers. Bound requests and time; disable paid retries. - Add durable invocation metadata and agentless decision ledger charges. Preserve fractional cents, pricing evidence, dispatch identity, and unresolved billing holds. - Share the company accounting lock and apply company, agent, and project budgets. Settle charges once, retain unknown holds, and recover interrupted calls without resubmission. - Reuse connection setup and management UI. Add a Decisions view under Costs, production-component Storybook coverage, database migration, and service documentation. ## Verification - Passed 166 current-code tests covering the decision service/provider, setup component, Costs, OpenAPI, and every failure from the earlier broad run. Coverage includes native SDK wire formats, refusals, billed malformed responses, permission and secret-rotation races, identity changes, concurrent budget admission, unresolved holds, agent/task deletion, and stale setup feedback. - Passed 170 existing connection, cost, budget, heartbeat-accounting, and profile regression tests. - Passed repository typecheck, production build, and design token gates after integrating master. Verified the generated migration on a fresh test database and upgraded the populated preview database from the branch's earlier migration without losing settings or usage. - Ran the required full `pnpm test:run`: its general phase completed with 16,354 passed and 10 failures across five files while this branch was still being updated. Every reported failure passes in the current-code rerun; the serialized phase did not run after that failure. The full GitHub CI suite passed on `b1b856a88`: general and serialized tests, browser shards, runner checks, typecheck, production build, packaging/canary, and policy gates. [CI evidence](https://github.com/paperclipai/paperclip/actions/runs/37673106360). - Passed the full-shell Storybook setup-to-history interaction test again after integrating master. Greptile rates the final revision 5/5 with zero unresolved threads. - Walked through the running app: empty setup, add each provider, save, reload, run all three sample questions, inspect fractional charges in history, switch provider while sponsorship is off, disable, and reconnect. Checked mobile settings. These tests used the actual UI, vault, server, SDKs, and database with simulated upstream responses. - Live paid setup tests remain unverified: this environment has no authorized OpenAI/OpenRouter credentials available. No mocked test is presented as live provider evidence. Reviewer journey: Company Settings → General → Decision model. Add/select a shared API connection, save, run the billed sample, open View usage, then disable decisions and verify Run test is disabled after reload. ## Risks - The SDK decision interface is experimental. Pinned versions and wire-format tests limit upgrade drift. - The migration allows agentless service charges and reservations. Existing agent cost-reporting APIs still require an agent, and decision receipts stay separate from run reconciliation. - Timeouts can have unknown provider charges. Holds remain until an audited accounting correction resolves them. - OpenAI prices use a versioned Decisions rate snapshot; OpenRouter costs use provider receipts. Unknown pricing is retained as unknown. - A configured company authorizes background spending by default. Setup explains this, and managers can turn it off. - Live provider account/model availability still needs the two credentialed acceptance checks. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, code execution, and browser tools. The exact serving revision and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
f1c44b7b56
commit
a7a244ab33
51 files changed
+4017
-68
No files matched your search
@@ -2091,3 +2091,9 @@ as free-form text in the instance database. Legacy agents use the default
|
||||
in standard, ask, and planning modes. Submission never changes task disposition
|
||||
or routes feedback externally. See [Agent commentary](agent-commentary.md) for
|
||||
authentication, replay, document-sized limits, inspection, and deletion semantics.
|
||||
|
||||
## Company decision-model service
|
||||
|
||||
Company Settings → General can configure one shared API-key connection for optional internal decisions. V1 supports OpenAI Decisions and Jev through OpenRouter. Companies start unconfigured; background sponsorship defaults on when configured, while an explicit off setting persists. Human and agent calls retain current responsible-user, connection audience, resource, and agent installation checks. Only explicitly registered internal background features can use company sponsorship.
|
||||
|
||||
The internal service provides local-only availability and bounded, reauthorized execution. It records metadata-only invocation history and independent fractional service charges in the existing cost ledger, applying company and applicable agent/project budgets under the accounting lock. Unknown dispatch charges retain reservations until audited resolution. Settings/testing require connection management permission; history uses existing cost visibility. See [decision-models.md](decision-models.md) for the contract, supported models, accounting, privacy, and endpoints.
|
||||
@@ -744,3 +744,7 @@ as free-form text in the instance database. Legacy agents use the default
|
||||
in standard, ask, and planning modes. Submission never changes task disposition
|
||||
or routes feedback externally. See [Agent commentary](agent-commentary.md) for
|
||||
authentication, replay, document-sized limits, inspection, and deletion semantics.
|
||||
|
||||
### Managed decision models
|
||||
|
||||
A company may configure a shared decision model for optional Paperclip features. The instance owns credential resolution, authorization, budget admission, and attributable service charges. Company-sponsored background use is enabled by default during configuration; explicit opt-out persists. User and agent requests keep their own access boundaries and cannot become sponsored background requests after denial. Availability is a cheap local capability check, and metadata-only request history makes service usage inspectable. The implemented V1 contract is in [decision-models.md](decision-models.md).
|
||||
@@ -0,0 +1,73 @@
|
||||
# Company decision models
|
||||
|
||||
Company Settings → General → **Decision model** configures one shared API connection for optional Paperclip features. New companies remain unconfigured. A connection manager selects or adds an OpenAI or OpenRouter connection, saves, and can run a small billed setup test. Test answers appear only in that session; request metadata and charges appear under Activity → Costs → Decisions.
|
||||
|
||||
**Allow company-sponsored background decisions defaults on.** Saving authorizes internal background features to charge the selected connection without a responsible human. An explicit off setting survives later edits, including changing the connection or omitting the field from an update. Deleting the selected connection or grant preserves the setting and blocks calls until a valid replacement is selected. Disabling decisions prevents new calls. It does not cancel or erase calls already dispatched.
|
||||
|
||||
V1 supports native OpenAI Decisions (`gpt-6-luna`) and OpenRouter Decisions (`typesafe/jev-1.13`). It accepts company-owned API keys only. Personal grants, subscription credentials, custom endpoints, fallback chains, images, and public agent/plugin execution tools are outside this version. Decision compatibility is separate from CLI-adapter compatibility.
|
||||
|
||||
## Internal service
|
||||
|
||||
Register a feature once in its server module, then call `decisionModelService(db)` with trusted server context:
|
||||
|
||||
```ts
|
||||
const classifyTask = defineDecisionFeature("tasks.classify");
|
||||
const service = decisionModelService(db, { budgetHooks });
|
||||
const context = { companyId, actor: req.actor, feature: classifyTask, issueId };
|
||||
const available = await service.availability(context);
|
||||
if (!available.available) return existingBehavior();
|
||||
const result = await service.decide(context, {
|
||||
state: { title: task.title },
|
||||
questions: {
|
||||
billing: { type: "boolean", instructions: "Is this about billing?" },
|
||||
team: { type: "choice", instructions: "Which team should handle it?",
|
||||
criteria: { billing: "Payments", support: "Product support" } },
|
||||
urgency: { type: "score", instructions: "How urgent is it?",
|
||||
criteria: ["Routine", "Soon", "Immediately"] },
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
The setup test is the only product caller in this release. Future callers must supply the existing budget enforcement hooks, handle unavailable/failure outcomes, and select their own thresholds. Boolean answers are probabilities, choice answers select a category, and scores use the ordered criteria's zero-based positions. Available distributions and provider confidence are preserved in returned answers. `succeeded` and `failed` results include an invocation ID and usage with reported, estimated, or unknown cost. Refusals and invalid responses never become invented answers.
|
||||
|
||||
`availability` reads local configuration and permissions only. It does not decrypt credentials, contact a provider, or promise reachability/budget admission. `availabilityForRequest()` returns a memoizing function; keep that function and its context object within one request. Dispatch never trusts this cached authorization.
|
||||
|
||||
Human calls use the authenticated user and retain the connection's audience checks. Agent calls reread the active persisted run and its current responsible user, check that user's current membership/resource access, and require an applicable agent/company connection installation. Task and project references must belong to that run/company. Attribution snapshots the effective identity at dispatch. Explicit internal system contexts additionally require a feature registered with `{ background: true }` and company sponsorship. A denied user/agent call cannot fall back to system context.
|
||||
|
||||
Credentials are resolved through the existing secret service immediately before admission, with the actual user/agent/system audit attribution. Authorization, connection/configuration revisions, current run identity, and budgets are rechecked before dispatch. Requests allow 1–20 named questions, text/JSON state, and at most 32 KB serialized request data. Execution is limited to 30 seconds and SDK paid retries are disabled.
|
||||
|
||||
## Accounting and privacy
|
||||
|
||||
`decision_invocations` records feature, actor/responsible user, applicable task/project/agent/run and identity revision, connection/grant, provider/model, request ID, timing, status, question types/count, token counters, and a cost-ledger reference. It does not store state, question text, answers, or raw provider payloads. SDK input/output telemetry is disabled and provider errors are reduced to bounded codes. These are local database records, not first-party telemetry events.
|
||||
|
||||
Cost events have `usageKind: decision` and may have no agent. Agentless reporting uses **Paperclip services**. The public agent cost-reporting API still requires an agent; only the internal service receipt path accepts agentless charges. Decision events snapshot responsible-user attribution; historical run costs continue to use their existing attribution rules. Decision events have no heartbeat receipt reference, so run receipt reconciliation cannot count them again.
|
||||
|
||||
Admission and settlement use the company accounting lock. Company and applicable agent/project budgets include held reservations and observed spend. Reservations use the largest configured applicable reservation amount. Settlement is idempotent. Explicit provider authentication/validation/rate-limit rejection releases the hold when no model work occurred. Uncertain dispatches, timeouts, and unknown prices remain unpriced with the reservation held. Startup/periodic accounting recovery marks stale invocations interrupted without resubmitting them. Use the existing audited cost adjustment workflow to resolve the charge and release its hold, including a verified zero charge when appropriate.
|
||||
|
||||
OpenRouter-reported USD costs are converted to fractional cents. OpenAI uses a dedicated, versioned Decisions rate snapshot: $0.10 per million input tokens for the bounded short requests at the native default endpoint. No chat-model price is substituted. Missing usage or an unsupported rate stays unknown; it is never displayed as a free request. The ledger retains seven decimal places in cents. Pricing provenance accompanies the cost event and remains available to accounting corrections.
|
||||
|
||||
History follows company cost visibility and rechecks task access before returning task/run links. It shows the latest 100 requests in the selected date window. Input/output contents remain absent after refresh.
|
||||
|
||||
## HTTP surface
|
||||
|
||||
All paths are under `/api/companies/:companyId/decision-model`:
|
||||
|
||||
| Method/path | Access and behavior |
|
||||
| --- | --- |
|
||||
| `GET /` | Board; returns manager capability and settings/compatible choices to connection managers |
|
||||
| `PUT /` | Connection manager; validates and audits configuration |
|
||||
| `GET /availability` | Authenticated company caller; local authorization check |
|
||||
| `POST /test` | Connection manager; fixed server-owned three-question sample, ignoring client prompts/identity |
|
||||
| `GET /history` | Existing company cost-read rules; date bounds and limit up to 500 |
|
||||
|
||||
There is no general HTTP decision execution endpoint.
|
||||
|
||||
## Provider dependencies and verification
|
||||
|
||||
The experimental Vercel contract is hidden behind Paperclip-owned types. Dependencies are pinned to `ai@7.0.130`, `@ai-sdk/openai@4.0.86`, and `@openrouter/ai-sdk-provider@3.1.0`. That OpenRouter release calls its compatible model factory `evaluationModel`; the wrapper adapts it to the current Decisions interface. Upgrade these together and rerun native wire-format tests.
|
||||
|
||||
CI owns lockfile updates. Before a direct Docker build of a feature checkout whose dependency manifests changed, refresh the local build context with `pnpm install --resolution-only --ignore-scripts --no-frozen-lockfile`. Do not commit that generated lockfile; the hosted Docker workflow performs the same preparation and the lockfile bot updates master.
|
||||
|
||||
References: [OpenAI Decisions](https://developers.openai.com/api/docs/guides/decisions), [Vercel AI SDK](https://ai-sdk.dev/docs/reference/ai-sdk-core/decide), [OpenRouter Decisions](https://openrouter.ai/docs/api/api-reference/alphadecisions/submit-a-decisions-questions-and-answers-request).
|
||||
|
||||
Automated coverage lives in the provider contract and database-backed decision service tests. Provider tests use actual pinned SDK adapters with simulated HTTP responses; they do not establish live account/model availability. Storybook's **Decision models** group uses production components and includes settings states, metadata history, and a settings-to-history journey. A live acceptance check must run the fixed setup test with each authorized shared provider connection and inspect the corresponding persisted invocation/charge. Record live and simulated evidence separately.
|
||||
@@ -0,0 +1,81 @@
|
||||
-- Idempotent for installations that applied the pre-merge decision migration.
|
||||
CREATE TABLE IF NOT EXISTS "company_decision_models" (
|
||||
"company_id" uuid PRIMARY KEY NOT NULL,
|
||||
"connection_id" uuid,
|
||||
"grant_id" uuid,
|
||||
"enabled" boolean DEFAULT false NOT NULL,
|
||||
"allow_background" boolean DEFAULT true NOT NULL,
|
||||
"provider" text,
|
||||
"model" text,
|
||||
"updated_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||
CONSTRAINT "company_decisions_configured" CHECK (("company_decision_models"."connection_id" is null) = ("company_decision_models"."grant_id" is null) and (not "company_decision_models"."enabled" or ("company_decision_models"."connection_id" is not null and "company_decision_models"."provider" is not null and "company_decision_models"."model" is not null)))
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE IF NOT EXISTS "decision_invocations" (
|
||||
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
|
||||
"company_id" uuid NOT NULL,
|
||||
"feature" text NOT NULL,
|
||||
"actor_type" text NOT NULL,
|
||||
"actor_id" text NOT NULL,
|
||||
"responsible_user_id" text,
|
||||
"agent_id" uuid,
|
||||
"issue_id" uuid,
|
||||
"project_id" uuid,
|
||||
"run_id" uuid,
|
||||
"identity_context_id" uuid,
|
||||
"connection_id" uuid NOT NULL,
|
||||
"grant_id" uuid NOT NULL,
|
||||
"provider" text NOT NULL,
|
||||
"model" text NOT NULL,
|
||||
"question_types" jsonb NOT NULL,
|
||||
"status" text DEFAULT 'running' NOT NULL,
|
||||
"error_code" text,
|
||||
"provider_request_id" text,
|
||||
"cost_event_id" uuid,
|
||||
"started_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||
"finished_at" timestamp with time zone,
|
||||
"duration_ms" integer,
|
||||
"input_tokens" integer,
|
||||
"output_tokens" integer,
|
||||
CONSTRAINT "decision_invocations_status_check" CHECK ("decision_invocations"."status" in ('running', 'succeeded', 'failed', 'unknown'))
|
||||
);
|
||||
--> statement-breakpoint
|
||||
ALTER TABLE "budget_reservations" ALTER COLUMN "run_id" DROP NOT NULL;--> statement-breakpoint
|
||||
ALTER TABLE "budget_reservations" ALTER COLUMN "agent_id" DROP NOT NULL;--> statement-breakpoint
|
||||
ALTER TABLE "cost_events" ALTER COLUMN "agent_id" DROP NOT NULL;--> statement-breakpoint
|
||||
ALTER TABLE "budget_reservations" ADD COLUMN IF NOT EXISTS "decision_invocation_id" uuid;--> statement-breakpoint
|
||||
ALTER TABLE "cost_events" ADD COLUMN IF NOT EXISTS "usage_kind" text DEFAULT 'agent' NOT NULL;--> statement-breakpoint
|
||||
ALTER TABLE "cost_events" ADD COLUMN IF NOT EXISTS "responsible_user_id" text;--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
IF NOT EXISTS (SELECT 1 FROM pg_constraint WHERE conname = 'company_decision_models_company_id_companies_id_fk' AND conrelid = 'company_decision_models'::regclass) THEN
|
||||
ALTER TABLE "company_decision_models" ADD CONSTRAINT "company_decision_models_company_id_companies_id_fk" FOREIGN KEY ("company_id") REFERENCES "public"."companies"("id") ON DELETE cascade ON UPDATE no action;
|
||||
END IF;
|
||||
END $$;--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
IF NOT EXISTS (SELECT 1 FROM pg_constraint WHERE conname = 'decision_invocations_company_id_companies_id_fk' AND conrelid = 'decision_invocations'::regclass) THEN
|
||||
ALTER TABLE "decision_invocations" ADD CONSTRAINT "decision_invocations_company_id_companies_id_fk" FOREIGN KEY ("company_id") REFERENCES "public"."companies"("id") ON DELETE cascade ON UPDATE no action;
|
||||
END IF;
|
||||
END $$;--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
IF NOT EXISTS (SELECT 1 FROM pg_constraint WHERE conname = 'decision_invocations_cost_event_id_cost_events_id_fk' AND conrelid = 'decision_invocations'::regclass) THEN
|
||||
ALTER TABLE "decision_invocations" ADD CONSTRAINT "decision_invocations_cost_event_id_cost_events_id_fk" FOREIGN KEY ("cost_event_id") REFERENCES "public"."cost_events"("id") ON DELETE no action ON UPDATE no action;
|
||||
END IF;
|
||||
END $$;--> statement-breakpoint
|
||||
CREATE INDEX IF NOT EXISTS "decision_invocations_company_started_idx" ON "decision_invocations" USING btree ("company_id","started_at","id");--> statement-breakpoint
|
||||
CREATE INDEX IF NOT EXISTS "decision_invocations_pending_idx" ON "decision_invocations" USING btree ("started_at") WHERE "decision_invocations"."status" = 'running';--> statement-breakpoint
|
||||
CREATE INDEX IF NOT EXISTS "decision_invocations_cost_idx" ON "decision_invocations" USING btree ("cost_event_id");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS "budget_reservations_decision_idx" ON "budget_reservations" USING btree ("company_id","decision_invocation_id");--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
IF NOT EXISTS (SELECT 1 FROM pg_constraint WHERE conname = 'budget_reservations_source_check' AND conrelid = 'budget_reservations'::regclass) THEN
|
||||
ALTER TABLE "budget_reservations" ADD CONSTRAINT "budget_reservations_source_check" CHECK (("budget_reservations"."run_id" is not null)::int + ("budget_reservations"."decision_invocation_id" is not null)::int = 1);
|
||||
END IF;
|
||||
END $$;--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
IF NOT EXISTS (SELECT 1 FROM pg_constraint WHERE conname = 'cost_events_usage_kind_check' AND conrelid = 'cost_events'::regclass) THEN
|
||||
ALTER TABLE "cost_events" ADD CONSTRAINT "cost_events_usage_kind_check" CHECK ("cost_events"."usage_kind" = 'decision' or ("cost_events"."usage_kind" = 'agent' and "cost_events"."agent_id" is not null));
|
||||
END IF;
|
||||
END $$;
|
||||
--> statement-breakpoint
|
||||
ALTER TABLE "company_decision_models" DROP CONSTRAINT IF EXISTS "company_decision_models_connection_id_tool_connections_id_fk";
|
||||
--> statement-breakpoint
|
||||
ALTER TABLE "company_decision_models" DROP CONSTRAINT IF EXISTS "company_decision_models_grant_id_connection_grants_id_fk";
|
||||
+2371
-15
File diff suppressed because it is too large.
Load diff
+7
@@ -2199,6 +2199,13 @@
|
||||
"when": 1791390905429,
|
||||
"tag": "0315_rapid_emma_frost",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 316,
|
||||
"version": "7",
|
||||
"when": 1791400295370,
|
||||
"tag": "0316_premium_slayback",
|
||||
"breakpoints": true
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -52,8 +52,9 @@ export const accountingRuntimeBaselines = pgTable("accounting_runtime_baselines"
|
||||
export const budgetReservations = pgTable("budget_reservations", {
|
||||
id: uuid("id").primaryKey().defaultRandom(),
|
||||
companyId: uuid("company_id").notNull().references(() => companies.id, { onDelete: "cascade" }),
|
||||
runId: uuid("run_id").notNull().references(() => heartbeatRuns.id, { onDelete: "cascade" }),
|
||||
agentId: uuid("agent_id").notNull().references(() => agents.id, { onDelete: "cascade" }),
|
||||
runId: uuid("run_id").references(() => heartbeatRuns.id, { onDelete: "cascade" }),
|
||||
decisionInvocationId: uuid("decision_invocation_id"),
|
||||
agentId: uuid("agent_id").references(() => agents.id, { onDelete: "cascade" }),
|
||||
projectId: uuid("project_id"),
|
||||
amountCents: numeric("amount_cents", { precision: 24, scale: 7 }).notNull(),
|
||||
state: text("state").notNull().default("held"),
|
||||
@@ -61,6 +62,8 @@ export const budgetReservations = pgTable("budget_reservations", {
|
||||
settledAt: timestamp("settled_at", { withTimezone: true }),
|
||||
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
|
||||
}, table => ({
|
||||
decision: uniqueIndex("budget_reservations_decision_idx").on(table.companyId, table.decisionInvocationId),
|
||||
source: check("budget_reservations_source_check", sql`(${table.runId} is not null)::int + (${table.decisionInvocationId} is not null)::int = 1`),
|
||||
run: uniqueIndex("budget_reservations_run_idx").on(table.companyId, table.runId),
|
||||
active: index("budget_reservations_active_idx").on(table.companyId, table.agentId, table.projectId).where(sql`${table.state} = 'held'`),
|
||||
stateCheck: check("budget_reservations_state_check", sql`${table.state} in ('held', 'settled', 'released')`),
|
||||
|
||||
@@ -12,7 +12,9 @@ export const costEvents = pgTable(
|
||||
{
|
||||
id: uuid("id").primaryKey().defaultRandom(),
|
||||
companyId: uuid("company_id").notNull().references(() => companies.id),
|
||||
agentId: uuid("agent_id").notNull().references(() => agents.id),
|
||||
agentId: uuid("agent_id").references(() => agents.id),
|
||||
usageKind: text("usage_kind").notNull().default("agent"),
|
||||
responsibleUserId: text("responsible_user_id"),
|
||||
issueId: uuid("issue_id").references(() => issues.id, { onDelete: "set null" }),
|
||||
projectId: uuid("project_id").references(() => projects.id),
|
||||
goalId: uuid("goal_id").references(() => goals.id),
|
||||
@@ -36,6 +38,7 @@ export const costEvents = pgTable(
|
||||
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
|
||||
},
|
||||
(table) => ({
|
||||
usageKindCheck: check("cost_events_usage_kind_check", sql`${table.usageKind} = 'decision' or (${table.usageKind} = 'agent' and ${table.agentId} is not null)`),
|
||||
receiptUniqueIdx: uniqueIndex("cost_events_company_receipt_idx").on(table.companyId, table.idempotencyKey),
|
||||
providerRequestIdx: index("cost_events_provider_request_idx").on(table.companyId, table.biller, table.providerRequestId),
|
||||
nonnegativeAmounts: check("cost_events_nonnegative_amounts", sql`${table.costCents} >= 0 and ${table.inputTokens} >= 0 and ${table.cachedInputTokens} >= 0 and ${table.outputTokens} >= 0`),
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
import { sql } from "drizzle-orm";
|
||||
import { boolean, check, index, integer, jsonb, pgTable, text, timestamp, uuid } from "drizzle-orm/pg-core";
|
||||
import { companies } from "./companies.js";
|
||||
import { costEvents } from "./cost_events.js";
|
||||
|
||||
export const companyDecisionModels = pgTable("company_decision_models", {
|
||||
companyId: uuid("company_id").primaryKey().references(() => companies.id, { onDelete: "cascade" }),
|
||||
// Retain company preferences when a connection/grant is deleted. Resolution
|
||||
// fails closed for stale IDs; choosing a replacement preserves sponsorship.
|
||||
connectionId: uuid("connection_id"),
|
||||
grantId: uuid("grant_id"),
|
||||
enabled: boolean("enabled").notNull().default(false),
|
||||
allowBackground: boolean("allow_background").notNull().default(true),
|
||||
provider: text("provider"),
|
||||
model: text("model"),
|
||||
updatedAt: timestamp("updated_at", { withTimezone: true }).notNull().defaultNow(),
|
||||
}, t => ({ configured: check("company_decisions_configured", sql`(${t.connectionId} is null) = (${t.grantId} is null) and (not ${t.enabled} or (${t.connectionId} is not null and ${t.provider} is not null and ${t.model} is not null))`) }));
|
||||
|
||||
/** Attribution references are archival: history survives deleted runs and connections. No input or answer content. */
|
||||
export const decisionInvocations = pgTable("decision_invocations", {
|
||||
id: uuid("id").primaryKey().defaultRandom(),
|
||||
companyId: uuid("company_id").notNull().references(() => companies.id, { onDelete: "cascade" }),
|
||||
feature: text("feature").notNull(),
|
||||
actorType: text("actor_type").notNull(),
|
||||
actorId: text("actor_id").notNull(),
|
||||
responsibleUserId: text("responsible_user_id"),
|
||||
agentId: uuid("agent_id"),
|
||||
issueId: uuid("issue_id"),
|
||||
projectId: uuid("project_id"),
|
||||
runId: uuid("run_id"),
|
||||
identityContextId: uuid("identity_context_id"),
|
||||
connectionId: uuid("connection_id").notNull(),
|
||||
grantId: uuid("grant_id").notNull(),
|
||||
provider: text("provider").notNull(),
|
||||
model: text("model").notNull(),
|
||||
questionTypes: jsonb("question_types").$type<string[]>().notNull(),
|
||||
status: text("status").notNull().default("running"),
|
||||
errorCode: text("error_code"),
|
||||
providerRequestId: text("provider_request_id"),
|
||||
costEventId: uuid("cost_event_id").references(() => costEvents.id),
|
||||
startedAt: timestamp("started_at", { withTimezone: true }).notNull().defaultNow(),
|
||||
finishedAt: timestamp("finished_at", { withTimezone: true }),
|
||||
durationMs: integer("duration_ms"),
|
||||
inputTokens: integer("input_tokens"),
|
||||
outputTokens: integer("output_tokens"),
|
||||
}, t => ({
|
||||
history: index("decision_invocations_company_started_idx").on(t.companyId, t.startedAt, t.id),
|
||||
pending: index("decision_invocations_pending_idx").on(t.startedAt).where(sql`${t.status} = 'running'`),
|
||||
cost: index("decision_invocations_cost_idx").on(t.costEventId),
|
||||
statusCheck: check("decision_invocations_status_check", sql`${t.status} in ('running', 'succeeded', 'failed', 'unknown')`),
|
||||
}));
|
||||
@@ -233,5 +233,5 @@ export { agentCommentary } from "./agent_commentary.js";
|
||||
|
||||
|
||||
export { agentIdentityKeys } from "./agent_identity_keys.js";
|
||||
|
||||
export * from "./decision_models.js";
|
||||
export { userCompanyPreferences } from "./user_company_preferences.js";
|
||||
@@ -87,7 +87,7 @@ export interface AccountingHealth {
|
||||
oldestPendingAt: string | null;
|
||||
pendingCancellationCount: number;
|
||||
heldReservationCents: string;
|
||||
items: Array<{ runId: string | null; costEventId?: string; agentId: string; state: "waiting_for_receipt" | "retryable" | "unpriced"; lastError: string | null; attempts: number; since: string; lastAttemptAt: string | null }>;
|
||||
items: Array<{ runId: string | null; costEventId?: string; agentId: string | null; state: "waiting_for_receipt" | "retryable" | "unpriced"; lastError: string | null; attempts: number; since: string; lastAttemptAt: string | null }>;
|
||||
}
|
||||
export interface BillingInvoice {
|
||||
id: string; companyId: string; biller: string; externalId: string; currency: string; createdAt: string;
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
import { z } from "zod";
|
||||
import type { AiManagedConnectionSummary } from "./ai-connections.js";
|
||||
|
||||
export const DECISION_MODELS = {
|
||||
openai: { id: "gpt-6-luna", name: "GPT-6 Luna" },
|
||||
openrouter: { id: "typesafe/jev-1.13", name: "Jev 1.13" },
|
||||
} as const;
|
||||
export type DecisionProvider = keyof typeof DECISION_MODELS;
|
||||
export const updateDecisionModelSchema = z.object({
|
||||
enabled: z.boolean(),
|
||||
connectionId: z.string().uuid().nullable(),
|
||||
grantId: z.string().uuid().nullable(),
|
||||
allowBackground: z.boolean().optional(),
|
||||
}).refine(v => Boolean(v.connectionId) === Boolean(v.grantId) && (!v.enabled || Boolean(v.connectionId)), {
|
||||
message: "Select a connection before enabling decisions",
|
||||
});
|
||||
export type UpdateDecisionModel = z.infer<typeof updateDecisionModelSchema>;
|
||||
export interface DecisionModelSettings extends UpdateDecisionModel {
|
||||
allowBackground: boolean;
|
||||
companyId: string;
|
||||
provider: DecisionProvider | null;
|
||||
model: string | null;
|
||||
}
|
||||
export type DecisionConnectionChoice = AiManagedConnectionSummary & { decisionModel: string };
|
||||
export type DecisionUnavailableReason = "not_configured" | "disabled" | "connection_unavailable" | "incompatible_connection" | "access_denied" | "responsible_user_missing" | "background_disabled" | "budget_blocked";
|
||||
export type DecisionAvailability = { available: true } | { available: false; reason: DecisionUnavailableReason };
|
||||
const instructions = z.string().trim().min(1).max(8000);
|
||||
export const decisionQuestionSchema = z.discriminatedUnion("type", [
|
||||
z.object({ type: z.literal("boolean"), instructions }),
|
||||
z.object({ type: z.literal("choice"), instructions, criteria: z.record(z.string().min(1).max(100), z.string().min(1).max(2000)).refine(v => Object.keys(v).length >= 2 && Object.keys(v).length <= 20) }),
|
||||
z.object({ type: z.literal("score"), instructions, criteria: z.array(z.string().min(1).max(2000)).min(2).max(20) }),
|
||||
]);
|
||||
export const decisionRequestSchema = z.object({
|
||||
state: z.union([z.string().min(1), z.record(z.string(), z.json()), z.array(z.json())]),
|
||||
questions: z.record(z.string().regex(/^[a-zA-Z][a-zA-Z0-9_]{0,63}$/), decisionQuestionSchema)
|
||||
.refine(v => Object.keys(v).length > 0 && Object.keys(v).length <= 20),
|
||||
});
|
||||
export type DecisionRequest = z.infer<typeof decisionRequestSchema>;
|
||||
export type DecisionAnswer =
|
||||
| { type: "boolean"; probability: number; confidence?: number }
|
||||
| { type: "choice"; choice: string; probabilities?: Record<string, number>; confidence?: number }
|
||||
| { type: "score"; score: number; probabilities?: Record<string, number>; confidence?: number };
|
||||
export interface DecisionUsage {
|
||||
inputTokens: number | null; outputTokens: number | null; costCents: string | null;
|
||||
costStatus: "reported" | "estimated" | "unpriced";
|
||||
}
|
||||
export type DecisionResult =
|
||||
| { status: "succeeded"; invocationId: string; answers: Record<string, DecisionAnswer>; usage: DecisionUsage }
|
||||
| { status: "unavailable"; reason: DecisionUnavailableReason; invocationId?: string }
|
||||
| { status: "failed"; invocationId: string; errorCode: string; usage: DecisionUsage };
|
||||
export interface DecisionHistoryEntry {
|
||||
id: string; feature: string; actorType: string; responsibleUserId: string | null; userName: string | null;
|
||||
issueId: string | null; issueIdentifier: string | null; agentId: string | null; runId: string | null;
|
||||
connectionId: string; provider: string; model: string; status: string; errorCode: string | null;
|
||||
startedAt: string; finishedAt: string | null; durationMs: number | null;
|
||||
inputTokens: number | null; outputTokens: number | null; costCents: string | null; costStatus: string | null;
|
||||
}
|
||||
export const DECISION_TEST_REQUEST: DecisionRequest = {
|
||||
state: "A customer reports being charged twice for an order. The product still works.",
|
||||
questions: {
|
||||
billing: { type: "boolean", instructions: "Is this a billing problem?" },
|
||||
team: { type: "choice", instructions: "Which team should handle the request?", criteria: { billing: "Charges and payments", technical: "Product failures" } },
|
||||
severity: { type: "score", instructions: "How severe is the product disruption?", criteria: ["Product works", "Some functionality is unavailable", "Product cannot be used"] },
|
||||
},
|
||||
};
|
||||
|
||||
/** Native endpoints only; OpenRouter's official routing metadata is compatible. */
|
||||
export function decisionProviderForConnection(metadata: { provider: string; method: string; routing?: { kind: string; baseUrl?: string } }): DecisionProvider | null {
|
||||
if (metadata.method !== "api_key") return null;
|
||||
if (metadata.provider === "openai" && !metadata.routing) return "openai";
|
||||
if (metadata.provider === "openrouter" && (!metadata.routing || (metadata.routing.kind === "openrouter" && !metadata.routing.baseUrl))) return "openrouter";
|
||||
return null;
|
||||
}
|
||||
@@ -2348,6 +2348,7 @@ export {
|
||||
type RunRoutine,
|
||||
type RotateRoutineTriggerSecret,
|
||||
createCostEventSchema,
|
||||
createServiceCostEventSchema,
|
||||
createFinanceEventSchema,
|
||||
updateBudgetSchema,
|
||||
ASSET_NAMESPACE_MAX_LENGTH,
|
||||
@@ -2850,5 +2851,5 @@ export { isAppAggregator, aggregatorManagementUrl, aggregatorAppsSyncSchema, agg
|
||||
|
||||
export * from "./connection-instructions.js";
|
||||
export * from "./customer-success.js";
|
||||
|
||||
export * from "./decision-models.js";
|
||||
export { updatePrimaryAgentSchema, type UpdatePrimaryAgent, type PrimaryAgentPreference } from "./primary-agent.js";
|
||||
@@ -2,9 +2,11 @@ import type { AgentAppearance } from "../agent-appearance.js";
|
||||
import type { BillingType, CostStatus } from "../constants.js";
|
||||
|
||||
export interface CostEvent {
|
||||
usageKind?: "agent" | "decision";
|
||||
responsibleUserId?: string | null;
|
||||
id: string;
|
||||
companyId: string;
|
||||
agentId: string;
|
||||
agentId: string | null;
|
||||
issueId: string | null;
|
||||
projectId: string | null;
|
||||
goalId: string | null;
|
||||
@@ -80,7 +82,7 @@ export interface CostByUserReport {
|
||||
}
|
||||
|
||||
export interface CostByAgent {
|
||||
agentId: string;
|
||||
agentId: string | null;
|
||||
agentName: string | null;
|
||||
agentAppearance?: AgentAppearance | null;
|
||||
avatarUrl?: string;
|
||||
@@ -135,7 +137,7 @@ export interface CostByBiller {
|
||||
|
||||
/** per-agent breakdown by provider + model, for identifying token-hungry agents */
|
||||
export interface CostByAgentModel {
|
||||
agentId: string;
|
||||
agentId: string | null;
|
||||
agentName: string | null;
|
||||
agentAppearance?: AgentAppearance | null;
|
||||
avatarUrl?: string;
|
||||
|
||||
@@ -59,7 +59,7 @@ export interface UserProfileActivitySummary {
|
||||
}
|
||||
|
||||
export interface UserProfileAgentUsage {
|
||||
agentId: string;
|
||||
agentId: string | null;
|
||||
agentName: string | null;
|
||||
costCents: number;
|
||||
inputTokens: number;
|
||||
|
||||
@@ -2,7 +2,7 @@ import { exactCentsSchema, pricingProvenanceSchema } from "../accounting.js";
|
||||
import { z } from "zod";
|
||||
import { BILLING_TYPES, COST_STATUSES } from "../constants.js";
|
||||
|
||||
export const createCostEventSchema = z.object({
|
||||
const costEventFields = z.object({
|
||||
idempotencyKey: z.string().trim().min(1).max(200).optional().nullable(),
|
||||
agentId: z.string().guid(),
|
||||
issueId: z.string().guid().optional().nullable(),
|
||||
@@ -22,11 +22,20 @@ export const createCostEventSchema = z.object({
|
||||
providerRequestId: z.string().min(1).max(250).nullable().optional(),
|
||||
pricingProvenance: pricingProvenanceSchema.nullable().optional(),
|
||||
occurredAt: z.string().datetime(),
|
||||
}).transform((value) => ({
|
||||
});
|
||||
|
||||
export const createCostEventSchema = costEventFields.transform((value) => ({
|
||||
...value,
|
||||
biller: value.biller ?? value.provider,
|
||||
}));
|
||||
|
||||
/** Internal service receipts; the public reporting endpoint retains its required agent. */
|
||||
export const createServiceCostEventSchema = costEventFields.extend({
|
||||
agentId: z.string().uuid().nullable(),
|
||||
usageKind: z.literal("decision"),
|
||||
responsibleUserId: z.string().nullable(),
|
||||
}).transform(value => ({ ...value, biller: value.biller ?? value.provider }));
|
||||
|
||||
export type CreateCostEvent = z.input<typeof createCostEventSchema>;
|
||||
|
||||
export const updateBudgetSchema = z.object({
|
||||
|
||||
@@ -714,7 +714,7 @@ export {
|
||||
} from "./routine.js";
|
||||
|
||||
export {
|
||||
createCostEventSchema,
|
||||
createCostEventSchema, createServiceCostEventSchema,
|
||||
updateBudgetSchema,
|
||||
type CreateCostEvent,
|
||||
type UpdateBudget,
|
||||
|
||||
@@ -44,6 +44,9 @@
|
||||
"typecheck": "pnpm run prepare:runner-vendor && pnpm --filter @paperclipai/plugin-sdk ensure-build-deps && tsc --noEmit"
|
||||
},
|
||||
"dependencies": {
|
||||
"ai": "7.0.130",
|
||||
"@ai-sdk/openai": "4.0.86",
|
||||
"@openrouter/ai-sdk-provider": "3.1.0",
|
||||
"@aws-sdk/client-s3": "^3.1141.0",
|
||||
"@chat-adapter/discord": "4.39.0",
|
||||
"@chat-adapter/github": "4.39.0",
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { DECISION_MODELS, DECISION_TEST_REQUEST, decisionProviderForConnection } from "@paperclipai/shared";
|
||||
import { decisionReceipt, runDecisionProvider } from "../services/decision-model-provider.js";
|
||||
|
||||
function openaiResponse() {
|
||||
return { model: "gpt-6-luna", usage: { input_tokens: 123, output_tokens: 0 }, answers: [
|
||||
{ type: "predicate", name: "billing", probability: 0.99 },
|
||||
{ type: "choice", name: "team", choice: "billing", confidence: 0.9, probabilities: [{ value: "billing", probability: 0.9 }, { value: "technical", probability: 0.1 }] },
|
||||
{ type: "score", name: "severity", score: 0.1, probabilities: [{ value: 0, probability: 0.9 }, { value: 1, probability: 0.1 }, { value: 2, probability: 0 }] },
|
||||
] };
|
||||
}
|
||||
function openrouterResponse() {
|
||||
return { id: "decision-fixture", model: "typesafe/jev-1.13", usage: { input_tokens: 123, output_tokens: 0, cost: 0.000005166 }, answers: {
|
||||
billing: { type: "noul", noul: 0.99 }, team: { type: "choice", choice: "billing", confidence: 0.85, probabilities: { billing: 0.9, technical: 0.1 } },
|
||||
severity: { type: "score", score: 0.1, probabilities: { "0": 0.9, "1": 0.1, "2": 0 } },
|
||||
} };
|
||||
}
|
||||
describe("decision provider contract", () => {
|
||||
it.each(["openai", "openrouter"] as const)("uses native %s decisions with the selected credential and preserves answers and fractional costs", async provider => {
|
||||
const transport = vi.fn<typeof fetch>().mockResolvedValue(Response.json(provider === "openai" ? openaiResponse() : openrouterResponse()));
|
||||
const result = await runDecisionProvider({ provider, model: DECISION_MODELS[provider].id, apiKey: "selected-credential", request: DECISION_TEST_REQUEST, signal: new AbortController().signal }, { fetch: transport });
|
||||
expect(transport).toHaveBeenCalledTimes(1);
|
||||
const [url, options] = transport.mock.calls[0];
|
||||
expect(String(url)).toBe(provider === "openai" ? "https://api.openai.com/v1/decisions" : "https://openrouter.ai/api/alpha/decisions");
|
||||
expect(new Headers(options?.headers).get("authorization")).toBe("Bearer selected-credential");
|
||||
expect(result.answers).toMatchObject({ billing: { type: "boolean", probability: 0.99 }, team: { type: "choice", choice: "billing", confidence: provider === "openai" ? 0.9 : 0.85 }, severity: { type: "score", score: 0.1 } });
|
||||
expect(result.receipt.costCents).toBe(provider === "openai" ? "0.0012300" : "0.0005166");
|
||||
expect(JSON.stringify(result.receipt)).not.toContain("customer");
|
||||
expect(JSON.parse(String(options?.body)).model).toBe(DECISION_MODELS[provider].id);
|
||||
});
|
||||
it("keeps usage for a charged refusal and never treats it as an answer", async () => {
|
||||
const transport = vi.fn<typeof fetch>().mockResolvedValue(Response.json({ ...openaiResponse(), answers: [{ type: "refusal", name: null }] }));
|
||||
const result = await runDecisionProvider({ provider: "openai", model: "gpt-6-luna", apiKey: "key", request: DECISION_TEST_REQUEST, signal: new AbortController().signal }, { fetch: transport });
|
||||
expect(result.answers).toBeUndefined(); expect(result.errorCode).toBe("refused"); expect(result.receipt.inputTokens).toBe(123);
|
||||
});
|
||||
it.each([401, 429, 500])("does not retry HTTP %i or retain its response body", async status => {
|
||||
const transport = vi.fn<typeof fetch>().mockResolvedValue(Response.json({ error: { message: "PRIVATE STATE AND KEY", type: "server_error" } }, { status }));
|
||||
const result = await runDecisionProvider({ provider: "openai", model: "gpt-6-luna", apiKey: "key", request: DECISION_TEST_REQUEST, signal: new AbortController().signal }, { fetch: transport });
|
||||
expect(transport).toHaveBeenCalledTimes(1); expect(result.receipt.costStatus).toBe(status === 500 ? "unpriced" : "estimated"); expect(result.noProviderWork).toBe(status !== 500); expect(JSON.stringify(result)).not.toContain("PRIVATE");
|
||||
});
|
||||
it("preserves usage when answer validation fails", async () => {
|
||||
const response = openrouterResponse(); response.answers.team.choice = "invented";
|
||||
const result = await runDecisionProvider({ provider: "openrouter", model: "typesafe/jev-1.13", apiKey: "key", request: DECISION_TEST_REQUEST, signal: new AbortController().signal }, { fetch: async () => Response.json(response) });
|
||||
expect(result.answers).toBeUndefined(); expect(result.receipt.costStatus).toBe("reported");
|
||||
});
|
||||
it.each(["openai", "openrouter"] as const)("retains the %s receipt when the SDK rejects malformed wire answers", async provider => {
|
||||
const wire = provider === "openai" ? { ...openaiResponse(), answers: [{ type: "predicate", name: "billing", probability: 2 }] }
|
||||
: { ...openrouterResponse(), answers: { billing: { type: "noul", noul: "not-a-probability" } } };
|
||||
const result = await runDecisionProvider({ provider, model: DECISION_MODELS[provider].id, apiKey: "key", request: DECISION_TEST_REQUEST, signal: new AbortController().signal }, { fetch: async () => Response.json(wire) });
|
||||
expect(result.answers).toBeUndefined(); expect(result.receipt.inputTokens).toBe(123);
|
||||
expect(result.receipt.costStatus).toBe(provider === "openai" ? "estimated" : "reported");
|
||||
});
|
||||
it("fails closed on an OpenRouter refusal while retaining any reported charge", async () => {
|
||||
const result = await runDecisionProvider({ provider: "openrouter", model: DECISION_MODELS.openrouter.id, apiKey: "key", request: DECISION_TEST_REQUEST, signal: new AbortController().signal },
|
||||
{ fetch: async () => Response.json({ ...openrouterResponse(), answers: { billing: { type: "refusal" } } }) });
|
||||
expect(result.answers).toBeUndefined(); expect(result.errorCode).toBe("refused"); expect(result.receipt.costStatus).toBe("reported");
|
||||
});
|
||||
it("does not invent prices when usage or a supported rate is missing", () => {
|
||||
expect(decisionReceipt("openai", {}).costCents).toBeNull();
|
||||
expect(decisionReceipt("openai", { usage: { inputTokens: 100_000 } }).costStatus).toBe("unpriced");
|
||||
expect(decisionReceipt("openrouter", { usage: { inputTokens: 1 } }).costCents).toBeNull();
|
||||
expect(decisionReceipt("openrouter", { providerMetadata: { openrouter: { usage: { cost: 0 } } } }).costStatus).toBe("reported");
|
||||
});
|
||||
it("rejects subscription, personal endpoint routing and other provider capabilities", () => {
|
||||
expect(decisionProviderForConnection({ provider: "openai", method: "subscription" })).toBeNull();
|
||||
expect(decisionProviderForConnection({ provider: "openai", method: "api_key", routing: { kind: "gateway", baseUrl: "https://example.com" } })).toBeNull();
|
||||
expect(decisionProviderForConnection({ provider: "anthropic", method: "api_key" })).toBeNull();
|
||||
expect(decisionProviderForConnection({ provider: "openrouter", method: "api_key", routing: { kind: "openrouter" } })).toBe("openrouter");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,291 @@
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { mkdtemp, realpath, rm } from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { and, eq, sql } from "drizzle-orm";
|
||||
import {
|
||||
createDb, companies, agents, companyMemberships, companyDecisionModels, costEvents, decisionInvocations,
|
||||
connectionGrants, connectionGrantMembers, toolConnectionInstalls, budgetPolicies, budgetReservations,
|
||||
heartbeatRuns, issues, projects, runIdentityContexts, secretAccessEvents, companySecrets,
|
||||
} from "@paperclipai/db";
|
||||
import { startEmbeddedPostgresTestDatabase } from "@paperclipai/db/test-embedded-postgres";
|
||||
import { DECISION_TEST_REQUEST, updateDecisionModelSchema, createCostEventSchema } from "@paperclipai/shared";
|
||||
import { aiConnectionService } from "../services/ai-connections.js";
|
||||
import { agentService } from "../services/agents.js";
|
||||
import { decisionModelService, settingsDecisionTest, defineDecisionFeature, type DecisionContext } from "../services/decision-models.js";
|
||||
import { decisionReceipt, type DecisionProviderOutcome } from "../services/decision-model-provider.js";
|
||||
import { costService } from "../services/costs.js";
|
||||
import { billingReconciliationService } from "../services/billing-reconciliation.js";
|
||||
import { withAccountingTransaction } from "../services/accounting-transaction.js";
|
||||
import express from "express";
|
||||
import request from "supertest";
|
||||
import { decisionModelRoutes } from "../routes/decision-models.js";
|
||||
import { errorHandler } from "../middleware/index.js";
|
||||
|
||||
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
|
||||
let db: ReturnType<typeof createDb>;
|
||||
let home: string;
|
||||
const backgroundFeature = defineDecisionFeature("test.background", { background: true });
|
||||
const outcome: DecisionProviderOutcome = { answers: { billing: { type: "boolean", probability: 0.9 } }, receipt: decisionReceipt("openai", { usage: { inputTokens: 100, outputTokens: 0 } }) };
|
||||
beforeAll(async () => {
|
||||
home = await realpath(await mkdtemp(path.join(os.tmpdir(), "paperclip-decisions-")));
|
||||
vi.stubEnv("PAPERCLIP_HOME", home);
|
||||
vi.stubEnv("PAPERCLIP_INSTANCE_ID", "decisions-fixture");
|
||||
database = await startEmbeddedPostgresTestDatabase("paperclip-decisions-db-");
|
||||
db = createDb(database.connectionString);
|
||||
}, 90000);
|
||||
afterAll(async () => { await database?.cleanup(); vi.unstubAllEnvs(); if (home) await rm(home, { recursive: true, force: true }); });
|
||||
let sequence = 0;
|
||||
async function fixture() {
|
||||
const companyId = randomUUID(), agentId = randomUUID();
|
||||
await db.insert(companies).values({ id: companyId, name: "Decision tests", issuePrefix: `DC${++sequence}` });
|
||||
await db.insert(agents).values({ id: agentId, companyId, name: "Test agent", adapterType: "codex_local" });
|
||||
await db.insert(companyMemberships).values(["alice", "bob"].map(principalId => ({ companyId, principalId, principalType: "user", status: "active", membershipRole: "owner" })));
|
||||
const connections = aiConnectionService(db);
|
||||
const binding = await connections.save(companyId, "alice", { provider: "openai", method: "api_key", name: "Decision key", ownership: "shared", apiKey: "fixture-key", agentIds: [], allAgents: true }, "fixture-key");
|
||||
const provider = vi.fn().mockResolvedValue(structuredClone(outcome));
|
||||
const service = decisionModelService(db, { provider });
|
||||
const context: DecisionContext = { companyId, feature: settingsDecisionTest,
|
||||
actor: { type: "board", userId: "alice", source: "session", companyIds: [companyId], memberships: [{ companyId, membershipRole: "owner", status: "active" }] } };
|
||||
const config = { ...binding, enabled: true, allowBackground: true };
|
||||
await service.configure(companyId, "alice", config);
|
||||
return { companyId, agentId, binding, provider, service, context, config, connections };
|
||||
}
|
||||
describe("company decision service", () => {
|
||||
it("starts unconfigured with background sponsorship on and makes availability free", async () => {
|
||||
const f = await fixture();
|
||||
await db.delete(companyDecisionModels).where(eq(companyDecisionModels.companyId, f.companyId));
|
||||
expect(await f.service.settings(f.companyId)).toMatchObject({ enabled: false, allowBackground: true, connectionId: null });
|
||||
expect(await f.service.availability(f.context)).toEqual({ available: false, reason: "not_configured" });
|
||||
await f.service.configure(f.companyId, "alice", updateDecisionModelSchema.parse({ ...f.binding, enabled: true }));
|
||||
const before = await db.select().from(secretAccessEvents).where(eq(secretAccessEvents.companyId, f.companyId));
|
||||
const memo = f.service.availabilityForRequest();
|
||||
expect(memo(f.context)).toBe(memo(f.context));
|
||||
expect(await memo(f.context)).toEqual({ available: true });
|
||||
expect(await db.select().from(secretAccessEvents).where(eq(secretAccessEvents.companyId, f.companyId))).toHaveLength(before.length);
|
||||
expect(f.provider).not.toHaveBeenCalled();
|
||||
});
|
||||
it("records a human service charge without an agent, exact cost and no input or answer content", async () => {
|
||||
const f = await fixture();
|
||||
const result = await f.service.decide(f.context, { ...DECISION_TEST_REQUEST, state: "PRIVATE PAYLOAD" });
|
||||
expect(result.status).toBe("succeeded");
|
||||
expect(f.provider.mock.calls[0][0].apiKey).toBe("fixture-key");
|
||||
const [event] = await db.select().from(costEvents).where(eq(costEvents.companyId, f.companyId));
|
||||
expect(event).toMatchObject({ agentId: null, responsibleUserId: "alice", usageKind: "decision", costCents: 0.001, costStatus: "estimated", heartbeatRunId: null });
|
||||
const rows = await db.select().from(decisionInvocations).where(eq(decisionInvocations.companyId, f.companyId));
|
||||
expect(rows[0]).toMatchObject({ status: "succeeded", actorType: "user", responsibleUserId: "alice", costEventId: event.id });
|
||||
expect(JSON.stringify(rows)).not.toContain("PRIVATE PAYLOAD"); expect(JSON.stringify(rows)).not.toContain("probability");
|
||||
expect((await costService(db).byUser(f.companyId)).rows.find(r => r.userId === "alice")?.costCentsExact).toBe("0.0010000");
|
||||
expect((await costService(db).byAgent(f.companyId))[0].agentId).toBeNull();
|
||||
expect((await db.select().from(budgetReservations).where(eq(budgetReservations.companyId, f.companyId)))[0].state).toBe("settled");
|
||||
});
|
||||
it("allows real background use, preserves explicit off, and never sponsors a denied human", async () => {
|
||||
const f = await fixture();
|
||||
const context: DecisionContext = { companyId: f.companyId, feature: backgroundFeature, actor: { type: "system" } };
|
||||
expect((await f.service.decide(context, DECISION_TEST_REQUEST)).status).toBe("succeeded");
|
||||
expect((await db.select().from(decisionInvocations).where(eq(decisionInvocations.companyId, f.companyId)))[0]).toMatchObject({ actorType: "system", responsibleUserId: null });
|
||||
await f.service.configure(f.companyId, "alice", { ...f.config, allowBackground: false });
|
||||
expect(await f.service.availability(context)).toEqual({ available: false, reason: "background_disabled" });
|
||||
const second = await f.connections.save(f.companyId, "alice", { provider: "openrouter", method: "api_key", name: "Other key", ownership: "shared", apiKey: "other", agentIds: [], allAgents: true }, "other");
|
||||
await f.service.configure(f.companyId, "alice", { ...second, enabled: true, allowBackground: false });
|
||||
expect((await f.service.settings(f.companyId)).allowBackground).toBe(false);
|
||||
await f.service.configure(f.companyId, "alice", f.config);
|
||||
await db.insert(connectionGrantMembers).values({ companyId: f.companyId, grantId: f.binding.grantId, subjectType: "user", subjectId: "bob" });
|
||||
expect(await f.service.decide({ ...f.context, feature: backgroundFeature }, DECISION_TEST_REQUEST)).toMatchObject({ status: "unavailable", reason: "access_denied" });
|
||||
expect(f.provider).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
it("retains the charge and dispatch snapshot when the task is deleted during provider work", async () => {
|
||||
const f = await fixture(), issueId = randomUUID();
|
||||
await db.insert(issues).values({ id: issueId, companyId: f.companyId, title: "Temporary task" });
|
||||
f.provider.mockImplementationOnce(async () => {
|
||||
await db.delete(issues).where(eq(issues.id, issueId));
|
||||
return structuredClone(outcome);
|
||||
});
|
||||
expect((await f.service.decide({ ...f.context, issueId }, DECISION_TEST_REQUEST)).status).toBe("succeeded");
|
||||
const [event] = await db.select().from(costEvents).where(eq(costEvents.companyId, f.companyId));
|
||||
expect(event).toMatchObject({ issueId: null, costCents: 0.001 });
|
||||
const [invocation] = await db.select().from(decisionInvocations).where(eq(decisionInvocations.companyId, f.companyId));
|
||||
expect(invocation).toMatchObject({ issueId, status: "succeeded", costEventId: event.id });
|
||||
});
|
||||
it("rechecks revoked grants after availability, and can disable a revoked connection", async () => {
|
||||
const f = await fixture(); expect(await f.service.availability(f.context)).toEqual({ available: true });
|
||||
await db.update(connectionGrants).set({ status: "revoked" }).where(eq(connectionGrants.id, f.binding.grantId));
|
||||
expect(await f.service.decide(f.context, DECISION_TEST_REQUEST)).toMatchObject({ status: "unavailable", reason: "connection_unavailable" });
|
||||
await f.service.configure(f.companyId, "alice", { ...f.config, enabled: false });
|
||||
expect(await f.service.availability(f.context)).toEqual({ available: false, reason: "disabled" }); expect(f.provider).not.toHaveBeenCalled();
|
||||
});
|
||||
it.each(["disabled", "rotated"])("rejects a secret %s while admission waits for accounting", async change => {
|
||||
const f = await fixture();
|
||||
const [secret] = await db.select().from(companySecrets).where(eq(companySecrets.companyId, f.companyId));
|
||||
let release!: () => void, locked!: (pid: number) => void;
|
||||
const gate = new Promise<void>(resolve => { release = resolve; });
|
||||
const acquired = new Promise<number>(resolve => { locked = resolve; });
|
||||
const holding = withAccountingTransaction(db, f.companyId, async tx => {
|
||||
const [row] = await tx.execute<{ pid: number }>(sql`select pg_backend_pid() as pid`);
|
||||
locked(row.pid); await gate;
|
||||
});
|
||||
const pid = await acquired;
|
||||
const pending = f.service.decide(f.context, DECISION_TEST_REQUEST);
|
||||
try {
|
||||
await vi.waitFor(async () => {
|
||||
const [row] = await db.execute<{ count: number }>(sql`select count(*)::int as count from pg_stat_activity where ${pid} = any(pg_blocking_pids(pid))`);
|
||||
expect(row.count).toBeGreaterThan(0);
|
||||
}, { timeout: 5000 });
|
||||
await db.update(companySecrets).set(change === "disabled" ? { status: "disabled" } : { latestVersion: secret.latestVersion + 1 }).where(eq(companySecrets.id, secret.id));
|
||||
} finally { release(); await holding; }
|
||||
expect(await pending).toMatchObject({ status: "unavailable", reason: "connection_unavailable" });
|
||||
expect(f.provider).not.toHaveBeenCalled();
|
||||
expect(await db.select().from(decisionInvocations).where(eq(decisionInvocations.companyId, f.companyId))).toHaveLength(0);
|
||||
});
|
||||
it("preserves sponsorship off when a deleted grant is replaced", async () => {
|
||||
const f = await fixture();
|
||||
await f.service.configure(f.companyId, "alice", { ...f.config, allowBackground: false });
|
||||
await db.delete(connectionGrants).where(eq(connectionGrants.id, f.binding.grantId));
|
||||
expect(await f.service.availability(f.context)).toMatchObject({ available: false, reason: "connection_unavailable" });
|
||||
expect((await f.service.settings(f.companyId)).allowBackground).toBe(false);
|
||||
const next = await f.connections.save(f.companyId, "alice", { provider: "openrouter", method: "api_key", name: "Replacement", ownership: "shared", apiKey: "other", agentIds: [], allAgents: true }, "other");
|
||||
await f.service.configure(f.companyId, "alice", { ...next, enabled: true });
|
||||
expect((await f.service.settings(f.companyId)).allowBackground).toBe(false);
|
||||
expect(f.provider).not.toHaveBeenCalled();
|
||||
});
|
||||
it("rejects foreign-company credentials, membership loss, and invented feature handles", async () => {
|
||||
const f = await fixture(), other = await fixture();
|
||||
await expect(f.service.configure(f.companyId, "alice", { ...other.config })).rejects.toThrow();
|
||||
await db.update(companyMemberships).set({ status: "inactive" }).where(and(eq(companyMemberships.companyId, f.companyId), eq(companyMemberships.principalId, "alice")));
|
||||
expect(await f.service.availability(f.context)).toMatchObject({ available: false, reason: "access_denied" });
|
||||
expect(await f.service.decide({ ...f.context, feature: { id: "settings.test", background: true }, actor: { type: "system" } }, DECISION_TEST_REQUEST)).toMatchObject({ status: "unavailable", reason: "access_denied" });
|
||||
});
|
||||
it("captures the active run identity and checks both human and agent access", async () => {
|
||||
const f = await fixture(), issueId = randomUUID(), runId = randomUUID();
|
||||
await db.insert(issues).values({ id: issueId, companyId: f.companyId, title: "Decision task", status: "in_progress", assigneeAgentId: f.agentId });
|
||||
await db.insert(heartbeatRuns).values({ id: runId, companyId: f.companyId, agentId: f.agentId, status: "running", invocationSource: "on_demand", responsibleUserId: "alice", contextSnapshot: { issueId } });
|
||||
const [identity] = await db.insert(runIdentityContexts).values({ companyId: f.companyId, runId, revision: 1, responsibleUserId: "bob", cause: "steering", correlationId: "test", acceptedAt: new Date() }).returning();
|
||||
await db.update(heartbeatRuns).set({ responsibleUserId: "bob", activeIdentityContextId: identity.id }).where(eq(heartbeatRuns.id, runId));
|
||||
const context: DecisionContext = { companyId: f.companyId, feature: settingsDecisionTest, actor: { type: "agent", agentId: f.agentId, runId, companyId: f.companyId, source: "agent_jwt", onBehalfOfUserId: "alice" } };
|
||||
expect((await f.service.decide(context, DECISION_TEST_REQUEST)).status).toBe("succeeded");
|
||||
expect((await db.select().from(decisionInvocations).where(eq(decisionInvocations.companyId, f.companyId)))[0]).toMatchObject({ responsibleUserId: "bob", runId, issueId, identityContextId: identity.id });
|
||||
await db.delete(toolConnectionInstalls).where(eq(toolConnectionInstalls.connectionId, f.binding.connectionId));
|
||||
expect(await f.service.availability(context)).toMatchObject({ available: false, reason: "access_denied" });
|
||||
await db.update(heartbeatRuns).set({ responsibleUserId: null }).where(eq(heartbeatRuns.id, runId));
|
||||
expect(await f.service.availability(context)).toMatchObject({ available: false, reason: "responsible_user_missing" });
|
||||
});
|
||||
it("serializes budget admission against concurrent requests", async () => {
|
||||
const f = await fixture();
|
||||
await db.insert(budgetPolicies).values({ companyId: f.companyId, scopeType: "company", scopeId: f.companyId, windowKind: "calendar_month_utc", amount: 1, reservationCents: "1", notifyEnabled: false });
|
||||
let release!: () => void; const gate = new Promise<void>(resolve => { release = resolve; });
|
||||
let entered!: () => void; const started = new Promise<void>(resolve => { entered = resolve; });
|
||||
f.provider.mockImplementationOnce(async () => { entered(); await gate; return structuredClone(outcome); });
|
||||
const first = f.service.decide(f.context, DECISION_TEST_REQUEST);
|
||||
await started;
|
||||
try { expect(await f.service.decide(f.context, DECISION_TEST_REQUEST)).toMatchObject({ status: "unavailable", reason: "budget_blocked" }); }
|
||||
finally { release(); }
|
||||
expect((await first).status).toBe("succeeded"); expect(f.provider).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
it("blocks agent deletion while an in-flight or unknown decision retains a budget hold", async () => {
|
||||
const f = await fixture(), runId = randomUUID();
|
||||
await db.insert(heartbeatRuns).values({ id: runId, companyId: f.companyId, agentId: f.agentId, status: "running", invocationSource: "on_demand", responsibleUserId: "alice" });
|
||||
await db.insert(budgetPolicies).values({ companyId: f.companyId, scopeType: "company", scopeId: f.companyId, windowKind: "calendar_month_utc", amount: 1, reservationCents: "1", notifyEnabled: false });
|
||||
const context: DecisionContext = { companyId: f.companyId, feature: settingsDecisionTest,
|
||||
actor: { type: "agent", agentId: f.agentId, runId, companyId: f.companyId, source: "agent_jwt" } };
|
||||
const remove = () => agentService(db).remove(f.agentId);
|
||||
f.provider.mockImplementationOnce(async () => {
|
||||
await expect(remove()).rejects.toMatchObject({ status: 409, details: { code: "agent_decision_accounting_pending" } });
|
||||
expect(await f.service.decide(f.context, DECISION_TEST_REQUEST)).toMatchObject({ status: "unavailable", reason: "budget_blocked" });
|
||||
return { errorCode: "timeout", receipt: decisionReceipt("openai", null) };
|
||||
});
|
||||
expect((await f.service.decide(context, DECISION_TEST_REQUEST)).status).toBe("failed");
|
||||
await expect(remove()).rejects.toMatchObject({ status: 409, details: { code: "agent_decision_accounting_pending" } });
|
||||
expect((await db.select().from(budgetReservations).where(eq(budgetReservations.companyId, f.companyId)))[0]).toMatchObject({ state: "held", amountCents: "1.0000000" });
|
||||
});
|
||||
it("holds unknown billing until an audited correction, without rerunning the provider", async () => {
|
||||
const f = await fixture(); f.provider.mockResolvedValue({ errorCode: "timeout", receipt: decisionReceipt("openai", null) });
|
||||
expect((await f.service.decide(f.context, DECISION_TEST_REQUEST)).status).toBe("failed");
|
||||
const [event] = await db.select().from(costEvents).where(eq(costEvents.companyId, f.companyId));
|
||||
expect(event.costStatus).toBe("unpriced");
|
||||
expect((await db.select().from(budgetReservations).where(eq(budgetReservations.companyId, f.companyId)))[0].state).toBe("held");
|
||||
await billingReconciliationService(db).adjust(f.companyId, event.id, { idempotencyKey: "correction", expectedCents: "0", correctedCents: "0.002", reason: "Verified provider invoice", pricing: { source: "operator" } }, "alice");
|
||||
expect((await db.select().from(budgetReservations).where(eq(budgetReservations.companyId, f.companyId)))[0].state).toBe("settled");
|
||||
expect(f.provider).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
it("recovers an interrupted dispatch once without paying for a retry", async () => {
|
||||
const f = await fixture();
|
||||
const [invocation] = await db.insert(decisionInvocations).values({ companyId: f.companyId, feature: "test.background", actorType: "system", actorId: "paperclip-decisions",
|
||||
...f.binding, provider: "openai", model: "gpt-6-luna", questionTypes: ["boolean"], startedAt: new Date(Date.now() - 180_000) }).returning();
|
||||
await db.insert(budgetReservations).values({ companyId: f.companyId, decisionInvocationId: invocation.id, amountCents: "0.01", providerStartedAt: invocation.startedAt });
|
||||
await Promise.all([f.service.recoverInterrupted(), f.service.recoverInterrupted()]);
|
||||
const events = await db.select().from(costEvents).where(eq(costEvents.companyId, f.companyId));
|
||||
expect(events).toHaveLength(1); expect(events[0].costStatus).toBe("unpriced"); expect(f.provider).not.toHaveBeenCalled();
|
||||
});
|
||||
it("keeps public cost reporting agent-scoped and bounds decision input before payment", async () => {
|
||||
expect(createCostEventSchema.safeParse({ agentId: null, provider: "openai", model: "gpt-6-luna", costCents: 1, occurredAt: new Date().toISOString() }).success).toBe(false);
|
||||
const f = await fixture(); await expect(f.service.decide(f.context, { ...DECISION_TEST_REQUEST, state: "x".repeat(32_001) })).rejects.toThrow(); expect(f.provider).not.toHaveBeenCalled();
|
||||
});
|
||||
it("preserves an explicit sponsorship opt-out when a subsequent edit omits the field", async () => {
|
||||
const f = await fixture();
|
||||
await f.service.configure(f.companyId, "alice", { ...f.config, allowBackground: false });
|
||||
const second = await f.connections.save(f.companyId, "alice", { provider: "openrouter", method: "api_key", name: "Other", ownership: "shared", apiKey: "other", agentIds: [], allAgents: true }, "other");
|
||||
await f.service.configure(f.companyId, "alice", { ...second, enabled: true });
|
||||
expect((await f.service.settings(f.companyId)).allowBackground).toBe(false);
|
||||
});
|
||||
it("rejects revoked secrets without provider work and returns usage on successful calls", async () => {
|
||||
const f = await fixture();
|
||||
expect(await f.service.decide(f.context, DECISION_TEST_REQUEST)).toMatchObject({ status: "succeeded", usage: { costCents: "0.0010000", costStatus: "estimated", inputTokens: 100 } });
|
||||
await db.update(companySecrets).set({ status: "revoked" }).where(eq(companySecrets.companyId, f.companyId));
|
||||
expect(await f.service.decide(f.context, DECISION_TEST_REQUEST)).toMatchObject({ status: "unavailable" });
|
||||
expect(f.provider).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
it.each(["agent", "project"] as const)("reserves and enforces applicable %s budgets", async scopeType => {
|
||||
const f = await fixture(), projectId = randomUUID(), issueId = randomUUID(), runId = randomUUID();
|
||||
await db.insert(projects).values({ id: projectId, companyId: f.companyId, name: "Decisions project" });
|
||||
await db.insert(issues).values({ id: issueId, companyId: f.companyId, projectId, title: "Budgeted task", assigneeAgentId: f.agentId });
|
||||
await db.insert(heartbeatRuns).values({ id: runId, companyId: f.companyId, agentId: f.agentId, status: "running", invocationSource: "on_demand", responsibleUserId: "alice", contextSnapshot: { issueId } });
|
||||
const context: DecisionContext = { companyId: f.companyId, feature: settingsDecisionTest, actor: { type: "agent", agentId: f.agentId, runId, companyId: f.companyId, source: "agent_jwt" } };
|
||||
const [policy] = await db.insert(budgetPolicies).values({ companyId: f.companyId, scopeType, scopeId: scopeType === "agent" ? f.agentId : projectId, windowKind: "calendar_month_utc", amount: 1, reservationCents: "2", notifyEnabled: false }).returning();
|
||||
expect(await f.service.availability(context)).toEqual({ available: true });
|
||||
expect(await f.service.decide(context, DECISION_TEST_REQUEST)).toMatchObject({ status: "unavailable", reason: "budget_blocked" });
|
||||
expect(f.provider).not.toHaveBeenCalled();
|
||||
await db.update(budgetPolicies).set({ amount: 5 }).where(eq(budgetPolicies.id, policy.id));
|
||||
expect((await f.service.decide(context, DECISION_TEST_REQUEST)).status).toBe("succeeded");
|
||||
expect((await db.select().from(budgetReservations).where(eq(budgetReservations.companyId, f.companyId)))[0]).toMatchObject({ agentId: f.agentId, projectId, amountCents: "2.0000000", state: "settled" });
|
||||
});
|
||||
it("releases reservations when the provider rejects a request before model work", async () => {
|
||||
const f = await fixture();
|
||||
f.provider.mockResolvedValue({ noProviderWork: true, errorCode: "provider_auth_failed", receipt: { ...outcome.receipt, inputTokens: 0, costCents: "0", pricingProvenance: { source: "unknown", evidence: "HTTP 401" } } });
|
||||
expect((await f.service.decide(f.context, DECISION_TEST_REQUEST)).status).toBe("failed");
|
||||
expect((await db.select().from(budgetReservations).where(eq(budgetReservations.companyId, f.companyId)))[0].state).toBe("released");
|
||||
expect((await db.select().from(costEvents).where(eq(costEvents.companyId, f.companyId)))[0].costCents).toBe(0);
|
||||
});
|
||||
it("enforces manager-only configuration and tests, company isolation, and fixed sample execution", async () => {
|
||||
const f = await fixture(), other = await fixture();
|
||||
const app = express(); app.use(express.json());
|
||||
let actor = f.context.actor;
|
||||
app.use((req, _res, next) => { req.actor = actor as typeof req.actor; next(); });
|
||||
app.use("/api", decisionModelRoutes(db)); app.use(errorHandler);
|
||||
const url = `/api/companies/${f.companyId}/decision-model`;
|
||||
expect((await request(app).get(url)).body.canManage).toBe(true);
|
||||
expect((await request(app).get(`/api/companies/${other.companyId}/decision-model`)).status).toBe(403);
|
||||
actor = { type: "board", userId: "alice", source: "session", companyIds: [f.companyId], memberships: [{ companyId: f.companyId, status: "active", membershipRole: "viewer" }] };
|
||||
expect((await request(app).get(url)).body).toEqual({ canManage: false, settings: null, choices: [] });
|
||||
expect((await request(app).put(url).send(f.config)).status).toBe(403);
|
||||
expect((await request(app).post(`${url}/test`).send({ actor: { type: "system" } })).status).toBe(403);
|
||||
actor = f.context.actor;
|
||||
await f.service.configure(f.companyId, "alice", { ...f.config, enabled: false });
|
||||
expect((await request(app).post(`${url}/test`).send({ state: "untrusted", actor: { type: "system" }, model: "untrusted" })).body).toEqual({ status: "unavailable", reason: "disabled" });
|
||||
expect((await request(app).post(`${url}/decide`).send(DECISION_TEST_REQUEST)).status).toBe(404);
|
||||
expect((await request(app).get(`${url}/history?limit=501`)).status).toBe(400);
|
||||
expect((await request(app).get(`${url}/history?from=2026-10-01T00:00:00.000Z&to=2026-10-31T23:59:59.000Z`)).status).toBe(200);
|
||||
});
|
||||
it("keeps private task links out of company cost history for unauthorized readers", async () => {
|
||||
const f = await fixture(), projectId = randomUUID(), issueId = randomUUID();
|
||||
await db.insert(projects).values({ id: projectId, companyId: f.companyId, name: "Private project", visibility: "private", privacyOwnerUserId: "bob" });
|
||||
await db.insert(issues).values({ id: issueId, companyId: f.companyId, projectId, title: "PRIVATE TITLE", identifier: "SECRET-1" });
|
||||
await f.service.decide({ companyId: f.companyId, feature: backgroundFeature, actor: { type: "system" }, issueId }, DECISION_TEST_REQUEST);
|
||||
await db.update(companyMemberships).set({ membershipRole: "member" }).where(and(eq(companyMemberships.companyId, f.companyId), eq(companyMemberships.principalId, "alice")));
|
||||
const actor = { type: "board" as const, userId: "alice", source: "session" as const, companyIds: [f.companyId], memberships: [{ companyId: f.companyId, status: "active", membershipRole: "member" }] };
|
||||
expect(await f.service.availability({ ...f.context, actor, issueId })).toMatchObject({ available: false, reason: "access_denied" });
|
||||
const history = await f.service.history(f.companyId, actor);
|
||||
expect(history[0]).toMatchObject({ issueId: null, issueIdentifier: null, runId: null });
|
||||
expect(JSON.stringify(history)).not.toContain("SECRET");
|
||||
});
|
||||
});
|
||||
@@ -23,6 +23,7 @@ const apiPrefixes: Record<string, string> = {
|
||||
"agent-avatars.ts": "/api",
|
||||
"announcements.ts": "/api",
|
||||
"ai-connections.ts": "/api",
|
||||
"decision-models.ts": "/api",
|
||||
"attention.ts": "/api",
|
||||
"approvals.ts": "/api",
|
||||
"assets.ts": "/api",
|
||||
|
||||
@@ -10,6 +10,7 @@ import { createMcpApiDispatch, createPublicMcpExecutor } from "./services/public
|
||||
import { createPublicMcpEvents, type PublicMcpEvents } from "./services/public-mcp/events.js";
|
||||
import { publicMcpIngressRoutes, publicMcpManagementRoutes } from "./routes/public-mcp.js";
|
||||
import { agentAvatarRoutes } from "./routes/agent-avatars.js";
|
||||
import { decisionModelRoutes } from "./routes/decision-models.js";
|
||||
import { aiConnectionRoutes } from "./routes/ai-connections.js";
|
||||
import { projectToolRoutes } from "./routes/project-tools.js";
|
||||
import { emailChannelService } from "./services/email-channels.js";
|
||||
@@ -877,6 +878,7 @@ export async function createApp(
|
||||
registerAssignedMcpGateway(db, toolGateway);
|
||||
app.locals.toolActionDeliveries = toolActionDeliveries;
|
||||
app.use(mcpGatewayProtocolRoutes(toolGateway));
|
||||
api.use(decisionModelRoutes(db));
|
||||
api.use(aiConnectionRoutes(db, { deploymentMode: opts.deploymentMode, deploymentExposure: opts.deploymentExposure, trustedLocalStdioRuntimeHost }));
|
||||
api.use(
|
||||
toolAccessRoutes(db, {
|
||||
|
||||
@@ -39,7 +39,7 @@ export function responsibleUserForAiRequest(req: Request): string | null {
|
||||
: getActorInfo(req).actorId;
|
||||
}
|
||||
|
||||
async function canManageAiConnections(db: Db, req: Request, companyId: string): Promise<boolean> {
|
||||
export async function canManageAiConnections(db: Db, req: Request, companyId: string): Promise<boolean> {
|
||||
const membership = req.actor.memberships?.find((m) => m.companyId === companyId && m.status === "active");
|
||||
if (membership?.membershipRole === "viewer") return false;
|
||||
return req.actor.source === "local_implicit" || Boolean(req.actor.isInstanceAdmin)
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
import { Router } from "express";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { DECISION_TEST_REQUEST, updateDecisionModelSchema } from "@paperclipai/shared";
|
||||
import { assertBoard, assertCompanyAccess, getActorInfo } from "./authz.js";
|
||||
import { canManageAiConnections } from "./ai-connections.js";
|
||||
import { forbidden } from "../errors.js";
|
||||
import { validate } from "../middleware/validate.js";
|
||||
import { decisionModelService, settingsDecisionTest } from "../services/decision-models.js";
|
||||
import { heartbeatService } from "../services/heartbeat.js";
|
||||
import { parseCostDateRange, parseCostLimit } from "./costs.js";
|
||||
|
||||
export function decisionModelRoutes(db: Db) {
|
||||
const router = Router();
|
||||
const service = decisionModelService(db, { budgetHooks: { cancelWorkForScope: heartbeatService(db).cancelBudgetScopeWork } });
|
||||
router.get("/companies/:companyId/decision-model", async (req, res) => {
|
||||
const companyId = req.params.companyId as string;
|
||||
assertCompanyAccess(req, companyId); assertBoard(req);
|
||||
const canManage = await canManageAiConnections(db, req, companyId);
|
||||
if (!canManage) { res.json({ canManage, settings: null, choices: [] }); return; }
|
||||
res.json({ canManage, settings: await service.settings(companyId), choices: await service.choices(companyId, getActorInfo(req).actorId) });
|
||||
});
|
||||
router.put("/companies/:companyId/decision-model", validate(updateDecisionModelSchema), async (req, res) => {
|
||||
const companyId = req.params.companyId as string;
|
||||
assertCompanyAccess(req, companyId); assertBoard(req);
|
||||
if (!(await canManageAiConnections(db, req, companyId))) throw forbidden("Manage connections permission is required");
|
||||
res.json(await service.configure(companyId, getActorInfo(req).actorId, req.body));
|
||||
});
|
||||
router.get("/companies/:companyId/decision-model/availability", async (req, res) => {
|
||||
const companyId = req.params.companyId as string;
|
||||
assertCompanyAccess(req, companyId);
|
||||
res.json(await service.availability({ companyId, actor: req.actor, feature: settingsDecisionTest }));
|
||||
});
|
||||
router.post("/companies/:companyId/decision-model/test", async (req, res) => {
|
||||
const companyId = req.params.companyId as string;
|
||||
assertCompanyAccess(req, companyId); assertBoard(req);
|
||||
if (!(await canManageAiConnections(db, req, companyId))) throw forbidden("Manage connections permission is required");
|
||||
// The client cannot supply a prompt, model, identity, or background flag.
|
||||
res.json(await service.decide({ companyId, actor: req.actor, feature: settingsDecisionTest }, DECISION_TEST_REQUEST));
|
||||
});
|
||||
router.get("/companies/:companyId/decision-model/history", async (req, res) => {
|
||||
const companyId = req.params.companyId as string;
|
||||
assertCompanyAccess(req, companyId);
|
||||
res.json(await service.history(companyId, req.actor, { ...parseCostDateRange(req.query), limit: parseCostLimit(req.query) }));
|
||||
});
|
||||
return router;
|
||||
}
|
||||
@@ -7,6 +7,7 @@ import { Router } from "express";
|
||||
import { z } from "zod";
|
||||
import {
|
||||
createAiConnectionSchema,
|
||||
updateDecisionModelSchema,
|
||||
aiConnectionPoolConfigSchema,
|
||||
aiConnectionLoginIntentSchema,
|
||||
localAiConnectionSchema,
|
||||
@@ -1355,6 +1356,9 @@ const browserUseOperations = [
|
||||
] as const;
|
||||
|
||||
const BOARD_ONLY_OPERATIONS = new Set([
|
||||
"GET /api/companies/{companyId}/decision-model",
|
||||
"PUT /api/companies/{companyId}/decision-model",
|
||||
"POST /api/companies/{companyId}/decision-model/test",
|
||||
...browserUseOperations.map(([method, path]) => `${method.toUpperCase()} ${path}`),
|
||||
"GET /api/companies/{companyId}/ai-connections",
|
||||
"POST /api/companies/{companyId}/ai-connections",
|
||||
@@ -10735,6 +10739,33 @@ registerCurrentRoute({
|
||||
|
||||
// --- Tool access -------------------------------------------------------------
|
||||
|
||||
registerCurrentRoute({
|
||||
method: "get", path: "/api/companies/{companyId}/decision-model", tags: ["decision-models"],
|
||||
summary: "Get decision settings, compatible connections, and manager capability",
|
||||
responses: { 200: r.ok(), 401: r.unauthorized, 403: r.forbidden },
|
||||
});
|
||||
registerCurrentRoute({
|
||||
method: "put", path: "/api/companies/{companyId}/decision-model", tags: ["decision-models"],
|
||||
summary: "Configure the company decision model as a connection manager", body: updateDecisionModelSchema,
|
||||
responses: { 200: r.ok(), 400: r.badRequest, 401: r.unauthorized, 403: r.forbidden, 422: r.unprocessable },
|
||||
});
|
||||
registerCurrentRoute({
|
||||
method: "get", path: "/api/companies/{companyId}/decision-model/availability", tags: ["decision-models"],
|
||||
summary: "Check local decision configuration and caller authorization without contacting a provider",
|
||||
responses: { 200: r.ok(), 401: r.unauthorized, 403: r.forbidden },
|
||||
});
|
||||
registerCurrentRoute({
|
||||
method: "post", path: "/api/companies/{companyId}/decision-model/test", tags: ["decision-models"],
|
||||
summary: "Run the fixed billed three-question setup test as a connection manager",
|
||||
responses: { 200: r.ok(), 401: r.unauthorized, 403: r.forbidden },
|
||||
});
|
||||
registerCurrentRoute({
|
||||
method: "get", path: "/api/companies/{companyId}/decision-model/history", tags: ["decision-models"],
|
||||
summary: "List decision metadata and charges with authorized task links",
|
||||
query: costReportQuerySchema.extend({ limit: z.coerce.number().int().min(1).max(500).optional() }),
|
||||
responses: { 200: r.ok(), 400: r.badRequest, 401: r.unauthorized, 403: r.forbidden },
|
||||
});
|
||||
|
||||
registerCurrentRoute({
|
||||
method: "get",
|
||||
path: "/api/companies/{companyId}/tools/gallery",
|
||||
|
||||
@@ -111,7 +111,7 @@ export function accountingIntegrityService(db: Db, hooks: BudgetServiceHooks = {
|
||||
and result_json->'executionRecovery'->>'providerWorkStarted' is distinct from 'false' then 'waiting_for_receipt' else 'retryable' end as state,
|
||||
accounting_last_error as "lastError", accounting_attempt_count as attempts, coalesce(finished_at,created_at)::text as since, accounting_last_attempt_at::text as "lastAttemptAt"
|
||||
from heartbeat_runs where company_id = ${companyId} and cost_accounting_pending and status in ('succeeded','failed','timed_out','cancelled','interrupted') order by coalesce(finished_at,created_at),id limit 100`);
|
||||
const unpriced = await tx.execute<{ runId: string | null; costEventId: string; agentId: string; since: string }>(sql`
|
||||
const unpriced = await tx.execute<{ runId: string | null; costEventId: string; agentId: string | null; since: string }>(sql`
|
||||
select heartbeat_run_id as "runId", id as "costEventId", agent_id as "agentId", occurred_at::text as since from cost_events
|
||||
where company_id = ${companyId} and cost_status = 'unpriced' and billing_type <> 'subscription_included' order by occurred_at,id limit 100`);
|
||||
return { companyId, pendingRunCount: counts.pending, unpricedEventCount: counts.unpriced, oldestPendingAt: counts.oldest, pendingCancellationCount: counts.cancellations,
|
||||
|
||||
@@ -15,6 +15,7 @@ import {
|
||||
agentWakeupRequests,
|
||||
activityLog,
|
||||
costEvents,
|
||||
budgetReservations,
|
||||
heartbeatRunEvents,
|
||||
heartbeatRuns,
|
||||
issueExecutionDecisions,
|
||||
@@ -1098,6 +1099,13 @@ export function agentService(db: Db, budgetHooks: BudgetServiceHooks = {}) {
|
||||
}
|
||||
|
||||
return withAccountingTransaction(db, existing.companyId, async (tx) => {
|
||||
const [decisionHold] = await tx.select({ id: budgetReservations.id }).from(budgetReservations).where(and(
|
||||
eq(budgetReservations.companyId, existing.companyId), eq(budgetReservations.agentId, id),
|
||||
eq(budgetReservations.state, "held"), sql`${budgetReservations.decisionInvocationId} is not null`,
|
||||
)).limit(1);
|
||||
if (decisionHold) throw conflict("Wait for active decisions or resolve their unknown charges in Costs before deleting this agent", {
|
||||
code: "agent_decision_accounting_pending",
|
||||
});
|
||||
await tx
|
||||
.select({ id: agents.id })
|
||||
.from(agents)
|
||||
|
||||
@@ -23,6 +23,7 @@ import {
|
||||
} from "@paperclipai/db";
|
||||
import {
|
||||
AI_CONNECTION_CAPABILITIES,
|
||||
decisionProviderForConnection,
|
||||
aiConnectionCatalogSlug,
|
||||
getAppStoreDefinition,
|
||||
aiConnectionMetadataSchema,
|
||||
@@ -407,7 +408,41 @@ export function aiConnectionService(db: Db) {
|
||||
} satisfies AiConnectionAttribution,
|
||||
};
|
||||
}
|
||||
async function credential(row: Pick<Awaited<ReturnType<typeof select>>, "connection" | "grant">, retry = 0): Promise<string> {
|
||||
/** Internal decision use has its own capability check, never a pretend CLI harness. */
|
||||
async function selectDecision(input: {
|
||||
companyId: string; connectionId: string; grantId: string;
|
||||
userId: string | null; agentId?: string | null; sponsoredBackground?: boolean;
|
||||
}) {
|
||||
const [row] = await db.select({ connection: toolConnections, grant: connectionGrants })
|
||||
.from(toolConnections).innerJoin(connectionGrants, and(
|
||||
eq(connectionGrants.connectionId, toolConnections.id), eq(connectionGrants.companyId, toolConnections.companyId),
|
||||
)).where(and(eq(toolConnections.companyId, input.companyId), eq(toolConnections.id, input.connectionId),
|
||||
eq(connectionGrants.id, input.grantId), eq(toolConnections.connectionPurpose, "ai"))).limit(1);
|
||||
if (!row || row.grant.kind !== "organization") throw unprocessable("Shared AI connection unavailable", { code: "connection_unavailable" });
|
||||
const metadata = aiConnectionMetadataSchema.safeParse(row.connection.config.ai);
|
||||
const provider = metadata.success ? decisionProviderForConnection(metadata.data) : null;
|
||||
if (!provider) throw unprocessable("Connection does not support decisions", { code: "incompatible_connection" });
|
||||
if (row.grant.status !== "active" || !row.connection.enabled || row.connection.status !== "active" || row.connection.healthStatus !== "ok")
|
||||
throw unprocessable("Reconnect or enable this connection", { code: "connection_unavailable" });
|
||||
// Only the decision service may sponsor background use; ordinary AI selection never bypasses an audience.
|
||||
if (!input.sponsoredBackground) {
|
||||
if (!(await membership(input.companyId, input.userId))) throw forbidden("Active responsible user required");
|
||||
const audience = await db.select().from(connectionGrantMembers).where(and(
|
||||
eq(connectionGrantMembers.companyId, input.companyId), eq(connectionGrantMembers.grantId, input.grantId),
|
||||
));
|
||||
if (!canUseCredential(row.grant, input.userId, audience)) throw forbidden("Connection is not shared with this user");
|
||||
} else if (input.userId || input.agentId) throw forbidden("Background sponsorship cannot replace caller permissions");
|
||||
if (input.agentId) {
|
||||
const [install] = await db.select({ id: toolConnectionInstalls.id }).from(toolConnectionInstalls).where(and(
|
||||
eq(toolConnectionInstalls.companyId, input.companyId), eq(toolConnectionInstalls.connectionId, input.connectionId),
|
||||
or(and(eq(toolConnectionInstalls.targetType, "company"), eq(toolConnectionInstalls.targetId, input.companyId)),
|
||||
and(eq(toolConnectionInstalls.targetType, "agent"), eq(toolConnectionInstalls.targetId, input.agentId))),
|
||||
)).limit(1);
|
||||
if (!install) throw forbidden("Connection is not permitted for this agent");
|
||||
}
|
||||
return { ...row, provider };
|
||||
}
|
||||
async function credential(row: Pick<Awaited<ReturnType<typeof select>>, "connection" | "grant">, retry = 0, audit?: { responsibleUserId: string | null; actorType: "user" | "agent" | "system"; actorId: string; issueId?: string | null; heartbeatRunId?: string | null }): Promise<string> {
|
||||
const metadata = aiConnectionMetadataSchema.parse(row.connection.config.ai);
|
||||
if (metadata.routing?.auth === "none") return "";
|
||||
const ref = row.grant.credentialSecretRefs.find(
|
||||
@@ -432,12 +467,14 @@ export function aiConnectionService(db: Db) {
|
||||
});
|
||||
if (row.grant.kind === "user" && secret.scope !== "user")
|
||||
throw forbidden("Credential ownership mismatch");
|
||||
if (row.grant.kind === "organization" && secret.scope !== "company") throw forbidden("Credential ownership mismatch");
|
||||
const context = {
|
||||
consumerType: "tool_connection" as const,
|
||||
consumerId: row.connection.id,
|
||||
configPath: ref.configPath,
|
||||
responsibleUserId: row.grant.subjectUserId,
|
||||
actorType: "system" as const,
|
||||
...audit,
|
||||
};
|
||||
let value: string;
|
||||
if (secret.scope === "user") {
|
||||
@@ -469,7 +506,7 @@ export function aiConnectionService(db: Db) {
|
||||
if (!latest || latest.status !== "active") throw unprocessable("Reconnect this AI account");
|
||||
if (latest.latestVersion !== secret.latestVersion) {
|
||||
if (retry >= 2) throw unprocessable("AI credentials are changing. Retry this execution.", { code: "ai_connection_busy" });
|
||||
return credential(row, retry + 1);
|
||||
return credential(row, retry + 1, audit);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
@@ -1070,5 +1107,5 @@ export function aiConnectionService(db: Db) {
|
||||
return [{ ...row, summary }];
|
||||
});
|
||||
}
|
||||
return { list, quotaAccounts, refreshQuotaCredential, select, credential, runtimeCredential, probeUsage, save, setDefault, membership, markAuthenticationFailed };
|
||||
return { list, selectDecision, quotaAccounts, refreshQuotaCredential, select, credential, runtimeCredential, probeUsage, save, setDefault, membership, markAuthenticationFailed };
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
import { createFinanceEventInTransaction } from "./finance.js";
|
||||
import { and, desc, eq, sql } from "drizzle-orm";
|
||||
import { agentRuntimeState, costEvents, costAdjustments, billingInvoices, billingInvoiceLines, financeEvents, heartbeatRuns, type Db } from "@paperclipai/db";
|
||||
import { decisionInvocations, budgetReservations, agentRuntimeState, costEvents, costAdjustments, billingInvoices, billingInvoiceLines, financeEvents, heartbeatRuns, type Db } from "@paperclipai/db";
|
||||
import { adjustCostSchema, importBillingInvoiceSchema, normalizeCents, subtractCents, type AdjustCost, type ImportBillingInvoice } from "@paperclipai/shared";
|
||||
import { conflict, notFound, unprocessable } from "../errors.js";
|
||||
import { withAccountingReadSnapshot, withAccountingTransaction } from "./accounting-transaction.js";
|
||||
@@ -147,12 +147,20 @@ export function billingReconciliationService(db: Db, hooks: BudgetServiceHooks =
|
||||
reportedCostCents: event.event.reportedCostCents ?? normalizeCents(event.exact), costStatus: input.pricing.source === "rate_card" ? "estimated" : "reported", pricingProvenance: input.pricing,
|
||||
}).where(eq(costEvents.id, eventId)).returning();
|
||||
await updateMonthlySpendProjections(tx, companyId, updated.agentId, subtractCents(input.correctedCents, input.expectedCents), updated.occurredAt);
|
||||
if (updated.heartbeatRunId) {
|
||||
if (updated.heartbeatRunId && updated.agentId) {
|
||||
const [run] = await tx.select().from(heartbeatRuns).where(and(eq(heartbeatRuns.id, updated.heartbeatRunId), eq(heartbeatRuns.companyId, companyId)));
|
||||
if (run?.costAccountedAt && run.accountingProjectionVersion === "v2") await tx.update(agentRuntimeState)
|
||||
.set({ totalCostCents: sql`${agentRuntimeState.totalCostCents} + ${subtractCents(input.correctedCents, input.expectedCents)}::numeric`, updatedAt: new Date() })
|
||||
.where(and(eq(agentRuntimeState.agentId, updated.agentId), eq(agentRuntimeState.companyId, companyId)));
|
||||
}
|
||||
if (updated.usageKind === "decision") {
|
||||
const [invocation] = await tx.select({ id: decisionInvocations.id }).from(decisionInvocations).where(and(
|
||||
eq(decisionInvocations.companyId, companyId), eq(decisionInvocations.costEventId, eventId),
|
||||
));
|
||||
if (invocation) await tx.update(budgetReservations).set({ state: "settled", settledAt: new Date() }).where(and(
|
||||
eq(budgetReservations.companyId, companyId), eq(budgetReservations.decisionInvocationId, invocation.id), eq(budgetReservations.state, "held"),
|
||||
));
|
||||
}
|
||||
const budgets = budgetServiceInTransaction(tx, publications);
|
||||
await budgets.evaluateCostEvent(updated);
|
||||
await budgets.getInvocationBlock(companyId, updated.agentId, { projectId: updated.projectId });
|
||||
|
||||
@@ -160,10 +160,10 @@ async function mapBudgetReads<T, R>(rows: T[], read: (row: T) => Promise<R>): Pr
|
||||
}
|
||||
|
||||
/** Filter in PostgreSQL so one run never loads unrelated agents' policies. */
|
||||
export function budgetPoliciesForRun(companyId: string, agentId: string, projectId: string | null) {
|
||||
export function budgetPoliciesForRun(companyId: string, agentId: string | null, projectId: string | null) {
|
||||
return or(
|
||||
and(eq(budgetPolicies.scopeType, "company"), eq(budgetPolicies.scopeId, companyId)),
|
||||
and(eq(budgetPolicies.scopeType, "agent"), eq(budgetPolicies.scopeId, agentId)),
|
||||
agentId ? and(eq(budgetPolicies.scopeType, "agent"), eq(budgetPolicies.scopeId, agentId)) : undefined,
|
||||
projectId ? and(eq(budgetPolicies.scopeType, "project"), eq(budgetPolicies.scopeId, projectId)) : undefined,
|
||||
);
|
||||
}
|
||||
@@ -858,11 +858,11 @@ export function budgetServiceInTransaction(db: Db, publications: ActivityPublica
|
||||
|
||||
getInvocationBlock: async (
|
||||
companyId: string,
|
||||
agentId: string,
|
||||
agentId: string | null,
|
||||
context?: { issueId?: string | null; projectId?: string | null },
|
||||
) => {
|
||||
const scopes: Array<{ scopeType: BudgetScopeType; scopeId: string }> = [
|
||||
{ scopeType: "company", scopeId: companyId }, { scopeType: "agent", scopeId: agentId },
|
||||
{ scopeType: "company", scopeId: companyId }, ...(agentId ? [{ scopeType: "agent" as const, scopeId: agentId }] : []),
|
||||
...(context?.projectId ? [{ scopeType: "project" as const, scopeId: context.projectId }] : []),
|
||||
];
|
||||
for (const { scopeType, scopeId } of scopes) {
|
||||
@@ -1043,7 +1043,7 @@ export function budgetService(db: Db, hooks: BudgetServiceHooks = {}) {
|
||||
mutate(companyId, (service) => service.upsertPolicy(companyId, input, actorUserId)),
|
||||
evaluateCostEvent: (event: typeof costEvents.$inferSelect) =>
|
||||
mutate(event.companyId, (service) => service.evaluateCostEvent(event)),
|
||||
getInvocationBlock: (companyId: string, agentId: string, context?: { issueId?: string | null; projectId?: string | null }) =>
|
||||
getInvocationBlock: (companyId: string, agentId: string | null, context?: { issueId?: string | null; projectId?: string | null }) =>
|
||||
mutate(companyId, (service) => service.getInvocationBlock(companyId, agentId, context)),
|
||||
resolveIncident: (companyId: string, incidentId: string, input: BudgetIncidentResolutionInput, actorUserId: string) =>
|
||||
mutate(companyId, (service) => service.resolveIncident(companyId, incidentId, input, actorUserId)),
|
||||
|
||||
@@ -19,6 +19,7 @@ import {
|
||||
runIdentityContexts,
|
||||
heartbeatRunEvents,
|
||||
costEvents,
|
||||
decisionInvocations,
|
||||
financeEvents,
|
||||
budgetPolicies,
|
||||
budgetIncidents,
|
||||
@@ -563,6 +564,7 @@ export function companyService(db: Db, budgetHooks: BudgetServiceHooks = {}) {
|
||||
// Finance can reference costs and both can reference runs. Incidents
|
||||
// reference policies and approvals; delete these dependents first.
|
||||
await tx.delete(financeEvents).where(eq(financeEvents.companyId, id));
|
||||
await tx.delete(decisionInvocations).where(eq(decisionInvocations.companyId, id));
|
||||
await tx.delete(costEvents).where(eq(costEvents.companyId, id));
|
||||
await tx.delete(budgetIncidents).where(eq(budgetIncidents.companyId, id));
|
||||
await tx.delete(budgetPolicies).where(eq(budgetPolicies.companyId, id));
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { receiptFingerprint } from "./receipt-fingerprint.js";
|
||||
import { agentAvatarUrl, resolveAgentAppearance, createCostEventSchema, normalizeCents, type MoneyInput, type CostByUserReport } from "@paperclipai/shared";
|
||||
import { agentAvatarUrl, resolveAgentAppearance, createCostEventSchema, createServiceCostEventSchema, normalizeCents, type MoneyInput, type CostByUserReport } from "@paperclipai/shared";
|
||||
import { and, desc, eq, gte, inArray, isNotNull, isNull, lt, lte, ne, sql } from "drizzle-orm";
|
||||
import { alias } from "drizzle-orm/pg-core";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
@@ -42,12 +42,13 @@ function currentUtcMonthWindow(now = new Date()) {
|
||||
/** Increment an initialized month under the company lock. On upgrade or UTC
|
||||
* rollover, initialize from the ledger in the same transaction exactly once.
|
||||
* Reports, budgets and the integrity checker continue reading the ledger. */
|
||||
export async function updateMonthlySpendProjections(db: Db, companyId: string, agentId: string, amountCents: MoneyInput, occurredAt: Date) {
|
||||
export async function updateMonthlySpendProjections(db: Db, companyId: string, agentId: string | null, amountCents: MoneyInput, occurredAt: Date) {
|
||||
const { start, end } = currentUtcMonthWindow();
|
||||
const month = start.toISOString().slice(0, 10);
|
||||
const delta = occurredAt >= start && occurredAt < end ? normalizeCents(amountCents) : "0.0000000";
|
||||
for (const table of [companies, agents] as const) {
|
||||
const id = table === companies ? companyId : agentId;
|
||||
if (!id) continue;
|
||||
const companyCondition = table === companies ? eq(companies.id, companyId) : eq(agents.companyId, companyId);
|
||||
await db.update(table).set({
|
||||
spentMonthlyCents: sql`case when ${table.spendMonthUtc} = ${month} then ${table.spentMonthlyCents} + ${delta}::numeric else (
|
||||
@@ -82,7 +83,7 @@ export async function getMonthlySpendTotal(
|
||||
}
|
||||
|
||||
export async function createCostEventInTransaction(db: Db, companyId: string, data: Omit<typeof costEvents.$inferInsert, "companyId" | "receiptHash" | "costCents"> & { costCents: MoneyInput }, publications: ActivityPublication[] = [], actor?: Pick<LogActivityInput, "actorType" | "actorId" | "agentId">) {
|
||||
const parsed = createCostEventSchema.safeParse({ ...data, occurredAt: data.occurredAt.toISOString() });
|
||||
const parsed = (data.usageKind === "decision" ? createServiceCostEventSchema : createCostEventSchema).safeParse({ ...data, occurredAt: data.occurredAt.toISOString() });
|
||||
if (!parsed.success) throw unprocessable("Invalid cost receipt", parsed.error.flatten());
|
||||
const values = {
|
||||
...parsed.data,
|
||||
@@ -132,7 +133,7 @@ export async function createCostEventInTransaction(db: Db, companyId: string, da
|
||||
// Separately reported charges linked to an already-accounted run contribute
|
||||
// to lifetime totals too. Before acknowledgement, accountRunCost includes
|
||||
// all of the run's events atomically when it initializes the projection.
|
||||
if (event.heartbeatRunId) {
|
||||
if (event.heartbeatRunId && event.agentId) {
|
||||
const [run] = await db.select({ acknowledged: heartbeatRuns.costAccountedAt, version: heartbeatRuns.accountingProjectionVersion })
|
||||
.from(heartbeatRuns).where(and(eq(heartbeatRuns.id, event.heartbeatRunId), eq(heartbeatRuns.companyId, companyId)));
|
||||
if (run.acknowledged && run.version === "v2") await db.update(agentRuntimeState).set({
|
||||
@@ -146,6 +147,7 @@ export async function createCostEventInTransaction(db: Db, companyId: string, da
|
||||
await budgetServiceInTransaction(db, publications).evaluateCostEvent(event);
|
||||
await logActivity(db, {
|
||||
companyId, actorType: actor?.actorType ?? "system", actorId: actor?.actorId ?? "cost_accounting", agentId: actor?.agentId ?? event.agentId,
|
||||
...(event.usageKind === "decision" ? { responsibleUserIdOverride: event.responsibleUserId } : {}),
|
||||
runId: event.heartbeatRunId, action: "cost.reported", entityType: "cost_event", entityId: event.id,
|
||||
details: { costCents: event.costCents, costCentsExact: values.costCents, model: event.model, costStatus: event.costStatus },
|
||||
}, publications);
|
||||
@@ -402,7 +404,7 @@ export function costService(db: Db, budgetHooks: BudgetServiceHooks = {}) {
|
||||
eq(companyMemberships.companyId, companyId),
|
||||
eq(companyMemberships.principalType, "user"),
|
||||
ne(companyMemberships.principalId, "local-board"),
|
||||
eq(companyMemberships.principalId, heartbeatRuns.responsibleUserId),
|
||||
eq(companyMemberships.principalId, sql`case when ${costEvents.usageKind} = 'decision' then ${costEvents.responsibleUserId} else ${heartbeatRuns.responsibleUserId} end`),
|
||||
))
|
||||
.where(and(...conditions))
|
||||
.groupBy(companyMemberships.principalId)
|
||||
@@ -463,8 +465,8 @@ export function costService(db: Db, budgetHooks: BudgetServiceHooks = {}) {
|
||||
.groupBy(costEvents.agentId, agents.name, agents.appearance, agents.status)
|
||||
.orderBy(desc(sumAsNumber(costEvents.costCents)));
|
||||
return rows.map(row => {
|
||||
const appearance = resolveAgentAppearance(row.agentAppearance, row.agentId);
|
||||
return { ...row, agentAppearance: appearance, avatarUrl: agentAvatarUrl(appearance, 512) };
|
||||
const appearance = row.agentId ? resolveAgentAppearance(row.agentAppearance, row.agentId) : null;
|
||||
return { ...row, agentAppearance: appearance, avatarUrl: appearance ? agentAvatarUrl(appearance, 512) : undefined };
|
||||
});
|
||||
},
|
||||
|
||||
@@ -630,8 +632,8 @@ export function costService(db: Db, budgetHooks: BudgetServiceHooks = {}) {
|
||||
)
|
||||
.orderBy(costEvents.provider, costEvents.biller, costEvents.billingType, costEvents.model);
|
||||
return rows.map(row => {
|
||||
const appearance = resolveAgentAppearance(row.agentAppearance, row.agentId);
|
||||
return { ...row, agentAppearance: appearance, avatarUrl: agentAvatarUrl(appearance, 512) };
|
||||
const appearance = row.agentId ? resolveAgentAppearance(row.agentAppearance, row.agentId) : null;
|
||||
return { ...row, agentAppearance: appearance, avatarUrl: appearance ? agentAvatarUrl(appearance, 512) : undefined };
|
||||
});
|
||||
},
|
||||
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
import { createOpenAI } from "@ai-sdk/openai";
|
||||
import { createOpenRouter } from "@openrouter/ai-sdk-provider";
|
||||
import { experimental_decide as decide, type Experimental_DecisionModel } from "ai";
|
||||
import { centsToUnits, unitsToCents, type DecisionAnswer, type DecisionProvider, type DecisionRequest } from "@paperclipai/shared";
|
||||
|
||||
export interface DecisionProviderReceipt {
|
||||
inputTokens: number | null; outputTokens: number | null; providerRequestId: string | null;
|
||||
costCents: string | null; costStatus: "reported" | "estimated" | "unpriced";
|
||||
pricingProvenance: { source: "provider_reported" | "rate_card" | "unknown"; version?: string; evidence?: string; inputCentsPerMillion?: string };
|
||||
}
|
||||
export interface DecisionProviderOutcome { noProviderWork?: boolean; answers?: Record<string, DecisionAnswer>; errorCode?: string; receipt: DecisionProviderReceipt }
|
||||
const record = (v: unknown): Record<string, unknown> => v !== null && typeof v === "object" && !Array.isArray(v) ? v as Record<string, unknown> : {};
|
||||
const tokenCount = (v: unknown) => typeof v === "number" && Number.isSafeInteger(v) && v >= 0 && v <= 2_147_483_647 ? v : null;
|
||||
function rejectedResponseData(error: unknown) {
|
||||
const value = record(error);
|
||||
if (value.data) return record(value.data);
|
||||
if (record(value.cause).value) return record(record(value.cause).value);
|
||||
// SDK schema rejection may retain only responseBody. Read a bounded body in
|
||||
// memory for accounting; never forward it to the ledger, logger, or caller.
|
||||
if (typeof value.responseBody === "string" && value.responseBody.length <= 1_000_000) {
|
||||
try { return record(JSON.parse(value.responseBody)); } catch { /* Not valid JSON. */ }
|
||||
}
|
||||
return {};
|
||||
}
|
||||
export function decisionReceipt(provider: DecisionProvider, result: unknown): DecisionProviderReceipt {
|
||||
const value = record(result), usage = record(value.usage), metadata = record(value.providerMetadata);
|
||||
const response = record(value.response);
|
||||
const headers = record(response.headers);
|
||||
const inputTokens = tokenCount(usage.inputTokens), outputTokens = tokenCount(usage.outputTokens);
|
||||
const reported = record(record(metadata.openrouter).usage).cost;
|
||||
const base = { inputTokens, outputTokens, providerRequestId: typeof response.id === "string" ? response.id.slice(0, 250) : typeof headers["x-request-id"] === "string" ? headers["x-request-id"].slice(0, 250) : null };
|
||||
if (provider === "openrouter" && typeof reported === "number" && Number.isFinite(reported) && reported >= 0) {
|
||||
return { ...base, costCents: unitsToCents(centsToUnits(reported * 100)), costStatus: "reported", pricingProvenance: { source: "provider_reported" } };
|
||||
}
|
||||
// V1 uses the official default endpoint and bounded short text requests. Decisions has its own rate, not the chat model's rate.
|
||||
if (provider === "openai" && inputTokens !== null && inputTokens <= 32_000) {
|
||||
return { ...base, costCents: unitsToCents(centsToUnits("10") * BigInt(inputTokens) / 1_000_000n), costStatus: "estimated",
|
||||
pricingProvenance: { source: "rate_card", version: "openai-decisions-2026-10-07", inputCentsPerMillion: "10.0000000" } };
|
||||
}
|
||||
return { ...base, costCents: null, costStatus: "unpriced", pricingProvenance: { source: "unknown" } };
|
||||
}
|
||||
export async function runDecisionProvider(input: {
|
||||
provider: DecisionProvider; model: string; apiKey: string; request: DecisionRequest; signal: AbortSignal;
|
||||
}, options: { fetch?: typeof fetch } = {}): Promise<DecisionProviderOutcome> {
|
||||
const native = input.provider === "openai"
|
||||
? createOpenAI({ apiKey: input.apiKey, fetch: options.fetch }).decisionModel(input.model)
|
||||
: createOpenRouter({ apiKey: input.apiKey, fetch: options.fetch }).evaluationModel(input.model);
|
||||
let receipt = decisionReceipt(input.provider, null);
|
||||
// Capture only accounting metadata before SDK answer validation/refusal errors.
|
||||
const model: Exclude<Experimental_DecisionModel, string> = {
|
||||
specificationVersion: "v4", provider: native.provider, modelId: native.modelId, supportedQuestionTypes: native.supportedQuestionTypes,
|
||||
async doDecide(args) {
|
||||
const response = await ("doDecide" in native ? native.doDecide(args) : native.doEvaluate(args));
|
||||
receipt = decisionReceipt(input.provider, response);
|
||||
return { ...response, warnings: [] };
|
||||
},
|
||||
};
|
||||
try {
|
||||
const result = await decide({ model, ...input.request, maxRetries: 0, abortSignal: input.signal,
|
||||
telemetry: { isEnabled: false, recordInputs: false, recordOutputs: false } });
|
||||
const metadata = record(result.providerMetadata);
|
||||
const answers = Object.fromEntries(Object.entries(result.answers).map(([id, answer]) => {
|
||||
const confidence = input.provider === "openai" ? record(record(metadata.openai).confidence)[id]
|
||||
: record(record(record(metadata.openrouter).answers)[id]).confidence;
|
||||
return [id, { ...answer, ...(typeof confidence === "number" && confidence >= 0 && confidence <= 1 ? { confidence } : {}) }];
|
||||
}));
|
||||
return { answers, receipt };
|
||||
} catch (error) {
|
||||
// Never expose provider error messages/bodies; they may contain inputs or credentials.
|
||||
const name = error instanceof Error ? error.name : "";
|
||||
const status = record(error).statusCode;
|
||||
// SDKs can reject refusals or malformed answers before normalized usage.
|
||||
// Keep only validated accounting fields, even in this failure path.
|
||||
const data = rejectedResponseData(error);
|
||||
const rawUsage = record(data.usage);
|
||||
if (receipt.costStatus === "unpriced") {
|
||||
receipt = decisionReceipt(input.provider, { usage: { inputTokens: rawUsage.input_tokens, outputTokens: rawUsage.output_tokens },
|
||||
providerMetadata: { openrouter: { usage: { cost: rawUsage.cost } } },
|
||||
response: { id: data.id, headers: record(error).responseHeaders } });
|
||||
}
|
||||
const rawAnswers = Array.isArray(data.answers) ? data.answers : Object.values(record(data.answers));
|
||||
const refused = name.includes("Refusal") || record(data.error).type === "refusal" || rawAnswers.some(answer => record(answer).type === "refusal");
|
||||
const errorCode = input.signal.aborted ? "timeout" : refused ? "refused"
|
||||
: status === 401 || status === 403 ? "provider_auth_failed" : status === 429 ? "provider_rate_limited" : "provider_failed";
|
||||
// An explicit authentication/validation rejection occurs before model work.
|
||||
// Server errors and network failures remain unresolved: their charge is unknown.
|
||||
const noProviderWork = receipt.costStatus === "unpriced" && [400, 401, 403, 404, 422, 429].includes(Number(status));
|
||||
if (noProviderWork) receipt = { ...receipt, inputTokens: 0, outputTokens: 0, costCents: "0.0000000", costStatus: "estimated",
|
||||
pricingProvenance: { source: "unknown", evidence: `Provider rejected request before model work (HTTP ${status})` } };
|
||||
return { errorCode, receipt, noProviderWork };
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,273 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { and, desc, eq, gte, lt, lte, or, sql } from "drizzle-orm";
|
||||
import {
|
||||
agents, authUsers, budgetPolicies, budgetReservations, companies, companyMemberships, companyDecisionModels, costEvents,
|
||||
decisionInvocations, heartbeatRuns, issues, projects, companySecrets, type Db,
|
||||
} from "@paperclipai/db";
|
||||
import {
|
||||
DECISION_MODELS, decisionProviderForConnection, decisionRequestSchema, updateDecisionModelSchema,
|
||||
centsToUnits, unitsToCents, type DecisionAvailability, type DecisionConnectionChoice, type DecisionHistoryEntry,
|
||||
type DecisionModelSettings, type DecisionProvider, type DecisionRequest, type DecisionResult, type DecisionUnavailableReason, type UpdateDecisionModel,
|
||||
} from "@paperclipai/shared";
|
||||
import { HttpError, forbidden, unprocessable } from "../errors.js";
|
||||
import { aiConnectionService } from "./ai-connections.js";
|
||||
import { accessService } from "./access.js";
|
||||
import { type AuthorizationActor } from "./authorization.js";
|
||||
import { withAccountingTransaction } from "./accounting-transaction.js";
|
||||
import { budgetPoliciesForRun, budgetService, budgetServiceInTransaction, computeObservedSpend, type BudgetServiceHooks } from "./budgets.js";
|
||||
import { createCostEventInTransaction } from "./costs.js";
|
||||
import { logActivity } from "./activity-log.js";
|
||||
import { decisionReceipt, runDecisionProvider, type DecisionProviderOutcome } from "./decision-model-provider.js";
|
||||
|
||||
const registeredFeatures = new WeakSet<object>();
|
||||
/** Register once in a server module. Feature handles cannot be supplied over JSON/API. */
|
||||
export function defineDecisionFeature(id: string, options: { background?: boolean } = {}) {
|
||||
if (!/^[a-z][a-z0-9_.-]{0,99}$/.test(id)) throw new Error("Invalid decision feature identifier");
|
||||
const feature = Object.freeze({ id, background: options.background === true });
|
||||
registeredFeatures.add(feature);
|
||||
return feature;
|
||||
}
|
||||
export const settingsDecisionTest = defineDecisionFeature("settings.test");
|
||||
export interface DecisionContext {
|
||||
companyId: string;
|
||||
feature: ReturnType<typeof defineDecisionFeature>;
|
||||
actor: AuthorizationActor | { type: "system" };
|
||||
issueId?: string;
|
||||
projectId?: string;
|
||||
}
|
||||
class DecisionUnavailable extends Error {
|
||||
constructor(readonly reason: DecisionUnavailableReason) { super(reason); }
|
||||
}
|
||||
function unavailableReason(error: unknown): DecisionUnavailableReason | null {
|
||||
if (error instanceof DecisionUnavailable) return error.reason;
|
||||
if (error instanceof HttpError) {
|
||||
if (error.status === 403 || error.status === 404) return "access_denied";
|
||||
if (error.status === 422) {
|
||||
return (error.details as { code?: string } | undefined)?.code === "incompatible_connection" ? "incompatible_connection" : "connection_unavailable";
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export function decisionModelService(db: Db, options: { provider?: typeof runDecisionProvider; budgetHooks?: BudgetServiceHooks } = {}) {
|
||||
const providerCall = options.provider ?? runDecisionProvider;
|
||||
async function settings(companyId: string): Promise<DecisionModelSettings> {
|
||||
const [row] = await db.select().from(companyDecisionModels).where(eq(companyDecisionModels.companyId, companyId));
|
||||
return row ? { ...row, provider: row.provider as DecisionProvider | null } : {
|
||||
companyId, enabled: false, allowBackground: true, connectionId: null, grantId: null, provider: null, model: null,
|
||||
};
|
||||
}
|
||||
async function choices(companyId: string, userId: string): Promise<DecisionConnectionChoice[]> {
|
||||
const connections = aiConnectionService(db);
|
||||
if (!(await connections.membership(companyId, userId))) throw forbidden();
|
||||
return (await connections.list(companyId, userId)).flatMap(row => {
|
||||
const provider = decisionProviderForConnection(row);
|
||||
return provider && row.ownership === "shared" ? [{ ...row, decisionModel: DECISION_MODELS[provider].id }] : [];
|
||||
});
|
||||
}
|
||||
async function configure(companyId: string, userId: string, input: UpdateDecisionModel) {
|
||||
const data = updateDecisionModelSchema.parse(input);
|
||||
return withAccountingTransaction(db, companyId, async (tx, publications) => {
|
||||
const [previous] = await tx.select().from(companyDecisionModels).where(eq(companyDecisionModels.companyId, companyId));
|
||||
// Managers can always turn off a broken/revoked connection without acquiring it.
|
||||
const disabling = !data.enabled && previous?.connectionId === data.connectionId && previous?.grantId === data.grantId;
|
||||
const row = !disabling && data.connectionId && data.grantId ? await aiConnectionService(tx).selectDecision({ companyId,
|
||||
userId, connectionId: data.connectionId, grantId: data.grantId }) : null;
|
||||
const provider = (disabling ? previous?.provider : row?.provider) as DecisionProvider | null | undefined;
|
||||
const values = { ...data, allowBackground: data.allowBackground ?? previous?.allowBackground ?? true, companyId, provider: provider ?? null, model: provider ? DECISION_MODELS[provider].id : null, updatedAt: new Date() };
|
||||
await tx.insert(companyDecisionModels).values(values).onConflictDoUpdate({ target: companyDecisionModels.companyId, set: values });
|
||||
await logActivity(tx, { companyId, actorType: "user", actorId: userId, action: "decision_model.configured", entityType: "company", entityId: companyId,
|
||||
details: { ...data, provider: values.provider, model: values.model } }, publications);
|
||||
return values;
|
||||
});
|
||||
}
|
||||
async function resolve(context: DecisionContext, executor = db) {
|
||||
if (!registeredFeatures.has(context.feature)) throw forbidden("A registered decision feature is required");
|
||||
const [config] = await executor.select().from(companyDecisionModels).where(eq(companyDecisionModels.companyId, context.companyId));
|
||||
if (!config?.connectionId || !config.grantId) throw new DecisionUnavailable("not_configured");
|
||||
if (!config.enabled) throw new DecisionUnavailable("disabled");
|
||||
let userId: string | null = null, agentId: string | null = null, runId: string | null = null, identityContextId: string | null = null;
|
||||
let issueId = context.issueId ?? null;
|
||||
let actor = context.actor;
|
||||
if (actor.type === "system") {
|
||||
if (!context.feature.background || !config.allowBackground) throw new DecisionUnavailable("background_disabled");
|
||||
} else if (actor.type === "agent") {
|
||||
if (!actor.agentId || !actor.runId || actor.companyId !== context.companyId) throw forbidden();
|
||||
const [run] = await executor.select().from(heartbeatRuns).where(and(eq(heartbeatRuns.companyId, context.companyId), eq(heartbeatRuns.id, actor.runId), eq(heartbeatRuns.agentId, actor.agentId)));
|
||||
if (!run || run.status !== "running") throw forbidden("An active run is required");
|
||||
agentId = run.agentId; runId = run.id; userId = run.responsibleUserId; identityContextId = run.activeIdentityContextId;
|
||||
const runIssue = run.nativeIssueId ?? run.contextSnapshot?.issueId ?? run.contextSnapshot?.taskId;
|
||||
if (issueId && issueId !== runIssue) throw forbidden("Task does not belong to this run");
|
||||
issueId = typeof runIssue === "string" ? runIssue : null;
|
||||
const memberships = userId ? await executor.select({ companyId: companyMemberships.companyId, membershipRole: companyMemberships.membershipRole, status: companyMemberships.status })
|
||||
.from(companyMemberships).where(and(eq(companyMemberships.companyId, context.companyId), eq(companyMemberships.principalType, "user"), eq(companyMemberships.principalId, userId))) : [];
|
||||
actor = { ...actor, onBehalfOfUserId: userId, onBehalfOfMemberships: memberships };
|
||||
} else if (actor.type === "board") userId = actor.userId ?? (actor.source === "local_implicit" ? "local-board" : null);
|
||||
else throw forbidden();
|
||||
if (actor.type !== "system" && !userId) throw new DecisionUnavailable("responsible_user_missing");
|
||||
const [issue] = issueId ? await executor.select().from(issues).where(and(eq(issues.id, issueId), eq(issues.companyId, context.companyId))) : [];
|
||||
if (issueId && !issue) throw forbidden();
|
||||
if (issue && context.projectId && context.projectId !== issue.projectId) throw forbidden("Task project mismatch");
|
||||
const projectId = issue?.projectId ?? context.projectId ?? null;
|
||||
if (projectId) {
|
||||
const [project] = await executor.select({ id: projects.id }).from(projects).where(and(eq(projects.id, projectId), eq(projects.companyId, context.companyId)));
|
||||
if (!project) throw forbidden();
|
||||
}
|
||||
if (actor.type !== "system") {
|
||||
const access = accessService(executor);
|
||||
const accessResult = await access.decide({ actor, action: issue ? "issue:read" : projectId ? "project:read" : "company_scope:read",
|
||||
resource: issue ? { type: "issue", companyId: context.companyId, issueId: issue.id, projectId: issue.projectId, parentIssueId: issue.parentId,
|
||||
assigneeAgentId: issue.assigneeAgentId, assigneeUserId: issue.assigneeUserId, status: issue.status }
|
||||
: projectId ? { type: "project", companyId: context.companyId, projectId } : { type: "company", companyId: context.companyId } });
|
||||
if (!accessResult.allowed) throw forbidden();
|
||||
}
|
||||
const connection = await aiConnectionService(executor).selectDecision({ companyId: context.companyId,
|
||||
connectionId: config.connectionId, grantId: config.grantId, userId, agentId, sponsoredBackground: actor.type === "system" });
|
||||
if (connection.provider !== config.provider || config.model !== DECISION_MODELS[connection.provider].id) throw new DecisionUnavailable("incompatible_connection");
|
||||
return { config, connection, userId, agentId, runId, identityContextId, issueId, projectId,
|
||||
actorType: actor.type === "board" ? "user" as const : actor.type === "agent" ? "agent" as const : "system" as const,
|
||||
actorId: actor.type === "system" ? "paperclip-decisions" : agentId ?? userId! };
|
||||
}
|
||||
async function availability(context: DecisionContext): Promise<DecisionAvailability> {
|
||||
try { await resolve(context); return { available: true }; }
|
||||
catch (error) { const reason = unavailableReason(error); if (reason) return { available: false, reason }; throw error; }
|
||||
}
|
||||
/** Callers may memoize within their request; dispatch never consumes a cached authorization. */
|
||||
function availabilityForRequest() {
|
||||
const cache = new Map<DecisionContext, Promise<DecisionAvailability>>();
|
||||
return (context: DecisionContext) => { let result = cache.get(context); if (!result) { result = availability(context); cache.set(context, result); } return result; };
|
||||
}
|
||||
async function settle(invocation: typeof decisionInvocations.$inferSelect, outcome: DecisionProviderOutcome) {
|
||||
await withAccountingTransaction(db, invocation.companyId, async (tx, publications) => {
|
||||
const [current] = await tx.select().from(decisionInvocations).where(and(eq(decisionInvocations.companyId, invocation.companyId), eq(decisionInvocations.id, invocation.id))).for("update");
|
||||
if (!current || current.status !== "running") return;
|
||||
const receipt = outcome.receipt;
|
||||
// Keep dispatch identity on the invocation even if its task or agent was
|
||||
// deleted while the provider worked. Ledger foreign keys use live rows.
|
||||
const links: { agentId: string | null; issueId: string | null; projectId: string | null } = { agentId: null, issueId: null, projectId: null };
|
||||
for (const [key, table] of [["agentId", agents], ["issueId", issues], ["projectId", projects]] as const) {
|
||||
const id = invocation[key];
|
||||
if (!id) continue;
|
||||
const [row] = await tx.select({ id: table.id }).from(table).where(and(eq(table.id, id), eq(table.companyId, invocation.companyId))).for("key share");
|
||||
if (row) links[key] = row.id;
|
||||
}
|
||||
const event = await createCostEventInTransaction(tx, invocation.companyId, {
|
||||
usageKind: "decision", responsibleUserId: invocation.responsibleUserId, ...links,
|
||||
// Run attribution lives on the invocation. Run receipt reconciliation must not ingest this independent charge.
|
||||
heartbeatRunId: null, idempotencyKey: `decision:${invocation.id}`,
|
||||
provider: invocation.provider === "openrouter" ? "typesafe" : "openai", biller: invocation.provider,
|
||||
billingType: "metered_api", costStatus: receipt.costStatus, model: invocation.model,
|
||||
inputTokens: receipt.inputTokens ?? 0, outputTokens: receipt.outputTokens ?? 0, costCents: receipt.costCents ?? "0",
|
||||
providerRequestId: receipt.providerRequestId, pricingProvenance: receipt.pricingProvenance, occurredAt: invocation.startedAt,
|
||||
}, publications, { actorType: invocation.actorType as "system" | "user" | "agent", actorId: invocation.actorId, agentId: links.agentId });
|
||||
const finishedAt = new Date();
|
||||
await tx.update(decisionInvocations).set({ status: outcome.answers ? "succeeded" : receipt.costStatus === "unpriced" ? "unknown" : "failed",
|
||||
errorCode: outcome.errorCode ?? null, costEventId: event.id, providerRequestId: receipt.providerRequestId,
|
||||
inputTokens: receipt.inputTokens, outputTokens: receipt.outputTokens, finishedAt, durationMs: Math.max(0, finishedAt.getTime() - invocation.startedAt.getTime()),
|
||||
}).where(eq(decisionInvocations.id, invocation.id));
|
||||
if (receipt.costStatus !== "unpriced") await tx.update(budgetReservations).set({ state: outcome.noProviderWork ? "released" : "settled", settledAt: finishedAt }).where(and(
|
||||
eq(budgetReservations.companyId, invocation.companyId), eq(budgetReservations.decisionInvocationId, invocation.id), eq(budgetReservations.state, "held"),
|
||||
));
|
||||
});
|
||||
await budgetService(db, options.budgetHooks).deliverPendingEnforcement(invocation.companyId);
|
||||
}
|
||||
async function decide(context: DecisionContext, request: DecisionRequest, signal?: AbortSignal): Promise<DecisionResult> {
|
||||
const parsed = decisionRequestSchema.safeParse(request);
|
||||
if (!parsed.success || Buffer.byteLength(JSON.stringify(parsed.data)) > 32_000) throw unprocessable("Decisions require 1–20 valid questions and at most 32 KB of text/JSON");
|
||||
let selected: Awaited<ReturnType<typeof resolve>>;
|
||||
try { selected = await resolve(context); }
|
||||
catch (error) { const reason = unavailableReason(error); if (reason) return { status: "unavailable", reason }; throw error; }
|
||||
const secretId = selected.connection.grant.credentialSecretRefs.find(ref => ref.configPath === "ai.credential")?.secretId;
|
||||
const [credentialSnapshot] = secretId ? await db.select({ id: companySecrets.id, status: companySecrets.status, version: companySecrets.latestVersion })
|
||||
.from(companySecrets).where(and(eq(companySecrets.companyId, context.companyId), eq(companySecrets.id, secretId))) : [];
|
||||
if (!credentialSnapshot || credentialSnapshot.status !== "active") return { status: "unavailable", reason: "connection_unavailable" };
|
||||
let credential: string;
|
||||
try { credential = await aiConnectionService(db).credential(selected.connection, 0, { responsibleUserId: selected.userId,
|
||||
actorType: selected.actorType, actorId: selected.actorId, issueId: selected.issueId, heartbeatRunId: selected.runId }); }
|
||||
catch (error) { return { status: "unavailable", reason: unavailableReason(error) ?? "connection_unavailable" }; }
|
||||
const admission = await withAccountingTransaction(db, context.companyId, async (tx, publications) => {
|
||||
let current: Awaited<ReturnType<typeof resolve>>;
|
||||
try { current = await resolve(context, tx); }
|
||||
catch (error) { const reason = unavailableReason(error); if (reason) return { reason }; throw error; }
|
||||
// A different setting or steering identity requires a fresh credential acquisition.
|
||||
if (current.config.updatedAt.getTime() !== selected.config.updatedAt.getTime() || current.userId !== selected.userId || current.identityContextId !== selected.identityContextId
|
||||
|| current.connection.grant.updatedAt.getTime() !== selected.connection.grant.updatedAt.getTime()
|
||||
|| current.connection.connection.updatedAt.getTime() !== selected.connection.connection.updatedAt.getTime())
|
||||
return { reason: "connection_unavailable" as const };
|
||||
// Secret changes do not update the connection revision. Serialize against
|
||||
// rotation/revocation through admission, including time spent waiting for
|
||||
// the accounting lock after credential resolution.
|
||||
const [currentSecret] = await tx.select({ status: companySecrets.status, version: companySecrets.latestVersion }).from(companySecrets)
|
||||
.where(and(eq(companySecrets.companyId, context.companyId), eq(companySecrets.id, credentialSnapshot.id))).for("share");
|
||||
if (!currentSecret || currentSecret.status !== "active" || currentSecret.version !== credentialSnapshot.version)
|
||||
return { reason: "connection_unavailable" as const };
|
||||
const budgets = budgetServiceInTransaction(tx, publications);
|
||||
const block = await budgets.getInvocationBlock(context.companyId, current.agentId, { projectId: current.projectId });
|
||||
const [company] = await tx.select({ status: companies.status }).from(companies).where(eq(companies.id, context.companyId));
|
||||
const [agent] = current.agentId ? await tx.select({ status: agents.status }).from(agents).where(and(eq(agents.id, current.agentId), eq(agents.companyId, context.companyId))) : [];
|
||||
if (block || company.status !== "active" || (current.agentId && (!agent || agent.status === "paused" || agent.status === "terminated"))) return { reason: "budget_blocked" as const };
|
||||
const policies = await tx.select().from(budgetPolicies).where(and(eq(budgetPolicies.companyId, context.companyId), eq(budgetPolicies.isActive, true),
|
||||
eq(budgetPolicies.hardStopEnabled, true), eq(budgetPolicies.metric, "billed_cents"), sql`${budgetPolicies.amount} > 0`, budgetPoliciesForRun(context.companyId, current.agentId, current.projectId)));
|
||||
const amount = policies.reduce((max, p) => { const n = centsToUnits(p.reservationCents); return n > max ? n : max; }, 0n);
|
||||
for (const policy of policies) {
|
||||
const [held] = await tx.select({ total: sql<string>`coalesce(sum(${budgetReservations.amountCents}),0)::text` }).from(budgetReservations).where(and(
|
||||
eq(budgetReservations.companyId, context.companyId), eq(budgetReservations.state, "held"),
|
||||
policy.scopeType === "agent" ? eq(budgetReservations.agentId, current.agentId!) : undefined,
|
||||
policy.scopeType === "project" ? eq(budgetReservations.projectId, current.projectId!) : undefined,
|
||||
));
|
||||
const committed = centsToUnits((await computeObservedSpend(tx, policy)).totalExact) + centsToUnits(held.total);
|
||||
if (committed >= centsToUnits(policy.amount) || committed + amount > centsToUnits(policy.amount)) return { reason: "budget_blocked" as const };
|
||||
}
|
||||
signal?.throwIfAborted();
|
||||
const [invocation] = await tx.insert(decisionInvocations).values({ id: randomUUID(), companyId: context.companyId, feature: context.feature.id,
|
||||
actorType: current.actorType, actorId: current.actorId, responsibleUserId: current.userId, agentId: current.agentId,
|
||||
issueId: current.issueId, projectId: current.projectId, runId: current.runId, identityContextId: current.identityContextId,
|
||||
connectionId: current.connection.connection.id, grantId: current.connection.grant.id, provider: current.connection.provider, model: current.config.model!,
|
||||
questionTypes: Object.values(parsed.data.questions).map(q => q.type),
|
||||
}).returning();
|
||||
await tx.insert(budgetReservations).values({ companyId: context.companyId, decisionInvocationId: invocation.id, agentId: current.agentId,
|
||||
projectId: current.projectId, amountCents: unitsToCents(amount), providerStartedAt: invocation.startedAt });
|
||||
return { invocation };
|
||||
});
|
||||
if (admission.reason) return { status: "unavailable", reason: admission.reason };
|
||||
const invocation = admission.invocation!;
|
||||
let outcome: DecisionProviderOutcome;
|
||||
try { outcome = await providerCall({ provider: selected.connection.provider, model: invocation.model, apiKey: credential,
|
||||
request: parsed.data, signal: signal ? AbortSignal.any([signal, AbortSignal.timeout(30_000)]) : AbortSignal.timeout(30_000) }); }
|
||||
catch { outcome = { errorCode: "provider_failed", receipt: decisionReceipt(selected.connection.provider, null) }; }
|
||||
await settle(invocation, outcome);
|
||||
const { inputTokens, outputTokens, costCents, costStatus } = outcome.receipt;
|
||||
const usage = { inputTokens, outputTokens, costCents, costStatus };
|
||||
return outcome.answers ? { status: "succeeded", invocationId: invocation.id, answers: outcome.answers, usage }
|
||||
: { status: "failed", invocationId: invocation.id, errorCode: outcome.errorCode ?? "provider_failed", usage };
|
||||
}
|
||||
async function recoverInterrupted() {
|
||||
const stale = await db.select().from(decisionInvocations).where(and(eq(decisionInvocations.status, "running"), lt(decisionInvocations.startedAt, new Date(Date.now() - 120_000)))).limit(100);
|
||||
for (const invocation of stale) await settle(invocation, { errorCode: "interrupted", receipt: decisionReceipt(invocation.provider as DecisionProvider, null) });
|
||||
return stale.length;
|
||||
}
|
||||
async function history(companyId: string, actor: AuthorizationActor, range: { from?: Date; to?: Date; before?: Date; limit?: number } = {}): Promise<DecisionHistoryEntry[]> {
|
||||
const access = accessService(db);
|
||||
if (!(await access.decide({ actor, action: "company_scope:read", resource: { type: "company", companyId } })).allowed) throw forbidden();
|
||||
const rows = await db.select({ invocation: decisionInvocations, userName: authUsers.name,
|
||||
inputTokens: costEvents.inputTokens, outputTokens: costEvents.outputTokens, costStatus: costEvents.costStatus,
|
||||
costCents: sql<string | null>`${costEvents.costCents}::text`, issue: issues,
|
||||
}).from(decisionInvocations).leftJoin(costEvents, and(eq(costEvents.id, decisionInvocations.costEventId), eq(costEvents.companyId, companyId)))
|
||||
.leftJoin(authUsers, eq(authUsers.id, decisionInvocations.responsibleUserId))
|
||||
.leftJoin(issues, and(eq(issues.id, decisionInvocations.issueId), eq(issues.companyId, companyId)))
|
||||
.where(and(eq(decisionInvocations.companyId, companyId), range.from ? gte(decisionInvocations.startedAt, range.from) : undefined,
|
||||
range.to ? lte(decisionInvocations.startedAt, range.to) : undefined, range.before ? lt(decisionInvocations.startedAt, range.before) : undefined))
|
||||
.orderBy(desc(decisionInvocations.startedAt), desc(decisionInvocations.id)).limit(Math.min(range.limit ?? 100, 500));
|
||||
return Promise.all(rows.map(async ({ invocation: row, issue, ...cost }) => {
|
||||
const visible = issue && (await access.decide({ actor, action: "issue:read", resource: { type: "issue", companyId, issueId: issue.id,
|
||||
projectId: issue.projectId, parentIssueId: issue.parentId, assigneeAgentId: issue.assigneeAgentId, assigneeUserId: issue.assigneeUserId, status: issue.status } })).allowed;
|
||||
return { id: row.id, feature: row.feature, actorType: row.actorType, responsibleUserId: row.responsibleUserId, userName: cost.userName,
|
||||
issueId: visible ? issue.id : null, issueIdentifier: visible ? issue.identifier : null, agentId: row.agentId, runId: visible ? row.runId : null,
|
||||
connectionId: row.connectionId, provider: row.provider, model: row.model, status: row.status, errorCode: row.errorCode,
|
||||
startedAt: row.startedAt.toISOString(), finishedAt: row.finishedAt?.toISOString() ?? null, durationMs: row.durationMs,
|
||||
inputTokens: row.inputTokens, outputTokens: row.outputTokens,
|
||||
costCents: cost.costStatus === "unpriced" ? null : cost.costCents, costStatus: cost.costStatus };
|
||||
}));
|
||||
}
|
||||
return { settings, choices, configure, availability, availabilityForRequest, decide, recoverInterrupted, history };
|
||||
}
|
||||
@@ -2,6 +2,7 @@ import { preserveWorkspaceRestoreRecoveryMetadataSql } from "./legacy-workspace-
|
||||
import { preserveWorkspaceRestoreRecoveryMetadata } from "./workspace-restore-recovery-state.js";
|
||||
import { recordLegacyWorkspaceRestoreFailure } from "./legacy-execution-recovery.js";
|
||||
import { CONFIGURED_ENVIRONMENT_KEYS, configuredEnvironmentProjection } from "../vendor/paperclip-runner/index.js";
|
||||
import { decisionModelService } from "./decision-models.js";
|
||||
import { activeIssueInteractionCondition, TASK_QUESTION_GUIDANCE } from "./issue-question-context.js";
|
||||
import { createAgentIdentityRedactor } from "./agent-identity-redaction.js";
|
||||
import { agentIdentityService, supportsManagedAgentIdentity } from "./agent-identity.js";
|
||||
@@ -30667,7 +30668,10 @@ export function heartbeatService(
|
||||
scanSilentActiveRuns,
|
||||
|
||||
reconcileTaskWatchdogs,
|
||||
reconcileCostAccounting: createCostAccountingReconciler(db, budgetHooks),
|
||||
reconcileCostAccounting: async () => {
|
||||
await decisionModelService(db, { budgetHooks }).recoverInterrupted();
|
||||
return createCostAccountingReconciler(db, budgetHooks)();
|
||||
},
|
||||
|
||||
buildRunOutputSilence,
|
||||
|
||||
|
||||
@@ -217,3 +217,5 @@ export {
|
||||
type RemoteAgentProfileInput,
|
||||
type RemoteAgentService,
|
||||
} from "./remote-agent-profiles.js";
|
||||
|
||||
export { decisionModelService, defineDecisionFeature } from "./decision-models.js";
|
||||
@@ -0,0 +1,14 @@
|
||||
import type { DecisionConnectionChoice, DecisionHistoryEntry, DecisionModelSettings, DecisionResult, UpdateDecisionModel } from "@paperclipai/shared";
|
||||
import { api } from "./client";
|
||||
export interface DecisionSettingsResponse { canManage: boolean; settings: DecisionModelSettings | null; choices: DecisionConnectionChoice[] }
|
||||
export const decisionModelsApi = {
|
||||
settings: (companyId: string) => api.get<DecisionSettingsResponse>(`/companies/${companyId}/decision-model`),
|
||||
update: (companyId: string, settings: UpdateDecisionModel) => api.put<DecisionModelSettings>(`/companies/${companyId}/decision-model`, settings),
|
||||
test: (companyId: string) => api.post<DecisionResult>(`/companies/${companyId}/decision-model/test`, {}),
|
||||
history: (companyId: string, from?: string | null, to?: string | null) => {
|
||||
const params = new URLSearchParams({ limit: "100" });
|
||||
if (from) params.set("from", from);
|
||||
if (to) params.set("to", to);
|
||||
return api.get<DecisionHistoryEntry[]>(`/companies/${companyId}/decision-model/history?${params}`);
|
||||
},
|
||||
};
|
||||
@@ -123,6 +123,7 @@ export function ToggleField({
|
||||
canonical ToggleSwitch (status-green on-state), DESIGN.md principle 1. */}
|
||||
<ToggleSwitch
|
||||
data-testid={toggleTestId}
|
||||
aria-label={label}
|
||||
checked={checked}
|
||||
onCheckedChange={onChange}
|
||||
/>
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
import { useQuery } from "@tanstack/react-query";
|
||||
import type { DecisionHistoryEntry } from "@paperclipai/shared";
|
||||
import { decisionModelsApi } from "@/api/decision-models";
|
||||
import { Link } from "@/lib/router";
|
||||
import { formatDateTime, formatTokens, formatDetailedCents } from "@/lib/utils";
|
||||
import { Button } from "@/components/ui/button";
|
||||
|
||||
export function DecisionHistoryTable({ entries }: { entries: DecisionHistoryEntry[] }) {
|
||||
if (!entries.length) return <p className="text-sm text-muted-foreground">No decision requests in this period. <Link to="/company/settings" className="underline underline-offset-4">Configure a decision model</Link> to run a test.</p>;
|
||||
return <div className="space-y-3">
|
||||
<p className="text-xs text-muted-foreground">Recent requests. Inputs and answers are not retained.</p>
|
||||
<div className="overflow-x-auto"><table className="w-full text-sm" aria-label="Decision requests">
|
||||
<thead><tr className="border-b border-border text-muted-foreground">{["When / feature", "Responsible", "Task", "Model", "Status", "Duration", "Tokens in / out", "Cost"].map(label => <th key={label} className="px-3 py-2 text-left font-medium">{label}</th>)}</tr></thead>
|
||||
<tbody>{entries.map(row => <tr key={row.id} className="border-b border-border last:border-0">
|
||||
<td className="px-3 py-3"><div className="whitespace-nowrap">{formatDateTime(row.startedAt)}</div><div className="text-xs text-muted-foreground">{row.feature === "settings.test" ? "Setup test" : row.feature}</div></td>
|
||||
<td className="px-3 py-3">{row.actorType === "system" ? "Paperclip services" : row.userName ?? (row.responsibleUserId === "local-board" ? "Board" : "Former user")}</td>
|
||||
<td className="px-3 py-3">{row.issueId ? <Link className="underline underline-offset-4" to={`/issues/${row.issueIdentifier ?? row.issueId}`}>{row.issueIdentifier ?? "View task"}</Link> : "—"}</td>
|
||||
<td className="px-3 py-3 font-mono text-xs">{row.model}</td>
|
||||
<td className="px-3 py-3"><div>{row.status === "succeeded" ? "Succeeded" : row.status === "running" ? "Running" : row.status === "unknown" ? "Unresolved" : "Failed"}</div>{row.errorCode && <div className="text-xs text-muted-foreground">{row.errorCode.replaceAll("_", " ")}</div>}</td>
|
||||
<td className="px-3 py-3 font-mono">{row.durationMs === null ? "—" : `${(row.durationMs / 1000).toFixed(1)}s`}</td>
|
||||
<td className="whitespace-nowrap px-3 py-3 font-mono">{row.inputTokens === null ? "—" : formatTokens(row.inputTokens)} / {row.outputTokens === null ? "—" : formatTokens(row.outputTokens)}</td>
|
||||
<td className="px-3 py-3 font-mono"><div>{row.costCents === null ? "Unknown" : formatDetailedCents(row.costCents)}</div>{row.costStatus !== "unpriced" && row.costStatus && <span className="text-xs text-muted-foreground">{row.costStatus === "estimated" ? "Estimated" : "Reported"}</span>}</td>
|
||||
</tr>)}</tbody>
|
||||
</table></div>
|
||||
</div>;
|
||||
}
|
||||
export function DecisionHistory({ companyId, from, to }: { companyId: string; from?: string | null; to?: string | null }) {
|
||||
const query = useQuery({ queryKey: ["decision-history", companyId, from, to], queryFn: () => decisionModelsApi.history(companyId, from, to), refetchInterval: q => q.state.data?.some(row => row.status === "running") ? 5000 : false });
|
||||
if (query.isPending) return <p className="text-sm text-muted-foreground">Loading decision requests…</p>;
|
||||
if (query.error) return <div role="alert" className="space-y-2"><p className="text-sm text-destructive">Could not load decision requests.</p><Button variant="outline" onClick={() => void query.refetch()}>Try again</Button></div>;
|
||||
return <DecisionHistoryTable entries={query.data ?? []} />;
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
// @vitest-environment jsdom
|
||||
import { act } from "react";
|
||||
import { createRoot } from "react-dom/client";
|
||||
import { MemoryRouter } from "react-router-dom";
|
||||
import { expect, it, vi } from "vitest";
|
||||
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
|
||||
import { decisionModelsApi } from "@/api/decision-models";
|
||||
import { DecisionModelSettingsSection, DecisionModelSettingsView } from "./DecisionModelSettings";
|
||||
import { choices, configured, result } from "../../../storybook/stories/decision-models/fixtures";
|
||||
|
||||
vi.mock("@/context/CompanyContext", () => ({ useCompany: () => ({ selectedCompanyId: "company-storybook" }) }));
|
||||
(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true;
|
||||
|
||||
it("consumes an added connection once and preserves a later choice across refetches", async () => {
|
||||
const container = document.createElement("div"); document.body.appendChild(container);
|
||||
const root = createRoot(container), onSave = vi.fn();
|
||||
const settings = { ...configured, allowBackground: false };
|
||||
const render = (refreshed = false) => <MemoryRouter><DecisionModelSettingsView settings={{ ...settings }}
|
||||
choices={choices.map(row => ({ ...row, name: refreshed ? `${row.name} refreshed` : row.name }))}
|
||||
newlyConnectedId={choices[0]!.id} onSave={onSave} onTest={vi.fn()} onAdd={vi.fn()} /></MemoryRouter>;
|
||||
try {
|
||||
await act(async () => root.render(render()));
|
||||
await act(async () => [...container.querySelectorAll("button")].find(button => button.textContent?.includes("Company OpenRouter"))!.click());
|
||||
await act(async () => root.render(render(true)));
|
||||
expect(container.querySelector('[aria-pressed="true"]')?.textContent).toContain("Company OpenRouter refreshed");
|
||||
await act(async () => [...container.querySelectorAll("button")].find(button => button.textContent === "Save decision model")!.click());
|
||||
expect(onSave).toHaveBeenCalledWith(expect.objectContaining({ connectionId: choices[1]!.id, grantId: choices[1]!.grantId, allowBackground: false }));
|
||||
} finally { act(() => root.unmount()); container.remove(); }
|
||||
});
|
||||
|
||||
it("clears test feedback when another manager changes the saved configuration", async () => {
|
||||
const container = document.createElement("div"); document.body.appendChild(container);
|
||||
const root = createRoot(container);
|
||||
const client = new QueryClient({ defaultOptions: { queries: { retry: false, staleTime: Infinity } } });
|
||||
const key = ["decision-model", configured.companyId];
|
||||
client.setQueryData(key, { canManage: true, settings: configured, choices });
|
||||
const test = vi.spyOn(decisionModelsApi, "test").mockResolvedValue(result);
|
||||
try {
|
||||
await act(async () => root.render(<QueryClientProvider client={client}><MemoryRouter>
|
||||
<DecisionModelSettingsSection companyId={configured.companyId} />
|
||||
</MemoryRouter></QueryClientProvider>));
|
||||
await act(async () => {
|
||||
[...container.querySelectorAll("button")].find(button => button.textContent === "Run test")!.click();
|
||||
await new Promise(resolve => setTimeout(resolve, 10));
|
||||
});
|
||||
expect(container.textContent).toContain("Decision model is working");
|
||||
await act(async () => {
|
||||
client.setQueryData(key, { canManage: true, settings: { ...configured, enabled: false }, choices });
|
||||
await new Promise(resolve => setTimeout(resolve, 10));
|
||||
});
|
||||
expect(container.textContent).not.toContain("Decision model is working");
|
||||
await act(async () => {
|
||||
client.setQueryData(key, { canManage: true, settings: configured, choices });
|
||||
await new Promise(resolve => setTimeout(resolve, 10));
|
||||
});
|
||||
expect(container.textContent).not.toContain("Decision model is working");
|
||||
} finally { act(() => root.unmount()); client.clear(); test.mockRestore(); container.remove(); }
|
||||
});
|
||||
@@ -0,0 +1,127 @@
|
||||
import { useEffect, useRef, useState } from "react";
|
||||
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { DECISION_MODELS, type DecisionConnectionChoice, type DecisionModelSettings, type DecisionProvider, type DecisionResult, type UpdateDecisionModel } from "@paperclipai/shared";
|
||||
import { Link } from "@/lib/router";
|
||||
import { decisionModelsApi } from "@/api/decision-models";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { Dialog, DialogContent, DialogHeader, DialogTitle, DialogDescription } from "@/components/ui/dialog";
|
||||
import { ToggleField } from "@/components/agent-config-primitives";
|
||||
import { ConnectionChoiceList } from "@/features/connections/ConnectionChoiceList";
|
||||
import { ConnectionSetupFlow } from "@/features/connections/ConnectionSetupFlow";
|
||||
import { AppLogo } from "@/pages/apps/AppLogo";
|
||||
|
||||
const failureMessages: Record<string, string> = {
|
||||
not_configured: "Choose and save a connection first.", disabled: "Enable and save decisions before running a test.",
|
||||
connection_unavailable: "The connection is unavailable. Reconnect it and try again.",
|
||||
access_denied: "This connection is not shared with you. Review its permissions.",
|
||||
incompatible_connection: "Choose a supported OpenAI or OpenRouter API connection.",
|
||||
budget_blocked: "The company or task budget prevents this request. Review Costs and Budgets.",
|
||||
provider_auth_failed: "The provider rejected the connection. Reconnect it and try again.",
|
||||
provider_rate_limited: "The provider is rate limited. Try again later.",
|
||||
timeout: "The provider did not respond in time. Billing is unresolved; review Costs.",
|
||||
refused: "The model declined the test. Review the connection and try again.",
|
||||
};
|
||||
export function DecisionModelSettingsView({ settings, choices, saving, testing, error, result, newlyConnectedId, onSave, onTest, onAdd }: {
|
||||
settings: DecisionModelSettings; choices: DecisionConnectionChoice[]; saving?: boolean; testing?: boolean;
|
||||
error?: string | null; result?: DecisionResult | null; newlyConnectedId?: string | null;
|
||||
onSave: (value: UpdateDecisionModel) => void; onTest: () => void; onAdd: () => void;
|
||||
}) {
|
||||
const [draft, setDraft] = useState<Required<UpdateDecisionModel>>(settings);
|
||||
const appliedConnection = useRef<string | null>(null);
|
||||
useEffect(() => { setDraft(settings); }, [settings.companyId, settings.connectionId, settings.grantId, settings.enabled, settings.allowBackground]);
|
||||
useEffect(() => {
|
||||
if (!newlyConnectedId) { appliedConnection.current = null; return; }
|
||||
if (appliedConnection.current === newlyConnectedId) return;
|
||||
const connection = choices.find(row => row.id === newlyConnectedId);
|
||||
if (connection) {
|
||||
appliedConnection.current = newlyConnectedId;
|
||||
setDraft(value => ({ ...value, connectionId: connection.id, grantId: connection.grantId, enabled: value.connectionId ? value.enabled : true }));
|
||||
}
|
||||
}, [newlyConnectedId, choices]);
|
||||
const selected = choices.find(row => row.grantId === draft.grantId);
|
||||
const dirty = draft.enabled !== settings.enabled || draft.allowBackground !== settings.allowBackground || draft.grantId !== settings.grantId;
|
||||
const unavailable = result && result.status !== "succeeded" ? failureMessages[result.status === "unavailable" ? result.reason : result.errorCode] ?? "The decision could not be completed. Review its entry in Costs." : null;
|
||||
return <section aria-labelledby="decision-model-heading" className="max-w-2xl space-y-4">
|
||||
<div className="flex flex-wrap items-center justify-between gap-3">
|
||||
<h2 id="decision-model-heading" className="text-xs font-medium uppercase tracking-wide text-muted-foreground">Decision model</h2>
|
||||
<Link to="/activity/costs?tab=decisions" className="text-sm underline underline-offset-4">View usage</Link>
|
||||
</div>
|
||||
<p className="text-sm text-muted-foreground">Choose the shared connection Paperclip uses for optional decision features. API usage is charged to this connection.</p>
|
||||
<ConnectionChoiceList selectedId={draft.grantId ?? undefined} disabled={saving || testing}
|
||||
choices={choices.map(row => ({ id: row.grantId, name: row.name,
|
||||
description: `${row.decisionModel} · ${row.status === "connected" ? "Connected" : "Needs attention"}`,
|
||||
icon: <AppLogo name={row.provider === "openai" ? "OpenAI" : "OpenRouter"} brandKey={row.provider} size={24} />,
|
||||
}))}
|
||||
onSelect={grantId => { const row = choices.find(choice => choice.grantId === grantId)!; setDraft(value => ({ ...value, connectionId: row.id, grantId, enabled: value.connectionId ? value.enabled : true })); }} />
|
||||
{choices.length === 0 && <p className="text-sm text-muted-foreground">Add a shared OpenAI or OpenRouter API connection to get started.</p>}
|
||||
{draft.connectionId && !selected && <p role="status" className="text-sm text-muted-foreground">The saved connection is unavailable to you. Disable decisions or choose another connection.</p>}
|
||||
<div className="flex flex-wrap items-center gap-3">
|
||||
<Button variant="outline" size="sm" onClick={onAdd} disabled={saving || testing}>Add connection</Button>
|
||||
{selected && <Link to={`/apps/${selected.id}/permissions`} className="text-sm underline underline-offset-4">{selected.status === "connected" ? "Manage connection" : "Reconnect connection"}</Link>}
|
||||
</div>
|
||||
{draft.connectionId && <fieldset disabled={saving || testing} className="space-y-4">
|
||||
<ToggleField label="Enable decisions" checked={draft.enabled} onChange={enabled => setDraft(value => ({ ...value, enabled }))} />
|
||||
<div className="space-y-2">
|
||||
<ToggleField label="Allow company-sponsored background decisions" checked={draft.allowBackground} onChange={allowBackground => setDraft(value => ({ ...value, allowBackground }))} />
|
||||
<p className="text-xs text-muted-foreground">Background features can charge this connection without a responsible person. These requests appear as Paperclip services.</p>
|
||||
</div>
|
||||
</fieldset>}
|
||||
{error && <p role="alert" className="text-sm text-destructive">{error}</p>}
|
||||
{!dirty && unavailable && <p role="alert" className="text-sm text-destructive">{unavailable}</p>}
|
||||
{!dirty && result?.status === "succeeded" && <div role="status" className="space-y-2 text-sm">
|
||||
<p className="font-medium">Decision model is working</p>
|
||||
<dl className="grid grid-cols-2 gap-2">{Object.entries(result.answers).map(([name, answer]) => <div key={name}>
|
||||
<dt className="text-muted-foreground">{name === "billing" ? "Billing problem" : name === "team" ? "Assigned team" : "Disruption score"}</dt>
|
||||
<dd className="font-mono">{answer.type === "boolean" ? `${Math.round(answer.probability * 100)}%` : answer.type === "choice" ? answer.choice : `${answer.score.toFixed(2)} / 2`}</dd>
|
||||
</div>)}</dl>
|
||||
<p className="text-xs text-muted-foreground">Usage was recorded. Test answers are not saved.</p>
|
||||
</div>}
|
||||
<div className="flex flex-wrap items-center justify-between gap-3">
|
||||
<div className="space-y-1">
|
||||
<Button variant="outline" size="sm" disabled={!settings.enabled || dirty || saving || testing || selected?.status !== "connected"} onClick={onTest}>{testing ? "Running test…" : "Run test"}</Button>
|
||||
<p className="text-xs text-muted-foreground">{dirty ? "Save changes before testing." : "Runs a small sample with an API charge."}</p>
|
||||
</div>
|
||||
{dirty && <Button onClick={() => onSave(draft)} disabled={saving || testing || (draft.enabled && selected?.status !== "connected")}>{saving ? "Saving…" : "Save decision model"}</Button>}
|
||||
</div>
|
||||
</section>;
|
||||
}
|
||||
|
||||
export function DecisionModelSettingsSection({ companyId }: { companyId: string }) {
|
||||
const client = useQueryClient();
|
||||
const key = ["decision-model", companyId];
|
||||
const query = useQuery({ queryKey: key, queryFn: () => decisionModelsApi.settings(companyId) });
|
||||
const [adding, setAdding] = useState(false);
|
||||
const [provider, setProvider] = useState<DecisionProvider | null>(null);
|
||||
const [newlyConnectedId, setNewlyConnectedId] = useState<string | null>(null);
|
||||
const settingsRevision = JSON.stringify(query.data?.settings ?? { companyId });
|
||||
const test = useMutation({ mutationFn: (_revision: string) => decisionModelsApi.test(companyId), onSettled: () => { void client.invalidateQueries({ queryKey: ["decision-history", companyId] }); void client.invalidateQueries({ predicate: q => String(q.queryKey[0]).includes("cost") }); } });
|
||||
const resetTest = test.reset;
|
||||
useEffect(() => { resetTest(); }, [settingsRevision, resetTest]);
|
||||
const save = useMutation({ mutationFn: (settings: UpdateDecisionModel) => decisionModelsApi.update(companyId, settings), onSuccess: settings => {
|
||||
setNewlyConnectedId(null); test.reset(); client.setQueryData(key, { ...query.data, settings });
|
||||
} });
|
||||
if (query.isPending) return <p className="text-sm text-muted-foreground">Loading decision model…</p>;
|
||||
if (query.error) return <div role="alert" className="space-y-2"><p className="text-sm text-destructive">Could not load decision settings.</p><Button variant="outline" onClick={() => void query.refetch()}>Try again</Button></div>;
|
||||
if (!query.data?.canManage || !query.data.settings) return null;
|
||||
return <>
|
||||
<DecisionModelSettingsView settings={query.data.settings} choices={query.data.choices} saving={save.isPending} testing={test.isPending}
|
||||
error={save.error?.message ?? (test.variables === settingsRevision ? test.error?.message : undefined)}
|
||||
result={test.variables === settingsRevision ? test.data : null} newlyConnectedId={newlyConnectedId}
|
||||
onSave={value => save.mutate(value)} onTest={() => test.mutate(settingsRevision)} onAdd={() => { setProvider(null); setAdding(true); }} />
|
||||
<Dialog open={adding} onOpenChange={setAdding}>
|
||||
<DialogContent className="sm:max-w-2xl max-h-(--sz-85vh) overflow-y-auto">
|
||||
<DialogHeader><DialogTitle>Add a decision connection</DialogTitle><DialogDescription>Use a shared API connection for company decisions.</DialogDescription></DialogHeader>
|
||||
{provider ? <ConnectionSetupFlow host="dialog" serviceSlug={provider} forceNewConnection
|
||||
aiConnection={{ provider, method: "api_key", mode: "shared" }} requiredAiOwnership="shared"
|
||||
onCancel={() => setAdding(false)} onComplete={async result => {
|
||||
const refreshed = await query.refetch();
|
||||
if ("connectionId" in result && result.connectionId && refreshed.data?.choices.some(row => row.id === result.connectionId)) setNewlyConnectedId(result.connectionId);
|
||||
setAdding(false);
|
||||
}} /> : <ConnectionChoiceList choices={(Object.keys(DECISION_MODELS) as DecisionProvider[]).map(id => ({ id,
|
||||
name: id === "openai" ? "OpenAI" : "OpenRouter", description: DECISION_MODELS[id].name,
|
||||
icon: <AppLogo name={id === "openai" ? "OpenAI" : "OpenRouter"} brandKey={id} size={24} />,
|
||||
}))} onSelect={id => setProvider(id as DecisionProvider)} />}
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
</>;
|
||||
}
|
||||
@@ -513,7 +513,9 @@ export interface ConnectionSetupFlowProps {
|
||||
connectionSettings?: ReactNode;
|
||||
additionalSettingsValid?: boolean;
|
||||
upstreamServiceName?: string;
|
||||
aiConnection?: import("@paperclipai/shared").AiConnectionBinding;
|
||||
aiConnection?: import("@paperclipai/shared").AiConnectionBinding | Pick<import("@paperclipai/shared").AiConnectionBinding, "provider" | "method" | "mode">;
|
||||
/** A company service requires a company-owned credential. Existing grants are never broadened. */
|
||||
requiredAiOwnership?: "shared";
|
||||
/** Provider-specific authentication inside the existing access/setup shell. Undefined retains the standard credential form. */
|
||||
renderCredentialStep?: (context: { app: AppDefinition; name: string; grantKind: ConnectionGrantKind; agentIds: string[]; allAgents: boolean; onBack: () => void }) => ReactNode;
|
||||
byoOnly?: boolean;
|
||||
@@ -587,6 +589,7 @@ function StandardConnectionSetupFlow({
|
||||
onComplete,
|
||||
onOAuthDeclined,
|
||||
aiConnection,
|
||||
requiredAiOwnership,
|
||||
onPhaseChange,
|
||||
onCancel,
|
||||
renderCredentialStep,
|
||||
@@ -975,7 +978,7 @@ function StandardConnectionSetupFlow({
|
||||
const galleryQuery = useQuery({
|
||||
queryKey: queryKeys.apps.gallery(selectedCompanyId ?? "__none__"),
|
||||
queryFn: () => toolsApi.listGallery(selectedCompanyId!),
|
||||
select: useCallback((data: Awaited<ReturnType<typeof toolsApi.listGallery>>) => connectionIntentId ? {
|
||||
select: useCallback((data: Awaited<ReturnType<typeof toolsApi.listGallery>>) => connectionIntentId || aiConnection ? {
|
||||
...data,
|
||||
apps: data.apps.map(app => ({ ...app, methods: app.methods.filter(method => aiConnection ? method.ai?.provider === aiConnection.provider && (aiConnection.mode === "responsible_user" || method.ai.method === aiConnection.method) : method.transport !== "runtime_auth") })).filter(app => app.methods.length > 0),
|
||||
} : data, [connectionIntentId, aiConnection?.provider, aiConnection?.method, aiConnection?.mode]),
|
||||
@@ -1183,7 +1186,7 @@ function StandardConnectionSetupFlow({
|
||||
? "agent"
|
||||
: "organization"
|
||||
: null;
|
||||
const fixedGrantKind = reconnectGrantKind ?? existingOAuthGrantKind;
|
||||
const fixedGrantKind = reconnectGrantKind ?? existingOAuthGrantKind ?? (requiredAiOwnership === "shared" ? "organization" : null);
|
||||
|
||||
useEffect(() => {
|
||||
if (fixedGrantKind) setGrantKind(fixedGrantKind);
|
||||
|
||||
@@ -281,3 +281,8 @@ export function projectWorkspaceUrl(
|
||||
): string {
|
||||
return `${projectUrl(project)}/workspaces/${workspaceId}`;
|
||||
}
|
||||
|
||||
/** Preserve sub-cent inference charges instead of showing a misleading $0.00. */
|
||||
export function formatDetailedCents(cents: string | number): string {
|
||||
return (Number(cents) / 100).toLocaleString("en-US", { style: "currency", currency: "USD", minimumFractionDigits: 2, maximumFractionDigits: 9 });
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
import { DecisionModelSettingsSection } from "../components/decision-models/DecisionModelSettings";
|
||||
import { ChangeEvent, useEffect, useState } from "react";
|
||||
import { useNavigate } from "react-router-dom";
|
||||
import { useMutation, useQueryClient } from "@tanstack/react-query";
|
||||
@@ -355,6 +356,8 @@ export function CompanySettings() {
|
||||
</div>
|
||||
|
||||
{/* Interaction governance */}
|
||||
{selectedCompanyId && <DecisionModelSettingsSection key={selectedCompanyId} companyId={selectedCompanyId} />}
|
||||
|
||||
<InteractionGovernancePanel
|
||||
governance={governance}
|
||||
onChange={handleGovernanceChange}
|
||||
|
||||
@@ -12,6 +12,7 @@ const resolveIncidentMock = vi.hoisted(() => vi.fn());
|
||||
const upsertPolicyMock = vi.hoisted(() => vi.fn());
|
||||
const budgetOverviewMock = vi.hoisted(() => vi.fn());
|
||||
const setBreadcrumbsMock = vi.hoisted(() => vi.fn());
|
||||
const decisionHistoryMock = vi.hoisted(() => vi.fn());
|
||||
const costsApiMocks = vi.hoisted(() => ({
|
||||
summary: vi.fn(),
|
||||
byAgent: vi.fn(),
|
||||
@@ -37,6 +38,7 @@ vi.mock("../api/budgets", () => ({
|
||||
}));
|
||||
|
||||
vi.mock("../api/costs", () => ({ costsApi: costsApiMocks }));
|
||||
vi.mock("../api/decision-models", () => ({ decisionModelsApi: { history: decisionHistoryMock } }));
|
||||
|
||||
vi.mock("../context/CompanyContext", () => ({
|
||||
useCompany: () => ({ selectedCompanyId: "company-1" }),
|
||||
@@ -61,6 +63,7 @@ describe("Shared Costs surfaces", () => {
|
||||
let root: ReturnType<typeof createRoot>;
|
||||
|
||||
beforeEach(() => {
|
||||
decisionHistoryMock.mockResolvedValue([]);
|
||||
costsApiMocks.byUser.mockResolvedValue({ activeUserCount: 1, rows: [] });
|
||||
container = document.createElement("div");
|
||||
document.body.appendChild(container);
|
||||
@@ -95,7 +98,7 @@ describe("Shared Costs surfaces", () => {
|
||||
await act(async () => {
|
||||
root.render(
|
||||
<QueryClientProvider client={queryClient}>
|
||||
<Costs embedded initialTab="budgets" lockTab />
|
||||
<MemoryRouter><Costs embedded initialTab="budgets" lockTab /></MemoryRouter>
|
||||
</QueryClientProvider>,
|
||||
);
|
||||
await Promise.resolve();
|
||||
@@ -113,6 +116,23 @@ describe("Shared Costs surfaces", () => {
|
||||
for (const mock of Object.values(costsApiMocks)) expect(mock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("opens linked decision history with real date bounds instead of report cache labels", async () => {
|
||||
for (const mock of Object.values(costsApiMocks)) mock.mockResolvedValue([]);
|
||||
costsApiMocks.byUser.mockResolvedValue({ activeUserCount: 1, rows: [] });
|
||||
costsApiMocks.summary.mockResolvedValue({ spendCents: 0, budgetCents: 0, pricingComplete: true });
|
||||
costsApiMocks.financeSummary.mockResolvedValue({ netCents: 0, debitCents: 0, creditCents: 0, estimatedDebitCents: 0, eventCount: 0 });
|
||||
const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } });
|
||||
root = createRoot(container);
|
||||
await act(async () => root.render(<MemoryRouter initialEntries={["/?tab=decisions"]}><QueryClientProvider client={queryClient}><Costs /></QueryClientProvider></MemoryRouter>));
|
||||
await act(async () => { await vi.waitFor(() => expect(container.textContent).toContain("No decision requests in this period")); });
|
||||
expect(container.querySelector('[role="tab"][data-state="active"]')?.textContent).toBe("Decisions");
|
||||
const [companyId, from, to] = decisionHistoryMock.mock.lastCall!;
|
||||
expect(companyId).toBe("company-1");
|
||||
expect(Number.isFinite(Date.parse(from))).toBe(true);
|
||||
expect(Number.isFinite(Date.parse(to))).toBe(true);
|
||||
queryClient.clear();
|
||||
});
|
||||
|
||||
it.each([
|
||||
["reservation", 500, "agent"], ["unknown price", 500, "agent"], ["amount", 500, "agent"],
|
||||
["amount", 0, "company"], ["amount", 0, "agent"], ["amount", 0, "project"],
|
||||
@@ -126,7 +146,7 @@ describe("Shared Costs surfaces", () => {
|
||||
upsertPolicyMock.mockResolvedValue({});
|
||||
const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } });
|
||||
root = createRoot(container);
|
||||
await act(async () => root.render(<QueryClientProvider client={queryClient}><Costs embedded initialTab="budgets" lockTab /></QueryClientProvider>));
|
||||
await act(async () => root.render(<QueryClientProvider client={queryClient}><MemoryRouter><Costs embedded initialTab="budgets" lockTab /></MemoryRouter></QueryClientProvider>));
|
||||
await act(async () => { await vi.waitFor(() => expect(container.querySelector('[aria-label="Reserve per run (USD)"]')).not.toBeNull()); });
|
||||
const advanced = container.querySelector("details")!;
|
||||
expect(advanced.open).toBe(false);
|
||||
@@ -159,7 +179,7 @@ describe("Shared Costs surfaces", () => {
|
||||
upsertPolicyMock.mockRejectedValueOnce(new Error("private SQL policy error"));
|
||||
const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false }, mutations: { retry: false } } });
|
||||
root = createRoot(container);
|
||||
await act(async () => root.render(<QueryClientProvider client={queryClient}><Costs embedded initialTab="budgets" lockTab /></QueryClientProvider>));
|
||||
await act(async () => root.render(<QueryClientProvider client={queryClient}><MemoryRouter><Costs embedded initialTab="budgets" lockTab /></MemoryRouter></QueryClientProvider>));
|
||||
await act(async () => { await vi.waitFor(() => expect(container.querySelector('input[type="checkbox"]')).not.toBeNull()); });
|
||||
const reservation = container.querySelector<HTMLInputElement>('[aria-label="Reserve per run (USD)"]')!;
|
||||
const originalReservation = reservation.value;
|
||||
@@ -335,7 +355,7 @@ describe("Shared Costs surfaces", () => {
|
||||
expect(setBreadcrumbsMock).toHaveBeenCalledWith([{ label: "Costs" }]);
|
||||
}
|
||||
expect([...container.querySelectorAll('[role="tab"]')].map((tab) => tab.textContent)).toEqual(
|
||||
streamlined ? ["Overview", "Providers", "Billers", "Finance"] : ["Overview", "Budgets", "Providers", "Billers", "Finance"],
|
||||
streamlined ? ["Overview", "Providers", "Billers", "Finance", "Decisions"] : ["Overview", "Budgets", "Providers", "Billers", "Finance", "Decisions"],
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
+16
-10
@@ -1,3 +1,5 @@
|
||||
import { DecisionHistory } from "../components/decision-models/DecisionHistory";
|
||||
import { useSearchParams } from "@/lib/router";
|
||||
import { CostEstimateLabel } from "../components/CostEstimateLabel";
|
||||
import { CostByUserTable } from "../components/CostByUserTable";
|
||||
import { AgentIdentity } from "@/components/AgentIdentity";
|
||||
@@ -39,7 +41,7 @@ import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/com
|
||||
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
|
||||
|
||||
const NO_COMPANY = "__none__";
|
||||
export type CostsMainTab = "overview" | "budgets" | "providers" | "billers" | "finance";
|
||||
export type CostsMainTab = "overview" | "budgets" | "providers" | "billers" | "finance" | "decisions";
|
||||
|
||||
export interface CostsProps {
|
||||
/** Render inside Audit without a second page-level title or breadcrumb. */
|
||||
@@ -172,6 +174,7 @@ export function Costs({
|
||||
const { setBreadcrumbs } = useBreadcrumbs();
|
||||
const queryClient = useQueryClient();
|
||||
|
||||
const [searchParams, setSearchParams] = useSearchParams();
|
||||
const [mainTab, setMainTab] = useState<CostsMainTab>(initialTab);
|
||||
const [activeProvider, setActiveProvider] = useState("all");
|
||||
const [activeBiller, setActiveBiller] = useState("all");
|
||||
@@ -194,8 +197,9 @@ export function Costs({
|
||||
}, [embedded, setBreadcrumbs]);
|
||||
|
||||
useEffect(() => {
|
||||
setMainTab(initialTab);
|
||||
}, [initialTab]);
|
||||
const tab = searchParams.get("tab");
|
||||
setMainTab(!lockTab && tab && ["overview", "providers", "billers", "finance", "budgets", "decisions"].includes(tab) ? tab as CostsMainTab : initialTab);
|
||||
}, [initialTab, lockTab, searchParams]);
|
||||
|
||||
const [today, setToday] = useState(() => new Date().toDateString());
|
||||
const todayTimerRef = useRef<ReturnType<typeof setTimeout> | null>(null);
|
||||
@@ -302,12 +306,12 @@ export function Costs({
|
||||
refetchInterval: 30_000,
|
||||
});
|
||||
|
||||
const [expandedAgents, setExpandedAgents] = useState<Set<string>>(new Set());
|
||||
const [expandedAgents, setExpandedAgents] = useState<Set<string | null>>(new Set());
|
||||
useEffect(() => {
|
||||
setExpandedAgents(new Set());
|
||||
}, [companyId, preset, customFrom, customTo]);
|
||||
|
||||
function toggleAgent(agentId: string) {
|
||||
function toggleAgent(agentId: string | null) {
|
||||
setExpandedAgents((prev) => {
|
||||
const next = new Set(prev);
|
||||
if (next.has(agentId)) next.delete(agentId);
|
||||
@@ -317,7 +321,7 @@ export function Costs({
|
||||
}
|
||||
|
||||
const agentModelRows = useMemo(() => {
|
||||
const map = new Map<string, CostByAgentModel[]>();
|
||||
const map = new Map<string | null, CostByAgentModel[]>();
|
||||
for (const row of spendData?.byAgentModel ?? []) {
|
||||
const rows = map.get(row.agentId) ?? [];
|
||||
rows.push(row);
|
||||
@@ -696,9 +700,9 @@ export function Costs({
|
||||
</p>
|
||||
) : null}
|
||||
|
||||
{!!spendData?.summary.estimatedEventCount && <p role="status" className="text-sm text-muted-foreground">Includes {spendData.summary.estimatedEventCount} estimated run charges. Token estimates use published rates and recorded assumptions; provider bills may differ.</p>}
|
||||
{!!spendData?.summary.estimatedEventCount && <p role="status" className="text-sm text-muted-foreground">Includes {spendData.summary.estimatedEventCount} estimated {spendData.summary.estimatedEventCount === 1 ? "charge" : "charges"}. Token estimates use published rates and recorded assumptions; provider bills may differ.</p>}
|
||||
{incidentMutation.error && <p role="alert" className="text-sm text-destructive">Could not update the budget. Check any pending runs or unpriced usage shown on this page, then try again.</p>}
|
||||
<Tabs value={mainTab} onValueChange={(value) => setMainTab(value as typeof mainTab)}>
|
||||
<Tabs value={mainTab} onValueChange={(value) => { setMainTab(value as typeof mainTab); setSearchParams(current => { const next = new URLSearchParams(current); next.set("tab", value); return next; }, { replace: true }); }}>
|
||||
{!lockTab ? (
|
||||
<TabsList variant="line" className="justify-start">
|
||||
<TabsTrigger value="overview">Overview</TabsTrigger>
|
||||
@@ -706,9 +710,11 @@ export function Costs({
|
||||
<TabsTrigger value="providers">Providers</TabsTrigger>
|
||||
<TabsTrigger value="billers">Billers</TabsTrigger>
|
||||
<TabsTrigger value="finance">Finance</TabsTrigger>
|
||||
<TabsTrigger value="decisions">Decisions</TabsTrigger>
|
||||
</TabsList>
|
||||
) : null}
|
||||
|
||||
<TabsContent value="decisions" className="mt-4">{showCustomPrompt ? <p className="text-sm text-muted-foreground">Select a start and end date to load data.</p> : <DecisionHistory companyId={companyId} from={from} to={to} />}</TabsContent>
|
||||
<TabsContent value="overview" className="mt-4 space-y-4">
|
||||
{showCustomPrompt ? (
|
||||
<p className="text-sm text-muted-foreground">Select a start and end date to load data.</p>
|
||||
@@ -733,7 +739,7 @@ export function Costs({
|
||||
const isExpanded = expandedAgents.has(row.agentId);
|
||||
const hasBreakdown = modelRows.length > 0;
|
||||
return (
|
||||
<div key={row.agentId} role="group" aria-label={`${row.agentName ?? row.agentId} costs`} className="border border-border px-4 py-3">
|
||||
<div key={row.agentId ?? "services"} role="group" aria-label={`${row.agentName ?? row.agentId ?? "Paperclip services"} costs`} className="border border-border px-4 py-3">
|
||||
<div
|
||||
className={cn("flex flex-col gap-3 sm:flex-row sm:items-start sm:justify-between", hasBreakdown ? "cursor-pointer select-none" : "")}
|
||||
onClick={() => hasBreakdown && toggleAgent(row.agentId)}
|
||||
@@ -746,7 +752,7 @@ export function Costs({
|
||||
) : (
|
||||
<span className="h-3 w-3 shrink-0" />
|
||||
)}
|
||||
<AgentIdentity agent={{ id: row.agentId, name: row.agentName ?? row.agentId, appearance: row.agentAppearance }} size="sm" />
|
||||
{row.agentId ? <AgentIdentity agent={{ id: row.agentId, name: row.agentName ?? row.agentId, appearance: row.agentAppearance }} size="sm" /> : <span className="font-medium">Paperclip services</span>}
|
||||
{row.agentStatus === "terminated" ? <StatusBadge status="terminated" /> : null}
|
||||
</div>
|
||||
<div className="text-right text-sm tabular-nums">
|
||||
|
||||
@@ -238,7 +238,7 @@ export function UserProfile() {
|
||||
() =>
|
||||
(data?.topAgents ?? []).map((row) => ({
|
||||
key: row.agentId ?? "unknown",
|
||||
label: row.agentName ?? (row.agentId ? row.agentId.slice(0, 8) : "unknown"),
|
||||
label: row.agentName ?? (row.agentId ? row.agentId.slice(0, 8) : "Paperclip services"),
|
||||
sublabel: "Task-linked usage",
|
||||
costCents: row.costCents,
|
||||
inputTokens: row.inputTokens,
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
import type { Meta, StoryObj } from "@storybook/react-vite";
|
||||
import { fn } from "storybook/test";
|
||||
import { DecisionModelSettingsView } from "@/components/decision-models/DecisionModelSettings";
|
||||
import { choices, configured, result, unconfigured } from "./fixtures";
|
||||
|
||||
const meta = { title: "Decision models/01 Settings", component: DecisionModelSettingsView,
|
||||
decorators: [Story => <main className="p-6"><Story /></main>],
|
||||
args: { settings: configured, choices, onSave: fn(), onTest: fn(), onAdd: fn() },
|
||||
} satisfies Meta<typeof DecisionModelSettingsView>;
|
||||
export default meta;
|
||||
type Story = StoryObj<typeof meta>;
|
||||
export const Empty: Story = { args: { settings: unconfigured, choices: [] } };
|
||||
export const SelectExisting: Story = { args: { settings: unconfigured } };
|
||||
export const Configured: Story = {};
|
||||
export const SponsorshipOff: Story = { args: { settings: { ...configured, allowBackground: false } } };
|
||||
export const Disabled: Story = { args: { settings: { ...configured, enabled: false } } };
|
||||
export const Reconnect: Story = { args: { choices: [{ ...choices[0]!, status: "needs_attention" }] } };
|
||||
export const Revoked: Story = { args: { choices: [] } };
|
||||
export const Testing: Story = { args: { testing: true } };
|
||||
export const Results: Story = { args: { result } };
|
||||
export const Failure: Story = { args: { result: { status: "failed", invocationId: "decision-failure", errorCode: "provider_auth_failed", usage: { inputTokens: 0, outputTokens: 0, costCents: "0", costStatus: "estimated" } } } };
|
||||
export const Mobile: Story = { args: { result }, globals: { viewport: { value: "mobile1", isRotated: false } } };
|
||||
@@ -0,0 +1,16 @@
|
||||
import type { Meta, StoryObj } from "@storybook/react-vite";
|
||||
import { DecisionHistoryTable } from "@/components/decision-models/DecisionHistory";
|
||||
import { entry } from "./fixtures";
|
||||
|
||||
const meta = { title: "Decision models/02 History", component: DecisionHistoryTable,
|
||||
decorators: [Story => <main className="p-6"><Story /></main>], args: { entries: [entry] },
|
||||
} satisfies Meta<typeof DecisionHistoryTable>;
|
||||
export default meta;
|
||||
type Story = StoryObj<typeof meta>;
|
||||
export const Empty: Story = { args: { entries: [] } };
|
||||
export const SetupTest: Story = {};
|
||||
export const MixedUsage: Story = { args: { entries: [entry,
|
||||
{ ...entry, id: "background", actorType: "system", responsibleUserId: null, userName: null, feature: "tasks.classification", model: "typesafe/jev-1.13", provider: "openrouter", costStatus: "reported", costCents: "0.0005166", issueId: "issue-1", issueIdentifier: "PAP-21" },
|
||||
{ ...entry, id: "timeout", status: "unknown", errorCode: "timeout", costStatus: "unpriced", costCents: null, inputTokens: null, outputTokens: null, durationMs: 30000 },
|
||||
] } };
|
||||
export const Mobile: Story = { ...MixedUsage, globals: { viewport: { value: "mobile1", isRotated: false } } };
|
||||
@@ -0,0 +1,80 @@
|
||||
import { useLayoutEffect, useState } from "react";
|
||||
import type { Meta, StoryObj } from "@storybook/react-vite";
|
||||
import { expect, userEvent, waitFor, within } from "storybook/test";
|
||||
import { useQueryClient } from "@tanstack/react-query";
|
||||
import { Route, Routes } from "@/lib/router";
|
||||
import { Layout } from "@/components/Layout";
|
||||
import { CompanySettings } from "@/pages/CompanySettings";
|
||||
import { Costs } from "@/pages/Costs";
|
||||
import { PluginLauncherProvider } from "@/plugins/launchers";
|
||||
import { choices, companyId, configured, entry, result, unconfigured } from "./fixtures";
|
||||
import type { DecisionHistoryEntry } from "@paperclipai/shared";
|
||||
|
||||
/** Production pages and shell; only API responses and paid provider execution are simulated. */
|
||||
function Journey({ alreadyConfigured = false }: { alreadyConfigured?: boolean }) {
|
||||
const [ready, setReady] = useState(false);
|
||||
const queryClient = useQueryClient();
|
||||
useLayoutEffect(() => {
|
||||
const original = window.fetch;
|
||||
let settings = { ...(alreadyConfigured ? configured : unconfigured) };
|
||||
const history: DecisionHistoryEntry[] = [];
|
||||
window.fetch = async (input, init) => {
|
||||
const url = new URL(typeof input === "string" ? input : input instanceof URL ? input.href : input.url, window.location.origin);
|
||||
const base = `/api/companies/${companyId}/decision-model`;
|
||||
if (url.pathname === base) {
|
||||
if (init?.method === "PUT") {
|
||||
const data = JSON.parse(String(init.body));
|
||||
const selected = choices.find(row => row.grantId === data.grantId)!;
|
||||
settings = { ...settings, ...data, provider: selected.provider, model: selected.decisionModel };
|
||||
return Response.json(settings);
|
||||
}
|
||||
return Response.json({ settings, choices, canManage: true });
|
||||
}
|
||||
if (url.pathname === `${base}/history`) return Response.json(history);
|
||||
if (url.pathname === `${base}/test`) {
|
||||
history.unshift({ ...entry, id: `decision-${history.length}`, model: settings.model!, provider: settings.provider! });
|
||||
return Response.json(result);
|
||||
}
|
||||
if (url.pathname === `/api/companies/${companyId}/budgets/overview`) return Response.json({ companyId,
|
||||
policies: [], activeIncidents: [], pausedAgentCount: 0, pausedProjectCount: 0, pendingApprovalCount: 0 });
|
||||
const costBase = `/api/companies/${companyId}/costs/`;
|
||||
if (url.pathname.startsWith(costBase)) {
|
||||
const report = url.pathname.slice(costBase.length);
|
||||
if (report === "summary") return Response.json({ companyId, eventCount: history.length, pendingRunCount: 0,
|
||||
unpricedEventCount: 0, estimatedEventCount: history.length, pricingComplete: true,
|
||||
spendCents: history.length * Number(entry.costCents), budgetCents: 0, utilizationPercent: 0 });
|
||||
if (report === "finance-summary") return Response.json({ companyId, currency: "USD", currencies: [], debitCents: 0,
|
||||
creditCents: 0, netCents: 0, estimatedDebitCents: 0, eventCount: 0 });
|
||||
if (report === "by-user") return Response.json({ activeUserCount: 1, rows: [] });
|
||||
if (report === "by-agent") return Response.json(history.length ? [{ agentId: null, agentName: null, agentStatus: null,
|
||||
eventCount: history.length, estimatedEventCount: history.length, costCents: history.length * Number(entry.costCents),
|
||||
inputTokens: history.length * entry.inputTokens!, cachedInputTokens: 0, outputTokens: 0 }] : []);
|
||||
return Response.json([]);
|
||||
}
|
||||
return original(input, init);
|
||||
};
|
||||
setReady(true);
|
||||
return () => { window.fetch = original; queryClient.removeQueries({ queryKey: ["decision-model"] }); queryClient.removeQueries({ queryKey: ["decision-history"] }); };
|
||||
}, [alreadyConfigured, queryClient]);
|
||||
return ready ? <PluginLauncherProvider><Routes><Route path="/:companyPrefix" element={<Layout />}>
|
||||
<Route path="company/settings" element={<CompanySettings />} />
|
||||
<Route path="activity/costs" element={<Costs />} />
|
||||
</Route></Routes></PluginLauncherProvider> : null;
|
||||
}
|
||||
const meta = { title: "Decision models/03 Journey", component: Journey, parameters: {
|
||||
layout: "fullscreen", initialEntries: ["/PAP/company/settings"], docs: { story: { inline: false }, description: { component: "Select an existing shared connection, preserve the background setting, save, run the sample, and follow View usage. Paid calls are fixtures." } },
|
||||
} } satisfies Meta<typeof Journey>;
|
||||
export default meta;
|
||||
type Story = StoryObj<typeof meta>;
|
||||
export const SetupToHistory: Story = { play: async ({ canvasElement }) => {
|
||||
const c = within(canvasElement);
|
||||
await userEvent.click(await c.findByRole("button", { name: /Company OpenAI/ }));
|
||||
await userEvent.click(c.getByRole("button", { name: "Save decision model" }));
|
||||
await waitFor(() => expect(c.getByRole("button", { name: "Run test" })).toBeEnabled());
|
||||
await userEvent.click(await c.findByRole("button", { name: "Run test" }));
|
||||
await expect(await c.findByText("Decision model is working")).toBeVisible();
|
||||
await userEvent.click(c.getByRole("link", { name: "View usage" }));
|
||||
await expect(await c.findByRole("table", { name: "Decision requests" })).toHaveTextContent("Setup test");
|
||||
} };
|
||||
export const ReturningManager: Story = { args: { alreadyConfigured: true } };
|
||||
export const Mobile: Story = { args: { alreadyConfigured: true }, globals: { viewport: { value: "mobile1", isRotated: false } } };
|
||||
@@ -0,0 +1,21 @@
|
||||
import type { DecisionConnectionChoice, DecisionHistoryEntry, DecisionModelSettings, DecisionResult } from "@paperclipai/shared";
|
||||
|
||||
export const companyId = "company-storybook";
|
||||
export const choices: DecisionConnectionChoice[] = ["openai", "openrouter"].map((provider, i) => ({
|
||||
id: `10000000-0000-4000-8000-00000000000${i + 1}`,
|
||||
grantId: `20000000-0000-4000-8000-00000000000${i + 1}`,
|
||||
companyId, provider: provider as "openai" | "openrouter", method: "api_key", ownership: "shared",
|
||||
name: i === 0 ? "Company OpenAI" : "Company OpenRouter", isDefault: false, status: "connected",
|
||||
decisionModel: i === 0 ? "gpt-6-luna" : "typesafe/jev-1.13",
|
||||
}));
|
||||
export const unconfigured: DecisionModelSettings = { companyId, enabled: false, allowBackground: true, connectionId: null, grantId: null, provider: null, model: null };
|
||||
export const configured: DecisionModelSettings = { ...unconfigured, enabled: true, connectionId: choices[0]!.id, grantId: choices[0]!.grantId, provider: "openai", model: "gpt-6-luna" };
|
||||
export const result: DecisionResult = { status: "succeeded", invocationId: "decision-fixture", answers: {
|
||||
billing: { type: "boolean", probability: 0.99 }, team: { type: "choice", choice: "billing", probabilities: { billing: 0.98, technical: 0.02 } }, severity: { type: "score", score: 0.08 },
|
||||
}, usage: { inputTokens: 140, outputTokens: 0, costCents: "0.0014000", costStatus: "estimated" } };
|
||||
export const entry: DecisionHistoryEntry = {
|
||||
id: "decision-fixture", feature: "settings.test", actorType: "user", responsibleUserId: "user-board", userName: "Board Operator",
|
||||
issueId: null, issueIdentifier: null, agentId: null, runId: null, connectionId: choices[0]!.id, provider: "openai", model: "gpt-6-luna",
|
||||
status: "succeeded", errorCode: null, startedAt: new Date().toISOString(), finishedAt: new Date().toISOString(), durationMs: 423,
|
||||
inputTokens: 140, outputTokens: 0, costCents: "0.0014000", costStatus: "estimated",
|
||||
};
|
||||
Reference in new issue
Block a user