mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-30 13:20:54 +01:00
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connections provide governed tools, chat channels, model accounts,
and gateways.
> - Several management routes require a human board actor even when an
agent has the required grant.
> - Default agents must configure these resources within the responsible
user's authority.
> - This pull request adds a shared authorization path to the existing
REST and native API surface.
> - Human authentication, execution approvals, and local host boundaries
remain enforced.
> - The benefit is that a standard agent can complete authorized setup
with a bounded human handoff.
## Linked Issues or Issue Description
**Subsystem affected**
Server authorization, connection services, shared contracts, native API
discovery, database schema.
**Problem or motivation**
Connection administration is available in the web app but often
unavailable to agents. Agent grants, responsible-user authority,
credential ownership, and actor attribution need separate checks.
Opening gateway-token creation also requires preserving the issuing
agent's delegated authority during token use.
**Proposed solution**
Use the existing authorization evaluator for every connection family.
Enforce current responsible-user authority, effective trust, grant
scope, resource visibility, and target-agent access. Keep existing agent
grants unchanged. Expose authorized configuration through default
`search_api` and `call_api`. Save human authentication steps and let
agents continue after completion.
**Roadmap alignment**
This change completes the API access path for the existing Connected
Apps, Personal & Shared AI Accounts, and governed MCP Gateway
capabilities in `ROADMAP.md`. It follows the maintainer's requested
design. Related work: #11899 enforces connection execution access and
creator/manager rules; this PR extends agent management across
connection families. It does not replace that execution-policy work.
## What Changed
- Add shared connection management context and mandatory
responsible-user, trust, scope, and target checks.
- Allow authorized agents to configure tool, chat/email, GitHub/Slack,
AI, profile, and gateway resources through existing routes.
- Preserve human credential ownership and record the actual agent,
responsible user, and run or key in audit data.
- Preserve saved browser authentication handoffs and callback session
checks.
- Bind agent-issued durable gateway tokens to their responsible user and
recheck delegated authority at use.
- Add explicit REST transport and management capability metadata,
built-in guidance, and sensitive-response handling.
- Intersect caller and endpoint-agent Slack test/invitation policies,
including independent and shared rates. Require the addressed human’s
current endpoint identity link before completing identity handoffs.
- Add production-authenticated integration scenarios. Companion #17739
contains the saved setup UI (including the required human
import/rotation step), product and permission documentation,
catalog/Vercel credential and callback ownership, and operation maps to
keep each review below 100 files. Both PRs must ship together; merge
this implementation first, then its companion.
## Verification
- Current implementation head:
`2ecd6451d0d0c8d82121a4e3fa4ce94c5befbbd6`, including master
`ea727dc3c2`, 99 files. The latest follow-up fixes the two valid
findings from the preceding f9 review. Exact-head Greptile has completed
at5/5 with no findings and Superagent passed. Full exact-head Linux CI
workflow40446606056 completed SUCCESS; only the queued external
Paperclip Review remains pending; earlier Linux passes do not qualify
this successor.
- Latest preserved-stack verification passed **2,165 tests** in fresh
PostgreSQL: **70 actual-auth scenarios**, 27 OpenAPI contracts, 2,022
native API tests and 46 saved GitHub wizard cases. **141 existing
gateway/policy acceptance tests** also passed. Workspace
typecheck/build, maps and token gates passed. The first workspace
typecheck was blocked by sandbox tsx IPC permissions; its authorized
rerun passed. This is focused local evidence; the full Linux CI matrix
is still required.
- Issuer/target gateway policy regressions were red before the fix. Both
decisions now intersect, including Block/Ask first/rates, same
credential identity and current token/user/agent authority after
approval. Invocations retain their token and actual requesting agent;
deleted tokens fail closed. Thirteen actual-auth cases cover revocation,
narrowed scope, approvals, shared rates and board compatibility.
Intermediate missing-token attribution, review-context and
implicit-session actor failures remain in the qualification record; all
final cases pass.
- Activity pagination now retains a bounded cursor across private-only
windows. The actual-JWT regression reaches three older readable events
after 27 private ones without exposing private details. A native
device-login regression was red for plaintext receipts; local check and
test-call results now use existing protected-response handling. The
passing actual-auth case proves encrypted persistence, exact replay,
redacted history and no response asset.
- Previous-head `5b55195593` Linux CI initially failed e2e shard 4/8 in
the existing connection-intent continuation test: periodic recovery
queued a repair at 01:21:47, the scripted journey completed the task at
01:21:48, and the extra queued run was cancelled at 01:21:49 because the
task was terminal. The assertion selected that latest cancelled run.
GitHub refused the first attempt while the workflow was nonterminal.
After completion, the single evidence-supported failed-job rerun was
accepted as **attempt 2** on the same head (workflow 40431202466). The
failed shard and final E2E aggregate passed on attempt 2, and the
complete Linux workflow for that head is successful. The original
failure remains part of qualification. No additional unchanged-code
rerun will be used for this failure.
- Companion CI also has an unchanged login-cleanup idleness timeout,
retained in its qualification record. That same 16-case suite passed in
this parent Linux run and a fresh local database. Both preceding
workflows passed after their single failed-job reruns as attempt 2; the
original failures remain recorded. No additional unchanged-code rerun
was used.
- Migration `0380_broken_aqueduct.sql` was regenerated through the
repository workflow after master took 0379 for project Git
authorization. It adds the responsible-user column with `IF NOT EXISTS`,
preserves upstream migrations, and satisfies snapshot retention and
migration safety checks.
- Upstream published Copilot v62 and v63 identities remain byte-for-byte
unchanged. The combined guidance is now v64, digest
`sha256:22e31f95ef972bcead98a4a707c9f33433558fd111e390e027a2bbb4346e112a`.
The normal shared TypeScript/Rust identity check and **61 Copilot
installation tests passed**. The old source-identity assertion first
failed against unchanged published v63; regenerating a new v64 preserves
the published release.
- Workspace `pnpm -r typecheck` and `pnpm build`, map validation, and UI
token gates passed on the final combined stack after the upstream merge.
Node 24.22.1 / pnpm 9.15.4 were used.
- **Earlier 82 API/OpenAPI tests passed** in fresh PostgreSQL 18: 55
assembled-server scenarios and 27 contracts. They use real persistent
agent keys, run JWTs, human sessions, production defaults, and
deterministic provider fixtures. No actor injection establishes access.
- The new email regression was red before per-account scope filtering.
The Arcade regression was red before human ownership and inventory-key
fixes; it also verifies actual-agent audit identity and a human-session
inventory read. The activity regression measured **200 authorization
checks before the fix and one afterward** for the same one-item page.
- **82 GitHub UI tests passed**, including all three new
failed-replacement regressions that were red before their fix. The UI
preserves an earlier copy only after confirming its committed
generation. Changed or unreadable results block copying and reconnecting
until reconciled. Human-only secret responses now include their
nonsecret committed generation.
- Private test status now rechecks the originating principal and linked
task, including mandatory agent/responsible-user privacy. Cross-agent
tests require the full caller tool access decision and the same
credential identity, regardless of auth label; caller and target
policies/approvals and rate limits apply before execution and after
human approval. A profile-only precondition was found to reject valid
allow-policy and scoped-grant access and has been removed. Both
permitted-access cases and a no-auth-label identity regression were red
before the correction; caller-specific Block still prevents provider
execution. The private-status regression was also red before its fix.
- The tool-activity regression measured 100 capability decisions and 102
run captures on a 100-event page before the fix. It now uses two task
decisions and at most two captures while retaining the same 50 visible
events, private-event filtering, and 403 after audit-grant revocation.
The regression was red before request-context reuse and task-decision
sharing; final server typecheck/build also pass.
- Generic chat configure/reconnect now hands agent-supplied verification
credentials to the addressed human before provider work or vault
changes. Actual key/JWT regressions were red for Slack signing-key
replacement and generic channel secret-reference rebinding. They now
prove unchanged refs and vaulted signing value, human setup, permitted
agent bot-token repair, and governed test messages. All 82 API/OpenAPI
cases pass in fresh PostgreSQL. Eight native receipt protections and
final server typecheck/build also pass.
- The preceding review found that old healthy state could retire a
credential-entry card before a human changed anything. The actual-JWT
regression was red. Saved field generations now change only with
protected values and commit with refs plus a verification fence; retries
retain the baseline. Old health, bot-token repair and unchanged entry
stay pending. New signing verification and current authority permit
continuation. A separate red race proved that user revocation during
provider authentication could return 403 after refs changed; authority
is now checked before persistence and inside the credential transaction.
The final 82 API/OpenAPI cases, four neighboring real-database
Slack/GitHub lifecycle cases, and final server typecheck/build pass.
- A preceding full run crossed a rate-window boundary. Database rows
prove one consumed slot at 23:35 and another at 23:36 for the same
policy/bucket. Only Date is frozen in the rate test cases; transport and
asynchronous timers remain real. Production rate behavior is unchanged,
and the final full run passes. An intermediate fixture failure that
overwrote hidden generation metadata with an API projection is retained
in the qualification record.
- Linux CI on an earlier head found two rate-admission assertions that
omitted the newly persisted bucket key. The failures were reproduced
locally; both assertions now verify the exact actor/tool bucket and all
49 policy-service tests pass. The companion qualification record retains
the initial and intermediate failures. Catalog creator attribution is
preserved in the companion to keep this implementation review at 99
files.
- The latest review reproduced three regressions before repair: complete
human replacements failed on unavailable old signing secrets, fresh
GitHub links incorrectly selected reconnect, and different
credential-entry requests shared an idempotency key. Each old protected
ref now has a narrow known-unavailable fallback; unexpected resolver
failures and company/scope violations still abort. Fresh imports keep
their credential form, configured endpoints keep reconnect, and
canonical field sets/completion modes distinguish handoffs while
matching retries retain their baseline. All 55 actual-auth scenarios and
27 separately run OpenAPI contracts pass, as do server typecheck/build.
Companion #17739 also passes all 41 wizard tests including actual
fresh-form submission, UI typecheck, maps and token gates. Earlier red
results and a mistyped OpenAPI filename are retained in qualification.
- The configured GitHub handoff regression was red in the
browser-component tests and actual-auth link/persistence tests. Saved
links now name requested fields and all/any completion mode; the
companion form submits those supplied fields through reconnect and
retains other saved credentials. Webhook/client-secret reconnect remains
human-only; first-setup secret restrictions remain. All 46 wizard tests
pass, including combined modes and legacy links. A duplicate review
referenced the older companion bf760912; the current form is already
fixed, linked in the reply, and its thread is resolved.
- Linux CI on bf760912 failed automatic Slack provisioning because vault
persistence cleared an already verified manifest-update challenge. The
same regression was reproduced locally. The provider-owned helper now
preserves proof only for the current signing fingerprint and callback
URL. Eight existing compatibility cases pass, including this case and
initial GitHub secret rejection. The first uncommitted GitHub-field
variant also failed that initial boundary and was narrowed to reconnect;
both failures remain in qualification.
- The upstream merge conflicted only in generated migration metadata.
Upstream 0379 is preserved, normal generation/pruning produced 0380, and
two repeated applications of the idempotent additive SQL passed. The
final combined stack passed **128 tests** (55 actual-auth, 27 OpenAPI
and 46 wizard cases) in fresh PostgreSQL. Eight Google UI cases also
pass; the two protocol cases could not initialize embedded PostgreSQL in
this fresh local attempt. Their earlier passing evidence remains
historical; fresh Linux CI must execute them. Vercel personal credential
attribution is preserved in companion #17739 with its other
ownership/consent follow-ups, keeping this PR at 99 files.
- The suite covers fresh defaults, restricted users/agents, scopes,
personal isolation, token revocation, human handoffs, callback replay,
execution approvals, authorized self-targeted policy changes,
self-approval denial, and current authority before commit. Reproduce API
checks with `pnpm exec vitest run
server/src/__tests__/connection-management.integration.test.ts
server/src/__tests__/openapi-routes.test.ts`, using a fresh database
through `PAPERCLIP_CONNECTION_MANAGEMENT_TEST_DATABASE_URL` or the
default embedded test database.
- Companion #17739 also records consent separately from token refresh so
saved Google choices survive refresh. Its eight service-picker and two
local-HTTP OAuth tests pass; both new UI cases were red before the fix.
Normal migration generation confirms no SQL change is needed for the
additive JSON metadata.
- The integration module shares one assembled server, authentication
factory, and provider lifecycle to preserve the
HTTP-to-vault-to-task-continuation proof. A reused external database
caused a Slack fixture identity conflict; the fresh final run passed.
Earlier fixture/transport and contract-update failures remain in the
companion qualification record.
- The broad local `pnpm test:run` attempt remains blocked by macOS
embedded PostgreSQL's host semaphore limit. It is not a clean pass. The
fresh Linux CI matrix must complete. Provider fixtures do not prove live
setup or an empirical LLM success rate.
- A current-head completed Paperclip assessment is not yet recorded. Its
earlier external recovery boundary has not been bypassed, and no
incomplete assessment is reported as passing. The user authorized both
merges when checks pass; merge this implementation first, then replay,
retarget, and requalify companion #17739 against the squash-merged
master.
- After the master refresh, local workspace typecheck/build, **97
actual-auth/OpenAPI cases** and **141 gateway/policy cases** pass. The
original fixed database port was occupied; Docker assigned a free
localhost port. The first neighbor setup command could not resolve root
`tsx`, leaving its schema uninitialized; the built database package
migrated the separate fixture and all unchanged cases passed. Both setup
failures are retained in qualification. Fresh Linux CI remains required;
no full local test-sweep pass is claimed.
- The f9 Greptile review scored **2/5** for two reproduced findings:
Slack test sends skipped the caller’s tool policy, and healthy App
credentials completed an unconfirmed identity handoff. Both are fixed in
this head and have resolved evidence threads
([Slack](https://github.com/paperclipai/paperclip/pull/17737#discussion_r4384927765),
[identity](https://github.com/paperclipai/paperclip/pull/17737#discussion_r4384927895)).
Actual key/JWT cases cover twelve Slack caller/target/recipient
combinations; the human-session GitHub OAuth callback alone remains
pending until explicit confirmation. All **99 API/OpenAPI tests** (72
actual-auth + 27 contracts), workspace typecheck and build pass. On the
combined companion, all four focused cases (including the twelve Slack
combinations), typecheck/build and maps also pass. The continuation
suite’s 50 embedded-PG cases were skipped because its database could not
start locally; fresh Linux CI must run them. Initial fixture
credential-collision and audit-projection assertion failures are
retained with the red reproductions in the companion qualification
record. No full local sweep or live consent is claimed.
## Risks
- The maintainer explicitly confirmed authorized policy administration,
including self-targeted changes that loosen controls. These writes
require management grants and the responsible user’s current authority.
Agents cannot approve their own governed action requests or edit their
own principal permission grants. The actual-auth regression exercises
both permitted administration and those denials.
- This is a broad authorization change. Incorrect scope or owner
resolution could deny permitted setup or expose another user's
resources. Integration cases cover both permitted and denied requests
with actual authentication.
- The additive nullable gateway-token column does not infer identities
for legacy clients or backfill agent grants. Legacy agent tokens without
a responsible user fail closed.
- Provider fixtures test supported setup and callback contracts. They do
not prove live provider availability.
## Model Used
- OpenAI Codex, GPT-6 family, with reasoning, repository tools, and code
execution. The runtime does not expose a more specific deployment ID or
context-window size.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass — focused suites above;
full local sweep remains blocked as documented
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green — exact `2ecd6451d0` Linux CI
completed SUCCESS; external Paperclip Review is still queued
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups —
exact `2ecd6451d0` review completed; both f9 findings have resolved
evidence replies
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
329 lines
29 KiB
TypeScript
329 lines
29 KiB
TypeScript
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
|
|
import { randomUUID } from "node:crypto";
|
|
import { mkdtemp, realpath, rm } from "node:fs/promises";
|
|
import os from "node:os";
|
|
import path from "node:path";
|
|
import { and, eq, sql } from "drizzle-orm";
|
|
import {
|
|
createDb, companies, agents, companyMemberships, companyDecisionModels, costEvents, decisionInvocations,
|
|
connectionGrants, connectionGrantMembers, toolConnectionInstalls, budgetPolicies, budgetReservations,
|
|
heartbeatRuns, issues, projects, runIdentityContexts, secretAccessEvents, companySecrets,
|
|
} from "@paperclipai/db";
|
|
import { startEmbeddedPostgresTestDatabase } from "@paperclipai/db/test-embedded-postgres";
|
|
import { DECISION_TEST_REQUEST, updateDecisionModelSchema, createCostEventSchema } from "@paperclipai/shared";
|
|
import { aiConnectionService } from "../services/ai-connections.js";
|
|
import { agentService } from "../services/agents.js";
|
|
import { decisionModelService, settingsDecisionTest, defineDecisionFeature, type DecisionContext } from "../services/decision-models.js";
|
|
import { decisionReceipt, type DecisionProviderOutcome } from "../services/decision-model-provider.js";
|
|
import { costService } from "../services/costs.js";
|
|
import { billingReconciliationService } from "../services/billing-reconciliation.js";
|
|
import { withAccountingTransaction } from "../services/accounting-transaction.js";
|
|
import express from "express";
|
|
import request from "supertest";
|
|
import { decisionModelRoutes } from "../routes/decision-models.js";
|
|
import { errorHandler } from "../middleware/index.js";
|
|
import { deliverBudgetEnforcement } from "../services/budgets.js";
|
|
|
|
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
|
|
let db: ReturnType<typeof createDb>;
|
|
let home: string;
|
|
const backgroundFeature = defineDecisionFeature("test.background", { background: true });
|
|
const outcome: DecisionProviderOutcome = { answers: { billing: { type: "boolean", probability: 0.9 } }, receipt: decisionReceipt("openai", { usage: { inputTokens: 100, outputTokens: 0 } }) };
|
|
beforeAll(async () => {
|
|
home = await realpath(await mkdtemp(path.join(os.tmpdir(), "paperclip-decisions-")));
|
|
vi.stubEnv("PAPERCLIP_HOME", home);
|
|
vi.stubEnv("PAPERCLIP_INSTANCE_ID", "decisions-fixture");
|
|
database = await startEmbeddedPostgresTestDatabase("paperclip-decisions-db-");
|
|
db = createDb(database.connectionString);
|
|
}, 90000);
|
|
afterAll(async () => { await database?.cleanup(); vi.unstubAllEnvs(); if (home) await rm(home, { recursive: true, force: true }); });
|
|
let sequence = 0;
|
|
async function fixture() {
|
|
const companyId = randomUUID(), agentId = randomUUID();
|
|
await db.insert(companies).values({ id: companyId, name: "Decision tests", issuePrefix: `DC${++sequence}` });
|
|
await db.insert(agents).values({ id: agentId, companyId, name: "Test agent", adapterType: "codex_local" });
|
|
await db.insert(companyMemberships).values(["alice", "bob"].map(principalId => ({ companyId, principalId, principalType: "user", status: "active", membershipRole: "owner" })));
|
|
const connections = aiConnectionService(db);
|
|
const binding = await connections.save(companyId, "alice", { provider: "openai", method: "api_key", name: "Decision key", ownership: "shared", apiKey: "fixture-key", agentIds: [], allAgents: true }, "fixture-key");
|
|
const provider = vi.fn().mockResolvedValue(structuredClone(outcome));
|
|
const service = decisionModelService(db, { provider });
|
|
const context: DecisionContext = { companyId, feature: settingsDecisionTest,
|
|
actor: { type: "board", userId: "alice", source: "session", companyIds: [companyId], memberships: [{ companyId, membershipRole: "owner", status: "active" }] } };
|
|
const config = { ...binding, enabled: true, allowBackground: true };
|
|
await service.configure(companyId, "alice", config);
|
|
return { companyId, agentId, binding, provider, service, context, config, connections };
|
|
}
|
|
describe("company decision service", () => {
|
|
it("returns answers and a committed charge before slow budget stop delivery", async () => {
|
|
const f = await fixture();
|
|
await db.insert(budgetPolicies).values({ companyId: f.companyId, scopeType: "company", scopeId: f.companyId,
|
|
windowKind: "calendar_month_utc", amount: 1, notifyEnabled: false });
|
|
f.provider.mockResolvedValue({ ...structuredClone(outcome), receipt: { ...outcome.receipt, costCents: "2.0000000" } });
|
|
let release!: () => void;
|
|
const gate = new Promise<void>(resolve => { release = resolve; });
|
|
const cancelWorkForScope = vi.fn(() => gate);
|
|
const service = decisionModelService(db, { provider: f.provider, budgetHooks: { cancelWorkForScope } });
|
|
const pending = service.decide(f.context, DECISION_TEST_REQUEST);
|
|
let delivery: Promise<void> | undefined;
|
|
let finished = false;
|
|
void pending.then(() => { finished = true; }, () => { finished = true; });
|
|
try {
|
|
await vi.waitFor(() => expect(finished).toBe(true), { timeout: 3000 });
|
|
expect(await pending).toMatchObject({ status: "succeeded", answers: outcome.answers });
|
|
expect(f.provider).toHaveBeenCalledOnce();
|
|
expect(cancelWorkForScope).not.toHaveBeenCalled();
|
|
const events = await db.select().from(costEvents).where(eq(costEvents.companyId, f.companyId));
|
|
expect(events).toHaveLength(1);
|
|
expect(events[0]).toMatchObject({ usageKind: "decision", costCents: 2 });
|
|
const [policy] = await db.select().from(budgetPolicies).where(eq(budgetPolicies.companyId, f.companyId));
|
|
expect(policy.enforcementVersion).toBeGreaterThan(policy.enforcementDeliveredVersion);
|
|
delivery = deliverBudgetEnforcement(db, { cancelWorkForScope }, f.companyId);
|
|
await vi.waitFor(() => expect(cancelWorkForScope).toHaveBeenCalledOnce());
|
|
expect((await db.select().from(budgetPolicies).where(eq(budgetPolicies.id, policy.id)))[0].enforcementDeliveredVersion).toBe(policy.enforcementDeliveredVersion);
|
|
release(); await delivery;
|
|
expect((await db.select().from(budgetPolicies).where(eq(budgetPolicies.id, policy.id)))[0].enforcementDeliveredVersion).toBe(policy.enforcementVersion);
|
|
} finally { release(); await pending.catch(() => undefined); await delivery?.catch(() => undefined); }
|
|
});
|
|
it("starts unconfigured with background sponsorship on and makes availability free", async () => {
|
|
const f = await fixture();
|
|
await db.delete(companyDecisionModels).where(eq(companyDecisionModels.companyId, f.companyId));
|
|
expect(await f.service.settings(f.companyId)).toMatchObject({ enabled: false, allowBackground: true, connectionId: null });
|
|
expect(await f.service.availability(f.context)).toEqual({ available: false, reason: "not_configured" });
|
|
await f.service.configure(f.companyId, "alice", updateDecisionModelSchema.parse({ ...f.binding, enabled: true }));
|
|
const before = await db.select().from(secretAccessEvents).where(eq(secretAccessEvents.companyId, f.companyId));
|
|
const memo = f.service.availabilityForRequest();
|
|
expect(memo(f.context)).toBe(memo(f.context));
|
|
expect(await memo(f.context)).toEqual({ available: true });
|
|
expect(await db.select().from(secretAccessEvents).where(eq(secretAccessEvents.companyId, f.companyId))).toHaveLength(before.length);
|
|
expect(f.provider).not.toHaveBeenCalled();
|
|
});
|
|
it("records a human service charge without an agent, exact cost and no input or answer content", async () => {
|
|
const f = await fixture();
|
|
const result = await f.service.decide(f.context, { ...DECISION_TEST_REQUEST, state: "PRIVATE PAYLOAD" });
|
|
expect(result.status).toBe("succeeded");
|
|
expect(f.provider.mock.calls[0][0].apiKey).toBe("fixture-key");
|
|
const [event] = await db.select().from(costEvents).where(eq(costEvents.companyId, f.companyId));
|
|
expect(event).toMatchObject({ agentId: null, responsibleUserId: "alice", usageKind: "decision", costCents: 0.001, costStatus: "estimated", heartbeatRunId: null });
|
|
const rows = await db.select().from(decisionInvocations).where(eq(decisionInvocations.companyId, f.companyId));
|
|
expect(rows[0]).toMatchObject({ status: "succeeded", actorType: "user", responsibleUserId: "alice", costEventId: event.id });
|
|
expect(JSON.stringify(rows)).not.toContain("PRIVATE PAYLOAD"); expect(JSON.stringify(rows)).not.toContain("probability");
|
|
expect((await costService(db).byUser(f.companyId)).rows.find(r => r.userId === "alice")?.costCentsExact).toBe("0.0010000");
|
|
expect((await costService(db).byAgent(f.companyId))[0].agentId).toBeNull();
|
|
expect((await db.select().from(budgetReservations).where(eq(budgetReservations.companyId, f.companyId)))[0].state).toBe("settled");
|
|
});
|
|
it("allows real background use, preserves explicit off, and never sponsors a denied human", async () => {
|
|
const f = await fixture();
|
|
const context: DecisionContext = { companyId: f.companyId, feature: backgroundFeature, actor: { type: "system" } };
|
|
expect((await f.service.decide(context, DECISION_TEST_REQUEST)).status).toBe("succeeded");
|
|
expect((await db.select().from(decisionInvocations).where(eq(decisionInvocations.companyId, f.companyId)))[0]).toMatchObject({ actorType: "system", responsibleUserId: null });
|
|
await f.service.configure(f.companyId, "alice", { ...f.config, allowBackground: false });
|
|
expect(await f.service.availability(context)).toEqual({ available: false, reason: "background_disabled" });
|
|
const second = await f.connections.save(f.companyId, "alice", { provider: "openrouter", method: "api_key", name: "Other key", ownership: "shared", apiKey: "other", agentIds: [], allAgents: true }, "other");
|
|
await f.service.configure(f.companyId, "alice", { ...second, enabled: true, allowBackground: false });
|
|
expect((await f.service.settings(f.companyId)).allowBackground).toBe(false);
|
|
await f.service.configure(f.companyId, "alice", f.config);
|
|
await db.insert(connectionGrantMembers).values({ companyId: f.companyId, grantId: f.binding.grantId, subjectType: "user", subjectId: "bob" });
|
|
expect(await f.service.decide({ ...f.context, feature: backgroundFeature }, DECISION_TEST_REQUEST)).toMatchObject({ status: "unavailable", reason: "access_denied" });
|
|
expect(f.provider).toHaveBeenCalledTimes(1);
|
|
});
|
|
it("retains the charge and dispatch snapshot when the task is deleted during provider work", async () => {
|
|
const f = await fixture(), issueId = randomUUID();
|
|
await db.insert(issues).values({ id: issueId, companyId: f.companyId, title: "Temporary task" });
|
|
f.provider.mockImplementationOnce(async () => {
|
|
await db.delete(issues).where(eq(issues.id, issueId));
|
|
return structuredClone(outcome);
|
|
});
|
|
expect((await f.service.decide({ ...f.context, issueId }, DECISION_TEST_REQUEST)).status).toBe("succeeded");
|
|
const [event] = await db.select().from(costEvents).where(eq(costEvents.companyId, f.companyId));
|
|
expect(event).toMatchObject({ issueId: null, costCents: 0.001 });
|
|
const [invocation] = await db.select().from(decisionInvocations).where(eq(decisionInvocations.companyId, f.companyId));
|
|
expect(invocation).toMatchObject({ issueId, status: "succeeded", costEventId: event.id });
|
|
});
|
|
it("rechecks revoked grants after availability, and can disable a revoked connection", async () => {
|
|
const f = await fixture(); expect(await f.service.availability(f.context)).toEqual({ available: true });
|
|
await db.update(connectionGrants).set({ status: "revoked" }).where(eq(connectionGrants.id, f.binding.grantId));
|
|
expect(await f.service.decide(f.context, DECISION_TEST_REQUEST)).toMatchObject({ status: "unavailable", reason: "connection_unavailable" });
|
|
await f.service.configure(f.companyId, "alice", { ...f.config, enabled: false });
|
|
expect(await f.service.availability(f.context)).toEqual({ available: false, reason: "disabled" }); expect(f.provider).not.toHaveBeenCalled();
|
|
});
|
|
it.each(["disabled", "rotated"])("rejects a secret %s while admission waits for accounting", async change => {
|
|
const f = await fixture();
|
|
const [secret] = await db.select().from(companySecrets).where(eq(companySecrets.companyId, f.companyId));
|
|
let release!: () => void, locked!: (pid: number) => void;
|
|
const gate = new Promise<void>(resolve => { release = resolve; });
|
|
const acquired = new Promise<number>(resolve => { locked = resolve; });
|
|
const holding = withAccountingTransaction(db, f.companyId, async tx => {
|
|
const [row] = await tx.execute<{ pid: number }>(sql`select pg_backend_pid() as pid`);
|
|
locked(row.pid); await gate;
|
|
});
|
|
const pid = await acquired;
|
|
const pending = f.service.decide(f.context, DECISION_TEST_REQUEST);
|
|
try {
|
|
await vi.waitFor(async () => {
|
|
const [row] = await db.execute<{ count: number }>(sql`select count(*)::int as count from pg_stat_activity where ${pid} = any(pg_blocking_pids(pid))`);
|
|
expect(row.count).toBeGreaterThan(0);
|
|
}, { timeout: 5000 });
|
|
await db.update(companySecrets).set(change === "disabled" ? { status: "disabled" } : { latestVersion: secret.latestVersion + 1 }).where(eq(companySecrets.id, secret.id));
|
|
} finally { release(); await holding; }
|
|
expect(await pending).toMatchObject({ status: "unavailable", reason: "connection_unavailable" });
|
|
expect(f.provider).not.toHaveBeenCalled();
|
|
expect(await db.select().from(decisionInvocations).where(eq(decisionInvocations.companyId, f.companyId))).toHaveLength(0);
|
|
});
|
|
it("preserves sponsorship off when a deleted grant is replaced", async () => {
|
|
const f = await fixture();
|
|
await f.service.configure(f.companyId, "alice", { ...f.config, allowBackground: false });
|
|
await db.delete(connectionGrants).where(eq(connectionGrants.id, f.binding.grantId));
|
|
expect(await f.service.availability(f.context)).toMatchObject({ available: false, reason: "connection_unavailable" });
|
|
expect((await f.service.settings(f.companyId)).allowBackground).toBe(false);
|
|
const next = await f.connections.save(f.companyId, "alice", { provider: "openrouter", method: "api_key", name: "Replacement", ownership: "shared", apiKey: "other", agentIds: [], allAgents: true }, "other");
|
|
await f.service.configure(f.companyId, "alice", { ...next, enabled: true });
|
|
expect((await f.service.settings(f.companyId)).allowBackground).toBe(false);
|
|
expect(f.provider).not.toHaveBeenCalled();
|
|
});
|
|
it("rejects foreign-company credentials, membership loss, and invented feature handles", async () => {
|
|
const f = await fixture(), other = await fixture();
|
|
await expect(f.service.configure(f.companyId, "alice", { ...other.config })).rejects.toThrow();
|
|
await db.update(companyMemberships).set({ status: "inactive" }).where(and(eq(companyMemberships.companyId, f.companyId), eq(companyMemberships.principalId, "alice")));
|
|
expect(await f.service.availability(f.context)).toMatchObject({ available: false, reason: "access_denied" });
|
|
expect(await f.service.decide({ ...f.context, feature: { id: "settings.test", background: true }, actor: { type: "system" } }, DECISION_TEST_REQUEST)).toMatchObject({ status: "unavailable", reason: "access_denied" });
|
|
});
|
|
it("captures the active run identity and checks both human and agent access", async () => {
|
|
const f = await fixture(), issueId = randomUUID(), runId = randomUUID();
|
|
await db.insert(issues).values({ id: issueId, companyId: f.companyId, title: "Decision task", status: "in_progress", assigneeAgentId: f.agentId });
|
|
await db.insert(heartbeatRuns).values({ id: runId, companyId: f.companyId, agentId: f.agentId, status: "running", invocationSource: "on_demand", responsibleUserId: "alice", contextSnapshot: { issueId } });
|
|
const [identity] = await db.insert(runIdentityContexts).values({ companyId: f.companyId, runId, revision: 1, responsibleUserId: "bob", cause: "steering", correlationId: "test", acceptedAt: new Date() }).returning();
|
|
await db.update(heartbeatRuns).set({ responsibleUserId: "bob", activeIdentityContextId: identity.id }).where(eq(heartbeatRuns.id, runId));
|
|
const context: DecisionContext = { companyId: f.companyId, feature: settingsDecisionTest, actor: { type: "agent", agentId: f.agentId, runId, companyId: f.companyId, source: "agent_jwt", onBehalfOfUserId: "alice" } };
|
|
expect((await f.service.decide(context, DECISION_TEST_REQUEST)).status).toBe("succeeded");
|
|
expect((await db.select().from(decisionInvocations).where(eq(decisionInvocations.companyId, f.companyId)))[0]).toMatchObject({ responsibleUserId: "bob", runId, issueId, identityContextId: identity.id });
|
|
await db.delete(toolConnectionInstalls).where(eq(toolConnectionInstalls.connectionId, f.binding.connectionId));
|
|
expect(await f.service.availability(context)).toMatchObject({ available: false, reason: "access_denied" });
|
|
await db.update(heartbeatRuns).set({ responsibleUserId: null }).where(eq(heartbeatRuns.id, runId));
|
|
expect(await f.service.availability(context)).toMatchObject({ available: false, reason: "responsible_user_missing" });
|
|
});
|
|
it("serializes budget admission against concurrent requests", async () => {
|
|
const f = await fixture();
|
|
await db.insert(budgetPolicies).values({ companyId: f.companyId, scopeType: "company", scopeId: f.companyId, windowKind: "calendar_month_utc", amount: 1, reservationCents: "1", notifyEnabled: false });
|
|
let release!: () => void; const gate = new Promise<void>(resolve => { release = resolve; });
|
|
let entered!: () => void; const started = new Promise<void>(resolve => { entered = resolve; });
|
|
f.provider.mockImplementationOnce(async () => { entered(); await gate; return structuredClone(outcome); });
|
|
const first = f.service.decide(f.context, DECISION_TEST_REQUEST);
|
|
await started;
|
|
try { expect(await f.service.decide(f.context, DECISION_TEST_REQUEST)).toMatchObject({ status: "unavailable", reason: "budget_blocked" }); }
|
|
finally { release(); }
|
|
expect((await first).status).toBe("succeeded"); expect(f.provider).toHaveBeenCalledTimes(1);
|
|
});
|
|
it("blocks agent deletion while an in-flight or unknown decision retains a budget hold", async () => {
|
|
const f = await fixture(), runId = randomUUID();
|
|
await db.insert(heartbeatRuns).values({ id: runId, companyId: f.companyId, agentId: f.agentId, status: "running", invocationSource: "on_demand", responsibleUserId: "alice" });
|
|
await db.insert(budgetPolicies).values({ companyId: f.companyId, scopeType: "company", scopeId: f.companyId, windowKind: "calendar_month_utc", amount: 1, reservationCents: "1", notifyEnabled: false });
|
|
const context: DecisionContext = { companyId: f.companyId, feature: settingsDecisionTest,
|
|
actor: { type: "agent", agentId: f.agentId, runId, companyId: f.companyId, source: "agent_jwt" } };
|
|
const remove = () => agentService(db).remove(f.agentId);
|
|
let inFlightDeletionError: unknown;
|
|
f.provider.mockImplementationOnce(async () => {
|
|
// Exercise the accounting guard after lifecycle termination.
|
|
await db.update(agents).set({ status: "terminated", lifecycleState: "terminated" }).where(eq(agents.id, f.agentId));
|
|
inFlightDeletionError = await remove().catch(error => error);
|
|
expect(await f.service.decide(f.context, DECISION_TEST_REQUEST)).toMatchObject({ status: "unavailable", reason: "budget_blocked" });
|
|
return { errorCode: "timeout", receipt: decisionReceipt("openai", null) };
|
|
});
|
|
expect((await f.service.decide(context, DECISION_TEST_REQUEST)).status).toBe("failed");
|
|
expect(inFlightDeletionError).toMatchObject({ status: 409, details: { code: "agent_decision_accounting_pending" } });
|
|
await expect(remove()).rejects.toMatchObject({ status: 409, details: { code: "agent_decision_accounting_pending" } });
|
|
expect((await db.select().from(budgetReservations).where(eq(budgetReservations.companyId, f.companyId)))[0]).toMatchObject({ state: "held", amountCents: "1.0000000" });
|
|
});
|
|
it("holds unknown billing until an audited correction, without rerunning the provider", async () => {
|
|
const f = await fixture(); f.provider.mockResolvedValue({ errorCode: "timeout", receipt: decisionReceipt("openai", null) });
|
|
expect((await f.service.decide(f.context, DECISION_TEST_REQUEST)).status).toBe("failed");
|
|
const [event] = await db.select().from(costEvents).where(eq(costEvents.companyId, f.companyId));
|
|
expect(event.costStatus).toBe("unpriced");
|
|
expect((await db.select().from(budgetReservations).where(eq(budgetReservations.companyId, f.companyId)))[0].state).toBe("held");
|
|
await billingReconciliationService(db).adjust(f.companyId, event.id, { idempotencyKey: "correction", expectedCents: "0", correctedCents: "0.002", reason: "Verified provider invoice", pricing: { source: "operator" } }, "alice");
|
|
expect((await db.select().from(budgetReservations).where(eq(budgetReservations.companyId, f.companyId)))[0].state).toBe("settled");
|
|
expect(f.provider).toHaveBeenCalledTimes(1);
|
|
});
|
|
it("recovers an interrupted dispatch once without paying for a retry", async () => {
|
|
const f = await fixture();
|
|
const [invocation] = await db.insert(decisionInvocations).values({ companyId: f.companyId, feature: "test.background", actorType: "system", actorId: "paperclip-decisions",
|
|
...f.binding, provider: "openai", model: "gpt-6-luna", questionTypes: ["boolean"], startedAt: new Date(Date.now() - 180_000) }).returning();
|
|
await db.insert(budgetReservations).values({ companyId: f.companyId, decisionInvocationId: invocation.id, amountCents: "0.01", providerStartedAt: invocation.startedAt });
|
|
await Promise.all([f.service.recoverInterrupted(), f.service.recoverInterrupted()]);
|
|
const events = await db.select().from(costEvents).where(eq(costEvents.companyId, f.companyId));
|
|
expect(events).toHaveLength(1); expect(events[0].costStatus).toBe("unpriced"); expect(f.provider).not.toHaveBeenCalled();
|
|
});
|
|
it("keeps public cost reporting agent-scoped and bounds decision input before payment", async () => {
|
|
expect(createCostEventSchema.safeParse({ agentId: null, provider: "openai", model: "gpt-6-luna", costCents: 1, occurredAt: new Date().toISOString() }).success).toBe(false);
|
|
const f = await fixture(); await expect(f.service.decide(f.context, { ...DECISION_TEST_REQUEST, state: "x".repeat(32_001) })).rejects.toThrow(); expect(f.provider).not.toHaveBeenCalled();
|
|
});
|
|
it("preserves an explicit sponsorship opt-out when a subsequent edit omits the field", async () => {
|
|
const f = await fixture();
|
|
await f.service.configure(f.companyId, "alice", { ...f.config, allowBackground: false });
|
|
const second = await f.connections.save(f.companyId, "alice", { provider: "openrouter", method: "api_key", name: "Other", ownership: "shared", apiKey: "other", agentIds: [], allAgents: true }, "other");
|
|
await f.service.configure(f.companyId, "alice", { ...second, enabled: true });
|
|
expect((await f.service.settings(f.companyId)).allowBackground).toBe(false);
|
|
});
|
|
it("rejects revoked secrets without provider work and returns usage on successful calls", async () => {
|
|
const f = await fixture();
|
|
expect(await f.service.decide(f.context, DECISION_TEST_REQUEST)).toMatchObject({ status: "succeeded", usage: { costCents: "0.0010000", costStatus: "estimated", inputTokens: 100 } });
|
|
await db.update(companySecrets).set({ status: "revoked" }).where(eq(companySecrets.companyId, f.companyId));
|
|
expect(await f.service.decide(f.context, DECISION_TEST_REQUEST)).toMatchObject({ status: "unavailable" });
|
|
expect(f.provider).toHaveBeenCalledTimes(1);
|
|
});
|
|
it.each(["agent", "project"] as const)("reserves and enforces applicable %s budgets", async scopeType => {
|
|
const f = await fixture(), projectId = randomUUID(), issueId = randomUUID(), runId = randomUUID();
|
|
await db.insert(projects).values({ id: projectId, companyId: f.companyId, name: "Decisions project" });
|
|
await db.insert(issues).values({ id: issueId, companyId: f.companyId, projectId, title: "Budgeted task", assigneeAgentId: f.agentId });
|
|
await db.insert(heartbeatRuns).values({ id: runId, companyId: f.companyId, agentId: f.agentId, status: "running", invocationSource: "on_demand", responsibleUserId: "alice", contextSnapshot: { issueId } });
|
|
const context: DecisionContext = { companyId: f.companyId, feature: settingsDecisionTest, actor: { type: "agent", agentId: f.agentId, runId, companyId: f.companyId, source: "agent_jwt" } };
|
|
const [policy] = await db.insert(budgetPolicies).values({ companyId: f.companyId, scopeType, scopeId: scopeType === "agent" ? f.agentId : projectId, windowKind: "calendar_month_utc", amount: 1, reservationCents: "2", notifyEnabled: false }).returning();
|
|
expect(await f.service.availability(context)).toEqual({ available: true });
|
|
expect(await f.service.decide(context, DECISION_TEST_REQUEST)).toMatchObject({ status: "unavailable", reason: "budget_blocked" });
|
|
expect(f.provider).not.toHaveBeenCalled();
|
|
await db.update(budgetPolicies).set({ amount: 5 }).where(eq(budgetPolicies.id, policy.id));
|
|
expect((await f.service.decide(context, DECISION_TEST_REQUEST)).status).toBe("succeeded");
|
|
expect((await db.select().from(budgetReservations).where(eq(budgetReservations.companyId, f.companyId)))[0]).toMatchObject({ agentId: f.agentId, projectId, amountCents: "2.0000000", state: "settled" });
|
|
});
|
|
it("releases reservations when the provider rejects a request before model work", async () => {
|
|
const f = await fixture();
|
|
f.provider.mockResolvedValue({ noProviderWork: true, errorCode: "provider_auth_failed", receipt: { ...outcome.receipt, inputTokens: 0, costCents: "0", pricingProvenance: { source: "unknown", evidence: "HTTP 401" } } });
|
|
expect((await f.service.decide(f.context, DECISION_TEST_REQUEST)).status).toBe("failed");
|
|
expect((await db.select().from(budgetReservations).where(eq(budgetReservations.companyId, f.companyId)))[0].state).toBe("released");
|
|
expect((await db.select().from(costEvents).where(eq(costEvents.companyId, f.companyId)))[0].costCents).toBe(0);
|
|
});
|
|
it("enforces manager-only configuration and tests, company isolation, and fixed sample execution", async () => {
|
|
const f = await fixture(), other = await fixture();
|
|
const app = express(); app.use(express.json());
|
|
let actor = f.context.actor;
|
|
app.use((req, _res, next) => { req.actor = actor as typeof req.actor; next(); });
|
|
app.use("/api", decisionModelRoutes(db)); app.use(errorHandler);
|
|
const url = `/api/companies/${f.companyId}/decision-model`;
|
|
expect((await request(app).get(url)).body.canManage).toBe(true);
|
|
expect((await request(app).get(`/api/companies/${other.companyId}/decision-model`)).status).toBe(403);
|
|
await db.update(companyMemberships).set({ membershipRole: "viewer" }).where(and(eq(companyMemberships.companyId, f.companyId), eq(companyMemberships.principalId, "alice")));
|
|
actor = { type: "board", userId: "alice", source: "session", companyIds: [f.companyId], memberships: [{ companyId: f.companyId, status: "active", membershipRole: "viewer" }] };
|
|
expect((await request(app).get(url)).body).toEqual({ canManage: false, settings: null, choices: [] });
|
|
expect((await request(app).put(url).send(f.config)).status).toBe(403);
|
|
expect((await request(app).post(`${url}/test`).send({ actor: { type: "system" } })).status).toBe(403);
|
|
await db.update(companyMemberships).set({ membershipRole: "owner" }).where(and(eq(companyMemberships.companyId, f.companyId), eq(companyMemberships.principalId, "alice")));
|
|
actor = f.context.actor;
|
|
await f.service.configure(f.companyId, "alice", { ...f.config, enabled: false });
|
|
expect((await request(app).post(`${url}/test`).send({ state: "untrusted", actor: { type: "system" }, model: "untrusted" })).body).toEqual({ status: "unavailable", reason: "disabled" });
|
|
expect((await request(app).post(`${url}/decide`).send(DECISION_TEST_REQUEST)).status).toBe(404);
|
|
expect((await request(app).get(`${url}/history?limit=501`)).status).toBe(400);
|
|
expect((await request(app).get(`${url}/history?from=2026-10-01T00:00:00.000Z&to=2026-10-31T23:59:59.000Z`)).status).toBe(200);
|
|
});
|
|
it("keeps private task links out of company cost history for unauthorized readers", async () => {
|
|
const f = await fixture(), projectId = randomUUID(), issueId = randomUUID();
|
|
await db.insert(projects).values({ id: projectId, companyId: f.companyId, name: "Private project", visibility: "private", privacyOwnerUserId: "bob" });
|
|
await db.insert(issues).values({ id: issueId, companyId: f.companyId, projectId, title: "PRIVATE TITLE", identifier: "SECRET-1" });
|
|
await f.service.decide({ companyId: f.companyId, feature: backgroundFeature, actor: { type: "system" }, issueId }, DECISION_TEST_REQUEST);
|
|
await db.update(companyMemberships).set({ membershipRole: "member" }).where(and(eq(companyMemberships.companyId, f.companyId), eq(companyMemberships.principalId, "alice")));
|
|
const actor = { type: "board" as const, userId: "alice", source: "session" as const, companyIds: [f.companyId], memberships: [{ companyId: f.companyId, status: "active", membershipRole: "member" }] };
|
|
expect(await f.service.availability({ ...f.context, actor, issueId })).toMatchObject({ available: false, reason: "access_denied" });
|
|
const history = await f.service.history(f.companyId, actor);
|
|
expect(history[0]).toMatchObject({ issueId: null, issueIdentifier: null, runId: null });
|
|
expect(JSON.stringify(history)).not.toContain("SECRET");
|
|
});
|
|
});
|