From 11a0ff02050657f73cdac3fc0ab49c076a834ca0 Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Tue, 17 Nov 2026 23:33:47 -0600 Subject: [PATCH] feat(connections): per-company workload identity issuer with public signing keys (#22083) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents use outside services through connections. The control plane holds each connection's credentials and makes the calls. > - Some clouds, such as AWS, do not need stored keys. They trust a signed token from an identity provider that they can verify against its public keys. > - Paperclip has no signing key whose public half it can publish. Agent identity keys are Ed25519, which AWS does not accept, and the agent's run receives the private key. Agent run tokens use a shared secret. > - This pull request gives each company its own signing key and publishes the public half as a standard OIDC issuer. > - The benefit is that a connection, such as the planned AWS connector, can get its own short-lived cloud credentials with no stored cloud keys. The connection's role in the customer's account sets its permissions. ## Linked Issues or Issue Description Refs #19414 and #10411. Those pull requests add OIDC *sign-in* to Paperclip. This pull request is the other direction: Paperclip *issues* tokens that outside clouds verify. **Subsystem affected** Connections (tool access) and the database. **Problem or motivation** An AWS connector should not store a long-lived AWS access key. AWS supports OIDC federation (`AssumeRoleWithWebIdentity`). AWS fetches the issuer's public keys over HTTPS and checks each token. Paperclip has no key that it can publish for this, so this method is not available. **Proposed solution** Each company gets an ES256 key that Paperclip creates on first use. The vault master key encrypts the private half. The issuer is `https://{instance}/oidc/companies/{companyId}`. Paperclip publishes a discovery document and a JWKS there. Tokens last 5 minutes and have `sub = connection:{id}`. Only the control plane signs them. The board can see key state and run planned rotation or emergency rotation. The design is in `doc/plans/2026-11-17-workload-identity-issuer.md`. The AWS connector plan that uses it is in `doc/plans/2026-11-17-aws-connector-and-minted-credentials.md`. **Alternatives considered** - Reuse agent identity keys. Rejected: AWS does not accept Ed25519, and the agent's run receives the private key. - Derive the key from the master key, with no table. Rejected: the master key alone would then sign as production. Worktree environments copy the master key on purpose. A master key rotation would also change the signing key and break every customer trust. - Use one issuer for the instance. Rejected: if a customer's role checks the issuer but not `sub`, any company on a shared instance could assume that role. ## What Changed - `packages/db`: new table `company_workload_identity_keys` and migration `0498`. The migration only creates a new table, so the previous release keeps working. Partial unique indexes allow one `active` key and one `next` key per company. - `server/src/services/workload-identity.ts`: lazy key creation with concurrent convergence, a private-key cache with a fail-closed key match check, the discovery document, the JWKS, `signConnectionToken`, and rotation steps. A key that is no longer active stays published for 24 hours. Paperclip calculates this when it reads the keys, so no background job is necessary. - `server/src/routes/workload-identity.ts`: public `GET /oidc/companies/{companyId}/.well-known/openid-configuration` and `GET /oidc/companies/{companyId}/jwks.json` (404 until a key exists). Board `GET /api/companies/{companyId}/workload-identity` and `POST .../workload-identity/rotation`, with an activity log entry for each rotation. - Token issuance writes a tool-access audit event. The event never contains the token. - `cli`: worktree seeds exclude the new table in the minimal mode and the full mode. This is the same as for `agent_identity_keys`, so a development copy never holds production's signing key. - Docs: `docs/api/workload-identity.md`, an entry in the API overview, and the two plan docs. - No connector uses `signConnectionToken` yet. The AWS connector pull request will use it. ## Verification - `cd server && npx vitest run src/__tests__/workload-identity.test.ts`: 8 tests pass. The tests cover these items: - The issuer requires https with no port and no loopback host. - The endpoints return 404 before a key exists. - Six concurrent first uses create one key. - The JWKS contains only `kty crv x y kid alg use`. - A token verifies against the published key and has the expected claims. - A company cannot sign for another company's connection, and its keys do not verify that company's tokens. - Signing stops when the public address is not usable or has changed. - Two-step rotation and emergency rotation work. - Board-only access is enforced, and each rotation writes an activity log entry. - Red/green: I put back a JWK that leaks `d` and removed the company check on the connection. Three tests failed. I restored the code, and all 8 tests passed. - `cd cli && npx vitest run src/__tests__/worktree.test.ts`: 63 tests pass. - `npx tsc --noEmit` passes for `packages/db`, `server` and `cli`. - I started this branch on a real dev server. Before a key existed, the public routes returned JSON 404 (not the UI page). After an emergency rotation from the board route, the JWKS returned one public P-256 key. On the http dev instance, discovery stayed 404 and the board route reported `not_https`. - `pnpm verify` exited 2 because of the migration. The required CI gates supply the broad coverage. - Not run: a real AWS `AssumeRoleWithWebIdentity` call. That needs a public HTTPS instance. It is the first step of the AWS connector work and will run against staging. ## Risks - The migration is low risk. It creates a new table with no backfill, and the compatibility checker classifies it as compatible with the running release. - The public routes are unauthenticated. They show only public keys, return 404 for companies without a key, and use random company UUIDs. Each request is one small indexed read. - The issuer URL shape is permanent after customers trust it. A change to the instance's public address breaks those trusts until the customer updates the identity provider. `signConnectionToken` stops on that mismatch when the caller supplies the recorded issuer. - Anyone who holds the production database and the master key could sign for that instance's companies. That is the same exposure as every other vault secret. - Edge routing on Paperclip Cloud must pass `/oidc/...` to the stack. I will confirm this on staging with the AWS connector work. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, web fetch of AWS documentation, a local dev server). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 Co-authored-by: Paperclip --- cli/src/__tests__/worktree.test.ts | 3 +- cli/src/commands/worktree-lib.ts | 3 +- ...17-aws-connector-and-minted-credentials.md | 299 +++ .../2026-11-17-workload-identity-issuer.md | 197 ++ docs/api/overview.md | 4 + docs/api/workload-identity.md | 54 + .../db/src/migrations/0498_quiet_hardball.sql | 18 + ...{0488_snapshot.json => 0498_snapshot.json} | 2215 ++++++++++++++++- packages/db/src/migrations/meta/_journal.json | 7 + .../schema/company_workload_identity_keys.ts | 34 + packages/db/src/schema/index.ts | 1 + server/src/__tests__/openapi-routes.test.ts | 2 + .../src/__tests__/workload-identity.test.ts | 356 +++ server/src/app.ts | 6 + server/src/routes/openapi.ts | 57 + server/src/routes/workload-identity.ts | 70 + server/src/services/workload-identity.ts | 384 +++ 17 files changed, 3703 insertions(+), 7 deletions(-) create mode 100644 doc/plans/2026-11-17-aws-connector-and-minted-credentials.md create mode 100644 doc/plans/2026-11-17-workload-identity-issuer.md create mode 100644 docs/api/workload-identity.md create mode 100644 packages/db/src/migrations/0498_quiet_hardball.sql rename packages/db/src/migrations/meta/{0488_snapshot.json => 0498_snapshot.json} (96%) create mode 100644 packages/db/src/schema/company_workload_identity_keys.ts create mode 100644 server/src/__tests__/workload-identity.test.ts create mode 100644 server/src/routes/workload-identity.ts create mode 100644 server/src/services/workload-identity.ts diff --git a/cli/src/__tests__/worktree.test.ts b/cli/src/__tests__/worktree.test.ts index d86751f0d21..67073b6cf8d 100644 --- a/cli/src/__tests__/worktree.test.ts +++ b/cli/src/__tests__/worktree.test.ts @@ -539,8 +539,9 @@ describe("worktree helpers", () => { expect(minimal.excludedTables).toContain("agent_task_sessions"); expect(minimal.nullifyColumns.issues).toEqual(["checkout_run_id", "execution_run_id"]); - expect(full.excludedTables).toEqual(["agent_identity_keys"]); + expect(full.excludedTables).toEqual(["agent_identity_keys", "company_workload_identity_keys"]); expect(minimal.excludedTables).toContain("agent_identity_keys"); + expect(minimal.excludedTables).toContain("company_workload_identity_keys"); expect(full.nullifyColumns).toEqual({}); }); diff --git a/cli/src/commands/worktree-lib.ts b/cli/src/commands/worktree-lib.ts index f4524c38a8e..dc69e8dcf00 100644 --- a/cli/src/commands/worktree-lib.ts +++ b/cli/src/commands/worktree-lib.ts @@ -60,6 +60,7 @@ export type WorktreeSeedPlan = { const MINIMAL_WORKTREE_EXCLUDED_TABLES = [ "agent_identity_keys", + "company_workload_identity_keys", "activity_log", "agent_runtime_state", "agent_task_sessions", @@ -122,7 +123,7 @@ export function resolveWorktreeSeedPlan(mode: WorktreeSeedMode): WorktreeSeedPla if (mode === "full") { return { mode, - excludedTables: ["agent_identity_keys"], + excludedTables: ["agent_identity_keys", "company_workload_identity_keys"], nullifyColumns: {}, }; } diff --git a/doc/plans/2026-11-17-aws-connector-and-minted-credentials.md b/doc/plans/2026-11-17-aws-connector-and-minted-credentials.md new file mode 100644 index 00000000000..b9bd880faf4 --- /dev/null +++ b/doc/plans/2026-11-17-aws-connector-and-minted-credentials.md @@ -0,0 +1,299 @@ +# AWS connector, and connectors that mint short-lived credentials + +Status: proposal, 2026-11-17 (revised the same day). Supersedes the earlier +draft that ran an AWS proxy process on the control plane. + +## The message + +AWS should be a **remote MCP connector with credentials minted on the control +plane**. Nothing needs to run on our side: + +1. The customer gives the AWS connection its own AWS identity. That is either + a role that trusts only this connection, using signed tokens from the + instance with no stored keys, or a dedicated access key. Nothing is shared + across the instance. +2. At call time, the control plane uses that connection's identity to get a + one-hour AWS bearer token, and calls AWS's hosted + **AWS MCP Server** through the existing remote MCP gateway. +3. IAM decides what is allowed, CloudTrail records it, and Paperclip's + gateway applies agent policy and logs every call. + +Every piece is AWS-owned and documented for automated agents. We add no +process, no Python runtime, and no third-party code. + +The product-wide feature this motivates is **minted credentials**: a +connection type can turn saved setup (a role, a service account, an app key) +into a short-lived token when it's needed, instead of storing a long-lived +key. AWS is the first provider. + +Running connector MCP servers on the control plane is still a real future +need, but AWS no longer requires it. It is moved to its own plan (see +"Deferred" below). + +## Where things run + +- Agents run in sandboxes or VMs and call back to the control plane's MCP + endpoints (for example `/mcp/native-runs/:runId`). +- The control plane's tool gateway executes connector calls + ([tool-gateway.ts:9653](../../server/src/services/tool-gateway.ts)). +- Credentials are resolved inside the gateway at call time + ([`resolveCredentialHeaders`, tool-gateway.ts:4392](../../server/src/services/tool-gateway.ts)). + Vercel Connect already uses this seam to resolve short-lived tokens at + invocation time (tool-gateway.ts:4460). + +So minting, caching and use of the AWS token all happen on the control +plane. The agent's sandbox never sees AWS credentials. + +## AWS options compared (checked 2026-11-17) + +| Option | Owner | Status | Runs on our side | Verdict | +|---|---|---|---|---| +| **A. Hosted AWS MCP Server + bearer token from IAM** (`signin:CreateOAuth2TokenWithIAM`, client-credentials grant) | AWS | AWS-recommended server. Token flow documented for "AI agents running automated workflows". Official JS SDK: `@aws-sdk/client-signin`, `CreateOAuth2TokenWithIAMCommand`. | Nothing: two AWS API calls, then our existing remote MCP path | **Recommended** | +| B. Hosted AWS MCP Server via `mcp-proxy-for-aws` (SigV4 proxy) | AWS (`github.com/aws/mcp-proxy-for-aws`, Apache-2.0, v1.8.0 on 2026-11-03) | Official, maintained. AWS positions it for terminal and IDE clients. | A Python process per run on the control plane | Works, but adds a runtime and processes for no gain over A | +| C. `awslabs.aws-api-mcp-server` (local `call_aws`) | AWS Labs | **Superseded** by the hosted server (notice in its README and a migration guide) | Python process | Don't build on it | +| D. Role credentials in the agent's sandbox + `aws` CLI | AWS CLI | Stable | Nothing (sandbox) | Later, opt-in, for Terraform/CDK-style work. Ungoverned per call. | + +What option A gives up compared with the proxy, and how we cover it: + +- **Proxy `--read-only`** (hides tools that write): our gateway already + classifies tools by reviewed risk. We can also mint read-only tokens through + an IAM session policy (below), which is stronger because AWS enforces it. +- **Switching between accounts in one session**: one Paperclip connection is + one AWS account and role. Companies with several accounts add several + connections. +- **Default region metadata**: the proxy injects `AWS_REGION` into requests. + The spike checks whether we can send the same metadata directly. Otherwise + the region goes in the connection's agent instructions, and agents pass it + per call. + +## Design + +### 1. Minted credentials (product-wide) + +A new credential source for remote MCP methods, next to `paperclip_vault` and +`vercel_connect`: + +```ts +interface CredentialMinter { + // Saved setup + grant + effective agent policy -> short-lived headers. + mint(input: { + setup: ConnectionSetup; + grant: Grant; + scope: "read" | "write"; + }): Promise<{ headers: Record; expiresAt: Date }>; +} +``` + +- **Cache** in memory per company, connection, grant and scope until five + minutes before expiry. Tokens are never persisted or logged. +- **401 from the provider**: drop the cached token, mint once, and retry that + call once. An authentication failure means the provider didn't execute the + call, so the retry is safe. +- **Mint failure** (role deleted, trust changed) marks the connection's health + with a plain-language reason ("Paperclip can no longer use the AWS role. Check + that it still trusts Paperclip.") and never shows raw AWS errors to end + users. +- **Restarts and deploys**: the cache is memory-only, so the next call after a + restart simply mints again. There is nothing to recover and no Retry. + +### 2. AWS minter + +**Each connection carries its own AWS identity. Nothing is instance-wide.** +A connection is one AWS identity the customer created for it. Two connections +in the same company, even to the same AWS account, have separate credentials, +and deleting a connection revokes only its own. + +A connection proves who it is in one of two ways. The customer picks one at +setup. + +**Method 1 (recommended): AWS role, with no stored keys.** This uses OIDC +federation, the same mechanism GitHub Actions, GitLab and Vercel use to reach +AWS. + +- *Once per company:* Paperclip publishes the company's issuer + `https:///oidc/companies/`. Its discovery document is + at `/.well-known/openid-configuration` and its public key set at + `/jwks.json` (see + [the issuer design](./2026-11-17-workload-identity-issuer.md)). The private + signing key stays in the instance's database, encrypted with the vault + master key. It is separate from sign-in keys and is used only for these + tokens. +- *Once per connection (customer, via CloudFormation):* + - Register the company's issuer URL as an IAM OIDC identity provider. + This is skipped if the account already has it. + - Create a role whose trust policy allows + `sts:AssumeRoleWithWebIdentity` only when the token's `sub` equals this + connection (for example `connection:`) and `aud` equals + `sts.amazonaws.com`. +- *At call time:* + 1. Paperclip signs a 5-minute JWT: issuer is the company's issuer URL, + `sub` is the connection, `aud` is `sts.amazonaws.com`. + 2. Paperclip calls `AssumeRoleWithWebIdentity`. That call needs no AWS + credentials at all; the signed JWT is the proof. + 3. AWS checks the signature against the public keys at the company's + issuer, then returns one-hour role credentials. +- *Requirements:* + - The instance does not have to run on AWS. It only has to serve those two + URLs over HTTPS with a publicly trusted certificate, reachable from the + internet. + - Paperclip Cloud instances qualify. Local, private-network or Tailscale-only + instances don't, and use method 2. + - The issuer URL must stay stable. Changing an instance's domain breaks + existing trusts until the customer updates the identity provider. +- *What a compromise means:* an attacker holding a company's signing key + could act as any of that company's connections, but only toward roles that + already trust that company's issuer and that connection. Keys rotate by publishing + the new key next to the old one. + +**Method 2: access key.** The customer creates a dedicated IAM user for this +connection, optionally with a role to assume, and pastes the key pair. It is +stored as this connection's secret, like any API-key connector, and never +leaves the control plane. This works on any instance, including local ones. + +Either way, identity belongs to the connection. Nothing is shared across the +instance. + +Minting then works as follows: + +1. Get role credentials: + - Method 1 calls `AssumeRoleWithWebIdentity` with the signed JWT. + - Method 2 with a role ARN calls `AssumeRole` with the stored key. + + Either call sets: + - Session name: `paperclip-`, so CloudTrail shows which + connection acted. + - Duration: one hour. + - Read-scope tokens add the AWS managed `ReadOnlyAccess` policy as a + session policy (`PolicyArns`). The session can then do at most what the + role allows and what read-only allows. + - With no role ARN, the dedicated IAM user's key is used directly. A + read-only scope still applies, via a `GetFederationToken` session policy. +2. Sign-in `CreateOAuth2TokenWithIAM`: + - Grant type is `client_credentials`. + - Resource is `aws-mcp.amazonaws.com`. + - The region is the endpoint region chosen at setup. + - It returns a bearer token valid for about one hour, with no refresh token. +3. The token is used as `Authorization: Bearer …` to + `https://aws-mcp..api.aws/mcp`. + +### How agents call AWS (no AWS CLI anywhere) + +Agents never receive the token. They see AWS tools the way they see any +connector's tools, through the control-plane MCP endpoint they're already +connected to: + +1. The agent calls the `aws___call_aws` tool with the operation it wants + (an AWS CLI-style command and its parameters), or calls `aws___run_script` + with a short Python script that chains several calls. +2. The gateway on the control plane checks policy, attaches the connection's + bearer token, and forwards the call to AWS's hosted MCP server. +3. **AWS runs the operation on its side.** That includes `run_script`, which + runs in AWS's own sandbox. The result returns through the gateway to the + agent. + +So the AWS CLI isn't installed on the instance or in any sandbox. All the +control plane needs is the two token calls from the AWS JavaScript SDK +(`@aws-sdk/client-sts`, `@aws-sdk/client-signin`). The only path that would +need the CLI is option D, which is a later opt-in. + +### 3. Governance + +- Tools are discovered with a minted token and published to the catalog as + for any remote MCP connector. +- Reviewed risk: + - `aws___call_aws` and `aws___run_script` are writes. + - Documentation, skill, region and task tools are reads. + - `aws___get_presigned_url` is a write, because it can grant upload access. +- **Read-only agents keep `aws___call_aws`, called with a read-scope token.** + AWS then refuses anything that isn't read-only, which beats hiding the + main tool. Writes need an agent granted write scope, plus the normal approval + policy. +- Every call goes through the gateway: activity log, timeouts, redaction. + CloudTrail on the customer side shows the session name, so customers can see + which Paperclip connection acted. + +### 4. Setup experience + +1. **Connect AWS** asks for one choice: **Role** (recommended, offered only + when AWS can reach the instance over public HTTPS) or **Access key**. + - The **Role** choice opens a one-click CloudFormation link, filled in + with this company's issuer URL and this connection. It creates the identity + provider (if missing) and a role trusted only by this connection, with + `AWSMCPSignInOAuthAccessPolicy` (which allows the token exchange) and + the permissions the customer picks (read-only to start, or their own + policy). + - The **Access key** choice's template creates a dedicated IAM user with + the same policies. +2. The customer pastes the role ARN (or the key pair) and picks an endpoint + region from the 8 + AWS publishes (us-east-1, us-west-2, eu-west-1, eu-west-2, eu-central-1, + ap-southeast-1, ap-southeast-2, ap-northeast-1) and a default region for + operations. +3. **Test** mints a token and lists tools, then reports the AWS account and role + in plain words. +4. Agents get AWS tools through the normal agent or company install. Read + scope is the default. + +An optional second method, "Sign in with AWS as yourself" (a personal grant), +may need no code: the hosted server supports standard MCP OAuth with dynamic +client registration. Verify it in the spike. + +## Phases (one PR each) + +| # | Deliverable | Proof | +|---|---|---| +| 0 | Spike with a test AWS account (`~/.secrets`): assume a role, exchange for a token, list and call tools from Node. Check that a `ReadOnlyAccess` session policy carries into the token (through `AssumeRoleWithWebIdentity`, `AssumeRole` and `GetFederationToken`); that AWS accepts a JWT issued by a tunnelled local issuer URL, what the call inputs of `aws___call_aws` look like, whether region metadata can be sent, latency of the mint and the first call, and that the DCR OAuth method works. | Findings recorded in this doc | +| 1 | Minted-credential source in the gateway: minter interface, cache, single retry on 401, health reasons. | Red/green tests: cache hit and expiry, a 401 leads to one re-mint, mint failure sets health, nothing persisted or logged | +| 1b | Per-company workload-identity issuer (`/oidc/companies/`): signing key stored in the database, encrypted with the vault master key; discovery and JWKS routes, key rotation, JWT signing for a connection. Built in [the issuer PR](./2026-11-17-workload-identity-issuer.md). Built generically, because GCP and Azure accept the same kind of federation. | Red/green tests; AWS accepts a token from a staging instance | +| 2 | AWS minter (role via OIDC, and access key), app definition, setup screen, CloudFormation template, reviewed tools, and `doc/connections/AWS.md` following the connector playbook. | Real-provider proof: a read-only agent's write is refused by AWS; a write agent's write succeeds after approval | +| 3 | Paperclip Cloud staging verification. | An AWS task completed on a staging stack | +| Later | Option D (sandbox credentials for Terraform/CDK). Further minters as connectors need them, such as GCP service-account impersonation or Azure client credentials. Each needs its own check of what the provider supports. | Separate plans | + +## Success criteria + +- An agent completes real AWS work end to end, locally and on staging. Its + permissions come only from the customer's role, narrowed to read-only by + default. +- No AWS credential or token reaches the agent's sandbox, the database or the + logs. +- No process is added and no Python is added to the image. +- After a deploy, the next AWS call works with no Retry. +- The mint adds a measured, cached cost: at most once an hour per connection + and scope. + +## Open decisions for Dotta + +1. **Decided: identity is per connection.** Role via OIDC is the default on + public HTTPS instances, and an access key is the fallback everywhere. + **Decided: one issuer per company** + (`https:///oidc/companies/`), so a customer's identity + provider trusts only their own company, even on a shared instance. +2. **Read-only by AWS session policy** (recommended), rather than hiding the + write tools. It needs `PolicyArns` support carried into the token, which the + spike verifies. +3. **Offer option D later** (credentials in the sandbox for infrastructure + agents), or never. + +## Deferred: running connector MCP servers on the control plane + +AWS no longer needs this. Other connectors will, whenever a provider ships +only a local server (Google Sheets is the current user). Findings to carry into +that plan: + +- Today the gateway starts a new process **for each tool call** and kills it + afterwards + ([tool-gateway.ts:5517](../../server/src/services/tool-gateway.ts)). +- Runtime "slots" are bookkeeping only (`process.simulated: true`); they never + own a process. +- Local processes are refused on public authenticated deployments unless + `PAPERCLIP_TRUSTED_MCP_RUNTIME_HOST` is set. +- The shape we sketched: + - A lease per run (start on first call, stop at run end, idle or max age). + - Pinned packages prefetched at connect time. + - Minted credentials passed as environment variables, using the same minter + interface as above. + - A temporary home directory per lease. + - Real process ownership in the supervisor. + - Reviewed-only packages on Cloud. +- The trigger to start that plan is the first catalog connector whose provider + has no hosted endpoint we can reach. diff --git a/doc/plans/2026-11-17-workload-identity-issuer.md b/doc/plans/2026-11-17-workload-identity-issuer.md new file mode 100644 index 00000000000..78b5be37f96 --- /dev/null +++ b/doc/plans/2026-11-17-workload-identity-issuer.md @@ -0,0 +1,197 @@ +# Workload identity issuer: public signing keys for connections + +Status: decided and built, 2026-11-17. Dotta chose one issuer per company and +a new table. The API is in [docs/api/workload-identity.md](../../docs/api/workload-identity.md). +Part of +[the AWS connector plan](./2026-11-17-aws-connector-and-minted-credentials.md) +(phase 1b). + +## The message + +Paperclip needs a signing key whose public half it publishes, so an outside +cloud (AWS first, later GCP or Azure) can verify "this request comes from +connection X of company Y on this Paperclip" without Paperclip storing any +cloud keys. + +None of our existing keys can do this safely, so we add one new kind of key. +It follows the pattern agent identity keys already use: + +- created lazily the first time it's needed, +- private half encrypted with the existing vault master key, +- stored in the database. + +No instance has to run a migration script. The only migration is the normal +schema migration that ships with the release and creates an empty table. + +## Existing keys, and why none can be reused + +| Key | Where | Algorithm | Why not | +|---|---|---|---| +| Agent identity keys | `agent_identity_keys`, [agent-identity.ts](../../server/src/services/agent-identity.ts) | Ed25519, one per agent | AWS accepts only RS256/384/512 and ES256/384/512, not Ed25519. **The private key is also handed to the agent's run** (that's why the run-output redactor exists), so an agent could mint AWS access itself and skip gateway policy. Disqualifying. | +| Agent run JWTs | [agent-auth-jwt.ts](../../server/src/agent-auth-jwt.ts) | HS256, derived from `PAPERCLIP_AGENT_JWT_SECRET` | Symmetric: publishing the verification key would publish the ability to sign. | +| Machine sign-in keys (MCP client credentials) | vault, per connection, `generateClientKeyPair` | RS256 / ES256 | Right algorithm, wrong purpose: each authenticates one connection to one provider's token endpoint. Using one key for two purposes invites token confusion. **The code is reusable**: key generation, JWK export, key IDs. | +| Cloud runtime identity, Cloud connector keys | Cloud Harness | Ed25519 | Held by Paperclip Cloud, not the instance, and absent on self-hosted. | +| Vault master key | `PAPERCLIP_SECRETS_MASTER_KEY` or key file | 32-byte symmetric | Not publishable. **It is the root that protects the new key.** Cloud already keeps it stable per stack across release rolls. | + +## Design + +### 1. One issuer per company (recommended) + +Issuer URL: `https:///oidc/companies/` + +AWS allows a path in the issuer URL. A per-company issuer, rather than one +for the whole instance, matters for two reasons: + +- **Misconfiguration blast radius.** The classic OIDC mistake (well known from + GitHub Actions) is a role trust policy that checks the issuer but forgets to + check `sub`. With a per-instance issuer, that mistake would let *any + company* on a shared instance assume the customer's role. With a + per-company issuer, the worst case is another connection in the customer's + own company. +- **It fits the company-scoping rule.** Keys are company-scoped entities, so + no new instance-wide exception is needed. + +**This choice is effectively permanent.** Changing the issuer later breaks +every customer trust, so it has to be decided before the first release. + +### 2. Keys + +- **Algorithm: ES256 (P-256).** AWS, GCP and Azure accept it, and signing is + fast and the keys are small. RS256 is the fallback if a later provider needs + it; the table records the algorithm per key. +- **New table `company_workload_identity_keys`.** Fields: `company_id`, + `key_id` (SHA-256 fingerprint, the same scheme agent keys use), `algorithm`, + `public_jwk`, `private_key_material` (encrypted by `localEncryptedProvider`), + `status` (`next`, `active`, `retiring` or `retired`), `created_at`, + `status_changed_at`. Partial unique indexes allow one `active` key and one + `next` key per company. A `retiring` key stays published for 24 hours after + it stops signing. This is computed when keys are read, so no background job + is needed. +- **Lazy creation.** The first time a connection in the company needs a token + (in practice, at AWS role setup), the server creates the key with + insert-on-conflict-do-nothing, then reads it back. Two processes racing end + up with the same key. Companies that never use federation never get a key. +- **No backfill and no script.** A new release only adds the empty table. +- **In memory:** the decrypted private key is cached per process. Signing a + JWT takes well under a millisecond. + +### 3. Public endpoints + +| Route | Returns | +|---|---| +| `GET /oidc/companies/:companyId/.well-known/openid-configuration` | `issuer`, `jwks_uri`, `id_token_signing_alg_values_supported: ["ES256"]`, `response_types_supported: ["id_token"]`, `subject_types_supported: ["public"]`, `claims_supported` | +| `GET /oidc/companies/:companyId/jwks.json` | Public keys with status `active` or `next` | + +- Both routes are unauthenticated, because AWS must fetch them anonymously. +- Both return **404 until the company has a key**, so they reveal nothing + about companies that don't use federation. Company IDs are random UUIDs. +- `Cache-Control: public, max-age=300`. +- **Enforced by a test:** the JWKS never contains private parameters (`d`, + and so on). AWS rejects the whole key set if one appears and treats the key + as compromised. +- Only served when the instance has a public HTTPS origin (Cloud, or + `authenticated` + `public` with `PAPERCLIP_PUBLIC_URL`). Elsewhere the setup + screen offers only the access-key method. + +### 4. Tokens + +```json +{ + "iss": "https:///oidc/companies/", + "sub": "connection:", + "aud": "sts.amazonaws.com", + "iat": 0, "nbf": 0, "exp": 300, "jti": "" +} +``` + +- **Lifetime: 5 minutes.** Each token is used once, immediately, for + `AssumeRoleWithWebIdentity`. +- **Only the credential minter in the gateway can request a token.** There is + no API that signs arbitrary claims, and agents never see these tokens. +- `sub` and `aud` come from the connection and its method, never from caller + input. +- Each issuance writes a tool-access audit event (connection and key ID, never + the token). +- **The issuer is fixed per connection.** It is stored when the connection is + set up. If the instance's public origin later changes, the connection shows + a health warning ("This instance's address changed. Update the identity + provider in AWS.") instead of silently signing for a new issuer. + +### 5. Rotation + +- **Planned rotation.** Create a `next` key, published alongside `active`. + After at least 24 hours (AWS caches key sets and doesn't publish how long), + switch signing to it, keep the old key published for another 24 hours, then + retire it. v1 offers this as an admin action; automatic yearly rotation can + come later. +- **Emergency rotation** (key believed exposed): retire immediately and + publish only the new key. Calls fail for at most the AWS cache window, and + the connection health message says so. Per AWS's guidance, the customer can + also revoke active role sessions. +- **Master key rotation is not supported yet.** No step re-encrypts vault + material when the master key changes, and that includes these rows and + `agent_identity_keys`. Replacing the master key makes the stored signing + keys unreadable, so signing fails closed. Operators must keep the original + master key; Cloud already keeps it stable per stack across release rolls. + Future work: a master-key re-encryption step that covers these rows. It + would keep the signing key itself, so customer trust would be unaffected. +- AWS allows at most 100 EC keys per key set. One active key plus one next key + per company is far below that. + +### 6. Threats and limits + +- **Holding the database plus the master key** lets someone sign as any + company on that instance, toward roles that already trust that company's + issuer and connection subject. That's the same exposure as today's vault + secrets, including stored AWS access keys under method 2, so it adds no new + class of risk. +- **Worktrees and forks** often copy the master key. Worktree seeding leaves + out `company_workload_identity_keys` in both the minimal and full modes, as + it already does for `agent_identity_keys`, so a development copy never holds + production's signing key. As a further guard, the signer refuses to sign + when the instance's current public origin doesn't match the issuer recorded + on the connection. +- **Not considered and rejected: deriving the key from the master key** (no + table at all). It would remove the schema migration, but then **anyone with + the master key alone could sign as production**. Master keys are copied into + worktree environments by design, and the agent JWT code carries an incident + note about exactly this. A rotated master key would also silently change the + signing key and break every customer's trust. + +### 7. Reuse beyond AWS + +The issuer is provider-neutral. Only `aud` and the minter differ: + +- GCP Workload Identity Federation accepts OIDC tokens from external issuers. +- Azure accepts federated credentials from external OIDC issuers. + +Each needs its own check before we claim support. + +## Delivery (one PR) + +1. Schema: `company_workload_identity_keys`, generated through the normal + `pnpm db:generate` flow. +2. Service: `ensureCompanySigningKey`, `signConnectionToken`, rotation + operations and the in-memory cache. This reuses the JWK and key-ID code + from `generateClientKeyPair`. +3. Routes: discovery and JWKS (public), plus admin rotate actions (board + only, written to the activity log). +4. Red/green tests: + - 404 before a key exists. + - Racing lazy creation produces one key. + - The JWKS has no private parameters. + - A token verifies against the published key. + - Signing is refused when the origin doesn't match. + - Rotation publishes two keys, then one. + - No token appears in audit records or logs. +5. Real proof: from a staging instance, AWS + `AssumeRoleWithWebIdentity` succeeds for a role that trusts the company + issuer and connection subject. It fails for a different connection's + subject and for another company's issuer. + +## Decisions (made 2026-11-17) + +1. **One issuer per company**, at `https:///oidc/companies/`. +2. **A new table through the normal release migration**, not a key derived + from the master key. A derived key would let the master key alone sign as + production. diff --git a/docs/api/overview.md b/docs/api/overview.md index f67e629812e..03ee8e978c6 100644 --- a/docs/api/overview.md +++ b/docs/api/overview.md @@ -67,3 +67,7 @@ No rate limiting is enforced in local deployments. Production deployments may ad [Tailscale connections](tailscale.md) let instance administrators connect a company tailnet and select its machines in SSH environments. + +[Workload identity](workload-identity.md) publishes the public keys that let +outside clouds, such as AWS, verify a company's connections without stored +cloud keys. diff --git a/docs/api/workload-identity.md b/docs/api/workload-identity.md new file mode 100644 index 00000000000..b975ed4cf33 --- /dev/null +++ b/docs/api/workload-identity.md @@ -0,0 +1,54 @@ +--- +title: Workload Identity +summary: Public signing keys that let outside clouds verify a company's connections +--- + +Each company can sign short-lived OIDC tokens that name one of its +connections. An outside cloud, such as AWS through `AssumeRoleWithWebIdentity`, +verifies them against the company's published public keys, so Paperclip never +stores that cloud's keys. Only the control plane signs. Agents never receive +these tokens. + +## Public endpoints + +These endpoints need no authentication, because the outside cloud fetches them +anonymously. They return only public keys. + +| Method | Path | Result | +|--------|------|--------| +| GET | `/oidc/companies/{companyId}/.well-known/openid-configuration` | OIDC discovery document | +| GET | `/oidc/companies/{companyId}/jwks.json` | Public key set (`ES256`, P-256) | + +Both return `404` until the company has a signing key. The discovery document +is also `404` unless the instance has a public HTTPS address with no port that +isn't a loopback host, because an outside cloud can't use any other issuer. +Responses are cacheable for five minutes. + +The issuer is `https://{instance}/oidc/companies/{companyId}`. Tokens have +`sub` set to `connection:{connectionId}`, an `aud` chosen by the connection +type, and a lifetime of five minutes. Only an enabled, active connection can get a token, +so disabling a connection stops new tokens at once. + +## Board endpoints + +These require board access to the company. Agent credentials are rejected. + +| Method | Path | Result | +|--------|------|--------| +| GET | `/api/companies/{companyId}/workload-identity` | Issuer (or why it is unavailable) and the published keys with their states | +| POST | `/api/companies/{companyId}/workload-identity/rotation` | Rotation step result and new state | + +The rotation body is `{ "step": "begin" | "complete" | "emergency" }`: + +- `begin` publishes a `next` key next to the active one. Repeating it while a + rotation is pending changes nothing and writes no log entry. It answers `409` + (`workload_identity_no_key`) when the company has no key yet. +- `complete` makes the `next` key sign once it has been published for 24 hours, + and keeps the old key published for another 24 hours. Before that it + answers `409` (`workload_identity_rotation_not_ready`) with `readyAt`. +- `emergency` stops publishing every current key and signs with a fresh one + immediately. Use it when a key may be exposed. + +Each step writes its activity log entry in the same transaction as the key +change. Responses never include private key +material. diff --git a/packages/db/src/migrations/0498_quiet_hardball.sql b/packages/db/src/migrations/0498_quiet_hardball.sql new file mode 100644 index 00000000000..50f349180d6 --- /dev/null +++ b/packages/db/src/migrations/0498_quiet_hardball.sql @@ -0,0 +1,18 @@ +CREATE TABLE "company_workload_identity_keys" ( + "id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL, + "company_id" uuid NOT NULL, + "key_id" text NOT NULL, + "algorithm" text NOT NULL, + "public_jwk" jsonb NOT NULL, + "private_key_material" jsonb NOT NULL, + "status" text NOT NULL, + "created_at" timestamp with time zone DEFAULT now() NOT NULL, + "status_changed_at" timestamp with time zone DEFAULT now() NOT NULL, + CONSTRAINT "company_workload_identity_keys_algorithm_check" CHECK ("company_workload_identity_keys"."algorithm" = 'ES256'), + CONSTRAINT "company_workload_identity_keys_status_check" CHECK ("company_workload_identity_keys"."status" in ('next', 'active', 'retiring', 'retired')) +); +--> statement-breakpoint +ALTER TABLE "company_workload_identity_keys" ADD CONSTRAINT "company_workload_identity_keys_company_id_companies_id_fk" FOREIGN KEY ("company_id") REFERENCES "public"."companies"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint +CREATE UNIQUE INDEX "company_workload_identity_keys_company_key_uq" ON "company_workload_identity_keys" USING btree ("company_id","key_id");--> statement-breakpoint +CREATE UNIQUE INDEX "company_workload_identity_keys_one_active_uq" ON "company_workload_identity_keys" USING btree ("company_id") WHERE "company_workload_identity_keys"."status" = 'active';--> statement-breakpoint +CREATE UNIQUE INDEX "company_workload_identity_keys_one_next_uq" ON "company_workload_identity_keys" USING btree ("company_id") WHERE "company_workload_identity_keys"."status" = 'next'; \ No newline at end of file diff --git a/packages/db/src/migrations/meta/0488_snapshot.json b/packages/db/src/migrations/meta/0498_snapshot.json similarity index 96% rename from packages/db/src/migrations/meta/0488_snapshot.json rename to packages/db/src/migrations/meta/0498_snapshot.json index bc96eb0f6a1..a17f7c4b16f 100644 --- a/packages/db/src/migrations/meta/0488_snapshot.json +++ b/packages/db/src/migrations/meta/0498_snapshot.json @@ -1,6 +1,6 @@ { - "id": "8f935a52-5f1e-4afe-8d13-c1770ca33968", - "prevId": "73c8fa97-70fc-49c0-804e-e49cfdffd156", + "id": "00aaeb32-9423-4cdd-ae8d-dc721ac83170", + "prevId": "3ea0a4c2-54f0-43fd-8c27-bcde98e3dd82", "version": "7", "dialect": "postgresql", "tables": { @@ -428,6 +428,12 @@ "primaryKey": false, "notNull": false }, + "responsibility_summary_invocation_id": { + "name": "responsibility_summary_invocation_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, "agent_id": { "name": "agent_id", "type": "uuid", @@ -537,6 +543,27 @@ "method": "btree", "with": {} }, + "budget_reservations_responsibility_summary_idx": { + "name": "budget_reservations_responsibility_summary_idx", + "columns": [ + { + "expression": "company_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "responsibility_summary_invocation_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, "budget_reservations_run_idx": { "name": "budget_reservations_run_idx", "columns": [ @@ -634,7 +661,7 @@ "checkConstraints": { "budget_reservations_source_check": { "name": "budget_reservations_source_check", - "value": "(\"budget_reservations\".\"run_id\" is not null)::int + (\"budget_reservations\".\"decision_invocation_id\" is not null)::int + (\"budget_reservations\".\"fast_response_request_id\" is not null)::int + (\"budget_reservations\".\"prompt_injection_invocation_id\" is not null)::int = 1" + "value": "(\"budget_reservations\".\"run_id\" is not null)::int + (\"budget_reservations\".\"decision_invocation_id\" is not null)::int + (\"budget_reservations\".\"fast_response_request_id\" is not null)::int + (\"budget_reservations\".\"prompt_injection_invocation_id\" is not null)::int + (\"budget_reservations\".\"responsibility_summary_invocation_id\" is not null)::int = 1" }, "budget_reservations_state_check": { "name": "budget_reservations_state_check", @@ -21928,6 +21955,153 @@ "checkConstraints": {}, "isRLSEnabled": false }, + "public.company_workload_identity_keys": { + "name": "company_workload_identity_keys", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "key_id": { + "name": "key_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "algorithm": { + "name": "algorithm", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "public_jwk": { + "name": "public_jwk", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "private_key_material": { + "name": "private_key_material", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "status_changed_at": { + "name": "status_changed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "company_workload_identity_keys_company_key_uq": { + "name": "company_workload_identity_keys_company_key_uq", + "columns": [ + { + "expression": "company_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "key_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "company_workload_identity_keys_one_active_uq": { + "name": "company_workload_identity_keys_one_active_uq", + "columns": [ + { + "expression": "company_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"company_workload_identity_keys\".\"status\" = 'active'", + "concurrently": false, + "method": "btree", + "with": {} + }, + "company_workload_identity_keys_one_next_uq": { + "name": "company_workload_identity_keys_one_next_uq", + "columns": [ + { + "expression": "company_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"company_workload_identity_keys\".\"status\" = 'next'", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "company_workload_identity_keys_company_id_companies_id_fk": { + "name": "company_workload_identity_keys_company_id_companies_id_fk", + "tableFrom": "company_workload_identity_keys", + "tableTo": "companies", + "columnsFrom": [ + "company_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "company_workload_identity_keys_algorithm_check": { + "name": "company_workload_identity_keys_algorithm_check", + "value": "\"company_workload_identity_keys\".\"algorithm\" = 'ES256'" + }, + "company_workload_identity_keys_status_check": { + "name": "company_workload_identity_keys_status_check", + "value": "\"company_workload_identity_keys\".\"status\" in ('next', 'active', 'retiring', 'retired')" + } + }, + "isRLSEnabled": false + }, "public.completion_contracts": { "name": "completion_contracts", "schema": "", @@ -23092,7 +23266,7 @@ "checkConstraints": { "cost_events_usage_kind_check": { "name": "cost_events_usage_kind_check", - "value": "\"cost_events\".\"usage_kind\" in ('decision', 'fast_response', 'prompt_injection') or (\"cost_events\".\"usage_kind\" = 'agent' and \"cost_events\".\"agent_id\" is not null)" + "value": "\"cost_events\".\"usage_kind\" in ('decision', 'fast_response', 'prompt_injection', 'responsibility_summary') or (\"cost_events\".\"usage_kind\" = 'agent' and \"cost_events\".\"agent_id\" is not null)" }, "cost_events_nonnegative_amounts": { "name": "cost_events_nonnegative_amounts", @@ -45081,6 +45255,123 @@ "checkConstraints": {}, "isRLSEnabled": false }, + "public.link_previews": { + "name": "link_previews", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "viewer_id": { + "name": "viewer_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "channel": { + "name": "channel", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "resource_kind": { + "name": "resource_kind", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "canonical_url": { + "name": "canonical_url", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "checked_at": { + "name": "checked_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "expires_at": { + "name": "expires_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + } + }, + "indexes": { + "link_previews_scope_url_uq": { + "name": "link_previews_scope_url_uq", + "columns": [ + { + "expression": "company_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "viewer_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "canonical_url", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "link_previews_company_id_companies_id_fk": { + "name": "link_previews_company_id_companies_id_fk", + "tableFrom": "link_previews", + "tableTo": "companies", + "columnsFrom": [ + "company_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, "public.managed_agent_profiles": { "name": "managed_agent_profiles", "schema": "", @@ -52200,6 +52491,1380 @@ }, "isRLSEnabled": false }, + "public.responsibilities": { + "name": "responsibilities", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "owner_user_id": { + "name": "owner_user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "area_id": { + "name": "area_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "title": { + "name": "title", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "standard": { + "name": "standard", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "metric_kind": { + "name": "metric_kind", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "metric_name": { + "name": "metric_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "metric_unit": { + "name": "metric_unit", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "metric_target": { + "name": "metric_target", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "metric_low": { + "name": "metric_low", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "metric_high": { + "name": "metric_high", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "metric_current": { + "name": "metric_current", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "metric_direction": { + "name": "metric_direction", + "type": "text", + "primaryKey": false, + "notNull": false, + "default": "'at_least'" + }, + "metric_updated_at": { + "name": "metric_updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "run_by_agent_id": { + "name": "run_by_agent_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'proposed'" + }, + "source": { + "name": "source", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'manual'" + }, + "evidence": { + "name": "evidence", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "health_override": { + "name": "health_override", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "health_note": { + "name": "health_note", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sort_order": { + "name": "sort_order", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "proposed_by_agent_id": { + "name": "proposed_by_agent_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "confirmed_at": { + "name": "confirmed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "responsibilities_company_owner_status_idx": { + "name": "responsibilities_company_owner_status_idx", + "columns": [ + { + "expression": "company_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "owner_user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "responsibilities_area_idx": { + "name": "responsibilities_area_idx", + "columns": [ + { + "expression": "area_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "responsibilities_company_id_companies_id_fk": { + "name": "responsibilities_company_id_companies_id_fk", + "tableFrom": "responsibilities", + "tableTo": "companies", + "columnsFrom": [ + "company_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "responsibilities_area_id_responsibility_areas_id_fk": { + "name": "responsibilities_area_id_responsibility_areas_id_fk", + "tableFrom": "responsibilities", + "tableTo": "responsibility_areas", + "columnsFrom": [ + "area_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + }, + "responsibilities_run_by_agent_id_agents_id_fk": { + "name": "responsibilities_run_by_agent_id_agents_id_fk", + "tableFrom": "responsibilities", + "tableTo": "agents", + "columnsFrom": [ + "run_by_agent_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + }, + "responsibilities_proposed_by_agent_id_agents_id_fk": { + "name": "responsibilities_proposed_by_agent_id_agents_id_fk", + "tableFrom": "responsibilities", + "tableTo": "agents", + "columnsFrom": [ + "proposed_by_agent_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "responsibilities_status_check": { + "name": "responsibilities_status_check", + "value": "\"responsibilities\".\"status\" in ('proposed', 'active', 'let_go', 'dismissed')" + }, + "responsibilities_source_check": { + "name": "responsibilities_source_check", + "value": "\"responsibilities\".\"source\" in ('manual', 'onboarding')" + }, + "responsibilities_health_override_check": { + "name": "responsibilities_health_override_check", + "value": "\"responsibilities\".\"health_override\" is null or \"responsibilities\".\"health_override\" in ('behind', 'at_risk', 'on_track')" + }, + "responsibilities_metric_direction_check": { + "name": "responsibilities_metric_direction_check", + "value": "\"responsibilities\".\"metric_direction\" is null or \"responsibilities\".\"metric_direction\" in ('at_least', 'at_most')" + } + }, + "isRLSEnabled": false + }, + "public.responsibility_areas": { + "name": "responsibility_areas", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "title": { + "name": "title", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'active'" + }, + "source": { + "name": "source", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'manual'" + }, + "proposed_by_agent_id": { + "name": "proposed_by_agent_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "sort_order": { + "name": "sort_order", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "responsibility_areas_company_status_idx": { + "name": "responsibility_areas_company_status_idx", + "columns": [ + { + "expression": "company_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "responsibility_areas_company_title_uq": { + "name": "responsibility_areas_company_title_uq", + "columns": [ + { + "expression": "company_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "lower(\"title\")", + "asc": true, + "isExpression": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"responsibility_areas\".\"status\" <> 'archived'", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "responsibility_areas_company_id_companies_id_fk": { + "name": "responsibility_areas_company_id_companies_id_fk", + "tableFrom": "responsibility_areas", + "tableTo": "companies", + "columnsFrom": [ + "company_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "responsibility_areas_proposed_by_agent_id_agents_id_fk": { + "name": "responsibility_areas_proposed_by_agent_id_agents_id_fk", + "tableFrom": "responsibility_areas", + "tableTo": "agents", + "columnsFrom": [ + "proposed_by_agent_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "responsibility_areas_status_check": { + "name": "responsibility_areas_status_check", + "value": "\"responsibility_areas\".\"status\" in ('proposed', 'active', 'archived')" + }, + "responsibility_areas_source_check": { + "name": "responsibility_areas_source_check", + "value": "\"responsibility_areas\".\"source\" in ('manual', 'onboarding')" + } + }, + "isRLSEnabled": false + }, + "public.responsibility_dashboard_versions": { + "name": "responsibility_dashboard_versions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "owner_user_id": { + "name": "owner_user_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "cause": { + "name": "cause", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "summary": { + "name": "summary", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "snapshot": { + "name": "snapshot", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "snapshot_hash": { + "name": "snapshot_hash", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "actor_user_id": { + "name": "actor_user_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "actor_agent_id": { + "name": "actor_agent_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "issue_id": { + "name": "issue_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "interaction_id": { + "name": "interaction_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "previous_version_id": { + "name": "previous_version_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "restored_from_version_id": { + "name": "restored_from_version_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "responsibility_dashboard_versions_owner_created_idx": { + "name": "responsibility_dashboard_versions_owner_created_idx", + "columns": [ + { + "expression": "company_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "owner_user_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "responsibility_dashboard_versions_company_id_companies_id_fk": { + "name": "responsibility_dashboard_versions_company_id_companies_id_fk", + "tableFrom": "responsibility_dashboard_versions", + "tableTo": "companies", + "columnsFrom": [ + "company_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "responsibility_dashboard_versions_actor_agent_id_agents_id_fk": { + "name": "responsibility_dashboard_versions_actor_agent_id_agents_id_fk", + "tableFrom": "responsibility_dashboard_versions", + "tableTo": "agents", + "columnsFrom": [ + "actor_agent_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + }, + "responsibility_dashboard_versions_issue_id_issues_id_fk": { + "name": "responsibility_dashboard_versions_issue_id_issues_id_fk", + "tableFrom": "responsibility_dashboard_versions", + "tableTo": "issues", + "columnsFrom": [ + "issue_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + }, + "responsibility_dashboard_versions_previous_version_id_responsibility_dashboard_versions_id_fk": { + "name": "responsibility_dashboard_versions_previous_version_id_responsibility_dashboard_versions_id_fk", + "tableFrom": "responsibility_dashboard_versions", + "tableTo": "responsibility_dashboard_versions", + "columnsFrom": [ + "previous_version_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + }, + "responsibility_dashboard_versions_restored_from_version_id_responsibility_dashboard_versions_id_fk": { + "name": "responsibility_dashboard_versions_restored_from_version_id_responsibility_dashboard_versions_id_fk", + "tableFrom": "responsibility_dashboard_versions", + "tableTo": "responsibility_dashboard_versions", + "columnsFrom": [ + "restored_from_version_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "responsibility_dashboard_versions_cause_check": { + "name": "responsibility_dashboard_versions_cause_check", + "value": "\"responsibility_dashboard_versions\".\"cause\" in ('change_set', 'person', 'restore')" + } + }, + "isRLSEnabled": false + }, + "public.responsibility_links": { + "name": "responsibility_links", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "responsibility_id": { + "name": "responsibility_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "project_id": { + "name": "project_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "issue_id": { + "name": "issue_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "routine_id": { + "name": "routine_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "responsibility_links_responsibility_idx": { + "name": "responsibility_links_responsibility_idx", + "columns": [ + { + "expression": "responsibility_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "responsibility_links_responsibility_project_uq": { + "name": "responsibility_links_responsibility_project_uq", + "columns": [ + { + "expression": "responsibility_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "project_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"responsibility_links\".\"project_id\" is not null", + "concurrently": false, + "method": "btree", + "with": {} + }, + "responsibility_links_responsibility_issue_uq": { + "name": "responsibility_links_responsibility_issue_uq", + "columns": [ + { + "expression": "responsibility_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "issue_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"responsibility_links\".\"issue_id\" is not null", + "concurrently": false, + "method": "btree", + "with": {} + }, + "responsibility_links_responsibility_routine_uq": { + "name": "responsibility_links_responsibility_routine_uq", + "columns": [ + { + "expression": "responsibility_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "routine_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"responsibility_links\".\"routine_id\" is not null", + "concurrently": false, + "method": "btree", + "with": {} + }, + "responsibility_links_project_idx": { + "name": "responsibility_links_project_idx", + "columns": [ + { + "expression": "project_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "responsibility_links_issue_idx": { + "name": "responsibility_links_issue_idx", + "columns": [ + { + "expression": "issue_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "responsibility_links_routine_idx": { + "name": "responsibility_links_routine_idx", + "columns": [ + { + "expression": "routine_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "responsibility_links_company_id_companies_id_fk": { + "name": "responsibility_links_company_id_companies_id_fk", + "tableFrom": "responsibility_links", + "tableTo": "companies", + "columnsFrom": [ + "company_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "responsibility_links_responsibility_id_responsibilities_id_fk": { + "name": "responsibility_links_responsibility_id_responsibilities_id_fk", + "tableFrom": "responsibility_links", + "tableTo": "responsibilities", + "columnsFrom": [ + "responsibility_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "responsibility_links_project_id_projects_id_fk": { + "name": "responsibility_links_project_id_projects_id_fk", + "tableFrom": "responsibility_links", + "tableTo": "projects", + "columnsFrom": [ + "project_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "responsibility_links_issue_id_issues_id_fk": { + "name": "responsibility_links_issue_id_issues_id_fk", + "tableFrom": "responsibility_links", + "tableTo": "issues", + "columnsFrom": [ + "issue_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "responsibility_links_routine_id_routines_id_fk": { + "name": "responsibility_links_routine_id_routines_id_fk", + "tableFrom": "responsibility_links", + "tableTo": "routines", + "columnsFrom": [ + "routine_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "responsibility_links_single_target_check": { + "name": "responsibility_links_single_target_check", + "value": "(\"responsibility_links\".\"project_id\" is not null)::int + (\"responsibility_links\".\"issue_id\" is not null)::int + (\"responsibility_links\".\"routine_id\" is not null)::int = 1" + } + }, + "isRLSEnabled": false + }, + "public.responsibility_metric_readings": { + "name": "responsibility_metric_readings", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "metric_id": { + "name": "metric_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "value": { + "name": "value", + "type": "numeric", + "primaryKey": false, + "notNull": true + }, + "recorded_at": { + "name": "recorded_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "recorded_by_agent_id": { + "name": "recorded_by_agent_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "recorded_by_user_id": { + "name": "recorded_by_user_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "note": { + "name": "note", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "responsibility_metric_readings_metric_recorded_idx": { + "name": "responsibility_metric_readings_metric_recorded_idx", + "columns": [ + { + "expression": "metric_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "recorded_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "responsibility_metric_readings_company_id_companies_id_fk": { + "name": "responsibility_metric_readings_company_id_companies_id_fk", + "tableFrom": "responsibility_metric_readings", + "tableTo": "companies", + "columnsFrom": [ + "company_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "responsibility_metric_readings_metric_id_responsibility_metrics_id_fk": { + "name": "responsibility_metric_readings_metric_id_responsibility_metrics_id_fk", + "tableFrom": "responsibility_metric_readings", + "tableTo": "responsibility_metrics", + "columnsFrom": [ + "metric_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "responsibility_metric_readings_recorded_by_agent_id_agents_id_fk": { + "name": "responsibility_metric_readings_recorded_by_agent_id_agents_id_fk", + "tableFrom": "responsibility_metric_readings", + "tableTo": "agents", + "columnsFrom": [ + "recorded_by_agent_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.responsibility_metrics": { + "name": "responsibility_metrics", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "responsibility_id": { + "name": "responsibility_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "position": { + "name": "position", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "title": { + "name": "title", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "display": { + "name": "display", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "unit": { + "name": "unit", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "target": { + "name": "target", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "direction": { + "name": "direction", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'active'" + }, + "created_by_agent_id": { + "name": "created_by_agent_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "created_by_user_id": { + "name": "created_by_user_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "responsibility_metrics_responsibility_status_position_idx": { + "name": "responsibility_metrics_responsibility_status_position_idx", + "columns": [ + { + "expression": "responsibility_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "position", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "responsibility_metrics_company_id_companies_id_fk": { + "name": "responsibility_metrics_company_id_companies_id_fk", + "tableFrom": "responsibility_metrics", + "tableTo": "companies", + "columnsFrom": [ + "company_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "responsibility_metrics_responsibility_id_responsibilities_id_fk": { + "name": "responsibility_metrics_responsibility_id_responsibilities_id_fk", + "tableFrom": "responsibility_metrics", + "tableTo": "responsibilities", + "columnsFrom": [ + "responsibility_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "responsibility_metrics_created_by_agent_id_agents_id_fk": { + "name": "responsibility_metrics_created_by_agent_id_agents_id_fk", + "tableFrom": "responsibility_metrics", + "tableTo": "agents", + "columnsFrom": [ + "created_by_agent_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "responsibility_metrics_display_check": { + "name": "responsibility_metrics_display_check", + "value": "\"responsibility_metrics\".\"display\" in ('number', 'percent', 'line', 'bar')" + }, + "responsibility_metrics_direction_check": { + "name": "responsibility_metrics_direction_check", + "value": "\"responsibility_metrics\".\"direction\" is null or \"responsibility_metrics\".\"direction\" in ('at_least', 'at_most')" + }, + "responsibility_metrics_status_check": { + "name": "responsibility_metrics_status_check", + "value": "\"responsibility_metrics\".\"status\" in ('active', 'archived')" + } + }, + "isRLSEnabled": false + }, + "public.responsibility_status_reports": { + "name": "responsibility_status_reports", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "responsibility_id": { + "name": "responsibility_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "agent_id": { + "name": "agent_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "user_id": { + "name": "user_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "message": { + "name": "message", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "detail": { + "name": "detail", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "responsibility_status_reports_responsibility_created_idx": { + "name": "responsibility_status_reports_responsibility_created_idx", + "columns": [ + { + "expression": "responsibility_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": false, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "responsibility_status_reports_company_id_companies_id_fk": { + "name": "responsibility_status_reports_company_id_companies_id_fk", + "tableFrom": "responsibility_status_reports", + "tableTo": "companies", + "columnsFrom": [ + "company_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "responsibility_status_reports_responsibility_id_responsibilities_id_fk": { + "name": "responsibility_status_reports_responsibility_id_responsibilities_id_fk", + "tableFrom": "responsibility_status_reports", + "tableTo": "responsibilities", + "columnsFrom": [ + "responsibility_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "responsibility_status_reports_agent_id_agents_id_fk": { + "name": "responsibility_status_reports_agent_id_agents_id_fk", + "tableFrom": "responsibility_status_reports", + "tableTo": "agents", + "columnsFrom": [ + "agent_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "responsibility_status_reports_status_check": { + "name": "responsibility_status_reports_status_check", + "value": "\"responsibility_status_reports\".\"status\" in ('behind', 'at_risk', 'on_track', 'unknown')" + }, + "responsibility_status_reports_author_check": { + "name": "responsibility_status_reports_author_check", + "value": "not (\"responsibility_status_reports\".\"agent_id\" is not null and \"responsibility_status_reports\".\"user_id\" is not null)" + } + }, + "isRLSEnabled": false + }, "public.routine_documents": { "name": "routine_documents", "schema": "", @@ -58584,6 +60249,197 @@ "checkConstraints": {}, "isRLSEnabled": false }, + "public.tool_connection_content_deletions": { + "name": "tool_connection_content_deletions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "connection_id": { + "name": "connection_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "requested_by_actor_type": { + "name": "requested_by_actor_type", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "requested_by_actor_id": { + "name": "requested_by_actor_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "requested_at": { + "name": "requested_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "progress": { + "name": "progress", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "attempts": { + "name": "attempts", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "last_error": { + "name": "last_error", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "tool_connection_content_deletions_connection_idx": { + "name": "tool_connection_content_deletions_connection_idx", + "columns": [ + { + "expression": "company_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "connection_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "tool_connection_content_deletions_pending_uq": { + "name": "tool_connection_content_deletions_pending_uq", + "columns": [ + { + "expression": "company_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "connection_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"tool_connection_content_deletions\".\"status\" = 'pending'", + "concurrently": false, + "method": "btree", + "with": {} + }, + "tool_connection_content_deletions_pending_idx": { + "name": "tool_connection_content_deletions_pending_idx", + "columns": [ + { + "expression": "requested_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"tool_connection_content_deletions\".\"status\" = 'pending'", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "tool_connection_content_deletions_company_id_companies_id_fk": { + "name": "tool_connection_content_deletions_company_id_companies_id_fk", + "tableFrom": "tool_connection_content_deletions", + "tableTo": "companies", + "columnsFrom": [ + "company_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "tool_connection_content_deletions_connection_id_tool_connections_id_fk": { + "name": "tool_connection_content_deletions_connection_id_tool_connections_id_fk", + "tableFrom": "tool_connection_content_deletions", + "tableTo": "tool_connections", + "columnsFrom": [ + "connection_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, "public.tool_connection_installs": { "name": "tool_connection_installs", "schema": "", @@ -59109,7 +60965,7 @@ }, "tool_connections_transport_check": { "name": "tool_connections_transport_check", - "value": "\"tool_connections\".\"transport\" in ('mcp_remote', 'rest_api', 'local_stdio', 'chat_sdk', 'voice', 'runtime_auth')" + "value": "\"tool_connections\".\"transport\" in ('mcp_remote', 'rest_api', 'local_stdio', 'chat_sdk', 'voice', 'runtime_auth', 'plugin')" }, "tool_connections_purpose_check": { "name": "tool_connections_purpose_check", @@ -69800,6 +71656,355 @@ } }, "isRLSEnabled": false + }, + "public.responsibility_summary_invocations": { + "name": "responsibility_summary_invocations", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "feature": { + "name": "feature", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "responsibility_id": { + "name": "responsibility_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "owner_user_id": { + "name": "owner_user_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "actor_type": { + "name": "actor_type", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "actor_id": { + "name": "actor_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "responsible_user_id": { + "name": "responsible_user_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "connection_id": { + "name": "connection_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "grant_id": { + "name": "grant_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "provider": { + "name": "provider", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "model": { + "name": "model", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "prompt_version": { + "name": "prompt_version", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'running'" + }, + "error_code": { + "name": "error_code", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "provider_request_id": { + "name": "provider_request_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "cost_event_id": { + "name": "cost_event_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "started_at": { + "name": "started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "finished_at": { + "name": "finished_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "duration_ms": { + "name": "duration_ms", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "input_tokens": { + "name": "input_tokens", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "output_tokens": { + "name": "output_tokens", + "type": "integer", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "responsibility_summary_invocations_company_started_idx": { + "name": "responsibility_summary_invocations_company_started_idx", + "columns": [ + { + "expression": "company_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "started_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "responsibility_summary_invocations_pending_idx": { + "name": "responsibility_summary_invocations_pending_idx", + "columns": [ + { + "expression": "started_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"responsibility_summary_invocations\".\"status\" = 'running'", + "concurrently": false, + "method": "btree", + "with": {} + }, + "responsibility_summary_invocations_cost_idx": { + "name": "responsibility_summary_invocations_cost_idx", + "columns": [ + { + "expression": "cost_event_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "responsibility_summary_invocations_company_id_companies_id_fk": { + "name": "responsibility_summary_invocations_company_id_companies_id_fk", + "tableFrom": "responsibility_summary_invocations", + "tableTo": "companies", + "columnsFrom": [ + "company_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "responsibility_summary_invocations_cost_event_id_cost_events_id_fk": { + "name": "responsibility_summary_invocations_cost_event_id_cost_events_id_fk", + "tableFrom": "responsibility_summary_invocations", + "tableTo": "cost_events", + "columnsFrom": [ + "cost_event_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "responsibility_summary_invocations_feature_check": { + "name": "responsibility_summary_invocations_feature_check", + "value": "\"responsibility_summary_invocations\".\"feature\" in ('where_things_stand', 'what_is_affecting_the_number')" + }, + "responsibility_summary_invocations_status_check": { + "name": "responsibility_summary_invocations_status_check", + "value": "\"responsibility_summary_invocations\".\"status\" in ('running', 'succeeded', 'failed', 'unknown')" + } + }, + "isRLSEnabled": false + }, + "public.plugin_connector_legacy_access": { + "name": "plugin_connector_legacy_access", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "company_id": { + "name": "company_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "plugin_key": { + "name": "plugin_key", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "connection_id": { + "name": "connection_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "claimed_at": { + "name": "claimed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "plugin_connector_legacy_access_company_plugin_uq": { + "name": "plugin_connector_legacy_access_company_plugin_uq", + "columns": [ + { + "expression": "company_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "plugin_key", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "plugin_connector_legacy_access_company_id_companies_id_fk": { + "name": "plugin_connector_legacy_access_company_id_companies_id_fk", + "tableFrom": "plugin_connector_legacy_access", + "tableTo": "companies", + "columnsFrom": [ + "company_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "plugin_connector_legacy_access_connection_id_tool_connections_id_fk": { + "name": "plugin_connector_legacy_access_connection_id_tool_connections_id_fk", + "tableFrom": "plugin_connector_legacy_access", + "tableTo": "tool_connections", + "columnsFrom": [ + "connection_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false } }, "enums": {}, diff --git a/packages/db/src/migrations/meta/_journal.json b/packages/db/src/migrations/meta/_journal.json index 92a1ad48fe3..d7377d29718 100644 --- a/packages/db/src/migrations/meta/_journal.json +++ b/packages/db/src/migrations/meta/_journal.json @@ -3473,6 +3473,13 @@ "when": 1794945756009, "tag": "0497_purple_lady_bullseye", "breakpoints": true + }, + { + "idx": 498, + "version": "7", + "when": 1794977210244, + "tag": "0498_quiet_hardball", + "breakpoints": true } ] } \ No newline at end of file diff --git a/packages/db/src/schema/company_workload_identity_keys.ts b/packages/db/src/schema/company_workload_identity_keys.ts new file mode 100644 index 00000000000..34914ec3e17 --- /dev/null +++ b/packages/db/src/schema/company_workload_identity_keys.ts @@ -0,0 +1,34 @@ +import { sql } from "drizzle-orm"; +import { check, jsonb, pgTable, text, timestamp, uniqueIndex, uuid } from "drizzle-orm/pg-core"; +import { companies } from "./companies.js"; + +export type CompanyWorkloadIdentityKeyStatus = "next" | "active" | "retiring" | "retired"; + +/** + * Signing keys a company's connections use to prove who they are to outside + * clouds (OIDC workload identity federation). The public half is published; + * the private half is encrypted with the vault master key and never leaves the + * control plane. Created lazily, one active key per company. + */ +export const companyWorkloadIdentityKeys = pgTable("company_workload_identity_keys", { + id: uuid("id").primaryKey().defaultRandom(), + companyId: uuid("company_id").notNull().references(() => companies.id, { onDelete: "cascade" }), + keyId: text("key_id").notNull(), + algorithm: text("algorithm").$type<"ES256">().notNull(), + publicJwk: jsonb("public_jwk").$type>().notNull(), + privateKeyMaterial: jsonb("private_key_material").$type>().notNull(), + status: text("status").$type().notNull(), + createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(), + statusChangedAt: timestamp("status_changed_at", { withTimezone: true }).notNull().defaultNow(), +}, (table) => ({ + companyKeyUq: uniqueIndex("company_workload_identity_keys_company_key_uq").on(table.companyId, table.keyId), + oneActiveUq: uniqueIndex("company_workload_identity_keys_one_active_uq") + .on(table.companyId).where(sql`${table.status} = 'active'`), + oneNextUq: uniqueIndex("company_workload_identity_keys_one_next_uq") + .on(table.companyId).where(sql`${table.status} = 'next'`), + algorithmCheck: check("company_workload_identity_keys_algorithm_check", sql`${table.algorithm} = 'ES256'`), + statusCheck: check( + "company_workload_identity_keys_status_check", + sql`${table.status} in ('next', 'active', 'retiring', 'retired')`, + ), +})); diff --git a/packages/db/src/schema/index.ts b/packages/db/src/schema/index.ts index 5bf2493f5a9..c09b5f48721 100644 --- a/packages/db/src/schema/index.ts +++ b/packages/db/src/schema/index.ts @@ -268,6 +268,7 @@ export { agentCommentary } from "./agent_commentary.js"; export { agentIdentityKeys } from "./agent_identity_keys.js"; +export { companyWorkloadIdentityKeys, type CompanyWorkloadIdentityKeyStatus } from "./company_workload_identity_keys.js"; export { chatVoiceReports } from "./chat_voice_reports.js"; diff --git a/server/src/__tests__/openapi-routes.test.ts b/server/src/__tests__/openapi-routes.test.ts index 9fcf86f4df9..3bb57cb75b4 100644 --- a/server/src/__tests__/openapi-routes.test.ts +++ b/server/src/__tests__/openapi-routes.test.ts @@ -118,6 +118,7 @@ const apiPrefixes: Record = { "tool-gateway.ts": "/api", "user-profiles.ts": "/api", "workspace-preferences.ts": "/api", + "workload-identity.ts": "/api", }; const ROUTE_LITERAL_PATTERN = @@ -213,6 +214,7 @@ function normalizeExpressPath(routePath: string) { function resolveMountedPath(file: string, prefix: string, routePath: string) { if (file === "connection-event-wake.ts") return routePath; + if (file === "workload-identity.ts" && routePath.startsWith("/oidc/")) return routePath; if (file === "public-mcp.ts" && (routePath.startsWith("/mcp/oauth/") || routePath.startsWith("/.well-known/"))) return routePath; if ( (file === "chat-channels.ts" || file === "email.ts") && diff --git a/server/src/__tests__/workload-identity.test.ts b/server/src/__tests__/workload-identity.test.ts new file mode 100644 index 00000000000..baa94d00e9c --- /dev/null +++ b/server/src/__tests__/workload-identity.test.ts @@ -0,0 +1,356 @@ +import { createPublicKey, randomUUID, verify } from "node:crypto"; +import { mkdtemp, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import express from "express"; +import request from "supertest"; +import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from "vitest"; +import { and, eq } from "drizzle-orm"; +import { + activityLog, + closeRegisteredClients, + companies, + companyWorkloadIdentityKeys, + createDb, + toolAccessAuditEvents, + toolApplications, + toolConnections, + type Db, +} from "@paperclipai/db"; +import { generateClientKeyPair } from "../services/tool-access/oauth/client-credentials.js"; +import { localEncryptedProvider } from "../secrets/local-encrypted-provider.js"; +import { startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js"; +import { resolveExternalTestDatabaseUrl } from "./helpers/external-test-database.js"; +import { createApp } from "../app.js"; +import { createLocalDiskStorageProvider } from "../storage/local-disk-provider.js"; +import { createStorageService } from "../storage/service.js"; +import { errorHandler } from "../middleware/error-handler.js"; +import { workloadIdentityPublicRoutes, workloadIdentityRoutes } from "../routes/workload-identity.js"; +import { + resetWorkloadIdentityKeyCacheForTests, + setWorkloadIdentityConfiguredPublicOrigin, + WORKLOAD_IDENTITY_PUBLISH_BEFORE_USE_MS, + WORKLOAD_IDENTITY_RETIRING_GRACE_MS, + workloadIdentityIssuerOrigin, + workloadIdentityService, +} from "../services/workload-identity.js"; + +const ORIGIN = "https://acme.paperclip.app"; +const actor = { actorType: "user" as const, actorId: "board-user" }; +const externalDatabase = resolveExternalTestDatabaseUrl("PAPERCLIP_WORKLOAD_IDENTITY_TEST_DATABASE_URL"); + +function decode(token: string) { + const [header, claims, signature] = token.split("."); + return { + header: JSON.parse(Buffer.from(header!, "base64url").toString("utf8")), + claims: JSON.parse(Buffer.from(claims!, "base64url").toString("utf8")), + signingInput: `${header}.${claims}`, + signature: Buffer.from(signature!, "base64url"), + }; +} + +/** What AWS does: find the key by `kid` in the published set and check the ES256 signature. */ +function verifiesAgainst(token: string, keys: Array>): boolean { + const { header, signingInput, signature } = decode(token); + const jwk = keys.find((key) => key.kid === header.kid); + if (!jwk) return false; + return verify("sha256", Buffer.from(signingInput), + { key: createPublicKey({ key: jwk as never, format: "jwk" }), dsaEncoding: "ieee-p1363" }, signature); +} + +describe("company workload identity issuer", () => { + let db: Db; + let database: { connectionString: string; cleanup: () => Promise }; + let directory: string; + let clock: Date; + const service = () => workloadIdentityService(db, { now: () => clock }); + + async function company() { + const id = randomUUID(); + await db.insert(companies).values({ + id, name: `Identity ${id.slice(0, 6)}`, issuePrefix: `W${id.slice(0, 4).toUpperCase()}`, + defaultResponsibleUserId: "workload-identity-owner", + }); + return id; + } + + async function connection(companyId: string, state: { status?: "draft" | "active" | "disabled"; enabled?: boolean } = {}) { + const [application] = await db.insert(toolApplications).values({ + companyId, applicationKey: `aws-${randomUUID()}`, name: `AWS ${randomUUID().slice(0, 8)}`, type: "mcp_http", status: "active", + }).returning(); + const [row] = await db.insert(toolConnections).values({ + companyId, applicationId: application!.id, name: `AWS ${randomUUID().slice(0, 6)}`, + uid: `aws-${randomUUID()}`, connectionKind: "managed", ownership: "customer", + transport: "mcp_remote", authKind: "oauth", credentialPolicy: "shared", + status: state.status ?? "active", enabled: state.enabled ?? true, + }).returning(); + return row!.id; + } + + beforeAll(async () => { + directory = await mkdtemp(join(tmpdir(), "paperclip-workload-identity-")); + vi.stubEnv("PAPERCLIP_HOME", directory); + vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY", ""); + vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY_FILE", join(directory, "master.key")); + vi.stubEnv("PAPERCLIP_PUBLIC_URL", ORIGIN); + database = externalDatabase + ? { connectionString: externalDatabase, cleanup: () => closeRegisteredClients(externalDatabase) } + : await startEmbeddedPostgresTestDatabase("paperclip-workload-identity-"); + db = createDb(database.connectionString); + }, 120_000); + + afterAll(async () => { + await database?.cleanup(); + vi.unstubAllEnvs(); + await rm(directory, { recursive: true, force: true }); + }); + + beforeEach(() => { + clock = new Date("2026-11-17T12:00:00.000Z"); + resetWorkloadIdentityKeyCacheForTests(); + }); + + it("only offers an issuer on a public https address AWS can reach", () => { + expect(workloadIdentityIssuerOrigin(ORIGIN)).toEqual({ available: true, origin: ORIGIN }); + expect(workloadIdentityIssuerOrigin(null)).toEqual({ available: false, reason: "no_public_url" }); + expect(workloadIdentityIssuerOrigin("http://acme.example.com")).toEqual({ available: false, reason: "not_https" }); + expect(workloadIdentityIssuerOrigin("https://acme.example.com:8443")).toEqual({ available: false, reason: "has_port" }); + expect(workloadIdentityIssuerOrigin("https://localhost")).toEqual({ available: false, reason: "loopback" }); + expect(workloadIdentityIssuerOrigin("https://127.0.0.1")).toEqual({ available: false, reason: "loopback" }); + }); + + it("falls back to the configured auth address when no public URL variable is set", () => { + vi.stubEnv("PAPERCLIP_PUBLIC_URL", ""); + vi.stubEnv("PAPERCLIP_AUTH_PUBLIC_BASE_URL", ""); + vi.stubEnv("PAPERCLIP_API_URL", ""); + try { + setWorkloadIdentityConfiguredPublicOrigin(null); + expect(workloadIdentityIssuerOrigin()).toEqual({ available: false, reason: "no_public_url" }); + setWorkloadIdentityConfiguredPublicOrigin("https://paperclip.example.com/"); + expect(workloadIdentityIssuerOrigin()).toEqual({ available: true, origin: "https://paperclip.example.com" }); + vi.stubEnv("PAPERCLIP_PUBLIC_URL", ORIGIN); + expect(workloadIdentityIssuerOrigin()).toEqual({ available: true, origin: ORIGIN }); + } finally { + setWorkloadIdentityConfiguredPublicOrigin(null); + vi.stubEnv("PAPERCLIP_PUBLIC_URL", ORIGIN); + } + }); + + it("publishes nothing until a company uses it, then converges on one key", async () => { + const companyId = await company(); + const app = express().use(workloadIdentityPublicRoutes(db)); + await request(app).get(`/oidc/companies/${companyId}/jwks.json`).expect(404); + await request(app).get(`/oidc/companies/${companyId}/.well-known/openid-configuration`).expect(404); + await request(app).get("/oidc/companies/not-a-uuid/jwks.json").expect(404); + expect(await db.select().from(companyWorkloadIdentityKeys) + .where(eq(companyWorkloadIdentityKeys.companyId, companyId))).toHaveLength(0); + + const keys = await Promise.all(Array.from({ length: 6 }, () => service().ensureActiveKey(companyId))); + expect(new Set(keys.map((key) => key.keyId)).size).toBe(1); + + const discovery = await request(app).get(`/oidc/companies/${companyId}/.well-known/openid-configuration`).expect(200); + const issuer = `${ORIGIN}/oidc/companies/${companyId}`; + expect(discovery.body).toMatchObject({ + issuer, jwks_uri: `${issuer}/jwks.json`, id_token_signing_alg_values_supported: ["ES256"], + response_types_supported: ["id_token"], subject_types_supported: ["public"], + }); + expect(discovery.headers["cache-control"]).toBe("public, max-age=300"); + const jwks = await request(app).get(`/oidc/companies/${companyId}/jwks.json`).expect(200); + expect(jwks.body.keys).toHaveLength(1); + }); + + it("never publishes private key members (AWS rejects the set and the key counts as exposed)", async () => { + const companyId = await company(); + await service().ensureActiveKey(companyId); + await service().beginRotation(companyId, actor); + const keySet = await service().publicKeySet(companyId); + expect(keySet!.keys).toHaveLength(2); + for (const key of keySet!.keys) { + expect(Object.keys(key).sort()).toEqual(["alg", "crv", "kid", "kty", "use", "x", "y"]); + expect(key).toMatchObject({ kty: "EC", crv: "P-256", alg: "ES256", use: "sig" }); + } + const [stored] = await db.select().from(companyWorkloadIdentityKeys) + .where(eq(companyWorkloadIdentityKeys.companyId, companyId)); + expect(JSON.stringify(stored!.publicJwk)).not.toMatch(/"d"/); + expect(JSON.stringify(stored!.privateKeyMaterial)).not.toMatch(/BEGIN PRIVATE KEY/); + }); + + it("signs a 5-minute token naming one connection that verifies against the published key", async () => { + const companyId = await company(); + const connectionId = await connection(companyId); + const signed = await service().signConnectionToken({ companyId, connectionId, audience: "sts.amazonaws.com" }); + const { header, claims } = decode(signed.token); + expect(header).toEqual({ alg: "ES256", typ: "JWT", kid: signed.keyId }); + expect(claims).toMatchObject({ + iss: `${ORIGIN}/oidc/companies/${companyId}`, + sub: `connection:${connectionId}`, + aud: "sts.amazonaws.com", + }); + expect(claims.exp - claims.iat).toBe(300); + expect(claims.nbf).toBe(claims.iat); + expect(verifiesAgainst(signed.token, (await service().publicKeySet(companyId))!.keys)).toBe(true); + + const [audit] = await db.select().from(toolAccessAuditEvents).where(and( + eq(toolAccessAuditEvents.companyId, companyId), eq(toolAccessAuditEvents.action, "workload_identity.token_issued"), + )); + expect(audit).toMatchObject({ connectionId, outcome: "success" }); + expect(JSON.stringify(audit)).not.toContain(signed.token.split(".")[2]!); + }); + + it("keeps companies and connections apart", async () => { + const companyA = await company(); + const companyB = await company(); + const connectionB = await connection(companyB); + await expect(service().signConnectionToken({ companyId: companyA, connectionId: connectionB, audience: "sts.amazonaws.com" })) + .rejects.toMatchObject({ status: 404 }); + const tokenB = await service().signConnectionToken({ companyId: companyB, connectionId: connectionB, audience: "sts.amazonaws.com" }); + await service().ensureActiveKey(companyA); + expect(verifiesAgainst(tokenB.token, (await service().publicKeySet(companyA))!.keys)).toBe(false); + }); + + it("refuses to sign without a usable public address or after the address changed", async () => { + const companyId = await company(); + const connectionId = await connection(companyId); + await expect(service().signConnectionToken({ + companyId, connectionId, audience: "sts.amazonaws.com", origin: workloadIdentityIssuerOrigin("http://localhost:3100"), + })).rejects.toMatchObject({ status: 422, details: { code: "workload_identity_unavailable", reason: "not_https" } }); + await expect(service().signConnectionToken({ + companyId, connectionId, audience: "sts.amazonaws.com", expectedIssuer: `https://old.example.com/oidc/companies/${companyId}`, + })).rejects.toMatchObject({ status: 409, details: { code: "workload_identity_issuer_changed" } }); + await expect(service().signConnectionToken({ companyId, connectionId, audience: "has space" })) + .rejects.toMatchObject({ status: 422 }); + }); + + it("rotates in two published steps, and an emergency rotation drops old keys at once", async () => { + const companyId = await company(); + const connectionId = await connection(companyId); + const original = await service().ensureActiveKey(companyId); + const begun = await service().beginRotation(companyId, actor); + expect(begun.created).toBe(true); + expect((await service().beginRotation(companyId, actor)).created).toBe(false); + await expect(service().completeRotation(companyId, actor)) + .rejects.toMatchObject({ status: 409, details: { code: "workload_identity_rotation_not_ready" } }); + expect((await service().signConnectionToken({ companyId, connectionId, audience: "sts.amazonaws.com" })).keyId) + .toBe(original.keyId); + + clock = new Date(clock.getTime() + WORKLOAD_IDENTITY_PUBLISH_BEFORE_USE_MS); + await service().completeRotation(companyId, actor); + const afterSwap = await service().signConnectionToken({ companyId, connectionId, audience: "sts.amazonaws.com" }); + expect(afterSwap.keyId).toBe(begun.keyId); + expect((await service().publicKeySet(companyId))!.keys.map((key) => key.kid).sort()) + .toEqual([original.keyId, begun.keyId].sort()); + + clock = new Date(clock.getTime() + WORKLOAD_IDENTITY_RETIRING_GRACE_MS); + expect((await service().publicKeySet(companyId))!.keys.map((key) => key.kid)).toEqual([begun.keyId]); + expect((await service().describe(companyId)).keys.map((key) => key.keyId)).toEqual([begun.keyId]); + await service().beginRotation(companyId, actor); + const [settled] = await db.select().from(companyWorkloadIdentityKeys).where(and( + eq(companyWorkloadIdentityKeys.companyId, companyId), eq(companyWorkloadIdentityKeys.keyId, original.keyId), + )); + expect(settled!.status).toBe("retired"); + + const emergency = await service().emergencyRotation(companyId, actor); + const published = (await service().publicKeySet(companyId))!.keys.map((key) => key.kid); + expect(published).toEqual([emergency.keyId]); + const next = await service().signConnectionToken({ companyId, connectionId, audience: "sts.amazonaws.com" }); + expect(next.keyId).toBe(emergency.keyId); + expect(verifiesAgainst(afterSwap.token, (await service().publicKeySet(companyId))!.keys)).toBe(false); + }); + + it("refuses to sign when the stored private key is not the published one", async () => { + const companyId = await company(); + const connectionId = await connection(companyId); + const key = await service().ensureActiveKey(companyId); + const other = await generateClientKeyPair("ES256"); + const { material } = await localEncryptedProvider.createSecret({ value: other.privateKeyPem }); + await db.update(companyWorkloadIdentityKeys).set({ privateKeyMaterial: material }) + .where(eq(companyWorkloadIdentityKeys.id, key.id)); + resetWorkloadIdentityKeyCacheForTests(); + await expect(service().signConnectionToken({ companyId, connectionId, audience: "sts.amazonaws.com" })) + .rejects.toThrow(/does not match its published key/); + expect(await db.select().from(toolAccessAuditEvents).where(and( + eq(toolAccessAuditEvents.companyId, companyId), eq(toolAccessAuditEvents.action, "workload_identity.token_issued"), + ))).toHaveLength(0); + }); + + it("stops issuing tokens for a disabled, draft or switched-off connection", async () => { + const companyId = await company(); + for (const state of [{ status: "disabled" as const }, { status: "draft" as const }, { enabled: false }]) { + const connectionId = await connection(companyId, state); + await expect(service().signConnectionToken({ companyId, connectionId, audience: "sts.amazonaws.com" })) + .rejects.toMatchObject({ status: 422, details: { code: "workload_identity_connection_inactive" } }); + } + }); + + it("writes a rotation's activity entry only when the key change commits", async () => { + const companyId = await company(); + await expect(service().beginRotation(companyId, actor)).rejects.toMatchObject({ status: 409 }); + const entries = () => db.select().from(activityLog).where(and( + eq(activityLog.companyId, companyId), eq(activityLog.action, "workload_identity.rotation_begin"), + )); + expect(await entries()).toHaveLength(0); + await service().ensureActiveKey(companyId); + await service().beginRotation(companyId, actor); + expect(await entries()).toHaveLength(1); + expect((await service().beginRotation(companyId, actor)).created).toBe(false); + expect(await entries()).toHaveLength(1); + }); + + it("answers the public paths anonymously in the assembled app", async () => { + const companyId = await company(); + await service().ensureActiveKey(companyId); + const app = await createApp(db, { + uiMode: "none", serverPort: 0, bindHost: "127.0.0.1", allowedHostnames: ["127.0.0.1"], + deploymentMode: "authenticated", deploymentExposure: "private", authReady: true, companyDeletionEnabled: false, + managedPluginAutoInstall: [], storageService: createStorageService(createLocalDiskStorageProvider(join(directory, "storage"))), + decisionServiceOptions: { wakeOriginAgent: async () => null }, + }); + await app.locals.bundledPluginsStartup; + const server = app.listen(0, "127.0.0.1"); + try { + await new Promise((resolve) => server.once("listening", resolve)); + const address = server.address(); + if (!address || typeof address === "string") throw new Error("No HTTP test address"); + const base = `http://127.0.0.1:${address.port}/oidc/companies/${companyId}`; + const discovery = await fetch(`${base}/.well-known/openid-configuration`); + expect(discovery.status).toBe(200); + expect((await discovery.json()).issuer).toBe(base.replace(`http://127.0.0.1:${address.port}`, ORIGIN)); + const jwks = await fetch(`${base}/jwks.json`); + expect(jwks.status).toBe(200); + expect((await jwks.json()).keys).toHaveLength(1); + expect((await fetch(`http://127.0.0.1:${address.port}/api/companies/${companyId}/workload-identity`)).status) + .not.toBe(200); + } finally { + await app.locals.paperclipShutdown?.(); + await new Promise((resolve, reject) => server.close((error) => error ? reject(error) : resolve())); + } + }, 120_000); + + it("limits status and rotation to the board and logs each rotation", async () => { + const companyId = await company(); + await service().ensureActiveKey(companyId); + const boardApp = express().use(express.json(), (req, _res, next) => { + req.actor = { + type: "board", userId: "board-user", source: "local_implicit", companyIds: [companyId], + memberships: [{ companyId, membershipRole: "owner", status: "active" }], isInstanceAdmin: true, + } as Express.Request["actor"]; + next(); + }).use("/api", workloadIdentityRoutes(db)).use(errorHandler); + const state = await request(boardApp).get(`/api/companies/${companyId}/workload-identity`).expect(200); + expect(state.body.issuer).toBe(`${ORIGIN}/oidc/companies/${companyId}`); + expect(JSON.stringify(state.body)).not.toMatch(/private|"d"/i); + await request(boardApp).post(`/api/companies/${companyId}/workload-identity/rotation`).send({ step: "nope" }).expect(400); + await request(boardApp).post(`/api/companies/${companyId}/workload-identity/rotation`).send({ step: "begin" }).expect(200); + const [entry] = await db.select().from(activityLog).where(and( + eq(activityLog.companyId, companyId), eq(activityLog.action, "workload_identity.rotation_begin"), + )); + expect(entry).toMatchObject({ actorType: "user", actorId: "board-user", entityType: "company" }); + + const agentApp = express().use(express.json(), (req, _res, next) => { + req.actor = { type: "agent", agentId: randomUUID(), companyId, source: "agent_key" } as Express.Request["actor"]; + next(); + }).use("/api", workloadIdentityRoutes(db)).use(errorHandler); + await request(agentApp).get(`/api/companies/${companyId}/workload-identity`).expect(403); + await request(agentApp).post(`/api/companies/${companyId}/workload-identity/rotation`).send({ step: "emergency" }).expect(403); + }); +}); diff --git a/server/src/app.ts b/server/src/app.ts index ff807278507..87591afcd81 100644 --- a/server/src/app.ts +++ b/server/src/app.ts @@ -139,6 +139,8 @@ import { routineRoutes } from "./routes/routines.js"; import { routineSuggestionRoutes } from "./routes/routine-suggestions.js"; import { environmentRoutes } from "./routes/environments.js"; import { tailscaleConnectionRoutes } from "./routes/tailscale-connections.js"; +import { workloadIdentityPublicRoutes, workloadIdentityRoutes } from "./routes/workload-identity.js"; +import { setWorkloadIdentityConfiguredPublicOrigin } from "./services/workload-identity.js"; import { executionWorkspaceRoutes } from "./routes/execution-workspaces.js"; import { goalRoutes } from "./routes/goals.js"; import { responsibilityRoutes } from "./routes/responsibilities.js"; @@ -618,6 +620,9 @@ export async function createApp( })); app.use(museTransportRoutes(db, opts.authPublicBaseUrl)); app.use(publicMcpIngress); + // Outside clouds fetch these anonymously to verify connection identity tokens. + setWorkloadIdentityConfiguredPublicOrigin(opts.authPublicBaseUrl); + app.use(workloadIdentityPublicRoutes(db)); // Connection-intent tools carry their own short-lived, run-bound bearer and // must be reachable by remote adapters that intentionally do not receive an // agent API key. Every request revalidates the active heartbeat row. @@ -866,6 +871,7 @@ export async function createApp( }) })); api.use(routineRoutes(db, { pluginWorkerManager: workerManager })); api.use(tailscaleConnectionRoutes(db)); + api.use(workloadIdentityRoutes(db)); api.use(routineSuggestionRoutes(db, { pluginWorkerManager: workerManager })); api.use( environmentRoutes(db, { diff --git a/server/src/routes/openapi.ts b/server/src/routes/openapi.ts index 4524123771e..190b466b397 100644 --- a/server/src/routes/openapi.ts +++ b/server/src/routes/openapi.ts @@ -1424,6 +1424,9 @@ const PUBLIC_OPERATIONS = new Set([ "POST /mcp/gateways/{gatewayPublicId}", "GET /api/tool-gateway/gateways/{gatewayId}/mcp", "POST /api/tool-gateway/gateways/{gatewayId}/mcp", + // Outside clouds fetch a company's workload identity issuer anonymously (routes/workload-identity.ts). + "GET /oidc/companies/{companyId}/.well-known/openid-configuration", + "GET /oidc/companies/{companyId}/jwks.json", ]); const BOARD_ONLY_PREFIXES = [ @@ -1467,6 +1470,9 @@ const mcpSkillsBoardOperations = [ ] as const; const BOARD_ONLY_OPERATIONS = new Set([ + // Workload identity status and key rotation are board-only (assertBoard in routes/workload-identity.ts). + "GET /api/companies/{companyId}/workload-identity", + "POST /api/companies/{companyId}/workload-identity/rotation", "POST /api/companies/{companyId}/chats/{agentRef}/onboarding", "POST /api/companies/{companyId}/chats/{agentRef}/onboarding/greeting", "PATCH /api/agents/{id}/permission-grants", @@ -9727,6 +9733,57 @@ registry.registerPath({ } }, ...tailscaleErrors }, }); +const workloadIdentityParams = z.object({ companyId: z.string().uuid() }); +const workloadIdentityPublicNotFound = { 404: { description: "The company has no published signing key, or this instance has no public HTTPS address" } }; +const workloadIdentityKeyState = z.object({ + issuer: z.string().url().nullable(), + unavailableReason: z.enum(["no_public_url", "not_https", "has_port", "loopback"]).nullable(), + keys: z.array(z.object({ + keyId: z.string(), status: z.enum(["next", "active", "retiring"]), + createdAt: z.string().datetime(), statusChangedAt: z.string().datetime(), + }).strict()), +}).strict(); + +registry.registerPath({ + method: "get", path: "/oidc/companies/{companyId}/.well-known/openid-configuration", tags: ["workload-identity"], + summary: "Company workload identity OIDC discovery document", + description: "Unauthenticated. Outside clouds read this to verify tokens that name one of the company's connections. Cacheable for five minutes.", + request: { params: workloadIdentityParams }, + responses: { 200: r.ok(z.object({ + issuer: z.string().url(), jwks_uri: z.string().url(), + response_types_supported: z.array(z.string()), subject_types_supported: z.array(z.string()), + id_token_signing_alg_values_supported: z.array(z.string()), claims_supported: z.array(z.string()), + }).strict()), ...workloadIdentityPublicNotFound }, +}); + +registry.registerPath({ + method: "get", path: "/oidc/companies/{companyId}/jwks.json", tags: ["workload-identity"], + summary: "Company workload identity public key set", + description: "Unauthenticated. Lists only public ES256 keys. Never contains private key members.", + request: { params: workloadIdentityParams }, + responses: { 200: r.ok(z.object({ keys: z.array(z.object({ + kty: z.literal("EC"), crv: z.string(), x: z.string(), y: z.string(), kid: z.string(), alg: z.literal("ES256"), use: z.literal("sig"), + }).strict()) }).strict()), ...workloadIdentityPublicNotFound }, +}); + +registry.registerPath({ + method: "get", path: "/api/companies/{companyId}/workload-identity", tags: ["workload-identity"], + summary: "Company workload identity issuer and key states", + description: "Board only. Returns the issuer (or why it is unavailable) and the published keys. No private key material.", + request: { params: workloadIdentityParams }, + responses: { 200: r.ok(workloadIdentityKeyState), 401: r.unauthorized, 403: r.forbidden, 404: r.notFound }, +}); + +registry.registerPath({ + method: "post", path: "/api/companies/{companyId}/workload-identity/rotation", tags: ["workload-identity"], + summary: "Rotate the company workload identity signing key", + description: "Board only. begin publishes a next key; complete makes it sign after 24 hours of publication; emergency replaces every key at once. Each step writes its activity entry in the same transaction as the key change.", + request: { params: workloadIdentityParams, body: jsonBody(z.object({ step: z.enum(["begin", "complete", "emergency"]) }).strict()) }, + responses: { 200: r.ok(z.object({ + step: z.enum(["begin", "complete", "emergency"]), keyId: z.string(), created: z.boolean().optional(), state: workloadIdentityKeyState, + }).strict()), 400: r.badRequest, 401: r.unauthorized, 403: r.forbidden, 404: r.notFound, 409: r.conflict }, +}); + // ─── Environments ───────────────────────────────────────────────────────────── registry.registerPath({ diff --git a/server/src/routes/workload-identity.ts b/server/src/routes/workload-identity.ts new file mode 100644 index 00000000000..062ac3f6db0 --- /dev/null +++ b/server/src/routes/workload-identity.ts @@ -0,0 +1,70 @@ +import { Router, type Response } from "express"; +import type { Db } from "@paperclipai/db"; +import { z } from "zod"; +import { validate } from "../middleware/validate.js"; +import { workloadIdentityService } from "../services/workload-identity.js"; +import { assertBoard, assertCompanyAccess, getActorInfo } from "./authz.js"; + +const companyIdSchema = z.string().uuid(); + +function notPublished(res: Response) { + res.setHeader("Cache-Control", "public, max-age=60"); + res.status(404).json({ error: "not_found" }); +} + +/** + * Public OIDC discovery and key set for a company's workload identity issuer. + * Outside clouds fetch these anonymously to verify tokens; they expose only + * public keys, and answer 404 until the company has created a key. + */ +export function workloadIdentityPublicRoutes(db: Db) { + const router = Router(); + const service = workloadIdentityService(db); + router.get("/oidc/companies/:companyId/.well-known/openid-configuration", async (req, res) => { + const companyId = companyIdSchema.safeParse(req.params.companyId); + if (!companyId.success) return notPublished(res); + const document = await service.discoveryDocument(companyId.data); + if (!document) return notPublished(res); + res.setHeader("Cache-Control", "public, max-age=300"); + res.json(document); + }); + router.get("/oidc/companies/:companyId/jwks.json", async (req, res) => { + const companyId = companyIdSchema.safeParse(req.params.companyId); + if (!companyId.success) return notPublished(res); + const keySet = await service.publicKeySet(companyId.data); + if (!keySet) return notPublished(res); + res.setHeader("Cache-Control", "public, max-age=300"); + res.json(keySet); + }); + return router; +} + +const rotationSchema = z.object({ step: z.enum(["begin", "complete", "emergency"]) }).strict(); + +/** Board-only status and key rotation for the company's issuer. */ +export function workloadIdentityRoutes(db: Db) { + const router = Router(); + const service = workloadIdentityService(db); + router.get("/companies/:companyId/workload-identity", async (req, res) => { + assertBoard(req); + const companyId = companyIdSchema.parse(req.params.companyId); + assertCompanyAccess(req, companyId); + res.json(await service.describe(companyId)); + }); + router.post("/companies/:companyId/workload-identity/rotation", validate(rotationSchema), async (req, res) => { + assertBoard(req); + const companyId = companyIdSchema.parse(req.params.companyId); + assertCompanyAccess(req, companyId); + const { step } = req.body as z.infer; + const { actorType, actorId } = getActorInfo(req); + // The service writes the activity entry in the same transaction as the key change. + const actor = { actorType, actorId }; + const result = step === "begin" + ? await service.beginRotation(companyId, actor) + : step === "complete" + ? await service.completeRotation(companyId, actor) + : await service.emergencyRotation(companyId, actor); + res.json({ step, ...result, state: await service.describe(companyId) }); + }); + return router; +} diff --git a/server/src/services/workload-identity.ts b/server/src/services/workload-identity.ts new file mode 100644 index 00000000000..74040949844 --- /dev/null +++ b/server/src/services/workload-identity.ts @@ -0,0 +1,384 @@ +import { createPrivateKey, createPublicKey, randomUUID, sign, type KeyObject } from "node:crypto"; +import { and, eq, gt, inArray, lte, or } from "drizzle-orm"; +import { + companies, + companyWorkloadIdentityKeys, + toolAccessAuditEvents, + toolConnections, + type Db, +} from "@paperclipai/db"; +import { conflict, notFound, unprocessable } from "../errors.js"; +import { persistActivity, publishActivity, type LogActivityInput } from "./activity-log.js"; +import { localEncryptedProvider } from "../secrets/local-encrypted-provider.js"; +import { runtimePublicOrigin } from "./cloud-runtime-identity.js"; +import { generateClientKeyPair, jwkThumbprint } from "./tool-access/oauth/client-credentials.js"; + +/** + * Workload identity issuer: each company signs short-lived OIDC tokens that + * name one of its connections, and publishes the public keys so an outside + * cloud (AWS, GCP, Azure) can verify them without Paperclip storing that + * cloud's keys. Only the control plane can sign; agents never see these tokens. + */ + +export const WORKLOAD_IDENTITY_ALGORITHM = "ES256"; +export const WORKLOAD_IDENTITY_TOKEN_LIFETIME_SECONDS = 300; +/** A `next` key is published this long before it may sign, so cached key sets pick it up. */ +export const WORKLOAD_IDENTITY_PUBLISH_BEFORE_USE_MS = 24 * 60 * 60 * 1000; +/** A replaced key stays published this long, so tokens it signed still verify. */ +export const WORKLOAD_IDENTITY_RETIRING_GRACE_MS = 24 * 60 * 60 * 1000; + +type KeyRow = typeof companyWorkloadIdentityKeys.$inferSelect; + +export type WorkloadIdentityIssuerOrigin = + | { available: true; origin: string } + | { available: false; reason: "no_public_url" | "not_https" | "has_port" | "loopback" }; + +let configuredPublicOrigin: string | null = null; + +/** + * The instance's configured public address (auth.publicBaseUrl, BETTER_AUTH_URL, + * BETTER_AUTH_BASE_URL), set once by createApp. Used after the live Cloud + * address and the PAPERCLIP_PUBLIC_URL family, which `runtimePublicOrigin` reads. + */ +export function setWorkloadIdentityConfiguredPublicOrigin(origin: string | null | undefined) { + configuredPublicOrigin = origin?.trim() || null; +} + +export function workloadIdentityPublicOrigin(): string | null { + return runtimePublicOrigin() ?? configuredPublicOrigin; +} + +/** AWS requires an https issuer with no port that it can reach from the internet. */ +export function workloadIdentityIssuerOrigin(origin: string | null = workloadIdentityPublicOrigin()): WorkloadIdentityIssuerOrigin { + if (!origin) return { available: false, reason: "no_public_url" }; + let url: URL; + try { + url = new URL(origin); + } catch { + return { available: false, reason: "no_public_url" }; + } + if (url.protocol !== "https:") return { available: false, reason: "not_https" }; + if (url.port !== "") return { available: false, reason: "has_port" }; + const host = url.hostname.toLowerCase(); + if (host === "localhost" || host.endsWith(".localhost") || host === "[::1]" || /^127\./.test(host)) { + return { available: false, reason: "loopback" }; + } + return { available: true, origin: url.origin }; +} + +export function workloadIdentityIssuer(origin: string, companyId: string): string { + return `${origin}/oidc/companies/${companyId}`; +} + +export function workloadIdentitySubject(connectionId: string): string { + return `connection:${connectionId}`; +} + +/** Only public EC members, so private material can never reach the published key set. */ +function publicJwkFor(row: Pick) { + const jwk = row.publicJwk; + return { + kty: "EC", + crv: jwk.crv, + x: jwk.x, + y: jwk.y, + kid: row.keyId, + alg: WORKLOAD_IDENTITY_ALGORITHM, + use: "sig", + }; +} + +/** Only each company's current signing key is held decrypted; a rotated-out key is dropped. */ +const signingKeyCache = new Map(); + +export function resetWorkloadIdentityKeyCacheForTests() { + signingKeyCache.clear(); +} + +function base64Url(value: Buffer | string): string { + return Buffer.from(value).toString("base64url"); +} + +export function workloadIdentityService(db: Db, options: { now?: () => Date } = {}) { + const now = options.now ?? (() => new Date()); + + async function newKeyValues(companyId: string, status: "next" | "active") { + const generated = await generateClientKeyPair(WORKLOAD_IDENTITY_ALGORITHM, now()); + const encrypted = await localEncryptedProvider.createSecret({ value: generated.privateKeyPem }); + const at = now(); + return { + companyId, + keyId: generated.publicKey.keyId, + algorithm: WORKLOAD_IDENTITY_ALGORITHM as "ES256", + publicJwk: generated.publicKey.publicJwk as unknown as Record, + privateKeyMaterial: encrypted.material, + status, + createdAt: at, + statusChangedAt: at, + }; + } + + async function assertCompany(companyId: string) { + const [company] = await db.select({ id: companies.id }).from(companies).where(eq(companies.id, companyId)); + if (!company) throw notFound("Company not found"); + } + + async function activeKey(companyId: string): Promise { + const [row] = await db.select().from(companyWorkloadIdentityKeys).where(and( + eq(companyWorkloadIdentityKeys.companyId, companyId), + eq(companyWorkloadIdentityKeys.status, "active"), + )); + return row; + } + + /** The company's signing key, created on first use. Concurrent callers converge on one key. */ + async function ensureActiveKey(companyId: string): Promise { + const existing = await activeKey(companyId); + if (existing) return existing; + await assertCompany(companyId); + await db.insert(companyWorkloadIdentityKeys).values(await newKeyValues(companyId, "active")).onConflictDoNothing(); + const created = await activeKey(companyId); + if (!created) throw new Error("Workload identity key could not be created"); + return created; + } + + async function privateKeyFor(row: KeyRow): Promise { + const cached = signingKeyCache.get(row.companyId); + if (cached?.keyId === row.keyId) return cached.privateKey; + const pem = await localEncryptedProvider.resolveVersion({ material: row.privateKeyMaterial, externalRef: null }); + const privateKey = createPrivateKey(pem); + // Fail closed on corruption or a wrong master key; never sign with a key that isn't the published one. + if (row.algorithm !== WORKLOAD_IDENTITY_ALGORITHM || privateKey.asymmetricKeyType !== "ec" + || jwkThumbprint(createPublicKey(privateKey).export({ format: "jwk" })) !== row.keyId) { + throw new Error("Workload identity private key does not match its published key"); + } + signingKeyCache.set(row.companyId, { keyId: row.keyId, privateKey }); + return privateKey; + } + + /** Published keys only, filtered in SQL so reads stay bounded however many rotations happened. */ + async function publishedKeyRows(companyId: string): Promise { + const graceStart = new Date(now().getTime() - WORKLOAD_IDENTITY_RETIRING_GRACE_MS); + return db.select().from(companyWorkloadIdentityKeys).where(and( + eq(companyWorkloadIdentityKeys.companyId, companyId), + or( + inArray(companyWorkloadIdentityKeys.status, ["next", "active"]), + and(eq(companyWorkloadIdentityKeys.status, "retiring"), + gt(companyWorkloadIdentityKeys.statusChangedAt, graceStart)), + ), + )); + } + + /** Retiring keys past their grace period become retired, so they leave every view. */ + async function settleRetiring(tx: Db, companyId: string, at: Date) { + await tx.update(companyWorkloadIdentityKeys).set({ status: "retired", statusChangedAt: at }).where(and( + eq(companyWorkloadIdentityKeys.companyId, companyId), + eq(companyWorkloadIdentityKeys.status, "retiring"), + lte(companyWorkloadIdentityKeys.statusChangedAt, new Date(at.getTime() - WORKLOAD_IDENTITY_RETIRING_GRACE_MS)), + )); + } + + type RotationActor = Pick; + + /** Runs one rotation step and its activity entry in one transaction; publishes the entry after commit. */ + async function rotate( + companyId: string, + actor: RotationActor, + step: "begin" | "complete" | "emergency", + work: (tx: Db, at: Date) => Promise<{ keyId: string; created?: boolean }>, + ) { + const { result, publication } = await db.transaction(async (rawTx) => { + const tx = rawTx as unknown as Db; + await lockCompany(tx, companyId); + const at = now(); + await settleRetiring(tx, companyId, at); + const result = await work(tx, at); + // A repeated "begin" while a rotation is pending changes nothing, so it is not logged. + if (result.created === false) return { result, publication: null }; + const { publication } = await persistActivity(tx, { + companyId, ...actor, + action: `workload_identity.rotation_${step}`, + entityType: "company", entityId: companyId, + details: { keyId: result.keyId }, + }); + return { result, publication }; + }); + if (publication) publishActivity(publication); + if (step !== "begin") signingKeyCache.delete(companyId); + return result; + } + + async function lockCompany(tx: Db, companyId: string) { + const [company] = await tx.select({ id: companies.id }).from(companies) + .where(eq(companies.id, companyId)).for("update"); + if (!company) throw notFound("Company not found"); + } + + return { + ensureActiveKey, + + /** Public key set, or null when the company has never created a key (callers answer 404). */ + async publicKeySet(companyId: string): Promise<{ keys: ReturnType[] } | null> { + const keys = (await publishedKeyRows(companyId)).map(publicJwkFor); + return keys.length > 0 ? { keys } : null; + }, + + /** OIDC discovery document, or null when the issuer isn't usable here or the company has no key. */ + async discoveryDocument(companyId: string, origin = workloadIdentityIssuerOrigin()) { + if (!origin.available) return null; + if ((await publishedKeyRows(companyId)).length === 0) return null; + const issuer = workloadIdentityIssuer(origin.origin, companyId); + return { + issuer, + jwks_uri: `${issuer}/jwks.json`, + response_types_supported: ["id_token"], + subject_types_supported: ["public"], + id_token_signing_alg_values_supported: [WORKLOAD_IDENTITY_ALGORITHM], + claims_supported: ["aud", "exp", "iat", "iss", "jti", "nbf", "sub"], + }; + }, + + /** + * A 5-minute token that names one connection. `sub` and `aud` come from the + * connection and its method, never from agent input. The token itself is + * never stored or logged. + */ + async signConnectionToken(input: { + companyId: string; + connectionId: string; + audience: string; + /** The issuer recorded when the connection was set up; signing stops if the instance address changed. */ + expectedIssuer?: string | null; + origin?: WorkloadIdentityIssuerOrigin; + }) { + if (!/^[\x21-\x7e]{1,256}$/.test(input.audience)) { + throw unprocessable("Invalid workload identity audience", { code: "workload_identity_audience_invalid" }); + } + const origin = input.origin ?? workloadIdentityIssuerOrigin(); + if (!origin.available) { + throw unprocessable("This Paperclip instance needs a public HTTPS address to sign identity tokens.", { + code: "workload_identity_unavailable", reason: origin.reason, + }); + } + const issuer = workloadIdentityIssuer(origin.origin, input.companyId); + if (input.expectedIssuer && input.expectedIssuer !== issuer) { + throw conflict("This Paperclip instance's address changed. Update the identity provider in your cloud account.", { + code: "workload_identity_issuer_changed", + }); + } + const [connection] = await db.select({ status: toolConnections.status, enabled: toolConnections.enabled }) + .from(toolConnections).where(and( + eq(toolConnections.id, input.connectionId), eq(toolConnections.companyId, input.companyId), + )); + if (!connection) throw notFound("Connection not found"); + // Disabling a connection must stop new cloud credentials at once. + if (!connection.enabled || connection.status !== "active") { + throw unprocessable("This connection is not active.", { code: "workload_identity_connection_inactive" }); + } + + const key = await ensureActiveKey(input.companyId); + const privateKey = await privateKeyFor(key); + const issuedAt = Math.floor(now().getTime() / 1000); + const expiresAt = issuedAt + WORKLOAD_IDENTITY_TOKEN_LIFETIME_SECONDS; + const subject = workloadIdentitySubject(input.connectionId); + const header = { alg: WORKLOAD_IDENTITY_ALGORITHM, typ: "JWT", kid: key.keyId }; + const claims = { + iss: issuer, sub: subject, aud: input.audience, + iat: issuedAt, nbf: issuedAt, exp: expiresAt, jti: randomUUID(), + }; + const signingInput = `${base64Url(JSON.stringify(header))}.${base64Url(JSON.stringify(claims))}`; + const signature = sign("sha256", Buffer.from(signingInput), { key: privateKey, dsaEncoding: "ieee-p1363" }); + await db.insert(toolAccessAuditEvents).values({ + companyId: input.companyId, + connectionId: input.connectionId, + actorType: "system", + actorId: "workload-identity", + action: "workload_identity.token_issued", + outcome: "success", + details: { keyId: key.keyId, audience: input.audience, expiresAt: new Date(expiresAt * 1000).toISOString() }, + }); + return { + token: `${signingInput}.${base64Url(signature)}`, + issuer, + subject, + keyId: key.keyId, + expiresAt: new Date(expiresAt * 1000), + }; + }, + + /** Board view: issuer and the published keys with their states, with no private material. */ + async describe(companyId: string, origin = workloadIdentityIssuerOrigin()) { + await assertCompany(companyId); + const rows = await publishedKeyRows(companyId); + return { + issuer: origin.available ? workloadIdentityIssuer(origin.origin, companyId) : null, + unavailableReason: origin.available ? null : origin.reason, + keys: rows + .sort((a, b) => a.createdAt.getTime() - b.createdAt.getTime()) + .map((row) => ({ + keyId: row.keyId, + status: row.status, + createdAt: row.createdAt.toISOString(), + statusChangedAt: row.statusChangedAt.toISOString(), + })), + }; + }, + + /** Step 1 of a planned rotation: publish a `next` key alongside the active one. */ + async beginRotation(companyId: string, actor: RotationActor) { + return rotate(companyId, actor, "begin", async (tx) => { + const rows = await tx.select().from(companyWorkloadIdentityKeys).where(and( + eq(companyWorkloadIdentityKeys.companyId, companyId), + inArray(companyWorkloadIdentityKeys.status, ["next", "active"]), + )); + if (!rows.some((row) => row.status === "active")) { + throw conflict("This company has no signing key to rotate yet.", { code: "workload_identity_no_key" }); + } + const pending = rows.find((row) => row.status === "next"); + if (pending) return { keyId: pending.keyId, created: false }; + const [created] = await tx.insert(companyWorkloadIdentityKeys) + .values(await newKeyValues(companyId, "next")).returning(); + return { keyId: created!.keyId, created: true }; + }); + }, + + /** Step 2: once the `next` key has been published long enough, it signs; the old key stays published for a grace period. */ + async completeRotation(companyId: string, actor: RotationActor) { + return rotate(companyId, actor, "complete", async (tx, at) => { + const [next] = await tx.select().from(companyWorkloadIdentityKeys).where(and( + eq(companyWorkloadIdentityKeys.companyId, companyId), + eq(companyWorkloadIdentityKeys.status, "next"), + )); + if (!next) throw conflict("Start a rotation first.", { code: "workload_identity_no_rotation" }); + const readyAt = new Date(next.createdAt.getTime() + WORKLOAD_IDENTITY_PUBLISH_BEFORE_USE_MS); + if (readyAt > at) { + throw conflict("The new key must be published for 24 hours before it can sign.", { + code: "workload_identity_rotation_not_ready", readyAt: readyAt.toISOString(), + }); + } + await tx.update(companyWorkloadIdentityKeys).set({ status: "retiring", statusChangedAt: at }).where(and( + eq(companyWorkloadIdentityKeys.companyId, companyId), + eq(companyWorkloadIdentityKeys.status, "active"), + )); + await tx.update(companyWorkloadIdentityKeys).set({ status: "active", statusChangedAt: at }) + .where(eq(companyWorkloadIdentityKeys.id, next.id)); + return { keyId: next.keyId }; + }); + }, + + /** For a key believed exposed: stop publishing every current key and sign with a fresh one immediately. */ + async emergencyRotation(companyId: string, actor: RotationActor) { + return rotate(companyId, actor, "emergency", async (tx, at) => { + await tx.update(companyWorkloadIdentityKeys).set({ status: "retired", statusChangedAt: at }).where(and( + eq(companyWorkloadIdentityKeys.companyId, companyId), + inArray(companyWorkloadIdentityKeys.status, ["next", "active", "retiring"]), + )); + const [created] = await tx.insert(companyWorkloadIdentityKeys) + .values(await newKeyValues(companyId, "active")).returning(); + return { keyId: created!.keyId }; + }); + }, + }; +} + +export type WorkloadIdentityService = ReturnType;