mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 03:31:44 +01:00
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Chat connections save each external message as a `chat_actions` row
before the server admits it as an agent wakeup.
> - The server checks if an issue holds chat input that it did not
admit. Recovery skips such an issue. Manual task start refuses it.
> - On a hosted instance, the recovery read with this check used the
most total database time: 488 calls at about 1 s each. The manual-start
check ran 155 times at about 0.9 s.
> - The check matches `chat_actions.payload->>'issueId'` against the
issue. No index covers this value, so each call reads the full
`chat_actions` table and its payloads.
> - This pull request adds one concurrent partial index for the check.
The check SQL does not change.
> - The benefit is less background database load, so request reads wait
less. The results stay the same.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
The chat-admission check `unadmittedChatWakeupCondition`. Recovery uses
it in the per-issue candidate read. Manual task start uses it in the
held-input check.
**Subsystem affected**
Server chat admission, issue recovery, manual task start, and the
database schema.
**Current behavior**
The check finds `inbound_wakeup` actions by `company_id` and
`payload->>'issueId'`. Two nested checks find `failed_run_retry` actions
and later `inbound_wakeup` actions on the same company and payload issue
ID. `chat_actions` has no index on `company_id` or on the payload issue
ID. Each call reads the full table and reads each payload. On a hosted
instance, the two statements used about 630 s of database time in one
statistics window.
**Proposed behavior**
The check and its nested checks read a small set of rows from one index.
The admission result does not change.
**Reason and benefit**
The hosted database had many concurrent active queries. Request reads
waited behind background work such as this check.
**Breaking changes**
None. The migration adds an index only.
## What Changed
- Migration `0458_chat_admission_issue_index` builds
`chat_actions_company_issue_kind_idx` with `CREATE INDEX CONCURRENTLY IF
NOT EXISTS`. It runs through the existing top-level statement path,
which repairs an invalid index on retry.
- Keys: `(company_id, payload->>'issueId', kind, status)`.
- Partial: `kind IN ('inbound_wakeup', 'failed_run_retry')`.
- One index serves the outer input lookup, the failed-run retry lookup,
and the later-input lookup. All three match on the same company and
payload issue ID.
- The Drizzle schema declares the index. The snapshot chain moves to
`0458` and keeps five snapshots.
- New regression test
`server/src/__tests__/chat-admission-condition.test.ts` pins the
admission semantics on real PostgreSQL:
- Pending and failed input without a receipt holds the issue. Received,
processed and cancelled input does not. Other action kinds do not.
- A skipped receipt, a receipt for another agent, or a receipt in
another company does not admit input. A failed or cancelled run still
counts as admission.
- The issue ID is compared as company-scoped text. Uppercase, array,
null, other-issue and other-company values do not match.
- Only an admitted retry in the same conversation and issue that lists
the action releases skipped input.
- Only later admitted input in the same conversation, issue and session
generation releases a failed input. Pending input stays held.
- The check reads `chat_actions` only through the new index.
- `scripts/benchmark-chat-admission-index.ts` measures both caller
statements with and without the index and fails if the rows differ.
- `scripts/benchmark-chat-admission-index-migration.ts` times the
migration through the normal migration runner.
- `doc/DATABASE.md` documents the index and the benchmark commands.
## Verification
Local SQL measurements on synthetic data. PostgreSQL 18.1 (embedded),
default settings. The fixture has 100,995 `chat_actions` rows (662 MB):
60,000 GitHub actions with 6 KiB incompressible payloads, 2,000 issues
with 20 admitted inbound messages each (3 KiB payloads), and 1,000
failed-run retries. Each case has 2 warmups and 9 samples of `EXPLAIN
ANALYZE` execution time. Runs alternate: no index, index, no index,
index.
| Caller statement | Case | No index, median (range) | Index, median
(range) |
| --- | --- | --- | --- |
| Manual-start held check | quiet issue | 3,093 ms (2,150–3,519) / 3,586
ms (2,375–11,235) | 0.036 ms (0.034–0.052) / 0.040 ms (0.033–0.047) |
| Manual-start held check | admitted input | 3,555 ms (3,310–8,080) /
5,205 ms (3,353–9,525) | 0.049 ms / 0.052 ms |
| Manual-start held check | unadmitted input | 3,002 ms (2,864–3,221) /
4,179 ms (3,200–6,585) | 0.051 ms / 0.044 ms |
| Manual-start held check | failed, later admitted | 3,586 ms / 3,722 ms
| 0.066 ms / 0.054 ms |
| Manual-start held check | admitted retry | 2,733 ms / 4,953 ms | 0.049
ms / 0.053 ms |
| Recovery candidate read | quiet issue | 43 ms (41–48) / 96 ms (44–219)
| 0.047 ms / 0.055 ms |
| Recovery candidate read | admitted input | 203 ms (48–575) / 174 ms
(44–314) | 0.055 ms / 0.058 ms |
| Recovery candidate read | unadmitted input | 66 ms / 144 ms | 0.053 ms
/ 0.054 ms |
- Buffers per call fall from about 40,000 shared hits to 5–23.
- Locally, the recovery read without the index uses a hashed full scan
of `chat_actions`. The held check uses a full scan per call. On the
hosted instance both took about 1 s per call. The local fixture is
smaller than the hosted table.
- Correctness: complete positional rows match across all four runs for
all six cases (quiet, admitted, unadmitted, failed then admitted,
retried, other-company decoy). The script asserts the expected held
result for each case.
- Migration timing: 600,001 `chat_actions` rows (1,759 MB), 300,000 of
them in the index scope. First run through `applyPendingMigrations`: 4.2
s. Rerun after the history row is removed: 0.5 s. Concurrent build
alone: 3.4 s. Index size: 30 MB.
Commands:
```sh
pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-chat-admission-index.ts
OTHER_PAYLOAD_KIB=2 pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-chat-admission-index-migration.ts
cd server && npx vitest run src/__tests__/chat-admission-condition.test.ts # 6 passed
pnpm --filter @paperclipai/db check:migrations # passed
pnpm --filter @paperclipai/server typecheck # passed
```
## Risks
- The migration builds one index concurrently. It does not block writes.
On the 1.76 GB local fixture it took about 4 s. A larger hosted table
takes longer, but the build does not lock out reads or writes.
- Writes to `inbound_wakeup` and `failed_run_retry` actions maintain one
more index. Status updates on these rows were already not HOT updates,
because other partial indexes use `status`.
- The planner has no expression statistics for a partial index. The new
test pins that the check uses the index on an analyzed table with
unrelated rows.
- The check SQL does not change, so admission behavior does not change.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through
Claude Code with tool use and code execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
849 lines
32 KiB
TypeScript
849 lines
32 KiB
TypeScript
import { sql } from "drizzle-orm";
|
|
import {
|
|
boolean,
|
|
check,
|
|
foreignKey,
|
|
index,
|
|
integer,
|
|
jsonb,
|
|
pgTable,
|
|
text,
|
|
timestamp,
|
|
unique,
|
|
uniqueIndex,
|
|
uuid,
|
|
} from "drizzle-orm/pg-core";
|
|
import type {
|
|
ChatExecutionDefaults,
|
|
ChatAdapterCapabilities,
|
|
ChatConcurrencyPolicy,
|
|
ChatDeliveryState,
|
|
ChatDeploymentMode,
|
|
ChatEndpointSetupState,
|
|
ChatEndpointStatus,
|
|
ChatEventKind,
|
|
ChatIdentityLinkStatus,
|
|
ChatPrincipalKind,
|
|
ChatProvider,
|
|
ChatPublicationState,
|
|
ChatResourceAvailability,
|
|
SafeChatPublicationPayload,
|
|
} from "@paperclipai/shared";
|
|
import { agents } from "./agents.js";
|
|
import { companies } from "./companies.js";
|
|
import { issueComments } from "./issue_comments.js";
|
|
import { issues } from "./issues.js";
|
|
import { toolConnections } from "./tool_access.js";
|
|
|
|
export const chatEndpoints = pgTable(
|
|
"chat_endpoints",
|
|
{
|
|
id: uuid("id").primaryKey().defaultRandom(),
|
|
companyId: uuid("company_id")
|
|
.notNull()
|
|
.references(() => companies.id, { onDelete: "cascade" }),
|
|
connectionId: uuid("connection_id").notNull(),
|
|
provider: text("provider").$type<ChatProvider>().notNull(),
|
|
publicId: text("public_id").notNull(),
|
|
publicationMode: text("publication_mode").$type<"automatic" | "explicit">().notNull().default("automatic"),
|
|
externalExecutionPolicy: text("external_execution_policy").$type<"restricted" | "agent">().notNull().default("restricted"),
|
|
assignedAgentId: uuid("assigned_agent_id")
|
|
.notNull()
|
|
.references(() => agents.id, { onDelete: "restrict" }),
|
|
sponsorUserId: text("sponsor_user_id"),
|
|
status: text("status")
|
|
.$type<ChatEndpointStatus>()
|
|
.notNull()
|
|
.default("draft"),
|
|
deploymentMode: text("deployment_mode")
|
|
.$type<ChatDeploymentMode>()
|
|
.notNull()
|
|
.default("direct"),
|
|
providerAccountId: text("provider_account_id"),
|
|
providerAccountLabel: text("provider_account_label"),
|
|
botExternalId: text("bot_external_id"),
|
|
botUsername: text("bot_username"),
|
|
botDisplayName: text("bot_display_name"),
|
|
botAvatarUrl: text("bot_avatar_url"),
|
|
communicationInstructions: text("communication_instructions").notNull().default(""),
|
|
executionDefaults: jsonb("execution_defaults").$type<ChatExecutionDefaults>(),
|
|
allowDirectMessages: boolean("allow_direct_messages")
|
|
.notNull()
|
|
.default(true),
|
|
requireAtMention: boolean("require_at_mention").notNull().default(false),
|
|
allowGroupChats: boolean("allow_group_chats").notNull().default(false),
|
|
allowUnlinkedPeople: boolean("allow_unlinked_people")
|
|
.notNull()
|
|
.default(true),
|
|
concurrencyPolicy: text("concurrency_policy")
|
|
.$type<ChatConcurrencyPolicy>()
|
|
.notNull()
|
|
.default("queue"),
|
|
capabilities: jsonb("capabilities")
|
|
.$type<ChatAdapterCapabilities>()
|
|
.notNull()
|
|
.default({
|
|
threads: false,
|
|
directMessages: false,
|
|
nativeStreaming: false,
|
|
messageEdits: false,
|
|
messageDeletes: false,
|
|
reactions: false,
|
|
files: false,
|
|
cards: false,
|
|
actions: false,
|
|
modals: false,
|
|
slashCommands: false,
|
|
ephemeralMessages: false,
|
|
proactiveDirectMessages: false,
|
|
}),
|
|
setup: jsonb("setup")
|
|
.$type<ChatEndpointSetupState>()
|
|
.notNull()
|
|
.default({ step: "provider_setup" }),
|
|
healthMessage: text("health_message"),
|
|
lastEventAt: timestamp("last_event_at", { withTimezone: true }),
|
|
lastPublicationAt: timestamp("last_publication_at", { withTimezone: true }),
|
|
lastError: text("last_error"),
|
|
activatedAt: timestamp("activated_at", { withTimezone: true }),
|
|
archivedAt: timestamp("archived_at", { withTimezone: true }),
|
|
createdAt: timestamp("created_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
updatedAt: timestamp("updated_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
},
|
|
(table) => [
|
|
check("chat_endpoints_publication_mode_check", sql`${table.publicationMode} in ('automatic', 'explicit')`),
|
|
check("chat_endpoints_execution_policy_check", sql`${table.externalExecutionPolicy} in ('restricted', 'agent')`),
|
|
check("chat_endpoints_email_policy_check", sql`${table.provider} <> 'agentmail' or (${table.publicationMode} = 'explicit' and ${table.externalExecutionPolicy} = 'agent')`),
|
|
check(
|
|
"chat_endpoints_provider_check",
|
|
sql`${table.provider} in ('slack', 'github', 'discord', 'microsoft-teams', 'telegram', 'speko', 'agentmail', 'imessage-photon')`,
|
|
),
|
|
check(
|
|
"chat_endpoints_status_check",
|
|
sql`${table.status} in ('draft', 'verifying', 'active', 'paused', 'attention', 'revoked', 'archived')`,
|
|
),
|
|
check(
|
|
"chat_endpoints_deployment_check",
|
|
sql`${table.deploymentMode} in ('direct', 'relay')`,
|
|
),
|
|
check(
|
|
"chat_endpoints_concurrency_check",
|
|
sql`${table.concurrencyPolicy} in ('burst', 'queue', 'debounce', 'drop', 'concurrent')`,
|
|
),
|
|
index("chat_endpoints_company_idx").on(table.companyId),
|
|
index("chat_endpoints_agent_idx").on(
|
|
table.companyId,
|
|
table.assignedAgentId,
|
|
),
|
|
index("chat_endpoints_status_idx").on(table.companyId, table.status),
|
|
uniqueIndex("chat_endpoints_public_id_uq").on(table.publicId),
|
|
uniqueIndex("chat_endpoints_agentmail_inbox_uq")
|
|
.on(table.botExternalId)
|
|
.where(sql`${table.provider} = 'agentmail' and ${table.status} != 'archived' and ${table.botExternalId} is not null`),
|
|
uniqueIndex("chat_endpoints_connection_uq").on(table.connectionId),
|
|
// A native provider identity can back only one live Paperclip endpoint.
|
|
// Historical archived/revoked endpoints retain attribution without
|
|
// preventing an operator from deliberately reusing the provider bot later.
|
|
uniqueIndex("chat_endpoints_live_bot_external_uq")
|
|
.on(table.provider, table.providerAccountId, table.botExternalId)
|
|
.where(
|
|
sql`${table.status} in ('verifying', 'active', 'paused', 'attention')
|
|
and ${table.providerAccountId} is not null
|
|
and ${table.botExternalId} is not null`,
|
|
),
|
|
// One Discord application can be installed in many guilds, but it remains
|
|
// one native bot identity. Excluding providerAccountId closes the race
|
|
// where concurrent setup in two guilds could otherwise claim that bot for
|
|
// two Paperclip agents after both application-level prechecks passed.
|
|
uniqueIndex("chat_endpoints_photon_number_uq")
|
|
.on(table.botExternalId)
|
|
.where(sql`${table.provider} = 'imessage-photon' and ${table.status} <> 'archived' and ${table.botExternalId} is not null`),
|
|
uniqueIndex("chat_endpoints_live_discord_bot_external_uq")
|
|
.on(table.provider, table.botExternalId)
|
|
.where(
|
|
sql`${table.provider} = 'discord'
|
|
and ${table.status} in ('verifying', 'active', 'paused', 'attention')
|
|
and ${table.botExternalId} is not null`,
|
|
),
|
|
// GitHub App and Microsoft Bot application ids are provider-global bot
|
|
// identities. Their mutable owner/tenant coordinate is useful metadata,
|
|
// but it cannot be part of the exclusivity key: an App transfer or a
|
|
// multi-tenant service principal must never let one native bot represent
|
|
// two Paperclip agents through two different webhook URLs.
|
|
uniqueIndex("chat_endpoints_live_global_app_bot_external_uq")
|
|
.on(table.provider, table.botExternalId)
|
|
.where(
|
|
sql`${table.provider} in ('github', 'microsoft-teams')
|
|
and ${table.status} in ('verifying', 'active', 'paused', 'attention')
|
|
and ${table.botExternalId} is not null`,
|
|
),
|
|
// GitHub App verification does not expose the bot user's numeric id, so
|
|
// retain an equivalent live-slot constraint on the provider-native name.
|
|
uniqueIndex("chat_endpoints_live_bot_username_uq")
|
|
.on(table.provider, table.providerAccountId, table.botUsername)
|
|
.where(
|
|
sql`${table.status} in ('verifying', 'active', 'paused', 'attention')
|
|
and ${table.providerAccountId} is not null
|
|
and ${table.botUsername} is not null`,
|
|
),
|
|
unique("chat_endpoints_company_id_uq").on(table.companyId, table.id),
|
|
foreignKey({
|
|
columns: [table.companyId, table.assignedAgentId],
|
|
foreignColumns: [agents.companyId, agents.id],
|
|
name: "chat_endpoints_company_agent_fk",
|
|
}),
|
|
foreignKey({
|
|
columns: [table.companyId, table.connectionId],
|
|
foreignColumns: [toolConnections.companyId, toolConnections.id],
|
|
name: "chat_endpoints_company_connection_fk",
|
|
}).onDelete("cascade"),
|
|
],
|
|
);
|
|
export const chatEndpointResources = pgTable(
|
|
"chat_endpoint_resources",
|
|
{
|
|
id: uuid("id").primaryKey().defaultRandom(),
|
|
companyId: uuid("company_id")
|
|
.notNull()
|
|
.references(() => companies.id, { onDelete: "cascade" }),
|
|
endpointId: uuid("endpoint_id").notNull(),
|
|
type: text("type").notNull(),
|
|
providerResourceId: text("provider_resource_id").notNull(),
|
|
parentProviderResourceId: text("parent_provider_resource_id"),
|
|
label: text("label").notNull(),
|
|
detail: text("detail"),
|
|
providerUrl: text("provider_url"),
|
|
availability: text("availability")
|
|
.$type<ChatResourceAvailability>()
|
|
.notNull()
|
|
.default("available"),
|
|
enabled: boolean("enabled").notNull().default(false),
|
|
executionDefaults: jsonb("execution_defaults").$type<ChatExecutionDefaults>(),
|
|
metadata: jsonb("metadata")
|
|
.$type<Record<string, unknown>>()
|
|
.notNull()
|
|
.default({}),
|
|
createdAt: timestamp("created_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
updatedAt: timestamp("updated_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
},
|
|
(table) => [
|
|
check(
|
|
"chat_endpoint_resources_availability_check",
|
|
sql`${table.availability} in ('available', 'unavailable', 'removed')`,
|
|
),
|
|
index("chat_endpoint_resources_endpoint_idx").on(
|
|
table.companyId,
|
|
table.endpointId,
|
|
),
|
|
uniqueIndex("chat_endpoint_resources_external_uq").on(
|
|
table.endpointId,
|
|
table.type,
|
|
table.providerResourceId,
|
|
),
|
|
unique("chat_endpoint_resources_company_id_uq").on(
|
|
table.companyId,
|
|
table.id,
|
|
),
|
|
foreignKey({
|
|
columns: [table.companyId, table.endpointId],
|
|
foreignColumns: [chatEndpoints.companyId, chatEndpoints.id],
|
|
name: "chat_endpoint_resources_company_endpoint_fk",
|
|
}).onDelete("cascade"),
|
|
],
|
|
);
|
|
|
|
export const chatExternalPrincipals = pgTable(
|
|
"chat_external_principals",
|
|
{
|
|
id: uuid("id").primaryKey().defaultRandom(),
|
|
companyId: uuid("company_id")
|
|
.notNull()
|
|
.references(() => companies.id, { onDelete: "cascade" }),
|
|
provider: text("provider").$type<ChatProvider>().notNull(),
|
|
providerAccountId: text("provider_account_id").notNull(),
|
|
externalId: text("external_id").notNull(),
|
|
kind: text("kind").$type<ChatPrincipalKind>().notNull().default("user"),
|
|
displayName: text("display_name"),
|
|
handle: text("handle"),
|
|
avatarUrl: text("avatar_url"),
|
|
isBot: boolean("is_bot").notNull().default(false),
|
|
lastSeenAt: timestamp("last_seen_at", { withTimezone: true }),
|
|
createdAt: timestamp("created_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
updatedAt: timestamp("updated_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
},
|
|
(table) => [
|
|
check(
|
|
"chat_external_principals_provider_check",
|
|
sql`${table.provider} in ('slack', 'github', 'discord', 'microsoft-teams', 'telegram', 'speko', 'agentmail', 'imessage-photon')`,
|
|
),
|
|
check(
|
|
"chat_external_principals_kind_check",
|
|
sql`${table.kind} in ('user', 'bot', 'app', 'system')`,
|
|
),
|
|
index("chat_external_principals_company_idx").on(table.companyId),
|
|
uniqueIndex("chat_external_principals_external_uq").on(
|
|
table.companyId,
|
|
table.provider,
|
|
table.providerAccountId,
|
|
table.externalId,
|
|
),
|
|
unique("chat_external_principals_company_id_uq").on(
|
|
table.companyId,
|
|
table.id,
|
|
),
|
|
],
|
|
);
|
|
|
|
export const chatIdentityLinks = pgTable(
|
|
"chat_identity_links",
|
|
{
|
|
id: uuid("id").primaryKey().defaultRandom(),
|
|
companyId: uuid("company_id")
|
|
.notNull()
|
|
.references(() => companies.id, { onDelete: "cascade" }),
|
|
endpointId: uuid("endpoint_id").notNull(),
|
|
principalId: uuid("principal_id").notNull(),
|
|
paperclipUserId: text("paperclip_user_id"),
|
|
status: text("status")
|
|
.$type<ChatIdentityLinkStatus>()
|
|
.notNull()
|
|
.default("pending"),
|
|
confirmationTokenHash: text("confirmation_token_hash"),
|
|
expiresAt: timestamp("expires_at", { withTimezone: true }),
|
|
confirmedAt: timestamp("confirmed_at", { withTimezone: true }),
|
|
revokedAt: timestamp("revoked_at", { withTimezone: true }),
|
|
createdAt: timestamp("created_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
updatedAt: timestamp("updated_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
},
|
|
(table) => [
|
|
check(
|
|
"chat_identity_links_status_check",
|
|
sql`${table.status} in ('pending', 'linked', 'revoked', 'expired')`,
|
|
),
|
|
index("chat_identity_links_user_idx").on(
|
|
table.companyId,
|
|
table.paperclipUserId,
|
|
),
|
|
uniqueIndex("chat_identity_links_endpoint_principal_uq").on(
|
|
table.endpointId,
|
|
table.principalId,
|
|
),
|
|
foreignKey({
|
|
columns: [table.companyId, table.endpointId],
|
|
foreignColumns: [chatEndpoints.companyId, chatEndpoints.id],
|
|
name: "chat_identity_links_company_endpoint_fk",
|
|
}).onDelete("cascade"),
|
|
foreignKey({
|
|
columns: [table.companyId, table.principalId],
|
|
foreignColumns: [
|
|
chatExternalPrincipals.companyId,
|
|
chatExternalPrincipals.id,
|
|
],
|
|
name: "chat_identity_links_company_principal_fk",
|
|
}).onDelete("cascade"),
|
|
],
|
|
);
|
|
|
|
export const chatConversations = pgTable(
|
|
"chat_conversations",
|
|
{
|
|
id: uuid("id").primaryKey().defaultRandom(),
|
|
companyId: uuid("company_id")
|
|
.notNull()
|
|
.references(() => companies.id, { onDelete: "cascade" }),
|
|
endpointId: uuid("endpoint_id").notNull(),
|
|
resourceId: uuid("resource_id").references(() => chatEndpointResources.id, {
|
|
onDelete: "set null",
|
|
}),
|
|
issueId: uuid("issue_id")
|
|
.notNull()
|
|
.references(() => issues.id, { onDelete: "restrict" }),
|
|
externalConversationId: text("external_conversation_id").notNull(),
|
|
externalThreadId: text("external_thread_id").notNull().default(""),
|
|
// Providers with linear conversations (DMs, Telegram groups, Teams group
|
|
// chats) reuse one native thread id. A generation preserves the native id
|
|
// used for replies while allowing completed Paperclip tasks to roll over.
|
|
sessionGeneration: integer("session_generation").notNull().default(1),
|
|
externalLabel: text("external_label").notNull(),
|
|
providerUrl: text("provider_url"),
|
|
isDirectMessage: boolean("is_direct_message").notNull().default(false),
|
|
// Immutable initial task context, never refreshed from endpoint settings.
|
|
communicationGuidance: text("communication_guidance"),
|
|
state: text("state").notNull().default("active"),
|
|
lastActivityAt: timestamp("last_activity_at", { withTimezone: true }),
|
|
createdAt: timestamp("created_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
updatedAt: timestamp("updated_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
},
|
|
(table) => [
|
|
check(
|
|
"chat_conversations_state_check",
|
|
sql`${table.state} in ('active', 'waiting', 'completed', 'unavailable', 'endpoint_removed')`,
|
|
),
|
|
index("chat_conversations_issue_idx").on(table.companyId, table.issueId),
|
|
uniqueIndex("chat_conversations_thread_uq").on(
|
|
table.endpointId,
|
|
table.externalConversationId,
|
|
table.externalThreadId,
|
|
table.sessionGeneration,
|
|
),
|
|
unique("chat_conversations_company_id_uq").on(table.companyId, table.id),
|
|
foreignKey({
|
|
columns: [table.companyId, table.issueId],
|
|
foreignColumns: [issues.companyId, issues.id],
|
|
name: "chat_conversations_company_issue_fk",
|
|
}),
|
|
foreignKey({
|
|
columns: [table.companyId, table.endpointId],
|
|
foreignColumns: [chatEndpoints.companyId, chatEndpoints.id],
|
|
name: "chat_conversations_company_endpoint_fk",
|
|
}).onDelete("cascade"),
|
|
foreignKey({
|
|
columns: [table.companyId, table.resourceId],
|
|
foreignColumns: [
|
|
chatEndpointResources.companyId,
|
|
chatEndpointResources.id,
|
|
],
|
|
name: "chat_conversations_company_resource_fk",
|
|
}),
|
|
],
|
|
);
|
|
|
|
export const chatDeliveries = pgTable(
|
|
"chat_deliveries",
|
|
{
|
|
id: uuid("id").primaryKey().defaultRandom(),
|
|
companyId: uuid("company_id")
|
|
.notNull()
|
|
.references(() => companies.id, { onDelete: "cascade" }),
|
|
endpointId: uuid("endpoint_id").notNull(),
|
|
conversationId: uuid("conversation_id").references(
|
|
() => chatConversations.id,
|
|
{
|
|
onDelete: "set null",
|
|
},
|
|
),
|
|
principalId: uuid("principal_id").references(
|
|
() => chatExternalPrincipals.id,
|
|
{
|
|
onDelete: "set null",
|
|
},
|
|
),
|
|
providerEventId: text("provider_event_id").notNull(),
|
|
deduplicationKey: text("deduplication_key").notNull(),
|
|
eventKind: text("event_kind").$type<ChatEventKind>().notNull(),
|
|
normalizedEvent: jsonb("normalized_event")
|
|
.$type<Record<string, unknown>>()
|
|
.notNull(),
|
|
state: text("state")
|
|
.$type<ChatDeliveryState>()
|
|
.notNull()
|
|
.default("received"),
|
|
attempts: integer("attempts").notNull().default(0),
|
|
redactedError: text("redacted_error"),
|
|
nextAttemptAt: timestamp("next_attempt_at", { withTimezone: true }),
|
|
receivedAt: timestamp("received_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
processedAt: timestamp("processed_at", { withTimezone: true }),
|
|
createdAt: timestamp("created_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
updatedAt: timestamp("updated_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
},
|
|
(table) => [
|
|
check(
|
|
"chat_deliveries_state_check",
|
|
sql`${table.state} in ('received', 'filtered', 'processing', 'processed', 'retry', 'failed')`,
|
|
),
|
|
index("chat_deliveries_work_idx").on(table.state, table.nextAttemptAt),
|
|
unique("chat_deliveries_company_id_uq").on(table.companyId, table.id),
|
|
uniqueIndex("chat_deliveries_event_uq").on(
|
|
table.endpointId,
|
|
table.providerEventId,
|
|
),
|
|
uniqueIndex("chat_deliveries_dedupe_uq").on(
|
|
table.endpointId,
|
|
table.deduplicationKey,
|
|
),
|
|
// Review checks find their delivery by the GitHub webhook delivery id.
|
|
index("chat_deliveries_github_automatic_delivery_idx")
|
|
.on(table.endpointId, sql`(${table.normalizedEvent}->'githubAutomatic'->'context'->>'deliveryId')`)
|
|
.where(sql`(${table.normalizedEvent}->'githubAutomatic'->'context'->>'deliveryId') is not null`)
|
|
.concurrently(),
|
|
foreignKey({
|
|
columns: [table.companyId, table.endpointId],
|
|
foreignColumns: [chatEndpoints.companyId, chatEndpoints.id],
|
|
name: "chat_deliveries_company_endpoint_fk",
|
|
}).onDelete("cascade"),
|
|
foreignKey({
|
|
columns: [table.companyId, table.conversationId],
|
|
foreignColumns: [chatConversations.companyId, chatConversations.id],
|
|
name: "chat_deliveries_company_conversation_fk",
|
|
}),
|
|
foreignKey({
|
|
columns: [table.companyId, table.principalId],
|
|
foreignColumns: [
|
|
chatExternalPrincipals.companyId,
|
|
chatExternalPrincipals.id,
|
|
],
|
|
name: "chat_deliveries_company_principal_fk",
|
|
}),
|
|
],
|
|
);
|
|
|
|
export const chatPublications = pgTable(
|
|
"chat_publications",
|
|
{
|
|
id: uuid("id").primaryKey().defaultRandom(),
|
|
companyId: uuid("company_id")
|
|
.notNull()
|
|
.references(() => companies.id, { onDelete: "cascade" }),
|
|
endpointId: uuid("endpoint_id").notNull(),
|
|
conversationId: uuid("conversation_id").notNull(),
|
|
issueId: uuid("issue_id")
|
|
.notNull()
|
|
.references(() => issues.id, { onDelete: "restrict" }),
|
|
commentId: uuid("comment_id").references(() => issueComments.id, {
|
|
onDelete: "set null",
|
|
}),
|
|
idempotencyKey: text("idempotency_key").notNull(),
|
|
payload: jsonb("payload").$type<SafeChatPublicationPayload>().notNull(),
|
|
state: text("state")
|
|
.$type<ChatPublicationState>()
|
|
.notNull()
|
|
.default("pending"),
|
|
providerMessageId: text("provider_message_id"),
|
|
providerUrl: text("provider_url"),
|
|
attempts: integer("attempts").notNull().default(0),
|
|
redactedError: text("redacted_error"),
|
|
nextAttemptAt: timestamp("next_attempt_at", { withTimezone: true }),
|
|
publishedAt: timestamp("published_at", { withTimezone: true }),
|
|
createdAt: timestamp("created_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
updatedAt: timestamp("updated_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
},
|
|
(table) => [
|
|
check(
|
|
"chat_publications_state_check",
|
|
sql`${table.state} in ('pending', 'streaming', 'published', 'retry', 'delivery_unknown', 'failed', 'cancelled', 'awaiting_consent')`,
|
|
),
|
|
uniqueIndex("chat_publications_company_id_uq").on(
|
|
table.companyId,
|
|
table.id,
|
|
),
|
|
foreignKey({
|
|
columns: [table.companyId, table.issueId],
|
|
foreignColumns: [issues.companyId, issues.id],
|
|
name: "chat_publications_company_issue_fk",
|
|
}),
|
|
// Retain the single-column SET NULL action above. The additional tenant
|
|
// key uses NO ACTION so deletion clears only comment_id, never company_id.
|
|
foreignKey({
|
|
columns: [table.companyId, table.commentId],
|
|
foreignColumns: [issueComments.companyId, issueComments.id],
|
|
name: "chat_publications_company_comment_fk",
|
|
}),
|
|
index("chat_publications_work_idx").on(table.state, table.nextAttemptAt),
|
|
index("chat_publications_comment_destination_idx").on(table.companyId, table.commentId, table.endpointId, table.conversationId)
|
|
.where(sql`${table.commentId} is not null`),
|
|
uniqueIndex("chat_publications_idempotency_uq").on(
|
|
table.companyId,
|
|
table.idempotencyKey,
|
|
),
|
|
foreignKey({
|
|
columns: [table.companyId, table.endpointId],
|
|
foreignColumns: [chatEndpoints.companyId, chatEndpoints.id],
|
|
name: "chat_publications_company_endpoint_fk",
|
|
}).onDelete("cascade"),
|
|
foreignKey({
|
|
columns: [table.companyId, table.conversationId],
|
|
foreignColumns: [chatConversations.companyId, chatConversations.id],
|
|
name: "chat_publications_company_conversation_fk",
|
|
}).onDelete("cascade"),
|
|
],
|
|
);
|
|
|
|
export const chatMessageLinks = pgTable(
|
|
"chat_message_links",
|
|
{
|
|
id: uuid("id").primaryKey().defaultRandom(),
|
|
companyId: uuid("company_id")
|
|
.notNull()
|
|
.references(() => companies.id, { onDelete: "cascade" }),
|
|
endpointId: uuid("endpoint_id").notNull(),
|
|
conversationId: uuid("conversation_id").notNull(),
|
|
deliveryId: uuid("delivery_id").references(() => chatDeliveries.id, {
|
|
onDelete: "set null",
|
|
}),
|
|
publicationId: uuid("publication_id").references(
|
|
() => chatPublications.id,
|
|
{ onDelete: "set null" },
|
|
),
|
|
commentId: uuid("comment_id").references(() => issueComments.id, {
|
|
onDelete: "set null",
|
|
}),
|
|
providerMessageId: text("provider_message_id").notNull(),
|
|
direction: text("direction").notNull(),
|
|
createdAt: timestamp("created_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
},
|
|
(table) => [
|
|
check(
|
|
"chat_message_links_direction_check",
|
|
sql`${table.direction} in ('inbound', 'outbound')`,
|
|
),
|
|
foreignKey({
|
|
columns: [table.companyId, table.endpointId],
|
|
foreignColumns: [chatEndpoints.companyId, chatEndpoints.id],
|
|
name: "chat_message_links_company_endpoint_fk",
|
|
}).onDelete("cascade"),
|
|
foreignKey({
|
|
columns: [table.companyId, table.deliveryId],
|
|
foreignColumns: [chatDeliveries.companyId, chatDeliveries.id],
|
|
name: "chat_message_links_company_delivery_fk",
|
|
}),
|
|
foreignKey({
|
|
columns: [table.companyId, table.publicationId],
|
|
foreignColumns: [chatPublications.companyId, chatPublications.id],
|
|
name: "chat_message_links_company_publication_fk",
|
|
}),
|
|
foreignKey({
|
|
columns: [table.companyId, table.commentId],
|
|
foreignColumns: [issueComments.companyId, issueComments.id],
|
|
name: "chat_message_links_company_comment_fk",
|
|
}),
|
|
uniqueIndex("chat_message_links_provider_message_uq").on(
|
|
table.endpointId,
|
|
table.conversationId,
|
|
table.providerMessageId,
|
|
),
|
|
// Source-comment and delivery lookups (GitHub review waits and checks).
|
|
// These also serve the comment and delivery `on delete set null` actions.
|
|
index("chat_message_links_company_comment_idx")
|
|
.on(table.companyId, table.commentId)
|
|
.where(sql`${table.commentId} is not null`)
|
|
.concurrently(),
|
|
index("chat_message_links_delivery_idx")
|
|
.on(table.deliveryId)
|
|
.where(sql`${table.deliveryId} is not null`)
|
|
.concurrently(),
|
|
foreignKey({
|
|
columns: [table.companyId, table.conversationId],
|
|
foreignColumns: [chatConversations.companyId, chatConversations.id],
|
|
name: "chat_message_links_company_conversation_fk",
|
|
}).onDelete("cascade"),
|
|
],
|
|
);
|
|
|
|
export const chatActions = pgTable(
|
|
"chat_actions",
|
|
{
|
|
id: uuid("id").primaryKey().defaultRandom(),
|
|
companyId: uuid("company_id")
|
|
.notNull()
|
|
.references(() => companies.id, { onDelete: "cascade" }),
|
|
endpointId: uuid("endpoint_id").notNull(),
|
|
deliveryId: uuid("delivery_id").references(() => chatDeliveries.id, {
|
|
onDelete: "set null",
|
|
}),
|
|
conversationId: uuid("conversation_id"),
|
|
principalId: uuid("principal_id"),
|
|
kind: text("kind").notNull(),
|
|
providerActionId: text("provider_action_id").notNull(),
|
|
payload: jsonb("payload")
|
|
.$type<Record<string, unknown>>()
|
|
.notNull()
|
|
.default({}),
|
|
status: text("status").notNull().default("received"),
|
|
result: jsonb("result").$type<Record<string, unknown>>(),
|
|
createdAt: timestamp("created_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
updatedAt: timestamp("updated_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
},
|
|
(table) => [
|
|
index("chat_actions_continuation_work_idx")
|
|
.on(table.companyId, table.id)
|
|
.where(sql`${table.kind} = 'interaction_wakeup' AND ${table.status} IN ('issued', 'processing')`),
|
|
index("chat_actions_confirmation_recovery_idx")
|
|
.on(table.companyId, table.id)
|
|
.where(sql`${table.kind} = 'confirmation_response' AND ${table.status} = 'processing'`),
|
|
uniqueIndex("chat_actions_provider_action_uq").on(
|
|
table.endpointId,
|
|
table.providerActionId,
|
|
),
|
|
// Chat-admission checks read an issue's inbound input and its failed-run
|
|
// retries by company and payload issue id.
|
|
index("chat_actions_company_issue_kind_idx")
|
|
.on(table.companyId, sql`(${table.payload}->>'issueId')`, table.kind, table.status)
|
|
.where(sql`${table.kind} IN ('inbound_wakeup', 'failed_run_retry')`)
|
|
.concurrently(),
|
|
index("chat_actions_github_stale_head_idx").on(table.companyId, table.endpointId,
|
|
sql`(${table.payload}->'event'->>'deliveryId')`).where(sql`${table.kind} = 'github_review_check'
|
|
and ${table.status} = 'cancelled' and ${table.result}->>'code' = 'stale_head'`),
|
|
index("chat_actions_github_check_work_idx").on(table.id).where(sql`${table.kind} = 'github_review_check'
|
|
and (${table.status} = 'received' or (${table.status} = 'failed' and ${table.result}->>'retryable' = 'true'))`),
|
|
index("chat_actions_github_terminal_work_idx").on(table.id).where(sql`${table.kind} = 'github_response_comment'
|
|
and ${table.status} = 'processed' and ${table.deliveryId} is not null
|
|
and coalesce(${table.result}->>'final', 'false') <> 'true'`),
|
|
index("chat_actions_github_ack_work_idx").on(table.id).where(sql`${table.kind} = 'github_response_comment'
|
|
and ${table.deliveryId} is not null and coalesce(${table.result}->>'final', 'false') <> 'true'
|
|
and (${table.status} = 'received' or (${table.status} = 'failed' and ${table.result}->>'retryable' = 'true'))`),
|
|
foreignKey({
|
|
columns: [table.companyId, table.deliveryId],
|
|
foreignColumns: [chatDeliveries.companyId, chatDeliveries.id],
|
|
name: "chat_actions_company_delivery_fk",
|
|
}),
|
|
foreignKey({
|
|
columns: [table.companyId, table.conversationId],
|
|
foreignColumns: [chatConversations.companyId, chatConversations.id],
|
|
name: "chat_actions_company_conversation_fk",
|
|
}),
|
|
foreignKey({
|
|
columns: [table.companyId, table.principalId],
|
|
foreignColumns: [
|
|
chatExternalPrincipals.companyId,
|
|
chatExternalPrincipals.id,
|
|
],
|
|
name: "chat_actions_company_principal_fk",
|
|
}),
|
|
foreignKey({
|
|
columns: [table.companyId, table.endpointId],
|
|
foreignColumns: [chatEndpoints.companyId, chatEndpoints.id],
|
|
name: "chat_actions_company_endpoint_fk",
|
|
}).onDelete("cascade"),
|
|
],
|
|
);
|
|
|
|
export const chatAgentRoutes = pgTable(
|
|
"chat_agent_routes",
|
|
{
|
|
id: uuid("id").primaryKey().defaultRandom(),
|
|
companyId: uuid("company_id")
|
|
.notNull()
|
|
.references(() => companies.id, { onDelete: "cascade" }),
|
|
sourceEndpointId: uuid("source_endpoint_id").notNull(),
|
|
destinationEndpointId: uuid("destination_endpoint_id").notNull(),
|
|
enabled: boolean("enabled").notNull().default(false),
|
|
triggerMode: text("trigger_mode").notNull().default("explicit_mention"),
|
|
maxHops: integer("max_hops").notNull().default(1),
|
|
createdByUserId: text("created_by_user_id"),
|
|
createdAt: timestamp("created_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
updatedAt: timestamp("updated_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
},
|
|
(table) => [
|
|
check(
|
|
"chat_agent_routes_hops_check",
|
|
sql`${table.maxHops} between 1 and 8`,
|
|
),
|
|
uniqueIndex("chat_agent_routes_pair_uq").on(
|
|
table.sourceEndpointId,
|
|
table.destinationEndpointId,
|
|
),
|
|
foreignKey({
|
|
columns: [table.companyId, table.sourceEndpointId],
|
|
foreignColumns: [chatEndpoints.companyId, chatEndpoints.id],
|
|
name: "chat_agent_routes_company_source_fk",
|
|
}).onDelete("cascade"),
|
|
foreignKey({
|
|
columns: [table.companyId, table.destinationEndpointId],
|
|
foreignColumns: [chatEndpoints.companyId, chatEndpoints.id],
|
|
name: "chat_agent_routes_company_destination_fk",
|
|
}).onDelete("cascade"),
|
|
],
|
|
);
|
|
|
|
export const chatEndpointLeases = pgTable(
|
|
"chat_endpoint_leases",
|
|
{
|
|
id: uuid("id").primaryKey().defaultRandom(),
|
|
companyId: uuid("company_id")
|
|
.notNull()
|
|
.references(() => companies.id, { onDelete: "cascade" }),
|
|
endpointId: uuid("endpoint_id").notNull(),
|
|
leaseKey: text("lease_key").notNull(),
|
|
token: text("token").notNull(),
|
|
expiresAt: timestamp("expires_at", { withTimezone: true }).notNull(),
|
|
createdAt: timestamp("created_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
updatedAt: timestamp("updated_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
},
|
|
(table) => [
|
|
uniqueIndex("chat_endpoint_leases_active_uq").on(
|
|
table.endpointId,
|
|
table.leaseKey,
|
|
),
|
|
index("chat_endpoint_leases_expiry_idx").on(table.expiresAt),
|
|
foreignKey({
|
|
columns: [table.companyId, table.endpointId],
|
|
foreignColumns: [chatEndpoints.companyId, chatEndpoints.id],
|
|
name: "chat_endpoint_leases_company_endpoint_fk",
|
|
}).onDelete("cascade"),
|
|
],
|
|
);
|
|
|
|
export const chatSdkState = pgTable(
|
|
"chat_sdk_state",
|
|
{
|
|
id: uuid("id").primaryKey().defaultRandom(),
|
|
companyId: uuid("company_id")
|
|
.notNull()
|
|
.references(() => companies.id, { onDelete: "cascade" }),
|
|
endpointId: uuid("endpoint_id").notNull(),
|
|
stateKey: text("state_key").notNull(),
|
|
version: integer("version").notNull().default(1),
|
|
value: jsonb("value").$type<unknown>().notNull(),
|
|
expiresAt: timestamp("expires_at", { withTimezone: true }),
|
|
createdAt: timestamp("created_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
updatedAt: timestamp("updated_at", { withTimezone: true })
|
|
.notNull()
|
|
.defaultNow(),
|
|
},
|
|
(table) => [
|
|
uniqueIndex("chat_sdk_state_key_uq").on(table.endpointId, table.stateKey),
|
|
index("chat_sdk_state_expiry_idx").on(table.expiresAt),
|
|
foreignKey({
|
|
columns: [table.companyId, table.endpointId],
|
|
foreignColumns: [chatEndpoints.companyId, chatEndpoints.id],
|
|
name: "chat_sdk_state_company_endpoint_fk",
|
|
}).onDelete("cascade"),
|
|
],
|
|
);
|