Files
PaperClipAI/server/src/services/documents.ts
T
DottaandClaude Opus 5.5 067f7a0815 feat(telemetry): send product lifecycle events and agent run failure cause (#22250)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - First-party telemetry tells the team which product steps people
finish
> - Telemetry did not record when a model is connected, when a task is
created, completed, or blocked, when a plan is created or approved, or
when an artifact is created
> - Telemetry also did not tell why an agent run failed, so failures
could not be grouped by cause
> - This pull request syncs the new telemetry contract and sends these
events from the server once per real change
> - The benefit is a product funnel and a failure breakdown that use
only enums, booleans, and counts

## Linked Issues or Issue Description

**Subsystem affected**
Server telemetry (`packages/shared/src/telemetry`,
`server/src/services/product-telemetry.ts`) and the emit sites in the
issue, document, work product, interaction, and AI connection services.

**Problem or motivation**
The team cannot measure the main product path. There is no event for a
new model connection, a new task, a completed task, a blocked task, a
new plan, an approved plan, or a new artifact. `agent.task_run` tells
that a run failed, but not why. The signup funnel also must know when
the onboarding wizard created the first task.

**Proposed solution**
Register the events in the telemetry backend. Sync the generated
contract here. Add typed helpers. Send each event once, after the
transaction that makes the change commits. Use fixed enums for every
kind, reason, and cause.

**Alternatives considered**
A post-commit action queue on the issue service only. The native status
path and many other callers use an external transaction without that
queue, so most agent completions would not send an event. A generic
after-commit hook on the database client covers every caller.

**Roadmap alignment**
This is telemetry for existing features. It adds no new product feature.

**Additional context**
This change depends on paperclipai/telemetry-backend#150, which is
merged. Deploy that backend change before this change ships. Until then,
the collector rejects the new event names.

## What Changed

- Sync `generated/paperclip-telemetry.ts` from the backend branch as a
direct copy. Regenerate `posthog-direct/enum-fallbacks.generated.ts`
with `scripts/generate-posthog-direct-fixtures.mjs`. The loader golden
file did not change.
- Add `afterTransactionCommit()` to `@paperclipai/db`. It runs a
callback after the outermost transaction commits. It drops the callback
on rollback. Outside a transaction it runs the callback at once. A
callback that throws never fails the committed work.
- Add helpers `trackModelConnected`, `trackTaskCreated`,
`trackTaskCompleted`, `trackTaskBlocked`, `trackPlanCreated`,
`trackPlanApproved`, and `trackArtifactCreated`. Add `failureCause` to
`trackAgentTaskRun`. Assign the new events to `operational_enum_count`
retention.
- Add `server/src/services/product-telemetry.ts`. It sends each event
after commit inside `withTelemetryCompany()`, so Cloud direct delivery
gets `company_id` for background writes.
- Emit sites:
- `model.connected`: `aiConnectionService.save`, for a new connection
only. A reconnect sends nothing. `first_for_company` is true when the
company had no earlier model connection.
- `task.created`: `issueService.create`, after the insert. A
deduplicated create, a company import, and a chat conversation send
nothing. `created_by_kind` is `routine` for routine runs, `onboarding`
for the onboarding wizard's first task, and otherwise `user`, `agent`,
or `system`.
- `task.completed` and `task.blocked`: `issueService.update`, when the
status changes to `done` or `blocked`. The same status again sends
nothing.
- `plan.created`: `documentService.upsertIssueDocument`, for the first
revision of the `plan` document.
- `plan.approved`: the accept path of a `request_confirmation` that
targets the task's `plan` document.
- `artifact.created`: `workProductService.createForIssue` (`direct`),
the promotion route (`promotion`), and a run file deliverable
(`run_deliverable`).
- `agent.task_run.failure_cause`: `emitAgentTaskRun`, for `failed`,
`timed_out`, and `interrupted` runs only.
- `blocked_reason` mapping: unfinished blocking tasks give
`waiting_on_task`. Otherwise an unblock owner of a person or the board
gives `waiting_for_person`, and an agent owner gives
`waiting_for_agent`. A system write with neither gives `needs_recovery`.
A person or agent write with neither gives `unspecified`.
- `failure_cause` mapping (stored `errorCode`, with
`resultJson.providerFailure.category` for generic provider and adapter
codes):
- `missing_access`: auth and permission codes (`acpx_auth_required`,
`provider_auth_failed`, `user_secret_missing`, `github_access_denied`,
`github_repo_not_accessible`, `permission_denied`,
`project_access_denied`, `configuration_incomplete`,
`connection_unavailable`, `signing_secret_unconfigured`), and provider
category `authentication` or `permission`.
- `provider_limit`: `provider_quota`, `native_provider_usage_limit`,
`native_provider_overloaded`, `github_rate_limited`,
`ai_connection_busy`, and provider category `rate_limit`,
`quota_exhausted`, or `overloaded`.
- `provider_error`: `native_provider_model_rejected`,
`provider_transport_failed`, `codex_transient_upstream`,
`provider_failed`, `provider_tool_definition_invalid`, and the other
provider categories.
- `waiting_for_person`: `execution_reconciliation_required`, every code
in `EXECUTION_RECONCILIATION_CAUSES`, `issue_waiting_for_response`, and
`issue_continuation_waiting_on_review`.
- `task_changed`: the task was reassigned, paused, closed, blocked, or
stopped by a person (`operator_interrupted`, `cancelled`, `interrupted`,
`issue_*` lifecycle codes, `agent_paused`, `queued_comment_discarded`).
- `platform_error`: process, server, workspace, sandbox, runner, and
storage codes (`process_lost`, `server_shutdown_interrupted`,
`setup_failed`, `workspace_*`, `environment_provider_unavailable`,
`controller_lease_lost`, and others).
- `agent_error`: `adapter_failed` without a provider failure,
`invalid_output`, `invalid_response`, `tool_error`, `acpx_turn_failed`,
`test_run_failed`, `prompt_injection_blocked`,
`tool_execution_outcome_unknown`, `timeout`.
  - `unknown`: any other code, or no code.
- On Cloud direct delivery, every new emit site names the responsible
person with `withTelemetryActor()` and `resolveTelemetryPerson()`, as
`agent.task_run` does: the task creator, else the task's responsible
user, else the run owner. `task.created` uses the creating user, else
the agent's on-behalf user. `model.connected` uses the user who owns the
connection. A live request still wins. The self-hosted collector
envelope does not change.
- `failure_cause` builds on the run failure code sets in
`continuation-cause.ts`. A test fails when a code in those sets has no
telemetry code. The provider failure category decides only for
`adapter_failed`, `provider_failed`, or no code.
- Work product providers outside `paperclip`, `github`, `vercel`, and
`s3` are sent as `custom`.
- The dispatch workspace check and execution recovery write `blocked`
directly. They call the same after-commit helper, with the task's real
blockers and unblock owner. The blocker read runs in a savepoint in the
issue dependency layer, so a failed read never aborts the block write.
- Update the telemetry README with the emit rules, the after-commit
rule, and the `failure_cause` source.
- Refresh `tests/runner-e2e/source-sdk-public-inputs.json`, because its
check reported a change.

## Privacy Review

Privacy review requested. Every new dimension is a fixed enum, a
boolean, or a count. No dimension carries a title, a body, a plan step,
a connection or account name, an email, a URL, a repository name, a
prompt, an error message, or an endpoint address. `failure_cause` is a
code that the server picks. The run's error text is never sent. The
tests check that a provider message and a connection name do not reach
the event.

## Verification

- `pnpm vitest run packages/shared/src/telemetry/`: 7 files pass,
including the new `product-events.test.ts` (helper payloads, omitted
optional values, collector envelope with only registered dimensions,
retention class).
- `packages/db/src/after-transaction-commit.test.ts`: 4 pass (after
commit, rollback drop, savepoint, outside a transaction).
- `server/src/__tests__/product-telemetry.test.ts`: 14 pass. They check
that each event fires once with the right kind and company scope, that a
rollback sends nothing, that a repeated status sends nothing, that a
later plan revision sends nothing, and that a reconnect sends nothing.
- `server/src/__tests__/agent-task-run-telemetry.test.ts`: 35 pass,
including the `failure_cause` mapping table and the emit path.
- `server/src/__tests__/issue-thread-interactions-service.test.ts -t
Plan`: `plan.approved` fires for a plan confirmation and not for an
unrelated confirmation.
- `server/src/__tests__/posthog-direct-telemetry.test.ts`: the Cloud
direct path sends `company_id`, `user_id`, and `attributed_via` for
background lifecycle events, an event with no person stays on the
workspace, and `failure_cause` passes through. A self-hosted test checks
that a person scope leaves the collector events unchanged.
- Existing telemetry tests (connector, interaction, run finalizer,
heartbeat, AI connection): 299 pass.
- `pnpm --filter @paperclipai/shared typecheck`, `pnpm --filter
@paperclipai/db typecheck`, and `pnpm --filter @paperclipai/server
typecheck`: pass.
- `pnpm verify` returned exit 2 (global CI input changed). Required CI
supplies the broad coverage.

## Risks

- Merge order: merge and deploy paperclipai/telemetry-backend#150 first.
If this change ships first, the collector quarantines the new events.
The product behavior does not change.
- `afterTransactionCommit` uses the existing open-transaction marker in
the database client. A callback that a savepoint registers runs when the
outer transaction commits, even if that savepoint rolled back.
- `task.blocked` adds one small indexed read inside the update
transaction, only when a task enters `blocked`.
- `failure_cause` adds one small read for a failed run when the caller
did not pass the error code. It runs off the run's critical path, as
before.
- Low risk otherwise. No schema migration. No change to existing event
dimensions.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), through Claude Code,
with tool use (shell, Git, tests).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 19:32:27 -06:00

916 lines
37 KiB
TypeScript

import { notifyStatusCardIssueChange } from "./status-card-notifications.js";
import { assertAgentInteraction } from "./agent-interaction-policy.js";
import { notifyIssueRecovery, notifyIssueGraphRecovery } from "./issue-recovery-notifications.js";
import { randomUUID } from "node:crypto";
import { and, asc, desc, eq, inArray, sql } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import { documentRevisions, documents, issueDocuments, issues } from "@paperclipai/db";
import { ISSUE_DOCUMENT_BATCH_MAX_IDS, isSystemIssueDocumentKey, issueDocumentKeySchema } from "@paperclipai/shared";
import { conflict, notFound, payloadTooLarge, unprocessable } from "../errors.js";
import { isUniqueViolation } from "../db-errors.js";
import { insertRowsInChunks } from "./batch-insert.js";
import type { ImportIssueDocumentRow } from "./import-write-types.js";
import { emitPlanCreatedAfterCommit } from "./product-telemetry.js";
// Stay below download_artifact's 10 MiB limit, including JSON escaping and metadata.
export const ISSUE_DOCUMENT_BATCH_MAX_BYTES = 8 * 1024 * 1024;
function normalizeDocumentKey(key: string) {
const normalized = key.trim().toLowerCase();
const parsed = issueDocumentKeySchema.safeParse(normalized);
if (!parsed.success) {
throw unprocessable("Invalid document key", parsed.error.issues);
}
return parsed.data;
}
function nextAvailableDocumentKey(sourceKey: string, existingKeys: string[]) {
const usedKeys = new Set(existingKeys);
for (let index = 2; index < 1000; index += 1) {
const suffix = `-${index}`;
const baseMaxLength = 64 - suffix.length;
const base = sourceKey.slice(0, baseMaxLength).replace(/[-_]+$/g, "") || "document";
const candidate = `${base}${suffix}`;
if (!usedKeys.has(candidate) && issueDocumentKeySchema.safeParse(candidate).success) {
return candidate;
}
}
throw conflict("Unable to choose a new document key for locked document", { key: sourceKey });
}
export function extractLegacyPlanBody(description: string | null | undefined) {
if (!description) return null;
const match = /<plan>\s*([\s\S]*?)\s*<\/plan>/i.exec(description);
if (!match) return null;
const body = match[1]?.trim();
return body ? body : null;
}
export function mapIssueDocumentRow(
row: {
id: string;
companyId: string;
issueId: string;
key: string;
title: string | null;
format: string;
latestBody: string;
latestRevisionId: string | null;
latestRevisionNumber: number;
createdByAgentId: string | null;
createdByUserId: string | null;
updatedByAgentId: string | null;
updatedByUserId: string | null;
lockedAt: Date | null;
lockedByAgentId: string | null;
lockedByUserId: string | null;
sourceTrust: typeof documents.$inferSelect.sourceTrust;
createdAt: Date;
updatedAt: Date;
},
includeBody: boolean,
) {
return {
id: row.id,
companyId: row.companyId,
issueId: row.issueId,
key: row.key,
title: row.title,
format: row.format,
...(includeBody ? { body: row.latestBody } : {}),
latestRevisionId: row.latestRevisionId ?? null,
latestRevisionNumber: row.latestRevisionNumber,
createdByAgentId: row.createdByAgentId,
createdByUserId: row.createdByUserId,
updatedByAgentId: row.updatedByAgentId,
updatedByUserId: row.updatedByUserId,
lockedAt: row.lockedAt,
lockedByAgentId: row.lockedByAgentId,
lockedByUserId: row.lockedByUserId,
sourceTrust: row.sourceTrust ?? null,
createdAt: row.createdAt,
updatedAt: row.updatedAt,
};
}
export const issueDocumentSelect = {
id: documents.id,
companyId: documents.companyId,
issueId: issueDocuments.issueId,
key: issueDocuments.key,
title: documents.title,
format: documents.format,
latestBody: documents.latestBody,
latestRevisionId: documents.latestRevisionId,
latestRevisionNumber: documents.latestRevisionNumber,
createdByAgentId: documents.createdByAgentId,
createdByUserId: documents.createdByUserId,
updatedByAgentId: documents.updatedByAgentId,
updatedByUserId: documents.updatedByUserId,
lockedAt: documents.lockedAt,
lockedByAgentId: documents.lockedByAgentId,
lockedByUserId: documents.lockedByUserId,
sourceTrust: documents.sourceTrust,
createdAt: documents.createdAt,
updatedAt: documents.updatedAt,
};
export function documentService(db: Db) {
const filterSystemDocuments = <T extends { key: string }>(rows: T[], includeSystem: boolean) =>
includeSystem ? rows : rows.filter((row) => !isSystemIssueDocumentKey(row.key));
return {
getIssueDocumentPayload: async (
issue: { id: string; description: string | null },
options: { includeSystem?: boolean } = {},
) => {
const [planDocument, documentSummaries] = await Promise.all([
db
.select(issueDocumentSelect)
.from(issueDocuments)
.innerJoin(documents, eq(issueDocuments.documentId, documents.id))
.where(and(eq(issueDocuments.issueId, issue.id), eq(issueDocuments.key, "plan")))
.then((rows) => rows[0] ?? null),
db
.select({ ...issueDocumentSelect, latestBody: sql<string>`''` })
.from(issueDocuments)
.innerJoin(documents, eq(issueDocuments.documentId, documents.id))
.where(eq(issueDocuments.issueId, issue.id))
.orderBy(asc(issueDocuments.key), desc(documents.updatedAt)),
]);
const legacyPlanBody = planDocument ? null : extractLegacyPlanBody(issue.description);
return {
planDocument: planDocument ? mapIssueDocumentRow(planDocument, true) : null,
documentSummaries: filterSystemDocuments(documentSummaries, options.includeSystem ?? false)
.map((row) => mapIssueDocumentRow(row, false)),
legacyPlanDocument: legacyPlanBody
? {
key: "plan" as const,
body: legacyPlanBody,
source: "issue_description" as const,
}
: null,
};
},
listIssueDocuments: async (issueId: string, options: { includeSystem?: boolean } = {}) => {
const rows = await db
.select(issueDocumentSelect)
.from(issueDocuments)
.innerJoin(documents, eq(issueDocuments.documentId, documents.id))
.where(eq(issueDocuments.issueId, issueId))
.orderBy(asc(issueDocuments.key), desc(documents.updatedAt));
return filterSystemDocuments(rows, options.includeSystem ?? false).map((row) => mapIssueDocumentRow(row, true));
},
getIssueDocumentByKey: async (issueId: string, rawKey: string) => {
const key = normalizeDocumentKey(rawKey);
const row = await db
.select(issueDocumentSelect)
.from(issueDocuments)
.innerJoin(documents, eq(issueDocuments.documentId, documents.id))
.where(and(eq(issueDocuments.issueId, issueId), eq(issueDocuments.key, key)))
.then((rows) => rows[0] ?? null);
return row ? mapIssueDocumentRow(row, true) : null;
},
/** Caller authorizes the task set first. One query reads only the requested key. */
getIssueDocumentsByKey: async (companyId: string, issueIds: string[], rawKey: string, includeBody = true) => {
const key = normalizeDocumentKey(rawKey);
if (issueIds.length > ISSUE_DOCUMENT_BATCH_MAX_IDS) throw unprocessable("Too many issue document IDs");
if (!issueIds.length) return [];
const join = and(eq(issueDocuments.documentId, documents.id), eq(documents.companyId, companyId));
const condition = and(eq(issueDocuments.companyId, companyId), inArray(issueDocuments.issueId, issueIds), eq(issueDocuments.key, key));
const read = async (connection: Db) => {
if (includeBody) {
// PostgreSQL counts encoded JSON one body at a time; no bodies cross
// into the app before the entire set fits. The same snapshot below
// prevents a concurrent edit from growing the admitted response.
const [size] = await connection.select({ bytes: sql<string>`coalesce(sum(octet_length(to_json(${documents.latestBody})::text)), 0)` })
.from(issueDocuments).innerJoin(documents, join).where(condition);
if (Number(size.bytes) > ISSUE_DOCUMENT_BATCH_MAX_BYTES) {
throw payloadTooLarge("Document batch exceeds 8 MiB; split issueIds into smaller requests or use includeBody=false", { maxResponseBytes: ISSUE_DOCUMENT_BATCH_MAX_BYTES });
}
}
const rows = await connection
.select({ ...issueDocumentSelect, latestBody: includeBody ? documents.latestBody : sql<string>`''` })
.from(issueDocuments).innerJoin(documents, join).where(condition);
return rows.map(row => mapIssueDocumentRow(row, includeBody));
};
return includeBody
? db.transaction(tx => read(tx as unknown as Db), { isolationLevel: "repeatable read", accessMode: "read only" })
: read(db);
},
/** Bounded receipt read: never load every historical body for one revision. */
getIssueDocumentRevision: async (issueId: string, rawKey: string, revisionId: string) => {
const key = normalizeDocumentKey(rawKey);
const [revision] = await db.select({
id: documentRevisions.id, companyId: documentRevisions.companyId,
documentId: documentRevisions.documentId, issueId: issueDocuments.issueId, key: issueDocuments.key,
revisionNumber: documentRevisions.revisionNumber, title: documentRevisions.title,
format: documentRevisions.format, body: documentRevisions.body, changeSummary: documentRevisions.changeSummary,
createdByAgentId: documentRevisions.createdByAgentId, createdByUserId: documentRevisions.createdByUserId,
createdByRunId: documentRevisions.createdByRunId, createdAt: documentRevisions.createdAt,
latestRevisionId: documents.latestRevisionId, latestRevisionNumber: documents.latestRevisionNumber,
}).from(issueDocuments)
.innerJoin(documents, eq(issueDocuments.documentId, documents.id))
.innerJoin(documentRevisions, and(eq(documentRevisions.documentId, documents.id), eq(documentRevisions.companyId, documents.companyId)))
.where(and(eq(issueDocuments.issueId, issueId), eq(issueDocuments.key, key), eq(documentRevisions.id, revisionId))).limit(1);
return revision ? { ...revision, isLatest: revision.latestRevisionId === revision.id } : null;
},
listIssueDocumentRevisions: async (issueId: string, rawKey: string) => {
const key = normalizeDocumentKey(rawKey);
return db
.select({
id: documentRevisions.id,
companyId: documentRevisions.companyId,
documentId: documentRevisions.documentId,
issueId: issueDocuments.issueId,
key: issueDocuments.key,
revisionNumber: documentRevisions.revisionNumber,
title: documentRevisions.title,
format: documentRevisions.format,
body: documentRevisions.body,
changeSummary: documentRevisions.changeSummary,
createdByAgentId: documentRevisions.createdByAgentId,
createdByUserId: documentRevisions.createdByUserId,
createdByRunId: documentRevisions.createdByRunId,
createdAt: documentRevisions.createdAt,
})
.from(issueDocuments)
.innerJoin(documents, eq(issueDocuments.documentId, documents.id))
.innerJoin(documentRevisions, eq(documentRevisions.documentId, documents.id))
.where(and(eq(issueDocuments.issueId, issueId), eq(issueDocuments.key, key)))
.orderBy(desc(documentRevisions.revisionNumber));
},
upsertIssueDocument: async (input: {
issueId: string;
key: string;
title?: string | null;
format: string;
body: string;
changeSummary?: string | null;
baseRevisionId?: string | null;
createdByAgentId?: string | null;
createdByUserId?: string | null;
createdByRunId?: string | null;
sourceTrust?: typeof documents.$inferInsert.sourceTrust;
lockedDocumentStrategy?: "conflict" | "create_new_document";
}, options: { initialTaskCreation?: boolean } = {}) => {
const key = normalizeDocumentKey(input.key);
const issue = await db
.select({ id: issues.id, companyId: issues.companyId, createdByUserId: issues.createdByUserId,
responsibleUserId: issues.responsibleUserId })
.from(issues)
.where(eq(issues.id, input.issueId))
.then((rows) => rows[0] ?? null);
if (!issue) throw notFound("Issue not found");
const maxAttempts = input.lockedDocumentStrategy === "create_new_document" ? 3 : 1;
for (let attempt = 0; attempt < maxAttempts; attempt += 1) {
try {
return await db.transaction(async (tx) => {
await notifyStatusCardIssueChange(tx, issue.companyId);
const now = new Date();
const existing = await tx
.select({
id: documents.id,
companyId: documents.companyId,
issueId: issueDocuments.issueId,
key: issueDocuments.key,
title: documents.title,
format: documents.format,
latestBody: documents.latestBody,
latestRevisionId: documents.latestRevisionId,
latestRevisionNumber: documents.latestRevisionNumber,
createdByAgentId: documents.createdByAgentId,
createdByUserId: documents.createdByUserId,
updatedByAgentId: documents.updatedByAgentId,
updatedByUserId: documents.updatedByUserId,
lockedAt: documents.lockedAt,
lockedByAgentId: documents.lockedByAgentId,
lockedByUserId: documents.lockedByUserId,
sourceTrust: documents.sourceTrust,
createdAt: documents.createdAt,
updatedAt: documents.updatedAt,
})
.from(issueDocuments)
.innerJoin(documents, eq(issueDocuments.documentId, documents.id))
.where(and(eq(issueDocuments.issueId, issue.id), eq(issueDocuments.key, key)))
.then((rows) => rows[0] ?? null);
await assertAgentInteraction(tx, {
companyId: issue.companyId, actorAgentId: input.createdByAgentId,
operation: options.initialTaskCreation && !existing ? "mention" : "edit", issueId: issue.id,
texts: [
...(!existing || input.body !== existing.latestBody ? [input.body] : []),
...(input.title !== undefined && input.title !== existing?.title ? [input.title ?? ""] : []),
input.changeSummary ?? "",
], lock: true,
});
if (existing) {
if (existing.lockedAt) {
if (input.lockedDocumentStrategy === "create_new_document") {
const issueDocumentKeys = await tx
.select({ key: issueDocuments.key })
.from(issueDocuments)
.where(eq(issueDocuments.issueId, issue.id));
const fallbackKey = nextAvailableDocumentKey(key, issueDocumentKeys.map((row) => row.key));
await notifyIssueGraphRecovery(tx, { companyId: issue.companyId, issueId: issue.id });
const [document] = await tx
.insert(documents)
.values({
companyId: issue.companyId,
title: input.title ?? null,
format: input.format,
latestBody: input.body,
latestRevisionId: null,
latestRevisionNumber: 1,
createdByAgentId: input.createdByAgentId ?? null,
createdByUserId: input.createdByUserId ?? null,
updatedByAgentId: input.createdByAgentId ?? null,
updatedByUserId: input.createdByUserId ?? null,
lockedAt: null,
lockedByAgentId: null,
lockedByUserId: null,
sourceTrust: input.sourceTrust ?? null,
createdAt: now,
updatedAt: now,
})
.returning();
const [revision] = await tx
.insert(documentRevisions)
.values({
companyId: issue.companyId,
documentId: document.id,
revisionNumber: 1,
title: input.title ?? null,
format: input.format,
body: input.body,
changeSummary: input.changeSummary ?? null,
createdByAgentId: input.createdByAgentId ?? null,
createdByUserId: input.createdByUserId ?? null,
createdByRunId: input.createdByRunId ?? null,
createdAt: now,
})
.returning();
await tx
.update(documents)
.set({ latestRevisionId: revision.id })
.where(eq(documents.id, document.id));
await tx.insert(issueDocuments).values({
companyId: issue.companyId,
issueId: issue.id,
documentId: document.id,
key: fallbackKey,
createdAt: now,
updatedAt: now,
});
return {
created: true as const,
redirectedFromLockedDocument: {
id: existing.id,
key: existing.key,
},
document: {
id: document.id,
companyId: issue.companyId,
issueId: issue.id,
key: fallbackKey,
title: document.title,
format: document.format,
body: document.latestBody,
latestRevisionId: revision.id,
latestRevisionNumber: 1,
createdByAgentId: document.createdByAgentId,
createdByUserId: document.createdByUserId,
updatedByAgentId: document.updatedByAgentId,
updatedByUserId: document.updatedByUserId,
lockedAt: null,
lockedByAgentId: null,
lockedByUserId: null,
sourceTrust: document.sourceTrust ?? null,
createdAt: document.createdAt,
updatedAt: document.updatedAt,
},
};
}
throw conflict("Document is locked", {
key: existing.key,
documentId: existing.id,
lockedAt: existing.lockedAt,
});
}
if (!input.baseRevisionId) {
throw conflict("Document update requires baseRevisionId. GET the current document, read its body and latestRevisionId, then set baseRevisionId to that latestRevisionId when updating.", {
currentRevisionId: existing.latestRevisionId,
});
}
if (input.baseRevisionId !== existing.latestRevisionId) {
throw conflict("Document was updated by someone else", {
currentRevisionId: existing.latestRevisionId,
});
}
await notifyIssueGraphRecovery(tx, { companyId: issue.companyId, issueId: issue.id });
const nextRevisionNumber = existing.latestRevisionNumber + 1;
const [revision] = await tx
.insert(documentRevisions)
.values({
companyId: issue.companyId,
documentId: existing.id,
revisionNumber: nextRevisionNumber,
title: input.title ?? null,
format: input.format,
body: input.body,
changeSummary: input.changeSummary ?? null,
createdByAgentId: input.createdByAgentId ?? null,
createdByUserId: input.createdByUserId ?? null,
createdByRunId: input.createdByRunId ?? null,
createdAt: now,
})
.returning();
await tx
.update(documents)
.set({
title: input.title ?? null,
format: input.format,
latestBody: input.body,
latestRevisionId: revision.id,
latestRevisionNumber: nextRevisionNumber,
updatedByAgentId: input.createdByAgentId ?? null,
updatedByUserId: input.createdByUserId ?? null,
sourceTrust: input.sourceTrust ?? null,
updatedAt: now,
})
.where(eq(documents.id, existing.id));
await tx
.update(issueDocuments)
.set({ updatedAt: now })
.where(eq(issueDocuments.documentId, existing.id));
return {
created: false as const,
document: {
...existing,
title: input.title ?? null,
format: input.format,
body: input.body,
latestRevisionId: revision.id,
latestRevisionNumber: nextRevisionNumber,
updatedByAgentId: input.createdByAgentId ?? null,
updatedByUserId: input.createdByUserId ?? null,
lockedAt: existing.lockedAt,
lockedByAgentId: existing.lockedByAgentId,
lockedByUserId: existing.lockedByUserId,
sourceTrust: input.sourceTrust ?? null,
updatedAt: now,
},
};
}
if (input.baseRevisionId) {
throw conflict("Document does not exist yet", { key });
}
await notifyIssueGraphRecovery(tx, { companyId: issue.companyId, issueId: issue.id });
const [document] = await tx
.insert(documents)
.values({
companyId: issue.companyId,
title: input.title ?? null,
format: input.format,
latestBody: input.body,
latestRevisionId: null,
latestRevisionNumber: 1,
createdByAgentId: input.createdByAgentId ?? null,
createdByUserId: input.createdByUserId ?? null,
updatedByAgentId: input.createdByAgentId ?? null,
updatedByUserId: input.createdByUserId ?? null,
lockedAt: null,
lockedByAgentId: null,
lockedByUserId: null,
sourceTrust: input.sourceTrust ?? null,
createdAt: now,
updatedAt: now,
})
.returning();
const [revision] = await tx
.insert(documentRevisions)
.values({
companyId: issue.companyId,
documentId: document.id,
revisionNumber: 1,
title: input.title ?? null,
format: input.format,
body: input.body,
changeSummary: input.changeSummary ?? null,
createdByAgentId: input.createdByAgentId ?? null,
createdByUserId: input.createdByUserId ?? null,
createdByRunId: input.createdByRunId ?? null,
createdAt: now,
})
.returning();
await tx
.update(documents)
.set({ latestRevisionId: revision.id })
.where(eq(documents.id, document.id));
await tx.insert(issueDocuments).values({
companyId: issue.companyId,
issueId: issue.id,
documentId: document.id,
key,
createdAt: now,
updatedAt: now,
});
if (key === "plan") {
emitPlanCreatedAfterCommit({
companyId: issue.companyId,
createdByAgentId: input.createdByAgentId,
createdByUserId: input.createdByUserId,
atTaskCreation: options.initialTaskCreation === true,
task: issue,
});
}
return {
created: true as const,
document: {
id: document.id,
companyId: issue.companyId,
issueId: issue.id,
key,
title: document.title,
format: document.format,
body: document.latestBody,
latestRevisionId: revision.id,
latestRevisionNumber: 1,
createdByAgentId: document.createdByAgentId,
createdByUserId: document.createdByUserId,
updatedByAgentId: document.updatedByAgentId,
updatedByUserId: document.updatedByUserId,
lockedAt: document.lockedAt,
lockedByAgentId: document.lockedByAgentId,
lockedByUserId: document.lockedByUserId,
sourceTrust: document.sourceTrust ?? null,
createdAt: document.createdAt,
updatedAt: document.updatedAt,
},
};
});
} catch (error) {
if (isUniqueViolation(error, "issue_documents_company_issue_key_uq")) {
if (input.lockedDocumentStrategy === "create_new_document" && attempt < maxAttempts - 1) {
continue;
}
throw conflict("Document key already exists on this issue", { key });
}
throw error;
}
}
throw conflict("Unable to choose a new document key for locked document", { key });
},
/**
* Batched issue-document insert for company import.
*
* Every imported document is a fresh create (the issue is brand new), so we
* skip {@link upsertIssueDocument}'s per-row existence/lock/base-revision
* dance and the follow-up latest-revision update: ids are pre-generated so
* `latest_revision_id` can be written inline. Documents, their initial
* revisions, and the issue links are each inserted in chunked statements.
*/
createIssueDocumentsForImport: async (rows: ImportIssueDocumentRow[]): Promise<void> => {
if (rows.length === 0) return;
const now = new Date();
const documentRows: Array<Record<string, unknown>> = [];
const revisionRows: Array<Record<string, unknown>> = [];
const linkRows: Array<Record<string, unknown>> = [];
for (const row of rows) {
const key = normalizeDocumentKey(row.key);
const documentId = randomUUID();
const revisionId = randomUUID();
documentRows.push({
id: documentId,
companyId: row.companyId,
title: row.title ?? null,
format: row.format,
latestBody: row.body,
latestRevisionId: revisionId,
latestRevisionNumber: 1,
createdByAgentId: row.createdByAgentId ?? null,
createdByUserId: row.createdByUserId ?? null,
updatedByAgentId: row.createdByAgentId ?? null,
updatedByUserId: row.createdByUserId ?? null,
lockedAt: null,
lockedByAgentId: null,
lockedByUserId: null,
sourceTrust: row.sourceTrust ?? null,
createdAt: now,
updatedAt: now,
});
revisionRows.push({
id: revisionId,
companyId: row.companyId,
documentId,
revisionNumber: 1,
title: row.title ?? null,
format: row.format,
body: row.body,
changeSummary: null,
createdByAgentId: row.createdByAgentId ?? null,
createdByUserId: row.createdByUserId ?? null,
createdByRunId: row.createdByRunId ?? null,
createdAt: now,
});
linkRows.push({
companyId: row.companyId,
issueId: row.issueId,
documentId,
key,
createdAt: now,
updatedAt: now,
});
}
await db.transaction(async (tx) => {
for (const companyId of new Set(rows.map(row => row.companyId))) {
await notifyStatusCardIssueChange(tx, companyId);
await notifyIssueRecovery(tx, { companyId });
}
await insertRowsInChunks(tx, documents, documentRows);
await insertRowsInChunks(tx, documentRevisions, revisionRows);
await insertRowsInChunks(tx, issueDocuments, linkRows);
});
},
restoreIssueDocumentRevision: async (input: {
issueId: string;
key: string;
revisionId: string;
createdByAgentId?: string | null;
createdByUserId?: string | null;
}) => {
const key = normalizeDocumentKey(input.key);
return db.transaction(async (tx) => {
const existing = await tx
.select(issueDocumentSelect)
.from(issueDocuments)
.innerJoin(documents, eq(issueDocuments.documentId, documents.id))
.where(and(eq(issueDocuments.issueId, input.issueId), eq(issueDocuments.key, key)))
.then((rows) => rows[0] ?? null);
if (!existing) throw notFound("Document not found");
if (existing.lockedAt) {
throw conflict("Document is locked", {
key: existing.key,
documentId: existing.id,
lockedAt: existing.lockedAt,
});
}
const revision = await tx
.select({
id: documentRevisions.id,
companyId: documentRevisions.companyId,
documentId: documentRevisions.documentId,
revisionNumber: documentRevisions.revisionNumber,
title: documentRevisions.title,
format: documentRevisions.format,
body: documentRevisions.body,
})
.from(documentRevisions)
.where(and(eq(documentRevisions.id, input.revisionId), eq(documentRevisions.documentId, existing.id)))
.then((rows) => rows[0] ?? null);
if (!revision) throw notFound("Document revision not found");
await assertAgentInteraction(tx, {
companyId: existing.companyId, actorAgentId: input.createdByAgentId,
operation: "edit", issueId: input.issueId, texts: [revision.body, revision.title ?? ""], lock: true,
});
if (existing.latestRevisionId === revision.id) {
throw conflict("Selected revision is already the latest revision", {
currentRevisionId: existing.latestRevisionId,
});
}
await notifyIssueGraphRecovery(tx, { companyId: existing.companyId, issueId: existing.issueId });
const now = new Date();
const nextRevisionNumber = existing.latestRevisionNumber + 1;
const [restoredRevision] = await tx
.insert(documentRevisions)
.values({
companyId: existing.companyId,
documentId: existing.id,
revisionNumber: nextRevisionNumber,
title: revision.title ?? null,
format: revision.format,
body: revision.body,
changeSummary: `Restored from revision ${revision.revisionNumber}`,
createdByAgentId: input.createdByAgentId ?? null,
createdByUserId: input.createdByUserId ?? null,
createdAt: now,
})
.returning();
await notifyStatusCardIssueChange(tx, existing.companyId);
await tx
.update(documents)
.set({
title: revision.title ?? null,
format: revision.format,
latestBody: revision.body,
latestRevisionId: restoredRevision.id,
latestRevisionNumber: nextRevisionNumber,
updatedByAgentId: input.createdByAgentId ?? null,
updatedByUserId: input.createdByUserId ?? null,
updatedAt: now,
})
.where(eq(documents.id, existing.id));
await tx
.update(issueDocuments)
.set({ updatedAt: now })
.where(eq(issueDocuments.documentId, existing.id));
return {
restoredFromRevisionId: revision.id,
restoredFromRevisionNumber: revision.revisionNumber,
document: {
...existing,
title: revision.title ?? null,
format: revision.format,
body: revision.body,
latestRevisionId: restoredRevision.id,
latestRevisionNumber: nextRevisionNumber,
updatedByAgentId: input.createdByAgentId ?? null,
updatedByUserId: input.createdByUserId ?? null,
updatedAt: now,
},
};
});
},
lockIssueDocument: async (input: {
issueId: string;
key: string;
lockedByAgentId?: string | null;
lockedByUserId?: string | null;
}) => {
const key = normalizeDocumentKey(input.key);
return db.transaction(async (tx) => {
const existing = await tx
.select(issueDocumentSelect)
.from(issueDocuments)
.innerJoin(documents, eq(issueDocuments.documentId, documents.id))
.where(and(eq(issueDocuments.issueId, input.issueId), eq(issueDocuments.key, key)))
.then((rows) => rows[0] ?? null);
if (!existing) throw notFound("Document not found");
await assertAgentInteraction(tx, {
companyId: existing.companyId, actorAgentId: input.lockedByAgentId,
operation: "edit", issueId: input.issueId, lock: true,
});
if (existing.lockedAt) {
return {
changed: false as const,
document: mapIssueDocumentRow(existing, true),
};
}
await notifyIssueGraphRecovery(tx, { companyId: existing.companyId, issueId: existing.issueId });
const now = new Date();
await tx
.update(documents)
.set({
lockedAt: now,
lockedByAgentId: input.lockedByAgentId ?? null,
lockedByUserId: input.lockedByUserId ?? null,
updatedAt: now,
})
.where(eq(documents.id, existing.id));
await tx
.update(issueDocuments)
.set({ updatedAt: now })
.where(eq(issueDocuments.documentId, existing.id));
return {
changed: true as const,
document: {
...mapIssueDocumentRow(existing, true),
lockedAt: now,
lockedByAgentId: input.lockedByAgentId ?? null,
lockedByUserId: input.lockedByUserId ?? null,
updatedAt: now,
},
};
});
},
unlockIssueDocument: async (issueId: string, rawKey: string, actorAgentId?: string | null) => {
const key = normalizeDocumentKey(rawKey);
return db.transaction(async (tx) => {
const existing = await tx
.select(issueDocumentSelect)
.from(issueDocuments)
.innerJoin(documents, eq(issueDocuments.documentId, documents.id))
.where(and(eq(issueDocuments.issueId, issueId), eq(issueDocuments.key, key)))
.then((rows) => rows[0] ?? null);
if (!existing) throw notFound("Document not found");
await assertAgentInteraction(tx, {
companyId: existing.companyId, actorAgentId, operation: "edit", issueId, lock: true,
});
if (!existing.lockedAt) {
return {
changed: false as const,
document: mapIssueDocumentRow(existing, true),
};
}
await notifyIssueGraphRecovery(tx, { companyId: existing.companyId, issueId: existing.issueId });
const now = new Date();
await tx
.update(documents)
.set({
lockedAt: null,
lockedByAgentId: null,
lockedByUserId: null,
updatedAt: now,
})
.where(eq(documents.id, existing.id));
await tx
.update(issueDocuments)
.set({ updatedAt: now })
.where(eq(issueDocuments.documentId, existing.id));
return {
changed: true as const,
document: {
...mapIssueDocumentRow(existing, true),
lockedAt: null,
lockedByAgentId: null,
lockedByUserId: null,
updatedAt: now,
},
};
});
},
deleteIssueDocument: async (issueId: string, rawKey: string, actorAgentId?: string | null) => {
const key = normalizeDocumentKey(rawKey);
return db.transaction(async (tx) => {
const existing = await tx
.select(issueDocumentSelect)
.from(issueDocuments)
.innerJoin(documents, eq(issueDocuments.documentId, documents.id))
.where(and(eq(issueDocuments.issueId, issueId), eq(issueDocuments.key, key)))
.then((rows) => rows[0] ?? null);
if (!existing) return null;
await assertAgentInteraction(tx, {
companyId: existing.companyId, actorAgentId, operation: "edit", issueId, lock: true,
});
if (existing.lockedAt) {
throw conflict("Document is locked", {
key: existing.key,
documentId: existing.id,
lockedAt: existing.lockedAt,
});
}
await notifyIssueGraphRecovery(tx, { companyId: existing.companyId, issueId: existing.issueId });
await tx.delete(issueDocuments).where(eq(issueDocuments.documentId, existing.id));
await notifyStatusCardIssueChange(tx, existing.companyId);
await tx.delete(documents).where(eq(documents.id, existing.id));
return {
...existing,
body: existing.latestBody,
latestRevisionId: existing.latestRevisionId ?? null,
};
});
},
};
}