mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-22 16:51:28 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent runs need durable records so Paperclip can explain results and final status changes. > - The current heartbeat tables support direct adapters, but they do not model native runner evidence. > - The runner transport and server coordinator must share a strict finalization contract before they write production data. > - This pull request adds that contract and its additive database boundary. > - It does not select the Paperclip Runner or change any existing adapter execution path. > - The benefit is a reviewable persistence layer that preserves all current behavior and supports later guarded integration. ## Linked Issues or Issue Description Refs #11962 Refs #12129 ## What Changed - Add native run result, finalization, completion, assessment, status decision, and status effect tables. - Add inert native metadata to heartbeat runs and events. Keep `legacy` as the default runtime mode. - Bind each evidence relationship to one company, issue, run, contract, result, assessment, and decision with composite constraints. - Add a strict `paperclip.native_finalization.v1` shared type and validator. - Preserve database functions, triggers, and the unique indexes required by foreign keys in JavaScript backups. - Add migration, backup, mixed-owner denial, validator, and direct-adapter compatibility tests. - Document the new records and their ownership rules. ## Verification - Run `pnpm -r typecheck`. - Run `pnpm build`. - Run `pnpm db:generate`. The schema output and migration safety checks remain current. - Run `PAPERCLIP_PSQL_PATH=/Applications/Postgres.app/Contents/Versions/latest/bin/psql pnpm exec vitest run packages/shared/src/validators/native-finalization.test.ts packages/db/src/client.test.ts packages/db/src/backup-lib.test.ts server/src/__tests__/heartbeat-workspace-busy.test.ts server/src/__tests__/heartbeat-comment-wake-batching.test.ts`. All 52 tests pass. - The full local `pnpm test:run` run completed 4,688 tests. It found 30 existing macOS test-environment failures. A serial rerun with the canonical `/private/tmp` path reduced those failures to six existing listener-diagnostics and skill-browser cases. None of those suites use files in this change. - The full Linux GitHub Actions matrix passes. This includes all general-server, serialized-server, workspace, browser, build, typecheck, canary, and aggregate verification jobs. - Greptile passes at 5/5. Contributor trust, Superagent, Socket, and Snyk pass with no finding from this change. - Storybook visual regression skips by path because this pull request has no UI or Storybook change. - Confirm that the diff contains 25 files. Confirm that it contains no workflow or `pnpm-lock.yaml` changes. ## Risks - The migration adds tables, columns, indexes, a function, a trigger, and ownership constraints. It does not remove or rename existing data. - Composite foreign keys reject mixed-company, mixed-issue, and mixed-run evidence even when each ID exists. - The status-version trigger runs only when an issue status changes. Backup tests confirm that restore retains this trigger and its dependencies. - Native source identifiers are unique when present. Legacy event rows remain unchanged. - This change does not add a unique run sequence constraint. The later native writer must allocate its sequence atomically before that invariant can be safe. - Existing adapters keep their current execution and finalization paths. New heartbeat runs default to `legacy` mode. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with GPT-5. The exact deployment ID and context-window size are not exposed. The model used agentic reasoning, repository tools, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
185 lines
9.9 KiB
TypeScript
185 lines
9.9 KiB
TypeScript
import { sql } from "drizzle-orm";
|
|
import {
|
|
type AnyPgColumn,
|
|
pgTable,
|
|
uuid,
|
|
text,
|
|
timestamp,
|
|
integer,
|
|
jsonb,
|
|
index,
|
|
uniqueIndex,
|
|
unique,
|
|
bigint,
|
|
} from "drizzle-orm/pg-core";
|
|
import { agents } from "./agents.js";
|
|
import { projects } from "./projects.js";
|
|
import { goals } from "./goals.js";
|
|
import { companies } from "./companies.js";
|
|
import { heartbeatRuns } from "./heartbeat_runs.js";
|
|
import { projectWorkspaces } from "./project_workspaces.js";
|
|
import { executionWorkspaces } from "./execution_workspaces.js";
|
|
import type { IssueReviewPolicy, IssueUnblockDescriptor, SourceTrustMetadata } from "@paperclipai/shared";
|
|
|
|
export const issues = pgTable(
|
|
"issues",
|
|
{
|
|
id: uuid("id").primaryKey().defaultRandom(),
|
|
companyId: uuid("company_id").notNull().references(() => companies.id),
|
|
projectId: uuid("project_id").references(() => projects.id),
|
|
projectWorkspaceId: uuid("project_workspace_id").references(() => projectWorkspaces.id, { onDelete: "set null" }),
|
|
goalId: uuid("goal_id").references(() => goals.id),
|
|
parentId: uuid("parent_id").references((): AnyPgColumn => issues.id),
|
|
title: text("title").notNull(),
|
|
description: text("description"),
|
|
status: text("status").notNull().default("backlog"),
|
|
statusVersion: bigint("status_version", { mode: "number" }).notNull().default(0),
|
|
lastStatusDecisionId: uuid("last_status_decision_id"),
|
|
workMode: text("work_mode").notNull().default("standard"),
|
|
harnessKind: text("harness_kind"),
|
|
priority: text("priority").notNull().default("medium"),
|
|
reviewPolicy: text("review_policy").$type<IssueReviewPolicy>(),
|
|
assigneeAgentId: uuid("assignee_agent_id").references(() => agents.id),
|
|
assigneeUserId: text("assignee_user_id"),
|
|
checkoutRunId: uuid("checkout_run_id").references(() => heartbeatRuns.id, { onDelete: "set null" }),
|
|
executionRunId: uuid("execution_run_id").references(() => heartbeatRuns.id, { onDelete: "set null" }),
|
|
executionAgentNameKey: text("execution_agent_name_key"),
|
|
executionLockedAt: timestamp("execution_locked_at", { withTimezone: true }),
|
|
createdByAgentId: uuid("created_by_agent_id").references(() => agents.id),
|
|
createdByUserId: text("created_by_user_id"),
|
|
responsibleUserId: text("responsible_user_id"),
|
|
issueNumber: integer("issue_number"),
|
|
identifier: text("identifier"),
|
|
originKind: text("origin_kind").notNull().default("manual"),
|
|
originId: text("origin_id"),
|
|
originRunId: text("origin_run_id"),
|
|
originFingerprint: text("origin_fingerprint").notNull().default("default"),
|
|
requestDepth: integer("request_depth").notNull().default(0),
|
|
billingCode: text("billing_code"),
|
|
assigneeAdapterOverrides: jsonb("assignee_adapter_overrides").$type<Record<string, unknown>>(),
|
|
executionPolicy: jsonb("execution_policy").$type<Record<string, unknown>>(),
|
|
executionState: jsonb("execution_state").$type<Record<string, unknown>>(),
|
|
monitorNextCheckAt: timestamp("monitor_next_check_at", { withTimezone: true }),
|
|
monitorWakeRequestedAt: timestamp("monitor_wake_requested_at", { withTimezone: true }),
|
|
monitorLastTriggeredAt: timestamp("monitor_last_triggered_at", { withTimezone: true }),
|
|
monitorAttemptCount: integer("monitor_attempt_count").notNull().default(0),
|
|
monitorNotes: text("monitor_notes"),
|
|
monitorScheduledBy: text("monitor_scheduled_by"),
|
|
executionWorkspaceId: uuid("execution_workspace_id")
|
|
.references((): AnyPgColumn => executionWorkspaces.id, { onDelete: "set null" }),
|
|
executionWorkspacePreference: text("execution_workspace_preference"),
|
|
executionWorkspaceSettings: jsonb("execution_workspace_settings").$type<Record<string, unknown>>(),
|
|
sourceTrust: jsonb("source_trust").$type<SourceTrustMetadata | null>(),
|
|
unblockDescriptor: jsonb("unblock_descriptor").$type<IssueUnblockDescriptor | null>(),
|
|
blockedTransitionAt: timestamp("blocked_transition_at", { withTimezone: true }),
|
|
blockedOwnerNotifiedAt: timestamp("blocked_owner_notified_at", { withTimezone: true }),
|
|
startedAt: timestamp("started_at", { withTimezone: true }),
|
|
completedAt: timestamp("completed_at", { withTimezone: true }),
|
|
cancelledAt: timestamp("cancelled_at", { withTimezone: true }),
|
|
hiddenAt: timestamp("hidden_at", { withTimezone: true }),
|
|
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
|
|
updatedAt: timestamp("updated_at", { withTimezone: true }).notNull().defaultNow(),
|
|
},
|
|
(table) => ({
|
|
companyIdUq: unique("issues_company_id_uq").on(table.companyId, table.id),
|
|
companyStatusIdx: index("issues_company_status_idx").on(table.companyId, table.status),
|
|
companyHarnessKindIdx: index("issues_company_harness_kind_idx").on(table.companyId, table.harnessKind),
|
|
assigneeStatusIdx: index("issues_company_assignee_status_idx").on(
|
|
table.companyId,
|
|
table.assigneeAgentId,
|
|
table.status,
|
|
),
|
|
assigneeUserStatusIdx: index("issues_company_assignee_user_status_idx").on(
|
|
table.companyId,
|
|
table.assigneeUserId,
|
|
table.status,
|
|
),
|
|
responsibleUserIdx: index("issues_company_responsible_user_idx").on(table.companyId, table.responsibleUserId),
|
|
parentIdx: index("issues_company_parent_idx").on(table.companyId, table.parentId),
|
|
projectIdx: index("issues_company_project_idx").on(table.companyId, table.projectId),
|
|
originIdx: index("issues_company_origin_idx").on(table.companyId, table.originKind, table.originId),
|
|
projectWorkspaceIdx: index("issues_company_project_workspace_idx").on(table.companyId, table.projectWorkspaceId),
|
|
executionWorkspaceIdx: index("issues_company_execution_workspace_idx").on(table.companyId, table.executionWorkspaceId),
|
|
dueMonitorIdx: index("issues_company_monitor_due_idx").on(table.companyId, table.monitorNextCheckAt),
|
|
companyUpdatedIdx: index("issues_company_updated_idx").on(table.companyId, table.updatedAt),
|
|
companyCreatedIdx: index("issues_company_created_idx").on(table.companyId, table.createdAt),
|
|
openNormalizedTitleCreatedIdx: index("issues_open_normalized_title_created_idx")
|
|
.on(
|
|
table.companyId,
|
|
table.parentId,
|
|
sql`lower(regexp_replace(btrim(${table.title}), '\\s+', ' ', 'g'))`,
|
|
table.createdAt,
|
|
)
|
|
.where(sql`${table.hiddenAt} is null and ${table.status} not in ('done', 'cancelled')`),
|
|
companyPriorityIdx: index("issues_company_priority_idx").on(table.companyId, table.priority),
|
|
identifierIdx: uniqueIndex("issues_identifier_idx").on(table.identifier),
|
|
titleSearchIdx: index("issues_title_search_idx").using("gin", table.title.op("gin_trgm_ops")),
|
|
identifierSearchIdx: index("issues_identifier_search_idx").using("gin", table.identifier.op("gin_trgm_ops")),
|
|
descriptionSearchIdx: index("issues_description_search_idx").using("gin", table.description.op("gin_trgm_ops")),
|
|
openRoutineExecutionIdx: uniqueIndex("issues_open_routine_execution_uq")
|
|
.on(table.companyId, table.originKind, table.originId, table.originFingerprint)
|
|
.where(
|
|
sql`${table.originKind} = 'routine_execution'
|
|
and ${table.originId} is not null
|
|
and ${table.hiddenAt} is null
|
|
and ${table.executionRunId} is not null
|
|
and ${table.status} in ('backlog', 'todo', 'in_progress', 'in_review', 'blocked')`,
|
|
),
|
|
activeLivenessRecoveryIncidentIdx: uniqueIndex("issues_active_liveness_recovery_incident_uq")
|
|
.on(table.companyId, table.originKind, table.originId)
|
|
.where(
|
|
sql`${table.originKind} = 'harness_liveness_escalation'
|
|
and ${table.originId} is not null
|
|
and ${table.hiddenAt} is null
|
|
and ${table.status} not in ('done', 'cancelled')`,
|
|
),
|
|
activeLivenessRecoveryLeafIdx: uniqueIndex("issues_active_liveness_recovery_leaf_uq")
|
|
.on(table.companyId, table.originKind, table.originFingerprint)
|
|
.where(
|
|
sql`${table.originKind} = 'harness_liveness_escalation'
|
|
and ${table.originFingerprint} <> 'default'
|
|
and ${table.hiddenAt} is null
|
|
and ${table.status} not in ('done', 'cancelled')`,
|
|
),
|
|
activeStaleRunEvaluationIdx: uniqueIndex("issues_active_stale_run_evaluation_uq")
|
|
.on(table.companyId, table.originKind, table.originId)
|
|
.where(
|
|
sql`${table.originKind} = 'stale_active_run_evaluation'
|
|
and ${table.originId} is not null
|
|
and ${table.hiddenAt} is null
|
|
and ${table.status} not in ('done', 'cancelled')`,
|
|
),
|
|
activeTaskWatchdogIdx: uniqueIndex("issues_active_task_watchdog_uq")
|
|
.on(table.companyId, table.originKind, table.originId)
|
|
.where(
|
|
sql`${table.originKind} = 'task_watchdog'
|
|
and ${table.originId} is not null
|
|
and ${table.hiddenAt} is null
|
|
and ${table.status} not in ('done', 'cancelled')`,
|
|
),
|
|
activeProductivityReviewIdx: uniqueIndex("issues_active_productivity_review_uq")
|
|
.on(table.companyId, table.originKind, table.originId)
|
|
.where(
|
|
sql`${table.originKind} = 'issue_productivity_review'
|
|
and ${table.originId} is not null
|
|
and ${table.hiddenAt} is null
|
|
and ${table.status} not in ('done', 'cancelled')`,
|
|
),
|
|
activeStrandedIssueRecoveryIdx: uniqueIndex("issues_active_stranded_issue_recovery_uq")
|
|
.on(table.companyId, table.originKind, table.originId)
|
|
.where(
|
|
sql`${table.originKind} = 'stranded_issue_recovery'
|
|
and ${table.originId} is not null
|
|
and ${table.hiddenAt} is null
|
|
and ${table.status} not in ('done', 'cancelled')`,
|
|
),
|
|
// The onboarding first-task origin grants privileged behavior (agent-attributed
|
|
// greeting, description suppression), so at most one issue per company may ever
|
|
// carry it — concurrent creates race on the pre-insert count check and this
|
|
// index is what atomically rejects the loser.
|
|
onboardingFirstTaskIdx: uniqueIndex("issues_onboarding_first_task_uq")
|
|
.on(table.companyId)
|
|
.where(sql`${table.originKind} = 'onboarding_first_task'`),
|
|
}),
|
|
);
|