mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 16:02:36 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents split work into tasks. A task can have a parent task. > - Today agents can nest tasks to any depth: parent, child, grandchild, and deeper. > - Deep agent-made trees are hard to follow, and an agent that works on a child task makes grandchildren by default. > - This pull request limits agent-created trees to two levels: a top-level task and its children. > - A request for a grandchild does not fail. The server puts the new task beside the requester and tells the agent. > - Board users keep full control. They can still nest to any depth. Existing deep trees stay valid. > - The pull request also records which task created each new task, so the audit trail stays complete when the parent changes. ## Linked Issues or Issue Description No public issue exists. This is the feature description. Related: #16853 (budgets that include descendant work) also reasons about task trees; this change does not alter its traversal. **Subsystem affected** Issues (tasks): create, child create and move in the issue service and routes. Agent tools, chat suggestions and creation provenance use these paths. **Problem or motivation** Agents create subtasks as deep as they want. An agent that runs on a child task gets that child as the default parent, so its follow-ups become grandchildren. The trees become deep and hard to read. The parent link is also the only record of which task created a follow-up, so that record is lost when the parent changes. **Proposed solution** Limit agent-created trees to a parent and its children. When an agent asks for a grandchild, put the new task under the requester's parent and return `parentAdjusted`. Do not reject the request. Keep board users unrestricted. Record the originating task in a new server-set `origin_issue_id` column. Keep `origin_run_id` as the originating run. **Alternatives considered** Reject grandchild requests with an error. We did not do this, because agents then fail mid-task. Add a database constraint. We did not do this, because existing deep trees must stay valid and nesting can return later. ## What Changed - Add `MAX_AGENT_TASK_HIERARCHY_DEPTH` (2) and the `IssueParentAdjustment` type in `packages/shared`. - Add `resolveNewTaskParent` and `planIssueParentChange` in `server/src/services/issues.ts`. `create`, `createChild` and `update` use the agent limit by default (`parentDepthPolicy: "flatten"`). - The issue create, `/children` and PATCH routes pass `allow_nested` for board users. For agents, the routes check authorization against the parent that the server actually writes. - The requester keeps its behavior when the server moves a task beside it: `blockParentUntilDone` blocks the requester, and the new task inherits the requester's workspace and privacy. - The server never moves a task above a low-trust boundary root that contains the requester. - These subsystems keep any depth, because they find their work by parent: watchdog reviews, monitor recovery, email-channel placement, and routines that a person configured. - An agent move of a task that has children skips only the parent change and returns `reason: "has_children"`. - Add `issues.origin_issue_id` (nullable, `on delete set null`, indexed) in migration `0434`. Only the server sets it. The `createdFrom` detail field, the `createdFromIssueId` filter and task-creation provenance read it first. - Chat prompts now let a chat agent create one task, a task with children, or separate tasks. The conversation is never a parent. - The runner `create_task` tool accepts an optional `parentTaskId`. A chat agent can create a top-level task and then put subtasks under it. The run must be able to read and change the parent. A conversation is never a parent. The depth limit still applies. Project tools map the field to `parentId` through the issue route. The protocol manifest, semantic contracts and eval fixture are regenerated. - Watchdog runs that have an active watchdog keep their requested parent on all create paths, because their access is limited to the watched subtree. - Fix a bug that the test drive found: the chat (MCP) `create_task` ignored the requested `status`, so a `backlog` task became `todo` and started to run. It now uses the same rule as the native tool. - Update `skills/paperclip`, `doc/SPEC-implementation.md`, and add the plan `doc/plans/2026-11-05-two-level-task-hierarchy.md`. ## Verification - `cd server && npx vitest run src/__tests__/agent-task-hierarchy-depth.test.ts src/__tests__/agent-task-hierarchy-depth-routes.test.ts`: 19 new tests for service and route behavior, all pass. - Related suites (64 files: issues service, thread interactions, privacy, watchdogs, provenance, runner tool authority, decisions, routines, project tools): 1208 tests pass. Two tests changed because of the intended behavior and pass now. Two `email-channels.integration` tests fail locally because the test cannot find a bundled skill file from this worktree path. This change does not touch that code. - `packages/db`: migration safety, snapshot drift and journal tests pass. - Typecheck passes for `server`, `packages/shared`, `packages/db`, `cli` and `ui` (local `ui` errors come only from generated Storybook catalog files that are absent). - Local test drive on an isolated instance with real Claude agents: 29 of 29 use cases pass, including an agent on a child task (follow-ups placed beside it, with the notice) and a chat that creates a top-level task with three subtasks. - I did not run the full `pnpm test:run` or `pnpm build` locally. CI runs them. ## Risks - Behavior change for agents: some follow-ups that were grandchildren are now siblings. The parent of the requester is woken when all its children finish, which now includes these tasks. - The runner `create_task` input contract adds one optional field, so the protocol manifest and the pack contract change. The tool output does not change: it shows the actual `parentId`, and the activity log records `parentAdjusted`. - Migration `0434` adds a nullable column, a foreign key and an index. It does not backfill rows. - `IssueCreationSource.run` is now nullable. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, local test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2290 lines
91 KiB
TypeScript
2290 lines
91 KiB
TypeScript
import { createHash, randomUUID } from "node:crypto";
|
|
import fs from "node:fs";
|
|
import { tmpdir } from "node:os";
|
|
import { join } from "node:path";
|
|
import { drizzle } from "drizzle-orm/postgres-js";
|
|
import { migrate } from "drizzle-orm/postgres-js/migrator";
|
|
import { afterEach, describe, expect, it } from "vitest";
|
|
import postgres from "postgres";
|
|
import {
|
|
DEFAULT_DATABASE_APPLICATION_NAME,
|
|
applyPendingMigrations,
|
|
closeRegisteredClients,
|
|
createDb,
|
|
ensurePostgresDatabase,
|
|
inspectMigrations,
|
|
resetPostgresDatabase,
|
|
} from "./client.js";
|
|
import {
|
|
EMBEDDED_POSTGRES_TEST_TIMEOUT_MS,
|
|
getEmbeddedPostgresTestSupport,
|
|
startEmbeddedPostgresTestDatabase,
|
|
} from "./test-embedded-postgres.js";
|
|
|
|
const cleanups: Array<() => Promise<void>> = [];
|
|
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
|
|
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
|
|
|
|
async function createTempDatabase(): Promise<string> {
|
|
const db = await startEmbeddedPostgresTestDatabase("paperclip-db-client-");
|
|
cleanups.push(db.cleanup);
|
|
return db.connectionString;
|
|
}
|
|
|
|
async function migrationHash(migrationFile: string): Promise<string> {
|
|
const content = await fs.promises.readFile(
|
|
new URL(`./migrations/${migrationFile}`, import.meta.url),
|
|
"utf8",
|
|
);
|
|
return createHash("sha256").update(content).digest("hex");
|
|
}
|
|
|
|
const userVisibleUpdatedAtTables = new Set([
|
|
"companies",
|
|
"heartbeat_runs",
|
|
"issue_comments",
|
|
"issues",
|
|
"routine_runs",
|
|
"routines",
|
|
]);
|
|
|
|
const migrationUpdatedAtUpdateAllowlist = new Map<string, ReadonlySet<string>>([
|
|
[
|
|
"0105_instance_scoped_environments.sql",
|
|
new Set(["issues"]),
|
|
],
|
|
[
|
|
"0131_repair_run_responsible_user_context_refs.sql",
|
|
new Set(["heartbeat_runs"]),
|
|
],
|
|
[
|
|
"0135_repair_run_responsible_user_updated_at_sweep.sql",
|
|
new Set(["companies", "heartbeat_runs", "issues", "routine_runs", "routines"]),
|
|
],
|
|
]);
|
|
|
|
function findUserVisibleUpdatedAtBackfillViolations(
|
|
migrationFile: string,
|
|
content: string,
|
|
): string[] {
|
|
const allowedTables = migrationUpdatedAtUpdateAllowlist.get(migrationFile) ?? new Set<string>();
|
|
const violations: string[] = [];
|
|
|
|
for (const statement of content.split("--> statement-breakpoint")) {
|
|
const updateMatch = statement.match(/\bUPDATE\s+"([^"]+)"/i);
|
|
if (!updateMatch) continue;
|
|
|
|
const tableName = updateMatch[1];
|
|
if (!userVisibleUpdatedAtTables.has(tableName)) continue;
|
|
if (!/\bSET\b[\s\S]*"updated_at"\s*=/i.test(statement)) continue;
|
|
if (allowedTables.has(tableName)) continue;
|
|
|
|
violations.push(`${migrationFile}: UPDATE "${tableName}" sets updated_at`);
|
|
}
|
|
|
|
return violations;
|
|
}
|
|
|
|
afterEach(async () => {
|
|
while (cleanups.length > 0) {
|
|
const cleanup = cleanups.pop();
|
|
await cleanup?.();
|
|
}
|
|
});
|
|
|
|
if (!embeddedPostgresSupport.supported) {
|
|
console.warn(
|
|
`Skipping embedded Postgres migration tests on this host: ${embeddedPostgresSupport.reason ?? "unsupported environment"}`,
|
|
);
|
|
}
|
|
|
|
describeEmbeddedPostgres("createDb pool defaults", () => {
|
|
it("names its backends and closes them once idle", async () => {
|
|
const url = await createTempDatabase();
|
|
const observer = postgres(url, { max: 1, onnotice: () => {} });
|
|
cleanups.push(async () => {
|
|
await observer.end({ timeout: 1 });
|
|
});
|
|
|
|
const backendsNamed = async (name: string) => {
|
|
const rows = await observer`
|
|
select count(*)::int as count from pg_stat_activity where application_name = ${name}
|
|
`;
|
|
return rows[0]?.count ?? 0;
|
|
};
|
|
|
|
const db = createDb(url);
|
|
cleanups.push(async () => {
|
|
await db.$client.end({ timeout: 1 });
|
|
});
|
|
const [self] = await db.$client`select application_name from pg_stat_activity where pid = pg_backend_pid()`;
|
|
expect(self?.application_name).toBe(DEFAULT_DATABASE_APPLICATION_NAME);
|
|
|
|
const shortLived = createDb(url, { applicationName: "paperclip-idle-test", idleTimeoutSeconds: 1 });
|
|
cleanups.push(async () => {
|
|
await shortLived.$client.end({ timeout: 1 });
|
|
});
|
|
await shortLived.$client`select 1`;
|
|
expect(await backendsNamed("paperclip-idle-test")).toBe(1);
|
|
|
|
// The driver closes the idle connection after `idle_timeout`; without the
|
|
// option (the driver default) the backend would stay until the process
|
|
// exits. Wait past the timeout, then poll PostgreSQL's own view.
|
|
const deadline = Date.now() + 10_000;
|
|
let remaining = await backendsNamed("paperclip-idle-test");
|
|
while (remaining > 0 && Date.now() < deadline) {
|
|
await new Promise((resolve) => setTimeout(resolve, 250));
|
|
remaining = await backendsNamed("paperclip-idle-test");
|
|
}
|
|
expect(remaining).toBe(0);
|
|
}, 30_000);
|
|
});
|
|
|
|
describeEmbeddedPostgres("resetPostgresDatabase", () => {
|
|
it("recreates an existing database so stale tables are removed", async () => {
|
|
const connectionString = await createTempDatabase();
|
|
const adminUrl = new URL(connectionString);
|
|
const databaseName = adminUrl.pathname.replace(/^\//, "");
|
|
adminUrl.pathname = "/postgres";
|
|
|
|
const setupSql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
await setupSql.unsafe(`CREATE TABLE stale_reseed_target_only (id integer PRIMARY KEY)`);
|
|
} finally {
|
|
await setupSql.end();
|
|
}
|
|
|
|
await resetPostgresDatabase(adminUrl.toString(), databaseName);
|
|
|
|
const verifySql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const rows = await verifySql.unsafe<{ stale_table: string | null }[]>(
|
|
`SELECT to_regclass('public.stale_reseed_target_only')::text AS stale_table`,
|
|
);
|
|
expect(rows[0]?.stale_table).toBeNull();
|
|
} finally {
|
|
await verifySql.end();
|
|
}
|
|
}, 30_000);
|
|
});
|
|
|
|
type MergedHistoryEntry = { idx: number; version: string; when: number; tag: string; breakpoints: boolean };
|
|
const mergedMasterTail = [
|
|
{ file: "0344_wide_blue_marvel.sql", when: 1791930226792, hash: "2df2e0b867f9ed65eb931b342e7f31fe8c59901da3108466c069b68cb3892f16" },
|
|
{ file: "0345_luxuriant_carmella_unuscione.sql", when: 1791978393174, hash: "d1b360d9e6acf060a67e91b5d6dc3952f9804da3d512c7097a928b30360f6426" },
|
|
{ file: "0346_cloudy_whiplash.sql", when: 1791989860950, hash: "a77d0b71eb96b4de14bbc60f5e5f00043bffacc09905ffc782f5f0cb818c4252" },
|
|
] as const;
|
|
const canonicalMasterTail = [
|
|
{ file: "0347_flippant_imperial_guard.sql", when: 1792100545989, hash: "e159aba238be8e27e16adec236adee0febe29191ff1b4cf9cbb8d254910f4f8f" },
|
|
{ file: "0348_agent_skill_runtime_hooks.sql", when: 1792101163248, hash: "22115f83622ba83f2f494a4637f3ce404afbdf8717a5dea106421223e64d0b93" },
|
|
] as const;
|
|
const mergedMuseTail = [
|
|
{ file: "0349_oval_dragon_lord.sql", oldFile: "0344_perpetual_eternity.sql", when: 1791904391794, hash: "5c422698d40da1d6cdc587134553e371f7c807ce9617e07d7ff77eb05810ae49" },
|
|
{ file: "0350_friendly_rhodey.sql", oldFile: "0345_lumpy_celestials.sql", when: 1791904404725, hash: "a9373b52e02fa985bf10c15da91f6681c49d8726d94d5d68fd77e5802fd47a68" },
|
|
{ file: "0351_hard_albert_cleary.sql", oldFile: "0346_powerful_epoch.sql", when: 1791904422315, hash: "65f4f08c6a8fed7396db491e0691e693636f48c4ca4fa63cd2057af38039861a" },
|
|
{ file: "0352_special_warbound.sql", oldFile: "0347_big_meggan.sql", when: 1791904430408, hash: "1d2d5834fd499b60a43778d6d14b2520e3e2980258b8c0997d80cb95465f4abf" },
|
|
{ file: "0353_skinny_gamma_corps.sql", oldFile: "0348_foamy_lightspeed.sql", when: 1791905947460, hash: "1a379bc03d80c34815546bb9f50bf1408ccf156f4a1df2d86edb0a01acdc55c0" },
|
|
] as const;
|
|
|
|
describeEmbeddedPostgres("merged Muse migration histories", () => {
|
|
async function schemaShape(sql: ReturnType<typeof postgres>) {
|
|
const columns = await sql`
|
|
SELECT table_name, column_name, data_type, udt_name, is_nullable, column_default
|
|
FROM information_schema.columns WHERE table_schema = 'public' ORDER BY table_name, ordinal_position
|
|
`;
|
|
const indexes = await sql`
|
|
SELECT tablename, indexname, indexdef FROM pg_indexes WHERE schemaname = 'public' ORDER BY tablename, indexname
|
|
`;
|
|
const constraints = await sql`
|
|
SELECT t.relname AS table_name, c.conname, pg_get_constraintdef(c.oid, true) AS definition
|
|
FROM pg_constraint c JOIN pg_class t ON t.oid = c.conrelid
|
|
JOIN pg_namespace n ON n.oid = t.relnamespace
|
|
WHERE n.nspname = 'public' ORDER BY t.relname, c.conname
|
|
`;
|
|
return { columns: Array.from(columns), indexes: Array.from(indexes), constraints: Array.from(constraints) };
|
|
}
|
|
|
|
async function historyRows(sql: ReturnType<typeof postgres>) {
|
|
return Array.from(await sql`SELECT id, hash, created_at::text AS created_at FROM drizzle.__drizzle_migrations ORDER BY id`);
|
|
}
|
|
|
|
async function replayHistoricalSchema(sql: ReturnType<typeof postgres>, entries: Array<{ file: string; oldFile?: string; when: number; hash: string }>) {
|
|
await sql`CREATE SCHEMA drizzle`;
|
|
await sql`CREATE TABLE drizzle.__drizzle_migrations (id SERIAL PRIMARY KEY, hash text NOT NULL, created_at bigint)`;
|
|
let createdAt = 0;
|
|
for (const entry of entries) {
|
|
const content = await fs.promises.readFile(new URL(entry.oldFile ? `./__fixtures__/muse-preview-migrations/${entry.oldFile}` : `./migrations/${entry.file}`, import.meta.url), "utf8");
|
|
expect(createHash("sha256").update(content).digest("hex")).toBe(entry.hash);
|
|
createdAt = Math.max(createdAt + 1, entry.when);
|
|
const applyMigration = async (executor: Pick<ReturnType<typeof postgres>, "unsafe">) => {
|
|
for (const statement of content.split("--> statement-breakpoint")) {
|
|
if (!statement.trim()) continue;
|
|
await executor.unsafe(statement.replace(/^(\s*ALTER TABLE "[^"]+" DROP CONSTRAINT) (?!IF EXISTS\b)/i, "$1 IF EXISTS "));
|
|
}
|
|
await executor.unsafe("INSERT INTO drizzle.__drizzle_migrations (hash, created_at) VALUES ($1, $2)", [entry.hash, createdAt]);
|
|
};
|
|
const requiresTopLevel = /\bCREATE\s+(?:UNIQUE\s+)?INDEX\s+CONCURRENTLY\b/i.test(content)
|
|
|| /^\s*COMMIT\s*;/im.test(content);
|
|
if (requiresTopLevel) await applyMigration(sql);
|
|
else await sql.begin(async transaction => { await applyMigration(transaction); });
|
|
}
|
|
}
|
|
|
|
// Later migrations may add columns; retained rows must keep every earlier value.
|
|
function retainedColumns<T extends Record<string, Record<string, unknown> | null>>(after: T, before: T): T {
|
|
return Object.fromEntries(Object.entries(after).map(([table, row]) => {
|
|
const earlier = before[table];
|
|
return [table, row && earlier ? Object.fromEntries(Object.keys(earlier).map(key => [key, row[key]])) : row];
|
|
})) as T;
|
|
}
|
|
|
|
async function seedRetainedRows(sql: ReturnType<typeof postgres>, withMuse: boolean) {
|
|
const userId = randomUUID(), clientId = `migration-client-${randomUUID()}`;
|
|
const [company] = await sql`INSERT INTO companies (name, issue_prefix) VALUES ('Retained migration company', 'MRG') RETURNING id`;
|
|
const [agent] = await sql`
|
|
INSERT INTO agents (company_id, name, adapter_type, adapter_config)
|
|
VALUES (${company.id}, 'Retained Muse', 'paperclip_runner', '{"provider":"muse"}'::jsonb) RETURNING id
|
|
`;
|
|
const [task] = await sql`
|
|
INSERT INTO issues (company_id, title, assignee_agent_id)
|
|
VALUES (${company.id}, 'Retained task', ${agent.id}) RETURNING id
|
|
`;
|
|
const [run] = await sql`
|
|
INSERT INTO heartbeat_runs (company_id, agent_id, invocation_source, trigger_detail, status, runtime_mode, native_issue_id, context_snapshot)
|
|
VALUES (${company.id}, ${agent.id}, 'assignment', 'system', 'failed', 'native', ${task.id},
|
|
${JSON.stringify({ paperclipIssue: { id: task.id }, issueId: task.id })}::jsonb) RETURNING id
|
|
`;
|
|
await sql`INSERT INTO "user" (id, name, email, created_at, updated_at) VALUES (${userId}, 'Migration operator', ${`${userId}@example.test`}, now(), now())`;
|
|
await sql`INSERT INTO mcp_oauth_clients (id, name, redirect_uris) VALUES (${clientId}, 'Synthetic migration client', '["https://example.test/callback"]'::jsonb)`;
|
|
const [grant] = await sql`
|
|
INSERT INTO mcp_oauth_grants (company_id, user_id, client_id, resource, purpose, agent_id, scopes)
|
|
VALUES (${company.id}, ${userId}, ${clientId}, 'https://example.test/api/muse/v1', 'agent', ${agent.id}, '["paperclip:agent","offline_access"]'::jsonb) RETURNING id
|
|
`;
|
|
const [binding] = withMuse ? await sql`
|
|
INSERT INTO muse_agent_bindings (company_id, agent_id, operator_id, grant_id, status)
|
|
VALUES (${company.id}, ${agent.id}, ${userId}, ${grant.id}, 'connected') RETURNING id
|
|
` : [];
|
|
return async () => {
|
|
const [companyRow] = await sql`SELECT to_jsonb(c) AS row FROM companies c WHERE id = ${company.id}`;
|
|
const [taskRow] = await sql`SELECT to_jsonb(t) AS row FROM issues t WHERE id = ${task.id}`;
|
|
const [runRow] = await sql`SELECT to_jsonb(r) AS row FROM heartbeat_runs r WHERE id = ${run.id}`;
|
|
const [grantRow] = await sql`SELECT to_jsonb(g) AS row FROM mcp_oauth_grants g WHERE id = ${grant.id}`;
|
|
const [bindingRow] = binding ? await sql`SELECT to_jsonb(b) AS row FROM muse_agent_bindings b WHERE id = ${binding.id}` : [];
|
|
return { company: companyRow.row, task: taskRow.row, run: runRow.row, grant: grantRow.row, binding: bindingRow?.row ?? null };
|
|
};
|
|
}
|
|
|
|
it.each(["fresh", "master0346", "muse0348"] as const)("migrates %s through the normal loader without rewriting historical receipts", async baseline => {
|
|
const referenceUrl = await createTempDatabase();
|
|
const adminUrl = new URL(referenceUrl), databaseName = `merged_muse_${baseline.toLowerCase()}`;
|
|
adminUrl.pathname = "/postgres";
|
|
await ensurePostgresDatabase(adminUrl.toString(), databaseName);
|
|
adminUrl.pathname = `/${databaseName}`;
|
|
const connectionString = adminUrl.toString();
|
|
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
const reference = postgres(referenceUrl, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const journal = JSON.parse(await fs.promises.readFile(new URL("./migrations/meta/_journal.json", import.meta.url), "utf8")) as { entries: MergedHistoryEntry[] };
|
|
const common = await Promise.all(journal.entries.filter(entry => entry.idx <= 343).map(async entry => ({
|
|
entry, file: `${entry.tag}.sql`, when: entry.when, hash: await migrationHash(`${entry.tag}.sql`),
|
|
})));
|
|
const prefixIdentity = common.map(({ entry, hash }) => [entry.idx, entry.version, entry.when, entry.tag, entry.breakpoints, hash]);
|
|
expect(createHash("sha256").update(JSON.stringify(prefixIdentity)).digest("hex")).toBe("18063d01d91c172c556ee0bc7e84eaa35dc7c4b00b07e817c471cdf392393454");
|
|
for (const entry of [...mergedMasterTail, ...canonicalMasterTail]) {
|
|
expect(await migrationHash(entry.file)).toBe(entry.hash);
|
|
expect(journal.entries.find(candidate => `${candidate.tag}.sql` === entry.file)?.when).toBe(entry.when);
|
|
}
|
|
for (const entry of mergedMuseTail) {
|
|
expect(await migrationHash(entry.file)).not.toBe(entry.hash);
|
|
expect(journal.entries.some(candidate => `${candidate.tag}.sql` === entry.file)).toBe(true);
|
|
}
|
|
if (baseline !== "fresh") await replayHistoricalSchema(sql, [...common, ...(baseline === "master0346" ? mergedMasterTail : mergedMuseTail)]);
|
|
const beforeHistory = baseline === "fresh" ? [] : await historyRows(sql);
|
|
const retainedRows = baseline === "fresh" ? null : await seedRetainedRows(sql, baseline === "muse0348");
|
|
const beforeRows = await retainedRows?.();
|
|
const state = await inspectMigrations(connectionString);
|
|
expect(state.status).toBe("needsMigrations");
|
|
const laterMigrations = journal.entries.filter(entry => entry.idx > 353).map(entry => `${entry.tag}.sql`);
|
|
const expectedPending = baseline === "master0346" ? [...[...canonicalMasterTail, ...mergedMuseTail].map(entry => entry.file), ...laterMigrations]
|
|
: baseline === "muse0348" ? [...[...mergedMasterTail, ...canonicalMasterTail, ...mergedMuseTail].map(entry => entry.file), ...laterMigrations]
|
|
: journal.entries.map(entry => `${entry.tag}.sql`);
|
|
expect(state).toMatchObject({ pendingMigrations: expectedPending });
|
|
if (baseline === "muse0348") {
|
|
expect(Number(beforeHistory.at(-1)!.created_at)).toBeGreaterThan(mergedMuseTail.at(-1)!.when);
|
|
expect(beforeHistory.slice(-5).map(row => row.hash)).toEqual(mergedMuseTail.map(entry => entry.hash));
|
|
expect(mergedMuseTail.map(entry => entry.oldFile)).toEqual([
|
|
"0344_perpetual_eternity.sql", "0345_lumpy_celestials.sql", "0346_powerful_epoch.sql", "0347_big_meggan.sql", "0348_foamy_lightspeed.sql",
|
|
]);
|
|
}
|
|
await applyPendingMigrations(connectionString);
|
|
expect((await inspectMigrations(connectionString)).status).toBe("upToDate");
|
|
const afterHistory = await historyRows(sql);
|
|
expect(afterHistory).toHaveLength(journal.entries.length + (baseline === "muse0348" ? mergedMuseTail.length : 0));
|
|
expect(afterHistory.slice(0, beforeHistory.length)).toEqual(beforeHistory);
|
|
for (const entry of [...mergedMasterTail, ...canonicalMasterTail]) expect(afterHistory.filter(row => row.hash === entry.hash)).toHaveLength(1);
|
|
for (const entry of mergedMuseTail) {
|
|
expect(afterHistory.filter(row => row.hash === entry.hash)).toHaveLength(baseline === "muse0348" ? 1 : 0);
|
|
const currentHash = await migrationHash(entry.file);
|
|
expect(afterHistory.filter(row => row.hash === currentHash)).toHaveLength(1);
|
|
}
|
|
if (retainedRows) expect(retainedColumns(await retainedRows(), beforeRows!)).toEqual(beforeRows);
|
|
const indexes = await sql`
|
|
SELECT c.relname, i.indisvalid, i.indisready, pg_get_indexdef(c.oid) AS definition
|
|
FROM pg_index i JOIN pg_class c ON c.oid = i.indexrelid JOIN pg_namespace n ON n.oid = c.relnamespace
|
|
WHERE n.nspname = 'public' AND c.relname IN (
|
|
'heartbeat_runs_company_ctx_paperclip_issue_created_idx', 'heartbeat_runs_company_origin_issue_idx', 'grok_dispatch_assignment_kind_idx'
|
|
) ORDER BY c.relname
|
|
`;
|
|
expect(indexes).toHaveLength(3);
|
|
for (const index of indexes) expect(index).toMatchObject({ indisvalid: true, indisready: true });
|
|
expect(indexes.find(index => index.relname === "heartbeat_runs_company_origin_issue_idx")!.definition).toContain("COALESCE");
|
|
const finalShape = await schemaShape(sql);
|
|
expect(finalShape).toEqual(await schemaShape(reference));
|
|
await applyPendingMigrations(connectionString);
|
|
expect((await inspectMigrations(connectionString)).status).toBe("upToDate");
|
|
expect(await historyRows(sql)).toEqual(afterHistory);
|
|
expect(await schemaShape(sql)).toEqual(finalShape);
|
|
if (retainedRows) expect(retainedColumns(await retainedRows(), beforeRows!)).toEqual(beforeRows);
|
|
} finally {
|
|
await Promise.all([sql.end(), reference.end()]);
|
|
}
|
|
}, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS);
|
|
});
|
|
|
|
describeEmbeddedPostgres("applyPendingMigrations", () => {
|
|
it("upgrades an older clone whose replaced connection constraint is already absent", async () => {
|
|
const clusterUrl = new URL(await createTempDatabase());
|
|
const databaseName = "missing_connection_constraint";
|
|
clusterUrl.pathname = "/postgres";
|
|
await ensurePostgresDatabase(clusterUrl.toString(), databaseName);
|
|
clusterUrl.pathname = "/" + databaseName;
|
|
const connectionString = clusterUrl.toString();
|
|
const prefixFolder = await fs.promises.mkdtemp(join(tmpdir(), "paperclip-migration-prefix-"));
|
|
cleanups.push(() => fs.promises.rm(prefixFolder, { recursive: true, force: true }));
|
|
await fs.promises.mkdir(join(prefixFolder, "meta"));
|
|
const journal = JSON.parse(await fs.promises.readFile(new URL("./migrations/meta/_journal.json", import.meta.url), "utf8"));
|
|
journal.entries = journal.entries.filter((entry: { idx: number }) => entry.idx < 255);
|
|
await fs.promises.writeFile(join(prefixFolder, "meta/_journal.json"), JSON.stringify(journal));
|
|
for (const entry of journal.entries) {
|
|
await fs.promises.copyFile(new URL(`./migrations/${entry.tag}.sql`, import.meta.url), join(prefixFolder, `${entry.tag}.sql`));
|
|
}
|
|
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
await migrate(drizzle(sql), { migrationsFolder: prefixFolder });
|
|
await sql`ALTER TABLE "tool_connections" DROP CONSTRAINT "tool_connections_transport_check"`;
|
|
await applyPendingMigrations(connectionString);
|
|
expect((await inspectMigrations(connectionString)).status).toBe("upToDate");
|
|
const [constraint] = await sql<{ definition: string }[]>`
|
|
SELECT pg_get_constraintdef(oid) AS definition FROM pg_constraint
|
|
WHERE conrelid = 'tool_connections'::regclass AND conname = 'tool_connections_transport_check'
|
|
`;
|
|
expect(constraint?.definition).toContain("chat_sdk");
|
|
expect(constraint?.definition).toContain("runtime_auth");
|
|
} finally { await sql.end(); }
|
|
}, 30_000);
|
|
|
|
it("upgrades renumbered recovery migrations and replays their schema idempotently", async () => {
|
|
const connectionString = await createTempDatabase();
|
|
const recoveryFiles = [
|
|
"0250_exotic_dakota_north.sql", "0251_narrow_mastermind.sql",
|
|
"0252_friendly_kate_bishop.sql", "0253_real_firebrand.sql",
|
|
"0254_military_calypso.sql",
|
|
];
|
|
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
// An instance may have applied this identical SQL under the pre-rebase
|
|
// numbers, before the new session-goal and tool-action migrations existed.
|
|
for (const file of recoveryFiles) {
|
|
const hash = await migrationHash(file);
|
|
await sql`UPDATE "drizzle"."__drizzle_migrations" SET created_at = 1788825600000 WHERE hash = ${hash}`;
|
|
const source = await fs.promises.readFile(new URL(`./migrations/${file}`, import.meta.url), "utf8");
|
|
for (const statement of source.split("--> statement-breakpoint")) {
|
|
if (statement.trim()) await sql.unsafe(statement);
|
|
}
|
|
}
|
|
for (const file of ["0248_small_manta.sql", "0249_fast_silverclaw.sql"]) {
|
|
const hash = await migrationHash(file);
|
|
await sql`DELETE FROM "drizzle"."__drizzle_migrations" WHERE hash = ${hash}`;
|
|
}
|
|
await applyPendingMigrations(connectionString);
|
|
expect((await inspectMigrations(connectionString)).status).toBe("upToDate");
|
|
for (const file of recoveryFiles) {
|
|
const hash = await migrationHash(file);
|
|
const rows = await sql`SELECT id FROM "drizzle"."__drizzle_migrations" WHERE hash = ${hash}`;
|
|
expect(rows).toHaveLength(1);
|
|
}
|
|
const indexes = await sql`SELECT indexname FROM pg_indexes WHERE indexname = 'heartbeat_runs_native_replacement_predecessor_uq'`;
|
|
expect(indexes).toHaveLength(1);
|
|
} finally {
|
|
await sql.end();
|
|
}
|
|
}, 30_000);
|
|
|
|
it("rejects unallowlisted migration backfills that bump updated_at on user-visible tables", async () => {
|
|
const entries = await fs.promises.readdir(new URL("./migrations", import.meta.url), {
|
|
withFileTypes: true,
|
|
});
|
|
const violations: string[] = [];
|
|
|
|
for (const entry of entries) {
|
|
if (!entry.isFile() || !entry.name.endsWith(".sql")) continue;
|
|
const content = await fs.promises.readFile(
|
|
new URL(`./migrations/${entry.name}`, import.meta.url),
|
|
"utf8",
|
|
);
|
|
violations.push(...findUserVisibleUpdatedAtBackfillViolations(entry.name, content));
|
|
}
|
|
|
|
expect(violations).toEqual([]);
|
|
expect(
|
|
findUserVisibleUpdatedAtBackfillViolations(
|
|
"9999_bad_backfill.sql",
|
|
`
|
|
UPDATE "issues" AS i
|
|
SET "responsible_user_id" = 'owner-user',
|
|
"updated_at" = now()
|
|
WHERE i."responsible_user_id" IS NULL;
|
|
`,
|
|
),
|
|
).toEqual(['9999_bad_backfill.sql: UPDATE "issues" sets updated_at']);
|
|
});
|
|
|
|
it(
|
|
"applies an inserted earlier migration without replaying later legacy migrations",
|
|
async () => {
|
|
const connectionString = await createTempDatabase();
|
|
|
|
|
|
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const richMagnetoHash = await migrationHash("0030_rich_magneto.sql");
|
|
|
|
await sql.unsafe(
|
|
`DELETE FROM "drizzle"."__drizzle_migrations" WHERE hash = '${richMagnetoHash}'`,
|
|
);
|
|
await sql.unsafe(`DROP TABLE "company_logos"`);
|
|
} finally {
|
|
await sql.end();
|
|
}
|
|
|
|
const pendingState = await inspectMigrations(connectionString);
|
|
expect(pendingState).toMatchObject({
|
|
status: "needsMigrations",
|
|
pendingMigrations: ["0030_rich_magneto.sql"],
|
|
reason: "pending-migrations",
|
|
});
|
|
|
|
await applyPendingMigrations(connectionString);
|
|
|
|
const finalState = await inspectMigrations(connectionString);
|
|
expect(finalState.status).toBe("upToDate");
|
|
|
|
const verifySql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const rows = await verifySql.unsafe<{ table_name: string }[]>(
|
|
`
|
|
SELECT table_name
|
|
FROM information_schema.tables
|
|
WHERE table_schema = 'public'
|
|
AND table_name IN ('company_logos', 'execution_workspaces')
|
|
ORDER BY table_name
|
|
`,
|
|
);
|
|
expect(rows.map((row) => row.table_name)).toEqual([
|
|
"company_logos",
|
|
"execution_workspaces",
|
|
]);
|
|
} finally {
|
|
await verifySql.end();
|
|
}
|
|
},
|
|
20_000,
|
|
);
|
|
|
|
it(
|
|
"replays migration 0044 safely when its schema changes already exist",
|
|
async () => {
|
|
const connectionString = await createTempDatabase();
|
|
|
|
|
|
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const illegalToadHash = await migrationHash("0044_illegal_toad.sql");
|
|
|
|
await sql.unsafe(
|
|
`DELETE FROM "drizzle"."__drizzle_migrations" WHERE hash = '${illegalToadHash}'`,
|
|
);
|
|
|
|
const columns = await sql.unsafe<{ column_name: string }[]>(
|
|
`
|
|
SELECT column_name
|
|
FROM information_schema.columns
|
|
WHERE table_schema = 'public'
|
|
AND table_name = 'instance_settings'
|
|
AND column_name = 'general'
|
|
`,
|
|
);
|
|
expect(columns).toHaveLength(1);
|
|
} finally {
|
|
await sql.end();
|
|
}
|
|
|
|
const pendingState = await inspectMigrations(connectionString);
|
|
expect(pendingState).toMatchObject({
|
|
status: "needsMigrations",
|
|
pendingMigrations: ["0044_illegal_toad.sql"],
|
|
reason: "pending-migrations",
|
|
});
|
|
|
|
await applyPendingMigrations(connectionString);
|
|
|
|
const finalState = await inspectMigrations(connectionString);
|
|
expect(finalState.status).toBe("upToDate");
|
|
},
|
|
20_000,
|
|
);
|
|
|
|
it(
|
|
"enforces a unique board_api_keys.key_hash after migration 0044",
|
|
async () => {
|
|
const connectionString = await createTempDatabase();
|
|
|
|
|
|
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
await sql.unsafe(`
|
|
INSERT INTO "user" ("id", "name", "email", "email_verified", "created_at", "updated_at")
|
|
VALUES ('user-1', 'User One', 'user@example.com', true, now(), now())
|
|
`);
|
|
await sql.unsafe(`
|
|
INSERT INTO "board_api_keys" ("id", "user_id", "name", "key_hash", "created_at")
|
|
VALUES ('00000000-0000-0000-0000-000000000001', 'user-1', 'Key One', 'dup-hash', now())
|
|
`);
|
|
await expect(
|
|
sql.unsafe(`
|
|
INSERT INTO "board_api_keys" ("id", "user_id", "name", "key_hash", "created_at")
|
|
VALUES ('00000000-0000-0000-0000-000000000002', 'user-1', 'Key Two', 'dup-hash', now())
|
|
`),
|
|
).rejects.toThrow();
|
|
} finally {
|
|
await sql.end();
|
|
}
|
|
},
|
|
20_000,
|
|
);
|
|
|
|
it(
|
|
"replays migration 0046 safely when document revision columns already exist",
|
|
async () => {
|
|
const connectionString = await createTempDatabase();
|
|
|
|
|
|
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const smoothSentinelsHash = await migrationHash("0046_smooth_sentinels.sql");
|
|
|
|
await sql.unsafe(
|
|
`DELETE FROM "drizzle"."__drizzle_migrations" WHERE hash = '${smoothSentinelsHash}'`,
|
|
);
|
|
|
|
const columns = await sql.unsafe<{ column_name: string; is_nullable: string; column_default: string | null }[]>(
|
|
`
|
|
SELECT column_name, is_nullable, column_default
|
|
FROM information_schema.columns
|
|
WHERE table_schema = 'public'
|
|
AND table_name = 'document_revisions'
|
|
AND column_name IN ('title', 'format')
|
|
ORDER BY column_name
|
|
`,
|
|
);
|
|
expect(columns).toHaveLength(2);
|
|
} finally {
|
|
await sql.end();
|
|
}
|
|
|
|
const pendingState = await inspectMigrations(connectionString);
|
|
expect(pendingState).toMatchObject({
|
|
status: "needsMigrations",
|
|
pendingMigrations: ["0046_smooth_sentinels.sql"],
|
|
reason: "pending-migrations",
|
|
});
|
|
|
|
await applyPendingMigrations(connectionString);
|
|
|
|
const finalState = await inspectMigrations(connectionString);
|
|
expect(finalState.status).toBe("upToDate");
|
|
|
|
const verifySql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const columns = await verifySql.unsafe<{ column_name: string; is_nullable: string; column_default: string | null }[]>(
|
|
`
|
|
SELECT column_name, is_nullable, column_default
|
|
FROM information_schema.columns
|
|
WHERE table_schema = 'public'
|
|
AND table_name = 'document_revisions'
|
|
AND column_name IN ('title', 'format')
|
|
ORDER BY column_name
|
|
`,
|
|
);
|
|
expect(columns).toEqual([
|
|
expect.objectContaining({
|
|
column_name: "format",
|
|
is_nullable: "NO",
|
|
}),
|
|
expect.objectContaining({
|
|
column_name: "title",
|
|
is_nullable: "YES",
|
|
}),
|
|
]);
|
|
expect(columns[0]?.column_default).toContain("'markdown'");
|
|
} finally {
|
|
await verifySql.end();
|
|
}
|
|
},
|
|
20_000,
|
|
);
|
|
|
|
it(
|
|
"replays migration 0047 safely when feedback tables and run columns already exist",
|
|
async () => {
|
|
const connectionString = await createTempDatabase();
|
|
|
|
|
|
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const overjoyedGrootHash = await migrationHash("0047_overjoyed_groot.sql");
|
|
|
|
await sql.unsafe(
|
|
`DELETE FROM "drizzle"."__drizzle_migrations" WHERE hash = '${overjoyedGrootHash}'`,
|
|
);
|
|
|
|
const tables = await sql.unsafe<{ table_name: string }[]>(
|
|
`
|
|
SELECT table_name
|
|
FROM information_schema.tables
|
|
WHERE table_schema = 'public'
|
|
AND table_name IN ('feedback_exports', 'feedback_votes')
|
|
ORDER BY table_name
|
|
`,
|
|
);
|
|
expect(tables.map((row) => row.table_name)).toEqual([
|
|
"feedback_exports",
|
|
"feedback_votes",
|
|
]);
|
|
|
|
const columns = await sql.unsafe<{ table_name: string; column_name: string }[]>(
|
|
`
|
|
SELECT table_name, column_name
|
|
FROM information_schema.columns
|
|
WHERE table_schema = 'public'
|
|
AND (
|
|
(table_name = 'companies' AND column_name IN (
|
|
'feedback_data_sharing_enabled',
|
|
'feedback_data_sharing_consent_at',
|
|
'feedback_data_sharing_consent_by_user_id',
|
|
'feedback_data_sharing_terms_version'
|
|
))
|
|
OR (table_name = 'document_revisions' AND column_name = 'created_by_run_id')
|
|
OR (table_name = 'issue_comments' AND column_name = 'created_by_run_id')
|
|
)
|
|
ORDER BY table_name, column_name
|
|
`,
|
|
);
|
|
expect(columns).toHaveLength(6);
|
|
} finally {
|
|
await sql.end();
|
|
}
|
|
|
|
const pendingState = await inspectMigrations(connectionString);
|
|
expect(pendingState).toMatchObject({
|
|
status: "needsMigrations",
|
|
pendingMigrations: ["0047_overjoyed_groot.sql"],
|
|
reason: "pending-migrations",
|
|
});
|
|
|
|
await applyPendingMigrations(connectionString);
|
|
|
|
const finalState = await inspectMigrations(connectionString);
|
|
expect(finalState.status).toBe("upToDate");
|
|
|
|
const verifySql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const constraints = await verifySql.unsafe<{ conname: string }[]>(
|
|
`
|
|
SELECT conname
|
|
FROM pg_constraint
|
|
WHERE conname IN (
|
|
'feedback_exports_company_id_companies_id_fk',
|
|
'feedback_exports_feedback_vote_id_feedback_votes_id_fk',
|
|
'feedback_exports_issue_id_issues_id_fk',
|
|
'feedback_votes_company_id_companies_id_fk',
|
|
'feedback_votes_issue_id_issues_id_fk'
|
|
)
|
|
ORDER BY conname
|
|
`,
|
|
);
|
|
expect(constraints.map((row) => row.conname)).toEqual([
|
|
"feedback_exports_company_id_companies_id_fk",
|
|
"feedback_exports_feedback_vote_id_feedback_votes_id_fk",
|
|
"feedback_exports_issue_id_issues_id_fk",
|
|
"feedback_votes_company_id_companies_id_fk",
|
|
"feedback_votes_issue_id_issues_id_fk",
|
|
]);
|
|
} finally {
|
|
await verifySql.end();
|
|
}
|
|
},
|
|
20_000,
|
|
);
|
|
|
|
it(
|
|
"replays migration 0048 safely when routines.variables already exists",
|
|
async () => {
|
|
const connectionString = await createTempDatabase();
|
|
|
|
|
|
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const flashyMarrowHash = await migrationHash("0048_flashy_marrow.sql");
|
|
|
|
await sql.unsafe(
|
|
`DELETE FROM "drizzle"."__drizzle_migrations" WHERE hash = '${flashyMarrowHash}'`,
|
|
);
|
|
|
|
const columns = await sql.unsafe<{ column_name: string }[]>(
|
|
`
|
|
SELECT column_name
|
|
FROM information_schema.columns
|
|
WHERE table_schema = 'public'
|
|
AND table_name = 'routines'
|
|
AND column_name = 'variables'
|
|
`,
|
|
);
|
|
expect(columns).toHaveLength(1);
|
|
} finally {
|
|
await sql.end();
|
|
}
|
|
|
|
const pendingState = await inspectMigrations(connectionString);
|
|
expect(pendingState).toMatchObject({
|
|
status: "needsMigrations",
|
|
pendingMigrations: ["0048_flashy_marrow.sql"],
|
|
reason: "pending-migrations",
|
|
});
|
|
|
|
await applyPendingMigrations(connectionString);
|
|
|
|
const finalState = await inspectMigrations(connectionString);
|
|
expect(finalState.status).toBe("upToDate");
|
|
|
|
const verifySql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const columns = await verifySql.unsafe<{ column_name: string; is_nullable: string; data_type: string }[]>(
|
|
`
|
|
SELECT column_name, is_nullable, data_type
|
|
FROM information_schema.columns
|
|
WHERE table_schema = 'public'
|
|
AND table_name = 'routines'
|
|
AND column_name = 'variables'
|
|
`,
|
|
);
|
|
expect(columns).toEqual([
|
|
expect.objectContaining({
|
|
column_name: "variables",
|
|
is_nullable: "NO",
|
|
data_type: "jsonb",
|
|
}),
|
|
]);
|
|
} finally {
|
|
await verifySql.end();
|
|
}
|
|
},
|
|
20_000,
|
|
);
|
|
|
|
it(
|
|
"replays migration 0050 safely when projects.env already exists",
|
|
async () => {
|
|
const connectionString = await createTempDatabase();
|
|
|
|
|
|
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const stiffLuckmanHash = await migrationHash("0050_stiff_luckman.sql");
|
|
|
|
await sql.unsafe(
|
|
`DELETE FROM "drizzle"."__drizzle_migrations" WHERE hash = '${stiffLuckmanHash}'`,
|
|
);
|
|
|
|
const columns = await sql.unsafe<{ column_name: string }[]>(
|
|
`
|
|
SELECT column_name
|
|
FROM information_schema.columns
|
|
WHERE table_schema = 'public'
|
|
AND table_name = 'projects'
|
|
AND column_name = 'env'
|
|
`,
|
|
);
|
|
expect(columns).toHaveLength(1);
|
|
} finally {
|
|
await sql.end();
|
|
}
|
|
|
|
const pendingState = await inspectMigrations(connectionString);
|
|
expect(pendingState).toMatchObject({
|
|
status: "needsMigrations",
|
|
pendingMigrations: ["0050_stiff_luckman.sql"],
|
|
reason: "pending-migrations",
|
|
});
|
|
|
|
await applyPendingMigrations(connectionString);
|
|
|
|
const finalState = await inspectMigrations(connectionString);
|
|
expect(finalState.status).toBe("upToDate");
|
|
|
|
const verifySql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const columns = await verifySql.unsafe<{ column_name: string; is_nullable: string; data_type: string }[]>(
|
|
`
|
|
SELECT column_name, is_nullable, data_type
|
|
FROM information_schema.columns
|
|
WHERE table_schema = 'public'
|
|
AND table_name = 'projects'
|
|
AND column_name = 'env'
|
|
`,
|
|
);
|
|
expect(columns).toEqual([
|
|
expect.objectContaining({
|
|
column_name: "env",
|
|
is_nullable: "YES",
|
|
data_type: "jsonb",
|
|
}),
|
|
]);
|
|
} finally {
|
|
await verifySql.end();
|
|
}
|
|
},
|
|
20_000,
|
|
);
|
|
|
|
it(
|
|
"replays migration 0059 safely when plugin_database_namespaces already exists",
|
|
async () => {
|
|
const connectionString = await createTempDatabase();
|
|
|
|
|
|
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const pluginNamespacesHash = await migrationHash(
|
|
"0059_plugin_database_namespaces.sql",
|
|
);
|
|
|
|
await sql.unsafe(
|
|
`DELETE FROM "drizzle"."__drizzle_migrations" WHERE hash = '${pluginNamespacesHash}'`,
|
|
);
|
|
|
|
const tables = await sql.unsafe<{ table_name: string }[]>(
|
|
`
|
|
SELECT table_name
|
|
FROM information_schema.tables
|
|
WHERE table_schema = 'public'
|
|
AND table_name IN ('plugin_database_namespaces', 'plugin_migrations')
|
|
ORDER BY table_name
|
|
`,
|
|
);
|
|
expect(tables.map((row) => row.table_name)).toEqual([
|
|
"plugin_database_namespaces",
|
|
"plugin_migrations",
|
|
]);
|
|
} finally {
|
|
await sql.end();
|
|
}
|
|
|
|
const pendingState = await inspectMigrations(connectionString);
|
|
expect(pendingState).toMatchObject({
|
|
status: "needsMigrations",
|
|
pendingMigrations: ["0059_plugin_database_namespaces.sql"],
|
|
reason: "pending-migrations",
|
|
});
|
|
|
|
await applyPendingMigrations(connectionString);
|
|
|
|
const finalState = await inspectMigrations(connectionString);
|
|
expect(finalState.status).toBe("upToDate");
|
|
|
|
const verifySql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const indexes = await verifySql.unsafe<{ indexname: string }[]>(
|
|
`
|
|
SELECT indexname
|
|
FROM pg_indexes
|
|
WHERE schemaname = 'public'
|
|
AND tablename IN ('plugin_database_namespaces', 'plugin_migrations')
|
|
ORDER BY indexname
|
|
`,
|
|
);
|
|
expect(indexes.map((row) => row.indexname)).toEqual(
|
|
expect.arrayContaining([
|
|
"plugin_database_namespaces_namespace_idx",
|
|
"plugin_database_namespaces_plugin_idx",
|
|
"plugin_database_namespaces_status_idx",
|
|
"plugin_migrations_plugin_idx",
|
|
"plugin_migrations_plugin_key_idx",
|
|
"plugin_migrations_status_idx",
|
|
]),
|
|
);
|
|
} finally {
|
|
await verifySql.end();
|
|
}
|
|
},
|
|
20_000,
|
|
);
|
|
|
|
it(
|
|
"replays the built-in managed resources migration after the legacy 0136 journal entry",
|
|
async () => {
|
|
const connectionString = await createTempDatabase();
|
|
|
|
|
|
const builtInResourcesHash = await migrationHash(
|
|
"0140_built_in_managed_resources.sql",
|
|
);
|
|
const legacyBuiltInResourcesHash = createHash("sha256")
|
|
.update("legacy 0136_built_in_managed_resources.sql")
|
|
.digest("hex");
|
|
|
|
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
await sql.unsafe(
|
|
`DELETE FROM "drizzle"."__drizzle_migrations" WHERE hash = '${builtInResourcesHash}'`,
|
|
);
|
|
await sql.unsafe(
|
|
`
|
|
INSERT INTO "drizzle"."__drizzle_migrations" ("hash", "created_at")
|
|
VALUES ('${legacyBuiltInResourcesHash}', 1783555200000)
|
|
`,
|
|
);
|
|
await sql.unsafe(`
|
|
ALTER TABLE "built_in_managed_resources"
|
|
DROP CONSTRAINT IF EXISTS "built_in_managed_resources_company_id_companies_id_fk"
|
|
`);
|
|
await sql.unsafe(`DROP INDEX IF EXISTS "built_in_managed_resources_company_idx"`);
|
|
await sql.unsafe(`DROP INDEX IF EXISTS "built_in_managed_resources_resource_idx"`);
|
|
await sql.unsafe(`DROP INDEX IF EXISTS "built_in_managed_resources_company_bundle_resource_uq"`);
|
|
} finally {
|
|
await sql.end();
|
|
}
|
|
|
|
const pendingState = await inspectMigrations(connectionString);
|
|
expect(pendingState).toMatchObject({
|
|
status: "needsMigrations",
|
|
pendingMigrations: ["0140_built_in_managed_resources.sql"],
|
|
reason: "pending-migrations",
|
|
});
|
|
|
|
await applyPendingMigrations(connectionString);
|
|
|
|
const finalState = await inspectMigrations(connectionString);
|
|
expect(finalState.status).toBe("upToDate");
|
|
|
|
const verifySql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const rows = await verifySql.unsafe<{
|
|
foreign_key_exists: boolean;
|
|
company_index_exists: boolean;
|
|
resource_index_exists: boolean;
|
|
unique_index_exists: boolean;
|
|
}[]>(`
|
|
SELECT
|
|
EXISTS (
|
|
SELECT 1
|
|
FROM "pg_constraint" c
|
|
JOIN "pg_class" t ON t.oid = c.conrelid
|
|
JOIN "pg_namespace" n ON n.oid = t.relnamespace
|
|
WHERE n.nspname = 'public'
|
|
AND t.relname = 'built_in_managed_resources'
|
|
AND c.conname = 'built_in_managed_resources_company_id_companies_id_fk'
|
|
) AS "foreign_key_exists",
|
|
EXISTS (
|
|
SELECT 1
|
|
FROM "pg_class" c
|
|
JOIN "pg_namespace" n ON n.oid = c.relnamespace
|
|
WHERE n.nspname = 'public'
|
|
AND c.relkind = 'i'
|
|
AND c.relname = 'built_in_managed_resources_company_idx'
|
|
) AS "company_index_exists",
|
|
EXISTS (
|
|
SELECT 1
|
|
FROM "pg_class" c
|
|
JOIN "pg_namespace" n ON n.oid = c.relnamespace
|
|
WHERE n.nspname = 'public'
|
|
AND c.relkind = 'i'
|
|
AND c.relname = 'built_in_managed_resources_resource_idx'
|
|
) AS "resource_index_exists",
|
|
EXISTS (
|
|
SELECT 1
|
|
FROM "pg_class" c
|
|
JOIN "pg_namespace" n ON n.oid = c.relnamespace
|
|
WHERE n.nspname = 'public'
|
|
AND c.relkind = 'i'
|
|
AND c.relname = 'built_in_managed_resources_company_bundle_resource_uq'
|
|
) AS "unique_index_exists"
|
|
`);
|
|
expect(rows[0]).toEqual({
|
|
foreign_key_exists: true,
|
|
company_index_exists: true,
|
|
resource_index_exists: true,
|
|
unique_index_exists: true,
|
|
});
|
|
} finally {
|
|
await verifySql.end();
|
|
}
|
|
},
|
|
20_000,
|
|
);
|
|
|
|
it(
|
|
"replays migration 0134 without bumping issue updated_at for inbox archives",
|
|
async () => {
|
|
const connectionString = await createTempDatabase();
|
|
|
|
|
|
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const runResponsibleUserHash = await migrationHash(
|
|
"0134_run_responsible_user_invariant.sql",
|
|
);
|
|
|
|
await sql.unsafe(`
|
|
INSERT INTO "companies" ("id", "name", "issue_prefix", "created_at", "updated_at")
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000120',
|
|
'Migration Inbox Co',
|
|
'TST120',
|
|
'2026-03-26T09:00:00.000Z',
|
|
'2026-03-26T09:00:00.000Z'
|
|
)
|
|
`);
|
|
await sql.unsafe(`
|
|
INSERT INTO "company_memberships" (
|
|
"id",
|
|
"company_id",
|
|
"principal_type",
|
|
"principal_id",
|
|
"status",
|
|
"membership_role",
|
|
"created_at",
|
|
"updated_at"
|
|
)
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000121',
|
|
'00000000-0000-0000-0000-000000000120',
|
|
'user',
|
|
'owner-user',
|
|
'active',
|
|
'owner',
|
|
'2026-03-26T09:00:00.000Z',
|
|
'2026-03-26T09:00:00.000Z'
|
|
)
|
|
`);
|
|
await sql.unsafe(`
|
|
INSERT INTO "issues" (
|
|
"id",
|
|
"company_id",
|
|
"title",
|
|
"status",
|
|
"responsible_user_id",
|
|
"created_at",
|
|
"updated_at"
|
|
)
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000122',
|
|
'00000000-0000-0000-0000-000000000120',
|
|
'Archived issue needing responsible user backfill',
|
|
'todo',
|
|
NULL,
|
|
'2026-03-26T10:00:00.000Z',
|
|
'2026-03-26T10:00:00.000Z'
|
|
)
|
|
`);
|
|
await sql.unsafe(`
|
|
INSERT INTO "issue_inbox_archives" (
|
|
"id",
|
|
"company_id",
|
|
"issue_id",
|
|
"user_id",
|
|
"archived_at",
|
|
"created_at",
|
|
"updated_at"
|
|
)
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000123',
|
|
'00000000-0000-0000-0000-000000000120',
|
|
'00000000-0000-0000-0000-000000000122',
|
|
'owner-user',
|
|
'2026-03-26T12:00:00.000Z',
|
|
'2026-03-26T12:00:00.000Z',
|
|
'2026-03-26T12:00:00.000Z'
|
|
)
|
|
`);
|
|
await sql.unsafe(
|
|
`DELETE FROM "drizzle"."__drizzle_migrations" WHERE hash = '${runResponsibleUserHash}'`,
|
|
);
|
|
} finally {
|
|
await sql.end();
|
|
}
|
|
|
|
const pendingState = await inspectMigrations(connectionString);
|
|
expect(pendingState).toMatchObject({
|
|
status: "needsMigrations",
|
|
pendingMigrations: ["0134_run_responsible_user_invariant.sql"],
|
|
reason: "pending-migrations",
|
|
});
|
|
|
|
await applyPendingMigrations(connectionString);
|
|
|
|
const verifySql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const rows = await verifySql.unsafe<{
|
|
responsible_user_id: string | null;
|
|
updated_at: Date;
|
|
inbox_archive_still_current: boolean;
|
|
}[]>(`
|
|
SELECT
|
|
i."responsible_user_id",
|
|
i."updated_at",
|
|
EXISTS (
|
|
SELECT 1
|
|
FROM "issue_inbox_archives" AS archive
|
|
WHERE archive."company_id" = i."company_id"
|
|
AND archive."issue_id" = i."id"
|
|
AND archive."user_id" = 'owner-user'
|
|
AND archive."archived_at" >= i."updated_at"
|
|
) AS "inbox_archive_still_current"
|
|
FROM "issues" AS i
|
|
WHERE i."id" = '00000000-0000-0000-0000-000000000122'
|
|
`);
|
|
expect(rows).toHaveLength(1);
|
|
expect(rows[0]?.responsible_user_id).toBe("owner-user");
|
|
expect(rows[0]?.updated_at.toISOString()).toBe("2026-03-26T10:00:00.000Z");
|
|
expect(rows[0]?.inbox_archive_still_current).toBe(true);
|
|
} finally {
|
|
await verifySql.end();
|
|
}
|
|
},
|
|
20_000,
|
|
);
|
|
|
|
it(
|
|
"replays migration 0135 to repair updated_at sweeps and no-op when clean",
|
|
async () => {
|
|
const connectionString = await createTempDatabase();
|
|
|
|
|
|
const repairSweepHash = await migrationHash(
|
|
"0135_repair_run_responsible_user_updated_at_sweep.sql",
|
|
);
|
|
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
await sql.unsafe(`
|
|
INSERT INTO "companies" ("id", "name", "issue_prefix", "created_at", "updated_at")
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000240',
|
|
'Clean Migration Co',
|
|
'CLN134',
|
|
'2026-04-01T09:00:00.000Z',
|
|
'2026-04-02T09:00:00.000Z'
|
|
)
|
|
`);
|
|
await sql.unsafe(`
|
|
INSERT INTO "issues" ("id", "company_id", "title", "status", "created_at", "updated_at")
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000241',
|
|
'00000000-0000-0000-0000-000000000240',
|
|
'Clean issue should not be touched',
|
|
'todo',
|
|
'2026-04-01T10:00:00.000Z',
|
|
'2026-04-02T10:00:00.000Z'
|
|
)
|
|
`);
|
|
await sql.unsafe(
|
|
`DELETE FROM "drizzle"."__drizzle_migrations" WHERE hash = '${repairSweepHash}'`,
|
|
);
|
|
} finally {
|
|
await sql.end();
|
|
}
|
|
|
|
await applyPendingMigrations(connectionString);
|
|
|
|
const afterCleanReplay = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const cleanRows = await afterCleanReplay.unsafe<{ updated_at: Date }[]>(`
|
|
SELECT "updated_at"
|
|
FROM "issues"
|
|
WHERE "id" = '00000000-0000-0000-0000-000000000241'
|
|
`);
|
|
expect(cleanRows[0]?.updated_at.toISOString()).toBe("2026-04-02T10:00:00.000Z");
|
|
|
|
await afterCleanReplay.unsafe(`
|
|
INSERT INTO "companies" ("id", "name", "issue_prefix", "created_at", "updated_at")
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000250',
|
|
'Sweep Migration Co',
|
|
'SWP134',
|
|
'2026-01-01T00:00:00.000Z',
|
|
'2026-04-03T12:00:00.123456Z'
|
|
)
|
|
`);
|
|
await afterCleanReplay.unsafe(`
|
|
INSERT INTO "agents" ("id", "company_id", "name", "role", "adapter_type", "created_at", "updated_at")
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000251',
|
|
'00000000-0000-0000-0000-000000000250',
|
|
'Sweep Agent',
|
|
'general',
|
|
'process',
|
|
'2026-01-02T00:00:00.000Z',
|
|
'2026-01-02T00:00:00.000Z'
|
|
)
|
|
`);
|
|
await afterCleanReplay.unsafe(`
|
|
INSERT INTO "issues" ("id", "company_id", "title", "status", "created_at", "updated_at")
|
|
SELECT
|
|
('10000000-0000-0000-0000-' || lpad(gs::text, 12, '0'))::uuid,
|
|
'00000000-0000-0000-0000-000000000250',
|
|
'Swept issue ' || gs::text,
|
|
'todo',
|
|
'2026-02-01T00:00:00.000Z'::timestamptz + (gs::text || ' minutes')::interval,
|
|
'2026-04-03T12:00:00.123456Z'
|
|
FROM generate_series(1, 101) AS gs
|
|
`);
|
|
await afterCleanReplay.unsafe(`
|
|
UPDATE "issues"
|
|
SET
|
|
"status" = 'done',
|
|
"completed_at" = '2026-04-03T12:00:00.123456Z'
|
|
WHERE "id" = '10000000-0000-0000-0000-000000000003'
|
|
`);
|
|
await afterCleanReplay.unsafe(`
|
|
INSERT INTO "issue_comments" ("id", "company_id", "issue_id", "body", "created_at", "updated_at")
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000252',
|
|
'00000000-0000-0000-0000-000000000250',
|
|
'10000000-0000-0000-0000-000000000001',
|
|
'Latest pre-sweep activity',
|
|
'2026-03-01T15:30:00.000Z',
|
|
'2026-03-02T16:45:00.000Z'
|
|
)
|
|
`);
|
|
await afterCleanReplay.unsafe(`
|
|
INSERT INTO "heartbeat_runs" (
|
|
"id",
|
|
"company_id",
|
|
"agent_id",
|
|
"status",
|
|
"started_at",
|
|
"finished_at",
|
|
"created_at",
|
|
"updated_at"
|
|
)
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000253',
|
|
'00000000-0000-0000-0000-000000000250',
|
|
'00000000-0000-0000-0000-000000000251',
|
|
'completed',
|
|
'2026-02-10T10:00:00.000Z',
|
|
'2026-02-10T10:30:00.000Z',
|
|
'2026-02-10T09:55:00.000Z',
|
|
'2026-04-03T12:00:00.123456Z'
|
|
)
|
|
`);
|
|
await afterCleanReplay.unsafe(`
|
|
INSERT INTO "heartbeat_runs" (
|
|
"id",
|
|
"company_id",
|
|
"agent_id",
|
|
"status",
|
|
"started_at",
|
|
"last_output_at",
|
|
"created_at",
|
|
"updated_at"
|
|
)
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000256',
|
|
'00000000-0000-0000-0000-000000000250',
|
|
'00000000-0000-0000-0000-000000000251',
|
|
'running',
|
|
'2026-02-10T11:00:00.000Z',
|
|
'2026-04-03T12:00:00.123456Z',
|
|
'2026-02-10T10:55:00.000Z',
|
|
'2026-04-03T12:00:00.123456Z'
|
|
)
|
|
`);
|
|
await afterCleanReplay.unsafe(`
|
|
INSERT INTO "routines" (
|
|
"id",
|
|
"company_id",
|
|
"title",
|
|
"last_triggered_at",
|
|
"last_enqueued_at",
|
|
"created_at",
|
|
"updated_at"
|
|
)
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000254',
|
|
'00000000-0000-0000-0000-000000000250',
|
|
'Swept routine',
|
|
'2026-03-20T10:00:00.000Z',
|
|
'2026-03-21T11:00:00.000Z',
|
|
'2026-02-11T00:00:00.000Z',
|
|
'2026-04-03T12:00:00.123456Z'
|
|
)
|
|
`);
|
|
await afterCleanReplay.unsafe(`
|
|
INSERT INTO "routines" (
|
|
"id",
|
|
"company_id",
|
|
"title",
|
|
"last_triggered_at",
|
|
"last_enqueued_at",
|
|
"created_at",
|
|
"updated_at"
|
|
)
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000257',
|
|
'00000000-0000-0000-0000-000000000250',
|
|
'Same-timestamp active routine',
|
|
'2026-03-20T10:00:00.000Z',
|
|
'2026-04-03T12:00:00.123456Z',
|
|
'2026-02-11T00:00:00.000Z',
|
|
'2026-04-03T12:00:00.123456Z'
|
|
)
|
|
`);
|
|
await afterCleanReplay.unsafe(`
|
|
INSERT INTO "routine_runs" (
|
|
"id",
|
|
"company_id",
|
|
"routine_id",
|
|
"source",
|
|
"status",
|
|
"completed_at",
|
|
"created_at",
|
|
"updated_at"
|
|
)
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000255',
|
|
'00000000-0000-0000-0000-000000000250',
|
|
'00000000-0000-0000-0000-000000000254',
|
|
'schedule',
|
|
'completed',
|
|
'2026-02-12T12:00:00.000Z',
|
|
'2026-02-12T11:00:00.000Z',
|
|
'2026-04-03T12:00:00.123456Z'
|
|
)
|
|
`);
|
|
await afterCleanReplay.unsafe(`
|
|
INSERT INTO "routine_runs" (
|
|
"id",
|
|
"company_id",
|
|
"routine_id",
|
|
"source",
|
|
"status",
|
|
"triggered_at",
|
|
"created_at",
|
|
"updated_at"
|
|
)
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000258',
|
|
'00000000-0000-0000-0000-000000000250',
|
|
'00000000-0000-0000-0000-000000000257',
|
|
'schedule',
|
|
'running',
|
|
'2026-04-03T12:00:00.123456Z',
|
|
'2026-02-12T13:00:00.000Z',
|
|
'2026-04-03T12:00:00.123456Z'
|
|
)
|
|
`);
|
|
await afterCleanReplay.unsafe(`
|
|
INSERT INTO "companies" ("id", "name", "issue_prefix", "created_at", "updated_at")
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000260',
|
|
'Coincident Timestamp Co',
|
|
'CTS134',
|
|
'2026-01-05T00:00:00.000Z',
|
|
'2026-04-03T12:00:00.123456Z'
|
|
)
|
|
`);
|
|
await afterCleanReplay.unsafe(`
|
|
INSERT INTO "agents" ("id", "company_id", "name", "role", "adapter_type", "created_at", "updated_at")
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000261',
|
|
'00000000-0000-0000-0000-000000000260',
|
|
'Coincident Agent',
|
|
'general',
|
|
'process',
|
|
'2026-01-05T00:10:00.000Z',
|
|
'2026-01-05T00:10:00.000Z'
|
|
)
|
|
`);
|
|
await afterCleanReplay.unsafe(`
|
|
INSERT INTO "issues" ("id", "company_id", "title", "status", "created_at", "updated_at")
|
|
VALUES (
|
|
'20000000-0000-0000-0000-000000000001',
|
|
'00000000-0000-0000-0000-000000000260',
|
|
'Coincident timestamp issue should not be touched',
|
|
'todo',
|
|
'2026-02-05T00:00:00.000Z',
|
|
'2026-04-03T12:00:00.123456Z'
|
|
)
|
|
`);
|
|
await afterCleanReplay.unsafe(`
|
|
INSERT INTO "heartbeat_runs" (
|
|
"id",
|
|
"company_id",
|
|
"agent_id",
|
|
"status",
|
|
"started_at",
|
|
"finished_at",
|
|
"created_at",
|
|
"updated_at"
|
|
)
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000262',
|
|
'00000000-0000-0000-0000-000000000260',
|
|
'00000000-0000-0000-0000-000000000261',
|
|
'completed',
|
|
'2026-02-05T10:00:00.000Z',
|
|
'2026-02-05T10:30:00.000Z',
|
|
'2026-02-05T09:55:00.000Z',
|
|
'2026-04-03T12:00:00.123456Z'
|
|
)
|
|
`);
|
|
await afterCleanReplay.unsafe(
|
|
`DELETE FROM "drizzle"."__drizzle_migrations" WHERE hash = '${repairSweepHash}'`,
|
|
);
|
|
} finally {
|
|
await afterCleanReplay.end();
|
|
}
|
|
|
|
await applyPendingMigrations(connectionString);
|
|
|
|
const afterRepair = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const repairedRows = await afterRepair.unsafe<{
|
|
subject: string;
|
|
updated_at: Date;
|
|
}[]>(`
|
|
SELECT 'company' AS subject, "updated_at"
|
|
FROM "companies"
|
|
WHERE "id" = '00000000-0000-0000-0000-000000000250'
|
|
UNION ALL
|
|
SELECT 'issue_with_comment' AS subject, "updated_at"
|
|
FROM "issues"
|
|
WHERE "id" = '10000000-0000-0000-0000-000000000001'
|
|
UNION ALL
|
|
SELECT 'issue_without_comment' AS subject, "updated_at"
|
|
FROM "issues"
|
|
WHERE "id" = '10000000-0000-0000-0000-000000000002'
|
|
UNION ALL
|
|
SELECT 'issue_with_state_activity' AS subject, "updated_at"
|
|
FROM "issues"
|
|
WHERE "id" = '10000000-0000-0000-0000-000000000003'
|
|
UNION ALL
|
|
SELECT 'heartbeat_run' AS subject, "updated_at"
|
|
FROM "heartbeat_runs"
|
|
WHERE "id" = '00000000-0000-0000-0000-000000000253'
|
|
UNION ALL
|
|
SELECT 'heartbeat_run_with_output' AS subject, "updated_at"
|
|
FROM "heartbeat_runs"
|
|
WHERE "id" = '00000000-0000-0000-0000-000000000256'
|
|
UNION ALL
|
|
SELECT 'other_company' AS subject, "updated_at"
|
|
FROM "companies"
|
|
WHERE "id" = '00000000-0000-0000-0000-000000000260'
|
|
UNION ALL
|
|
SELECT 'other_heartbeat_run' AS subject, "updated_at"
|
|
FROM "heartbeat_runs"
|
|
WHERE "id" = '00000000-0000-0000-0000-000000000262'
|
|
UNION ALL
|
|
SELECT 'other_issue' AS subject, "updated_at"
|
|
FROM "issues"
|
|
WHERE "id" = '20000000-0000-0000-0000-000000000001'
|
|
UNION ALL
|
|
SELECT 'routine' AS subject, "updated_at"
|
|
FROM "routines"
|
|
WHERE "id" = '00000000-0000-0000-0000-000000000254'
|
|
UNION ALL
|
|
SELECT 'routine_with_activity' AS subject, "updated_at"
|
|
FROM "routines"
|
|
WHERE "id" = '00000000-0000-0000-0000-000000000257'
|
|
UNION ALL
|
|
SELECT 'routine_run' AS subject, "updated_at"
|
|
FROM "routine_runs"
|
|
WHERE "id" = '00000000-0000-0000-0000-000000000255'
|
|
UNION ALL
|
|
SELECT 'routine_run_with_trigger' AS subject, "updated_at"
|
|
FROM "routine_runs"
|
|
WHERE "id" = '00000000-0000-0000-0000-000000000258'
|
|
ORDER BY subject
|
|
`);
|
|
const repaired = Object.fromEntries(
|
|
repairedRows.map((row) => [row.subject, row.updated_at.toISOString()]),
|
|
);
|
|
expect(repaired).toEqual({
|
|
company: "2026-01-01T00:00:00.000Z",
|
|
heartbeat_run: "2026-02-10T10:30:00.000Z",
|
|
heartbeat_run_with_output: "2026-04-03T12:00:00.123Z",
|
|
issue_with_comment: "2026-03-02T16:45:00.000Z",
|
|
issue_with_state_activity: "2026-04-03T12:00:00.123Z",
|
|
issue_without_comment: "2026-02-01T00:02:00.000Z",
|
|
other_company: "2026-04-03T12:00:00.123Z",
|
|
other_heartbeat_run: "2026-04-03T12:00:00.123Z",
|
|
other_issue: "2026-04-03T12:00:00.123Z",
|
|
routine: "2026-03-21T11:00:00.000Z",
|
|
routine_run_with_trigger: "2026-04-03T12:00:00.123Z",
|
|
routine_with_activity: "2026-04-03T12:00:00.123Z",
|
|
routine_run: "2026-02-12T12:00:00.000Z",
|
|
});
|
|
|
|
await afterRepair.unsafe(
|
|
`DELETE FROM "drizzle"."__drizzle_migrations" WHERE hash = '${repairSweepHash}'`,
|
|
);
|
|
} finally {
|
|
await afterRepair.end();
|
|
}
|
|
|
|
await applyPendingMigrations(connectionString);
|
|
|
|
const afterSecondRun = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const secondRunRows = await afterSecondRun.unsafe<{ updated_at: Date }[]>(`
|
|
SELECT "updated_at"
|
|
FROM "issues"
|
|
WHERE "id" = '10000000-0000-0000-0000-000000000001'
|
|
`);
|
|
expect(secondRunRows[0]?.updated_at.toISOString()).toBe("2026-03-02T16:45:00.000Z");
|
|
} finally {
|
|
await afterSecondRun.end();
|
|
}
|
|
},
|
|
20_000,
|
|
);
|
|
|
|
it(
|
|
"replays the run responsible user repair migration when heartbeat run issue refs are identifiers",
|
|
async () => {
|
|
const connectionString = await createTempDatabase();
|
|
|
|
|
|
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const runResponsibleUserRepairHash = await migrationHash(
|
|
"0131_repair_run_responsible_user_context_refs.sql",
|
|
);
|
|
|
|
await sql.unsafe(`
|
|
INSERT INTO "companies" ("id", "name", "issue_prefix", "created_at", "updated_at")
|
|
VALUES ('00000000-0000-0000-0000-000000000130', 'Migration Test Co', 'TST130', now(), now())
|
|
`);
|
|
await sql.unsafe(`
|
|
INSERT INTO "company_memberships" (
|
|
"id",
|
|
"company_id",
|
|
"principal_type",
|
|
"principal_id",
|
|
"status",
|
|
"membership_role",
|
|
"created_at",
|
|
"updated_at"
|
|
)
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000131',
|
|
'00000000-0000-0000-0000-000000000130',
|
|
'user',
|
|
'owner-user',
|
|
'active',
|
|
'owner',
|
|
now(),
|
|
now()
|
|
)
|
|
`);
|
|
await sql.unsafe(`
|
|
INSERT INTO "agents" ("id", "company_id", "name", "role", "adapter_type", "created_at", "updated_at")
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000132',
|
|
'00000000-0000-0000-0000-000000000130',
|
|
'Migration Agent',
|
|
'general',
|
|
'process',
|
|
now(),
|
|
now()
|
|
)
|
|
`);
|
|
await sql.unsafe(`
|
|
INSERT INTO "issues" (
|
|
"id",
|
|
"company_id",
|
|
"title",
|
|
"status",
|
|
"responsible_user_id",
|
|
"identifier",
|
|
"created_at",
|
|
"updated_at"
|
|
)
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000133',
|
|
'00000000-0000-0000-0000-000000000130',
|
|
'Identifier referenced issue',
|
|
'todo',
|
|
'issue-user',
|
|
'TST130-1',
|
|
now(),
|
|
now()
|
|
)
|
|
`);
|
|
await sql.unsafe(`
|
|
INSERT INTO "heartbeat_runs" (
|
|
"id",
|
|
"company_id",
|
|
"agent_id",
|
|
"status",
|
|
"responsible_user_id",
|
|
"context_snapshot",
|
|
"created_at",
|
|
"updated_at"
|
|
)
|
|
VALUES (
|
|
'00000000-0000-0000-0000-000000000134',
|
|
'00000000-0000-0000-0000-000000000130',
|
|
'00000000-0000-0000-0000-000000000132',
|
|
'completed',
|
|
NULL,
|
|
'{"issueId":"TST130-1"}'::jsonb,
|
|
now(),
|
|
now()
|
|
)
|
|
`);
|
|
await sql.unsafe(
|
|
`DELETE FROM "drizzle"."__drizzle_migrations" WHERE hash = '${runResponsibleUserRepairHash}'`,
|
|
);
|
|
} finally {
|
|
await sql.end();
|
|
}
|
|
|
|
const pendingState = await inspectMigrations(connectionString);
|
|
expect(pendingState).toMatchObject({
|
|
status: "needsMigrations",
|
|
pendingMigrations: ["0131_repair_run_responsible_user_context_refs.sql"],
|
|
reason: "pending-migrations",
|
|
});
|
|
|
|
await applyPendingMigrations(connectionString);
|
|
|
|
const verifySql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const runs = await verifySql.unsafe<{ responsible_user_id: string | null }[]>(`
|
|
SELECT "responsible_user_id"
|
|
FROM "heartbeat_runs"
|
|
WHERE "id" = '00000000-0000-0000-0000-000000000134'
|
|
`);
|
|
expect(runs).toEqual([{ responsible_user_id: "issue-user" }]);
|
|
|
|
} finally {
|
|
await verifySql.end();
|
|
}
|
|
},
|
|
20_000,
|
|
);
|
|
|
|
it(
|
|
"preserves legacy runs while adding native persistence and replay-safe status versioning",
|
|
async () => {
|
|
const clusterUrl = await createTempDatabase();
|
|
await ensurePostgresDatabase(clusterUrl, "native_legacy");
|
|
const legacyUrl = new URL(clusterUrl);
|
|
legacyUrl.pathname = "/native_legacy";
|
|
const connectionString = legacyUrl.href;
|
|
cleanups.push(() => closeRegisteredClients(connectionString));
|
|
const directory = await fs.promises.mkdtemp(join(tmpdir(), "paperclip-native-prior-migrations-"));
|
|
cleanups.push(() => fs.promises.rm(directory, { recursive: true, force: true }));
|
|
const migrationsRoot = new URL("./migrations/", import.meta.url);
|
|
const journal = JSON.parse(await fs.promises.readFile(new URL("meta/_journal.json", migrationsRoot), "utf8"));
|
|
const priorEntries = journal.entries.filter((entry: { idx: number }) => entry.idx < 227);
|
|
await fs.promises.mkdir(join(directory, "meta"));
|
|
for (const entry of priorEntries) {
|
|
await fs.promises.copyFile(new URL(`${entry.tag}.sql`, migrationsRoot), join(directory, `${entry.tag}.sql`));
|
|
}
|
|
await fs.promises.writeFile(join(directory, "meta/_journal.json"), JSON.stringify({ ...journal, entries: priorEntries }));
|
|
|
|
const nativePersistenceHash = await migrationHash("0227_modern_pandemic.sql");
|
|
const eventSequenceUniquenessHash = await migrationHash(
|
|
"0235_heartbeat_run_event_sequence_uniqueness.sql",
|
|
);
|
|
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
const companyId = "10000000-0000-4000-8000-000000000227";
|
|
const agentId = "20000000-0000-4000-8000-000000000227";
|
|
const runId = "30000000-0000-4000-8000-000000000227";
|
|
const issueId = "40000000-0000-4000-8000-000000000227";
|
|
const contractId = "50000000-0000-4000-8000-000000000227";
|
|
const resultId = "60000000-0000-4000-8000-000000000227";
|
|
const assessmentId = "70000000-0000-4000-8000-000000000227";
|
|
const decisionId = "80000000-0000-4000-8000-000000000227";
|
|
const otherCompanyId = "11000000-0000-4000-8000-000000000227";
|
|
const otherAgentId = "21000000-0000-4000-8000-000000000227";
|
|
const otherRunId = "31000000-0000-4000-8000-000000000227";
|
|
const otherIssueId = "41000000-0000-4000-8000-000000000227";
|
|
const otherContractId = "51000000-0000-4000-8000-000000000227";
|
|
const otherResultId = "61000000-0000-4000-8000-000000000227";
|
|
const otherAssessmentId = "71000000-0000-4000-8000-000000000227";
|
|
const otherDecisionId = "81000000-0000-4000-8000-000000000227";
|
|
|
|
try {
|
|
// Build the real pre-native schema. Downgrading the latest schema by
|
|
// dropping its unique index is invalid once later tenant FKs use it.
|
|
await migrate(drizzle(sql), { migrationsFolder: directory });
|
|
expect(await sql`SELECT to_regclass('public.native_run_results') AS native_results`).toEqual([{ native_results: null }]);
|
|
await sql`
|
|
INSERT INTO companies (id, name, issue_prefix)
|
|
VALUES (${companyId}, 'Native persistence fixture', 'NPF')
|
|
`;
|
|
await sql`
|
|
INSERT INTO agents (id, company_id, name)
|
|
VALUES (${agentId}, ${companyId}, 'Legacy migration agent')
|
|
`;
|
|
await sql`
|
|
INSERT INTO heartbeat_runs (id, company_id, agent_id, status)
|
|
VALUES (${runId}, ${companyId}, ${agentId}, 'succeeded')
|
|
`;
|
|
await sql`
|
|
INSERT INTO issues (id, company_id, title, status)
|
|
VALUES (${issueId}, ${companyId}, 'Legacy migration issue', 'in_progress')
|
|
`;
|
|
await sql.unsafe(`
|
|
INSERT INTO heartbeat_run_events
|
|
(company_id, run_id, agent_id, seq, event_type, stream, level, message, payload, created_at)
|
|
VALUES
|
|
('${companyId}', '${runId}', '${agentId}', 1, 'legacy.start', 'system', 'info', 'one', '{"bytes":"alpha-1"}'::jsonb, '2026-08-01T00:00:01.000Z'),
|
|
('${companyId}', '${runId}', '${agentId}', 5, 'legacy.log', 'stdout', 'info', 'first-five', '{"bytes":"beta-5a"}'::jsonb, '2026-08-01T00:00:02.000Z'),
|
|
('${companyId}', '${runId}', '${agentId}', 5, 'legacy.log', 'stderr', 'warn', 'duplicate-five', '{"bytes":"gamma-5b"}'::jsonb, '2026-08-01T00:00:03.000Z'),
|
|
('${companyId}', '${runId}', '${agentId}', 9, 'legacy.end', 'system', 'info', 'nine', '{"bytes":"delta-9"}'::jsonb, '2026-08-01T00:00:04.000Z')
|
|
`);
|
|
} finally {
|
|
await sql.end();
|
|
}
|
|
|
|
await applyPendingMigrations(connectionString);
|
|
|
|
const verifySql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const events = await verifySql.unsafe<{
|
|
seq: string;
|
|
event_type: string;
|
|
stream: string;
|
|
level: string;
|
|
message: string;
|
|
payload: { bytes: string };
|
|
created_at: Date;
|
|
}[]>(`
|
|
SELECT seq, event_type, stream, level, message, payload, created_at
|
|
FROM heartbeat_run_events
|
|
WHERE run_id = '${runId}'
|
|
ORDER BY id
|
|
`);
|
|
// 0235 preserves every legacy event while moving only duplicate
|
|
// sequence values above the old run maximum before installing the
|
|
// durable (run_id, seq) uniqueness invariant.
|
|
expect(events.map((event) => Number(event.seq))).toEqual([1, 5, 10, 9]);
|
|
expect(events.map(({ seq: _seq, ...event }) => ({
|
|
...event,
|
|
created_at: event.created_at.toISOString(),
|
|
}))).toEqual([
|
|
{
|
|
event_type: "legacy.start",
|
|
stream: "system",
|
|
level: "info",
|
|
message: "one",
|
|
payload: { bytes: "alpha-1" },
|
|
created_at: "2026-08-01T00:00:01.000Z",
|
|
},
|
|
{
|
|
event_type: "legacy.log",
|
|
stream: "stdout",
|
|
level: "info",
|
|
message: "first-five",
|
|
payload: { bytes: "beta-5a" },
|
|
created_at: "2026-08-01T00:00:02.000Z",
|
|
},
|
|
{
|
|
event_type: "legacy.log",
|
|
stream: "stderr",
|
|
level: "warn",
|
|
message: "duplicate-five",
|
|
payload: { bytes: "gamma-5b" },
|
|
created_at: "2026-08-01T00:00:03.000Z",
|
|
},
|
|
{
|
|
event_type: "legacy.end",
|
|
stream: "system",
|
|
level: "info",
|
|
message: "nine",
|
|
payload: { bytes: "delta-9" },
|
|
created_at: "2026-08-01T00:00:04.000Z",
|
|
},
|
|
]);
|
|
|
|
const runs = await verifySql.unsafe<{ runtime_mode: string; next_event_seq: string }[]>(`
|
|
SELECT runtime_mode, next_event_seq
|
|
FROM heartbeat_runs
|
|
WHERE id = '${runId}'
|
|
`);
|
|
expect(runs.map((run) => ({
|
|
runtimeMode: run.runtime_mode,
|
|
nextEventSeq: Number(run.next_event_seq),
|
|
}))).toEqual([{ runtimeMode: "legacy", nextEventSeq: 11 }]);
|
|
|
|
const nativeRowsBefore = await verifySql.unsafe<{ table_name: string; row_count: number }[]>(`
|
|
SELECT 'completion_contracts' AS table_name, count(*)::int AS row_count FROM completion_contracts
|
|
UNION ALL SELECT 'native_run_results', count(*)::int FROM native_run_results
|
|
UNION ALL SELECT 'native_run_finalizations', count(*)::int FROM native_run_finalizations
|
|
UNION ALL SELECT 'work_assessments', count(*)::int FROM work_assessments
|
|
UNION ALL SELECT 'status_decisions', count(*)::int FROM status_decisions
|
|
UNION ALL SELECT 'status_decision_effects', count(*)::int FROM status_decision_effects
|
|
ORDER BY table_name
|
|
`);
|
|
expect(nativeRowsBefore.every((row) => row.row_count === 0)).toBe(true);
|
|
|
|
await verifySql`
|
|
INSERT INTO companies (id, name, issue_prefix)
|
|
VALUES (${otherCompanyId}, 'Other native persistence fixture', 'ONP')
|
|
`;
|
|
await verifySql`
|
|
INSERT INTO agents (id, company_id, name)
|
|
VALUES (${otherAgentId}, ${otherCompanyId}, 'Other native migration agent')
|
|
`;
|
|
await verifySql`
|
|
INSERT INTO heartbeat_runs (
|
|
id, company_id, agent_id, status, native_issue_id, completion_contract_id
|
|
) VALUES (
|
|
${otherRunId}, ${otherCompanyId}, ${otherAgentId}, 'succeeded',
|
|
${otherIssueId}, ${otherContractId}
|
|
)
|
|
`;
|
|
await verifySql`
|
|
INSERT INTO issues (id, company_id, title, status)
|
|
VALUES (${otherIssueId}, ${otherCompanyId}, 'Other native issue', 'in_progress')
|
|
`;
|
|
await verifySql`
|
|
UPDATE heartbeat_runs
|
|
SET native_issue_id = ${issueId}, completion_contract_id = ${contractId}
|
|
WHERE id = ${runId}
|
|
`;
|
|
|
|
await expect(verifySql`
|
|
INSERT INTO completion_contracts (
|
|
company_id, issue_id, revision, schema_version, policy_version,
|
|
risk, completion_authority, incomplete_criteria_policy, contract_json,
|
|
canonical_sha256, created_by_actor_type, created_by_actor_id
|
|
) VALUES (
|
|
${companyId}, ${otherIssueId}, 1, 'paperclip.completion-contract.v1',
|
|
'policy-v1', 'low', 'server', 'review', ${JSON.stringify({ criteria: [] })}::jsonb,
|
|
'cross-company-contract-sha', 'system', 'migration-test'
|
|
)
|
|
`).rejects.toThrow(/completion_contracts_issue_company_fk/);
|
|
|
|
await verifySql`
|
|
INSERT INTO completion_contracts (
|
|
id, company_id, issue_id, revision, schema_version, policy_version,
|
|
risk, completion_authority, incomplete_criteria_policy, contract_json,
|
|
canonical_sha256, created_by_actor_type, created_by_actor_id
|
|
) VALUES (
|
|
${contractId}, ${companyId}, ${issueId}, 1, 'paperclip.completion-contract.v1',
|
|
'policy-v1', 'low', 'server', 'review', ${JSON.stringify({ criteria: [] })}::jsonb,
|
|
'contract-sha', 'system', 'migration-test'
|
|
)
|
|
`;
|
|
await verifySql`
|
|
INSERT INTO completion_contracts (
|
|
id, company_id, issue_id, revision, schema_version, policy_version,
|
|
risk, completion_authority, incomplete_criteria_policy, contract_json,
|
|
canonical_sha256, created_by_actor_type, created_by_actor_id
|
|
) VALUES (
|
|
${otherContractId}, ${otherCompanyId}, ${otherIssueId}, 1,
|
|
'paperclip.completion-contract.v1', 'policy-v1', 'low', 'server', 'review',
|
|
${JSON.stringify({ criteria: [] })}::jsonb, 'other-contract-sha',
|
|
'system', 'migration-test'
|
|
)
|
|
`;
|
|
|
|
await expect(verifySql`
|
|
INSERT INTO native_run_results (
|
|
company_id, issue_id, run_id, completion_contract_id,
|
|
server_fingerprint, schema_status, result_json, canonical_sha256
|
|
) VALUES (
|
|
${companyId}, ${issueId}, ${otherRunId}, ${contractId},
|
|
'cross-company-run', 'valid', ${JSON.stringify({ summary: "invalid" })}::jsonb,
|
|
'cross-company-run-sha'
|
|
)
|
|
`).rejects.toThrow(/native_run_results_run_contract_owner_fk/);
|
|
|
|
await expect(verifySql`
|
|
INSERT INTO native_run_results (
|
|
company_id, issue_id, run_id, completion_contract_id,
|
|
server_fingerprint, schema_status, result_json, canonical_sha256
|
|
) VALUES (
|
|
${companyId}, ${issueId}, ${runId}, ${otherContractId},
|
|
'cross-company-contract', 'valid', ${JSON.stringify({ summary: "invalid" })}::jsonb,
|
|
'cross-company-contract-sha'
|
|
)
|
|
`).rejects.toThrow(/native_run_results_run_contract_owner_fk/);
|
|
|
|
await verifySql`
|
|
INSERT INTO native_run_results (
|
|
id, company_id, issue_id, run_id, completion_contract_id,
|
|
server_fingerprint, schema_status, result_json, canonical_sha256
|
|
) VALUES (
|
|
${resultId}, ${companyId}, ${issueId}, ${runId}, ${contractId},
|
|
'result-fingerprint', 'valid', ${JSON.stringify({ summary: "done" })}::jsonb, 'result-sha'
|
|
)
|
|
`;
|
|
await verifySql`
|
|
INSERT INTO native_run_results (
|
|
id, company_id, issue_id, run_id, completion_contract_id,
|
|
server_fingerprint, schema_status, result_json, canonical_sha256
|
|
) VALUES (
|
|
${otherResultId}, ${otherCompanyId}, ${otherIssueId}, ${otherRunId},
|
|
${otherContractId}, 'other-result-fingerprint', 'valid',
|
|
${JSON.stringify({ summary: "other" })}::jsonb, 'other-result-sha'
|
|
)
|
|
`;
|
|
|
|
await expect(verifySql`
|
|
INSERT INTO work_assessments (
|
|
company_id, issue_id, run_id, contract_id, result_id,
|
|
trigger_kind, trigger_actor_company_id, prior_issue_status,
|
|
prior_status_version, policy_version, assessment_json, input_digest
|
|
) VALUES (
|
|
${companyId}, ${issueId}, ${runId}, ${contractId}, ${otherResultId},
|
|
'run_terminal', ${companyId}, 'in_progress', 0, 'policy-v1',
|
|
${JSON.stringify({ disposition: "invalid" })}::jsonb,
|
|
'cross-company-assessment-input-sha'
|
|
)
|
|
`).rejects.toThrow(/work_assessments_result_owner_fk/);
|
|
|
|
await expect(verifySql`
|
|
INSERT INTO work_assessments (
|
|
company_id, issue_id, run_id, contract_id, result_id,
|
|
trigger_kind, trigger_actor_company_id, prior_issue_status,
|
|
prior_status_version, policy_version, assessment_json, input_digest
|
|
) VALUES (
|
|
${companyId}, ${issueId}, ${runId}, ${contractId}, ${resultId},
|
|
'run_terminal', ${otherCompanyId}, 'in_progress', 0, 'policy-v1',
|
|
${JSON.stringify({ disposition: "invalid" })}::jsonb,
|
|
'cross-company-trigger-input-sha'
|
|
)
|
|
`).rejects.toThrow(/work_assessments_trigger_actor_company_check/);
|
|
|
|
await verifySql`
|
|
INSERT INTO work_assessments (
|
|
id, company_id, issue_id, run_id, contract_id, result_id,
|
|
trigger_kind, trigger_actor_company_id, prior_issue_status,
|
|
prior_status_version, policy_version, assessment_json, input_digest
|
|
) VALUES (
|
|
${assessmentId}, ${companyId}, ${issueId}, ${runId}, ${contractId}, ${resultId},
|
|
'run_terminal', ${companyId}, 'in_progress', 0, 'policy-v1',
|
|
${JSON.stringify({ disposition: "done" })}::jsonb, 'assessment-input-sha'
|
|
)
|
|
`;
|
|
await verifySql`
|
|
INSERT INTO work_assessments (
|
|
id, company_id, issue_id, run_id, contract_id, result_id,
|
|
trigger_kind, trigger_actor_company_id, prior_issue_status,
|
|
prior_status_version, policy_version, assessment_json, input_digest
|
|
) VALUES (
|
|
${otherAssessmentId}, ${otherCompanyId}, ${otherIssueId}, ${otherRunId},
|
|
${otherContractId}, ${otherResultId}, 'run_terminal', ${otherCompanyId},
|
|
'in_progress', 0, 'policy-v1',
|
|
${JSON.stringify({ disposition: "done" })}::jsonb, 'other-assessment-input-sha'
|
|
)
|
|
`;
|
|
|
|
await expect(verifySql`
|
|
INSERT INTO status_decisions (
|
|
company_id, issue_id, run_id, assessment_id, decision_version,
|
|
policy_version, from_status, to_status, reason_code,
|
|
decision_json, decision_digest
|
|
) VALUES (
|
|
${companyId}, ${issueId}, ${runId}, ${otherAssessmentId}, 1,
|
|
'policy-v1', 'in_progress', 'done', 'native_result_accepted',
|
|
${JSON.stringify({ toStatus: "done" })}::jsonb, 'cross-company-decision-sha'
|
|
)
|
|
`).rejects.toThrow(/status_decisions_assessment_owner_fk/);
|
|
|
|
await verifySql`
|
|
INSERT INTO status_decisions (
|
|
id, company_id, issue_id, run_id, assessment_id, decision_version,
|
|
policy_version, from_status, to_status, reason_code,
|
|
decision_json, decision_digest
|
|
) VALUES (
|
|
${decisionId}, ${companyId}, ${issueId}, ${runId}, ${assessmentId}, 1,
|
|
'policy-v1', 'in_progress', 'done', 'native_result_accepted',
|
|
${JSON.stringify({ toStatus: "done" })}::jsonb, 'decision-sha'
|
|
)
|
|
`;
|
|
await verifySql`
|
|
INSERT INTO status_decisions (
|
|
id, company_id, issue_id, run_id, assessment_id, decision_version,
|
|
policy_version, from_status, to_status, reason_code,
|
|
decision_json, decision_digest
|
|
) VALUES (
|
|
${otherDecisionId}, ${otherCompanyId}, ${otherIssueId}, ${otherRunId},
|
|
${otherAssessmentId}, 1, 'policy-v1', 'in_progress', 'done',
|
|
'native_result_accepted', ${JSON.stringify({ toStatus: "done" })}::jsonb,
|
|
'other-decision-sha'
|
|
)
|
|
`;
|
|
|
|
await expect(verifySql`
|
|
INSERT INTO status_decision_effects (
|
|
company_id, issue_id, decision_id, ordinal, effect_kind,
|
|
target_type, idempotency_key, payload
|
|
) VALUES (
|
|
${companyId}, ${issueId}, ${otherDecisionId}, 0, 'update_issue_status',
|
|
'issue', 'cross-company-decision-effect', ${JSON.stringify({ status: "done" })}::jsonb
|
|
)
|
|
`).rejects.toThrow(/status_decision_effects_decision_owner_fk/);
|
|
|
|
await verifySql`
|
|
INSERT INTO status_decision_effects (
|
|
company_id, issue_id, decision_id, ordinal, effect_kind,
|
|
target_type, idempotency_key, payload
|
|
) VALUES (
|
|
${companyId}, ${issueId}, ${decisionId}, 0, 'update_issue_status',
|
|
'issue', 'decision-effect-1', ${JSON.stringify({ status: "done" })}::jsonb
|
|
)
|
|
`;
|
|
|
|
await expect(verifySql`
|
|
INSERT INTO native_run_finalizations (
|
|
run_id, company_id, issue_id, phase, result_id, assessment_id, decision_id
|
|
) VALUES (
|
|
${runId}, ${companyId}, ${issueId}, 'committed', ${resultId},
|
|
${assessmentId}, ${otherDecisionId}
|
|
)
|
|
`).rejects.toThrow(/native_run_finalizations_decision_owner_fk/);
|
|
|
|
await verifySql`
|
|
INSERT INTO native_run_finalizations (
|
|
run_id, company_id, issue_id, phase, result_id, assessment_id, decision_id
|
|
) VALUES (
|
|
${runId}, ${companyId}, ${issueId}, 'committed', ${resultId}, ${assessmentId}, ${decisionId}
|
|
)
|
|
`;
|
|
|
|
await verifySql`UPDATE issues SET title = 'Renamed legacy issue' WHERE id = ${issueId}`;
|
|
await verifySql`UPDATE issues SET status = 'done' WHERE id = ${issueId}`;
|
|
const issues = await verifySql.unsafe<{ status: string; status_version: string }[]>(`
|
|
SELECT status, status_version
|
|
FROM issues
|
|
WHERE id = '${issueId}'
|
|
`);
|
|
expect(issues.map((issue) => ({
|
|
status: issue.status,
|
|
statusVersion: Number(issue.status_version),
|
|
}))).toEqual([{ status: "done", statusVersion: 1 }]);
|
|
|
|
await verifySql`DELETE FROM "drizzle"."__drizzle_migrations" WHERE "hash" = ${nativePersistenceHash}`;
|
|
await verifySql`DELETE FROM "drizzle"."__drizzle_migrations" WHERE "hash" = ${eventSequenceUniquenessHash}`;
|
|
} finally {
|
|
await verifySql.end();
|
|
}
|
|
|
|
await expect(applyPendingMigrations(connectionString)).resolves.toBeUndefined();
|
|
await expect(inspectMigrations(connectionString)).resolves.toMatchObject({
|
|
status: "upToDate",
|
|
});
|
|
|
|
const replaySql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const replayed = await replaySql.unsafe<{
|
|
status_version: string;
|
|
next_event_seq: string;
|
|
trigger_count: number;
|
|
finalization_count: number;
|
|
}[]>(`
|
|
SELECT
|
|
issue.status_version,
|
|
run.next_event_seq,
|
|
(
|
|
SELECT count(*)::int
|
|
FROM pg_trigger
|
|
WHERE tgname = 'paperclip_issue_status_version_trigger'
|
|
AND NOT tgisinternal
|
|
) AS trigger_count,
|
|
(
|
|
SELECT count(*)::int
|
|
FROM native_run_finalizations
|
|
WHERE run_id = '${runId}'
|
|
) AS finalization_count
|
|
FROM issues issue
|
|
CROSS JOIN heartbeat_runs run
|
|
WHERE issue.id = '${issueId}' AND run.id = '${runId}'
|
|
`);
|
|
expect(replayed.map((row) => ({
|
|
statusVersion: Number(row.status_version),
|
|
nextEventSeq: Number(row.next_event_seq),
|
|
triggerCount: row.trigger_count,
|
|
finalizationCount: row.finalization_count,
|
|
}))).toEqual([{
|
|
statusVersion: 1,
|
|
nextEventSeq: 11,
|
|
triggerCount: 1,
|
|
finalizationCount: 1,
|
|
}]);
|
|
await replaySql`UPDATE issues SET assignee_agent_id = ${agentId} WHERE id = ${issueId}`;
|
|
expect(await replaySql`SELECT status_version::int FROM issues WHERE id = ${issueId}`)
|
|
.toEqual([{ status_version: 2 }]);
|
|
await replaySql`UPDATE issues SET status = 'backlog', assignee_agent_id = null,
|
|
assignee_user_id = 'handoff-owner' WHERE id = ${issueId}`;
|
|
expect(await replaySql`SELECT status_version::int FROM issues WHERE id = ${issueId}`)
|
|
.toEqual([{ status_version: 3 }]);
|
|
const assignmentVersionHash = await migrationHash("0358_issue_assignment_status_version.sql");
|
|
await replaySql`DELETE FROM "drizzle"."__drizzle_migrations" WHERE "hash" = ${assignmentVersionHash}`;
|
|
await applyPendingMigrations(connectionString);
|
|
expect(await replaySql`SELECT status_version::int FROM issues WHERE id = ${issueId}`)
|
|
.toEqual([{ status_version: 3 }]);
|
|
expect(await replaySql`SELECT count(*)::int AS count FROM pg_trigger
|
|
WHERE tgname = 'paperclip_issue_assignment_version_trigger' AND NOT tgisinternal`)
|
|
.toEqual([{ count: 1 }]);
|
|
await replaySql`UPDATE issues SET assignee_user_id = null WHERE id = ${issueId}`;
|
|
expect(await replaySql`SELECT status_version::int FROM issues WHERE id = ${issueId}`)
|
|
.toEqual([{ status_version: 4 }]);
|
|
} finally {
|
|
await replaySql.end();
|
|
}
|
|
},
|
|
60_000,
|
|
);
|
|
|
|
it(
|
|
"replays the idempotent provider trace migration",
|
|
async () => {
|
|
const connectionString = await createTempDatabase();
|
|
const hash = await migrationHash(
|
|
"0234_provider_trace_records.sql",
|
|
);
|
|
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
await sql`
|
|
DELETE FROM "drizzle"."__drizzle_migrations"
|
|
WHERE "hash" = ${hash}
|
|
`;
|
|
} finally {
|
|
await sql.end();
|
|
}
|
|
|
|
await expect(
|
|
applyPendingMigrations(connectionString),
|
|
).resolves.toBeUndefined();
|
|
await expect(inspectMigrations(connectionString)).resolves.toMatchObject({
|
|
status: "upToDate",
|
|
});
|
|
},
|
|
30_000,
|
|
);
|
|
|
|
it(
|
|
"removes retired model profiles from live records and configuration revisions",
|
|
async () => {
|
|
const connectionString = await createTempDatabase();
|
|
const hash = await migrationHash("0236_remove_cheap_model_profiles.sql");
|
|
const companyId = "10000000-0000-4000-8000-000000000236";
|
|
const agentId = "20000000-0000-4000-8000-000000000236";
|
|
const issueId = "30000000-0000-4000-8000-000000000236";
|
|
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
|
|
try {
|
|
await sql`
|
|
INSERT INTO companies (id, name, issue_prefix)
|
|
VALUES (${companyId}, 'Model profile migration fixture', 'MPF')
|
|
`;
|
|
await sql`
|
|
INSERT INTO agents (id, company_id, name, runtime_config)
|
|
VALUES (
|
|
${agentId},
|
|
${companyId},
|
|
'Legacy model profile agent',
|
|
'{"heartbeat":{"enabled":true},"modelProfiles":{"cheap":{"model":"legacy"}}}'::jsonb
|
|
)
|
|
`;
|
|
await sql`
|
|
INSERT INTO issues (id, company_id, title, assignee_adapter_overrides)
|
|
VALUES (
|
|
${issueId},
|
|
${companyId},
|
|
'Legacy model profile issue',
|
|
'{"modelProfile":"cheap","workingDirectory":"/workspace"}'::jsonb
|
|
)
|
|
`;
|
|
await sql`
|
|
INSERT INTO agent_config_revisions (
|
|
company_id,
|
|
agent_id,
|
|
changed_keys,
|
|
before_config,
|
|
after_config
|
|
)
|
|
VALUES (
|
|
${companyId},
|
|
${agentId},
|
|
'["runtimeConfig"]'::jsonb,
|
|
'{"name":"Legacy model profile agent","runtimeConfig":{"modelProfiles":{"cheap":{"model":"legacy-before"}},"heartbeat":{"enabled":true}}}'::jsonb,
|
|
'{"name":"Legacy model profile agent","runtimeConfig":{"modelProfiles":{"cheap":{"model":"legacy-after"}},"heartbeat":{"enabled":false}}}'::jsonb
|
|
)
|
|
`;
|
|
await sql`
|
|
DELETE FROM "drizzle"."__drizzle_migrations"
|
|
WHERE "hash" = ${hash}
|
|
`;
|
|
} finally {
|
|
await sql.end();
|
|
}
|
|
|
|
await applyPendingMigrations(connectionString);
|
|
|
|
const verifySql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
|
try {
|
|
const [result] = await verifySql.unsafe<{
|
|
runtime_config: Record<string, unknown>;
|
|
assignee_adapter_overrides: Record<string, unknown> | null;
|
|
before_config: Record<string, unknown>;
|
|
after_config: Record<string, unknown>;
|
|
}[]>(`
|
|
SELECT
|
|
agent.runtime_config,
|
|
issue.assignee_adapter_overrides,
|
|
revision.before_config,
|
|
revision.after_config
|
|
FROM agents agent
|
|
JOIN issues issue ON issue.company_id = agent.company_id
|
|
JOIN agent_config_revisions revision ON revision.agent_id = agent.id
|
|
WHERE agent.id = '${agentId}' AND issue.id = '${issueId}'
|
|
`);
|
|
|
|
expect(result.runtime_config).toEqual({ heartbeat: { enabled: true } });
|
|
expect(result.assignee_adapter_overrides).toEqual({ workingDirectory: "/workspace" });
|
|
expect(result.before_config).toEqual({
|
|
name: "Legacy model profile agent",
|
|
runtimeConfig: { heartbeat: { enabled: true } },
|
|
});
|
|
expect(result.after_config).toEqual({
|
|
name: "Legacy model profile agent",
|
|
runtimeConfig: { heartbeat: { enabled: false } },
|
|
});
|
|
} finally {
|
|
await verifySql.end();
|
|
}
|
|
},
|
|
30_000,
|
|
);
|
|
});
|