feat(db): classify migrations for application under the running release (#20913)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - `packages/db` owns the schema and its migrations. There are 467
migrations, and several ship each day
> - A zero-downtime deploy starts the new release while the previous
release still serves the same database
> - Core does not say whether a migration is safe to apply under the
previous release. A deployer must treat each pending migration as a
reason to pause, or a person must review it
> - On stacks that may not pause, releases wait for hours
> - Core owns the schema, so Core must own this knowledge and publish it
as data
> - This pull request adds a conservative classifier, an in-file
override, the published result, and a guard for new migrations
> - The benefit is that a deployer can apply routine compatible
migrations automatically, and incompatible changes become visible at
review time

## Linked Issues or Issue Description

**Subsystem affected**

Database migrations (`packages/db`), the cloud migrator artifact
(`scripts/cloud-migrator-artifacts.mjs`), and the pull-request policy
job.

**Problem or motivation**

Core has no machine-readable statement that a migration can be applied
while the previous release still reads and writes the database. A
deployer cannot tell an added nullable column from a dropped column. It
must pause for both, or refuse the release.

**Proposed solution**

Classify each migration as `online_compatible`, `requires_staging`, or
`unknown`. Publish the result in the migrator manifest. Fail a pull
request when a migration it adds is `unknown` and has no reviewed
override in the file.

**Alternatives considered**

- A hand-kept list in the deployer. It goes stale and it is not reviewed
with the SQL.
- A sibling JSON file in the artifact. It needs a workflow change, and
the attestation of `manifest.json` would not cover it.

**Roadmap alignment**

This is release infrastructure. It adds no product feature.

## What Changed

- `packages/db/src/migration-compatibility.ts`:
`classifyMigrationSql(sql)` returns `{ compatibility, classified,
reasons, override, overrideError }`. It parses statement by statement
(breakpoints, `;`, comments, string and dollar quoting, guarded `DO`
blocks). A statement that it does not recognize is `unknown`. A
migration takes the worst class of its statements.
- Lock and duration hazards have their own reason codes
(`lock_hazard_*`) and classify as `unknown`, not as compatible with a
warning.
- Override: `-- paperclip:migration-compat online_compatible:
<justification>` (or `requires_staging`) in the leading comment block of
the migration file. The justification is mandatory (10 characters or
more). A malformed, repeated, or misplaced directive is an error and
never improves the verdict.
- A table counts as new after a plain `CREATE TABLE`, or after `CREATE
TABLE IF NOT EXISTS` when no earlier migration created that table.
`classifyMigrationSequence` classifies migrations in apply order with
that knowledge.
- `packages/db/scripts/migration-compatibility.ts` (run with `pnpm exec
tsx` in `packages/db`): classify files, print the distribution, print
JSON, or run the guard (`--check`). No new package script.
- Published data: `manifest.json` of the cloud migrator artifact has a
new additive field `migrationCompatibility`. `version` stays `1`. No
existing field changes. The build reads the SQL from the packed
`db.tgz`, so the data describes what the migrator applies. `validate`
(publisher) and `verify` reject data that is malformed or does not match
the packed migrations. The workflow file does not change.
- Guard for new migrations:
`.github/scripts/check-pr-migration-order.mjs` (the existing "Validate
migration ordering against target branch" policy step) now also
classifies the migrations that the pull request adds. `check:migrations`
runs the same guard against the merge base with `origin/master`, so the
`@paperclipai/db` typecheck and build report it locally. `unknown`
without an override fails. `requires_staging` prints a notice and
passes. A new migration that uses `CONCURRENTLY` also fails when it is
not in `MIGRATIONS_WITH_TOP_LEVEL_STATEMENTS`
(`packages/db/src/client.ts`), because the migrator would run it inside
a transaction.
- `doc/DATABASE.md`: new section "Migration compatibility with the
running release" under "Schema change workflow".

Published shape:

```json
"migrationCompatibility": {
  "version": 1,
  "migrations": [
    {
      "idx": 468,
      "tag": "0468_busy_wendell_rand",
      "sqlSha256": "56a6e272…",
      "compatibility": "online_compatible",
      "classified": "online_compatible",
      "override": false,
      "reasons": [
        { "code": "add_column", "compatibility": "online_compatible", "message": "Adds a column that is nullable or has a constant default.", "statements": [1] }
      ]
    }
  ]
}
```

`sqlSha256` is the hash that the migrator records in the migration
history. `overrideJustification` is present only when `override` is
`true`. Entries are in apply order (journal `idx`, then file name; the
shipped journal repeats `idx` 178).

### Distribution over the existing migrations

No existing migration is rewritten or annotated.

| Class | All 467 | Newest 100 |
| --- | --- | --- |
| `online_compatible` | 228 (48.8%) | 67 |
| `requires_staging` | 114 (24.4%) | 12 |
| `unknown` | 125 (26.8%) | 21 |

Migrations per blocking reason (one migration can have several):

| Migrations | Class | Reason |
| --- | --- | --- |
| 74 | `requires_staging` | `create_unique_index_existing_table` |
| 70 | `unknown` | `lock_hazard_create_index` |
| 66 | `unknown` | `lock_hazard_validated_foreign_key` |
| 56 | `unknown` | `lock_hazard_data_backfill` |
| 54 | `unknown` | `drop_constraint` |
| 41 | `requires_staging` | `add_validated_check` |
| 33 | `unknown` | `drop_index` |
| 16 | `unknown` | `do_block_procedural` |
| 15 | `unknown` | `replace_function` |
| 13 | `unknown` | `create_trigger_existing_table` |
| 12 | `unknown` | `data_change_existing_table` |
| 10 | `unknown` | `drop_object`, `lock_hazard_set_not_null`,
`set_default` (each) |
| 8 | `requires_staging` | `drop_column` |

## Verification

- `pnpm --filter @paperclipai/db exec vitest run
src/migration-compatibility.test.ts`: 113 tests pass. They cover a
positive and a negative case for each rule, each reason code, the
override (missing, short, misplaced, and repeated justification are
rejected), unknown by default, the guard, the shipped migrations, the
`CONCURRENTLY` list, and the command line.
- `node --test scripts/cloud-migrator-artifacts.test.mjs
.github/scripts/tests/check-pr-migration-order.test.mjs`: 21 tests pass.
- Real bundle at the first commit of this branch: `preview-artifacts.mjs
pack`, then `cloud-migrator-artifacts.mjs build`, `validate`, and
`verify-install` pass. The manifest lists 467 migrations and is 312 KB.
This run found the repeated journal `idx`, which the second commit
handles.
- `pnpm --filter @paperclipai/db run check:migrations` passes. With
`--check --base HEAD~150` the guard fails on three `unknown` migrations
and prints notices for two `requires_staging` migrations.
- `pnpm verify` exits 2 because `.github/scripts/**` is a global CI
input. Required CI supplies the broad coverage.
- Not run: the publish job (it needs the master-only AWS role), and no
deployer reads the new field yet.

## Risks

- The manifest grows from about 1 KB to about 312 KB. A consumer that
rejects unknown manifest fields would fail. The verifier in this
repository ignores unknown fields; consumers outside this repository
were not inspected.
- From this merge, an open pull request that adds an `unknown` migration
fails the policy step and the `@paperclipai/db` typecheck until the
author rewrites the migration or adds the override. `drizzle-kit` emits
validated foreign keys and plain indexes for existing tables, so this is
common. The error text and `doc/DATABASE.md` give the fix.
- `ADD CONSTRAINT ... NOT VALID` classifies as `online_compatible`.
PostgreSQL still enforces it on new writes, so the author must make sure
that the previous release does not write rows that violate it.
- The classifier reads SQL only. It cannot see that new code writes
values that the previous release cannot read (for example a new enum
value). The `requires_staging` override exists for that case.
- `master` cannot fail the guard: the guard only judges migrations that
are not on the merge base.

## Model Used

- Claude (Anthropic), model ID `claude-fable-5-2`, in Claude Code with
tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Fable 5.2 authored and GitHub committed 2026-11-11 14:14:40 -06:00
1 parent fc14f1ade2
commit e8e548e856
9 files changed
+1621 -28

No files matched your search

+116 -3
View File
@@ -7,6 +7,7 @@ import { copyFileSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "
import os from "node:os";
import path from "node:path";
import { pathToFileURL } from "node:url";
import { gunzipSync } from "node:zlib";
import { assertMetadata, tarManifest, versionFor } from "./preview-artifacts.mjs";
export const artifactBase = "https://d1p6rlowie26tp.cloudfront.net/cloud-migrators/v1";
@@ -30,10 +31,111 @@ function assertDescriptor(pin, extension) {
}
}
const compatibilityValues = ["online_compatible", "requires_staging", "unknown"];
const migrationsPrefix = "package/dist/migrations/";
/** Regular files under `prefix` in a package archive, by archive path. */
function tarFiles(bytes, prefix) {
const tar = gunzipSync(bytes, { maxOutputLength: 128 * 1024 * 1024 });
const files = new Map();
for (let offset = 0; offset + 512 <= tar.length;) {
const h = tar.subarray(offset, offset + 512);
if (h.every((v) => v === 0)) break;
const field = (start, size) => h.subarray(start, start + size).toString("utf8").split("\0")[0].trim();
const sizeText = field(124, 12);
if (!/^[0-7]+$/.test(sizeText)) throw new Error("Invalid package archive.");
const size = Number.parseInt(sizeText, 8);
if (offset + 512 + size > tar.length) throw new Error("Truncated package archive.");
const name = `${field(345, 155) ? field(345, 155) + "/" : ""}${field(0, 100)}`;
if ([0, 48].includes(h[156]) && name.startsWith(prefix)) {
if (files.has(name)) throw new Error("Duplicate package archive entry.");
files.set(name, tar.subarray(offset + 512, offset + 512 + size));
}
offset += 512 + Math.ceil(size / 512) * 512;
}
return files;
}
/** The journal of the packed migrator, in apply order, with each file's bytes. */
function packedMigrations(dbBytes) {
const files = tarFiles(dbBytes, migrationsPrefix);
const journal = files.get(`${migrationsPrefix}meta/_journal.json`);
if (!journal) throw new Error("Migrator package has no migration journal.");
const entries = JSON.parse(journal.toString("utf8")).entries;
if (!Array.isArray(entries) || entries.length === 0) throw new Error("Migrator package has an empty migration journal.");
// The migrator's apply order: journal idx, then file name. The shipped
// journal repeats an idx, so idx alone does not identify a migration.
return [...entries].sort((left, right) => left.idx - right.idx || (String(left.tag) < String(right.tag) ? -1 : 1)).map(({ idx, tag }) => {
const sql = files.get(`${migrationsPrefix}${tag}.sql`);
if (!sql) throw new Error(`Migrator package is missing migration ${tag}.`);
// The same digest the migrator records in the database's migration history.
return { idx, tag, sql, sqlSha256: createHash("sha256").update(sql).digest("hex") };
});
}
/**
* Whether each packed migration can be applied while the previous release is
* still running. `classify` is classifyMigrationSequence from @paperclipai/db:
* it takes the SQL of every migration in apply order. The overrides it honours
* come only from the packed SQL itself.
*/
export function migrationCompatibility(dbBytes, classify) {
const packed = packedMigrations(dbBytes);
const results = classify(packed.map(({ sql }) => sql.toString("utf8")));
if (!Array.isArray(results) || results.length !== packed.length) throw new Error("The classifier did not return one result per migration.");
return {
version: 1,
migrations: packed.map(({ idx, tag, sqlSha256 }, position) => {
const result = results[position];
return {
idx, tag, sqlSha256,
compatibility: result.compatibility,
classified: result.classified,
override: Boolean(result.override),
...(result.override ? { overrideJustification: result.override.justification } : {}),
reasons: result.reasons.map(({ code, compatibility, message, statements }) => ({ code, compatibility, message, statements: [...statements] })),
};
}),
};
}
function assertMigrationCompatibility(value) {
const fail = () => { throw new Error("Invalid migration compatibility data."); };
if (value?.version !== 1 || !Array.isArray(value.migrations) || value.migrations.length === 0) fail();
const text = (candidate) => typeof candidate === "string" && candidate.length > 0 && candidate.length <= 2000;
value.migrations.forEach((migration, position) => {
const previous = value.migrations[position - 1];
if (!Number.isSafeInteger(migration?.idx) || migration.idx < 0 ||
(previous && (migration.idx < previous.idx || (migration.idx === previous.idx && !(migration.tag > previous.tag)))) ||
typeof migration.tag !== "string" || !/^\d{4}_[A-Za-z0-9_]+$/.test(migration.tag) ||
typeof migration.sqlSha256 !== "string" || !/^[0-9a-f]{64}$/.test(migration.sqlSha256) ||
!compatibilityValues.includes(migration.compatibility) || !compatibilityValues.includes(migration.classified) ||
typeof migration.override !== "boolean" || !Array.isArray(migration.reasons)) fail();
// An override always carries its justification and never yields `unknown`.
if (migration.override ? !text(migration.overrideJustification) || migration.compatibility === "unknown" : migration.overrideJustification !== undefined) fail();
for (const reason of migration.reasons) {
if (typeof reason?.code !== "string" || !/^[a-z_]+$/.test(reason.code) || !compatibilityValues.includes(reason.compatibility) || !text(reason.message) ||
!Array.isArray(reason.statements) || !reason.statements.every((statement) => Number.isSafeInteger(statement) && statement > 0)) fail();
}
});
}
export function assertManifest(manifest, sha) {
if (manifest?.version !== 1 || manifest.sourceSha !== sha || manifest.packageVersion !== versionFor(sha)) throw new Error("Artifact source identity mismatch.");
for (const name of names) assertDescriptor(manifest.packages?.[name], "tgz");
assertDescriptor(manifest.lockfile, "json");
// Additive since the classifier shipped; manifests published earlier lack it.
if (manifest.migrationCompatibility !== undefined) assertMigrationCompatibility(manifest.migrationCompatibility);
}
/** The published classification must describe exactly the migrations in db.tgz. */
function assertCompatibilityMatchesPackage(manifest, dbBytes) {
const published = manifest.migrationCompatibility?.migrations;
const packed = packedMigrations(dbBytes);
if (!published || published.length !== packed.length || packed.some(({ idx, tag, sqlSha256 }, position) =>
published[position].idx !== idx || published[position].tag !== tag || published[position].sqlSha256 !== sqlSha256)) {
throw new Error("Migration compatibility data does not match the packed migrations.");
}
}
export function assertLockfile(lock, manifest) {
@@ -61,8 +163,9 @@ export function assertLockfile(lock, manifest) {
}
}
export function buildBundle(directory, sha, { exec = execFileSync } = {}) {
export function buildBundle(directory, sha, { exec = execFileSync, classify } = {}) {
versionFor(sha);
if (typeof classify !== "function") throw new Error("buildBundle needs the migration compatibility classifier.");
directory = path.resolve(directory);
const packages = {};
for (const name of names) {
@@ -83,8 +186,11 @@ export function buildBundle(directory, sha, { exec = execFileSync } = {}) {
lock.packages[""].dependencies = { "@paperclipai/db": versionFor(sha) };
for (const name of names) lock.packages[`node_modules/@paperclipai/${name}`].resolved = packages[name].url;
const lockBytes = Buffer.from(JSON.stringify(lock) + "\n");
const manifest = { version: 1, sourceSha: sha, packageVersion: versionFor(sha), packages, lockfile: descriptor(lockBytes, "json") };
const dbBytes = readFileSync(path.join(directory, "db.tgz"));
const manifest = { version: 1, sourceSha: sha, packageVersion: versionFor(sha), packages, lockfile: descriptor(lockBytes, "json"),
migrationCompatibility: migrationCompatibility(dbBytes, classify) };
assertManifest(manifest, sha);
assertCompatibilityMatchesPackage(manifest, dbBytes);
assertLockfile(lock, manifest);
writeFileSync(path.join(directory, "package-lock.json"), lockBytes);
writeFileSync(path.join(directory, "manifest.json"), JSON.stringify(manifest) + "\n");
@@ -103,6 +209,7 @@ export function validateBundle(directory, sha) {
const bytes = readFileSync(path.join(directory, `${name}.tgz`));
verifyBytes(bytes, manifest.packages[name]);
assertMetadata(tarManifest(bytes), `@paperclipai/${name}`, sha);
if (name === "db") assertCompatibilityMatchesPackage(manifest, bytes);
}
const bytes = readFileSync(path.join(directory, "package-lock.json"));
verifyBytes(bytes, manifest.lockfile);
@@ -160,6 +267,7 @@ export async function verifyPublished(sha, fetchImpl = fetch, { verifyProvenance
const bytes = await download(manifest.packages[name].url, fetchImpl);
verifyBytes(bytes, manifest.packages[name]);
assertMetadata(tarManifest(bytes), `@paperclipai/${name}`, sha);
if (name === "db" && manifest.migrationCompatibility !== undefined) assertCompatibilityMatchesPackage(manifest, bytes);
}));
const lock = await download(manifest.lockfile.url, fetchImpl);
verifyBytes(lock, manifest.lockfile);
@@ -200,7 +308,12 @@ export async function publishBundle(directory, sha, { exec = execFileSync, fetch
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
const [command, directory, sha] = process.argv.slice(2);
try {
if (command === "build") buildBundle(directory, sha);
if (command === "build") {
// Type-stripped by Node; the module has no imports, so the build job
// needs nothing beyond the checkout.
const { classifyMigrationSequence } = await import(new URL("../packages/db/src/migration-compatibility.ts", import.meta.url).href);
buildBundle(directory, sha, { classify: classifyMigrationSequence });
}
else if (command === "validate") validateBundle(directory, sha);
else if (command === "verify-install") verifyInstall(directory, sha);
else if (command === "publish") await publishBundle(directory, sha);
+95 -14
View File
@@ -7,25 +7,45 @@ import { mkdtempSync, readFileSync, writeFileSync, rmSync } from "node:fs";
import os from "node:os";
import path from "node:path";
import { gzipSync } from "node:zlib";
import { artifactBase, assertManifest, assertLockfile, buildBundle, descriptor, validateBundle, verifyPublished, publishBundle } from "./cloud-migrator-artifacts.mjs";
import { createHash } from "node:crypto";
import { artifactBase, assertManifest, assertLockfile, buildBundle, descriptor, migrationCompatibility, validateBundle, verifyPublished, publishBundle } from "./cloud-migrator-artifacts.mjs";
import { previewManifest, versionFor } from "./preview-artifacts.mjs";
import { classifyMigrationSequence as classify } from "../packages/db/src/migration-compatibility.ts";
const sha = "a".repeat(40);
const migrations = {
"0000_first": 'CREATE TABLE IF NOT EXISTS "widgets" ("id" uuid PRIMARY KEY NOT NULL);\n--> statement-breakpoint\nCREATE INDEX "widgets_idx" ON "widgets" ("id");\n',
// IF NOT EXISTS does not make a table from an earlier migration new again.
"0001_second": 'CREATE TABLE IF NOT EXISTS "widgets" ("id" uuid);\n--> statement-breakpoint\nCREATE INDEX "widgets_again_idx" ON "widgets" ("id");\n',
"0002_third": '-- paperclip:migration-compat online_compatible: no release since 0.2 reads issues.legacy\nALTER TABLE "issues" DROP COLUMN "legacy";\n',
};
function tarEntry(name, content) {
const bytes = Buffer.from(content);
const header = Buffer.alloc(512);
header.write(name); header.write(bytes.length.toString(8).padStart(11, "0"), 124, 11); header[156] = 48;
// A real tar header, so npm can install this fixture as well as inspect it.
header.fill(32, 148, 156);
const sum = header.reduce((a, b) => a + b, 0);
header.write(sum.toString(8).padStart(6, "0") + "\0 ", 148, 8);
const padded = Buffer.alloc(Math.ceil(bytes.length / 512) * 512); bytes.copy(padded);
return Buffer.concat([header, padded]);
}
function dbArchive(files = migrations) {
const journal = { version: "7", dialect: "postgresql", entries: Object.keys(files).map((tag, idx) => ({ idx, version: "7", when: idx, tag, breakpoints: true })) };
return gzipSync(Buffer.concat([
tarEntry("package/package.json", JSON.stringify(previewManifest({ name: "@paperclipai/db", dependencies: {} }, sha))),
tarEntry("package/dist/migrations/meta/_journal.json", JSON.stringify(journal)),
...Object.entries(files).map(([tag, sql]) => tarEntry(`package/dist/migrations/${tag}.sql`, sql)),
Buffer.alloc(1024),
]));
}
function fixture(t) {
const dir = mkdtempSync(path.join(os.tmpdir(), "migrator-artifact-test-"));
t.after(() => rmSync(dir, { recursive: true, force: true }));
for (const name of ["db", "shared"]) {
const bytes = Buffer.from(JSON.stringify(previewManifest({ name: `@paperclipai/${name}`, dependencies: {} }, sha)));
const header = Buffer.alloc(512);
header.write("package/package.json"); header.write(bytes.length.toString(8).padStart(11, "0"), 124, 11); header[156] = 48;
// A real tar header, so npm can install this fixture as well as inspect it.
header.fill(32, 148, 156);
const sum = header.reduce((a, b) => a + b, 0);
header.write(sum.toString(8).padStart(6, "0") + "\0 ", 148, 8);
const padded = Buffer.alloc(Math.ceil(bytes.length / 512) * 512); bytes.copy(padded);
writeFileSync(path.join(dir, `${name}.tgz`), gzipSync(Buffer.concat([header, padded, Buffer.alloc(1024)])));
}
const manifest = buildBundle(dir, sha, { exec: (cmd, args, options) => {
writeFileSync(path.join(dir, "db.tgz"), dbArchive());
writeFileSync(path.join(dir, "shared.tgz"), gzipSync(Buffer.concat([
tarEntry("package/package.json", JSON.stringify(previewManifest({ name: "@paperclipai/shared", dependencies: {} }, sha))), Buffer.alloc(1024)])));
const manifest = buildBundle(dir, sha, { classify, exec: (cmd, args, options) => {
assert.equal(cmd, "npm"); assert.ok(args.includes("--ignore-scripts"));
const localRoot = JSON.parse(readFileSync(path.join(options.cwd, "package.json")));
assert.deepEqual(localRoot.dependencies, { "@paperclipai/db": "file:db.tgz", "@paperclipai/shared": "file:shared.tgz" });
@@ -53,6 +73,67 @@ test("bundle pins the exact source pair and complete lockfile without new npm lo
assert.deepEqual(await verifyPublished(sha, fetchImpl), manifest);
});
test("manifest publishes how each packed migration behaves under the previous release", (t) => {
const { manifest } = fixture(t);
const digest = (tag) => createHash("sha256").update(migrations[tag]).digest("hex");
assert.deepEqual(manifest.migrationCompatibility.version, 1);
assert.deepEqual(manifest.migrationCompatibility.migrations.map(({ reasons, ...entry }) => ({ ...entry, reasons: reasons.map((reason) => reason.code) })), [
{ idx: 0, tag: "0000_first", sqlSha256: digest("0000_first"), compatibility: "online_compatible", classified: "online_compatible", override: false,
reasons: ["create_table", "create_index_new_table"] },
{ idx: 1, tag: "0001_second", sqlSha256: digest("0001_second"), compatibility: "unknown", classified: "unknown", override: false,
reasons: ["create_table_if_not_exists", "lock_hazard_create_index"] },
{ idx: 2, tag: "0002_third", sqlSha256: digest("0002_third"), compatibility: "online_compatible", classified: "requires_staging", override: true,
overrideJustification: "no release since 0.2 reads issues.legacy", reasons: ["drop_column"] },
]);
const reason = manifest.migrationCompatibility.migrations[1].reasons[1];
assert.deepEqual(Object.keys(reason), ["code", "compatibility", "message", "statements"]);
assert.deepEqual([reason.compatibility, reason.statements], ["unknown", [2]]);
// Everything a consumer pinned before this field existed is unchanged.
assert.deepEqual(Object.keys(manifest), ["version", "sourceSha", "packageVersion", "packages", "lockfile", "migrationCompatibility"]);
assert.equal(manifest.version, 1);
});
test("migration compatibility is optional for earlier manifests and fails closed when malformed or stale", async (t) => {
const { dir, manifest, files, fetchImpl } = fixture(t);
const earlier = structuredClone(manifest); delete earlier.migrationCompatibility;
assert.doesNotThrow(() => assertManifest(earlier, sha));
for (const mutate of [
(c) => { c.version = 2; },
(c) => { c.migrations = []; },
(c) => { c.migrations[0].compatibility = "safe"; },
(c) => { c.migrations[0].classified = undefined; },
(c) => { c.migrations[0].sqlSha256 = "abc"; },
(c) => { c.migrations[2].idx = 0; },
(c) => { c.migrations[1].tag = c.migrations[0].tag; c.migrations[1].idx = 0; },
(c) => { c.migrations[0].override = true; },
(c) => { c.migrations[2].overrideJustification = ""; },
(c) => { c.migrations[2].compatibility = "unknown"; },
(c) => { c.migrations[0].overrideJustification = "without an override"; },
(c) => { c.migrations[1].reasons[0].statements = [0]; },
(c) => { c.migrations[1].reasons[1].compatibility = "fine"; },
(c) => { c.migrations[1].reasons[0].code = 7; },
]) {
const bad = structuredClone(manifest); mutate(bad.migrationCompatibility);
assert.throws(() => assertManifest(bad, sha), /Invalid migration compatibility data/);
}
assert.throws(() => buildBundle(dir, sha), /classifier/);
// A well-formed classification of different SQL must not be published or accepted.
const stale = structuredClone(manifest);
stale.migrationCompatibility = migrationCompatibility(dbArchive({ ...migrations, "0001_second": 'DROP TABLE "issues";\n' }), classify);
assertManifest(stale, sha);
files.set(`${artifactBase}/${sha}/manifest.json`, Buffer.from(JSON.stringify(stale)));
await assert.rejects(verifyPublished(sha, fetchImpl), /does not match the packed migrations/);
writeFileSync(path.join(dir, "manifest.json"), JSON.stringify(stale));
assert.throws(() => validateBundle(dir, sha), /does not match the packed migrations/);
writeFileSync(path.join(dir, "manifest.json"), JSON.stringify(earlier));
assert.throws(() => validateBundle(dir, sha), /does not match the packed migrations/);
assert.throws(() => migrationCompatibility(dbArchive({}), classify), /empty migration journal/);
assert.throws(() => migrationCompatibility(dbArchive(), () => []), /one result per migration/);
// The shipped journal repeats idx 178; order then follows the file name, as in the migrator.
const repeated = structuredClone(manifest); repeated.migrationCompatibility.migrations[1].idx = 0;
assert.doesNotThrow(() => assertManifest(repeated, sha));
});
test("source identity, content hashes, size, and origin fail closed", async (t) => {
const { dir, manifest, files, fetchImpl } = fixture(t);
for (const mutate of [
@@ -109,7 +190,7 @@ test("download failures and oversized objects never count as available", async (
test("real npm ci installs the new pair from pinned archives with an empty cache", async (t) => {
const { dir } = fixture(t);
// Real npm resolution uses local archives; neither package version exists on npm.
const manifest = buildBundle(dir, sha);
const manifest = buildBundle(dir, sha, { classify });
const lock = JSON.parse(readFileSync(path.join(dir, "package-lock.json")));
const requests = [];
const server = createServer((req, res) => {