mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-07 16:11:46 +02:00
Nicky LeachandClaude Opus 5
aeef493f4a
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - `@paperclipai/db` owns the Drizzle schema and the migration history > - Drizzle writes a full copy of the schema as a snapshot for each generated migration. Each snapshot is now about 1.3 MB. > - The `meta/` folder is about 103 MB. That is about half of each checkout and each worktree. The build also copies it into `dist`, so the published `@paperclipai/db` package is 112.8 MB unpacked. > - `drizzle-kit generate` reads only the newest snapshot. The runtime migrator reads only the `.sql` files and `_journal.json`. > - This pull request keeps the newest 5 snapshots and removes snapshots from `dist`. > - The benefit is a checkout that is about 100 MB smaller, and a published package that is about 1.5 MB instead of 113 MB. ## Linked Issues or Issue Description Refs #11240, #11254, #12333 (earlier snapshot work: diff collapse, binary diffs, drift repair) **What existing behavior does this improve?** The size of the Drizzle migration snapshots in the repository and in the published `@paperclipai/db` package. **Subsystem affected** `packages/db`: migrations and the build. **Current behavior** `packages/db/src/migrations/meta/` holds 141 snapshots (102.8 MB). The size grows faster than the number of migrations, because each snapshot is a full copy of the schema. `build` runs `cp -r src/migrations dist/migrations`. `@paperclipai/db@2026.916.0` contains 147 snapshot files. It is 112.8 MB unpacked and 5.3 MB as a tarball. **Proposed behavior** Keep the newest 5 snapshots. `generate` deletes older snapshots after it runs. `dist` gets only `*.sql` and `meta/_journal.json`. **Reason and benefit** - In drizzle-kit 0.31.10, `generate` sorts `meta/*` and diffs against the last snapshot only (`bin.cjs`, `preparePrevSnapshot`). The [generate docs](https://orm.drizzle.team/docs/drizzle-kit-generate) also say it compares against "the most recent" snapshot. - Gaps in the snapshot history already work. 138 of the 280 migrations never had a snapshot, because they were written by hand before `doc/DATABASE.md` required `generate`. - We keep 5 snapshots instead of 1. This lets a developer undo the latest generated migration, and it keeps the `prevId` chain for recent branches. - Git keeps the history cheaply. The 631 snapshot versions use only 2.2 MB of the pack, because git stores each version as a delta of the previous one. The cost is in the checked-out files, not the clone download. Therefore this change does not use git-lfs and does not rewrite history. Old snapshots stay available with `git show <rev>:<path>`. ## What Changed - `packages/db/package.json`: a new `prune:snapshots` script keeps the newest 5 `*_snapshot.json` files. It is `ls | sort -r | tail -n +6 | xargs rm -f`, which works with the BSD tools on macOS and the GNU tools on Linux. `generate` runs this script after `drizzle-kit generate`. - `packages/db/package.json`: `build` copies only `src/migrations/*.sql` and `meta/_journal.json` into `dist/migrations`. - Deleted 137 older snapshots. `0277`–`0281` remain. - `chat-identity-migration-reconciliation.test.ts`: removed the walk over snapshots `0254`–`0268`. Those files do not change after merge, and the walk would fail after pruning. The journal-order assertions in the same test remain. `migration-snapshot-drift.test.ts` still makes sure that the newest snapshot matches the schema. - `doc/DATABASE.md`: documented the retention rule. - The snapshots were already marked `linguist-generated=true -diff -merge` by `packages/db/.gitattributes` (#11240). No change there. `git check-attr` confirms it. ## Verification - `pnpm --filter @paperclipai/db exec vitest run`: 43 files, 160 tests pass. - `pnpm --filter @paperclipai/db build`: `dist/migrations` contains 280 `.sql` files and `meta/_journal.json`. It is 1.5 MB, compared with about 105 MB before. - `prune:snapshots` was run on macOS (BSD) and in `debian:stable-slim` (GNU findutils 4.10). With 7 fixture snapshots, it keeps the newest 5. When 5 or fewer are present, it deletes nothing and exits 0 on both. ## Risks - Low risk. Runtime migration does not read snapshots. The only commands that read older snapshots are `drizzle-kit check` and `drizzle-kit drop`. No script or CI job calls them, and they still have the newest 5 snapshots. - A branch that is open now can still add its own snapshot. If the branch conflicts, the rule is the same as today: renumber the migration and run `generate` again. - When we upgrade to drizzle-kit v1 (folder per migration), check whether its new cross-branch "commutativity" checks need a longer snapshot history. ## Model Used - Claude Opus 5 (`claude-opus-5`) in Claude Code, with tool use (shell, file edits, web fetch). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
67 lines
1.9 KiB
JSON
67 lines
1.9 KiB
JSON
{
|
|
"name": "@paperclipai/db",
|
|
"version": "0.3.1",
|
|
"license": "MIT",
|
|
"homepage": "https://github.com/paperclipai/paperclip",
|
|
"bugs": {
|
|
"url": "https://github.com/paperclipai/paperclip/issues"
|
|
},
|
|
"repository": {
|
|
"type": "git",
|
|
"url": "https://github.com/paperclipai/paperclip",
|
|
"directory": "packages/db"
|
|
},
|
|
"type": "module",
|
|
"exports": {
|
|
".": "./src/index.ts",
|
|
"./*": "./src/*.ts"
|
|
},
|
|
"publishConfig": {
|
|
"access": "public",
|
|
"exports": {
|
|
".": {
|
|
"types": "./dist/index.d.ts",
|
|
"import": "./dist/index.js"
|
|
},
|
|
"./*": {
|
|
"types": "./dist/*.d.ts",
|
|
"import": "./dist/*.js"
|
|
}
|
|
},
|
|
"main": "./dist/index.js",
|
|
"types": "./dist/index.d.ts"
|
|
},
|
|
"files": [
|
|
"dist"
|
|
],
|
|
"bundleDependencies": [
|
|
"embedded-postgres"
|
|
],
|
|
"scripts": {
|
|
"check:migrations": "tsx src/check-migration-numbering.ts && tsx src/check-migration-safety.ts",
|
|
"build": "pnpm run check:migrations && tsc && rm -rf dist/migrations && mkdir -p dist/migrations/meta && cp src/migrations/*.sql dist/migrations/ && cp src/migrations/meta/_journal.json dist/migrations/meta/",
|
|
"clean": "rm -rf dist",
|
|
"typecheck": "pnpm run check:migrations && tsc --noEmit",
|
|
"generate": "pnpm run check:migrations && pnpm run clean && tsc -p tsconfig.json && drizzle-kit generate && pnpm run prune:snapshots",
|
|
"prune:snapshots": "ls src/migrations/meta/*_snapshot.json | sort -r | tail -n +6 | xargs rm -f",
|
|
"migrate": "pnpm run check:migrations && tsx src/migrate.ts",
|
|
"seed": "tsx src/seed.ts"
|
|
},
|
|
"dependencies": {
|
|
"@paperclipai/shared": "workspace:*",
|
|
"drizzle-orm": "^0.45.2",
|
|
"embedded-postgres": "^18.1.0-beta.16",
|
|
"postgres": "^3.4.9"
|
|
},
|
|
"devDependencies": {
|
|
"@types/node": "^24.0.0",
|
|
"drizzle-kit": "^0.31.10",
|
|
"tsx": "^4.23.12",
|
|
"typescript": "^7.0.2",
|
|
"vitest": "^4.1.11"
|
|
},
|
|
"engines": {
|
|
"node": ">=24.11.0"
|
|
}
|
|
}
|