mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 16:27:18 +02:00
ea4aa185b0f3bc30be1c5fc160cdfd0eb433fe41
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
85f4205de3 |
perf(test): reuse a migrated embedded Postgres template across suites (#15861)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Pull requests to this repository run a large CI matrix. The slowest
check sets the time a contributor waits for a green run.
> - After #15853, the twelve `server (n/12)` shards are the slowest
check in every fully green PR run created since its merge (518s to 548s;
`Build` is 482s to 504s). Each shard is about 60s of setup and 454s to
490s of Vitest, with one worker, so its wall time is the sum of its
suites.
> - From two shard logs, 22 to 24 of the 71 to 74 suites in a shard
start an embedded Postgres cluster. Each one runs `initdb`, starts the
cluster, and applies all 331 migrations before its first test. The
smallest such suite takes 6s with one or two trivial tests. On top of
that, the `getEmbeddedPostgresTestSupport` probe starts and stops a
second throwaway cluster at import time in 19 of those suites, which is
why their per-file import overhead is 3.5s to 4.1s against 0.9s to 1.0s
for suites without Postgres.
> - A cluster data directory that was shut down cleanly can be copied
and started again as an independent cluster, and the migrated schema is
the same for every suite.
> - This pull request builds one migrated cluster per host, publishes
its data directory as a template under the OS temp directory, and starts
every later suite from a copy of it. The probe reuses the template
instead of starting a cluster.
> - The benefit is about 7s to 8s per Postgres suite on the CI runners
(measured 3.2s to 0.35s per start and 0.75s to 0.03s per probe locally),
about 120s to 170s per server shard, and a faster local test loop for
every suite that uses the fixture.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
The `server (n/12)` PR checks are the slowest check in every fully green
PR run since 2026-10-11 05:27Z. Their Vitest step runs suites one at a
time, and about a third of each shard's time is embedded Postgres setup
that produces the same migrated cluster every time.
**Subsystem affected**
Test infrastructure: the embedded Postgres test fixture in
`@paperclipai/db` (`packages/db/src/test-embedded-postgres.ts`), used by
349 server suites, the db package suites, and the CLI suites.
**Current behavior**
`startEmbeddedPostgresTestDatabase` runs `initdb`, starts the cluster,
creates the database, and applies all 331 migrations for every suite.
`getEmbeddedPostgresTestSupport` starts and stops a second throwaway
cluster once per test file, because the module-level cache does not
survive Vitest's per-file isolation. Measured locally: 3.2s per suite
start plus 0.4s cleanup, and 0.75s per probe. On the CI runners the
smallest Postgres suite takes 6s and the probe adds 2.5s to 3s of import
time per file.
**Proposed behavior**
The first caller on a host builds one migrated cluster, stops it, and
renames its data directory to
`$TMPDIR/paperclip-embedded-postgres-templates-<uid>/<key>`. Every later
caller copies the template (about 90 MB) and starts Postgres on the
copy, then still runs `ensurePostgresDatabase` and
`applyPendingMigrations`, which are no-ops on a current template. The
probe treats a published template as proof that the host can run the
cluster. The key hashes the embedded-postgres version, the initdb flags,
the user, and the name and content of every migration file plus the
journal, so a changed migration builds a new template.
`PAPERCLIP_TEST_POSTGRES_TEMPLATE=0` bypasses the cache. Any template
failure falls back to the fresh path.
A template is a Postgres data directory, and starting it executes its
configuration, so the cache is private to the current OS user: the root
name carries the uid, the root is created with mode 0700, and both the
root and each template are used only when `lstat` shows a real directory
(not a symlink) owned by the current user with no group or world write
bit. A root that fails those checks (for example one pre-created by
another user in a shared `/tmp`) is left alone and the process uses
fresh clusters.
**Reason and benefit**
The slowest check decides the wall-clock time of every PR run. Measured
locally, a template start takes 0.3s to 0.4s instead of 3.2s, cleanup
0.15s instead of 0.43s, and the probe 0.03s instead of 0.75s. Four
server suites that passed in both modes dropped by 3.4s to 6.0s each;
the db `backup-lib` suite dropped from 42.7s to 20.9s. The CI runners
are about twice as slow, so the expected saving per server shard is 120s
to 170s.
**Breaking changes**
None. The returned database is fully migrated in both paths, each suite
still gets its own cluster and data directory, and a failure on the
template path falls back to the existing fresh path.
## What Changed
- `packages/db/src/test-embedded-postgres.ts`: adds the template cache.
`computeEmbeddedPostgresTemplateKey` hashes the inputs that determine a
migrated cluster. `buildEmbeddedPostgresTemplate` starts a fresh cluster
through the existing bounded retry, creates the database, applies
migrations, waits for the real `stop()` (30s bound), and publishes the
data directory with an atomic rename; a concurrent builder that loses
the rename discards its copy.
`startEmbeddedPostgresFromTemplateWithRetry` copies the template into a
fresh temp directory, sets mode 0700, and starts the cluster with the
same per-attempt port and directory rotation as the fresh path.
`startEmbeddedPostgresTestDatabase` tries the template first and falls
back to the fresh path with one warning per process.
`probeEmbeddedPostgresSupport` returns supported when a template exists
or can be built, and keeps its throwaway-cluster path for the case where
the template could not be published. Templates for other keys are pruned
after seven days. The connection strings and initdb flags move into
shared constants; no other behavior changes.
- `packages/db/src/test-embedded-postgres.test.ts`: two unit tests for
the key (stable for identical inputs; changes with migration content, a
new file, flags, version, or user; hashes this package's real migrations
directory) and two real-cluster tests (the published template is
complete and private, a suite starts from it in `template` mode with
migrations up to date, a table written to one copy is not visible in the
next copy, and `PAPERCLIP_TEST_POSTGRES_TEMPLATE=0` runs the `fresh`
path). The real-cluster tests skip on hosts without embedded Postgres,
like every other fixture suite.
- `doc/DEVELOPING.md`: a short section under Test Commands describing
the template, its key, and the bypass variable.
**Review fixes (second commit), one per finding:**
- Private cache root (Greptile P1, Superagent P2). The root is
`$TMPDIR/paperclip-embedded-postgres-templates-<uid>` (user name on
Windows), created with mode 0700. `prepareTemplateRootDir` and
`isUsableTemplate` both reject, via `lstat`, anything that is a symlink,
not a directory, owned by another uid, or writable by group or others; a
rejected root logs one warning and the process uses fresh clusters. A
root created by the first commit with a wider mode is narrowed to 0700
if this user owns it.
- Template failures never skip suites (Greptile P1).
`getEmbeddedPostgresTemplateDir` no longer rejects. A build or root
failure resolves `null` with one warning, and
`probeEmbeddedPostgresSupport` then runs the throwaway-cluster probe, so
only a cluster that cannot start reports `supported: false`. A migration
that cannot apply fails the suite on the fresh path instead of being
hidden by `describe.skip`.
- Idle-based pruning (Greptile P2). Each reuse refreshes the template's
timestamp with `utimes`, so the seven-day prune measures idle time. A
template that disappears after a process resolved its path is not
retried five times: `startEmbeddedPostgresFromTemplateWithRetry` checks
the template before each copy and throws at once, the suite falls back
to a fresh cluster, and the fallback forgets the path so the next caller
in that process rebuilds the template.
- The template-path test enables the cache for itself and restores the
original `PAPERCLIP_TEST_POSTGRES_TEMPLATE` value afterwards (Greptile
P1), so the suite passes under `PAPERCLIP_TEST_POSTGRES_TEMPLATE=0` as
well.
- Eight new tests: `isUsableTemplate` accepts a private cluster
directory and rejects a missing `PG_VERSION`, a leftover
`postmaster.pid`, a symlink, and (POSIX) group- or world-writable modes;
a world-writable pre-created root and a symlinked root both fall back to
fresh clusters, publish nothing, log one warning, and still report
`supported: true` from the probe; a ten-day-old template is refreshed on
reuse; a pruned template is rebuilt on the next start; and a template
build that fails after the cluster started falls through to the fresh
probe. The tests that need an empty cache point `os.tmpdir()` at a
scratch directory (`TMPDIR`, `TEMP` and `TMP` together, so the redirect
holds on Windows as well) and restore it.
**Review fixes (third commit,
|
||
|
Nicky LeachandPaperclip
|
4813ed3f0c |
fix(db): harden embedded Postgres test start with bounded retry (#10540)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The database layer uses embedded Postgres for isolated test runs > - A port probe can fail when another process takes the same port before Postgres binds it > - That race can make a test fail even when the code under test is fine > - This pull request adds bounded retry and clearer error text to the embedded Postgres start path > - The benefit is more stable tests and faster diagnosis when startup still fails ## Linked Issues or Issue Description No public GitHub issue exists for this change. This PR fixes a flaky embedded Postgres test start path. The helper can lose a port between probe and bind. This PR retries the start with a fresh port and a fresh data directory. Related public context: - Refs #7259 - Refs #9769 ## What Changed - Add bounded retry around embedded Postgres initialization and start. - Stop each failed attempt and remove its data directory before the next attempt. - Capture Postgres output in the thrown error so the failure is easier to read. - Add unit coverage for retry success, retry exhaustion, and the improved error text. ## Verification - `pnpm --filter @paperclipai/db exec vitest run src/test-embedded-postgres.test.ts src/embedded-postgres-error.test.ts` - `pnpm --filter @paperclipai/db exec vitest run` - `pnpm --filter @paperclipai/db exec vitest run` passed in the worktree after the change. - `worktree.test.ts > quarantines copied live execution state in seeded worktree databases` passed. - A real cluster loop of 100 starts passed with 0 failures. ## Risks - The retry can hide a real startup fault until the fifth try. - The bound keeps the wait short, and the final error still shows the captured Postgres log. - This change only affects the embedded Postgres test start helper. ## Model Used OpenAI Codex, GPT-5, tool-use enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |