mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 13:01:05 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - CI selects only the server tests that a pull request can affect. The Vitest selection graph trusts a setup file only while its content matches a reviewed hash, and it follows every local import of that setup > - #21768 added a `beforeEach` to `server/src/__tests__/setup-supertest.ts` that resets the instance settings cache, through `../services/instance-settings-cache.js` > - The reviewed hash was not updated, and the cache module was not reviewed > - So server test selection fails with "Opaque setup dependency", and CI runs every server test instead. The qualification test also fails on every pull request that changes CI files > - This pull request reviews the new setup content and the cache reset > - The benefit is that server test selection works again, and pull requests that change CI files pass `qualify_test_selection` ## Linked Issues or Issue Description Refs #21768. **What happened?** `qualify_test_selection` ("Incremental CI qualification") fails on pull requests that change CI files, for example `perf/canary-git-numbering` and `fix/server-shard2-hang`: ``` ✖ real graph retains direct and indirect filesystem, process and opaque dependencies Error: Opaque setup dependency: .../server/src/__tests__/setup-supertest.ts ``` With only the hash updated, selection of the real server project stops at `Opaque setup package: drizzle-orm`, which `packages/db/src/work-signals.ts` imports through the cache module. **Expected behavior** The reviewed setup permits file selection again. A change to the setup, its helpers or the cache module still selects every server test. **Steps to reproduce** 1. On `master`, run `node --test scripts/ci/test-selection.integration.test.mjs`. 2. The real graph test fails with "Opaque setup dependency". ## What Changed - `scripts/ci/vitest-selection-graph.mjs`: - Pin the current `setup-supertest.ts` content. - Add `server/src/services/instance-settings-cache.ts` as a reviewed setup dependency. The setup calls only `invalidateInstanceSettingsCache`, which increments an in-memory generation. The module performs no IO when it loads. Its `drizzle-orm`, `drizzle-orm/pg-core` and `@paperclipai/db/work-signals` imports serve the read-through path, which the setup does not call. The review lists them as external, like `supertest/lib/test.js` for the setup itself. - `scripts/ci/test-selection.integration.test.mjs`: the fixture copies the cache module with the reviewed setup files. It also checks that a change to the cache module selects every test. ## Verification - `node --test scripts/ci/test-selection*.test.mjs scripts/ci/vitest-impact-report.integration.test.mjs` (the `qualify_test_selection` step): 42 passed. On `master`, the real graph case fails. - The "Qualify selection against the real Paperclip workspace" script from `ci-incremental-qualification.yml`, run locally with `NODE_ENV=test VITEST=true`: passes. - Real server project selection (`selectProjectTests` with the `@paperclipai/server` project): a change to `server/src/services/heartbeat-run-summary.ts` selects 2004 of 2188 tests. A change to `instance-settings-cache.ts` sets `setupChanged` and selects all 2188. ## Risks - A later edit to `instance-settings-cache.ts` or `setup-supertest.ts` disables the review again until the hash is renewed. Selection then fails closed and runs every server test, as it does today. - Changes to `packages/db/src/work-signals.ts` no longer select every server test through the setup. The setup's cache reset does not call it. Tests that import it directly are still selected. - CI scripts and tests only. No product code changes. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>