mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 03:31:44 +01:00
test(skills): budget list read bytes for fixture rows, not the bundled catalog (#21285)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The skill library list reconciles the bundled skill catalog, folders, and local sources on each read > - #18023 added a test that limits the `company_skills` bytes one warm `list()` call reads, so the folder pass and the list query keep their narrow projections > - The limit counts every `company_skills` read, so it also counts the bundled catalog rows that every company gets > - Each new bundled skill adds about 9.6 KB to that total, so pull requests that only add skills under `skills/` fail a test about projections > - This pull request measures a catalog-only company next to the fixture company and applies the same limit to the difference > - The benefit is that the test fails only when the fixture rows are read too widely, not when the catalog grows ## Linked Issues or Issue Description Refs #18023 (added the test). Refs #20760, #20764, #20767, #20768, #20769 (failed on this test). **What happened?** `companySkillService.list > keeps folder classification and stored content while narrowing inventory refresh reads` failed on five open pull requests that only add bundled skills under `skills/`. The error was `expected 1163219 to be less than 950000`. Other runs had 983607, 1035956, 982321, and 985561. The test passes on master. **Expected behavior** The test must fail when `list()` reads the unused metadata or the Markdown of company skill rows. It must not fail when the bundled catalog gets larger. **Steps to reproduce** 1. Check out master. 2. Run `git checkout <head of #20760> -- skills` to add its 34 bundled skills. 3. Run `pnpm --filter @paperclipai/server exec vitest run src/__tests__/company-skills-service.test.ts -t "narrowing inventory refresh"`. 4. The test fails with `expected 1163504 to be less than 950000`. **Paperclip version or commit** `9b613562b70b7bea9e7562ddc1087d4c1b1788b1` (master). ## What Changed - The test creates a second company that has only the bundled catalog. - The test lists both companies warm and captures the `company_skills` response-value bytes of each call. - The test applies the existing 950,000-byte limit to the difference. The difference is about 663 KB and comes only from the ten fixture rows. - The test also checks that the fixture company lists the catalog rows plus the ten fixture rows. - No service code changes. ## Verification Where the bytes come from (master, one warm `list()` call): | Read | Master catalog | Catalog of #20760 | | --- | --- | --- | | Main list query (`company-skills.ts:3324`) | 671,003 B, 20 rows | 691,694 B, 52 rows | | Bundled reconciliation `getByKey` reads (one per bundled skill) | 165,423 B, 13 calls | 471,771 B, 45 calls | | Total (old assertion) | 836,465 B, pass | 1,163,504 B, fail | | Fixture company minus catalog-only company (new assertion) | 663,153 B, pass | 663,166 B, pass | Red and green: - Old test, master `skills/` plus the `skills/` tree of #20760: fails, `expected 1163504 to be less than 950000`. - New test, same enlarged catalog: passes, difference 663,166. - New test, master: passes, difference 663,153. The new test still catches the regression it exists for. I widened each projection on purpose with the enlarged catalog, ran the test, and reverted: - Folder pass reads the full `metadata` column instead of `sourceKind` only: fails, `expected 1323122 to be less than 950000`. - Main list query also reads `markdown`: fails, `expected 1183162 to be less than 950000`. Commands: - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/company-skills-service.test.ts`: 81 passed. - `pnpm verify`: not completed. It selected 2,615 of 3,435 test files (conservative selection), and I stopped it to keep load low on a shared machine. CI ran the full required checks on the final commit: 54 passed, 4 skipped. ## Risks - Low risk. The change is in one test only. - The new test runs one more warm `list()` call, about 0.5 s more. - The bound is unchanged. A widened projection on the fixture rows still adds at least 540 KB and fails. - This test no longer limits the bytes the bundled catalog adds to each list call. Each bundled skill adds about 9.6 KB per warm call, because `ensureBundledSkills` calls `getByKey` with the full row for each bundled skill. That cost is real but separate. A fix should narrow that read, not let this test limit the catalog size. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits) and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes (not applicable: test-only change) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green (Superagent Security Scan is neutral: "Scan inconclusive") - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
c6c7a61e2c
commit
0c63f0fdb3
1 file changed
+23
-9
@@ -668,8 +668,11 @@ describeEmbeddedPostgres("companySkillService.list", () => {
|
||||
});
|
||||
|
||||
it("keeps folder classification and stored content while narrowing inventory refresh reads", async () => {
|
||||
const companyId = randomUUID();
|
||||
await db.insert(companies).values({ id: companyId, name: "Inventory projections", issuePrefix: "IVP" });
|
||||
const companyId = randomUUID(), catalogOnlyCompanyId = randomUUID();
|
||||
await db.insert(companies).values([
|
||||
{ id: companyId, name: "Inventory projections", issuePrefix: "IVP" },
|
||||
{ id: catalogOnlyCompanyId, name: "Catalog only", issuePrefix: "IVC" },
|
||||
]);
|
||||
const unused = "unused".repeat(11_000);
|
||||
const savedMarkdown = "# Required detail content\n".repeat(2_000);
|
||||
const inserted = await db.insert(companySkills).values(Array.from({ length: 10 }, (_, index) => ({
|
||||
@@ -679,14 +682,25 @@ describeEmbeddedPostgres("companySkillService.list", () => {
|
||||
}))).returning();
|
||||
const before = await svc.list(companyId);
|
||||
expect(before.find(item => item.id === inserted[0]!.id)?.folderPath).toBe("bundled/paperclip-core");
|
||||
const events: Record<string, unknown>[] = [], now = Date.now();
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 1, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { companySkills }, capture);
|
||||
let after: Awaited<ReturnType<typeof svc.list>>;
|
||||
try { after = await svc.list(companyId); } finally { capture.flush(); }
|
||||
const catalogBefore = await svc.list(catalogOnlyCompanyId);
|
||||
async function listWithReadBytes(targetCompanyId: string) {
|
||||
const events: Record<string, unknown>[] = [], now = Date.now();
|
||||
const capture = createFullDatabaseResponseCapture({ start: now - 1, end: now + 59_000 }, event => events.push(event));
|
||||
installDatabaseResponseCapture(db, { companySkills }, capture);
|
||||
let listed: Awaited<ReturnType<typeof svc.list>>;
|
||||
try { listed = await svc.list(targetCompanyId); } finally { capture.flush(); }
|
||||
const bytes = events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1"
|
||||
&& event.table === "company_skills").reduce((sum, event) => sum + Number(event.responseValueBytes), 0);
|
||||
return { listed, bytes };
|
||||
}
|
||||
const catalogOnly = await listWithReadBytes(catalogOnlyCompanyId);
|
||||
expect(catalogOnly.listed).toEqual(catalogBefore);
|
||||
const { listed: after, bytes: catalogAndFixtureBytes } = await listWithReadBytes(companyId);
|
||||
expect(after).toEqual(before);
|
||||
const reads = events.filter(event => event.event === "paperclip.test_db_response_capture.entry.v1" && event.table === "company_skills");
|
||||
expect(reads.reduce((sum, event) => sum + Number(event.responseValueBytes), 0)).toBeLessThan(950_000);
|
||||
expect(after).toHaveLength(catalogBefore.length + inserted.length);
|
||||
const fixtureRowReadBytes = catalogAndFixtureBytes - catalogOnly.bytes;
|
||||
const fixtureRowBudgetWithoutUnusedMetadataOrMarkdownRereads = 950_000;
|
||||
expect(fixtureRowReadBytes).toBeLessThan(fixtureRowBudgetWithoutUnusedMetadataOrMarkdownRereads);
|
||||
const stored = await svc.getById(companyId, inserted[0]!.id);
|
||||
expect(stored?.markdown).toBe(savedMarkdown);
|
||||
expect(stored?.metadata).toEqual({ unused, sourceKind: " paperclip_bundled " });
|
||||
|
||||
Reference in new issue
Block a user