Files
PaperClipAI/server/scripts/profile-tool-projection/register.mjs
T
DottaandPaperclip 00fea0da52 refactor(perf): reuse database test setup for profile benchmark (#18683)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Tool profiles govern the tools that agents can use.
> - Profile listing and app attention shared nested selector scans that
consumed the server CPU.
> - #18681 and #18673 replaced those scans with indexes for each
computation.
> - Their performance benchmarks need consistent setup and reproducible
per-service measurements.
> - This pull request uses the shared helper entry point and retains the
exact measurement harness and results.
> - Reviewers can inspect both the CPU savings and the limits of the
evidence.

## Linked Issues or Issue Description

Refs #18681 and #18673.

**What existing behavior does this improve?**

The consistency of benchmark setup and the reproducibility of evidence
for the two merged CPU fixes.

**Current behavior**

The profile-review benchmark imports the database starter directly. The
matching benchmark uses the shared server test-helper entry point. On
the current base, that helper re-exports the database starter, which
already supports `PAPERCLIP_TEST_POSTGRES_ADMIN_URL`. The previous
committed benchmark measures both services together; the per-service
measurements and separate HTTP/Inspector runs lacked a committed
harness.

**Proposed behavior**

Use the same test-helper entry point in both benchmarks. Retain the
compile, per-service timing, Inspector, and HTTP-probe programs with
their exact commands and raw samples. Preserve the existing embedded and
admin-server database behavior.

**Reason and benefit**

A shared entry point keeps benchmark setup consistent. The committed
harness lets reviewers reproduce and compare each real owning service
before and after the merged fixes. The failure-path checks verify that
result-write errors still release the database and probe resources.

**Breaking changes**

None. This change affects benchmark setup and documentation.

## What Changed

- Import the database starter from the repository test helper in
`benchmark-profile-review-projection.ts`.
- Record separate CPU and elapsed measurements for profile listing and
app attention.
- Retain the actual compile, per-service measurement, Inspector capture,
and separate-process HTTP probe harness.
- Always tear down the test database, HTTP server, and probe even when
result writing fails.
- Retain all timing and responsiveness samples as sanitized JSON.
- State the runtime, fixture, source revisions, and limits of the local
comparison.

## Verification

- 24 focused tests passed across selector matching, pending projection,
review-service freshness, and effective assignments.
- Four selected real database-backed service/route tests passed for
summaries, review changes, auto-allow behavior, and company isolation.
The other 494 tests in that file were not selected.
- Injected an `EISDIR` result-write failure into both profiling programs
on an owned local admin server. Each child exited with the expected
error and removed its disposable database. The responsiveness child also
terminated its probe/server handles.
- The changed benchmark completed with a configured local PostgreSQL
admin URL. Its full responses matched the baseline, and query/row counts
matched.
- The exact documented compile, benchmark, and responsiveness commands
completed from the committed harness at
`ca458baddc6461fb6c888984246e329e2bb4bdfe`. The JSON records that build
revision and the four owner source hashes.
- Five alternating samples per service and implementation used Node
24.22.2 with compiled ES modules. No runtime TypeScript loader ran in
those measurements.
- Median process CPU from the committed harness: profile listing 3,164
ms to 385 ms; app attention 2,466 ms to 221 ms. Both versions returned
identical ordered results and made seven queries per service.
- Separate Inspector profiles confirmed that the repeated scope and
summary scans were removed. Three separate-process HTTP probe trials per
service showed less event-loop blocking. These are local synthetic
service results, not hosted page-load acceptance.
- Initial local setup failed on missing embedded PostgreSQL library
aliases and an unbuilt plugin SDK. Restoring local aliases and building
the SDK resolved those setup failures. The original failed setup logs
were retained; the 28 selected tests then passed.
- The merged summary fix CI run
[41535304968](https://github.com/paperclipai/paperclip/actions/runs/41535304968)
hit two 15-second timeouts in the unchanged attention-retention history
cases. The server shard passed unchanged on one failed-jobs-only retry.
Original and retry logs are retained. The compared runs used different
runners and synthetic merge trees, so the original timeout cause is not
established. No timeout or assertion was changed.
- Full repository typecheck, test matrix, and build passed in CI on
final head `2f484ed4284f3fa632d8e73d9be084827d1c987e`: 54 successful
check runs, three expected skips, one non-blocking Paperclip review, and
a successful Snyk status. No failures or pending checks remain. Those
broad commands were not rerun locally for this benchmark-tooling
follow-up; the 28 selected local regressions and the actual
profiling/failure-path programs ran locally.
- Fresh Greptile is 5/5 on that exact head. Both review P2 threads are
resolved. The Paperclip review's current-base documentation note is
corrected in this description; its optional tooling suggestions remain
outside this bounded change.

## Risks

- The test helper creates a unique disposable database on the configured
admin server and removes it afterward. The operator must use a database
server where those test operations are appropriate.
- Synthetic data and shared-host activity affect absolute timings. The
measurements do not establish a production latency percentage or
capacity limit.
- Production behavior is covered by the two linked merged fixes. This
follow-up changes only benchmark setup and documentation.

## Model Used

OpenAI GPT-6 through Codex, with code execution and repository tools.
The exact runtime model revision and context window were not exposed in
this session. The original linked performance fixes record their own
model details.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-02 16:50:58 -06:00

14 lines
655 B
JavaScript

import { registerHooks } from 'node:module';
import { pathToFileURL, fileURLToPath } from 'node:url';
import path from 'node:path';
const root = fileURLToPath(new URL("../../..", import.meta.url));
registerHooks({ resolve(specifier, context, next) {
for (const [name, dir] of [['@paperclipai/db', 'packages/db'], ['@paperclipai/shared', 'packages/shared']]) {
if (specifier === name || specifier.startsWith(name + '/')) {
const file = specifier === name ? 'index' : specifier.slice(name.length + 1);
return next(pathToFileURL(path.join(root, dir, 'dist', `${file}.js`)).href, context);
}
}
return next(specifier, context);
} });