## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The skill library lists current company skills and their agent
attachments.
> - Each list request also reconciles bundled skills, folders, and local
files.
> - These reads return full agent records and large metadata that their
callers do not use.
> - Conditional requests do this work before they return an unchanged
response.
> - This pull request selects the fields each caller needs and avoids
unused folder lookups.
> - The result is less database transfer and fewer round trips while
reconciliation stays current.
## Linked Issues or Issue Description
Refs #17941. That change reuses bundled categories within an import.
This change removes other repeated work in the same list path.
**What happened?**
Warm skill-list requests still made 119 SQL calls in a local HTTP
fixture. The folder pass returned full skill metadata. The local-source
pass also returned remote skills. Attachment counts loaded full agent
configurations and monthly spend. Bundled reconciliation resolved folder
paths that its release check did not use.
**Expected behavior**
The list must preserve its current result, attachment counts, access
rules, local-file reconciliation, and bundled release checks. Internal
reads should return only the data their callers need.
**Steps to reproduce**
1. Use a company with many remote skills and agents with large saved
configurations.
2. Read its skill list, then repeat the read with its ETag.
3. Measure the queries and response values before the server returns
`200` or `304`.
4. Run `pnpm --filter @paperclipai/server exec tsx
scripts/benchmark-skill-list.ts --samples=5 --network-delay-ms=3.5` for
a synthetic reproduction.
**Paperclip version or commit**
Baseline service: `e1ba77f06771a27114b3999004b0ca5e3c172dd4`. The branch
is based on current master. The paired mode changes only the
skill-service module; both versions use the same imported helpers.
**Deployment mode**
Built from source. Local HTTP server and disposable PostgreSQL 17. No
hosted deployment is part of this PR's local proof.
## What Changed
- Read only the desired-skill selections of nonterminated company agents
for attachment counts. Resolve each agent's references once against the
full visible skill list. Preserve IDs, keys, aliases, duplicate
handling, and ambiguous slugs.
- Read only `sourceKind` from metadata for bundled folder
classification.
- Limit local-source reconciliation to local-path rows.
- Skip folder-path enrichment only for bundled reconciliation. Normal
imports and list responses keep their folder paths.
- Add real-database byte regressions and a local HTTP benchmark. The
benchmark can alternate the old and new services against one database,
actor, and process. It checks complete response equality and conditional
responses.
The company-skill service owns these reads. The query predicates and
freshness checks remain at their existing owners. This change adds no
cache, schema change, or new background work.
## Verification
- The attachment byte regression failed before the fix: 601,418 captured
response-value bytes exceeded its 2,000-byte bound. The same case passed
after the fix. It checks fresh configuration edits, IDs and keys,
duplicate aliases, ambiguous slugs, paused agents, terminated agents,
and company scope.
- 111 focused service and authorization tests passed against isolated
real PostgreSQL 17 databases. They include full result equality,
retained stored Markdown and metadata, local-file updates, release
checks, project visibility, and import authorization.
- The default embedded PostgreSQL probe skipped the first local attempt
on this Mac. The completed run used a local-only bootstrap adapter for
disposable databases. Queries, migrations, services, authorization, and
filesystem work used their real implementations. The adapter is not part
of this PR.
- In repeated synthetic HTTP byte runs, calls fell from 119 to 98 per
request. Result-value bytes fell from about 7.75 MB to 1.64 MB per
request, about 79%. These values are measured before Drizzle decoding.
They exclude protocol framing and do not measure provider billing.
- A separate-process run without the byte observer did not show a
latency gain. The paired experiment removes that comparison gap: it
alternates both services against the same database and checks identical
complete responses. On final commit
`3076fe1402085f7286ff58e642d7caca892a5f80`, five observer-free pairs had
median times of 2.582 s before and 1.850 s after (28.3% lower). The
candidate was faster in all five pairs. Its range was 1.802–2.199 s; the
baseline range was 2.327–2.725 s. Every normal response had the same
complete body hash, and both versions returned the expected unchanged
conditional responses. These are local service/HTTP timings with
synthetic data and 3.5 ms delay per network direction. They are not
hosted acceptance.
- Final-head workspace typecheck and build passed. The full local test
command did not complete because embedded PostgreSQL bootstrap failed on
this Mac; I stopped that owned runner after 16 native-runtime cases
failed. All 16 cases then passed when rerun alone against disposable
PostgreSQL 17 using their existing external-database path. The 111
focused PostgreSQL 17 tests above also completed successfully. This does
not constitute a completed full local suite; all current-head remote CI
gates passed (55 successful checks and 2 skipped). Module-boundary and
whitespace checks pass. An initial nested package command used the
global pnpm; subsequent commands were pinned to pnpm 9.15.4.
- The first canary dry run failed while installing an unrelated
standalone package: npm returned 404 for the newly published Rollup
4.66.0 tarball. The exact download subsequently returned HTTP 200. The
one retry passed without code changes.
- Benchmark commands: add `--capture-values=false` for observer-free
timings. Add `--baseline-ref=e1ba77f06771a27114b3999004b0ca5e3c172dd4
--capture-values=false` for paired service acceptance. The normal mode
uses the production router. The paired mode uses equivalent default-list
HTTP handlers and a synthetic authenticated board actor; it does not
test login.
## Risks
The SQL selection must stay in sync if the desired-skill format or
folder classifier needs new fields. Existing parsing and reference
resolution still interpret the selected data.
The synthetic fixture has large unused payloads. It proves the mechanism
and preserves required content. Its percentage is not a projection of
whole-instance transfer or billed savings. Hosted latency still needs
verification after merge and deployment.
## Model Used
OpenAI GPT-6 through Codex, with repository inspection, shell execution,
real PostgreSQL, and local HTTP testing. The host does not expose a
precise backend revision or context-window size.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Co-authored-by: Paperclip <noreply@paperclip.ing>
Server tests that need a real PostgreSQL process must use
./helpers/embedded-postgres.ts instead of constructing embedded-postgres
directly.
The shared helper creates a throwaway data directory and a reserved-safe
loopback port for each test database. This protects the live Paperclip
control-plane Postgres from server vitest runs; see PAP-2033 for the incident
that introduced this guard.
