perf(tools): index profile selectors per resolution (#18673)

## Thinking Path

> - Paperclip manages AI agents and their work.
> - Core resolves tool profiles before it prepares runtime MCP
assignments.
> - A selected profile can contain a rule for each individual tool.
> - The resolver and profile summary scan these rules for each catalog
row.
> - This repeats millions of exact comparisons for a large selected
profile.
> - This pull request indexes each selector kind once per computation.
> - The benefit is less server CPU with the same current access
decisions and result contract.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Effective tool assignment and profile summary computation for large
selected profiles.

**Current behavior**

Each catalog row scans the profile's includes and excludes. The work
grows with catalog size times rule count. The runtime assignment path
also computes a profile summary with the same scans.

**Proposed behavior**

Build five selector sets for each include or exclude group. Match
catalog rows with exact set membership. Retain all current database
reads and the full result contract.

**Reason and benefit**

Reduce repeated JavaScript comparisons. On the synthetic PostgreSQL
fixture below, median process CPU per resolution falls from 50.068 ms to
10.129 ms. This is independent of
[#18371](https://github.com/paperclipai/paperclip/pull/18371), which
removes unused administrative results and narrows instruction
descriptors. This PR retains those results and does not include that
change.

## What Changed

- Index application, connection, catalog-entry, tool-name, and risk
selectors in the profile owner.
- Use indexed matching in effective access resolution and profile
summaries.
- Add a differential test against the original single-rule matcher.
Cover null applications, exact special-character names, unknown
selectors, and empty rule groups.
- Add a PostgreSQL regression for per-profile exclusion precedence, the
union of multiple effective profiles, and current rule removal.
- Add a real PostgreSQL resolver benchmark and document reproduction and
limits.

## Verification

Baseline: `410b0e27015495ab8951768b5b0b94727de33785`. Treatment:
`895ee2eec15cf4cdedee4c257fcdddb2a7000a1b`. Node 24.22.2 on macOS arm64.
Source modules run with `tsx`; these are local service measurements, not
production-build or hosted acceptance.

The real owning resolver ran against disposable PostgreSQL with 1,890
active catalog tools, 1,575 individual include/exclude rules, one
deny-by-default effective profile, and 12 agents. The installation
dependency returns an empty list because the fixture has no installed
connections. There are no provider calls. Full returned assignments,
profile summaries, rules, bindings, and names are identical to baseline.

After two warmups per version, seven alternating samples ran ten
resolutions each. Other workers paused CPU measurements and builds.
Unrelated host work was not controlled.

| Per resolution | Baseline median (range) | Change median (range) |
| --- | --- | --- |
| Process CPU | 50.068 ms (46.380–58.880) | 10.129 ms (8.089–11.072) |
| Elapsed time | 50.362 ms (44.799–106.408) | 8.853 ms (7.719–18.590) |
| Queries | 6 | 6 |
| Returned database rows | 3,480 | 3,480 |

Median process CPU falls 79.8%; median elapsed time falls 82.4%. The
long baseline elapsed sample had no matching process CPU increase. Query
and row counts do not decrease. Small or broadly allowed profiles can
save less. This does not establish whole-run latency or hosted
performance.

Reproduce with `server/scripts/benchmark-tool-profile-matching.ts`;
baseline module preparation is in `doc/mcp-discovery-performance.md`.

Passed on the treatment source:

```sh
node node_modules/vitest/vitest.mjs run server/src/__tests__/tool-profile-matching.test.ts server/src/__tests__/tool-effective-assignments.test.ts server/src/services/native-runtime/runtime-context.test.ts server/src/__tests__/heartbeat-runtime-mcp-servers.test.ts
```

All 45 tests in four suites passed. `git diff --check` passed. The first
PostgreSQL test attempt failed at setup because the sandbox blocked its
loopback listener (`EPERM`); the authorized unsandboxed retry passed. No
behavior failure was discarded.

At measured head `895ee2eec15cf4cdedee4c257fcdddb2a7000a1b`, CI
typecheck, build, canary, Runner checks, and all eight browser shards
passed. Server general shard 8/12 failed an unchanged
background-quietness assertion in `heartbeat-run-dispatch-work.test.ts`:
expected 235 select calls, observed 236 after advancing fake timers. Its
cause remains unverified; no baseline reproduction was run.

That head also failed the module-boundary policy because the benchmark
inserted synthetic agents outside the test fixture tree. Follow-up head
`e20165b6eb` moves the fixture body byte-for-byte into
`server/src/__tests__/helpers/tool-profile-matching-fixture.ts`,
following the existing listing benchmark pattern. The production matcher
and benchmark measurement loop are unchanged. `node
scripts/check-module-boundaries.mjs`, a Node 24/tsx fixture import
check, and `git diff --check` pass. The benchmark was not rerun for this
extraction. New-head CI and review are pending. Greptile gave the
measured head 5/5 with no actionable issues. No deployment or live
runtime mutation was performed.

## Risks

Exact equality must remain identical to the original matcher, especially
for null application selectors and exclusions. The differential and
PostgreSQL tests cover these cases. Sets add linear temporary memory in
rule count. They live only for the current computation. Current
company-scoped state reads, profile precedence, revocation, and runtime
assignment digest checks remain in place. Revert this commit to roll
back.

## Model Used

OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with source inspection, code
execution, tests, and local PostgreSQL benchmarking. The context window
size is not available in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

doc/mcp-discovery-performance.md

@@ -106,3 +106,35 @@ Catalog payload memory and policy-matching CPU still grow with the catalog.
 The measurements do not establish a safe production heap limit or the share of
 active installations affected. Verify restart bursts and sustained traffic on the
 actual deployment before reducing its temporary heap allowance.
+
+## Profile selector matching
+
+Effective profile resolution and profile summaries index each profile's include
+and exclude selectors once per decision. Catalog rows then use exact set
+membership checks for application, connection, catalog entry, tool name, and
+risk level. Exclusions still take precedence within a profile; effective profiles
+still contribute the union of their allowed tools. Null application selectors
+retain their exact-match behavior.
+
+These indexes live only for the current computation. Each resolver call still
+reads current company-scoped bindings, profiles, and rules. Query counts and
+returned database rows are unchanged. This reduces JavaScript matching work when
+a profile selects many individual tools; it does not cache authorization or
+remove the runtime assignment digest check.
+
+For a real PostgreSQL comparison, save `projection.ts` and `resolution.ts` from
+the baseline commit beside their current versions as `.baseline-projection.ts`
+and `.baseline-resolution.ts`. Change only the baseline resolver's projection
+import to `./.baseline-projection.js`. Then run from `server/`:
+
+```sh
+node --import tsx scripts/benchmark-tool-profile-matching.ts --baseline "$PWD/src/services/tool-access/profiles/.baseline-resolution.ts"
+```
+
+Remove the two baseline files afterward. The probe uses 1,890 catalog tools,
+1,575 individual include/exclude rules, one effective profile, and 12 agents. It
+checks full assignment-result equality, warms both versions, and alternates
+seven samples of ten resolutions. It reports actual query and returned-row
+counts, process CPU, and elapsed time. The installation dependency returns an
+empty list for this fixture, which has no installed connections. It does not
+measure provider work, whole-run latency, or hosted performance.

server/scripts/benchmark-tool-profile-matching.ts

@@ -0,0 +1,51 @@
+import assert from "node:assert/strict";
+import { pathToFileURL } from "node:url";
+import { and, eq } from "drizzle-orm";
+import { agents, createDb, type Db } from "@paperclipai/db";
+import { startEmbeddedPostgresTestDatabase } from "../src/__tests__/helpers/embedded-postgres.js";
+import { createToolProfileMatchingFixture } from "../src/__tests__/helpers/tool-profile-matching-fixture.js";
+import { createToolProfileResolver } from "../src/services/tool-access/profiles/resolution.js";
+
+const baselinePath = process.argv[process.argv.indexOf("--baseline") + 1];
+if (!process.argv.includes("--baseline") || !baselinePath) throw new Error("--baseline module required");
+const baselineFactory = (await import(pathToFileURL(baselinePath).href)).createToolProfileResolver as typeof createToolProfileResolver;
+const database = await startEmbeddedPostgresTestDatabase("profile-matching-benchmark-");
+const db = createDb(database.connectionString, { maxConnections: 1 });
+let queries = 0, rows = 0;
+const observedDb = new Proxy(db, { get(target, key) {
+  if (key !== "select") return Reflect.get(target, key);
+  return (...fields: Parameters<Db["select"]>) => {
+    const builder = target.select(...fields), from = builder.from.bind(builder);
+    builder.from = ((...args: Parameters<typeof from>) => {
+      const selection = from(...args), execute = selection.execute.bind(selection);
+      selection.execute = async () => { const result = await execute(); queries++; rows += result.length; return result; };
+      return selection;
+    }) as typeof builder.from;
+    return builder;
+  };
+} });
+try {
+  const { company, agent } = await createToolProfileMatchingFixture(db);
+  const dependencies = {
+    assertOptionalAgent: async (companyId: string, agentId: string | null | undefined) => {
+      const valid = await observedDb.select({ id: agents.id }).from(agents).where(and(eq(agents.companyId, companyId), eq(agents.id, agentId!)));
+      assert.equal(valid.length, 1);
+    },
+    resolveInstalledConnectionsForAgent: async () => [],
+    toCatalogEntry: () => { throw new Error("Full catalog definitions are not used by this assignment benchmark"); },
+  };
+  const before = baselineFactory(observedDb, dependencies);
+  const after = createToolProfileResolver(observedDb, dependencies);
+  const invoke = (resolver: typeof after) => resolver.getEffectiveToolAssignmentsForAgent(company.id, agent.id);
+  assert.deepEqual(await invoke(after), await invoke(before));
+  for (let i = 0; i < 2; i++) { await invoke(before); await invoke(after); }
+  for (let round = 0; round < 7; round++) {
+    for (const [label, resolver] of round % 2 ? [["after", after], ["before", before]] as const : [["before", before], ["after", after]] as const) {
+      queries = 0; rows = 0;
+      const cpu = process.cpuUsage(), start = performance.now();
+      for (let i = 0; i < 10; i++) await invoke(resolver);
+      const elapsedMs = performance.now() - start, used = process.cpuUsage(cpu);
+      console.log(JSON.stringify({ round, label, operations: 10, tools: 1890, rules: 1575, agents: 12, queries, rows, elapsedMs, cpuMs: (used.user + used.system) / 1000 }));
+    }
+  }
+} finally { await db.$client.end({ timeout: 1 }); await database.cleanup(); }

server/src/__tests__/helpers/tool-profile-matching-fixture.ts

@@ -0,0 +1,15 @@
+import { randomUUID } from "node:crypto";
+import { agents, companies, toolApplications, toolCatalogEntries, toolConnections, toolProfileBindings, toolProfileEntries, toolProfiles, type Db } from "@paperclipai/db";
+
+export async function createToolProfileMatchingFixture(db: Db) {
+  const [company] = await db.insert(companies).values({ name: "Synthetic profile matching", issuePrefix: `BM${randomUUID().slice(0, 6)}` }).returning();
+  const companyAgents = await db.insert(agents).values(Array.from({ length: 12 }, (_, i) => ({ companyId: company.id, name: `Agent ${i}` }))).returning();
+  const agent = companyAgents[0]!;
+  const [app] = await db.insert(toolApplications).values({ companyId: company.id, name: "Synthetic app", type: "mcp_http" }).returning();
+  const [connection] = await db.insert(toolConnections).values({ companyId: company.id, applicationId: app.id, name: "Synthetic connection", uid: randomUUID(), transport: "mcp_remote" }).returning();
+  const catalog = await db.insert(toolCatalogEntries).values(Array.from({ length: 1890 }, (_, i) => ({ companyId: company.id, applicationId: app.id, connectionId: connection.id, name: `tool_${i}`, toolName: `tool_${i}`, versionHash: `v${i}`, riskLevel: "read" as const }))).returning();
+  const [profile] = await db.insert(toolProfiles).values({ companyId: company.id, profileKey: "selected", name: "Selected tools", defaultAction: "deny" }).returning();
+  await db.insert(toolProfileBindings).values({ companyId: company.id, profileId: profile.id, targetType: "agent", targetId: agent.id });
+  await db.insert(toolProfileEntries).values(catalog.slice(0, 1575).map((entry, i) => ({ companyId: company.id, profileId: profile.id, selectorType: "catalog_entry" as const, catalogEntryId: entry.id, effect: i % 5 === 0 ? "exclude" as const : "include" as const })));
+  return { company, agent };
+}

server/src/__tests__/tool-effective-assignments.test.ts

@@ -76,6 +76,20 @@ describeDatabase("effective tool assignments", () => {
     expect(full.allowedTools[0].annotations).toEqual(f.read.annotations);
   });
 
+  it("keeps exclusions local to each profile when effective profiles grant the same tool", async () => {
+    const f = await fixture();
+    await db.update(toolProfiles).set({ defaultAction: "allow" }).where(eq(toolProfiles.id, f.profile.id));
+    await db.insert(toolProfileEntries).values({ companyId: f.company.id, profileId: f.profile.id,
+      selectorType: "catalog_entry", effect: "exclude", catalogEntryId: f.read.id });
+    const [second] = await db.insert(toolProfiles).values({ companyId: f.company.id, profileKey: "second", name: "Second", defaultAction: "deny" }).returning();
+    await db.insert(toolProfileBindings).values({ companyId: f.company.id, profileId: second.id, targetType: "agent", targetId: f.agent.id });
+    const [include] = await db.insert(toolProfileEntries).values({ companyId: f.company.id, profileId: second.id,
+      selectorType: "catalog_entry", effect: "include", catalogEntryId: f.read.id }).returning();
+    expect((await compare(f)).assignments.allowedTools.map(tool => tool.id)).toEqual([f.read.id, f.write.id]);
+    await db.delete(toolProfileEntries).where(eq(toolProfileEntries.id, include.id));
+    expect((await compare(f)).assignments.allowedTools.map(tool => tool.id)).toEqual([f.write.id]);
+  });
+
   it("keeps unresolved exact names and observes changed catalog, profile and binding state", async () => {
     const f = await fixture();
     await db.insert(toolProfileEntries).values([

server/src/__tests__/tool-profile-matching.test.ts

@@ -0,0 +1,21 @@
+import { describe, expect, it } from "vitest";
+import { profileCatalogMatcher, profileEntryMatchesCatalog, type ProfileCatalogMatch, type ProfileEntryMatch } from "../services/tool-access/profiles/projection.js";
+
+describe("indexed profile catalog matching", () => {
+  it("preserves all selector kinds, null matches and exact strings in mixed rule sets", () => {
+    const catalog: ProfileCatalogMatch[] = Array.from({ length: 75 }, (_, index) => ({
+      id: `tool-${index}`, applicationId: index % 3 ? `app-${index % 5}` : null,
+      connectionId: `connection-${index % 7}`, toolName: index % 4 ? `tool-${index}` : "quoted' path\\tool_🙂",
+      riskLevel: index % 2 ? "read" : "write",
+    }));
+    const selectors = ["application", "connection", "catalog_entry", "tool_name", "risk_level", "unknown"];
+    const rules: ProfileEntryMatch[] = catalog.flatMap(entry => selectors.map(selectorType => ({
+      selectorType, applicationId: entry.applicationId, connectionId: entry.connectionId,
+      catalogEntryId: entry.id, toolName: entry.toolName, riskLevel: entry.riskLevel,
+    })));
+    for (const selected of [[], ...rules.map(rule => [rule]), rules.slice(0, 25), rules]) {
+      const matches = profileCatalogMatcher(selected);
+      expect(catalog.filter(matches)).toEqual(catalog.filter(entry => selected.some(rule => profileEntryMatchesCatalog(rule, entry))));
+    }
+  });
+});

server/src/services/tool-access/profiles/projection.ts

@@ -128,22 +128,18 @@ export function summarizeProfile(input: {
   catalog: ProfileCatalogEntry[];
   agentIds: string[];
 }): ToolProfileSummary {
-  const includes = input.entries.filter((entry) => entry.effect === "include");
-  const excludes = input.entries.filter((entry) => entry.effect === "exclude");
+  const included = profileCatalogMatcher(input.entries.filter((entry) => entry.effect === "include"));
+  const excluded = profileCatalogMatcher(input.entries.filter((entry) => entry.effect === "exclude"));
   const allowedCatalogIds = new Set<string>();
   const allowedApplicationIds = new Set<string>();
   const excludedCatalogIds = new Set<string>();
 
   for (const catalogEntry of input.catalog) {
-    const excluded = excludes.some((entry) =>
-      profileEntryMatchesCatalog(entry, catalogEntry),
-    );
-    if (excluded) excludedCatalogIds.add(catalogEntry.id);
-    if (excluded) continue;
-    const included = includes.some((entry) =>
-      profileEntryMatchesCatalog(entry, catalogEntry),
-    );
-    if (input.profile.defaultAction === "allow" || included) {
+    if (excluded(catalogEntry)) {
+      excludedCatalogIds.add(catalogEntry.id);
+      continue;
+    }
+    if (input.profile.defaultAction === "allow" || included(catalogEntry)) {
       allowedCatalogIds.add(catalogEntry.id);
       if (catalogEntry.applicationId)
         allowedApplicationIds.add(catalogEntry.applicationId);

server/src/services/tool-access/profiles/resolution.ts

@@ -3,7 +3,7 @@ import type { Db } from "@paperclipai/db";
 import { agents, toolCatalogEntries, toolProfileBindings, toolProfileEntries, toolProfiles } from "@paperclipai/db";
 import type { ToolCatalogEntry, ToolConnection, ToolProfileEffectiveSummary, ToolProfileWithDetails } from "@paperclipai/shared";
 import { effectiveToolProfileBindings, profileIdsInBindingOrder } from "../../tool-profile-binding-precedence.js";
-import { buildProfileDetails, profileCatalogColumns, profileEntryMatchesCatalog, toProfileBinding, toProfileEntry, type ProfileCatalogEntry, type ProfileEntryMatch, type ProfileCatalogMatch } from "./projection.js";
+import { buildProfileDetails, profileCatalogColumns, profileCatalogMatcher, toProfileBinding, toProfileEntry, type ProfileCatalogEntry, type ProfileEntryMatch, type ProfileCatalogMatch } from "./projection.js";
 type ProfileAssignmentDependencies = {
   assertOptionalAgent(companyId: string, agentId: string | null | undefined, label: string): Promise<void>;
   resolveInstalledConnectionsForAgent(companyId: string, agentId: string): Promise<ToolConnection[]>;
@@ -37,18 +37,13 @@ function resolveCatalogAccess(
     const excludes = profileEntries.filter(
       (entry) => entry.effect === "exclude",
     );
+    const included = profileCatalogMatcher(includes);
+    const excluded = profileCatalogMatcher(excludes);
     for (const catalogEntry of catalog) {
-      if (
-        excludes.some((entry) =>
-          profileEntryMatchesCatalog(entry, catalogEntry),
-        )
-      )
-        continue;
+      if (excluded(catalogEntry)) continue;
       if (
         profile.defaultAction === "allow" ||
-        includes.some((entry) =>
-          profileEntryMatchesCatalog(entry, catalogEntry),
-        )
+        included(catalogEntry)
       ) {
         allowedCatalogIds.add(catalogEntry.id);
         allowedToolNames.add(catalogEntry.toolName);