mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-11 05:11:49 +01:00
fix(api-search): stop function words from pushing case lookup off page one (#20382)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Chat and task agents find API operations with `search_api`, which
ranks the runner API catalog by word matches in each operation's method,
path and summary
> - The ranker scored function words such as "for", "the" and "of" like
resource words, so any summary that says "for" got 5 points for that
word alone
> - #20355 added two chat-bootstrap summaries that say "for up to ten
tasks" and "for one task". After that, the query "case for task" put
`GET /api/issues/{issueId}/cases` at rank 6, off the first page of 5
> - #20354 promised that agents find a task's case on the first search
page, and its test `case-history-api-discovery.test.ts` now fails on
master
> - This pull request stops the ranker from scoring function words, so
the resource words in the query decide the rank
> - The benefit is that master is green again and agents find the case
lookup (and other "X for an issue" reads) on the first page, whatever
new summaries say "for"
## Linked Issues or Issue Description
Refs #20354, #20355. Supersedes #19388 (the same ranker change, opened
earlier and not merged).
**What happened**
On a clean checkout of master at 0e3fdebea8 and later,
`server/src/__tests__/case-history-api-discovery.test.ts` fails:
"expected [ …(5) ] to include 'GET /api/issues/{issueId}/cases'".
`search_api` for "case for task" returns case handoffs, both
chat-bootstrap reads, task-drain, pending questions and privacy
constraints first. Most of these rank high only because their summaries
contain "for".
**Expected behavior**
An agent that searches "case for task" finds `GET
/api/issues/{issueId}/cases` on the first page. Words such as "for",
"an" and "the" do not change the rank.
**Steps to reproduce**
1. Check out master.
2. Run `cd server && npx vitest run
src/__tests__/case-history-api-discovery.test.ts`.
3. See the assertion failure for the "case for task" query.
## What Changed
- `server/src/services/native-runtime/runner-api-catalog.ts`: the search
tokenizer drops a small set of function words (`a`, `an`, `the`, `for`,
`of`, `to`, `in`, `on`, `at`, `by`, `with`, `and`, `or`, `from`) before
scoring. This applies to the query and to the catalog text.
- `server/src/services/native-runtime/runner-api.test.ts`: a regression
test. "list comments for an issue" returns the comments route first.
"queued comments", "activity", "runs" and "case for task" return their
routes on the first page. A query of only function words returns no
results.
## Verification
- Red on master before the change: `cd server && npx vitest run
src/__tests__/case-history-api-discovery.test.ts` fails (1 failed, 2
passed). Before the fix, the top 10 for "case for task" put the case
lookup at rank 6.
- Green after the change: `npx vitest run
src/__tests__/case-history-api-discovery.test.ts
src/services/native-runtime/runner-api.test.ts
src/__tests__/openapi-routes.test.ts
src/__tests__/agent-conversations.test.ts
src/__tests__/heartbeat-list.test.ts` passes.
## Risks
- Low risk. The change affects only `search_api` ranking. Exact
operationId lookup is unchanged. A query that has only function words
now returns no results instead of a list of arbitrary matches.
- Rank order can change for queries that used function words. The
existing ranking tests still pass.
## Model Used
- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), agentic coding with
tool use (shell, file edits) in Claude Code.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
3cc1032f6a
commit
46ca7d4377
2 files changed
+17
-1
No files matched your search
@@ -34,8 +34,12 @@ const SYNONYMS: Record<string, string> = {
|
||||
blocker: "dependencies", blockers: "dependencies", files: "attachments",
|
||||
};
|
||||
|
||||
// Function words carry no resource meaning; scoring them let any summary that
|
||||
// happens to say "for" or "the" outrank the operation a query names.
|
||||
const SEARCH_FUNCTION_WORDS = new Set(["a", "an", "the", "for", "of", "to", "in", "on", "at", "by", "with", "and", "or", "from"]);
|
||||
|
||||
function words(text: string): string[] {
|
||||
return text.toLowerCase().split(/[^a-z0-9]+/).filter(Boolean)
|
||||
return text.toLowerCase().split(/[^a-z0-9]+/).filter((word) => word.length > 0 && !SEARCH_FUNCTION_WORDS.has(word))
|
||||
.flatMap((word) => [word, ...(SYNONYMS[word] ? [SYNONYMS[word]] : [])])
|
||||
.map((word) => word.length > 3 && word.endsWith("s") ? word.slice(0, -1) : word);
|
||||
}
|
||||
|
||||
@@ -202,6 +202,18 @@ describe("runner API catalog", () => {
|
||||
expect(operation.authorization.actor).toBeTruthy();
|
||||
});
|
||||
|
||||
it("keeps natural-language function words from hiding the requested API resource", () => {
|
||||
expect(searchRunnerApi({ query: "list comments for an issue" }).results[0]?.operationId)
|
||||
.toBe("GET /api/issues/{id}/comments");
|
||||
for (const [query, operation] of [
|
||||
["queued comments for an issue", "GET /api/issues/{id}/queued-comments"],
|
||||
["activity for an issue", "GET /api/issues/{id}/activity"],
|
||||
["runs for an issue", "GET /api/issues/{id}/runs"],
|
||||
["case for task", "GET /api/issues/{issueId}/cases"],
|
||||
]) expect(searchRunnerApi({ query }).results.map(row => row.operationId)).toContain(operation);
|
||||
expect(searchRunnerApi({ query: "for an the" }).results).toEqual([]);
|
||||
});
|
||||
|
||||
it("ranks natural language, explains dedicated alternatives, and supports exact lookup", () => {
|
||||
expect(
|
||||
searchRunnerApi({ query: "create project" }).results.map(
|
||||
|
||||
Reference in new issue
Block a user