Files
PaperClipAI/packages/shared/src/validators/search.ts
T
DottaandPaperclip ea0e899905 fix(search): honor extract match limits + harden pr-gardening candidate discovery (#9652)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The `/pr-gardening` skill drives a bundled agent that scans a
company's issues for those linked to open GitHub PRs, then reports on
their state; it relies on the server's company-search **extract**
endpoint to pull PR references out of issue bodies
> - Two gaps surfaced during end-to-end QA of the gardening workflow:
the extract service silently ignored a per-issue match cap, so callers
could not bound how many matches came back per issue, and the skill's
candidate-discovery scripts fell over on large repos and on issues that
referenced deleted PRs
> - Left unaddressed, the gardener either truncated its scan
unpredictably or aborted outright, so it could not reliably enumerate PR
candidates
> - This pull request honors an explicit `matchesPerIssue` limit in the
extract search API and hardens the skill's candidate discovery against
missing/unavailable PRs and oversized `gh` output
> - The benefit is a PR-gardening workflow that scans deterministically
and finishes cleanly on real-world companies

## Linked Issues or Issue Description

No pre-existing public GitHub issue — describing the bug in-PR following
the bug report template (`.github/ISSUE_TEMPLATE/bug_report.yml`).

### What happened?

The company-search extract endpoint accepted a per-issue match limit but
did not apply it, returning matches capped only by the old hardcoded
constant regardless of the caller's request. Separately, the
`/pr-gardening` skill's candidate-discovery scripts crashed when a
scanned issue referenced a deleted PR (GitHub `Not Found (HTTP 404)` /
GraphQL `Could not resolve to a PullRequest`) and could exceed the
default `gh` output buffer on large result sets, aborting the whole
scan.

### Expected behavior

The extract API bounds matches per issue when a caller passes
`matchesPerIssue` (default 20, max 200), and omitting it preserves the
previous default. The gardening scripts skip PRs that are
deleted/unavailable and tolerate large `gh` responses without aborting
the scan.

### Steps to reproduce

1. Call the company-search extract endpoint with a `matchesPerIssue`
value against an issue containing many PR references — previously the
value was ignored.
2. Run the pr-gardening candidate scan against a company whose issues
reference a since-deleted PR — previously the scan threw instead of
skipping that PR.

### Paperclip version or commit

`master` at the base of this PR (branch cut from current
`origin/master`).

### Deployment mode

Local Paperclip instance / self-hosted.

## What Changed

- **Extract search honors `matchesPerIssue`**: added the
`matchesPerIssue` field to the shared search validator/types and applied
the cap in `company-search-extract` so results are bounded per issue
(`packages/shared`, `server/src/services/company-search-extract.ts`,
`doc/SPEC-implementation.md`).
- **Hardened pr-gardening candidate discovery**: `find-candidates.mjs` /
`lib.mjs` now request `matchesPerIssue=200`, treat missing/unavailable
PRs (deleted PR → `isMissingPullRequestError` / `unavailable`) as skips
instead of fatal errors, and raise the `gh` `maxBuffer` to 50 MB for
large repos.
- **Tests**: expanded `company-search-extract-{routes,service}.test.ts`
for the new limit and added coverage in `pr-gardening.test.mjs`.

## Verification

Re-run on a fresh worktree cherry-picked onto current `master`:

- `node --test
.agents/skills/pr-gardening/scripts/pr-gardening.test.mjs` → 8/8 pass
- `pnpm vitest run
server/src/__tests__/company-search-extract-routes.test.ts
server/src/__tests__/company-search-extract-service.test.ts` → 10/10
pass

## Risks

Low risk. `matchesPerIssue` is optional and backward-compatible
(omitting it preserves prior behavior). The skill changes only add
skip/tolerance paths and a larger buffer; no schema or migration
changes.

## Model Used

Claude — Opus 4.8 (`claude-opus-4-8`), extended thinking, tool use /
code execution via the Claude Agent SDK.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-15 21:44:21 -05:00

271 lines
10 KiB
TypeScript

import { z } from "zod";
import { ISSUE_PRIORITIES, ISSUE_STATUSES } from "../constants.js";
import { isUuidLike } from "../agent-url-key.js";
import {
COMPANY_SEARCH_EXTRACT_KINDS,
COMPANY_SEARCH_EXTRACT_SCOPES,
COMPANY_SEARCH_SCOPES,
COMPANY_SEARCH_SORTS,
} from "../types/search.js";
export const COMPANY_SEARCH_MAX_QUERY_LENGTH = 200;
export const COMPANY_SEARCH_MAX_TOKENS = 8;
export const COMPANY_SEARCH_DEFAULT_LIMIT = 20;
export const COMPANY_SEARCH_MAX_LIMIT = 50;
export const COMPANY_SEARCH_MAX_OFFSET = 200;
export const COMPANY_SEARCH_EXTRACT_DEFAULT_LIMIT = 100;
export const COMPANY_SEARCH_EXTRACT_MAX_LIMIT = 200;
export const COMPANY_SEARCH_EXTRACT_MAX_OFFSET = 5_000;
export const COMPANY_SEARCH_EXTRACT_DEFAULT_MATCHES_PER_ISSUE = 20;
export const COMPANY_SEARCH_EXTRACT_MAX_MATCHES_PER_ISSUE = 200;
const UPDATED_WITHIN_RE = /^[1-9]\d{0,2}(h|d|w|m)$/;
function firstQueryValue(value: unknown): unknown {
return Array.isArray(value) ? value[0] : value;
}
function queryValues(value: unknown): unknown[] {
if (value === undefined || value === null) return [];
return Array.isArray(value) ? value : [value];
}
function parseOptionalString(value: unknown, ctx: z.RefinementCtx, field: string): string | undefined {
const raw = firstQueryValue(value);
if (raw === undefined || raw === null) return undefined;
if (typeof raw !== "string" && typeof raw !== "number") {
ctx.addIssue({ code: z.ZodIssueCode.custom, message: `${field} must be a string` });
return undefined;
}
const normalized = String(raw).trim();
return normalized.length > 0 ? normalized : undefined;
}
function parseIntegerQuery(
value: unknown,
ctx: z.RefinementCtx,
field: string,
fallback: number,
min: number,
max: number,
): number {
const raw = firstQueryValue(value);
if (raw === undefined || raw === null || raw === "") return fallback;
const text = typeof raw === "number" ? String(raw) : typeof raw === "string" ? raw.trim() : "";
if (!/^-?\d+$/.test(text)) {
ctx.addIssue({ code: z.ZodIssueCode.custom, message: `${field} must be an integer` });
return fallback;
}
const numeric = Number.parseInt(text, 10);
if (!Number.isInteger(numeric) || numeric < min || numeric > max) {
const range = min === 0 ? `between 0 and ${max}` : `between ${min} and ${max}`;
ctx.addIssue({ code: z.ZodIssueCode.custom, message: `${field} must be ${range}` });
return fallback;
}
return numeric;
}
function parseEnumList<T extends string>(
value: unknown,
ctx: z.RefinementCtx,
field: string,
allowed: readonly T[],
): T[] {
const allowedSet = new Set<string>(allowed);
const values: T[] = [];
for (const rawEntry of queryValues(value)) {
if (typeof rawEntry !== "string") {
ctx.addIssue({ code: z.ZodIssueCode.custom, message: `${field} must be a comma-separated string` });
continue;
}
for (const rawItem of rawEntry.split(",")) {
const item = rawItem.trim();
if (!item) continue;
if (!allowedSet.has(item)) {
ctx.addIssue({ code: z.ZodIssueCode.custom, message: `${field} contains an unsupported value` });
continue;
}
if (!values.includes(item as T)) values.push(item as T);
}
}
return values;
}
function parseOptionalUuid(value: unknown, ctx: z.RefinementCtx, field: string): string | undefined {
const normalized = parseOptionalString(value, ctx, field);
if (normalized === undefined) return undefined;
if (!isUuidLike(normalized)) {
ctx.addIssue({ code: z.ZodIssueCode.custom, message: `${field} must be a UUID` });
return undefined;
}
return normalized;
}
function parseAssigneeAgentId(value: unknown, ctx: z.RefinementCtx): string | null | undefined {
const normalized = parseOptionalString(value, ctx, "assigneeAgentId");
if (normalized === undefined) return undefined;
if (normalized.toLowerCase() === "null") return null;
if (!isUuidLike(normalized)) {
ctx.addIssue({ code: z.ZodIssueCode.custom, message: "assigneeAgentId must be a UUID or 'null'" });
return undefined;
}
return normalized;
}
function parseUpdatedAfter(value: unknown, ctx: z.RefinementCtx): string | undefined {
const normalized = parseOptionalString(value, ctx, "updatedAfter");
if (normalized === undefined) return undefined;
const date = new Date(normalized);
if (Number.isNaN(date.getTime())) {
ctx.addIssue({ code: z.ZodIssueCode.custom, message: "updatedAfter must be a valid date" });
return undefined;
}
return date.toISOString();
}
function parseUpdatedWithin(value: unknown, ctx: z.RefinementCtx): string | undefined {
const normalized = parseOptionalString(value, ctx, "updatedWithin");
if (normalized === undefined) return undefined;
if (!UPDATED_WITHIN_RE.test(normalized)) {
ctx.addIssue({ code: z.ZodIssueCode.custom, message: "updatedWithin must be a duration like 24h, 7d, 4w, or 3m" });
return undefined;
}
return normalized;
}
export const companySearchQuerySchema = z.object({
q: z.unknown()
.optional()
.transform((value, ctx) => (parseOptionalString(value, ctx, "q") ?? "").slice(0, COMPANY_SEARCH_MAX_QUERY_LENGTH)),
scope: z.unknown()
.optional()
.transform((value, ctx) => {
const normalized = parseOptionalString(value, ctx, "scope") ?? "all";
if (!(COMPANY_SEARCH_SCOPES as readonly string[]).includes(normalized)) {
ctx.addIssue({ code: z.ZodIssueCode.custom, message: "scope must be a supported search scope" });
return "all";
}
return normalized as (typeof COMPANY_SEARCH_SCOPES)[number];
}),
limit: z.unknown()
.optional()
.transform((value, ctx) => parseIntegerQuery(value, ctx, "limit", COMPANY_SEARCH_DEFAULT_LIMIT, 1, COMPANY_SEARCH_MAX_LIMIT)),
offset: z.unknown()
.optional()
.transform((value, ctx) => parseIntegerQuery(value, ctx, "offset", 0, 0, COMPANY_SEARCH_MAX_OFFSET)),
status: z.unknown()
.optional()
.transform((value, ctx) => parseEnumList(value, ctx, "status", ISSUE_STATUSES)),
priority: z.unknown()
.optional()
.transform((value, ctx) => parseEnumList(value, ctx, "priority", ISSUE_PRIORITIES)),
assigneeAgentId: z.unknown()
.optional()
.transform((value, ctx) => parseAssigneeAgentId(value, ctx)),
assigneeUserId: z.unknown()
.optional()
.transform((value, ctx) => parseOptionalString(value, ctx, "assigneeUserId")),
projectId: z.unknown()
.optional()
.transform((value, ctx) => parseOptionalUuid(value, ctx, "projectId")),
labelId: z.unknown()
.optional()
.transform((value, ctx) => parseOptionalUuid(value, ctx, "labelId")),
updatedWithin: z.unknown()
.optional()
.transform((value, ctx) => parseUpdatedWithin(value, ctx)),
updatedAfter: z.unknown()
.optional()
.transform((value, ctx) => parseUpdatedAfter(value, ctx)),
sort: z.unknown()
.optional()
.transform((value, ctx) => {
const normalized = parseOptionalString(value, ctx, "sort") ?? "relevance";
if (!(COMPANY_SEARCH_SORTS as readonly string[]).includes(normalized)) {
ctx.addIssue({ code: z.ZodIssueCode.custom, message: "sort must be relevance, updated, created, or priority" });
return "relevance";
}
return normalized as (typeof COMPANY_SEARCH_SORTS)[number];
}),
});
export type CompanySearchQuery = z.infer<typeof companySearchQuerySchema>;
export const companySearchExtractQuerySchema = z.object({
contains: z.unknown().transform((value, ctx) => {
const normalized = parseOptionalString(value, ctx, "contains");
if (!normalized || normalized.length < 2) {
ctx.addIssue({ code: z.ZodIssueCode.custom, message: "contains must be at least 2 characters" });
return "";
}
if (normalized.length > COMPANY_SEARCH_MAX_QUERY_LENGTH) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
message: `contains must be at most ${COMPANY_SEARCH_MAX_QUERY_LENGTH} characters`,
});
}
return normalized.slice(0, COMPANY_SEARCH_MAX_QUERY_LENGTH);
}),
kind: z.unknown()
.optional()
.transform((value, ctx) => {
const normalized = parseOptionalString(value, ctx, "kind") ?? "literal";
if (!(COMPANY_SEARCH_EXTRACT_KINDS as readonly string[]).includes(normalized)) {
ctx.addIssue({ code: z.ZodIssueCode.custom, message: "kind must be literal or url" });
return "literal";
}
return normalized as (typeof COMPANY_SEARCH_EXTRACT_KINDS)[number];
}),
scope: z.unknown()
.optional()
.transform((value, ctx) => {
const normalized = parseOptionalString(value, ctx, "scope") ?? "all";
if (!(COMPANY_SEARCH_EXTRACT_SCOPES as readonly string[]).includes(normalized)) {
ctx.addIssue({ code: z.ZodIssueCode.custom, message: "scope must be all, issues, comments, or documents" });
return "all";
}
return normalized as (typeof COMPANY_SEARCH_EXTRACT_SCOPES)[number];
}),
limit: z.unknown()
.optional()
.transform((value, ctx) => parseIntegerQuery(
value,
ctx,
"limit",
COMPANY_SEARCH_EXTRACT_DEFAULT_LIMIT,
1,
COMPANY_SEARCH_EXTRACT_MAX_LIMIT,
)),
offset: z.unknown()
.optional()
.transform((value, ctx) => parseIntegerQuery(value, ctx, "offset", 0, 0, COMPANY_SEARCH_EXTRACT_MAX_OFFSET)),
matchesPerIssue: z.unknown()
.optional()
.transform((value, ctx) => parseIntegerQuery(
value,
ctx,
"matchesPerIssue",
COMPANY_SEARCH_EXTRACT_DEFAULT_MATCHES_PER_ISSUE,
1,
COMPANY_SEARCH_EXTRACT_MAX_MATCHES_PER_ISSUE,
)),
status: z.unknown()
.optional()
.transform((value, ctx) => parseEnumList(value, ctx, "status", ISSUE_STATUSES)),
updatedWithin: z.unknown()
.optional()
.transform((value, ctx) => parseUpdatedWithin(value, ctx)),
updatedAfter: z.unknown()
.optional()
.transform((value, ctx) => parseUpdatedAfter(value, ctx)),
}).superRefine((value, ctx) => {
if (value.updatedWithin && value.updatedAfter) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
message: "updatedWithin and updatedAfter cannot be used together",
});
}
});
export type CompanySearchExtractQuery = z.infer<typeof companySearchExtractQuerySchema>;