fix(native-runtime): name the closest assigned tools when a tool name is mistyped (#22327)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native runners give agents their assigned app tools through two
wrapper tools: `paperclip_search_assigned_tools` and
`paperclip_call_assigned_tool`.
> - Search returns each tool with a hashed alias
(`app_<readable>_<digest>`). #22213 also accepts an unambiguous gateway
name or display name.
> - Some models copy the alias with an error: they drop or change one
digest character, change the characters of the bot id, or attach the
digest of a different tool.
> - The wrapper correctly rejects these names, because a guess could
select a write tool when the agent meant a read tool.
> - But the error only said "name must be an exact name". The agent
searched again and often repeated the same mistake.
> - This pull request adds the closest names that the agent can use to
the error. It does not resolve any name by approximation.
> - The benefit is a faster recovery after a mistyped name, with no
change to authorization.

## Linked Issues or Issue Description

Refs #22213

**What happened?**
An agent searched its assigned tools and then called
`paperclip_call_assigned_tool` with a name that is almost the returned
alias. For example, one digest character was missing, or the agent used
the digest of the read tool with the name of the submit tool. The call
failed with `assigned_mcp_tool_unknown: name must be an exact name
returned by paperclip_search_assigned_tools. Search assigned tools again
and use the returned name.` In observed runs, the agent made more failed
calls with other wrong names.

**Expected behavior**
The call is still rejected. The error also shows the closest assigned
names that the agent can use, so that the agent can correct the name
directly.

**Paperclip version or commit**
`master` at c0aa7751f8.

**Steps to reproduce**
1. Assign the tools `github_bot:read_pull_request` (read) and
`github_bot:begin_review` (write) to an agent on an on-demand native
runner.
2. Call `paperclip_search_assigned_tools` with `{"query":"github_bot"}`
and copy the alias of `begin_review`.
3. Remove one character from the digest and call
`paperclip_call_assigned_tool` with that name.
4. See `assigned_mcp_tool_unknown` with no information about the correct
name.

## What Changed

- `server/src/services/native-runtime/assigned-mcp-tools.ts`:
- Add `closestNames`. It returns up to three candidate names whose edit
distance (case-insensitive) is not more than one third of the longer
name. It skips the distance computation when the length difference is
already more than this limit. Ties sort by name.
- The unknown-name error of `paperclip_call_assigned_tool` now includes
`Closest assigned names: <a>, <b>, <c>.` when a near name exists. If no
name is near, the error text is the same as before.
  - The unknown `schemaTool` error of search includes the same list.
- If the fresh grant list fails on the call path (for example, a revoked
token), the call still returns `assigned_mcp_tool_unknown`, with no
hint. The hint never replaces the error.
- Both lists use one new helper, `authorizedTools`. It is the
fresh-grant and work-mode filter that search already used. A revoked
tool and a tool that the current work mode does not permit are never
shown.
- `editDistance` is a local helper in the same module. It has the same
algorithm as the private helper in
`packages/shared/src/config-schema.ts`. That shared file is pinned as a
reviewed SDK reference source, so an export from it would require a new
pin review. A second commit tried this, and the runner support suite
failed on the stale pin, so the third commit reverted it.
- `server/src/services/native-runtime/assigned-mcp-tools.test.ts`: add a
regression test.

## Verification

- Failing before the first fix: `pnpm --filter @paperclipai/server exec
vitest run src/services/native-runtime/assigned-mcp-tools.test.ts -t
"closest permitted"` failed with `AssertionError: expected
'assigned_mcp_tool_unknown: name must …' to contain 'Closest assigned
names: app_github_bo…'`.
- Failing before the review fixes (second commit). I added each
assertion before its fix:
- Distance limit: the planning-mode error was expected to have no hint.
It failed with `Closest assigned names:
app_github_bot_read_pull_request_…, app_calendar_event_0_…,
app_calendar_event_1_…`.
- Length bound: a 4 MiB name on the call path and on the `schemaTool`
path took 20045 ms. The test now uses an 8 MiB name and the 15 s test
timeout, not a wall-clock assertion. Without the length bound it fails
with `Test timed out in 15000ms`.
- Fresh grants: after the gateway revoked `begin_review`, the call error
still listed `app_github_bot_begin_review_…`.
- Failing before the fourth commit: with a gateway listing that rejects
with `gateway_token_revoked`, the call error was `gateway_token_revoked`
instead of `assigned_mcp_tool_unknown: name must be an exact name ...`.
- Passing after the fixes: the full file
`src/services/native-runtime/assigned-mcp-tools.test.ts` passes (60 of
60 tests). The oversized-name checks finish in a few milliseconds.
- CI on the second commit: only `Runner support suite` failed for a code
reason (the stale SDK pin above). `Build`, `Canary Dry Run`, and three
server test shards stopped because the CI runner received a shutdown
signal. No test failed.
- The new test checks these conditions:
- A digest with one missing character lists the correct alias first.
Unrelated tools are not in the list.
- A write-tool name with the digest of the read tool lists the write
tool first.
- In planning mode, a typo of a write tool gives the old error with no
hint. Write tools and distant read tools are not shown.
  - An unknown `schemaTool` lists the correct alias first.
  - A very long name gets no hint and finishes quickly on both paths.
  - A revoked tool is not shown.
- A failed grant listing gives the plain unknown-name error, not the
gateway error.
  - The gateway receives no call for any of these names.
- No other code or test depends on the changed error text (`git grep`).
- Not run locally: the full server `tsc --noEmit`. The sandbox stopped
it for memory. CI runs the typecheck. It passed on the first commit.

## Risks

- Low risk. The change affects only the text of errors that are already
returned. No name is resolved by approximation, and every call still
needs an exact name.
- The list shows only names that search would return to the caller now:
fresh grants and the current work mode apply.
- A name that is not near any tool gets the old error, so the hint does
not send the agent to an unrelated tool.
- The cost is one fresh grant list and one bounded edit-distance pass
over the session catalog, only when a name is unknown. A long name skips
the distance computation.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution
in an agent harness.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed 2026-11-19 14:29:30 -06:00
1 parent 806440d12c
commit cf89f0436c
2 files changed
+83 -12

No files matched your search

@@ -131,6 +131,44 @@ describe("assigned MCP runner tools", () => {
expect(f.executeTool).not.toHaveBeenCalled();
});
it("names the closest permitted assigned tools when a name is mistyped, without calling any of them", async () => {
const read = descriptor("github_bot:read_pull_request");
const begin = descriptor("github_bot:begin_review", "write");
const submit = descriptor("github_bot:submit_review", "write");
const f = fixture([read, begin, submit, ...Array.from({ length: 6 }, (_, i) => descriptor(`calendar:event_${i}`))]);
const assigned = await createAssignedMcpTools({ ...f, delivery: "on_demand" });
const page = await assigned.execute({ tool: searchName, arguments: { query: "github_bot", limit: 20 } }) as { tools: Array<{ name: string }> };
const alias = (tool: ToolGatewayDescriptor) => page.tools.find(entry => entry.name.includes(tool.name.split(":")[1]!))!.name;
const unknown = (name: string, mode?: "planning") =>
assigned.execute({ tool: callName, arguments: { name, arguments: {} } }, mode).then(() => "", (error: Error) => error.message);
const dropped = await unknown(`${alias(begin).slice(0, -6)}${alias(begin).slice(-5)}`);
expect(dropped).toMatch(/^assigned_mcp_tool_unknown: name must be an exact name returned by paperclip_search_assigned_tools\./);
expect(dropped).toContain(`Closest assigned names: ${alias(begin)}`);
expect(dropped).not.toContain("calendar");
const borrowed = await unknown(`${alias(submit).slice(0, -12)}${alias(read).slice(-12)}`);
expect(borrowed).toContain(`Closest assigned names: ${alias(submit)}`);
const planning = await unknown(`${alias(begin).slice(0, -1)}0`, "planning");
expect(planning).toBe("assigned_mcp_tool_unknown: name must be an exact name returned by paperclip_search_assigned_tools. Search assigned tools again and use the returned name.");
await expect(assigned.execute({ tool: searchName, arguments: { query: "", schemaTool: `${alias(read)}x` } }))
.rejects.toThrow(`Closest assigned names: ${alias(read)}`);
const oversized = `${alias(read)}${"x".repeat(8 * 1024 * 1024)}`;
expect(await unknown(oversized)).not.toContain("Closest assigned names");
await expect(assigned.execute({ tool: searchName, arguments: { query: "", schemaTool: oversized } }))
.rejects.toThrow("schemaTool must match an authorized inputSchemaRef. Omit schemaTool");
f.listToolsForNamedGateway.mockResolvedValue([read, submit]);
expect(await unknown(`${alias(begin).slice(0, -1)}0`)).not.toContain(alias(begin));
f.listToolsForNamedGateway.mockRejectedValue(new Error("gateway_token_revoked"));
expect(await unknown(`${alias(read).slice(0, -1)}0`)).toBe("assigned_mcp_tool_unknown: name must be an exact name returned by paperclip_search_assigned_tools. Search assigned tools again and use the returned name.");
expect(f.executeTool).not.toHaveBeenCalled();
});
it.each([undefined, "on_demand"] as const)("advertises usable page limits through MCP discovery (%s)", async delivery => {
const f = fixture(Array.from({ length: 25 }, (_, i) => descriptor(`operation_${i}`)));
const assigned = await createAssignedMcpTools({ ...f, delivery });
@@ -85,6 +85,32 @@ export function assignedToolName(name: string): string {
return `app_${readable}_${digest}`;
}
function editDistance(left: string, right: string): number {
let previous = Array.from({ length: right.length + 1 }, (_, index) => index);
for (let leftIndex = 1; leftIndex <= left.length; leftIndex += 1) {
const current = [leftIndex];
for (let rightIndex = 1; rightIndex <= right.length; rightIndex += 1) {
const substitutionCost = left[leftIndex - 1] === right[rightIndex - 1] ? 0 : 1;
current[rightIndex] = Math.min(current[rightIndex - 1]! + 1, previous[rightIndex]! + 1, previous[rightIndex - 1]! + substitutionCost);
}
previous = current;
}
return previous[right.length]!;
}
function closestNames(reference: string, names: string[]): string {
const target = reference.trim().toLowerCase();
const closest = names
.map(name => ({ name, limit: Math.floor(Math.max(target.length, name.length) / 3) }))
.filter(({ name, limit }) => Math.abs(target.length - name.length) <= limit)
.map(({ name, limit }) => ({ name, limit, distance: editDistance(target, name.toLowerCase()) }))
.filter(({ distance, limit }) => distance <= limit)
.sort((a, b) => a.distance - b.distance || (a.name < b.name ? -1 : a.name > b.name ? 1 : 0))
.slice(0, 3)
.map(({ name }) => name);
return closest.length > 0 ? ` Closest assigned names: ${closest.join(", ")}.` : "";
}
export const ASSIGNED_SCHEMA_LISTING_BUDGET_BYTES = 768 * 1024;
export const ASSIGNED_SCHEMA_ADMISSION_CACHE_ENTRIES = 1024;
@@ -316,6 +342,19 @@ export async function createAssignedMcpTools(input: {
return matches[0]?.[0];
}
// Intersect fresh grants with this session's pinned catalog. Revocations
// take effect immediately; newly assigned tools require a new session.
async function authorizedTools(currentWorkMode?: WorkMode) {
const current = new Map(deduplicateToolDescriptors(await input.gateway.listToolsForNamedGateway({
gatewayPublicId: input.gatewayPublicId, bearerToken: input.bearerToken,
})).map(tool => [tool.name, tool]));
return [...tools].filter(([, { tool }]) => {
const fresh = current.get(tool.name);
return fresh && permits(tool) && permits(tool, currentWorkMode) &&
permits(fresh) && permits(fresh, currentWorkMode);
});
}
async function search(argumentsValue: unknown, currentWorkMode?: WorkMode) {
const args = object(argumentsValue);
const offset = args.offset ?? 0;
@@ -332,21 +371,14 @@ export async function createAssignedMcpTools(input: {
if (schemaOffset > 0 && !hasSchemaTool) {
throw new Error("assigned_mcp_tool_invalid_arguments: A positive schemaOffset requires an exact inputSchemaRef in schemaTool. Omit schemaTool and schemaOffset to search assigned tools.");
}
// Intersect fresh grants with this session's pinned catalog. Revocations
// take effect immediately; newly assigned tools require a new session.
const current = new Map(deduplicateToolDescriptors(await input.gateway.listToolsForNamedGateway({
gatewayPublicId: input.gatewayPublicId, bearerToken: input.bearerToken,
})).map(tool => [tool.name, tool]));
const terms = args.query.toLowerCase().trim().split(/\s+/).filter(Boolean);
const authorized = [...tools].filter(([, { tool }]) => {
const fresh = current.get(tool.name);
return fresh && permits(tool) && permits(tool, currentWorkMode) &&
permits(fresh) && permits(fresh, currentWorkMode);
});
const authorized = await authorizedTools(currentWorkMode);
if (hasSchemaTool) {
const reference = resolveAssignedName(args.schemaTool as string, "schemaTool");
const selected = authorized.find(([name]) => name === reference);
if (!selected) throw new Error("assigned_mcp_tool_unknown: schemaTool must match an authorized inputSchemaRef. Omit schemaTool to search assigned tools, then use an exact inputSchemaRef from the results.");
if (!selected) {
throw new Error(`assigned_mcp_tool_unknown: schemaTool must match an authorized inputSchemaRef.${closestNames(args.schemaTool as string, authorized.map(([name]) => name))} Omit schemaTool to search assigned tools, then use an exact inputSchemaRef from the results.`);
}
const schema = JSON.stringify(selected[1].schema);
const end = schemaOffset + schemaChunkCharacters;
return {
@@ -405,7 +437,8 @@ export async function createAssignedMcpTools(input: {
parameters = object(args.arguments);
const resolved = resolveAssignedName(args.name, "name");
if (!resolved) {
throw new Error("assigned_mcp_tool_unknown: name must be an exact name returned by paperclip_search_assigned_tools. Search assigned tools again and use the returned name.");
const authorized = await authorizedTools(currentWorkMode).then(entries => entries.map(([candidate]) => candidate), () => []);
throw new Error(`assigned_mcp_tool_unknown: name must be an exact name returned by paperclip_search_assigned_tools.${closestNames(args.name, authorized)} Search assigned tools again and use the returned name.`);
}
name = resolved;
}