fix(runner): name missing tool fields and what the call included (#21252)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents act through Paperclip tools, and the runner validates each
tool call against its input schema
> - When a call is invalid, the runner returns an "Invalid tool input"
error to the model
> - For a call that left out required fields, that error was a list of
JSON Schema messages such as `/ must have required property 'kickoff'`
> - A model sent `paperclip_create_case` with only
`workflow_document_text` for about 5 minutes, believed it had sent every
field, and never replied to the person
> - This pull request makes the error name the missing fields, the
fields the call did include, and a short example of the missing fields
> - The benefit is that a model can see its own mistake and correct the
call on the next attempt

## Linked Issues or Issue Description

**What happened?**
In the paid chat eval `cases.create-case` (OpenCode with
`openrouter/deepseek/deepseek-v4.1-flash-1017`), the agent called
`paperclip_create_case` about 20 times with only
`workflow_document_text`. Each call returned `Invalid tool input: / must
have required property 'idempotencyKey'; / must have required property
'caseType'; ...`. The model's reasoning said that the other fields were
being stripped. It never created the case and never replied.

**Steps to reproduce**
1. Validate `{ "workflow_document_text": "..." }` against the
`create_case` input schema.
2. Read `toolInputError(validator)`.

**Expected behavior**
The error says which required fields are missing, says that the call
included only `workflow_document_text`, and shows a short example of the
missing fields.

## What Changed

- `packages/paperclip-runner/src/drivers/tool-input-error.ts`: for
missing top-level required fields, the error now says `missing required
fields: idempotencyKey, caseType, title, kickoff.`, `This call included
only: workflow_document_text.`, and a placeholder example built from the
schema, such as
`{"idempotencyKey":"<idempotencyKey>",...,"kickoff":{"toAgentId":"<toAgentId>","phase":"<phase>","instructions":"<instructions>"}}`.
Other validation errors follow after `Also:`. Field-description guidance
is unchanged.
- `packages/paperclip-runner/src/drivers/opencode/mcp-bridge.ts` and
`runner-tool-bridge.ts`: pass the call arguments to the error builder.
Only argument names are read; values are never echoed.
- `packages/paperclip-runner/src/drivers/tool-input-error.test.ts`: a
test on the real `create_case` schema and a test that other errors still
follow the missing fields.
- The `create_case` schema is not flattened. The model also left out the
flat fields (`idempotencyKey`, `caseType`, `title`), so nesting was not
the cause, and `kickoff` is shared with `handoff_case`.

## Verification

- `cd packages/paperclip-runner && npx vitest run
src/drivers/tool-input-error.test.ts
src/drivers/opencode/mcp-bridge.test.ts
src/drivers/opencode/opencode-server-driver.test.ts` passes (115 tests).
The two new tests fail on master.

## Risks

- Low risk. Only the error text for invalid calls changes. Validation
and dispatch are unchanged. The example is bounded to 600 characters and
the field lists to 12 names.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed 2026-11-12 20:06:48 -06:00
1 parent bc2cdeaa7c
commit 52817dff55
4 files changed
+78 -9

No files matched your search

@@ -342,7 +342,7 @@ async function handleRequest(
? "continuation" : errors.some(error => error.instancePath === "") ? "root" : "other";
context.diagnoseFinishInput({ input: params.arguments, callId, validationCategory });
}
const error = toolInputError(validator);
const error = toolInputError(validator, args);
try { context.onRejectedCall?.({ tool, callId, arguments: args }, error); } catch {}
outcome = "error";
writeRpc(response, id, {
@@ -373,7 +373,7 @@ async function handleRequest(
writeToolError(
response,
id,
toolInputError(validator),
toolInputError(validator, args),
);
return;
}
@@ -1,6 +1,7 @@
import { Ajv2020 } from "ajv/dist/2020.js";
import { expect, it } from "vitest";
import { toolInputError } from "./tool-input-error.js";
import { createCaseAction } from "../protocol-actions/case-data.js";
it("leaves ordinary validation errors unchanged and excludes guidance from unrelated scopes", () => {
const validate = new Ajv2020({ allErrors: true }).compile({ type: "object", required: ["id", "child"], properties: {
@@ -23,3 +24,25 @@ it("bounds and deduplicates direct missing-property descriptions", () => {
expect(result).not.toContain("e".repeat(1000));
expect(result).not.toContain("a".repeat(1001));
});
it("names the missing create_case fields, what the call included, and a short example", () => {
const validate = new Ajv2020({ allErrors: true, strict: false }).compile(createCaseAction.live.descriptor.inputSchema);
const input = { workflow_document_text: "Investigate, then verify." };
expect(validate(input)).toBe(false);
const result = toolInputError(validate, input);
expect(result).toContain("missing required fields: idempotencyKey, caseType, title, kickoff.");
expect(result).toContain("This call included only: workflow_document_text.");
expect(result).toContain('Missing fields example: {"idempotencyKey":"<idempotencyKey>","caseType":"<caseType>","title":"<title>","kickoff":{"toAgentId":"<toAgentId>","phase":"<phase>","instructions":"<instructions>"}}');
expect(result).not.toContain("must have required property");
});
it("keeps other validation errors after the missing fields", () => {
const validate = new Ajv2020({ allErrors: true }).compile({ type: "object", required: ["id", "count"], properties: {
id: { type: "string" }, count: { type: "integer" }, child: { type: "object", required: ["id"], properties: { id: { type: "string" } } },
} });
const input = { child: {} };
validate(input);
expect(toolInputError(validate, input)).toBe(
'Invalid tool input: missing required fields: id, count. This call included only: child. Send every required field together in one call, alongside the fields you already sent. Missing fields example: {"id":"<id>","count":0} Also: /child must have required property \'id\'',
);
});
@@ -1,15 +1,50 @@
import type { ValidateFunction } from "ajv/dist/2020.js";
import type { ErrorObject, ValidateFunction } from "ajv/dist/2020.js";
type JsonSchemaNode = {
type?: unknown;
properties?: Record<string, JsonSchemaNode>;
required?: unknown;
description?: unknown;
};
const MAX_LISTED_FIELDS = 12;
const MAX_EXAMPLE_LENGTH = 600;
function isRootRequiredError(error: ErrorObject) {
return error.keyword === "required" && error.instancePath === "" && error.schemaPath === "#/required";
}
function placeholder(key: string, schema: JsonSchemaNode | undefined, depth: number): unknown {
const type = Array.isArray(schema?.type) ? schema?.type.find(entry => entry !== "null") : schema?.type;
if (type === "object") {
const required = Array.isArray(schema?.required) ? schema.required.filter((entry): entry is string => typeof entry === "string") : [];
if (depth >= 2 || required.length === 0) return {};
return Object.fromEntries(required.map(child => [child, placeholder(child, schema?.properties?.[child], depth + 1)]));
}
if (type === "array") return [];
if (type === "integer" || type === "number") return 0;
if (type === "boolean") return false;
return `<${key}>`;
}
function listFields(keys: string[]) {
const shown = keys.slice(0, MAX_LISTED_FIELDS).join(", ");
return keys.length > MAX_LISTED_FIELDS ? `${shown}, and ${keys.length - MAX_LISTED_FIELDS} more` : shown;
}
/** Return schema-owned recovery guidance without changing validation or dispatch. */
export function toolInputError(validator: ValidateFunction | undefined): string {
export function toolInputError(validator: ValidateFunction | undefined, input?: unknown): string {
const errors = validator?.errors ?? [];
const detail = errors.map(error => `${error.instancePath || "/"} ${error.message ?? "is invalid"}`).join("; ");
const schema = validator?.schema;
const properties = typeof schema === "object" && schema !== null ? schema.properties : undefined;
const rootSchema = typeof schema === "object" && schema !== null ? schema as JsonSchemaNode : undefined;
const properties = rootSchema?.properties;
const missing = [...new Set(errors.filter(isRootRequiredError)
.map(error => error.params.missingProperty)
.filter((key): key is string => typeof key === "string"))];
const otherErrors = missing.length ? errors.filter(error => !isRootRequiredError(error)) : errors;
const detail = otherErrors.map(error => `${error.instancePath || "/"} ${error.message ?? "is invalid"}`).join("; ");
const guidance = new Set<string>();
for (const error of errors) {
// Only direct required properties are supported. Nested schemas and refs
// need their own resolution; do not attach guidance from a different scope.
if (error.keyword !== "required" || error.instancePath !== "") continue;
const key = error.params.missingProperty;
if (typeof key !== "string" || !properties || !Object.hasOwn(properties, key)) continue;
@@ -17,5 +52,16 @@ export function toolInputError(validator: ValidateFunction | undefined): string
if (typeof description === "string" && description.trim()) guidance.add(description.slice(0, 1000));
if (guidance.size === 3) break;
}
return `Invalid tool input${detail ? `: ${detail}` : "."}${guidance.size ? ` ${[...guidance].join(" ")}` : ""}`;
const parts: string[] = [];
if (missing.length) {
const received = typeof input === "object" && input !== null && !Array.isArray(input) ? Object.keys(input) : [];
const example = JSON.stringify(Object.fromEntries(missing.map(key => [key, placeholder(key, properties?.[key], 0)])));
parts.push(`missing required field${missing.length === 1 ? "" : "s"}: ${listFields(missing)}.`);
parts.push(received.length ? `This call included only: ${listFields(received)}.` : "This call included no fields.");
parts.push("Send every required field together in one call, alongside the fields you already sent.");
if (example.length <= MAX_EXAMPLE_LENGTH) parts.push(`Missing fields example: ${example}`);
}
if (detail) parts.push(missing.length ? `Also: ${detail}` : detail);
const body = parts.join(" ");
return `Invalid tool input${body ? `: ${body}` : "."}${guidance.size ? ` ${[...guidance].join(" ")}` : ""}`;
}