mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-29 04:25:52 +01:00
fix(runner): keep human input tool visible to Claude (#18046)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native task agents must create durable human questions and approvals. > - The request_human_input schema has an object root with a conditional wrapped in allOf. > - The pinned Claude client omits that tool from the API request in an isolated gateway environment. > - An equivalent direct conditional keeps the tool available and preserves validation. > - This pull request fixes the schema and tests discovery through the real pinned client. ## Linked Issues or Issue Description **What happened?** A retained native Claude Plan attempt could save its Plan but received `No such tool available: mcp__paperclip__request_human_input`. A local comparison with Claude Code 2.1.286 reproduces the discovery failure without a provider: the client offers the control tool but omits the human-input tool. It offers both tools after this schema change. **Expected behavior** The client must offer the authorized human-input tool. Invalid question forms must still fail validation. **Steps to reproduce** 1. Serve the current request_human_input definition through Runner's local MCP bridge. 2. Start the pinned Claude client with a local mock Messages API and tool search disabled. 3. Inspect the API request's offered tools. Before the fix, request_human_input is absent. After the fix, it is present. Refs #18034 for the separate saved-Plan readback repair. This PR addresses tool discovery. It does not reclassify or rerun the original failed live attempts. ## What Changed - Put the existing question-payload condition directly on the live and scenario object schemas. A one-branch allOf wrapper is unnecessary. - Add a regression through the pinned native Claude executable, real Runner MCP bridge and loopback mock API. The test uses fake credentials and executes no tools. - Keep validation tests for mixed forms and malformed payloads. Add an explicit missing-payload rejection. - Refresh generated schema identity and the explicitly seeded catalog fixture with the maintained generator. Historical live evidence is unchanged. ## Verification - Native discovery regression red: the control tool is present and request_human_input is absent. - Green: 92 tests pass across native discovery, Runner tool bridge, core contracts and semantic tools. The final argument-visibility regression and core contracts also pass together (21 tests). - Runner package typecheck passes. - Runner package build passes. Its first run found stale generated contract identities; regeneration resolved that check. Both logs are retained. - No real provider request, paid eval, sandbox allocation or automatic retry. - Local verification is scoped to the Runner package. Full repository CI must pass on this PR's exact head before merge. New live Plan acceptance is still required. ## Risks - The conditional has the same JSON Schema validation meaning. The provider-facing object shape changes, so the generated catalog identity changes. - The native regression uses the optional platform CLI already required by the pinned ACP profile. It does not establish provider billing, model behavior, or full Plan acceptance. - Optional case/workflow tool discovery is a separate confirmed gap tracked in #18051. This human-input fix does not qualify those tools. - No database migration, permission, budget, workflow, lockfile or UI change. ## Model Used - OpenAI Codex, GPT-6, with reasoning and code execution tools. The exact serving variant and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed
1 parent
567c043b17
commit
4f8104dec6
6 files changed
+74
-5
No files matched your search
@@ -1 +1 @@
|
||||
[{"annotations":{"exposure":"always","operationId":"get_task_context","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the active task and actor, including the exact approved Markdown revision when this issue has an accepted plan.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"get_task_context","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"get_task_history","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded comments on the active task.","inputSchema":{"additionalProperties":false,"properties":{"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":[],"type":"object"},"name":"get_task_history","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_documents","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"List revisioned documents on the active task.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"list_documents","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"read_document","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the current revision of one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable issue-document key.","maxLength":120,"minLength":1,"type":"string"}},"required":["key"],"type":"object"},"name":"read_document","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_document_revisions","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded revision history for one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable issue-document key.","maxLength":120,"minLength":1,"type":"string"},"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":["key"],"type":"object"},"name":"list_document_revisions","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"report_progress","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append a durable progress comment to the active task.","inputSchema":{"additionalProperties":false,"properties":{"body":{"description":"Multiline progress update.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","body"],"type":"object"},"name":"report_progress","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"answer_status_question","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append the answer to a status-only wake without changing task disposition.","inputSchema":{"additionalProperties":false,"properties":{"body":{"description":"Concise status answer.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","body"],"type":"object"},"name":"answer_status_question","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"write_document","requiredClaimLine truncated
|
||||
[{"annotations":{"exposure":"always","operationId":"get_task_context","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the active task and actor, including the exact approved Markdown revision when this issue has an accepted plan.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"get_task_context","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"get_task_history","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded comments on the active task.","inputSchema":{"additionalProperties":false,"properties":{"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":[],"type":"object"},"name":"get_task_history","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_documents","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"List revisioned documents on the active task.","inputSchema":{"additionalProperties":false,"properties":{},"required":[],"type":"object"},"name":"list_documents","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"read_document","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read the current revision of one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable issue-document key.","maxLength":120,"minLength":1,"type":"string"}},"required":["key"],"type":"object"},"name":"read_document","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"list_document_revisions","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Read bounded revision history for one active-task document.","inputSchema":{"additionalProperties":false,"properties":{"key":{"description":"Stable issue-document key.","maxLength":120,"minLength":1,"type":"string"},"limit":{"default":50,"maximum":200,"minimum":1,"type":"integer"}},"required":["key"],"type":"object"},"name":"list_document_revisions","outputSchema":{"additionalProperties":true,"type":"object"}},{"annotations":{"exposure":"always","operationId":"report_progress","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append a durable progress comment to the active task.","inputSchema":{"additionalProperties":false,"properties":{"body":{"description":"Multiline progress update.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","body"],"type":"object"},"name":"report_progress","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"answer_status_question","requiredClaims":[],"semanticContract":"paperclip.semantic-tool.v1","version":1},"description":"Append the answer to a status-only wake without changing task disposition.","inputSchema":{"additionalProperties":false,"properties":{"body":{"description":"Concise status answer.","maxLength":20000,"minLength":1,"type":"string"},"idempotencyKey":{"description":"Caller-stable retry key.","maxLength":240,"minLength":1,"type":"string"}},"required":["idempotencyKey","body"],"type":"object"},"name":"answer_status_question","outputSchema":{"additionalProperties":false,"properties":{"commandId":{"description":"Stable command identifier.","maxLength":200,"minLength":1,"type":"string"},"disposition":{"enum":["applied","duplicate"]},"entityRefs":{"description":"Entities affected by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"scheduledWakeIds":{"description":"Wake identifiers scheduled by the operation.","items":{"minLength":1,"type":"string"},"maxItems":200,"type":"array","uniqueItems":true},"stateRevision":{"minimum":0,"type":"integer"}},"required":["commandId","disposition","stateRevision","entityRefs","scheduledWakeIds"],"type":"object"}},{"annotations":{"exposure":"always","operationId":"write_document","requiredClaimLine truncated
|
||||
@@ -24,7 +24,7 @@
|
||||
"prpVersion": 1,
|
||||
"nativeExecutionVersion": 1,
|
||||
"catalogVersion": 1,
|
||||
"catalogSha256": "sha256:bfd48a7478b44ea295d3f260ba2d228d89b6b752794922628c9736737ba49dcb",
|
||||
"catalogSha256": "sha256:8b0a124566a82c7827352a576ba1c77c14fd28b89a2c874e15e60b9fc49f1f8e",
|
||||
"driverContractVersion": 1,
|
||||
"driverKind": "paperclip-deterministic",
|
||||
"driverVersion": "1.0.0"
|
||||
|
||||
@@ -180,7 +180,7 @@
|
||||
},
|
||||
{
|
||||
"path": "fixtures/evals/native-execution-seeded.json",
|
||||
"sha256": "080a41b9041afda7b34a082d7615a5b6f71b10caeb5386bb99045f34aa38af18",
|
||||
"sha256": "369a626b8721d0f464b75dc3e39db8b2acab148599f3d14e5551557d2cc143e5",
|
||||
"expectation": "accept",
|
||||
"compatibilityCase": "canonical"
|
||||
},
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { createServer } from "node:http";
|
||||
import { mkdtemp, mkdir, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
import { claudeCliPath } from "@agentclientprotocol/claude-agent-acp/dist/acp-agent.js";
|
||||
import { expect, it } from "vitest";
|
||||
import { requestHumanInputAction } from "../protocol-actions/request-human-input.js";
|
||||
import { startRunnerToolBridge } from "./runner-tool-bridge.js";
|
||||
|
||||
it("keeps the human-input tool available in the pinned Claude client", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "claude-question-schema-"));
|
||||
const config = join(root, "config"), cwd = join(root, "workspace");
|
||||
await Promise.all([mkdir(config), mkdir(cwd)]);
|
||||
const offered: Array<Array<{ name: string; input_schema: Record<string, unknown> }>> = [];
|
||||
const api = createServer(async (request, response) => {
|
||||
let raw = "";
|
||||
for await (const chunk of request) raw += chunk;
|
||||
const body = JSON.parse(raw || "{}");
|
||||
if (Array.isArray(body.tools)) offered.push(body.tools);
|
||||
response.writeHead(200, { "content-type": "text/event-stream" });
|
||||
// The real client reaches only this local API, which never asks it to run tools.
|
||||
const frames = [
|
||||
{ type: "message_start", message: { id: "msg_fixture", type: "message", role: "assistant", model: "claude-sonnet-5", content: [], stop_reason: null, stop_sequence: null, usage: { input_tokens: 1, output_tokens: 0 } } },
|
||||
{ type: "content_block_start", index: 0, content_block: { type: "text", text: "" } },
|
||||
{ type: "content_block_delta", index: 0, delta: { type: "text_delta", text: "Local fixture complete." } },
|
||||
{ type: "content_block_stop", index: 0 },
|
||||
{ type: "message_delta", delta: { stop_reason: "end_turn", stop_sequence: null }, usage: { output_tokens: 1 } },
|
||||
{ type: "message_stop" },
|
||||
];
|
||||
for (const frame of frames) response.write(`event: ${frame.type}\ndata: ${JSON.stringify(frame)}\n\n`);
|
||||
response.end();
|
||||
});
|
||||
const bridge = await startRunnerToolBridge({ tools: [
|
||||
{ name: "request_human_input", description: "Ask a durable question.", inputSchema: requestHumanInputAction.live.descriptor.inputSchema },
|
||||
{ name: "write_document", description: "Discovery control.", inputSchema: { type: "object", properties: { key: { type: "string" } } } },
|
||||
], handler: async () => { throw new Error("This fixture must not execute tools"); } });
|
||||
try {
|
||||
await new Promise<void>(resolve => api.listen(0, "127.0.0.1", resolve));
|
||||
const address = api.address() as { port: number };
|
||||
const mcpConfig = { mcpServers: { paperclip: { type: "http", url: bridge.url, headers: { Authorization: `Bearer ${bridge.secret}` } } } };
|
||||
await promisify(execFile)(await claudeCliPath(), [
|
||||
"--bare", "--print", "--model", "claude-sonnet-5", "--strict-mcp-config", "--mcp-config", JSON.stringify(mcpConfig),
|
||||
"--setting-sources", "", "--permission-mode", "dontAsk", "--no-session-persistence", "--tools", "", "--max-turns", "1",
|
||||
"Report this local fixture's result. Do not call tools.",
|
||||
], { cwd, timeout: 20_000, env: {
|
||||
PATH: process.env.PATH, TMPDIR: root, CLAUDE_CONFIG_DIR: config,
|
||||
ANTHROPIC_API_KEY: "local-fixture-only", ANTHROPIC_BASE_URL: `http://127.0.0.1:${address.port}`,
|
||||
CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC: "1", DISABLE_TELEMETRY: "1", DISABLE_ERROR_REPORTING: "1", ENABLE_TOOL_SEARCH: "false",
|
||||
} });
|
||||
expect(offered.length).toBeGreaterThan(0);
|
||||
expect(offered.at(-1)?.map(tool => tool.name)).toContain("mcp__paperclip__write_document");
|
||||
const question = offered.at(-1)?.find(tool => tool.name === "mcp__paperclip__request_human_input");
|
||||
expect(question).toBeDefined();
|
||||
expect(question!.input_schema).toMatchObject({ type: "object", properties: {
|
||||
idempotencyKey: { type: "string" }, interactionKind: { enum: expect.arrayContaining(["questions", "confirmation"]) },
|
||||
title: { type: "string" }, prompt: { type: "string" }, payload: { type: "object" },
|
||||
}, required: expect.arrayContaining(["idempotencyKey", "interactionKind", "title", "prompt", "continuationPolicy"]) });
|
||||
} finally {
|
||||
api.closeAllConnections();
|
||||
await new Promise<void>(resolve => api.close(() => resolve()));
|
||||
await bridge.close();
|
||||
await rm(root, { recursive: true, force: true, maxRetries: 5, retryDelay: 50 });
|
||||
}
|
||||
}, 30_000);
|
||||
@@ -16,6 +16,7 @@ describe("core Paperclip protocol action contracts", () => {
|
||||
const input = { ...action.examples.call.input, interactionKind: "questions", payload };
|
||||
if (schema === action.scenario!.descriptor.inputSchema) delete (input as { idempotencyKey?: string }).idempotencyKey;
|
||||
expect(ajv.validate(schema, input), JSON.stringify(ajv.errors)).toBe(true);
|
||||
expect(ajv.validate(schema, { ...input, payload: undefined })).toBe(false);
|
||||
expect(ajv.validate(schema, { ...input, payload: { version: 1 } })).toBe(false);
|
||||
expect(ajv.validate(schema, { ...input, payload: { version: 1, questionSet: { ...payload.questionSet, questions: [{ ...payload.questionSet.questions[0], options: [{ id: "a", label: "A" }] }] } } })).toBe(false);
|
||||
}
|
||||
|
||||
@@ -86,7 +86,9 @@ export const requestHumanInputAction = {
|
||||
"skill_test"
|
||||
],
|
||||
"inputSchema": {
|
||||
"allOf": [questionsPayloadRequirement],
|
||||
// The pinned Claude client drops a tool with this conditional in root allOf.
|
||||
// Keep the equivalent direct condition on the concrete object root.
|
||||
...questionsPayloadRequirement,
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"idempotencyKey": {
|
||||
@@ -216,7 +218,7 @@ export const requestHumanInputAction = {
|
||||
"title": "Request human input",
|
||||
"description": "Create a typed confirmation, checkbox, question, task suggestion, or item-verdict request.",
|
||||
"inputSchema": {
|
||||
"allOf": [questionsPayloadRequirement],
|
||||
...questionsPayloadRequirement,
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"interactionKind": {
|
||||
|
||||
Reference in new issue
Block a user