From 314b8936780b428b6ed9d03143e782ff9a2f75b7 Mon Sep 17 00:00:00 2001 From: Nicky Leach Date: Fri, 13 Nov 2026 00:16:58 -0800 Subject: [PATCH] fix(native-runtime): deliver assigned gateway tools to every native harness through the PRP relay (#21274) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - An agent can have app connections. Paperclip gives the agent the tools of those apps through an MCP gateway, with its approval and audit rules. > - Native harnesses reach the gateway in two ways today. In the relay, the server keeps the gateway token and offers the tools over the authenticated Runner channel. In the direct way, the server gives the gateway URL and a bearer token to the harness. > - Which way a harness uses comes from a list of special cases that grew one harness at a time. Local Codex and all ACPX agents use the direct way. The others use the relay. > - The direct way puts a run-scoped token in a provider process. A process that the agent starts can read it. A provider process that serves more than one run can hold a revoked token. > - This pull request replaces the list with one rule. Each native harness gets its assigned gateway tools through the relay. > - The benefit is one behavior for all native harnesses, and no gateway token in a provider process. ## Linked Issues or Issue Description No public issue exists. The issue is described here. Refs #13942 (the relay), #17149 (ACPX accepts relayed tools) and #19912 (the design for turn-scoped ACPX MCP authority). **What existing behavior does this improve?** How a native harness reaches the tools of its assigned MCP gateway. **Subsystem affected** Native runtime in the server (`native-session-executor.ts`, `native-session-resume.ts`). One Runner test file. One Rust test. **Current behavior** - `relayAssignedMcp` is true for plugin tasks, Cursor Cloud, OpenCode, Codex on a remote target, and Dot. - For local Codex and for each ACPX agent (Claude, Codex, Cursor, Pi, Kimi, Grok, Copilot, Antigravity), Core puts `PAPERCLIP_NATIVE_MCP_NAME`, `PAPERCLIP_NATIVE_MCP_URL` and `PAPERCLIP_NATIVE_MCP_TOKEN` in the Runner environment. The ACPX host gives them to the provider as an HTTP MCP server. The local Codex home gets them in `config.toml`. - The token is run-scoped. Core revokes it when the run ends. **Proposed behavior** - A native run that has an assigned gateway uses the relay. The condition names no provider. - Core builds the gateway tools on the control plane, offers them with the other Runner tools, dispatches each call over PRP, and authorizes each call again through the gateway. - Core removes the three `PAPERCLIP_NATIVE_MCP_*` variables from each Runner environment. - The tool contract fingerprint has the relay marker for each target, not only for remote targets. **Reason and benefit** - All native harnesses have the same behavior. - No provider process holds a gateway token. - A sandbox does not need to reach the gateway URL of the host. ## What Changed - `server/src/services/native-runtime/native-session-executor.ts` - The list of special cases is gone. Core validates the binding and builds the relayed tools when a binding is present. - Core does not list the gateway for a review run, Grok Bot or Muse. The tool authority never offered app tools to those. A gateway error can therefore not fail those runs. - OpenCode keeps its `on_demand` catalog. That option depends on the provider, not on the relay. - Core removes the three variables from each Runner environment. - `server/src/services/native-runtime/native-session-resume.ts`: the relay marker in the tool contract fingerprint is unconditional. - `doc/execution-semantics.md`: the text about the tool contract fingerprint now states the rule, the tool names and the cutover. - App tool schemas. New module `packages/paperclip-runner/src/drivers/tool-input-schema.ts` compiles each schema by itself, in its declared dialect (draft-07, 2019-09 or 2020-12), with no shared `$id` registry. Core uses it when it builds the relayed tools (`assigned-mcp-tools.ts`). Core does not offer a tool whose schema has an unsupported dialect or does not compile, and writes a log entry with the tool name and the reason. The run does not fail. The ACPX tool bridge and the OpenCode MCP bridge use the same function. The tools of Paperclip itself stay strict. - Large arguments. The ACPX sidecar sends the arguments of a tool call to runnerd with no display limit. The limit is the tool-call admission limit of runnerd, 256 KiB. A larger call fails with a tool error that gives the limit. The sidecar does not dispatch it. - Tests in `native-session-executor.test.ts`, `native-session-resume.test.ts`, `runtime-context-materializer.test.ts` and `acpx_provider_backend.rs`. Not changed in this pull request: the direct code path. It is still in the Runner, but it receives no binding. #21311 deletes it. ## Behavior that each harness sees | Harness | Before | After | |---|---|---| | Codex on a remote target, OpenCode, Cursor Cloud, Dot, plugin tasks | Relay | No change | | Local Codex | Direct: a `paperclip-assigned` server in `config.toml` | Relay. App tools are dynamic tools named `app__` | | ACPX agents | Direct: a second HTTP MCP server | Relay. App tools are on the `paperclip` bridge as `mcp__paperclip__app__` | | ACPX with a preconfigured home | No gateway | No change | | Grok Bot, Muse | Got the variables and ignored them | Do not get the variables | - A relayed tool name has a maximum of 64 characters. With the `mcp__paperclip__` prefix it has a maximum of 80. - Permission behavior keeps the same intent. For Claude in the `approve-reads` and `approve-paperclip` modes, app tools ask for approval, as before. For the other ACPX agents the permission decision does not use the server name. - In a planning run or an ask run, the relay offers only tools with read risk. The direct gateway offered all tools. This is stricter. - When the catalog is too large, each harness gets the two compact tools `paperclip_search_assigned_tools` and `paperclip_call_assigned_tool`. - The four helper tools for resources and prompts (`paperclip_list_resources`, `paperclip_read_resource`, `paperclip_list_prompts`, `paperclip_get_prompt`) are not in the relay. No harness loses them: a run token permits only `tools/list` and `tools/call` (`run-preparation.ts`), so these four tools already fail with `gateway_token_action_denied` on the direct path. ## State saved by the previous release - **Local sessions of each provider start again one time.** The tool contract fingerprint changes for local targets. The next run sees a different fingerprint, starts a new provider session with the full task context in the same workspace, and keeps the old saved state. This is the existing rotation for a tool contract change. Codex needs it, because Codex cannot add tools to a thread that it resumes. For the local providers that already used the relay, the restart is not necessary. It is accepted so that the rule names no provider. - **Remote sessions continue.** Their fingerprint does not change. For ACPX, the attach of the next run replaces the saved tool set. - **A kept Codex home.** Paperclip writes `config.toml` again at each start and recovery. The old gateway server and its token are removed. - **A deploy lands during a run, and the Runner survives.** The controller attaches to the same Runner for the same run. The old provider keeps its direct gateway connection. The controller detach keeps the gateway token of that run, and Paperclip revokes it when the run ends. - **A deploy lands during a run, and the Runner is dead.** The Runner starts again for the same run with its saved tool set and no direct gateway. The assigned app tools are not available for the remainder of that one run. The next run gets them through the relay. This applies to each harness that moves to the relay. It has no test. - Paperclip refuses no saved state and discards none. ## Verification - `cd server && npx vitest run src/services/native-runtime/native-session-executor.test.ts src/services/native-runtime/native-session-resume.test.ts`: `1508 passed | 7 skipped`, 1 failed. The failed test is "archives a damaged prior epoch before completing guarded same-task replacement with real runnerd" (`ENOENT … authority-epochs`). It fails in the same way on `master` at 53608bd1d6 and with the product change removed, on macOS. - `cd server && npx tsc --noEmit -p tsconfig.json`: no errors. - `cd packages/paperclip-runner && npx vitest run src/drivers/runtime-context-materializer.test.ts`: `18 passed`. `npx tsc --noEmit -p tsconfig.test.json`: no errors. - `cargo fmt --manifest-path runner/Cargo.toml --all -- --check`: no changes. `cargo test -p paperclip-runner-core --lib acpx_provider_backend`: `50 passed`. - `npx vitest run src/drivers/tool-input-schema.test.ts src/drivers/runner-tool-bridge.schemas.test.ts src/drivers/runner-tool-bridge.test.ts src/cli/acpx-runtime-sidecar.test.ts src/drivers/opencode` in `packages/paperclip-runner`: `21 passed (21)` files, `377 passed` tests. `pnpm run typecheck:typescript`: no errors. - `cargo test -p paperclip-runner-core --test acpx_event_payload --test tool_input_schema_fixtures`: 10 and 3 passed. - Schema tests use one shared fixture (`tool-input-schema-fixtures.json`) for the TypeScript compiler, the tool bridge and the Rust `begin_call`: a draft-07 schema, a second schema with the same `$id`, 2019-09, 2020-12, draft-04, an unknown dialect, a `$ref` with no target, and an invalid keyword. Before the fix the bridge failed with `no schema with key or ref "http://json-schema.org/draft-07/schema#"`. - A tool schema whose `$id` is the URI of a meta-schema cannot damage the shared compiler. Before this fix, `removeSchema` deleted that meta-schema, and later schemas failed to compile for each run in the process. A test with 7 reserved ids failed before the fix and passes now. - Core limits the compile work of each run start. A schema over 512 KiB is not compiled, and one listing compiles a maximum of 768 KiB of schemas. These are the limits of the Runner. A tool over a limit is served only through the search and call tools. Core keeps the outcome for each schema digest (1,024 entries) and yields to the event loop before each compile. Four tests failed before this change. - A sidecar test sends a tool call with 200 KiB of arguments and receives them byte for byte, refuses a call with 300 KiB with the named error, and keeps the replacement of a lone surrogate. Tests, each seen to fail without the product change: - "keeps assigned MCP credentials on the control plane for %s" now runs for remote Codex, local Codex, OpenCode and ACPX Claude, Pi, Kimi, Copilot and Cursor. It checks that Core builds the relay, offers the tool, dispatches a call, and gives no `PAPERCLIP_NATIVE_MCP_*` variable to the Runner. Without the change, the five new harness cases fail. - "relays the assigned gateway when a deploy lands mid-run…" runs for Claude and Pi. - The fingerprint tests keep the old local fingerprint as a retained value and check that the new one is different. The remote fingerprints for Codex and OpenCode do not change. - "drops a retained direct gateway server when the relayed launch has no binding": a Codex home with the old `config.toml` entry loses that entry at the next start and at a recovery of the same run. - `next_run_attach_offers_relayed_gateway_tools_to_a_direct_gateway_checkpoint` runs for Claude and Pi. Not run: a live run with a real provider and a real app connection for each harness. Not run: `pnpm verify` and the full suite. CI supplies that coverage. ## Risks - **An ACPX app call has a limit of 256 KiB for its arguments.** The direct gateway accepted a request body of 10 MB. Codex and OpenCode permit 480 KiB. To get nearer to 10 MB, runnerd needs a larger admission limit for tool calls or a protocol that sends arguments in parts. That is not in this pull request. - **A tool with a draft-04 or draft-06 schema is not offered.** Ajv 8 does not support draft-04, and the TypeScript and Rust validators must agree. On the direct path the provider got such a tool. - **Core compiles app schemas on its event loop.** Before this pull request Core did not compile them. With the limits above, one valid schema near 512 KiB can hold the loop for about 0.65 s, and one listing for about 1.2 s in total (measured on a developer machine). An app that changes its schemas at each listing gets no help from the cache. A worker thread with a timeout would remove this; it is not in this pull request. - **Tool names change for local Codex and all ACPX agents.** A saved agent instruction that names a gateway tool directly does not match the new name. - **Each local native session starts again one time after the deploy.** The provider conversation of that session does not continue. The task context and the workspace stay. - **A run that is in flight at deploy, with a dead Runner, has no app tools until it ends.** - **Names of 80 characters are not tested live** with Kimi, Copilot, Grok, Antigravity and Codex through ACPX. Pi already shortens a name longer than 64 characters. - **Managed Claude and AWS AgentCore now receive relayed app tools.** This is not tested live. - **A gateway token expires after 1 hour.** The relay keeps the token on the controller for the whole run, as OpenCode and remote Codex do today. A longer run loses its app tools on both paths. This pull request does not change that. - **The gateway URL must have the path `/mcp/gateways/`.** With a path prefix in the API base URL, a run with an assigned gateway fails with `assigned native MCP gateway path is invalid`. This was true for the relayed harnesses before. It is now true for all. - No test covers the rule that Core does not list the gateway for Grok Bot and Muse. ## Follow-up The direct path is now unreachable. #21311 deletes it. That pull request is in the same stack, above the pull requests that change the same ACPX host files. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the change, reviewed the diff and ran the verification again. - Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with tool use (shell, file edits, local test runs), investigated each harness and wrote the change and the tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.2 --- doc/execution-semantics.md | 18 ++- .../runner-core/src/acpx_provider_backend.rs | 131 ++++++++++++++++++ .../runner-core/tests/acpx_event_payload.rs | 30 ++++ .../tests/tool_input_schema_fixtures.rs | 94 +++++++++++++ .../src/cli/acpx-runtime-sidecar.test.ts | 57 +++++++- .../src/cli/acpx-runtime-sidecar.ts | 20 +-- .../src/drivers/opencode/mcp-bridge.ts | 10 +- .../runner-tool-bridge.schemas.test.ts | 64 +++++++++ .../src/drivers/runner-tool-bridge.ts | 21 +-- .../runtime-context-materializer.test.ts | 25 ++++ .../src/drivers/tool-input-schema.test.ts | 78 +++++++++++ .../src/drivers/tool-input-schema.ts | 68 +++++++++ packages/paperclip-runner/src/index.ts | 1 + .../fixtures/tool-input-schema-fixtures.json | 93 +++++++++++++ .../native-runtime/assigned-mcp-tools.test.ts | 113 ++++++++++++++- .../native-runtime/assigned-mcp-tools.ts | 73 +++++++++- .../native-session-executor.test.ts | 85 +++++++++++- .../native-runtime/native-session-executor.ts | 29 ++-- .../native-session-resume.test.ts | 13 ++ .../native-runtime/native-session-resume.ts | 4 +- server/src/vendor/paperclip-runner/index.ts | 3 + 21 files changed, 975 insertions(+), 55 deletions(-) create mode 100644 packages/paperclip-runner/runner/crates/runner-core/tests/tool_input_schema_fixtures.rs create mode 100644 packages/paperclip-runner/src/drivers/runner-tool-bridge.schemas.test.ts create mode 100644 packages/paperclip-runner/src/drivers/tool-input-schema.test.ts create mode 100644 packages/paperclip-runner/src/drivers/tool-input-schema.ts create mode 100644 packages/paperclip-runner/test/fixtures/tool-input-schema-fixtures.json diff --git a/doc/execution-semantics.md b/doc/execution-semantics.md index a3913dac9c2..e54d096968f 100644 --- a/doc/execution-semantics.md +++ b/doc/execution-semantics.md @@ -5626,8 +5626,22 @@ on the controller. The MCP bridge preserves exact declared operation names, including names that start with `paperclip_`; only a catalog-bound transport prefix may resolve to that declared name. Shortened or unknown names do not gain authority. OpenCode sessions with the former direct catalog rotate their -provider checkpoint before reuse; other providers retain their existing tool -contract fingerprints. +provider checkpoint before reuse. + +Every native Runner harness, local or remote, receives its assigned gateway +tools through this relay. The controller lists them with the run's gateway +token, offers them as dynamic tools on the Runner's task bridge (the +`paperclip` MCP server for ACPX agents), and executes each call through the +gateway on the controller. The provider process never receives +`PAPERCLIP_NATIVE_MCP_*` or the gateway token. Grok Bot and Muse runs, which +cannot advertise app tools, skip the listing. Relayed names are +`app___`, at most 64 characters, and follow each +harness's existing permission rules for task-bridge tools. Local provider +checkpoints saved with the former direct gateway rotate before reuse, so the +next run starts a fresh provider session with full task context in the same +workspace. Remote checkpoints, which already used the relay for Codex, keep +their fingerprint. A run recovered after its Runner died during the cutover +continues with the tool set it saved and has no app tools until it ends. ### Use real connection requests (2026-09-14) diff --git a/packages/paperclip-runner/runner/crates/runner-core/src/acpx_provider_backend.rs b/packages/paperclip-runner/runner/crates/runner-core/src/acpx_provider_backend.rs index 7fbb581728e..5a27596232e 100644 --- a/packages/paperclip-runner/runner/crates/runner-core/src/acpx_provider_backend.rs +++ b/packages/paperclip-runner/runner/crates/runner-core/src/acpx_provider_backend.rs @@ -4823,6 +4823,137 @@ mod tests { fs::remove_dir_all(directory).unwrap(); } + #[cfg(unix)] + #[test] + fn next_claude_run_attach_offers_relayed_gateway_tools_to_a_direct_gateway_checkpoint() { + next_run_attach_offers_relayed_gateway_tools("claude"); + } + + #[cfg(unix)] + #[test] + fn next_pi_run_attach_offers_relayed_gateway_tools_to_a_direct_gateway_checkpoint() { + next_run_attach_offers_relayed_gateway_tools("pi"); + } + + #[cfg(unix)] + fn next_run_attach_offers_relayed_gateway_tools(agent: &str) { + let directory = temporary_directory(&format!("relayed-gateway-attach-{agent}")); + let runtime = directory.join("runtime"); + let workspace = directory.join("workspace"); + fs::create_dir_all(&runtime).unwrap(); + fs::create_dir_all(&workspace).unwrap(); + fs::set_permissions(&runtime, fs::Permissions::from_mode(0o700)).unwrap(); + fs::set_permissions(&workspace, fs::Permissions::from_mode(0o700)).unwrap(); + let marker = directory.join("provider-started"); + let command = directory.join("sidecar"); + write_artifact( + &command, + format!("#!/bin/sh\ntouch '{}'\n", marker.display()).as_bytes(), + true, + ); + let launch_profile = AcpxLaunchProfile { + candidate_admission: None, + authority_digest: format!("sha256:{}", "d".repeat(64)), + command: command.clone(), + args: Vec::new(), + artifacts: vec![artifact(&command)], + }; + let mut descriptor_value = descriptor(agent); + descriptor_value["sidecarCommand"] = json!(command); + descriptor_value["sidecarArgs"] = json!([]); + descriptor_value["runtimeDirectory"] = json!(runtime); + descriptor_value["cwd"] = json!(workspace); + descriptor_value["runtimeContext"] = json!({ + "aggregateDigest": "a".repeat(64), + "prompt": {"revision": "pinned", "digest": "b".repeat(64)}, + "mcp": {"assignmentSetId": "assignment-1", "digest": "e".repeat(64), "bindingId": "old-run-binding"}, + }); + let saved_descriptor: AcpxProviderDescriptor = + serde_json::from_value(descriptor_value.clone()).unwrap(); + let identity = AcpxProviderSessionIdentity { + kind: "acpx".to_owned(), + normalized_session_id: "session-1".to_owned(), + acpx_record_id: "record-1".to_owned(), + backend_session_id: "backend-1".to_owned(), + agent_session_id: "agent-1".to_owned(), + profile_digest: saved_descriptor.command_digest.clone(), + workspace_digest: format!("sha256:{}", "a".repeat(64)), + requested_model: saved_descriptor.model.clone(), + effective_model: saved_descriptor.model.clone(), + permission_mode: Some(saved_descriptor.permission_mode), + mode: saved_descriptor.mode.clone(), + pi_thinking_level: saved_descriptor.pi_thinking_level, + claude_effort: saved_descriptor.claude_effort.clone(), + grok_effort: saved_descriptor.grok_effort.clone(), + grok_disable_web_search: saved_descriptor.grok_disable_web_search.clone(), + provider_lifetime_fence_candidates: [60_001, 60_002, 60_003], + }; + let tool = |operation_id: &str| crate::provider_bridge::AuthorizedTool { + operation_id: operation_id.to_owned(), + version: 1, + description: format!("{operation_id} fixture"), + input_schema: json!({"type": "object"}), + response_schema: json!({"type": "object"}), + }; + let tool_set = + |operations: Vec| AuthorizedToolSet { + schema: TOOL_SET_SCHEMA.to_owned(), + schema_version: 1, + catalog_digest: authorized_tool_catalog_digest(&operations).unwrap(), + operations, + }; + let direct_gateway_tools = tool_set(vec![tool("get_task_context")]); + let relayed_tools = tool_set(vec![ + tool("app_linear_create_issue_0123456789ab"), + tool("get_task_context"), + ]); + let mut state = AcpxDurableState::new( + saved_descriptor, + direct_gateway_tools, + launch_profile.canonical_digest().unwrap(), + ); + state.lifecycle = "session_open".to_owned(); + state.identity = Some(identity); + let saved_config = test_config(&directory, Some(launch_profile.clone())); + let mut saved = AcpxCommandExecutor::with_runner_config(&directory, &saved_config); + saved.state = Some(state); + saved.save_state().unwrap(); + + let mut next_config = saved_config.clone(); + next_config.run_id = "run-2".to_owned(); + let mut next = AcpxCommandExecutor::with_runner_config(&directory, &next_config); + next.restore().unwrap(); + descriptor_value["runId"] = json!("run-2"); + descriptor_value["runtimeContext"]["mcp"]["bindingId"] = json!("new-run-binding"); + next.attach_run(&json!({ + "provider": descriptor_value, + "authorizedTools": relayed_tools, + })) + .unwrap(); + assert!(!marker.exists()); + let attached = next.state.as_ref().unwrap(); + assert_eq!(attached.descriptor.run_id, "run-2"); + assert_eq!(attached.lifecycle, "suspended"); + assert_eq!(attached.tool_set, relayed_tools); + let persisted: AcpxDurableState = + serde_json::from_slice(&fs::read(next.state_path()).unwrap()).unwrap(); + assert_eq!(persisted.tool_set, relayed_tools); + let session_config = attached + .descriptor + .session_config( + attached.tool_set.clone(), + attached.identity.clone(), + Some(&launch_profile), + ) + .unwrap(); + assert!(session_config + .tool_set + .operations + .iter() + .any(|operation| operation.operation_id == "app_linear_create_issue_0123456789ab")); + fs::remove_dir_all(directory).unwrap(); + } + #[cfg(unix)] #[test] fn cursor_attachment_rotates_only_authenticated_run_grants() { diff --git a/packages/paperclip-runner/runner/crates/runner-core/tests/acpx_event_payload.rs b/packages/paperclip-runner/runner/crates/runner-core/tests/acpx_event_payload.rs index f14a6244c95..5fb37f45e11 100644 --- a/packages/paperclip-runner/runner/crates/runner-core/tests/acpx_event_payload.rs +++ b/packages/paperclip-runner/runner/crates/runner-core/tests/acpx_event_payload.rs @@ -173,6 +173,36 @@ fn rejects_unclassified_and_malformed_runtime_payloads() { } } +#[test] +fn admits_app_tool_arguments_intact_up_to_the_tool_call_limit() { + let scope = active_scope(); + let content = "abcdefghijklmnopqrstuvwxyz".repeat(200 * 1024 / 26); + let input = json!({"files": [{"path": "site/index.html", "content": content}]}); + let decoded = decode_acpx_event( + &scope, + &event( + GeneratedAcpxSidecarEventType::RuntimeToolCalled, + json!({"callId": "inline-file", "operationId": "app_deploy_create_0123456789ab", "input": input}), + ), + ) + .unwrap(); + assert!( + matches!(&decoded, AcpxEventPayload::ToolCalled {input: decoded, ..} if decoded == &input) + ); + + let oversized = + json!({"files": [{"path": "site/index.html", "content": "x".repeat(300 * 1024)}]}); + let error = decode_acpx_event( + &scope, + &event( + GeneratedAcpxSidecarEventType::RuntimeToolCalled, + json!({"callId": "oversized-file", "operationId": "app_deploy_create_0123456789ab", "input": oversized}), + ), + ) + .unwrap_err(); + assert!(error.to_string().contains("256 KiB")); +} + #[test] fn decodes_tool_and_permission_requests_after_scope_validation() { let scope = active_scope(); diff --git a/packages/paperclip-runner/runner/crates/runner-core/tests/tool_input_schema_fixtures.rs b/packages/paperclip-runner/runner/crates/runner-core/tests/tool_input_schema_fixtures.rs new file mode 100644 index 00000000000..e939e0b4063 --- /dev/null +++ b/packages/paperclip-runner/runner/crates/runner-core/tests/tool_input_schema_fixtures.rs @@ -0,0 +1,94 @@ +use paperclip_runner_core::provider_bridge::{ + authorized_tool_catalog_digest, AuthorizedTool, AuthorizedToolSet, ProviderToolBridge, + TOOL_SET_SCHEMA, +}; +use serde_json::{json, Value}; + +const FIXTURES: &str = include_str!("../../../../test/fixtures/tool-input-schema-fixtures.json"); + +fn accepted_cases() -> Vec { + let fixtures: Value = serde_json::from_str(FIXTURES).unwrap(); + fixtures["cases"] + .as_array() + .unwrap() + .iter() + .filter(|case| case["accepted"] == json!(true)) + .cloned() + .collect() +} + +fn admitted_schema(case: &Value) -> Value { + let mut schema = case["schema"].clone(); + if let Some(dialect) = case.get("dialectUri") { + schema["$schema"] = dialect.clone(); + } + schema +} + +fn tool_set(cases: &[Value]) -> AuthorizedToolSet { + let operations = cases + .iter() + .map(|case| AuthorizedTool { + operation_id: format!("app_{}", case["name"].as_str().unwrap()), + version: 1, + description: "Fixture app tool.".to_owned(), + input_schema: admitted_schema(case), + response_schema: json!({"type": "object"}), + }) + .collect::>(); + AuthorizedToolSet { + schema: TOOL_SET_SCHEMA.to_owned(), + schema_version: 1, + catalog_digest: authorized_tool_catalog_digest(&operations).unwrap(), + operations, + } +} + +#[test] +fn every_schema_the_controller_admits_compiles_in_one_tool_set() { + let cases = accepted_cases(); + assert!(cases.len() >= 5); + ProviderToolBridge::default() + .prepare(tool_set(&cases)) + .unwrap(); +} + +#[test] +fn schemas_the_controller_finds_uncompilable_are_refused_here_too() { + let fixtures: Value = serde_json::from_str(FIXTURES).unwrap(); + let refused = fixtures["cases"] + .as_array() + .unwrap() + .iter() + .filter(|case| case["reason"] == json!("uncompilable")) + .cloned() + .collect::>(); + assert!(!refused.is_empty()); + for case in &refused { + assert!( + ProviderToolBridge::default() + .prepare(tool_set(std::slice::from_ref(case))) + .is_err(), + "{}", + case["name"] + ); + } +} + +#[test] +fn begin_call_agrees_with_the_shared_argument_fixtures() { + let cases = accepted_cases(); + for case in &cases { + let operation_id = format!("app_{}", case["name"].as_str().unwrap()); + for (expected, key) in [(true, "valid"), (false, "invalid")] { + for (index, input) in case[key].as_array().unwrap().iter().enumerate() { + let mut bridge = ProviderToolBridge::default(); + bridge.prepare(tool_set(&cases)).unwrap(); + let admitted = bridge + .begin_call(format!("call-{index}"), operation_id.clone(), input.clone()) + .is_ok(); + assert_eq!(admitted, expected, "{operation_id} {key} {input}"); + } + } + } +} diff --git a/packages/paperclip-runner/src/cli/acpx-runtime-sidecar.test.ts b/packages/paperclip-runner/src/cli/acpx-runtime-sidecar.test.ts index c24f274c567..6a7be569b2b 100644 --- a/packages/paperclip-runner/src/cli/acpx-runtime-sidecar.test.ts +++ b/packages/paperclip-runner/src/cli/acpx-runtime-sidecar.test.ts @@ -1116,6 +1116,60 @@ describe("qualified ACPX runtime sidecar", () => { } finally { await sidecar.close(); rmSync(root,{recursive:true,force:true}); } }, 15000); + it("forwards large app-tool arguments intact and refuses only a call over the tool-call limit", async () => { + const root = mkdtempSync(join(tmpdir(), "acpx-large-arguments-")); + const loader = join(root, "loader.mjs"), fixture = join(root, "host.mjs"); + const base = new URL("../test-support/acpx-blocked-output-host.mjs", import.meta.url).href; + writeFileSync(fixture, `import { AcpxRuntimeHost as Base } from ${JSON.stringify(base)}; + import { appendFileSync } from "node:fs"; + import { join } from "node:path"; + export class AcpxRuntimeHost extends Base { + static async open(options) { const host = await Base.open(options); Object.setPrototypeOf(host, this.prototype); return host; } + startTurn({ requestId }) { + for (const [callId, bytes] of [["inline-file", 200 * 1024], ["oversized-file", 300 * 1024]]) { + const content = Array.from({ length: bytes }, (_, index) => String.fromCharCode(97 + index % 26)).join(""); + this.options.semanticTools.handler({callId, tool:"app_deploy_create_0123456789ab", arguments:{files:[{path:"site/index.html", content}]}, signal:new AbortController().signal}) + .then(() => appendFileSync(join(process.env.PAPERCLIP_TEST_OUTPUT_ROOT,"outcomes"), callId+":resolved\\n"), + error => appendFileSync(join(process.env.PAPERCLIP_TEST_OUTPUT_ROOT,"outcomes"), callId+":"+error.message+"\\n")); + } + this.options.semanticTools.handler({callId:"lone-surrogate", tool:"app_deploy_create_0123456789ab", arguments:{note:"half \\ud83d emoji"}, signal:new AbortController().signal}) + .then(() => undefined, error => appendFileSync(join(process.env.PAPERCLIP_TEST_OUTPUT_ROOT,"outcomes"), "lone-surrogate:"+error.message+"\\n")); + return {requestId, promptStarted:Promise.resolve(), result:this.result, events:(async function*(){})()}; + } + }`); + writeFileSync(loader, `import { registerHooks } from "node:module"; import { pathToFileURL } from "node:url"; + registerHooks({resolve(specifier,context,next){ + if(context.parentURL?.includes("/acpx-runtime-sidecar.") && specifier.endsWith("/runtime-host.js")) + return {url:pathToFileURL(${JSON.stringify(fixture)}).href,shortCircuit:true}; + return next(specifier,context); + }});`); + const sidecar = new SidecarProcess({loader, environment:{PAPERCLIP_TEST_OUTPUT_ROOT:root}}); + children.add(sidecar); + const send = async (id:number, command:string, params:Record) => { + sidecar.write({protocolVersion:ACPX_SIDECAR_PROTOCOL_VERSION,id,command,params}); + expect(await sidecar.next(frame=>frame.id === id)).toMatchObject({ok:true}); + }; + const expected = Array.from({ length: 200 * 1024 }, (_, index) => String.fromCharCode(97 + index % 26)).join(""); + try { + sidecar.write(initializeRequest(1,"codex")); + expect(await sidecar.next(frame=>frame.id === 1)).toMatchObject({ok:true}); + await send(2,"session.open",{agent:"codex",model:"gpt-5.6-sol",runtimeDirectory:root,workingDirectory:root, + normalizedSessionId:"large-arguments-session",permissionMode:"deny-all",systemInstructions:"Test large arguments.",tools:[]}); + await send(3,"run.attach",{runId:"large-arguments-run",catalogRevision:1,tools:[]}); + await send(4,"turn.start",{turnId:"large-arguments-turn",message:"Deploy the site."}); + const called = await sidecar.next(frame=>frame.eventType === "runtime.tool_called"); + expect(called.payload).toEqual({callId:"inline-file",operationId:"app_deploy_create_0123456789ab", + input:{files:[{path:"site/index.html",content:expected}]}}); + await vi.waitFor(() => expect(readFileSync(join(root,"outcomes"),"utf8")) + .toContain("oversized-file:ACPX tool arguments exceed the 256 KiB tool-call limit")); + const replaced = await sidecar.next(frame=>frame.eventType === "runtime.tool_called"); + expect(replaced.payload).toEqual({callId:"lone-surrogate",operationId:"app_deploy_create_0123456789ab",input:{note:"half \uFFFD emoji"}}); + expect(sidecar.pending(frame=>frame.eventType === "runtime.tool_called")).toBe(0); + await send(5,"tool.resolve",{turnId:"large-arguments-turn",callId:"inline-file",result:{deployed:true}}); + await vi.waitFor(() => expect(readFileSync(join(root,"outcomes"),"utf8")).toContain("inline-file:resolved")); + } finally { await sidecar.close(); rmSync(root,{recursive:true,force:true}); } + }, 15000); + it("finishes cleanup when the controller closed the output pipe before shutdown", async () => { const root = mkdtempSync(join(tmpdir(), "acpx-shutdown-")); const loader = join(root, "loader.mjs"), fixture = join(root, "host.mjs"); @@ -2023,7 +2077,7 @@ function loadWaitForTool(input: { const factory = new Function( "boundedIdentity", "tools", "turnId", "emit", "PRP_COMPLETION_TOOL_NAME", "PRP_BLOCK_TOOL_NAME", "validatePrpStructuredRunResult", "boundedSidecarValue", "record", "MAX_PENDING_TOOLS", "stringifyAcpxSidecarFrame", - "resolveCompletionContinuation", "runId", "normalizingTools", "randomUUID", "callIdNamespace", + "resolveCompletionContinuation", "runId", "normalizingTools", "randomUUID", "callIdNamespace", "MAX_EVENT_PAYLOAD_BYTES", `return (${functionSource});`, ); return factory( @@ -2046,6 +2100,7 @@ function loadWaitForTool(input: { new Set(), randomUUID, input.callIdNamespace ?? null, + 256 * 1024, ); } diff --git a/packages/paperclip-runner/src/cli/acpx-runtime-sidecar.ts b/packages/paperclip-runner/src/cli/acpx-runtime-sidecar.ts index b850c3a1284..c0e8e6fd4ef 100644 --- a/packages/paperclip-runner/src/cli/acpx-runtime-sidecar.ts +++ b/packages/paperclip-runner/src/cli/acpx-runtime-sidecar.ts @@ -938,15 +938,17 @@ async function waitForTool(call: RunnerToolCall): Promise { if (call.signal.aborted) abort(); }); } - emit( - "runtime.tool_called", - { - callId, - operationId, - input: boundedSidecarValue(record(call.arguments)), - }, - activeTurnId, - ); + const toolCall = { callId, operationId, input: record(call.arguments) }; + let encodedToolCall: string; + try { encodedToolCall = stringifyAcpxSidecarFrame(toolCall); } catch { + throw new Error("ACPX tool arguments cannot be encoded for the Runner"); + } + if (Buffer.byteLength(encodedToolCall) > MAX_EVENT_PAYLOAD_BYTES) { + throw new Error(`ACPX tool arguments exceed the ${MAX_EVENT_PAYLOAD_BYTES / 1024} KiB tool-call limit`); + } + if (!emit("runtime.tool_called", toolCall, activeTurnId)) { + throw new Error("ACPX tool call exceeds the sidecar frame limit"); + } return await new Promise((settle, reject) => { const abort = () => { const pending = tools.get(callId); diff --git a/packages/paperclip-runner/src/drivers/opencode/mcp-bridge.ts b/packages/paperclip-runner/src/drivers/opencode/mcp-bridge.ts index 293788a626b..cef4e5d88bc 100644 --- a/packages/paperclip-runner/src/drivers/opencode/mcp-bridge.ts +++ b/packages/paperclip-runner/src/drivers/opencode/mcp-bridge.ts @@ -1,7 +1,8 @@ import { createHash, randomBytes, timingSafeEqual } from "node:crypto"; import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http"; import type { Socket } from "node:net"; -import { Ajv2020, type ValidateFunction } from "ajv/dist/2020.js"; +import type { ValidateFunction } from "ajv/dist/2020.js"; +import { admitToolInputSchema } from "../tool-input-schema.js"; import { toolInputError } from "../tool-input-error.js"; import { @@ -216,8 +217,11 @@ function normalizeTools( } function compileValidators(tools: readonly OpenCodeMcpToolDefinition[]): Map { - const ajv = new Ajv2020({ allErrors: true, strict: false }); - return new Map(tools.map((tool) => [tool.name, ajv.compile(tool.inputSchema)])); + return new Map(tools.map((tool) => { + const admission = admitToolInputSchema(tool.inputSchema); + if (!admission.ok) throw new Error(`OpenCode tool ${tool.name} has an invalid input schema`); + return [tool.name, admission.validate]; + })); } async function handleRequest( diff --git a/packages/paperclip-runner/src/drivers/runner-tool-bridge.schemas.test.ts b/packages/paperclip-runner/src/drivers/runner-tool-bridge.schemas.test.ts new file mode 100644 index 00000000000..c5df2413d92 --- /dev/null +++ b/packages/paperclip-runner/src/drivers/runner-tool-bridge.schemas.test.ts @@ -0,0 +1,64 @@ +import { readFileSync } from "node:fs"; +import { afterEach, expect, it } from "vitest"; +import { startRunnerToolBridge, type RunnerToolBridge } from "./runner-tool-bridge.js"; + +type FixtureCase = { name: string; accepted: boolean; schema: Record; valid?: unknown[]; invalid?: unknown[] }; +const fixtures = JSON.parse(readFileSync(new URL("../../test/fixtures/tool-input-schema-fixtures.json", import.meta.url), "utf8")) as { cases: FixtureCase[] }; +const bridges: RunnerToolBridge[] = []; +afterEach(async () => { await Promise.all(bridges.splice(0).map(bridge => bridge.close())); }); + +async function rpc(bridge: RunnerToolBridge, body: Record) { + const response = await fetch(bridge.url, { + method: "POST", + headers: { Authorization: `Bearer ${bridge.secret}`, "Content-Type": "application/json" }, + body: JSON.stringify({ jsonrpc: "2.0", ...body }), + }); + return await response.json() as { result: { tools?: Array<{ name: string }>; isError?: boolean } }; +} + +it("starts with every app schema dialect, reused $id values and an uncompilable schema, validating each tool on its own", async () => { + const calls: string[] = []; + const bridge = await startRunnerToolBridge({ + preserveToolNames: true, + tools: [ + { name: "get_task_context", inputSchema: { type: "object", properties: {}, additionalProperties: false } }, + ...fixtures.cases.map(fixture => ({ name: `app_${fixture.name}`, description: fixture.name, inputSchema: fixture.schema })), + ], + handler: async call => { calls.push(call.tool); return { ok: true }; }, + }); + bridges.push(bridge); + const listed = (await rpc(bridge, { id: 1, method: "tools/list" })).result.tools!.map(tool => tool.name); + expect(listed).toEqual([ + "get_task_context", + ...fixtures.cases.filter(fixture => fixture.accepted).map(fixture => `app_${fixture.name}`), + "paperclip_finish", + "paperclip_block", + ]); + let id = 2; + for (const fixture of fixtures.cases) { + const name = `app_${fixture.name}`; + if (!fixture.accepted) { + expect((await rpc(bridge, { id: id++, method: "tools/call", params: { name, arguments: {} } })).result.isError).toBe(true); + continue; + } + for (const input of fixture.valid ?? []) { + expect((await rpc(bridge, { id: id++, method: "tools/call", params: { name, arguments: input } })).result.isError).not.toBe(true); + } + for (const input of fixture.invalid ?? []) { + expect((await rpc(bridge, { id: id++, method: "tools/call", params: { name, arguments: input } })).result.isError).toBe(true); + } + } + expect(calls.filter(name => !fixtures.cases.some(fixture => fixture.accepted && `app_${fixture.name}` === name))).toEqual([]); + expect((await rpc(bridge, { id: id++, method: "tools/call", params: { name: "get_task_context", arguments: { unexpected: true } } })).result.isError).toBe(true); + expect((await rpc(bridge, { id: id++, method: "tools/call", params: { name: "paperclip_finish", arguments: { summary: 1 } } })).result.isError).toBe(true); + expect(calls).not.toContain("get_task_context"); + expect(calls).not.toContain("paperclip_finish"); +}); + +it("refuses a Paperclip-owned private tool whose schema cannot compile", async () => { + await expect(startRunnerToolBridge({ + tools: [], + privateTools: [{ name: "__paperclip_permission", inputSchema: { type: "object", properties: { a: { type: "text" } } } }], + handler: async () => ({ ok: true }), + })).rejects.toThrow("has an invalid input schema"); +}); diff --git a/packages/paperclip-runner/src/drivers/runner-tool-bridge.ts b/packages/paperclip-runner/src/drivers/runner-tool-bridge.ts index b67b76b36e0..b8022158d77 100644 --- a/packages/paperclip-runner/src/drivers/runner-tool-bridge.ts +++ b/packages/paperclip-runner/src/drivers/runner-tool-bridge.ts @@ -7,7 +7,8 @@ import { type ServerResponse, } from "node:http"; -import { Ajv2020, type ValidateFunction } from "ajv/dist/2020.js"; +import type { ValidateFunction } from "ajv/dist/2020.js"; +import { admitToolInputSchema } from "./tool-input-schema.js"; import { toolInputError } from "./tool-input-error.js"; import { @@ -129,7 +130,8 @@ export async function startRunnerToolBridge( ); } } - const visibleTools = [...tools, ...structuredClone(RESERVED_TOOLS)]; + const validators = compileValidators(tools, [...structuredClone(RESERVED_TOOLS), ...privateTools]); + const visibleTools = [...tools.filter((tool) => validators.has(tool.name)), ...structuredClone(RESERVED_TOOLS)]; if (options.preloadClaudeTaskTools) { for (const tool of visibleTools) { if (tool.name === "set_task_title" || tool.name === PRP_COMPLETION_TOOL_NAME || tool.name === PRP_BLOCK_TOOL_NAME) { @@ -138,7 +140,6 @@ export async function startRunnerToolBridge( } } const admittedTools = [...visibleTools, ...privateTools]; - const validators = compileValidators(admittedTools); const calls = new Map(); const controllers = new Map(); const context = { @@ -502,12 +503,16 @@ function normalizeTools( } function compileValidators( - tools: readonly RunnerToolDefinition[], + offered: readonly RunnerToolDefinition[], + required: readonly RunnerToolDefinition[], ): Map { - const ajv = new Ajv2020({ allErrors: true, strict: false }); - return new Map( - tools.map((tool) => [tool.name, ajv.compile(tool.inputSchema)]), - ); + const validators = new Map(); + for (const tool of [...offered, ...required]) { + const admission = admitToolInputSchema(tool.inputSchema); + if (admission.ok) validators.set(tool.name, admission.validate); + else if (required.includes(tool)) throw new Error(`Runner tool ${tool.name} has an invalid input schema`); + } + return validators; } function authorized(value: string | undefined, secret: string): boolean { diff --git a/packages/paperclip-runner/src/drivers/runtime-context-materializer.test.ts b/packages/paperclip-runner/src/drivers/runtime-context-materializer.test.ts index 8f6cc5a169e..c5ab6c9c433 100644 --- a/packages/paperclip-runner/src/drivers/runtime-context-materializer.test.ts +++ b/packages/paperclip-runner/src/drivers/runtime-context-materializer.test.ts @@ -549,4 +549,29 @@ requires_openai_auth = false expect((await stat(join(codexHome, "auth.json"))).mode & 0o777).toBe(0o600); } else await expect(stat(join(codexHome, "auth.json"))).rejects.toMatchObject({ code: "ENOENT" }); }); + + it.each(["next_run", "same_run_recovery"] as const)("drops a retained direct gateway server when the relayed launch has no binding (%s)", async mode => { + const root = await mkdtemp(join(tmpdir(), "paperclip-relayed-codex-home-")); + roots.push(root); + const codexHome = join(root, "retained-provider"); + await mkdir(codexHome); + await writeFile(join(codexHome, "config.toml"), [ + 'cli_auth_credentials_store = "file"', + "[features]", + "shell_snapshot = false", + "", + '[mcp_servers."paperclip-assigned"]', + 'url = "http://127.0.0.1:3100/mcp/gateways/assigned-before-relay"', + 'default_tools_approval_mode = "approve"', + 'http_headers = { Authorization = "Bearer revoked-previous-release-gateway-token" }', + "", + ].join("\n")); + const nativeMcp = nativeMcpLaunchBinding({}); + if (mode === "next_run") await prepareIsolatedCodexHome({ context: null, codexHome, nativeMcp }); + else await prepareIsolatedCodexRecoveryHome({ codexHome, nativeMcp }); + const config = await readFile(join(codexHome, "config.toml"), "utf8"); + expect(parse(config).mcp_servers).toBeUndefined(); + expect(config).not.toContain("revoked-previous-release-gateway-token"); + expect(config).not.toContain("assigned-before-relay"); + }); }); diff --git a/packages/paperclip-runner/src/drivers/tool-input-schema.test.ts b/packages/paperclip-runner/src/drivers/tool-input-schema.test.ts new file mode 100644 index 00000000000..2b513d258d3 --- /dev/null +++ b/packages/paperclip-runner/src/drivers/tool-input-schema.test.ts @@ -0,0 +1,78 @@ +import { readFileSync } from "node:fs"; +import { Ajv } from "ajv"; +import { Ajv2019 } from "ajv/dist/2019.js"; +import { Ajv2020 } from "ajv/dist/2020.js"; +import { describe, expect, it, vi } from "vitest"; +import { admitToolInputSchema, TOOL_INPUT_SCHEMA_MAX_BYTES, toolInputSchemaBytes } from "./tool-input-schema.js"; + +type FixtureCase = { + name: string; + accepted: boolean; + reason?: string; + dialectUri?: string; + schema: Record; + valid?: unknown[]; + invalid?: unknown[]; +}; + +const fixtures = JSON.parse(readFileSync(new URL("../../test/fixtures/tool-input-schema-fixtures.json", import.meta.url), "utf8")) as { cases: FixtureCase[] }; + +describe("tool input schema admission", () => { + it.each([Ajv, Ajv2019, Ajv2020].map(compiler => [compiler.name, compiler] as const))("refuses a valid schema over the Runner byte limit before %s compiles it", (_name, compiler) => { + const compile = vi.spyOn(compiler.prototype, "compile"); + try { + const $schema = compiler === Ajv ? "http://json-schema.org/draft-07/schema#" + : compiler === Ajv2019 ? "https://json-schema.org/draft/2019-09/schema" : "https://json-schema.org/draft/2020-12/schema"; + const properties = Object.fromEntries(Array.from({ length: 6_000 }, (_, index) => [`field_${index}`, { type: "string", description: "x".repeat(64) }])); + const schema = { $schema, type: "object", properties }; + expect(toolInputSchemaBytes(schema)).toBeGreaterThan(TOOL_INPUT_SCHEMA_MAX_BYTES); + expect(admitToolInputSchema(schema)).toEqual({ ok: false, reason: "oversized" }); + expect(compile).not.toHaveBeenCalled(); + expect(admitToolInputSchema({ $schema, type: "object", properties: { name: { type: "string" } } }).ok).toBe(true); + expect(compile).toHaveBeenCalledOnce(); + } finally { + compile.mockRestore(); + } + }); + + it.each(fixtures.cases.map(fixture => [fixture.name, fixture] as const))("matches the shared fixture for %s", (_name, fixture) => { + const admission = admitToolInputSchema(fixture.schema); + if (!fixture.accepted) { + expect(admission).toEqual({ ok: false, reason: fixture.reason }); + return; + } + if (!admission.ok) throw new Error(`expected ${fixture.name} to compile`); + expect(admission.schema.$schema).toBe(fixture.dialectUri); + for (const input of fixture.valid ?? []) expect(admission.validate(input)).toBe(true); + for (const input of fixture.invalid ?? []) expect(admission.validate(input)).toBe(false); + }); + + it("compiles every accepted schema independently, including a reused $id", () => { + const accepted = fixtures.cases.filter(fixture => fixture.accepted); + const first = accepted.map(fixture => admitToolInputSchema(fixture.schema)); + const second = accepted.map(fixture => admitToolInputSchema(fixture.schema)); + expect([...first, ...second].every(admission => admission.ok)).toBe(true); + }); + + const dialects = [ + "http://json-schema.org/draft-07/schema#", + "https://json-schema.org/draft/2019-09/schema", + "https://json-schema.org/draft/2020-12/schema", + ]; + + it.each([ + ...dialects, + "http://json-schema.org/draft-07/schema", + "https://json-schema.org/draft/2020-12/schema#", + "https://json-schema.org/draft/2019-09/meta/core", + "https://json-schema.org/draft/2020-12/meta/core", + ])("keeps every compiler usable after a schema that claims the reserved $id %s", reserved => { + for (const $schema of dialects) admitToolInputSchema({ $schema, $id: reserved, type: "object" }); + for (const $schema of dialects) { + const admission = admitToolInputSchema({ $schema, type: "object", properties: { name: { type: "string" } }, required: ["name"] }); + if (!admission.ok) throw new Error(`expected ${$schema} to compile after ${reserved}`); + expect(admission.validate({ name: "site" })).toBe(true); + expect(admission.validate({})).toBe(false); + } + }); +}); diff --git a/packages/paperclip-runner/src/drivers/tool-input-schema.ts b/packages/paperclip-runner/src/drivers/tool-input-schema.ts new file mode 100644 index 00000000000..f37362b9e22 --- /dev/null +++ b/packages/paperclip-runner/src/drivers/tool-input-schema.ts @@ -0,0 +1,68 @@ +import { Ajv } from "ajv"; +import { Ajv2019 } from "ajv/dist/2019.js"; +import { Ajv2020, type ValidateFunction } from "ajv/dist/2020.js"; + +export type ToolInputSchemaDialect = "draft-07" | "2019-09" | "2020-12"; + +export type ToolInputSchemaAdmission = + | { ok: true; dialect: ToolInputSchemaDialect; schema: Record; validate: ValidateFunction } + | { ok: false; reason: "oversized" | "unsupported_dialect" | "uncompilable" }; + +export const TOOL_INPUT_SCHEMA_MAX_BYTES = 512 * 1024; + +export function toolInputSchemaBytes(schema: Readonly>): number { + return Buffer.byteLength(JSON.stringify(schema), "utf8"); +} + +const CANONICAL_DIALECTS: Record = { + "draft-07": "http://json-schema.org/draft-07/schema#", + "2019-09": "https://json-schema.org/draft/2019-09/schema", + "2020-12": "https://json-schema.org/draft/2020-12/schema", +}; + +const compilers: Partial> = {}; + +function compiler(dialect: ToolInputSchemaDialect) { + const options = { allErrors: true, strict: false, addUsedSchema: false }; + return compilers[dialect] ??= dialect === "draft-07" ? new Ajv(options) + : dialect === "2019-09" ? new Ajv2019(options) : new Ajv2020(options); +} + +export function toolInputSchemaDialect(value: unknown): ToolInputSchemaDialect | null { + if (value === undefined) return "2020-12"; + if (typeof value !== "string") return null; + const match = /^https?:\/\/json-schema\.org\/(draft-07\/schema|draft\/2019-09\/schema|draft\/2020-12\/schema)#?$/.exec(value); + if (!match) return null; + return match[1] === "draft-07/schema" ? "draft-07" : match[1] === "draft/2019-09/schema" ? "2019-09" : "2020-12"; +} + +export function canonicalToolInputSchema( + schema: Readonly>, +): { dialect: ToolInputSchemaDialect; schema: Record } | null { + const dialect = toolInputSchemaDialect(schema.$schema); + if (dialect === null) return null; + const normalized = structuredClone(schema) as Record; + if (normalized.$schema !== undefined) normalized.$schema = CANONICAL_DIALECTS[dialect]; + return { dialect, schema: normalized }; +} + +export function admitToolInputSchema(schema: Readonly>): ToolInputSchemaAdmission { + if (toolInputSchemaBytes(schema) > TOOL_INPUT_SCHEMA_MAX_BYTES) return { ok: false, reason: "oversized" }; + const canonical = canonicalToolInputSchema(schema); + if (canonical === null) return { ok: false, reason: "unsupported_dialect" }; + const { dialect, schema: normalized } = canonical; + const ajv = compiler(dialect); + const compiled = structuredClone(normalized); + try { + return { ok: true, dialect, schema: normalized, validate: ajv.compile(compiled) }; + } catch { + return { ok: false, reason: "uncompilable" }; + } finally { + forgetCompiledSchema(ajv, compiled); + } +} + +function forgetCompiledSchema(ajv: Ajv | Ajv2019 | Ajv2020, compiled: Record): void { + delete compiled.$id; + ajv.removeSchema(compiled); +} diff --git a/packages/paperclip-runner/src/index.ts b/packages/paperclip-runner/src/index.ts index fcd60bbfcf9..52ece3d2a7f 100644 --- a/packages/paperclip-runner/src/index.ts +++ b/packages/paperclip-runner/src/index.ts @@ -73,6 +73,7 @@ export * from "./drivers/codex/app-server-transport.js"; export * from "./drivers/codex/codex-app-server-driver.js"; export { codexUsageBaselineForNextRun } from "./drivers/codex/codex-usage-baseline.js"; export { resolveCodexCommand, resolvePinnedCodexCommand } from "./drivers/codex/codex-command.js"; +export { admitToolInputSchema, canonicalToolInputSchema, TOOL_INPUT_SCHEMA_MAX_BYTES, type ToolInputSchemaAdmission } from "./drivers/tool-input-schema.js"; export { runnerBinaryTarget } from "./live/runner-binary.js"; export * from "./drivers/opencode/opencode-server-driver.js"; export * from "./drivers/opencode/mcp-bridge.js"; diff --git a/packages/paperclip-runner/test/fixtures/tool-input-schema-fixtures.json b/packages/paperclip-runner/test/fixtures/tool-input-schema-fixtures.json new file mode 100644 index 00000000000..ddaeb5bcb54 --- /dev/null +++ b/packages/paperclip-runner/test/fixtures/tool-input-schema-fixtures.json @@ -0,0 +1,93 @@ +{ + "schema": "paperclip.runner.tool-input-schema-fixtures.v1", + "cases": [ + { + "name": "implicit_2020_12", + "accepted": true, + "schema": {"type": "object", "properties": {"title": {"type": "string"}}, "required": ["title"], "additionalProperties": false}, + "valid": [{"title": "Ship it"}], + "invalid": [{}, {"title": 1}, {"title": "Ship it", "extra": true}] + }, + { + "name": "draft_07_with_fragment_and_id", + "accepted": true, + "dialectUri": "http://json-schema.org/draft-07/schema#", + "schema": { + "$schema": "http://json-schema.org/draft-07/schema#", + "$id": "https://apps.example/deploy/create.json", + "type": "object", + "definitions": {"file": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}, + "properties": {"files": {"type": "array", "items": {"$ref": "#/definitions/file"}}, "pair": {"type": "array", "items": [{"type": "string"}, {"type": "integer"}], "additionalItems": false}}, + "required": ["files"] + }, + "valid": [{"files": [{"path": "a.txt", "content": "x"}]}, {"files": [], "pair": ["a", 1]}], + "invalid": [{"files": [{"path": "a.txt"}]}, {"files": [], "pair": ["a", "b"]}, {"files": [], "pair": ["a", 1, 2]}] + }, + { + "name": "draft_07_same_id_second_connection", + "accepted": true, + "dialectUri": "http://json-schema.org/draft-07/schema#", + "schema": { + "$schema": "http://json-schema.org/draft-07/schema", + "$id": "https://apps.example/deploy/create.json", + "type": "object", + "properties": {"name": {"type": "string", "minLength": 2}}, + "required": ["name"] + }, + "valid": [{"name": "ok"}], + "invalid": [{"name": "x"}, {}] + }, + { + "name": "draft_2019_09_defs", + "accepted": true, + "dialectUri": "https://json-schema.org/draft/2019-09/schema", + "schema": { + "$schema": "https://json-schema.org/draft/2019-09/schema", + "type": "object", + "$defs": {"id": {"type": "string", "pattern": "^[a-z]+-[0-9]+$"}}, + "properties": {"issue": {"$ref": "#/$defs/id"}}, + "required": ["issue"], + "dependentRequired": {"assignee": ["issue"]} + }, + "valid": [{"issue": "eng-12"}, {"issue": "eng-12", "assignee": "a"}], + "invalid": [{"issue": "ENG"}, {}] + }, + { + "name": "draft_2020_12_explicit_prefix_items", + "accepted": true, + "dialectUri": "https://json-schema.org/draft/2020-12/schema", + "schema": { + "$schema": "https://json-schema.org/draft/2020-12/schema", + "type": "object", + "properties": {"point": {"type": "array", "prefixItems": [{"type": "number"}, {"type": "number"}], "items": false}}, + "required": ["point"] + }, + "valid": [{"point": [1, 2.5]}], + "invalid": [{"point": [1, "2"]}, {"point": [1, 2, 3]}] + }, + { + "name": "draft_04", + "accepted": false, + "reason": "unsupported_dialect", + "schema": {"$schema": "http://json-schema.org/draft-04/schema#", "type": "object"} + }, + { + "name": "unknown_dialect", + "accepted": false, + "reason": "unsupported_dialect", + "schema": {"$schema": "https://apps.example/meta/v1", "type": "object"} + }, + { + "name": "unresolvable_reference", + "accepted": false, + "reason": "uncompilable", + "schema": {"type": "object", "properties": {"a": {"$ref": "#/definitions/missing"}}} + }, + { + "name": "invalid_keyword_value", + "accepted": false, + "reason": "uncompilable", + "schema": {"type": "object", "properties": {"a": {"type": "text"}}} + } + ] +} diff --git a/server/src/services/native-runtime/assigned-mcp-tools.test.ts b/server/src/services/native-runtime/assigned-mcp-tools.test.ts index cfcef3b18e2..aeb63ca7517 100644 --- a/server/src/services/native-runtime/assigned-mcp-tools.test.ts +++ b/server/src/services/native-runtime/assigned-mcp-tools.test.ts @@ -1,9 +1,29 @@ +import { randomUUID } from "node:crypto"; +import { readFileSync } from "node:fs"; import { describe, expect, it, vi } from "vitest"; import type { Db } from "@paperclipai/db"; import { ToolGatewayHttpError, type ToolGatewayDescriptor, type ToolGatewayService } from "../tool-gateway.js"; -import { createAssignedMcpTools, getAssignedMcpGateway, registerAssignedMcpGateway } from "./assigned-mcp-tools.js"; +import { + ASSIGNED_SCHEMA_ADMISSION_CACHE_ENTRIES, ASSIGNED_SCHEMA_LISTING_BUDGET_BYTES, assignedSchemaAdmissionCacheSize, + createAssignedMcpTools, getAssignedMcpGateway, registerAssignedMcpGateway, +} from "./assigned-mcp-tools.js"; +import { admitToolInputSchema } from "../../vendor/paperclip-runner/index.js"; import { startOpenCodeMcpBridge } from "../../vendor/paperclip-runner/testing.js"; +vi.mock("../../vendor/paperclip-runner/index.js", async importOriginal => { + const actual = await importOriginal(); + return { ...actual, admitToolInputSchema: vi.fn(actual.admitToolInputSchema) }; +}); + +function sizedSchema(bytes: number): Record { + const marker = randomUUID(); + return { type: "object", description: `${marker}${"x".repeat(Math.max(0, bytes - 80))}`, properties: { name: { type: "string" } } }; +} + +function compiledDescriptions(): string[] { + return vi.mocked(admitToolInputSchema).mock.calls.map(([schema]) => String(schema.description).slice(0, 36)); +} + function descriptor(name: string, risk: ToolGatewayDescriptor["risk"] = "read"): ToolGatewayDescriptor { return { name, displayName: name, description: `Use ${name}`, parametersSchema: { type: "object", properties: { query: { type: "string" } } }, pluginId: "fixture", providerType: "mcp_remote_http", risk }; } @@ -348,14 +368,17 @@ describe("assigned MCP runner tools", () => { const schema = { type: "object", description: "\u0000🙂".repeat(150_000), properties: {} }; const f = fixture([{ ...descriptor("a_large"), parametersSchema: schema }, descriptor("z_small")]); const assigned = await createAssignedMcpTools(f); - const large = assigned.definitions()[0]!.name as string; const page = await assigned.execute({ tool: searchName, arguments: { query: "", limit: 1 } }) as { tools: Array>; nextOffset: number; }; + const large = page.tools[0]!.name as string; expect(page.tools).toEqual([{ name: large, description: "a_large: Use a_large", inputSchemaRef: large }]); expect(page.nextOffset).toBe(1); - await expect(assigned.execute({ tool: searchName, arguments: { query: "", offset: page.nextOffset } })) - .resolves.toEqual({ tools: [assigned.definitions()[1]], nextOffset: null }); + const rest = await assigned.execute({ tool: searchName, arguments: { query: "", offset: page.nextOffset } }) as { + tools: Array>; nextOffset: number | null; + }; + expect(rest.tools.map(tool => tool.description)).toEqual(["z_small: Use z_small"]); + expect(rest.nextOffset).toBeNull(); let schemaOffset: number | null = 0; let serialized = ""; do { @@ -390,6 +413,88 @@ describe("assigned MCP runner tools", () => { expect(getAssignedMcpGateway(firstDb)).toBe(second); }); + it("never compiles an app schema over the Runner byte limit and serves that tool only through search", async () => { + const large = sizedSchema(600 * 1024), small = sizedSchema(200); + const f = fixture([{ ...descriptor("deploy:large"), parametersSchema: large }, { ...descriptor("deploy:small"), parametersSchema: small }]); + vi.mocked(admitToolInputSchema).mockClear(); + const assigned = await createAssignedMcpTools(f); + expect(assigned.definitions().map(tool => tool.name)).toEqual(["paperclip_search_assigned_tools", "paperclip_call_assigned_tool"]); + expect(compiledDescriptions()).toEqual([String(small.description).slice(0, 36)]); + const page = await assigned.execute({ tool: searchName, arguments: { query: "deploy", limit: 20 } }) as { tools: Array> }; + expect(page.tools.map(tool => tool.description)).toEqual(["deploy:large: Use deploy:large", "deploy:small: Use deploy:small"]); + }); + + it("stops compiling at the listing budget, keeps earlier tools offered, and serves the rest only through search", async () => { + const schemas = [300, 300, 300, 1].map(kib => sizedSchema(kib * 1024)); + const listing = schemas.map((schema, index) => ({ ...descriptor(`deploy:tool_${index}`), parametersSchema: schema })); + expect(schemas.slice(0, 2).reduce((total, schema) => total + Buffer.byteLength(JSON.stringify(schema)), 0)).toBeLessThanOrEqual(ASSIGNED_SCHEMA_LISTING_BUDGET_BYTES); + expect(schemas.slice(0, 3).reduce((total, schema) => total + Buffer.byteLength(JSON.stringify(schema)), 0)).toBeGreaterThan(ASSIGNED_SCHEMA_LISTING_BUDGET_BYTES); + for (const attempt of [1, 2]) { + const f = fixture(listing); + vi.mocked(admitToolInputSchema).mockClear(); + const assigned = await createAssignedMcpTools(f); + expect(compiledDescriptions()).toEqual(attempt === 1 ? schemas.slice(0, 2).map(schema => String(schema.description).slice(0, 36)) : []); + expect(assigned.definitions().map(tool => tool.name)).toEqual(["paperclip_search_assigned_tools", "paperclip_call_assigned_tool"]); + const found: Array> = []; + for (let offset: number | null = 0; offset !== null;) { + const page = await assigned.execute({ tool: searchName, arguments: { query: "deploy", offset, limit: 20 } }) as { + tools: Array>; nextOffset: number | null; + }; + found.push(...page.tools); + offset = page.nextOffset; + } + expect(found.map(tool => tool.description)).toEqual(listing.map(tool => `${tool.name}: Use ${tool.name}`)); + for (const tool of found) await assigned.execute({ tool: callName, arguments: { name: tool.name, arguments: { name: "site" } } }); + expect(f.executeTool.mock.calls.map(([call]) => call.tool)).toEqual(listing.map(tool => tool.name)); + } + }); + + it("compiles a new schema one time when several runs start together", async () => { + const listing = [{ ...descriptor("deploy:shared"), parametersSchema: sizedSchema(400) }]; + vi.mocked(admitToolInputSchema).mockClear(); + const assigned = await Promise.all([1, 2, 3, 4].map(() => createAssignedMcpTools(fixture(listing)))); + expect(admitToolInputSchema).toHaveBeenCalledTimes(1); + for (const tools of assigned) expect(tools.definitions().map(tool => tool.description)).toEqual(["deploy:shared: Use deploy:shared"]); + }); + + it("does not compile an unchanged schema again and keeps the outcome cache bounded", async () => { + const listing = Array.from({ length: ASSIGNED_SCHEMA_ADMISSION_CACHE_ENTRIES + 10 }, (_, index) => + ({ ...descriptor(`bulk:tool_${index}`), parametersSchema: sizedSchema(120) })); + vi.mocked(admitToolInputSchema).mockClear(); + await createAssignedMcpTools(fixture(listing.slice(-5))); + await createAssignedMcpTools(fixture(listing.slice(-5))); + expect(admitToolInputSchema).toHaveBeenCalledTimes(5); + await createAssignedMcpTools(fixture(listing)); + expect(assignedSchemaAdmissionCacheSize()).toBe(ASSIGNED_SCHEMA_ADMISSION_CACHE_ENTRIES); + }, 30_000); + + it.each([undefined, "on_demand"] as const)("offers and executes every app schema the Runner validates and withholds the rest (%s)", async delivery => { + const shared = JSON.parse(readFileSync(new URL("../../../../packages/paperclip-runner/test/fixtures/tool-input-schema-fixtures.json", import.meta.url), "utf8")) as { + cases: Array<{ name: string; accepted: boolean; dialectUri?: string; schema: Record; valid?: Record[] }>; + }; + const descriptors = shared.cases.map(entry => ({ ...descriptor(`deploy:${entry.name}`), parametersSchema: entry.schema }) as ToolGatewayDescriptor); + const f = fixture(descriptors); + const assigned = await createAssignedMcpTools({ ...f, ...(delivery ? { delivery } : {}) }); + const offered = delivery + ? (await assigned.execute({ tool: searchName, arguments: { query: "", limit: 20 } }) as { tools: Array> }).tools + : assigned.definitions(); + expect(offered.map(tool => tool.description)).toEqual(expect.arrayContaining( + shared.cases.filter(entry => entry.accepted).map(entry => expect.stringContaining(entry.name)))); + expect(offered).toHaveLength(shared.cases.filter(entry => entry.accepted).length); + for (const entry of shared.cases) { + const tool = offered.find(candidate => String(candidate.description).endsWith(`deploy:${entry.name}`)); + if (!entry.accepted) { + expect(tool).toBeUndefined(); + continue; + } + expect((tool!.inputSchema as Record).$schema).toBe(entry.dialectUri); + const args = entry.valid![0]!; + await assigned.execute(delivery ? { tool: callName, arguments: { name: tool!.name, arguments: args } } : { tool: String(tool!.name), arguments: args }); + expect(f.executeTool).toHaveBeenLastCalledWith(expect.objectContaining({ tool: `deploy:${entry.name}`, parameters: args })); + } + expect(f.executeTool).toHaveBeenCalledTimes(shared.cases.filter(entry => entry.accepted).length); + }); + it("projects only authorized gateway tools with stable bounded collision-resistant names", async () => { const names = ["memory.search", "memory-search", "x".repeat(120), "🧠"]; const f = fixture(names.map(name => descriptor(name))); diff --git a/server/src/services/native-runtime/assigned-mcp-tools.ts b/server/src/services/native-runtime/assigned-mcp-tools.ts index c7472fa5c4e..057f247f529 100644 --- a/server/src/services/native-runtime/assigned-mcp-tools.ts +++ b/server/src/services/native-runtime/assigned-mcp-tools.ts @@ -1,5 +1,7 @@ import { createHash } from "node:crypto"; import type { Db } from "@paperclipai/db"; +import { logger } from "../../middleware/logger.js"; +import { admitToolInputSchema, canonicalToolInputSchema, TOOL_INPUT_SCHEMA_MAX_BYTES } from "../../vendor/paperclip-runner/index.js"; import { deduplicateToolDescriptors, ToolGatewayHttpError, type ToolGatewayDescriptor, type ToolGatewayService } from "../tool-gateway.js"; type WorkMode = "standard" | "planning" | "ask" | "skill_test"; @@ -45,8 +47,8 @@ function object(value: unknown): Record { return value as Record; } -function definition(name: string, tool: ToolGatewayDescriptor): ToolDefinition { - return { name, description: `${tool.displayName}: ${tool.description}`, inputSchema: structuredClone(tool.parametersSchema) }; +function definition(name: string, tool: ToolGatewayDescriptor, inputSchema: Record): ToolDefinition { + return { name, description: `${tool.displayName}: ${tool.description}`, inputSchema: structuredClone(inputSchema) }; } // Execution must use the app's configured gateway, including deployment @@ -86,6 +88,36 @@ function assignedToolName(name: string): string { return `app_${readable}_${digest}`; } +export const ASSIGNED_SCHEMA_LISTING_BUDGET_BYTES = 768 * 1024; +export const ASSIGNED_SCHEMA_ADMISSION_CACHE_ENTRIES = 1024; + +type AssignedSchemaOutcome = { ok: true } | { ok: false; reason: string }; +type AssignedSchemaAdmission = { ok: true; schema: Record } | { ok: false; reason: string }; +const assignedSchemaOutcomes = new Map(); + +export function assignedSchemaAdmissionCacheSize(): number { + return assignedSchemaOutcomes.size; +} + +async function admitAssignedToolSchema(serialized: string, schema: Record): Promise { + const key = createHash("sha256").update(serialized).digest("hex"); + let outcome = assignedSchemaOutcomes.get(key); + if (!outcome) { + await new Promise(resolve => setImmediate(resolve)); + outcome = assignedSchemaOutcomes.get(key); + } + if (!outcome) { + const admission = admitToolInputSchema(schema); + outcome = admission.ok ? { ok: true } : { ok: false, reason: admission.reason }; + } + assignedSchemaOutcomes.delete(key); + assignedSchemaOutcomes.set(key, outcome); + while (assignedSchemaOutcomes.size > ASSIGNED_SCHEMA_ADMISSION_CACHE_ENTRIES) { + assignedSchemaOutcomes.delete(assignedSchemaOutcomes.keys().next().value!); + } + return outcome.ok ? { ok: true, schema: canonicalToolInputSchema(schema)!.schema } : outcome; +} + /** Server-owned MCP bindings projected into the runner's existing tool channel. */ export async function createAssignedMcpTools(input: { gateway: ToolGatewayService; @@ -100,6 +132,10 @@ export async function createAssignedMcpTools(input: { bearerToken: input.bearerToken, }); const tools = new Map(); + const schemas = new Map>(); + const unavailable: Array<{ tool: string; reason: string }> = []; + const unchecked: string[] = []; + let listingSchemaBytes = 0; for (const descriptor of deduplicateToolDescriptors(listed)) { const name = assignedToolName(descriptor.name); const previous = tools.get(name); @@ -108,7 +144,32 @@ export async function createAssignedMcpTools(input: { `Assigned MCP tool name collision for ${JSON.stringify([previous.name, descriptor.name])}.`, "assigned_mcp_tool_name_collision"); } + const raw = (descriptor.parametersSchema ?? { type: "object", properties: {} }) as Record; + const serialized = JSON.stringify(raw); + const bytes = Buffer.byteLength(serialized, "utf8"); + const compilable = bytes <= TOOL_INPUT_SCHEMA_MAX_BYTES + && (listingSchemaBytes += bytes) <= ASSIGNED_SCHEMA_LISTING_BUDGET_BYTES; + if (!compilable) { + tools.set(name, descriptor); + schemas.set(name, structuredClone(raw)); + unchecked.push(descriptor.name); + continue; + } + const admission = await admitAssignedToolSchema(serialized, raw); + if (!admission.ok) { + unavailable.push({ tool: descriptor.name, reason: admission.reason }); + continue; + } tools.set(name, descriptor); + schemas.set(name, admission.schema); + } + if (unavailable.length > 0) { + logger.warn({ gatewayPublicId: input.gatewayPublicId, unavailable }, + "assigned MCP tools whose input schema the Runner cannot validate are not offered"); + } + if (unchecked.length > 0) { + logger.warn({ gatewayPublicId: input.gatewayPublicId, unchecked }, + "assigned MCP tools whose input schema is over the compile budget are offered only through assigned-tool search"); } const permits = (tool: ToolGatewayDescriptor, mode: WorkMode = input.workMode ?? "standard") => mode === "standard" || tool.risk === "read"; // OpenCode's HTTP bridge returns at most 64K JSON characters. Leave room @@ -147,7 +208,7 @@ export async function createAssignedMcpTools(input: { if (hasSchemaTool) { const selected = authorized.find(([name]) => name === args.schemaTool); if (!selected) throw new Error("assigned_mcp_tool_unknown: schemaTool must match an authorized inputSchemaRef. Omit schemaTool to search assigned tools, then use an exact inputSchemaRef from the results."); - const schema = JSON.stringify(selected[1].parametersSchema); + const schema = JSON.stringify(schemas.get(selected[0])); const end = schemaOffset + schemaChunkCharacters; return { name: selected[0], schemaJson: schema.slice(schemaOffset, end), @@ -168,7 +229,7 @@ export async function createAssignedMcpTools(input: { nextOffset += 1; continue; } - let next = definition(name, tool); + let next = definition(name, tool, schemas.get(name)!); if (Buffer.byteLength(JSON.stringify([next]), "utf8") > searchPageBytes) { // One large schema must not block browsing the tools after it. Expose // a reference whose schema can be fetched in bounded, reauthorized @@ -186,8 +247,8 @@ export async function createAssignedMcpTools(input: { return { definitions(fits?: (definitions: ToolDefinition[]) => boolean): ToolDefinition[] { - const direct = [...tools].filter(([, tool]) => permits(tool)).map(([name, tool]) => definition(name, tool)); - if (input.delivery !== "on_demand" && (!fits || fits(direct))) return direct; + const direct = [...tools].filter(([, tool]) => permits(tool)).map(([name, tool]) => definition(name, tool, schemas.get(name)!)); + if (input.delivery !== "on_demand" && unchecked.length === 0 && (!fits || fits(direct))) return direct; const compact = structuredClone(ON_DEMAND_TOOLS); if (fits && !fits(compact)) throw new Error("assigned_mcp_tool_catalog_capacity_exceeded"); return compact; diff --git a/server/src/services/native-runtime/native-session-executor.test.ts b/server/src/services/native-runtime/native-session-executor.test.ts index fdd1f03c1bf..b8a49c3cb1e 100644 --- a/server/src/services/native-runtime/native-session-executor.test.ts +++ b/server/src/services/native-runtime/native-session-executor.test.ts @@ -179,6 +179,8 @@ type BackendFactoryOptions = { }; }) => unknown; dynamicToolHandler?: (call: unknown) => Promise; + acpxDynamicToolHandler?: (call: unknown) => Promise; + dynamicTools?: unknown[]; completionFeedback?: (result: Parameters[2]) => Promise; onSpawn?: (meta: { pid: number; @@ -17760,6 +17762,58 @@ describe("runnerd provider runtime wiring", () => { || request.args.includes("paperclip-runner-claim-unstarted-session"))).toBe(false); }); + it.each(["claude", "pi"] as const)("relays the assigned gateway when a deploy lands mid-run and the same ACPX %s run reattaches its surviving runner", async agent => { + const base = scopedNativeExecution(`relay-mid-run-${agent}`); + const current = { ...base, provider: { kind: "acpx", agent, model: null }, + session: { ...base.session, driverKind: "acpx_runtime" } } as unknown as NativeExecutionInputV1; + const identity = { runId: current.binding.runId, normalizedSessionId: current.session.normalizedSessionId!, + runnerInstanceId: `admitted-${agent}`, environmentLeaseId: "admitted-lease", + turnId: `turn_attachment_${randomUUID()}`, itemId: `item_attachment_${randomUUID()}` }; + const root = await initializeScopedRunnerRoot(current, identity.runnerInstanceId); + await mkdir(join(root, "control-plane"), { recursive: true }); + const journal = JSON.stringify(durableControlPlaneState(identity)); + await writeFile(join(root, "control-plane", "control-plane-state.json"), journal); + const execute = vi.fn(async (request: { args: string[] }) => ({ exitCode: 0, timedOut: false, stderr: "", + stdout: request.args[1]?.includes("base64") ? Buffer.from(JSON.stringify({ ...identity, lifecycle: "running" })).toString("base64") + : request.args[1]?.includes("cat --") ? `nonce\n123\n2026-09-19T10:00:00.000Z\n${identity.runnerInstanceId}\nlinux:abcd-1234:100\n` : "" })); + const assignedTool = { name: "app_linear_create_issue_0123456789ab", description: "Linear: Create issue", inputSchema: { type: "object" } }; + const assignedMcpTools = { definitions: () => [assignedTool], has: (name: string) => name === assignedTool.name, execute: vi.fn() }; + state.createAssignedMcpTools.mockResolvedValueOnce(assignedMcpTools); + state.toolAuthorityDefinitions.mockImplementation(async binding => + ((binding.assignedMcpTools as typeof assignedMcpTools | undefined)?.definitions() ?? []) as never); + state.createBackend.mockClear(); state.createTransport.mockClear(); + try { + await createRunnerdBackend({ db: leaseDb(current), execution: current, runnerInstanceId: identity.runnerInstanceId, + runnerIngressAuthorized: true, + runnerEnvironment: { + PAPERCLIP_NATIVE_MCP_NAME: "paperclip-assigned", + PAPERCLIP_NATIVE_MCP_URL: "http://127.0.0.1:3217/mcp/gateways/assigned-mid-run", + PAPERCLIP_NATIVE_MCP_TOKEN: "private-mid-run-token", + }, + controllerPersistence: { bootId: "successor", runnerArtifact: { version: "admitted-version", digest: `sha256:${"a".repeat(64)}` } }, + runnerExecutionTarget: { kind: "remote", transport: "sandbox", remoteCwd: "/workspace", + environmentId: "environment", leaseId: identity.environmentLeaseId, providerKey: "daytona", runner: { execute }, + effectiveCapabilities: { runnerWebSocketIngress: true }, + sandboxLeaseAcquisition: { outcome: "resumed", providerLeaseId: "sandbox" } } as never, + restartRecovery: { kind: "reattach_remote_runner", runId: current.binding.runId, leaseOwner: "controller", + controllerGeneration: 2, providerAttempt: 1, restartKind: "hard", recoveryRequestId: null, + remote: { providerLeaseId: "sandbox", remoteCwd: "/workspace" } }, + }); + expect(state.createAssignedMcpTools).toHaveBeenCalledWith(expect.objectContaining({ + gatewayPublicId: "assigned-mid-run", bearerToken: "private-mid-run-token" })); + state.createBackend.mock.calls.at(-1)![1].codexTransportFactory!(); + const options = state.createTransport.mock.calls.at(-1)![0] as RunnerTransportOptions & { resumeDynamicTools?: unknown[] }; + expect(options).toMatchObject({ prpIdentity: identity, adoptExistingRunner: { pid: 123, startedAt: "2026-09-19T10:00:00.000Z" } }); + expect(options.resumeDynamicTools).toContainEqual(assignedTool); + for (const name of ["PAPERCLIP_NATIVE_MCP_NAME", "PAPERCLIP_NATIVE_MCP_URL", "PAPERCLIP_NATIVE_MCP_TOKEN"]) { + expect(options.environment?.[name]).toBeUndefined(); + } + expect(await readFile(join(root, "control-plane", "control-plane-state.json"), "utf8")).toBe(journal); + } finally { + state.toolAuthorityDefinitions.mockImplementation(async () => []); + } + }); + it.each(["codex", "acpx"] as const)("opens a remote %s replacement as a new runner, not the surviving runner it archived", async provider => { const base = scopedNativeExecution(`replacement-after-reattach-${provider}`); const current = { ...base, ...(provider === "acpx" ? { provider: { kind: "acpx", agent: "claude", model: null }, @@ -18824,21 +18878,36 @@ describe("runnerd provider runtime wiring", () => { } }); - it.each(["codex", "opencode"] as const)("keeps assigned MCP credentials on the control plane for %s", async (providerKind) => { + it.each(["codex", "codex-local", "opencode", "acpx-claude", "acpx-pi", "acpx-kimi", "acpx-copilot", "acpx-cursor"] as const)("keeps assigned MCP credentials on the control plane for %s", async (providerKind) => { + const acpxAgent = providerKind.startsWith("acpx-") ? providerKind.slice("acpx-".length) : null; const selectedExecution: NativeExecutionInput = providerKind === "opencode" ? { ...execution, provider: { kind: "opencode", model: "openrouter/deepseek/deepseek-v4.1-flash-1017", permissionMode: "allow" } } - : execution; + : acpxAgent + ? { ...execution, provider: { kind: "acpx", agent: acpxAgent, model: `${acpxAgent}-test-model`, permissionMode: "approve-reads" } } as unknown as NativeExecutionInput + : execution; const endpoint = process.env.OTEL_EXPORTER_OTLP_ENDPOINT; vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", "http://collector.test"); const exporter = new InMemorySpanExporter(); const provider = new TracerProvider({ spanProcessors: [new SimpleSpanProcessor({ exporter })] }); expect(trace.setGlobalTracerProvider(provider)).toBe(true); try { - const assignedMcpTools = { definitions: () => [], has: () => false, execute: vi.fn() }; + const assignedTool = { name: "app_linear_create_issue_0123456789ab", description: "Linear: Create issue", inputSchema: { type: "object" } }; + const assignedMcpTools = { + definitions: () => [assignedTool], has: (name: string) => name === assignedTool.name, + execute: vi.fn(async (_call: unknown) => ({ content: [{ type: "text", text: "created" }] })), + }; state.createAssignedMcpTools.mockResolvedValueOnce(assignedMcpTools); state.createBackend.mockClear(); state.createTransport.mockClear(); state.toolAuthorityDefinitions.mockClear(); + state.toolAuthorityDefinitions.mockImplementation(async binding => + ((binding.assignedMcpTools as typeof assignedMcpTools | undefined)?.definitions() ?? []) as never); + state.toolAuthorityExecute.mockReset().mockImplementation(async (binding: Record, call: unknown) => { + const tools = binding.assignedMcpTools as typeof assignedMcpTools | undefined; + const { tool, arguments: args } = call as { tool: string; arguments: unknown }; + if (!tools?.has(tool)) throw new Error("unknown tool"); + return tools.execute({ tool, arguments: args }); + }); await createRunnerdBackend({ db: leaseDb(selectedExecution), execution: selectedExecution, runnerInstanceId: "runner-assigned-mcp", runnerEnvironment: { @@ -18846,7 +18915,7 @@ describe("runnerd provider runtime wiring", () => { PAPERCLIP_NATIVE_MCP_URL: "http://127.0.0.1:3217/mcp/gateways/assigned-test", PAPERCLIP_NATIVE_MCP_TOKEN: "private-run-token", }, - runnerExecutionTarget: providerKind === "opencode" ? undefined : { + runnerExecutionTarget: providerKind !== "codex" ? undefined : { kind: "remote", transport: "sandbox", providerKey: "daytona", leaseId: "lease-assigned-mcp", remoteCwd: "/home/daytona/paperclip-workspace", runner: { execute: vi.fn() }, @@ -18862,6 +18931,12 @@ describe("runnerd provider runtime wiring", () => { expect(options.environment.PAPERCLIP_NATIVE_MCP_NAME).toBeUndefined(); expect(options.environment.PAPERCLIP_NATIVE_MCP_URL).toBeUndefined(); expect(options.environment.PAPERCLIP_NATIVE_MCP_TOKEN).toBeUndefined(); + const backendOptions = state.createBackend.mock.calls[0]![1]; + expect(backendOptions.dynamicTools).toContainEqual(assignedTool); + const handler = acpxAgent ? backendOptions.acpxDynamicToolHandler! : backendOptions.dynamicToolHandler!; + await expect(handler({ tool: assignedTool.name, callId: "call-assigned", arguments: { title: "Ship it" } } as never)) + .resolves.toEqual({ content: [{ type: "text", text: "created" }] }); + expect(assignedMcpTools.execute).toHaveBeenCalledWith({ tool: assignedTool.name, arguments: { title: "Ship it" } }); await provider.forceFlush(); const spans = exporter.getFinishedSpans(); expect(spans.map(span => span.name)).toEqual(expect.arrayContaining([ @@ -18873,6 +18948,8 @@ describe("runnerd provider runtime wiring", () => { expect(JSON.stringify(span.attributes)).not.toContain("private-run-token"); } } finally { + state.toolAuthorityDefinitions.mockImplementation(async () => []); + state.toolAuthorityExecute.mockReset(); vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", endpoint); trace.disable(); await provider.shutdown(); diff --git a/server/src/services/native-runtime/native-session-executor.ts b/server/src/services/native-runtime/native-session-executor.ts index 74185833543..a00299d9cd6 100644 --- a/server/src/services/native-runtime/native-session-executor.ts +++ b/server/src/services/native-runtime/native-session-executor.ts @@ -16155,23 +16155,24 @@ async function createRunnerdBackendWithinSessionClaim( if (nativeReview && !await getNativeReviewAssignment(input.db, { ...input.execution.binding, contextSnapshot: nativeReview, })) throw new Error("native_review_assignment_no_longer_available"); - // Codex and OpenCode send dynamic tool calls over authenticated PRP. Keep + // Native harnesses send dynamic tool calls over authenticated PRP. Keep // the assigned gateway on the control plane instead of asking the sandbox to // reach the host's HTTP origin (which may be private or loopback-only). - const relayAssignedMcp = Boolean(pluginTask) || input.execution.provider.kind === "cursor_cloud" || input.execution.provider.kind === "opencode" || (remoteTarget !== null && input.execution.provider.kind === "codex") || input.execution.schema === "paperclip.native-execution-input.v6"; const assignedMcpUrl = input.runnerEnvironment?.PAPERCLIP_NATIVE_MCP_URL; const assignedMcpToken = input.runnerEnvironment?.PAPERCLIP_NATIVE_MCP_TOKEN; const assignedMcpName = input.runnerEnvironment?.PAPERCLIP_NATIVE_MCP_NAME; const hasAssignedMcp = Boolean(assignedMcpName || assignedMcpUrl || assignedMcpToken); - if (relayAssignedMcp && hasAssignedMcp && (!assignedMcpName?.trim() || !assignedMcpUrl?.trim() || !assignedMcpToken?.trim())) { + if (hasAssignedMcp && (!assignedMcpName?.trim() || !assignedMcpUrl?.trim() || !assignedMcpToken?.trim())) { throw new Error("assigned native MCP launch binding is incomplete"); } - const assignedGatewayPublicId = relayAssignedMcp && assignedMcpUrl + const assignedGatewayPublicId = assignedMcpUrl ? new URL(assignedMcpUrl).pathname.match(/^\/mcp\/gateways\/([a-zA-Z0-9_-]+)$/)?.[1] : undefined; - if (relayAssignedMcp && hasAssignedMcp && !assignedGatewayPublicId) { + if (hasAssignedMcp && !assignedGatewayPublicId) { throw new Error("assigned native MCP gateway path is invalid"); } + const firstPartyOnly = input.execution.provider.kind === "grok_bot"; + const museRuntime = input.execution.provider.kind === "muse"; let durableState: ReturnType | undefined; let pluginTaskOwner: ReturnType | undefined; const earlyPluginTaskOwner = pluginTask && remoteTarget?.transport === "sandbox" && input.beginRetainedRunAttachment @@ -16180,7 +16181,7 @@ async function createRunnerdBackendWithinSessionClaim( : undefined; const [preparedEnvironment, preparedAssignedMcp, preparedPluginTaskOwner] = await Promise.allSettled([ Promise.resolve().then(() => input.prepareRunnerEnvironment?.() ?? input.runnerEnvironment), - Promise.resolve().then(() => relayAssignedMcp && !nativeReview && assignedMcpUrl && assignedMcpToken + Promise.resolve().then(() => !nativeReview && !firstPartyOnly && !museRuntime && assignedMcpUrl && assignedMcpToken ? timeController("assigned_mcp", () => createAssignedMcpTools({ gateway: getAssignedMcpGateway(input.db), gatewayPublicId: assignedGatewayPublicId!, @@ -16221,8 +16222,8 @@ async function createRunnerdBackendWithinSessionClaim( workspaceBridge: input.execution.schema === "paperclip.native-execution-input.v6" && !!input.dotWorkspaceRoot, taskAttachmentRead: dotAttachmentActor?.config.dotAttachmentAccess === true, dotRuntime: input.execution.schema === "paperclip.native-execution-input.v6", - firstPartyOnly: input.execution.provider.kind === "grok_bot", - museRuntime: input.execution.provider.kind === "muse", + firstPartyOnly, + museRuntime, externalAuthorizingUserId: input.execution.provider.kind === "muse" ? await museRunnerBroker(input.db).authorizingUserIdForBinding(input.execution.provider.binding) : undefined, assertBridgeAuthority: input.execution.schema === "paperclip.native-execution-input.v6" || input.execution.schema === "paperclip.native-execution-input.v8" ? (database = input.db) => assertExternalNativeRunAuthority(input.db, input.execution, database) : undefined, @@ -17964,13 +17965,11 @@ async function createRunnerdBackendWithinSessionClaim( lifecycleMode: input.execution.session.lifecyclePolicy.mode, }) : null; if (antigravityCredential) effectiveRunnerEnvironmentBase = antigravityCredential.environment; - if (relayAssignedMcp) { - // The server-held tool authority owns this credential. Do not deliver a - // duplicate HTTP MCP server or its bearer token to the remote provider. - delete effectiveRunnerEnvironmentBase.PAPERCLIP_NATIVE_MCP_NAME; - delete effectiveRunnerEnvironmentBase.PAPERCLIP_NATIVE_MCP_URL; - delete effectiveRunnerEnvironmentBase.PAPERCLIP_NATIVE_MCP_TOKEN; - } + // The server-held tool authority owns this credential. Do not deliver a + // duplicate HTTP MCP server or its bearer token to the provider. + delete effectiveRunnerEnvironmentBase.PAPERCLIP_NATIVE_MCP_NAME; + delete effectiveRunnerEnvironmentBase.PAPERCLIP_NATIVE_MCP_URL; + delete effectiveRunnerEnvironmentBase.PAPERCLIP_NATIVE_MCP_TOKEN; // This authority bit is derived only from the selected execution target. // Never let an agent, environment binding, or host variable disable the // Codex sandbox for a local runner by supplying the same key. diff --git a/server/src/services/native-runtime/native-session-resume.test.ts b/server/src/services/native-runtime/native-session-resume.test.ts index 1f88c898e36..44feab70bd8 100644 --- a/server/src/services/native-runtime/native-session-resume.test.ts +++ b/server/src/services/native-runtime/native-session-resume.test.ts @@ -1960,6 +1960,11 @@ describe("rebindNativeSessionCheckpoint", () => { retainedFingerprint: "sha256:b64efcd063a575925aa95dbd2a20953386eaa760f05b0e0e73b4ae04a97679b0", }, + { + contract: "direct assigned gateway", + retainedFingerprint: + "sha256:c46f05862e87525db392ebf3157d50c424d6e7010e5640680d0ec5075723d7f5", + }, ])( "refreshes retained $contract without changing the Paperclip task or prior history", ({ contract, retainedFingerprint }) => { @@ -2038,6 +2043,14 @@ describe("rebindNativeSessionCheckpoint", () => { expect(rebindNativeSessionCheckpoint({ previousRun: previousRun({ nativeToolContractFingerprint: current, nativeExecutionInput: previousExecution }), currentExecution, executionTargetKind: target })).not.toBeNull(); }); + it.each([ + ["codex", "sha256:07b116cd7f04b0c9a53b87818a910be32682d5df6181c2019cb1ba8e0077be99"], + ["opencode", "sha256:310dd0367d7157669e9fbf4c41698372a9977ef94183330317167fe07a8987cb"], + ] as const)("keeps a remote %s checkpoint that already relayed its assigned gateway", (providerKind, retainedFingerprint) => { + expect(nativeToolContractFingerprintForTarget("remote", providerKind)).toBe(retainedFingerprint); + expect(nativeToolContractFingerprintForTarget("local", providerKind)).not.toBe(retainedFingerprint); + }); + it("binds the provider tool catalog to the local or remote execution target", () => { const remoteFingerprint = nativeToolContractFingerprintForTarget("remote"); expect(remoteFingerprint).not.toBe(NATIVE_TOOL_CONTRACT_FINGERPRINT); diff --git a/server/src/services/native-runtime/native-session-resume.ts b/server/src/services/native-runtime/native-session-resume.ts index bb9fe4f458d..5b10620f53c 100644 --- a/server/src/services/native-runtime/native-session-resume.ts +++ b/server/src/services/native-runtime/native-session-resume.ts @@ -60,9 +60,7 @@ export function nativeToolContractFingerprintForTarget( chatTaskTools: "durable_task_binding_and_accepted_origin.v1", githubResponses: "bound_working_comment_progress_and_final.v1", githubInstructionSkills: "trusted_configuration_native_selection.v1", - ...(executionTargetKind === "remote" - ? { assignedMcpTools: "codex_server_gateway_prp_relay.v1" } - : {}), + assignedMcpTools: "codex_server_gateway_prp_relay.v1", }, tools: [ { name: "register_deliverable", version: 2 }, diff --git a/server/src/vendor/paperclip-runner/index.ts b/server/src/vendor/paperclip-runner/index.ts index ba3fe4e7e17..a7c2708f747 100644 --- a/server/src/vendor/paperclip-runner/index.ts +++ b/server/src/vendor/paperclip-runner/index.ts @@ -103,6 +103,9 @@ export const inspectUnclaimedBootstrapState = runner.inspectUnclaimedBootstrapSt export const RUNNER_PRP_PROTOCOL_MIN = runner.RUNNER_PRP_PROTOCOL_MIN; export const RUNNER_PRP_PROTOCOL_MAX = runner.RUNNER_PRP_PROTOCOL_MAX; export const runnerCodexDynamicToolsFit = runner.runnerCodexDynamicToolsFit; +export const admitToolInputSchema = runner.admitToolInputSchema; +export const TOOL_INPUT_SCHEMA_MAX_BYTES = runner.TOOL_INPUT_SCHEMA_MAX_BYTES; +export const canonicalToolInputSchema = runner.canonicalToolInputSchema; export const inspectWarmRunTransition = runner.inspectWarmRunTransition; export const inspectStagedRunAttachment = runner.inspectStagedRunAttachment; export const inspectRetainedRunInstructions = runner.inspectRetainedRunInstructions;