diff --git a/doc/connections/CONNECTOR-PLAYBOOK.md b/doc/connections/CONNECTOR-PLAYBOOK.md index 07651e601a..8b786adc98 100644 --- a/doc/connections/CONNECTOR-PLAYBOOK.md +++ b/doc/connections/CONNECTOR-PLAYBOOK.md @@ -438,6 +438,26 @@ Avoid separate methods for: The default setup screen should ask only for information required to make the connection work or enforce a real tenant boundary. Follow these rules: +- Do not ask for a Paperclip **Connection name** during setup. Derive the display + name from the provider and observed account/workspace identity. A provider-required + app/bot name is a separate configuration requirement, not a connection label. +- Use the traditional Gmail/Google Docs setup structure: a compact horizontal + progress header and **Access → Connect**, without a numbered sidebar, + tool-permissions step or optional Test step. Authentication and successfully + reading the tool catalog are enough to complete setup. Do not require a sample + action or add an empty-catalog onboarding detour. Reuse the existing access screen: + “Which humans can use this credential?” and “Which agents can use this connection?” +- After setup, use the regular connection **Permissions** screen: reuse its + canonical searchable Actions list, Read/Write filters, Off / Ask first / Allowed + controls and per-action Test dialog. Do not build a parallel permissions list or + provider-specific testing page. Discovery errors stay inline on Connect; an empty + returned catalog belongs to the ordinary saved-connection state. +- Enable every discovered tool automatically. Put later Allowed / Ask first / Off + controls in management, separate from connection access. Reconnect and refresh + retain existing restrictions; new tools are Allowed under the existing access rules. +- Show browser sign-in/pending/return only for a real OAuth handoff supported by + the chosen authentication. Zapier's pasted MCP URL or bearer token requires no + Paperclip sign-in window. Advanced token/header setups need no invented OAuth step. - Put optional narrowing in fields marked `advanced: true`. - Give hidden fields a `defaultValue`; never create a hidden required field the server cannot fill. @@ -841,8 +861,9 @@ At minimum, add or update tests in these layers: declared. - Finish setup resumes the exact draft using `resumeConnectionId`. - Optional customer OAuth details stay folded when automatic OAuth exists. -- Setup success leads to the connection's Test page, or to Permissions when a - separate agent-resource assignment is the next step. Follow the +- Successful authentication and tool discovery finish setup and lead to the + connection's regular Permissions screen. Testing is available there through each + action's Test button; it is never an onboarding step. Follow the [connection UX guidance](#connection-ux-and-user-journeys). - Interactive Storybooks cover setup and ongoing task interactions, including relevant failure states; the walkthrough matches the implemented journey. diff --git a/doc/connections/REMOTE-MCP-BRAND-SOURCES.md b/doc/connections/REMOTE-MCP-BRAND-SOURCES.md new file mode 100644 index 0000000000..7b7ee7d1eb --- /dev/null +++ b/doc/connections/REMOTE-MCP-BRAND-SOURCES.md @@ -0,0 +1,16 @@ +# MCP connector brand assets + +Retrieved 2026-09-21 from the providers’ own sites for the independent connector +design review. Brand-library membership does not publish a catalog connector. +The four connectors are available for setup only when the MCP aggregators experimental flag is enabled. +Zapier and Composio reuse the existing reviewed local marks. + +| File | Source | SHA-256 | +| --- | --- | --- | +| `ui/public/brands/apps/arcade.png` | [Arcade site touch icon](https://www.arcade.dev/_astro/webclip.BYnZsC5R.png), linked from [arcade.dev](https://www.arcade.dev/) | `5b3704e210b6dc70fffab260d5416624f37db19108399a6d2c18c848d33465ad` | +| `ui/public/brands/apps/executor.png` | [Executor site icon](https://executor.sh/favicon-192.png), linked from [executor.sh](https://executor.sh/) | `e5fd761a0cd80d51d451cc06e9e6d0236dcc39317248a13ac5db5fcf4d52d0ca` | + +Both files preserve provider artwork without recoloring. The shared AppLogo +provides the standard gray frame and contained padding. Arcade’s SVG favicon +wrapped embedded raster content; the 180px touch icon linked by the same official +page was used to satisfy the repository’s artwork safety checks. diff --git a/doc/connections/REMOTE-MCP-LIVE-ACCEPTANCE.md b/doc/connections/REMOTE-MCP-LIVE-ACCEPTANCE.md new file mode 100644 index 0000000000..cf201584c7 --- /dev/null +++ b/doc/connections/REMOTE-MCP-LIVE-ACCEPTANCE.md @@ -0,0 +1,76 @@ +# Independent MCP connectors — acceptance, 2026-09-21 + +Test environment: an isolated development worktree with a fresh database and organization. No production data was cloned. The API served the compiled UI with browser-reachable OAuth callbacks. Detailed account and local-instance evidence remains in a private acceptance report. + +**This PR delivers default-off experimental support at the maintainer's request. General provider acceptance remains incomplete: Zapier needs its generated credential entered before live validation can finish.** Three providers have real browser and real agent proof. Simulations are recorded separately below. The maintainer's subsequent delivery instruction was to put these connectors behind an experimental MCP aggregators flag and open a PR with passing checks. The Zapier gap is an explicit limitation of that experimental scope, not a claim that the connector playbook's full provider acceptance is complete. Complete that acceptance before promoting the feature out of experimental status. + +## Experimental rollout + +Setup is behind **Settings → Experimental → MCP aggregators** (`enableMcpAggregators`). It defaults off on self-hosted and managed instances. When off, all four fresh catalog entries are hidden and direct setup, reconnect setup, and OAuth-start requests are rejected server-side. Existing connections keep running and remain available for management. Legacy Composio API-key/child connections are unchanged. An in-progress OAuth callback may complete; the flag does not revoke already issued credentials or grants. + +Focused regression tests cover flag defaults, persistence, managed metadata, cached catalog visibility, all four direct setup routes, and server-side rejection before network/credential writes. + +## Live results + +| Provider | Functional correctness | UX readiness | Observed account, catalog, and actual results | +| --- | --- | --- | --- | +| Arcade | Passed OAuth setup, Test, real agent, Off, Ask first, denied agent, catalog additions/removal, reconnect, disconnect, and restoration. | Ready for the tested gateway OAuth flow after the fixes below. | Dedicated test gateway with a verified GitHub account. Started with 4 exposed tools, added two read actions, then removed one (5 remain). `Github.GetRepository` returned `paperclipai/paperclip`, branch `master`, repository ID `1170821064`. | +| Composio | Passed default endpoint OAuth, Test, real agent, Off, Ask first, denied agent, refresh, reconnect, disconnect, and restoration. | Ready for Composio Connect. Optional GitHub app consent remains at GitHub's user-verification screen; this does not block the verified DeepWiki action. | Verified provider account; 11 meta tools. Search discovered `DEEPWIKI_MCP_READ_WIKI_STRUCTURE`; Multi Execute returned the real Paperclip documentation hierarchy, 1 success / 0 errors. The real agent repeated discovery and execution. | +| Executor | Passed workspace OAuth, Test, real agent, Off, Ask first, denied agent, refresh, reconnect, disconnect, and restoration. Provider approve/resume, decline, cancel, and Paperclip approval → provider approval were exercised. | Ready for the tested hosted workspace with model-side resume. Decline/cancel copy now describes the deliberate outcome rather than suggesting a retry. | Dedicated test workspace with a verified provider account; 7 tools. Execution returned integration slugs `executor`, `context7`. A disposable Context7 read paused for approval; resuming the same execution returned real React library IDs. | +| Zapier | Live proof blocked after provider setup; production integration and deterministic tests are implemented. | Approved setup design and Storybook checks pass. Real URL-paste workflow still needs completion. | Created a dedicated Managed-mode server with Google Sheets Find Spreadsheet / Get Spreadsheet by ID, using an existing test account. Its generated credential is masked. Copy buttons returned an empty clipboard through automation. No Paperclip credential has been entered or live tool call made. | + +The real Paperclip agent used a vaulted model credential. No secret value is stored in this report or the source tree. + +### Agent evidence + +- Gateway acceptance task: six real gateway calls; Arcade repository read and Executor skills → search → integrations list. +- Execution and disabled-tool task: Composio discovery and DeepWiki execution returned actual headings including Overview, Core Concepts, Getting Started, Server Architecture, and User Interface. Arcade's Off tool was absent from callable discovery. Executor public helper paths were readable and executable. +- Revocation and access-denial task: disconnected Arcade exposed no tools; ungranted Composio exposed no tools and reported that identity/access needed review. Executor remained available and returned its actual integration list. This verifies cross-connection isolation through a real agent. + +### Governance and lifecycle evidence + +- Arcade CountStargazers Off prevented a Test call and disappeared from the real agent's tool surface. GetRepository Ask first made no call until “Allow once”; the saved Test result then showed the actual repository response. +- Composio Multi Execute Off prevented Test execution. Search Ask first completed only after Paperclip review. Removing all agent access made Search Off despite its saved Ask first choice. +- Executor skills Off prevented testing. Removing agent access also prevented execute despite an Ask first choice. A separately allowed execution paused at the provider, preserving its execution ID. Inline acceptance resumed that ID; decline and cancel each stopped their respective execution. Paperclip Ask first → Allow once → provider pending survived navigation to Review and back. +- Arcade catalog refresh enabled newly added GetFileContents and GetIssue automatically, preserved CountStargazers Off and GetRepository Ask first, and removed GetIssue after its removal upstream. Composio/Executor refreshes retained their stable catalogs and rules. Provider-controlled additions to Composio's meta catalog were not manufactured; changed-catalog fixtures cover the common path. +- All three OAuth reconnects retained an empty agent selection and existing Off/Ask first choices. Reloaded screens agreed with effective Test access. +- Disconnected each through the real UI. Subsequent requests for previously valid Test actions returned HTTP 404 `tool_not_found` for all three. Reconnected through Apps and verified fresh Arcade repository, Composio search, and Executor skills calls. The three connections are left connected with default permissions for review. +- Removed the temporary Executor `context7.*` approval policy after testing. The dedicated test gateway, Zapier server, and Context7 connection remain available for review. Existing unrelated provider configurations were not changed. + +## Defects found, fixed, and retested + +1. Enabled DCR ownership for direct MCP OAuth; Composio no longer incorrectly requires a configured client ID. +2. Added explicit provider authorization/approval states, validated handoff links, execution identifiers, and resume controls. Executor's `resume.content` must be a JSON string; corrected it after an observed provider rejection and retested successfully. +3. Added an object JSON editor for open-ended schemas. Composio's nested `arguments` object was otherwise impossible to enter. The real Multi Execute call passed afterward. +4. Corrected broad execution risk classification and the false JWT redaction of three exact public Executor helper selectors. Actual secrets, bearer assignments, and arbitrary dotted values retain redaction. The execution acceptance task verified the selectors live. +5. Prevented app-generated Ask first policies from granting access to an ungranted agent. Negative fixture and live Test/agent checks pass. +6. Allowed an empty selected-agent list and made installation reach plus permission binding updates atomic. OAuth completion no longer substitutes all agents for an empty saved selection. Real reconnects and a dedicated OAuth callback regression pass. +7. Refreshed permission caches with the catalog so newly added tools display Allowed immediately. +8. Retired disappeared tools in stored discovery, not merely the refresh response. Reappearing tools keep existing restrictions. The regression checks persisted database state; the Arcade removal was retested live. +9. Prevented retries of an already-approved provider-pending runtime call from starting another execution. The fixture verifies one dispatch and retained execution identity. Test requires an explicit Reset before starting another approval-controlled call. +10. Added production Reconnect, Manage in provider, and Disconnect controls using the same management component as Storybook. Saved access loading and setup failures remain actionable. + +## Supporting checks + +All newly added isolated checks passed, with one test worker: + +- 27 Vitest checks: protocol (7), provider pending (5), connector lifecycle/governance/OAuth (4), redaction (2), shared contracts (7), schema form (2). +- 18 connector-only Storybook browser checks: four complete setup journeys, four draft/auth journeys, desktop/narrow state matrices, keyboard recovery, Zapier's absence of OAuth, normal Test states, and Executor approve/decline/cancel. +- All 85 connector stories rendered at 1280px/dark and 390px/light without page errors or horizontal overflow. Inspected generated setup and Test screenshots. Evidence is under `tests/storybook-visual/test-results/remote-mcp/` (ignored local test output). +- UI and server direct TypeScript checks, token gates, UI build, Storybook build, and `git diff --check` passed. +- **No full repository test suite, recursive typecheck, or repository-wide build was run locally**, following the maintainer's explicit resource constraint. The PR's CI runs the required broad typecheck, test shards, and build; those gates must pass before handoff. Narrow local verification is not a waiver of CI. No schema migration or lockfile change is included. + +Storybook links: `apps-connections-zapier--complete-setup-journey`, `apps-connections-arcade--complete-setup-journey`, `apps-connections-composio--complete-setup-journey`, `apps-connections-executor--complete-setup-journey`. `apps-connections-executor--provider-handoff-after-setup` uses mocked responses and makes no real authorization request. + +## Remaining work and limits + +PR review regressions: expired MCP sessions now trigger one safe discovery handshake; action calls fail visibly and wait for an explicit retry. Provider handoff detection recognizes protocol/provider envelopes rather than arbitrary app-data statuses. Buffered transports preserve response-ID matching, size limits, and distinct error codes. Dedicated regression checks and selected existing connector/gateway cases pass. Ordinary connector resume/reconnect continues through its existing controller. + +Aggregator action risk is conservative: an unfamiliar or renamed tool defaults to write risk even if the provider advertises it as read-only. Only an explicit reviewed allowlist receives read risk; annotations can still escalate it. This affects risk labels and risk-based governance, not the approved default-enabled behavior or saved Off/Ask first choices. Legacy Composio child connections retain their existing classification path. + +- Paste Zapier's generated Full URL into the already-open local Zapier setup field (not chat), then complete its browser Test, agent, governance, refresh, and lifecycle acceptance. Do not rotate the displayed credential unnecessarily. +- Optional Composio GitHub account authorization awaits the user's GitHub verification. The no-auth DeepWiki app path is proven; GitHub app execution is not claimed. +- Hosted OAuth paths were tested live. Custom-header/session imports, credential-bearing URLs, protocol URL elicitation, pagination edge cases, and revocation races have deterministic fixture coverage rather than separate live accounts/endpoints for every variant. +- Provider auth links are kept briefly in memory; durable records retain redacted handoff/execution identifiers. After an application restart, a one-time auth link may need reopening from the provider dashboard. Calls are never automatically replayed to recover it. + +Completion still requires observed live results for Zapier. Passing stories and fixtures do not substitute for that proof. diff --git a/doc/design/COMPONENT-INVENTORY.md b/doc/design/COMPONENT-INVENTORY.md index 84bdccd4ab..91bf50d192 100644 --- a/doc/design/COMPONENT-INVENTORY.md +++ b/doc/design/COMPONENT-INVENTORY.md @@ -6,6 +6,20 @@ Run scope: `ui/src/components/` and `ui/src/pages/` on branch `design/token-extr ## Counts +### Independent MCP connection setup — 2026-09-21 + +`ui/src/features/connections/remote-mcp/RemoteMcpConnectionSetup.tsx` is the controlled +Access → Connect and management composition for Zapier, Arcade, +Composio and Executor. It reuses Gmail’s StepHeader and AccessStepContent, plus SetupWizardFooter, +AppLogo, InlineBanner and existing form/dialog primitives. Saved connections reuse +the canonical PermissionsPanel ActionsSection and its ActionTestDialog; there is +no separate setup test or permission list. Each +provider has an independent state and controller. It does not initiate network +authentication requests or persist credentials. The shared Test dialog uses normal +API calls, intercepted by scoped in-memory Storybook fixtures. The first milestone is design review; production +routes do not use it yet. See `/design-guide` and Storybook **Apps / Connections** +for the provider variants and complete interactive state matrix. + Execution recovery reuses the existing transcript header and task status. Routine phases add no list badges, status cards, or reconciliation dialogs. Only a transient reconnection changes the header text. Automatic recovery decisions remain in the local run log. Storybook **Tasks / Execution recovery** demonstrates quiet task lists, native and legacy transcript headers, and dashboard composition. | Area | Count | diff --git a/doc/plans/2026-09-21-independent-mcp-connectors.md b/doc/plans/2026-09-21-independent-mcp-connectors.md new file mode 100644 index 0000000000..c9103bdbc0 --- /dev/null +++ b/doc/plans/2026-09-21-independent-mcp-connectors.md @@ -0,0 +1,220 @@ +# Four independent MCP connectors + +Branch: `codex/unified-mcp-connectors` +Worktree: dedicated checkout on `codex/unified-mcp-connectors` +Base: `b19307758` + +Zapier, Arcade, Composio and Executor each have their own connection, credentials, +catalog, agent access, permissions, sessions and lifecycle. Their setup shares the +same structure: **Access → Connect**. Underlying apps remain managed +in the provider; Paperclip does not create child connections for every app. + +## Delivery checkpoints + +- [x] Create the fresh worktree. +- [x] Build controlled application views and four interactive Storybook groups. +- [x] Review the designs with the user and incorporate revisions. +- [x] Wire the reviewed views to production APIs and implement shared protocol support. +- [ ] Conduct real browser acceptance testing for all four providers, fix and retest. +- [x] Complete the user-authorized narrow checks: new isolated tests, direct UI/server typechecks, token gates, UI and Storybook builds. Full repository suites/builds were explicitly excluded. + +Design review is complete and the user authorized production integration and live browser testing. The implementation is running in fresh local data. Three providers have real browser and agent proof; Zapier is waiting for a credential handoff. See [the current acceptance report](../connections/REMOTE-MCP-LIVE-ACCEPTANCE.md). Simulations are not live proof. + +## Review links and operation + +The review server uses the build from this worktree on port 6137: + +| Provider | Complete journey | +| --- | --- | +| Zapier | `apps-connections-zapier--complete-setup-journey` | +| Arcade | `apps-connections-arcade--complete-setup-journey` | +| Composio | `apps-connections-composio--complete-setup-journey` | +| Executor | `apps-connections-executor--complete-setup-journey` | + +Choose who can use the connection, continue, then use **Use example configuration**. The separate +**Storybook simulation** area supplies provider responses and browser sign-in or +completion events. After connection, Test opens the regular per-action dialog with +scoped mock API responses; approval requests stay pending in this preview. Provider links are intercepted by this simulator; +they do not open real authorization pages. Example catalogs and schemas are +representative, not production constants. Never enter real credentials here. + +Each provider group exposes initial access, selected agents, URL and advanced +setup, connecting, invalid URL, rejected credentials, unreachable endpoint, +and post-setup permissions/management states. Successful authentication and catalog +discovery complete setup. There is no empty-catalog or Test setup step. Tool restrictions +and action tests use the regular Permissions screen after setup. All new tools are enabled automatically. + +Browser sign-in pending/return/cancel stories apply only to Arcade, Composio and +Executor OAuth configurations. Zapier uses the pasted URL/token without an OAuth +handoff. Upstream app authorization and Executor resume now use the shared +post-setup Test dialog; bespoke setup test stories +have been removed. Narrow access, connection details +and tool management are directly accessible. + +### Design feedback incorporated — 2026-09-21 + +- Reuse the actual Gmail `AccessStepContent` and `StepHeader`: human credential + sharing and agent reach come before credentials, with compact top progress bars. +- Remove Connection name, the numbered sidebar and setup tool-permission controls. +- Finish setup as soon as authentication and tool discovery succeed, with all + tools enabled. Remove the empty-catalog story and every setup Test step. +- Reuse the actual Permissions `ActionsSection` and `ActionTestDialog` after setup, + including search, Read/Write filters, permission radios and per-action testing. +- Keep tool restrictions in management, separate from who can use the connection. +- Remove Zapier OAuth stories/options and unsupported provider resume examples. +- Update the canonical connector playbook and local `chat-connector-ux` skill with + these conventions, including the explicit prohibition on setup connection names. + +Drafts live only in React state in the previews and clear on reload. The production +controller must persist progress through refresh and OAuth redirects using the +existing server draft and vault mechanisms. Do not copy preview persistence into +production. + +To rebuild and serve: + +```sh +pnpm --filter @paperclipai/ui build-storybook +node scripts/serve-storybook-static.mjs --port 6137 +``` + +## Provider setup contracts + +| Provider | New setup | Permission boundary | +| --- | --- | --- | +| Zapier | Paste generated secret-bearing URL; alternatively endpoint plus bearer token. | Recommend Managed mode for individual action controls. Agentic mode exposes discovery/execution tools; permissions cover the whole exposed call. | +| Arcade | Paste gateway URL and sign in through the gateway’s configured User Source. Advanced: bearer API key plus `Arcade-User-ID`. | Actual exposed gateway tools. Individual apps can require further authorization. | +| Composio | Prefill `https://connect.composio.dev/mcp`, then authenticate. Advanced: externally configured session URL and headers. | Connect’s discovery, execution, connection-management and sandbox tools. Direct-tools sessions can expose individual actions. | +| Executor | Paste hosted workspace URL or reachable self-hosted HTTP endpoint and authenticate. Advanced user API key when supported. | Execution and helper tools; action policies remain in Executor. Preserve execution and MCP session identity for resume. | + +The production catalog always comes from discovery, never the Storybook fixture. +Only expose a pending-state recovery action when the actual provider response and +protocol support it. Executor’s browser approval/resume is a specific supported +pattern; implement it in the shared post-setup Test dialog, not in onboarding. + +Existing Composio project-API-key and child connections must remain supported. +New direct-MCP setup must not migrate their credentials or grants. Narrow legacy +broker checks by setup/transport instead of treating all Composio connections as +parent brokers. Vercel Connect remains a separate follow-up: its documented role +supplies credentials for an app’s MCP connection rather than this aggregator flow. + +Research references (checked 2026-09-21): + +- Zapier [setup](https://docs.zapier.com/mcp/get-started/connect/other) and [tool modes](https://docs.zapier.com/mcp/overview/how-tools-work). +- Arcade [MCP gateways](https://docs.arcade.dev/en/operate/governance/mcp-gateways). +- Composio [Connect](https://docs.composio.dev/docs/composio-connect) and [sessions](https://docs.composio.dev/docs/sessions-via-mcp). +- Executor [MCP proxy](https://executor.sh/docs/mcp-proxy) and [implementation](https://github.com/UsefulSoftwareCo/executor). +- Vercel [AI SDK and MCP integration](https://vercel.com/docs/connect/frameworks/ai-sdk-and-mcp). + +## Production implementation scope + +The views live in `ui/src/features/connections/remote-mcp/`; fixtures and the +simulator live under `ui/storybook/`. Reuse the views in the production controller +and keep the stories synchronized. The production controller is routed into Apps. + +1. **Definitions and contracts:** give all four providers independent catalog + definitions, setup/branding metadata and capabilities. Permit branded setup + and configuration imports to use OAuth discovery, bearer tokens, custom + headers and credential-bearing URLs. Extend existing connection/test types + with provider presentation and upstream pending state. No new connection + model or database tables are planned. +2. **MCP transport:** complete initialized Streamable HTTP operation, paginated + `tools/list`, response matching by JSON-RPC ID, and session continuity across + execution/resume. Existing `server/src/services/mcp-http.ts` and tool-access + discovery need this work; a one-request proxy is insufficient. +3. **Gateway governance:** enforce company membership, agent grant and tool + permission on both discovery and invocation, including resume. Allow new + catalog entries under existing connection access; preserve explicit Off and + Ask first, remove disappeared tools, and audit changes. Broad-tool permission + covers the entire exposed call. +4. **Authentication and recovery:** keep credentials/catalogs/policies/sessions + isolated by connection and effective identity. Use the vault, endpoint + validation and secret redaction. Preserve execution IDs and handle provider + authorization URLs and URL elicitation in both runtime and Test. Never + automatically repeat a call when a write may already have happened. Disconnect + must revoke access and invalidate sessions without affecting other connections. +5. **UI controller:** connect these controlled views to `ConnectionSetupFlow`, + current permissions and Test interfaces, draft storage, OAuth and existing + service APIs. Resolve provider pending actions using observed capabilities. + Save errors remain visible and do not pretend to persist a draft. The test’s + acting agent uses the same effective rules as a real agent gateway call. +6. **Verification:** protocol/governance fixtures, Storybook interaction and + visual checks, token gates, Storybook build, then required repository typecheck, + tests and build. Run the final checks on the integrated production change. + +## Historical design milestone verification + +Review environment: macOS, this worktree, static Storybook, example company, +agents and catalogs. Browser UI simulation and production acceptance are recorded +separately. Screenshots generated by the browser suite are local test evidence; +they are not Linux visual-regression baselines. + +Commands: + +```sh +pnpm --filter @paperclipai/ui typecheck +pnpm check:token-gates +pnpm exec vitest run ui/src/components/SetupWizard.test.tsx ui/src/components/JsonSchemaForm.test.tsx ui/src/features/connections/ConnectionSetupFlow.architecture.test.ts ui/src/pages/apps/app-detail/TestPanel.test.tsx +pnpm --filter @paperclipai/ui build-storybook +pnpm exec playwright test --config tests/storybook-visual/remote-mcp-connections.config.ts +``` + +Observed fixes during review: Save & exit now explicitly avoids submitting its +form; schema inputs expose accessible names; setup reuses the Gmail access choices and compact top progress header; newly discovered fixture tools default to Allowed without resetting saved +restrictions. The shared components retain their existing behavior otherwise. + +### Design revision verification (before production integration) + +- UI typecheck, token gates, 34 focused tests and Storybook build passed. +- All 15 browser checks passed, including four complete Access → Connect journeys + that finish with zero action calls and every discovered action Allowed. +- The canonical Permissions action list and Test dialog cover effective agent + access, denied agents, disabled tools, success, errors and pending approval. + Permission changes survive catalog refresh and reconnect; disconnect blocks use. +- All 82 stories render at 1280px/dark and 390px/light without page errors or + horizontal overflow: Zapier 18, Arcade 21, Composio 22, Executor 21. +- Browser inspection confirmed Zapier goes straight from Connect to saved + permissions, then opens the shared Test dialog. Narrow Permissions screenshots + for Zapier/Composio and the shared Test dialog were visually reviewed. +- Fake credentials stay out of browser storage, and scoped test requests never + leave the Storybook mock. No real provider action ran. + +Screenshots are local test evidence in +`tests/storybook-visual/test-results/remote-mcp/`. Earlier 117/122-story results +included the removed setup Test flow and are superseded by this revision. + +The removed simulations were never live provider proof. Production results and the final narrow check commands are recorded in the acceptance report. The user subsequently prohibited running the full suite locally; that later constraint supersedes the earlier repository-wide verification plan. + +Dependency installation previously used the pinned pnpm after a base-branch +patch/lockfile mismatch (`postgres@3.4.9`); the tracked lockfile was restored. +No dependency or lockfile changes are included. + +## Current live acceptance + +See [REMOTE-MCP-LIVE-ACCEPTANCE.md](../connections/REMOTE-MCP-LIVE-ACCEPTANCE.md) for provider-specific results, actual agent task links, fixes, retests, and remaining gaps. + +- Arcade, Composio, and Executor: real OAuth, catalog, Test and agent calls observed; governance and lifecycle exercised. +- Zapier: dedicated provider server configured, but its generated secret URL still needs to be pasted into the local setup form. No live execution proof is claimed. +- Supporting checks: 27 newly added isolated Vitest checks, 18 connector-only Storybook checks, 85 stories at desktop/narrow widths, direct UI/server typechecks, token gates, UI build, and Storybook build passed. One test worker; no full repository suite or recursive build/typecheck. + +The isolated test instance has a fresh database and a browser-reachable OAuth callback. All three tested connections were restored after revocation checks and are available for review. + +For **each** provider, record environment, account identity, observed catalog, +journeys, actual results, useful screenshots, defects/fixes/retests and gaps: + +1. Discover it from Apps, connect it, and verify account and catalog. +2. Assign agent access and exercise Allowed, Ask first and Off. +3. Run a useful action in Test and verify its external result; use disposable + data for writes. +4. Have a real Paperclip agent call through the Paperclip gateway. Verify an + ungranted agent and disabled tool are denied. +5. Exercise applicable provider authorization/approval, including Executor resume + without starting a second execution. +6. Add/remove tools upstream and refresh. New tools become Allowed; prior Off and + Ask first remain. Confirm reload, draft return, reconnect and disconnect. +7. Change this connection and prove the other connections remain unaffected. +8. Assess redirects, transient errors, duplicate writes, copy, keyboard access, + desktop and narrow layouts. Fix defects and repeat affected journeys. + +Completion requires observed live results for **all four**, with functional +correctness and UX readiness assessed separately. Storybook successes do not +check any live acceptance box. diff --git a/packages/adapter-utils/src/command-redaction.ts b/packages/adapter-utils/src/command-redaction.ts index 5890973961..d922367c1d 100644 --- a/packages/adapter-utils/src/command-redaction.ts +++ b/packages/adapter-utils/src/command-redaction.ts @@ -1,5 +1,16 @@ export const REDACTED_COMMAND_TEXT_VALUE = "***REDACTED***"; +// These exact public Executor helper addresses resemble dotted bearer tokens. +// Do not exempt arbitrary provider paths, prefixes, or user-defined selectors. +const PUBLIC_EXECUTOR_TOOL_SELECTORS = new Set([ + "executor.coreTools.integrations.list", + "executor.coreTools.connections.list", + "executor.coreTools.policies.list", +]); +export function isPublicExecutorToolSelector(value: string): boolean { + return PUBLIC_EXECUTOR_TOOL_SELECTORS.has(value); +} + const SECRET_NAME_PATTERN = String.raw`[A-Za-z0-9_-]*(?:api[-_]?key|(?:access[-_]?|auth[-_]?)?token|token|authorization|bearer|secret|passwd|password|credential|jwt|private[-_]?key|cookie|connectionstring)[A-Za-z0-9_-]*`; const COMMAND_CLI_SECRET_OPTION_RE = new RegExp( @@ -76,7 +87,12 @@ export function redactCommandText( ) .replace(COMMAND_OPENAI_KEY_RE, redactedValue) .replace(COMMAND_GITHUB_TOKEN_RE, redactedValue) - .replace(COMMAND_JWT_RE, redactedValue); + .replace(COMMAND_JWT_RE, (match, offset: number, source: string) => { + // The JWT heuristic may match only the first three segments; inspect the + // complete address so a longer secret sharing a prefix stays redacted. + const address = source.slice(offset).match(/^[A-Za-z0-9_-]+(?:\.[A-Za-z0-9_-]+)*/)?.[0]; + return address && isPublicExecutorToolSelector(address) ? match : redactedValue; + }); } // A JSON secret field is a key/value pair such as `"token":"opaque-value"`. The diff --git a/packages/shared/src/app-definitions-url.test.ts b/packages/shared/src/app-definitions-url.test.ts index 5503545e72..f5bcea4768 100644 --- a/packages/shared/src/app-definitions-url.test.ts +++ b/packages/shared/src/app-definitions-url.test.ts @@ -33,9 +33,10 @@ describe("tool app gallery URL matching", () => { expect(getAppDefinitionForUrl("https://drivemcp.googleapis.com/mcp/v1")?.slug).toBe("google-drive"); }); - it("lists Composio as a connectable API-key app", () => { + it("lists Composio Connect alongside the legacy API-key method", () => { const composio = CONNECTABLE_APP_DEFINITIONS.find((app) => app.slug === "composio"); expect(composio?.methods).toEqual([ + expect.objectContaining({ key: "mcp", transport: "mcp_remote", ownershipModes: ["dcr", "customer"] }), expect.objectContaining({ key: "api-key", transport: "rest_api", auth: "api_key" }), ]); }); diff --git a/packages/shared/src/app-definitions.generated.ts b/packages/shared/src/app-definitions.generated.ts index 90aabc8775..1b8139bb8a 100644 --- a/packages/shared/src/app-definitions.generated.ts +++ b/packages/shared/src/app-definitions.generated.ts @@ -1,74 +1,76 @@ import a0 from "./app-definitions/agentmail.json" with { type: "json" }; import a1 from "./app-definitions/zapier.json" with { type: "json" }; -import a2 from "./app-definitions/railway.json" with { type: "json" }; -import a3 from "./app-definitions/github.json" with { type: "json" }; -import a4 from "./app-definitions/slack.json" with { type: "json" }; -import a5 from "./app-definitions/microsoft-teams.json" with { type: "json" }; -import a6 from "./app-definitions/imessage-photon.json" with { type: "json" }; -import a7 from "./app-definitions/telegram.json" with { type: "json" }; -import a8 from "./app-definitions/discord.json" with { type: "json" }; -import a9 from "./app-definitions/notion.json" with { type: "json" }; -import a10 from "./app-definitions/posthog.json" with { type: "json" }; -import a11 from "./app-definitions/linear.json" with { type: "json" }; -import a12 from "./app-definitions/context7.json" with { type: "json" }; -import a13 from "./app-definitions/shopify.json" with { type: "json" }; -import a14 from "./app-definitions/composio.json" with { type: "json" }; -import a15 from "./app-definitions/oauth-generic.json" with { type: "json" }; -import a16 from "./app-definitions/api-key-generic.json" with { type: "json" }; -import a17 from "./app-definitions/sentry.json" with { type: "json" }; -import a18 from "./app-definitions/vercel.json" with { type: "json" }; -import a19 from "./app-definitions/anthropic.json" with { type: "json" }; -import a20 from "./app-definitions/jira.json" with { type: "json" }; -import a21 from "./app-definitions/airtable.json" with { type: "json" }; -import a22 from "./app-definitions/beehiiv.json" with { type: "json" }; -import a23 from "./app-definitions/bitly.json" with { type: "json" }; -import a24 from "./app-definitions/candid.json" with { type: "json" }; -import a25 from "./app-definitions/cloudflare.json" with { type: "json" }; -import a26 from "./app-definitions/cloudinary.json" with { type: "json" }; -import a27 from "./app-definitions/coda.json" with { type: "json" }; -import a28 from "./app-definitions/hugging-face.json" with { type: "json" }; -import a29 from "./app-definitions/kernel.json" with { type: "json" }; -import a30 from "./app-definitions/local-falcon.json" with { type: "json" }; -import a31 from "./app-definitions/make.json" with { type: "json" }; -import a32 from "./app-definitions/manufact.json" with { type: "json" }; -import a33 from "./app-definitions/miro.json" with { type: "json" }; -import a34 from "./app-definitions/netlify.json" with { type: "json" }; -import a35 from "./app-definitions/oreilly.json" with { type: "json" }; -import a36 from "./app-definitions/planetscale.json" with { type: "json" }; -import a37 from "./app-definitions/resend.json" with { type: "json" }; -import a38 from "./app-definitions/ticktick.json" with { type: "json" }; -import a39 from "./app-definitions/todoist.json" with { type: "json" }; -import a40 from "./app-definitions/webflow.json" with { type: "json" }; -import a41 from "./app-definitions/wix.json" with { type: "json" }; -import a42 from "./app-definitions/brex.json" with { type: "json" }; -import a43 from "./app-definitions/clickhouse.json" with { type: "json" }; -import a44 from "./app-definitions/egnyte.json" with { type: "json" }; -import a45 from "./app-definitions/embat.json" with { type: "json" }; -import a46 from "./app-definitions/mixpanel.json" with { type: "json" }; -import a47 from "./app-definitions/postman.json" with { type: "json" }; -import a48 from "./app-definitions/razorpay.json" with { type: "json" }; -import a49 from "./app-definitions/sanity.json" with { type: "json" }; -import a50 from "./app-definitions/stripe.json" with { type: "json" }; -import a51 from "./app-definitions/supabase.json" with { type: "json" }; -import a52 from "./app-definitions/ticket-tailor.json" with { type: "json" }; -import a53 from "./app-definitions/asana.json" with { type: "json" }; -import a54 from "./app-definitions/box.json" with { type: "json" }; -import a55 from "./app-definitions/mem0.json" with { type: "json" }; -import a56 from "./app-definitions/pagerduty.json" with { type: "json" }; -import a57 from "./app-definitions/similarweb.json" with { type: "json" }; -import a58 from "./app-definitions/xero.json" with { type: "json" }; -import a59 from "./app-definitions/youcom.json" with { type: "json" }; -import a60 from "./app-definitions/gmail.json" with { type: "json" }; -import a61 from "./app-definitions/google-drive.json" with { type: "json" }; -import a62 from "./app-definitions/google-docs.json" with { type: "json" }; -import a63 from "./app-definitions/google-sheets.json" with { type: "json" }; -import a64 from "./app-definitions/google-slides.json" with { type: "json" }; -import a65 from "./app-definitions/google-calendar.json" with { type: "json" }; -import a66 from "./app-definitions/google-chat.json" with { type: "json" }; -import a67 from "./app-definitions/google-people.json" with { type: "json" }; -import a68 from "./app-definitions/google-workspace-search.json" with { type: "json" }; -import a69 from "./app-definitions/openai.json" with { type: "json" }; -import a70 from "./app-definitions/openrouter.json" with { type: "json" }; -import a71 from "./app-definitions/xai.json" with { type: "json" }; +import a2 from "./app-definitions/arcade.json" with { type: "json" }; +import a3 from "./app-definitions/executor.json" with { type: "json" }; +import a4 from "./app-definitions/railway.json" with { type: "json" }; +import a5 from "./app-definitions/github.json" with { type: "json" }; +import a6 from "./app-definitions/slack.json" with { type: "json" }; +import a7 from "./app-definitions/microsoft-teams.json" with { type: "json" }; +import a8 from "./app-definitions/imessage-photon.json" with { type: "json" }; +import a9 from "./app-definitions/telegram.json" with { type: "json" }; +import a10 from "./app-definitions/discord.json" with { type: "json" }; +import a11 from "./app-definitions/notion.json" with { type: "json" }; +import a12 from "./app-definitions/posthog.json" with { type: "json" }; +import a13 from "./app-definitions/linear.json" with { type: "json" }; +import a14 from "./app-definitions/context7.json" with { type: "json" }; +import a15 from "./app-definitions/shopify.json" with { type: "json" }; +import a16 from "./app-definitions/composio.json" with { type: "json" }; +import a17 from "./app-definitions/oauth-generic.json" with { type: "json" }; +import a18 from "./app-definitions/api-key-generic.json" with { type: "json" }; +import a19 from "./app-definitions/sentry.json" with { type: "json" }; +import a20 from "./app-definitions/vercel.json" with { type: "json" }; +import a21 from "./app-definitions/anthropic.json" with { type: "json" }; +import a22 from "./app-definitions/jira.json" with { type: "json" }; +import a23 from "./app-definitions/airtable.json" with { type: "json" }; +import a24 from "./app-definitions/beehiiv.json" with { type: "json" }; +import a25 from "./app-definitions/bitly.json" with { type: "json" }; +import a26 from "./app-definitions/candid.json" with { type: "json" }; +import a27 from "./app-definitions/cloudflare.json" with { type: "json" }; +import a28 from "./app-definitions/cloudinary.json" with { type: "json" }; +import a29 from "./app-definitions/coda.json" with { type: "json" }; +import a30 from "./app-definitions/hugging-face.json" with { type: "json" }; +import a31 from "./app-definitions/kernel.json" with { type: "json" }; +import a32 from "./app-definitions/local-falcon.json" with { type: "json" }; +import a33 from "./app-definitions/make.json" with { type: "json" }; +import a34 from "./app-definitions/manufact.json" with { type: "json" }; +import a35 from "./app-definitions/miro.json" with { type: "json" }; +import a36 from "./app-definitions/netlify.json" with { type: "json" }; +import a37 from "./app-definitions/oreilly.json" with { type: "json" }; +import a38 from "./app-definitions/planetscale.json" with { type: "json" }; +import a39 from "./app-definitions/resend.json" with { type: "json" }; +import a40 from "./app-definitions/ticktick.json" with { type: "json" }; +import a41 from "./app-definitions/todoist.json" with { type: "json" }; +import a42 from "./app-definitions/webflow.json" with { type: "json" }; +import a43 from "./app-definitions/wix.json" with { type: "json" }; +import a44 from "./app-definitions/brex.json" with { type: "json" }; +import a45 from "./app-definitions/clickhouse.json" with { type: "json" }; +import a46 from "./app-definitions/egnyte.json" with { type: "json" }; +import a47 from "./app-definitions/embat.json" with { type: "json" }; +import a48 from "./app-definitions/mixpanel.json" with { type: "json" }; +import a49 from "./app-definitions/postman.json" with { type: "json" }; +import a50 from "./app-definitions/razorpay.json" with { type: "json" }; +import a51 from "./app-definitions/sanity.json" with { type: "json" }; +import a52 from "./app-definitions/stripe.json" with { type: "json" }; +import a53 from "./app-definitions/supabase.json" with { type: "json" }; +import a54 from "./app-definitions/ticket-tailor.json" with { type: "json" }; +import a55 from "./app-definitions/asana.json" with { type: "json" }; +import a56 from "./app-definitions/box.json" with { type: "json" }; +import a57 from "./app-definitions/mem0.json" with { type: "json" }; +import a58 from "./app-definitions/pagerduty.json" with { type: "json" }; +import a59 from "./app-definitions/similarweb.json" with { type: "json" }; +import a60 from "./app-definitions/xero.json" with { type: "json" }; +import a61 from "./app-definitions/youcom.json" with { type: "json" }; +import a62 from "./app-definitions/gmail.json" with { type: "json" }; +import a63 from "./app-definitions/google-drive.json" with { type: "json" }; +import a64 from "./app-definitions/google-docs.json" with { type: "json" }; +import a65 from "./app-definitions/google-sheets.json" with { type: "json" }; +import a66 from "./app-definitions/google-slides.json" with { type: "json" }; +import a67 from "./app-definitions/google-calendar.json" with { type: "json" }; +import a68 from "./app-definitions/google-chat.json" with { type: "json" }; +import a69 from "./app-definitions/google-people.json" with { type: "json" }; +import a70 from "./app-definitions/google-workspace-search.json" with { type: "json" }; +import a71 from "./app-definitions/openai.json" with { type: "json" }; +import a72 from "./app-definitions/openrouter.json" with { type: "json" }; +import a73 from "./app-definitions/xai.json" with { type: "json" }; import type { AppDefinition } from "./types/app-definition.js"; -export const APP_DEFINITIONS=[a0,a1,a2,a3,a4,a5,a6,a7,a8,a9,a10,a11,a12,a13,a14,a15,a16,a17,a18,a19,a20,a21,a22,a23,a24,a25,a26,a27,a28,a29,a30,a31,a32,a33,a34,a35,a36,a37,a38,a39,a40,a41,a42,a43,a44,a45,a46,a47,a48,a49,a50,a51,a52,a53,a54,a55,a56,a57,a58,a59,a60,a61,a62,a63,a64,a65,a66,a67,a68,a69,a70,a71] as AppDefinition[]; +export const APP_DEFINITIONS=[a0,a1,a2,a3,a4,a5,a6,a7,a8,a9,a10,a11,a12,a13,a14,a15,a16,a17,a18,a19,a20,a21,a22,a23,a24,a25,a26,a27,a28,a29,a30,a31,a32,a33,a34,a35,a36,a37,a38,a39,a40,a41,a42,a43,a44,a45,a46,a47,a48,a49,a50,a51,a52,a53,a54,a55,a56,a57,a58,a59,a60,a61,a62,a63,a64,a65,a66,a67,a68,a69,a70,a71,a72,a73] as AppDefinition[]; diff --git a/packages/shared/src/app-definitions.test.ts b/packages/shared/src/app-definitions.test.ts index e1ae262d54..b4ae9cd614 100644 --- a/packages/shared/src/app-definitions.test.ts +++ b/packages/shared/src/app-definitions.test.ts @@ -692,7 +692,6 @@ describe("AppDefinition catalog", () => { "brex", "candid", "coda", - "composio", "context7", "egnyte", "embat", @@ -709,7 +708,7 @@ describe("AppDefinition catalog", () => { "ticktick", "xero", ]); - expect(APP_STORE_DEFINITIONS).toHaveLength(48); + expect(APP_STORE_DEFINITIONS).toHaveLength(51); const connectableSlugs = new Set( CONNECTABLE_APP_DEFINITIONS.map((entry) => entry.slug), ); diff --git a/packages/shared/src/app-definitions.ts b/packages/shared/src/app-definitions.ts index 0ff98f037e..1e16e35aa2 100644 --- a/packages/shared/src/app-definitions.ts +++ b/packages/shared/src/app-definitions.ts @@ -8,6 +8,8 @@ export const CONNECTABLE_APP_SLUGS = new Set([ "agentmail", ...SELF_SERVE_MCP_CANDIDATES.map((entry) => entry.slug), "zapier", + "arcade", + "executor", "slack", "notion", "railway", @@ -48,7 +50,6 @@ export const APP_STORE_HIDDEN_SLUGS = new Set([ "brex", "candid", "coda", - "composio", "context7", "egnyte", "embat", diff --git a/packages/shared/src/app-definitions/arcade.json b/packages/shared/src/app-definitions/arcade.json new file mode 100644 index 0000000000..c837945005 --- /dev/null +++ b/packages/shared/src/app-definitions/arcade.json @@ -0,0 +1,36 @@ +{ + "schemaVersion": 1, + "slug": "arcade", + "name": "Arcade", + "description": "Use the tools exposed by your Arcade MCP connection.", + "categories": [ + "productivity" + ], + "featured": true, + "branding": { + "logoUrl": "/brands/apps/arcade.png" + }, + "urlPatterns": [ + "https://api.arcade.dev/*" + ], + "methods": [ + { + "key": "mcp", + "transport": "mcp_remote", + "auth": "none", + "ownershipModes": [ + "dcr", + "customer" + ], + "whenToUse": "Use the provider-hosted connection for the quickest setup.", + "defaults": {}, + "guidanceMd": "Paste your Arcade MCP URL. Sign in if required, or add a token or headers under Advanced authentication.", + "riskTier": "S3", + "label": "Connect MCP server", + "consoleLinks": { + "docs": "https://docs.arcade.dev/en/operate/governance/mcp-gateways" + } + } + ], + "docsUrl": "https://docs.arcade.dev/en/operate/governance/mcp-gateways" +} diff --git a/packages/shared/src/app-definitions/composio.json b/packages/shared/src/app-definitions/composio.json index 29f3efe967..b5c880da38 100644 --- a/packages/shared/src/app-definitions/composio.json +++ b/packages/shared/src/app-definitions/composio.json @@ -2,7 +2,7 @@ "schemaVersion": 1, "slug": "composio", "name": "Composio", - "description": "Connect Composio so Paperclip can discover and manage the toolkits in your project.", + "description": "Discover and use connected apps through Composio Connect.", "categories": [ "productivity" ], @@ -12,9 +12,28 @@ "darkLogoUrl": "/brands/apps/composio-dark.svg" }, "urlPatterns": [ - "https://backend.composio.dev/*" + "https://backend.composio.dev/*", + "https://connect.composio.dev/*", + "https://mcp.composio.dev/*", + "https://*.composio.dev/*" ], "methods": [ + { + "key": "mcp", + "transport": "mcp_remote", + "auth": "none", + "ownershipModes": [ + "dcr", + "customer" + ], + "whenToUse": "Use the provider-hosted connection for the quickest setup.", + "defaults": { + "serverUrl": "https://connect.composio.dev/mcp" + }, + "guidanceMd": "Sign in to Composio Connect, or paste an externally configured MCP session URL and headers.", + "riskTier": "S3", + "label": "Composio Connect" + }, { "key": "api-key", "transport": "rest_api", @@ -48,5 +67,6 @@ "docs": "https://docs.composio.dev/reference/authenticating-to-composio/project-api-key-permissions" } } - ] + ], + "docsUrl": "https://docs.composio.dev/docs/composio-connect" } diff --git a/packages/shared/src/app-definitions/executor.json b/packages/shared/src/app-definitions/executor.json new file mode 100644 index 0000000000..f01e0cd4e1 --- /dev/null +++ b/packages/shared/src/app-definitions/executor.json @@ -0,0 +1,36 @@ +{ + "schemaVersion": 1, + "slug": "executor", + "name": "Executor", + "description": "Use the tools exposed by your Executor MCP connection.", + "categories": [ + "productivity" + ], + "featured": true, + "branding": { + "logoUrl": "/brands/apps/executor.png" + }, + "urlPatterns": [ + "https://executor.sh/*" + ], + "methods": [ + { + "key": "mcp", + "transport": "mcp_remote", + "auth": "none", + "ownershipModes": [ + "dcr", + "customer" + ], + "whenToUse": "Use the provider-hosted connection for the quickest setup.", + "defaults": {}, + "guidanceMd": "Paste your Executor MCP URL. Sign in if required, or add a token or headers under Advanced authentication.", + "riskTier": "S3", + "label": "Connect MCP server", + "consoleLinks": { + "docs": "https://executor.sh/docs/mcp-proxy" + } + } + ], + "docsUrl": "https://executor.sh/docs/mcp-proxy" +} diff --git a/packages/shared/src/feature-catalog.ts b/packages/shared/src/feature-catalog.ts index 879661d9dd..00e4945e9f 100644 --- a/packages/shared/src/feature-catalog.ts +++ b/packages/shared/src/feature-catalog.ts @@ -116,6 +116,14 @@ export const INSTANCE_FEATURE_CATALOG: Record { + it("requires an explicit MCP aggregators opt-in for self-hosted and managed instances", () => { + expect(instanceExperimentalSettingsSchema.parse({}).enableMcpAggregators).toBe(false); + expect(patchInstanceExperimentalSettingsSchema.parse({ enableMcpAggregators: true })).toEqual({ enableMcpAggregators: true }); + expect(patchInstanceExperimentalSettingsSchema.parse({})).not.toHaveProperty("enableMcpAggregators"); + expect(INSTANCE_FEATURE_CATALOG.enableMcpAggregators).toMatchObject({ tier: "managed", cloudDefault: false, selfHostedDefault: false }); + }); + for (const [provider, methodKey] of Object.entries(REMOTE_MCP_CONNECTOR_METHODS)) { + it(`${provider} has its own catalog and accepts URL plus explicit credentials`, () => { + const definition = getConnectableAppDefinition(provider)!; + expect(definition.slug).toBe(provider); + const method = definition.methods.find((method) => method.key === methodKey)!; + expect(method.transport).toBe("mcp_remote"); + if (provider !== "zapier") expect(method.ownershipModes).toContain("dcr"); + expect(connectToolAppSchema.safeParse({ galleryKey: provider, connectionMethodKey: methodKey, link: "https://example.com/mcp", authMode: "bearer", credentialValues: { "credentials.authorization": "test-secret", "headers.X-User-ID": "test-user" } }).success).toBe(true); + }); + } + it("keeps the legacy Composio API-key broker distinct", () => { + expect(isRemoteMcpConnectorMethod("composio", "api-key")).toBe(false); + expect(getConnectableAppDefinition("composio")?.methods.find((method) => method.key === "api-key")?.transport).toBe("rest_api"); + expect(connectToolAppSchema.safeParse({ galleryKey: "composio", connectionMethodKey: "api-key", authMode: "bearer" }).success).toBe(false); + }); + it("allows removing all agent access without changing the tool choices", () => { + expect(finishToolAppSchema.safeParse({ access: { agentIds: [] } }).success).toBe(true); + expect(finishToolAppSchema.safeParse({ access: { agentIds: ["not-an-agent-id"] } }).success).toBe(false); + }); + it("rejects unsafe header overrides and draft saving on unrelated connectors", () => { + expect(connectToolAppSchema.safeParse({ galleryKey: "arcade", connectionMethodKey: "mcp", credentialValues: { "headers.Host": "evil.example" } }).success).toBe(false); + expect(connectToolAppSchema.safeParse({ galleryKey: "github", saveDraft: true }).success).toBe(false); + }); +}); diff --git a/packages/shared/src/remote-mcp-connectors.ts b/packages/shared/src/remote-mcp-connectors.ts new file mode 100644 index 0000000000..a32e4d9fff --- /dev/null +++ b/packages/shared/src/remote-mcp-connectors.ts @@ -0,0 +1,15 @@ +/** Providers whose own MCP endpoint defines the catalog and authentication. */ +export const REMOTE_MCP_CONNECTOR_METHODS = { + zapier: "generated-url", + arcade: "mcp", + composio: "mcp", + executor: "mcp", +} as const; + +export type RemoteMcpConnectorId = keyof typeof REMOTE_MCP_CONNECTOR_METHODS; +export function isRemoteMcpConnectorId(value: unknown): value is RemoteMcpConnectorId { + return typeof value === "string" && Object.hasOwn(REMOTE_MCP_CONNECTOR_METHODS, value); +} +export function isRemoteMcpConnectorMethod(provider: unknown, method: unknown): boolean { + return isRemoteMcpConnectorId(provider) && method === REMOTE_MCP_CONNECTOR_METHODS[provider]; +} diff --git a/packages/shared/src/types/index.ts b/packages/shared/src/types/index.ts index d950a57212..f7e61aa8c4 100644 --- a/packages/shared/src/types/index.ts +++ b/packages/shared/src/types/index.ts @@ -608,6 +608,7 @@ export type { ToolConnectionTestAgentAccessResponse, ToolConnectionTestAgentsResponse, ToolConnectionTestCallResult, + ToolUpstreamPending, ToolConnectionTestCallStatus, ToolConnectionTestCallStatusPhase, } from "./tool-access.js"; diff --git a/packages/shared/src/types/instance.ts b/packages/shared/src/types/instance.ts index 2eb8da2094..5ed400e3b2 100644 --- a/packages/shared/src/types/instance.ts +++ b/packages/shared/src/types/instance.ts @@ -71,6 +71,8 @@ export interface InstanceExperimentalSettings { enableApps: boolean; /** Exposes chat connector setup and Board surfaces; existing delivery continues when hidden. */ enableChatConnectors: boolean; + /** Exposes MCP aggregator setup; existing connections keep running when hidden. */ + enableMcpAggregators: boolean; enablePipelines: boolean; enableCases: boolean; enableAgentChat: boolean; diff --git a/packages/shared/src/types/tool-access.ts b/packages/shared/src/types/tool-access.ts index db50522bca..527bf6f8de 100644 --- a/packages/shared/src/types/tool-access.ts +++ b/packages/shared/src/types/tool-access.ts @@ -1657,6 +1657,17 @@ export interface ToolConnectionTestAgentAccessResponse { access: ToolConnectionAccessSummary; } +export interface ToolUpstreamPending { + kind: "authorization" | "approval"; + links: Array<{ url: string; host: string; elicitationId?: string }>; + executionId?: string; + elicitationId?: string; + resumeTool?: string; + expiresAt?: string; + message?: string; + requestedSchema?: Record; +} + /** Result of `POST /tool-connections/:id/test-calls`. */ export interface ToolConnectionTestCallResult { decision: ToolConnectionTestDecision; @@ -1667,6 +1678,8 @@ export interface ToolConnectionTestCallResult { error?: { message: string; reasonCode: ToolAccessReasonCode | string | null }; /** Present (with `decision: "ask_first"`) — the parked approval request. */ actionRequestId?: string; + /** Provider handoff, distinct from a Paperclip permission approval. */ + upstreamPending?: ToolUpstreamPending; } /** @@ -1692,6 +1705,7 @@ export interface ToolConnectionTestCallStatus { parameters?: Record | null; /** Present once `phase === "done"` and the tool succeeded. */ result?: unknown; + upstreamPending?: ToolUpstreamPending; /** Present once `phase === "done"` and the tool failed, or when the request was denied/expired. */ error?: { message: string; reasonCode: ToolAccessReasonCode | string | null }; /** Wall-clock duration of the executed call in ms, when known. */ diff --git a/packages/shared/src/validators/instance.ts b/packages/shared/src/validators/instance.ts index 3380139e20..136386708a 100644 --- a/packages/shared/src/validators/instance.ts +++ b/packages/shared/src/validators/instance.ts @@ -52,6 +52,7 @@ export const instanceExperimentalSettingsSchema = z.object({ // configs continue to load during upgrades. enableApps: z.boolean().default(true), enableChatConnectors: z.boolean().default(false), + enableMcpAggregators: z.boolean().default(false), enablePipelines: z.boolean().default(false), enableCases: z.boolean().default(false), enableAgentChat: z.boolean().default(false), diff --git a/packages/shared/src/validators/tool-access.ts b/packages/shared/src/validators/tool-access.ts index f1da754604..3933aa3495 100644 --- a/packages/shared/src/validators/tool-access.ts +++ b/packages/shared/src/validators/tool-access.ts @@ -1,3 +1,4 @@ +import { isRemoteMcpConnectorMethod } from "../remote-mcp-connectors.js"; import { z } from "zod"; import { CONNECTION_TOKEN_ISSUANCE_PATHS, @@ -419,6 +420,7 @@ export const connectToolAppSchema = z.object({ resumeConnectionId: z.string().guid().optional(), /** Exact configured connection to reauthorize without replacing its identity. */ reconnectConnectionId: z.string().guid().optional(), + saveDraft: z.boolean().optional(), authMode: genericMcpAuthModeSchema.optional(), oauthClient: genericMcpOAuthClientSchema.optional(), credentialSource: z.enum(["paperclip_vault", "vercel_connect"]).optional(), @@ -438,7 +440,10 @@ export const connectToolAppSchema = z.object({ if ((value.grantKind === "agent") !== Boolean(value.subjectAgentId)) { ctx.addIssue({ code: z.ZodIssueCode.custom, path: ["subjectAgentId"], message: "subjectAgentId is required exactly for an agent grant" }); } - if (value.authMode && value.galleryKey) { + if (value.saveDraft && !isRemoteMcpConnectorMethod(value.galleryKey, value.connectionMethodKey)) { + ctx.addIssue({ code: z.ZodIssueCode.custom, path: ["saveDraft"], message: "Draft saving requires a remote MCP connector" }); + } + if (value.authMode && value.galleryKey && !isRemoteMcpConnectorMethod(value.galleryKey, value.connectionMethodKey)) { ctx.addIssue({ code: z.ZodIssueCode.custom, path: ["authMode"], @@ -490,7 +495,8 @@ export const finishToolAppSchema = z.object({ reviewedCatalogEntryIds: z.array(z.string().guid()).max(500).optional(), access: z.union([ z.literal("all_agents"), - z.object({ agentIds: z.array(z.string().guid()).min(1).max(250) }), + // Choosing specific agents may intentionally leave the connection unassigned. + z.object({ agentIds: z.array(z.string().guid()).max(250) }), ]), }); diff --git a/scripts/ingest-app-definitions.mjs b/scripts/ingest-app-definitions.mjs index 1e33b374e0..cc8ca176be 100644 --- a/scripts/ingest-app-definitions.mjs +++ b/scripts/ingest-app-definitions.mjs @@ -207,6 +207,16 @@ const apps = [ }, ), ], + ...[ + ["arcade", "Arcade", "https://api.arcade.dev/*", "https://docs.arcade.dev/en/operate/governance/mcp-gateways"], + ["executor", "Executor", "https://executor.sh/*", "https://executor.sh/docs/mcp-proxy"], + ].map(([slug, name, pattern, docsUrl]) => [ + slug, name, `Use the tools exposed by your ${name} MCP connection.`, "productivity", new URL(pattern).hostname, [pattern], + method("mcp", "mcp_remote", "none", {}, "S3", `Paste your ${name} MCP URL. Sign in if required, or add a token or headers under Advanced authentication.`, { + label: "Connect MCP server", ownershipModes: ["dcr", "customer"], consoleLinks: { docs: docsUrl }, + }), + { featured: true, docsUrl }, + ]), [ "railway", "Railway", @@ -693,11 +703,11 @@ const apps = [ [ "composio", "Composio", - "Connect Composio so Paperclip can discover and manage the toolkits in your project.", + "Discover and use connected apps through Composio Connect.", "productivity", "composio.dev", - ["https://backend.composio.dev/*"], - method( + ["https://backend.composio.dev/*", "https://connect.composio.dev/*", "https://mcp.composio.dev/*", "https://*.composio.dev/*"], + [method("mcp", "mcp_remote", "none", { serverUrl: "https://connect.composio.dev/mcp" }, "S3", "Sign in to Composio Connect, or paste an externally configured MCP session URL and headers.", { label: "Composio Connect", ownershipModes: ["dcr", "customer"] }), method( "api-key", "rest_api", "api_key", @@ -721,8 +731,8 @@ const apps = [ docs: "https://docs.composio.dev/reference/authenticating-to-composio/project-api-key-permissions", }, }, - ), - { featured: true }, + )], + { featured: true, docsUrl: "https://docs.composio.dev/docs/composio-connect" }, ], [ "oauth-generic", diff --git a/server/src/__tests__/generic-mcp-connection.test.ts b/server/src/__tests__/generic-mcp-connection.test.ts index 09fb4aae4f..77b7c16580 100644 --- a/server/src/__tests__/generic-mcp-connection.test.ts +++ b/server/src/__tests__/generic-mcp-connection.test.ts @@ -39,6 +39,7 @@ import { startEmbeddedPostgresTestDatabase, } from "./helpers/embedded-postgres.js"; import { toolAccessService } from "../services/tool-access.js"; +import { instanceSettingsService } from "../services/instance-settings.js"; import { ComposioApiError, type ComposioClient } from "../services/composio.js"; import { createComposioSessionManager } from "../services/composio-session-manager.js"; import { toolAccessPolicyService } from "../services/tool-access-policy.js"; @@ -468,6 +469,7 @@ describeEmbeddedPostgres("generic remote MCP connections", () => { }); it("keeps a generated Zapier URL attached to the curated Zapier identity", async () => { + await instanceSettingsService(db).updateExperimental({ enableMcpAggregators: true }); const secretUrl = "https://mcp.zapier.com/api/v1/connect?token=zapier-secret"; const publicUrl = "https://mcp.zapier.com/api/v1/connect"; const company = await createCompany(db); @@ -477,10 +479,12 @@ describeEmbeddedPostgres("generic remote MCP connections", () => { remoteHttpEndpointLookup: async () => [{ address: "8.8.8.8", family: 4 }], remoteHttpRequest: async (url, init) => { if (url === secretUrl && (init.method ?? "GET").toUpperCase() === "POST") { + const body = JSON.parse(String(init.body)); + if (body.method === "notifications/initialized") return new Response(null, { status: 202 }); return jsonResponse({ jsonrpc: "2.0", - id: "paperclip-catalog-refresh", - result: { tools: FIXTURE_TOOLS }, + id: body.id, + result: body.method === "initialize" ? { protocolVersion: "2025-06-18" } : { tools: FIXTURE_TOOLS }, }); } return jsonResponse({ error: "not_found" }, 404); diff --git a/server/src/__tests__/instance-settings-service.test.ts b/server/src/__tests__/instance-settings-service.test.ts index 38218a50e6..f5a3cfab06 100644 --- a/server/src/__tests__/instance-settings-service.test.ts +++ b/server/src/__tests__/instance-settings-service.test.ts @@ -43,6 +43,7 @@ describe("instance settings service", () => { enableApps: true, enableAgentChat: false, enableChatConnectors: false, + enableMcpAggregators: false, enableConferenceRoomChat: false, enableClassicTaskInterface: false, enableExternalObjects: false, diff --git a/server/src/__tests__/remote-mcp-connectors.test.ts b/server/src/__tests__/remote-mcp-connectors.test.ts new file mode 100644 index 0000000000..48b7bb7270 --- /dev/null +++ b/server/src/__tests__/remote-mcp-connectors.test.ts @@ -0,0 +1,251 @@ +import { randomUUID } from "node:crypto"; +import { mkdtemp, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterAll, beforeAll, describe, expect, it, vi } from "vitest"; +import { agents, heartbeatRuns, issues, toolCatalogEntries, toolConnectionInstalls, toolInvocations, companies, companyMemberships, createDb, toolConnections, toolPolicies, toolProfileEntries } from "@paperclipai/db"; +import { eq } from "drizzle-orm"; +import { startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js"; +import { toolAccessService } from "../services/tool-access.js"; +import { createToolGatewayService } from "../services/tool-gateway.js"; +import { instanceSettingsService, normalizeExperimentalSettings } from "../services/instance-settings.js"; +import express from "express"; +import request from "supertest"; +import { toolAccessRoutes } from "../routes/tool-access.js"; +const actor = { actorType: "user" as const, actorId: "mcp-test-user", actorSource: "local_implicit" as const }; +const tool = (name: string) => ({ name, description: name, inputSchema: { type: "object", properties: {} }, annotations: { readOnlyHint: true } }); +describe("remote connector lifecycle", () => { + let fixture: Awaited>; + let db: ReturnType; + let keyDir: string; + beforeAll(async () => { + keyDir = await mkdtemp(join(tmpdir(), "mcp-connector-secrets-")); + vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY_FILE", join(keyDir, "key")); + fixture = await startEmbeddedPostgresTestDatabase("mcp-connectors-test-"); + db = createDb(fixture.connectionString); + await instanceSettingsService(db).updateExperimental({ enableMcpAggregators: true }); + }); + afterAll(async () => { await fixture?.cleanup(); vi.unstubAllEnvs(); if (keyDir) await rm(keyDir, { recursive: true, force: true }); }); + async function company() { + const [row] = await db.insert(companies).values({ name: `MCP ${randomUUID()}`, issuePrefix: `M${randomUUID().slice(0, 5)}` }).returning(); + await db.insert(companyMemberships).values({ companyId: row.id, principalType: "user", principalId: actor.actorId, status: "active", membershipRole: "admin" }); + return row; + } + function remoteFixture() { + const requests: { url: string; headers: Headers }[] = []; + let added = false; + let removed = false; + const service = toolAccessService(db, { deploymentMode: "local_trusted", deploymentExposure: "private", + remoteHttpEndpointLookup: async () => [{ address: "8.8.8.8", family: 4 }], + remoteHttpRequest: async (url, init) => { + const body = JSON.parse(String(init.body)); const headers = new Headers(init.headers); + requests.push({ url, headers }); + if (body.method === "initialize") return Response.json({ id: body.id, jsonrpc: "2.0", result: { protocolVersion: "2025-06-18", capabilities: {} } }, { headers: { "Mcp-Session-Id": "session" } }); + expect(headers.get("mcp-session-id")).toBe("session"); + if (body.method === "notifications/initialized") return new Response(null, { status: 202 }); + expect(body.method).toBe("tools/list"); + return Response.json({ id: body.id, jsonrpc: "2.0", result: body.params?.cursor + ? { tools: [tool("ask"), ...(added ? [tool("new_tool")] : [])] } + : { tools: [tool("read"), ...(!removed ? [tool("off")] : [])], nextCursor: "page-two" } }); + }, + }); + return { service, requests, add: () => { added = true; }, remove: () => { removed = true; }, restore: () => { removed = false; } }; + } + it("defaults MCP aggregators off and rejects direct setup without provider requests or credential writes", async () => { + expect(normalizeExperimentalSettings({}).enableMcpAggregators).toBe(false); + const org = await company(); const remote = remoteFixture(); + await instanceSettingsService(db).updateExperimental({ enableMcpAggregators: false }); + try { + const app = express(); + app.use((req, _res, next) => { req.actor = { type: "board", userId: actor.actorId, source: "local_implicit", isInstanceAdmin: true }; next(); }); + app.use("/api", toolAccessRoutes(db, { paperclipCloudConnector: null })); + const gallery = await request(app).get(`/api/companies/${org.id}/tools/gallery`); + expect(gallery.status).toBe(200); + expect(gallery.body.apps.some((entry: { slug: string }) => ["zapier", "arcade", "composio", "executor"].includes(entry.slug))).toBe(false); + expect(gallery.body.apps.some((entry: { slug: string }) => entry.slug === "notion")).toBe(true); + for (const [galleryKey, connectionMethodKey] of [["zapier", "generated-url"], ["arcade", "mcp"], ["composio", "mcp"], ["executor", "mcp"]]) { + await expect(remote.service.connectGalleryApp(org.id, { galleryKey, connectionMethodKey, saveDraft: true }, actor)) + .rejects.toMatchObject({ status: 403, details: { code: "mcp_aggregators_disabled" } }); + } + await expect(remote.service.preflightGalleryAppMetadata("composio", "mcp")) + .rejects.toMatchObject({ status: 403 }); + expect(remote.requests).toHaveLength(0); + expect(await db.select().from(toolConnections).where(eq(toolConnections.companyId, org.id))).toHaveLength(0); + } finally { + await instanceSettingsService(db).updateExperimental({ enableMcpAggregators: true }); + } + }); + it("vaults generated URLs, paginates discovery, preserves Off/Ask during refresh and reconnect, and isolates companies", async () => { + const org = await company(); const other = await company(); const remote = remoteFixture(); + const input = { galleryKey: "zapier", connectionMethodKey: "generated-url", link: "https://mcp.zapier.com/api/v1/connect?token=fixture-secret", authMode: "none" as const }; + const connected = await remote.service.connectGalleryApp(org.id, input, actor); + expect(connected.catalog).toHaveLength(3); + expect(JSON.stringify(connected)).not.toContain("fixture-secret"); + expect(remote.requests.every((r) => r.url.includes("token=fixture-secret"))).toBe(true); + const ids = Object.fromEntries(connected.catalog.map((entry) => [entry.toolName, entry.id])); + const finished = await remote.service.finishGalleryAppConnection(org.id, connected.connectionId, { enabledCatalogEntryIds: [ids.read, ids.ask], askFirstCatalogEntryIds: [ids.ask], access: "all_agents" }, actor); + remote.add(); + const refreshed = await remote.service.refreshCatalog(connected.connectionId, actor, { enableAllByDefault: true }); + const newId = refreshed.catalog.find((entry) => entry.toolName === "new_tool")!.id; + const entries = await db.select().from(toolProfileEntries).where(eq(toolProfileEntries.profileId, finished.profile.id)); + expect(entries.map((entry) => entry.catalogEntryId)).toEqual(expect.arrayContaining([ids.read, ids.ask, newId])); + expect(entries.map((entry) => entry.catalogEntryId)).not.toContain(ids.off); + await expect(remote.service.connectGalleryApp(other.id, { ...input, reconnectConnectionId: connected.connectionId }, actor)).rejects.toThrow("not found"); + const reconnected = await remote.service.connectGalleryApp(org.id, { ...input, reconnectConnectionId: connected.connectionId }, actor); + const restored = await remote.service.finishGalleryAppConnection(org.id, connected.connectionId, { enabledCatalogEntryIds: reconnected.catalog.map((entry) => entry.id), askFirstCatalogEntryIds: [], access: "all_agents" }, actor); + expect(restored.connection.status).toBe("active"); + expect(restored.profileEntries.map((entry) => entry.catalogEntryId)).not.toContain(ids.off); + const policies = await db.select().from(toolPolicies).where(eq(toolPolicies.companyId, org.id)); + expect(policies.some((policy) => policy.enabled && policy.config.catalogEntryId === ids.ask)).toBe(true); + remote.remove(); + const afterRemoval = await remote.service.refreshCatalog(connected.connectionId, actor); + expect(afterRemoval.catalog.find((entry) => entry.toolName === "off")?.status).not.toBe("active"); + const [retired] = await db.select().from(toolCatalogEntries).where(eq(toolCatalogEntries.id, ids.off)); + expect(retired.status).toBe("disabled"); // Check persisted discovery, not just this refresh response. + remote.restore(); + const afterReturn = await remote.service.refreshCatalog(connected.connectionId, actor); + expect(afterReturn.catalog.find((entry) => entry.toolName === "off")?.status).toBe("active"); + const retainedEntries = await db.select().from(toolProfileEntries).where(eq(toolProfileEntries.profileId, finished.profile.id)); + expect(retainedEntries.map((entry) => entry.catalogEntryId)).not.toContain(ids.off); + await remote.service.archiveConnection(connected.connectionId, org.id, actor); + const [removed] = await db.select().from(toolConnections).where(eq(toolConnections.id, connected.connectionId)); + expect(removed.enabled).toBe(false); + expect(removed.status).toBe("archived"); + }); + it("renews expired discovery sessions and requires an explicit retry after an expired execution session", async () => { + const org = await company(); + const [agent] = await db.insert(agents).values({ companyId: org.id, name: "Session tester", role: "engineer", adapterType: "process", adapterConfig: {}, runtimeConfig: {} }).returning(); + let initialized = 0; + const expired = new Set(); + const calls: string[] = []; + const send = async (_url: string, init: RequestInit) => { + const body = JSON.parse(String(init.body)); + if (body.method === "initialize") return Response.json({ id: body.id, result: { protocolVersion: "2025-06-18" } }, { headers: { "Mcp-Session-Id": `session-${++initialized}` } }); + if (body.method === "notifications/initialized") return new Response(null, { status: 202 }); + const session = new Headers(init.headers).get("mcp-session-id")!; + if (body.method === "tools/call") calls.push(session); + if (expired.has(session)) return new Response(null, { status: 404 }); + return Response.json({ id: body.id, result: body.method === "tools/list" ? { tools: [tool("read")] } : { content: [{ type: "text", text: "ok" }] } }); + }; + const service = toolAccessService(db, { deploymentMode: "local_trusted", deploymentExposure: "private", remoteHttpEndpointLookup: async () => [{ address: "8.8.8.8", family: 4 }], remoteHttpRequest: send }); + const connected = await service.connectGalleryApp(org.id, { galleryKey: "arcade", connectionMethodKey: "mcp", link: "https://api.arcade.dev/mcp/expiration", authMode: "none" }, actor); + const beforeRefresh = initialized; + expired.add(`session-${initialized}`); + expect((await service.refreshCatalog(connected.connectionId, actor)).catalog).toHaveLength(1); + expect(initialized).toBe(beforeRefresh + 1); + await service.finishGalleryAppConnection(org.id, connected.connectionId, { enabledCatalogEntryIds: connected.catalog.map((entry) => entry.id), askFirstCatalogEntryIds: [], access: "all_agents" }, actor); + const gateway = createToolGatewayService(db, { deploymentMode: "local_trusted", deploymentExposure: "private", remoteHttpRequest: send }); + const call = () => gateway.executeTestCall({ companyId: org.id, connectionId: connected.connectionId, agentId: agent.id, userId: actor.actorId, toolName: "read", parameters: {} }); + expect(await call()).toMatchObject({ decision: "allowed", result: { data: { isError: false } } }); + const executionSession = calls[0]; + expired.add(executionSession); + const failed = await call(); + expect(failed).toMatchObject({ error: { reasonCode: "mcp_remote_status" } }); + expect(calls).toEqual([executionSession, executionSession]); + expect(await call()).toMatchObject({ decision: "allowed", result: { data: { isError: false } } }); + expect(calls).toHaveLength(3); + expect(calls[2]).not.toBe(executionSession); + }); + it("saves a vaulted draft without contacting the provider and resumes with custom headers", async () => { + const org = await company(); const remote = remoteFixture(); + const input = { galleryKey: "arcade", connectionMethodKey: "mcp", link: "https://api.arcade.dev/mcp/fixture", authMode: "bearer" as const }; + const draft = await remote.service.connectGalleryApp(org.id, { ...input, saveDraft: true, credentialValues: { "credentials.authorization": "fixture-key", "headers.Arcade-User-ID": "test-user" } }, actor); + expect(remote.requests).toHaveLength(0); + expect(JSON.stringify(draft)).not.toContain("fixture-key"); + const connected = await remote.service.connectGalleryApp(org.id, { ...input, resumeConnectionId: draft.connectionId }, actor); + expect(connected.catalog).toHaveLength(3); + expect(remote.requests.every((request) => request.headers.get("authorization") === "Bearer fixture-key" && request.headers.get("arcade-user-id") === "test-user")).toBe(true); + }); + it("does not widen an empty agent selection after an OAuth callback or reconnect", async () => { + const org = await company(); + const endpoint = "https://api.arcade.dev/mcp/fixture-oauth"; + const send = async (url: string, init: RequestInit) => { + if (url.includes(".well-known/oauth-protected-resource")) return Response.json({ resource: endpoint, authorization_servers: ["https://auth.arcade.dev"] }); + if (url.includes(".well-known/")) return Response.json({ issuer: "https://auth.arcade.dev", authorization_endpoint: "https://auth.arcade.dev/authorize", token_endpoint: "https://auth.arcade.dev/token", response_types_supported: ["code"], code_challenge_methods_supported: ["S256"], token_endpoint_auth_methods_supported: ["none"] }); + if (url === "https://auth.arcade.dev/token") return Response.json({ access_token: "fixture-access", token_type: "Bearer", expires_in: 3600 }); + if (url !== endpoint) throw new Error(`Unexpected fixture URL: ${url}`); + if (!new Headers(init.headers).has("authorization")) return new Response(null, { status: 401, headers: { "WWW-Authenticate": 'Bearer resource_metadata="https://api.arcade.dev/.well-known/oauth-protected-resource"' } }); + const body = JSON.parse(String(init.body)); + if (body.method === "notifications/initialized") return new Response(null, { status: 202 }); + return Response.json({ id: body.id, result: body.method === "initialize" ? { protocolVersion: "2025-06-18" } : { tools: [tool("read")] } }); + }; + const service = toolAccessService(db, { deploymentMode: "local_trusted", deploymentExposure: "private", remoteHttpEndpointLookup: async () => [{ address: "8.8.8.8", family: 4 }], remoteHttpRequest: send }); + const input = { galleryKey: "arcade", connectionMethodKey: "mcp", link: endpoint, authMode: "auto" as const, oauthClient: { clientId: "fixture-client" } }; + const connected = await service.connectGalleryApp(org.id, input, actor); + const callback = async () => { + const redirectUri = "http://127.0.0.1:3116/api/tools/oauth/callback"; + const started = await service.startOAuth(org.id, connected.connectionId, { redirectUri, actor }); + return service.completeOAuthCallback({ state: new URL(started.authorizationUrl).searchParams.get("state")!, code: "fixture-code", redirectUri, actor }); + }; + await service.putConnectionInstalls(connected.connectionId, { installs: [] }, actor); + const first = await callback(); + expect(first.connection.status).toBe("active"); + expect((await service.listConnectionInstalls(connected.connectionId))).toHaveLength(0); + await service.connectGalleryApp(org.id, { ...input, reconnectConnectionId: connected.connectionId }, actor); + await callback(); + expect((await service.listConnectionInstalls(connected.connectionId))).toHaveLength(0); + }); + it("enforces agent and action permissions before dispatch, and preserves provider resume after Paperclip approval", async () => { + const org = await company(); + const [allowed, denied] = await db.insert(agents).values(["Allowed", "Denied"].map((name) => ({ companyId: org.id, name, role: "engineer", adapterType: "process", adapterConfig: {}, runtimeConfig: {} }))).returning(); + const calls: { name: string; arguments: unknown; session: string | null }[] = []; + const send = async (_url: string, init: RequestInit) => { + const body = JSON.parse(String(init.body)); + if (body.method === "initialize") return Response.json({ id: body.id, result: { protocolVersion: "2025-06-18" } }, { headers: { "Mcp-Session-Id": "execution-session" } }); + if (body.method === "notifications/initialized") return new Response(null, { status: 202 }); + if (body.method === "tools/list") return Response.json({ id: body.id, result: { tools: [tool("execute"), tool("resume"), tool("off")] } }); + calls.push({ ...body.params, session: new Headers(init.headers).get("mcp-session-id") }); + return Response.json({ id: body.id, result: { structuredContent: body.params.name === "execute" + ? { status: "waiting_for_interaction", executionId: "fixture-execution", interaction: { kind: "form", message: "Approve read?", requestedSchema: { type: "object", properties: {} } } } + : { ok: true, resumed: body.params.arguments.executionId }, content: [] } }); + }; + const service = toolAccessService(db, { deploymentMode: "local_trusted", deploymentExposure: "private", remoteHttpEndpointLookup: async () => [{ address: "8.8.8.8", family: 4 }], remoteHttpRequest: send }); + const connection = await service.connectGalleryApp(org.id, { galleryKey: "executor", connectionMethodKey: "mcp", link: "https://executor.sh/test/mcp", authMode: "none" }, actor); + const ids = Object.fromEntries(connection.catalog.map((entry) => [entry.toolName, entry.id])); + await service.finishGalleryAppConnection(org.id, connection.connectionId, { enabledCatalogEntryIds: [ids.execute, ids.resume], askFirstCatalogEntryIds: [ids.execute], access: { agentIds: [allowed.id] } }, actor); + const gateway = createToolGatewayService(db, { deploymentMode: "local_trusted", deploymentExposure: "private", remoteHttpRequest: send, toolActionSigningSecret: "fixture-signing-key" }); + const call = (agentId: string, toolName: string, parameters: Record = {}) => gateway.executeTestCall({ companyId: org.id, connectionId: connection.connectionId, agentId, userId: actor.actorId, toolName, parameters }); + expect((await call(denied.id, "execute")).decision).toBe("off"); + expect((await call(allowed.id, "off")).decision).toBe("off"); + expect(calls).toHaveLength(0); + const asked = await call(allowed.id, "execute"); + expect(asked.decision).toBe("ask_first"); + expect(calls).toHaveLength(0); + if (!("actionRequestId" in asked)) throw new Error("Missing approval request"); + await gateway.approveActionRequest({ companyId: org.id, actionRequestId: asked.actionRequestId!, actor: { userId: actor.actorId } }); + const status = await gateway.getTestCallStatus({ companyId: org.id, connectionId: connection.connectionId, actionRequestId: asked.actionRequestId! }); + expect(status.phase).toBe("done"); + expect(status.upstreamPending).toMatchObject({ executionId: "fixture-execution", resumeTool: "resume" }); + const [invocation] = await db.select().from(toolInvocations).where(eq(toolInvocations.id, asked.invocationId)); + expect(invocation.resultSummary?.summary).toContain("fixture-execution"); + const resumed = await call(allowed.id, "resume", { executionId: status.upstreamPending!.executionId, action: "accept", content: "{}" }); + expect(resumed.decision).toBe("allowed"); + expect(calls.map((call) => call.name)).toEqual(["execute", "resume"]); + expect(calls.every((call) => call.session === "execution-session")).toBe(true); + const [issue] = await db.insert(issues).values({ companyId: org.id, title: "Provider approval", status: "in_progress", assigneeAgentId: allowed.id }).returning(); + const [run] = await db.insert(heartbeatRuns).values({ companyId: org.id, agentId: allowed.id, invocationSource: "assignment", status: "running", contextSnapshot: { issueId: issue.id } }).returning(); + const session = await gateway.createSession({ companyId: org.id, agentId: allowed.id, runId: run.id }); + const execute = (await gateway.listToolsForSession(session.token)).find((entry) => entry.upstreamToolName === "execute")!; + expect(execute).toBeDefined(); + let approvalId = ""; + try { await gateway.executeTool({ sessionToken: session.token, tool: execute.name, parameters: {} }); } + catch (error) { + expect(error).toMatchObject({ reasonCode: "approval_required" }); + approvalId = (error as { details: { actionRequestId: string } }).details.actionRequestId; + } + expect(approvalId).not.toBe(""); + await gateway.approveActionRequest({ companyId: org.id, actionRequestId: approvalId, actor: { userId: actor.actorId } }); + await expect(gateway.executeTool({ sessionToken: session.token, tool: execute.name, parameters: {} })).rejects.toMatchObject({ + reasonCode: "provider_interaction_required", details: { upstreamPending: { executionId: "fixture-execution" } }, + }); + expect(calls).toHaveLength(3); // Retrying the approved action did not start another execution. + await service.finishGalleryAppConnection(org.id, connection.connectionId, { enabledCatalogEntryIds: [ids.execute, ids.resume], askFirstCatalogEntryIds: [ids.execute], access: { agentIds: [] } }, actor); + expect(await db.select().from(toolConnectionInstalls).where(eq(toolConnectionInstalls.connectionId, connection.connectionId))).toHaveLength(0); + expect((await call(allowed.id, "execute")).decision).toBe("off"); + expect((await gateway.listToolsForSession(session.token)).filter((entry) => entry.connectionId === connection.connectionId)).toHaveLength(0); + await service.archiveConnection(connection.connectionId, org.id, actor); + await expect(call(allowed.id, "resume")).rejects.toThrow("not found"); + expect(calls).toHaveLength(3); + }); + +}); diff --git a/server/src/__tests__/remote-mcp-pending.test.ts b/server/src/__tests__/remote-mcp-pending.test.ts new file mode 100644 index 0000000000..2da80822e3 --- /dev/null +++ b/server/src/__tests__/remote-mcp-pending.test.ts @@ -0,0 +1,59 @@ +import { describe, expect, it } from "vitest"; +import { extractRemoteMcpPending } from "../services/remote-mcp-pending.js"; +import { classifyRisk } from "../services/tool-access.js"; + +describe("remote MCP provider handoffs", () => { + it("recognizes Arcade's JSON text authorization response before redaction", () => { + const result = { result: { isError: true, content: [{ type: "text", text: JSON.stringify({ authorization_url: "https://github.com/login/oauth/authorize?state=test", message: "Not executed; authorize first" }) }] } }; + expect(extractRemoteMcpPending(result, "arcade")).toMatchObject({ kind: "authorization", links: [{ host: "github.com", url: "https://github.com/login/oauth/authorize?state=test" }] }); + }); + it("recognizes nested Composio connection links without treating ordinary results as handoffs", () => { + expect(extractRemoteMcpPending({ data: { results: [{ redirect_url: "https://connect.composio.dev/link/test" }] } }, "composio", "COMPOSIO_MANAGE_CONNECTIONS")?.links).toHaveLength(1); + expect(extractRemoteMcpPending({ url: "https://example.com/article", execution_id: "completed" }, "executor")).toBeNull(); + expect(extractRemoteMcpPending({ redirect_url: "https://example.com/article" }, "unrelated")).toBeNull(); + }); + it("preserves URL elicitation identities and drops unsafe navigation targets", () => { + const elicitations = [ + { mode: "url", elicitationId: "consent-1", url: "https://provider.example/approve" }, + { mode: "url", elicitationId: "consent-2", url: "javascript:alert(1)" }, + { mode: "url", elicitationId: "consent-3", url: "https://user:password@example.com" }, + ]; + const pending = extractRemoteMcpPending({ error: { code: -32042, data: { elicitations } } }); + expect(pending?.links).toEqual([{ url: "https://provider.example/approve", host: "provider.example", elicitationId: "consent-1" }]); + expect(pending?.elicitationId).toBe("consent-1"); + }); + it("keeps an Executor execution identity for manual resume, without inventing a retry", () => { + expect(extractRemoteMcpPending({ status: "waiting_for_interaction", executionId: "run-42", interaction: { kind: "form", message: "Approve read?", requestedSchema: { type: "object", properties: {} } } }, "executor")).toMatchObject({ kind: "approval", executionId: "run-42", resumeTool: "resume" }); + }); + it("does not turn ordinary successful app data into an approval or authorization handoff", () => { + for (const provider of ["arcade", "composio", "executor"]) { + for (const status of ["suspended", "pending_approval", "awaiting_approval", "waiting_for_interaction"]) { + expect(extractRemoteMcpPending({ result: { structuredContent: { records: [{ status, executionId: "app-job", interaction: { kind: "form" }, authorization_url: "https://example.com/auth", redirect_url: "https://connect.composio.dev/link/test" }] } } }, provider, "get_records")).toBeNull(); + expect(extractRemoteMcpPending({ result: { structuredContent: { status } } }, provider, "get_record")).toBeNull(); + } + expect(extractRemoteMcpPending({ result: { structuredContent: { mode: "url", elicitationId: "app-field", url: "https://example.com" } } }, provider)).toBeNull(); + } + }); + it("classifies broad execution and resume as writes even without annotations", () => { + for (const name of ["execute", "resume", "edit-artifact"]) expect(classifyRisk({ name }, "executor")).toBe("write"); + expect(classifyRisk({ name: "skills", annotations: { readOnlyHint: true } }, "executor")).toBe("read"); + expect(classifyRisk({ name: "COMPOSIO_MULTI_EXECUTE_TOOL", annotations: { readOnlyHint: true } }, "composio")).toBe("write"); + }); + it("defaults unfamiliar and namespaced aggregator capabilities to writes despite read-only hints", () => { + for (const provider of ["executor", "composio", "arcade", "zapier"]) { + for (const name of ["vendor.execute", "custom_resume", "code", "workbench", "new_capability", "get_and_run_action"]) { + for (const annotations of [undefined, { readOnlyHint: true }, { readOnlyHint: false }]) { + expect(classifyRisk({ name, annotations }, provider)).toBe("write"); + } + } + expect(classifyRisk({ name: "delete_everything", annotations: { readOnlyHint: true } }, provider)).toBe("destructive"); + } + for (const [provider, name] of [["executor", "skills"], ["composio", "COMPOSIO_SEARCH_TOOLS"], ["arcade", "Github.GetRepository"]]) { + expect(classifyRisk({ name }, provider)).toBe("read"); + expect(classifyRisk({ name, annotations: { readOnlyHint: false } }, provider)).toBe("write"); + expect(classifyRisk({ name, annotations: { destructiveHint: true } }, provider)).toBe("destructive"); + expect(classifyRisk({ name: `custom.${name}`, annotations: { readOnlyHint: true } }, provider)).toBe("write"); + } + expect(classifyRisk({ name: "GITHUB_LIST_REPOSITORIES", annotations: { readOnlyHint: true } })).toBe("read"); + }); +}); diff --git a/server/src/__tests__/remote-mcp-protocol.test.ts b/server/src/__tests__/remote-mcp-protocol.test.ts new file mode 100644 index 0000000000..1167e514d4 --- /dev/null +++ b/server/src/__tests__/remote-mcp-protocol.test.ts @@ -0,0 +1,79 @@ +import { describe, expect, it, vi } from "vitest"; +import { forgetMcpHttpSessions, getMcpHttpSession, initializeMcpHttpSession, McpHttpInitializationError, readMcpHttpResponse } from "../services/mcp-http.js"; + +const encoder = new TextEncoder(); +function stream(chunks: string[], close = true) { + const cancel = vi.fn(); + const response = new Response(new ReadableStream({ start(controller) { for (const chunk of chunks) controller.enqueue(encoder.encode(chunk)); if (close) controller.close(); }, cancel }), { headers: { "content-type": "text/event-stream" } }); + return { response, cancel }; +} +function event(message: unknown) { return `data: ${JSON.stringify(message)}\r\n\r\n`; } + +describe("remote connector Streamable HTTP", () => { + it("matches the requested ID after progress and unrelated messages without waiting for stream closure", async () => { + const fixture = stream([event({ jsonrpc: "2.0", method: "notifications/progress", params: { progress: 1 } }), event({ jsonrpc: "2.0", id: "other", result: {} }), event({ jsonrpc: "2.0", id: "call", result: { content: [] } })], false); + expect(await readMcpHttpResponse(fixture.response, "call")).toEqual({ jsonrpc: "2.0", id: "call", result: { content: [] } }); + expect(fixture.cancel).toHaveBeenCalledOnce(); + }); + it("decodes split CRLF and UTF-8 chunks", async () => { + const data = event({ id: 7, result: { text: "café" } }); + expect(await readMcpHttpResponse(stream([...data]).response, 7)).toMatchObject({ result: { text: "café" } }); + }); + it("rejects an unrelated JSON result and bounds streamed responses", async () => { + await expect(readMcpHttpResponse(Response.json({ id: "wrong", result: {} }), "call")).rejects.toThrow("message ID"); + await expect(readMcpHttpResponse(stream([event({ id: 1, result: "too much" })]).response, 1, { maxBytes: 8 })).rejects.toThrow("size limit"); + }); + it("applies matching, parse errors, and size limits to buffered HTTP transports", async () => { + const buffered = (body: string) => ({ headers: new Headers(), text: async () => body }) as Response; + expect(await readMcpHttpResponse(buffered('{"id":"call","result":{}}'), "call")).toMatchObject({ id: "call" }); + await expect(readMcpHttpResponse(buffered('{"id":"other","result":{}}'), "call")).rejects.toMatchObject({ reason: "malformed_response" }); + await expect(readMcpHttpResponse(buffered("not json"), "call")).rejects.toMatchObject({ reason: "invalid_json" }); + await expect(readMcpHttpResponse(buffered("too large"), "call", { maxBytes: 2 })).rejects.toMatchObject({ reason: "too_large" }); + }); + it("delivers server requests before the matching response", async () => { + const onRequest = vi.fn(async () => {}); + const request = { jsonrpc: "2.0", id: "auth", method: "elicitation/create", params: { mode: "url", url: "https://example.com/auth", elicitationId: "consent" } }; + await readMcpHttpResponse(stream([event(request), event({ id: "call", result: {} })]).response, "call", { onRequest }); + expect(onRequest).toHaveBeenCalledWith(request); + }); + it("preserves an initialization authentication challenge for OAuth discovery", async () => { + const response = new Response(null, { status: 401, headers: { "www-authenticate": 'Bearer resource_metadata="https://example.com/.well-known/oauth-protected-resource"' } }); + try { await initializeMcpHttpSession({ requestId: "a", send: async () => response }); throw new Error("Expected challenge"); } + catch (error) { expect(error).toBeInstanceOf(McpHttpInitializationError); expect((error as McpHttpInitializationError).response).toBe(response); } + }); + it("initializes once per connection and identity and resets after revocation", async () => { + let count = 0; + const send = vi.fn(async (init: RequestInit) => { + const body = JSON.parse(String(init.body)); + if (body.method === "initialize") { count++; return Response.json({ jsonrpc: "2.0", id: body.id, result: { protocolVersion: "2025-06-18", capabilities: {} } }, { headers: { "Mcp-Session-Id": `session-${count}` } }); } + expect(body.method).toBe("notifications/initialized"); + return new Response(null, { status: 202 }); + }); + const input = { send, requestId: "a", scope: "connection-a:grant-a", headers: { Authorization: "Bearer secret-a" } }; + const [a, repeated] = await Promise.all([getMcpHttpSession(input), getMcpHttpSession(input)]); + expect(a).toEqual(repeated); + expect((await getMcpHttpSession(input))["Mcp-Session-Id"]).toBe("session-1"); + expect((await getMcpHttpSession({ ...input, scope: "connection-b:grant-a" }))["Mcp-Session-Id"]).toBe("session-2"); + expect((await getMcpHttpSession({ ...input, scope: "connection-a:grant-b" }))["Mcp-Session-Id"]).toBe("session-3"); + expect((await getMcpHttpSession({ ...input, headers: { Authorization: "Bearer rotated" } }))["Mcp-Session-Id"]).toBe("session-4"); + forgetMcpHttpSessions("connection-a"); + expect((await getMcpHttpSession(input))["Mcp-Session-Id"]).toBe("session-5"); + expect(count).toBe(5); + }); + it("does not resurrect an in-flight session after disconnect", async () => { + let finish: (() => void) | undefined; + const gate = new Promise((resolve) => { finish = resolve; }); + const send = async (init: RequestInit) => { + const body = JSON.parse(String(init.body)); + if (body.method === "initialize") { + await gate; + return Response.json({ id: body.id, result: { protocolVersion: "2025-06-18" } }, { headers: { "Mcp-Session-Id": "revoked" } }); + } + return new Response(null, { status: 202 }); + }; + const pending = getMcpHttpSession({ send, requestId: "revoking", scope: "revoking-connection:agent" }); + forgetMcpHttpSessions("revoking-connection"); + finish!(); + await expect(pending).rejects.toThrow("connection changed"); + }); +}); diff --git a/server/src/__tests__/remote-mcp-redaction.test.ts b/server/src/__tests__/remote-mcp-redaction.test.ts new file mode 100644 index 0000000000..65fa64b4ee --- /dev/null +++ b/server/src/__tests__/remote-mcp-redaction.test.ts @@ -0,0 +1,18 @@ +import { describe, expect, it } from "vitest"; +import { redactEventPayload, redactSensitiveText } from "../redaction.js"; + +describe("public Executor selectors and secret redaction", () => { + it("keeps only exact known public helper addresses readable in MCP results", () => { + const path = "executor.coreTools.integrations.list"; + expect(redactEventPayload({ path, text: `Call tools.${path}({})` })).toEqual({ path, text: `Call tools.${path}({})` }); + expect(redactSensitiveText(`${path}.privateSecretSuffix`)).toContain("REDACTED"); + expect(redactEventPayload({ path: "arbitrary.provider.path" })?.path).toContain("REDACTED"); + }); + it("still redacts secret fields, bearer values, and JWT-shaped strings", () => { + const value = "executor.coreTools.integrations.list"; + expect(redactEventPayload({ token: value })?.token).toContain("REDACTED"); + expect(redactSensitiveText(`Authorization: Bearer ${value}`)).not.toContain(value); + expect(redactSensitiveText(`TOKEN=${value}`)).not.toContain(value); + expect(redactSensitiveText("eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.signature12345678")).toContain("REDACTED"); + }); +}); diff --git a/server/src/__tests__/sentry.test.ts b/server/src/__tests__/sentry.test.ts index 5f34a1ff97..030029c9b0 100644 --- a/server/src/__tests__/sentry.test.ts +++ b/server/src/__tests__/sentry.test.ts @@ -546,6 +546,7 @@ describe("buildSentryInitOptions serverName", () => { describe("buildSentryInitOptions release", () => { const commit = "0123456789abcdef0123456789abcdef01234567"; + const readBuildCommit = vi.fn<() => string | null>(); const integrations = { httpIntegration: () => ({ name: "Http" }), onUnhandledRejectionIntegration: () => ({ name: "OnUnhandledRejection" }), @@ -553,12 +554,14 @@ describe("buildSentryInitOptions release", () => { beforeEach(() => { vi.stubEnv("SENTRY_RELEASE", ""); - vi.doMock("../build-commit.js", () => ({ readBuildCommit: () => commit })); + readBuildCommit.mockReturnValue(commit); + vi.doMock("../build-commit.js", () => ({ readBuildCommit })); }); afterEach(() => { vi.unstubAllEnvs(); vi.doUnmock("../build-commit.js"); + readBuildCommit.mockReset(); }); it("uses the server build commit", async () => { @@ -573,9 +576,12 @@ describe("buildSentryInitOptions release", () => { }); it("leaves an unknown build unattributed", async () => { - vi.doMock("../build-commit.js", () => ({ readBuildCommit: () => null })); + // Keep one module factory and change its return value explicitly for this + // case, rather than depending on a second factory replacing the first. + readBuildCommit.mockReturnValue(null); const { buildSentryInitOptions } = await importFreshSentry(); expect(buildSentryInitOptions("test-dsn", integrations).release).toBeUndefined(); + expect(readBuildCommit).toHaveBeenCalled(); }); }); diff --git a/server/src/__tests__/tool-access-service.test.ts b/server/src/__tests__/tool-access-service.test.ts index 4831c866e8..6b8356dd06 100644 --- a/server/src/__tests__/tool-access-service.test.ts +++ b/server/src/__tests__/tool-access-service.test.ts @@ -72,6 +72,7 @@ import { toolAccessService, } from "../services/tool-access.js"; import { accessService } from "../services/access.js"; +import { instanceSettingsService } from "../services/instance-settings.js"; import { toolAccessPolicyService } from "../services/tool-access-policy.js"; import { secretService } from "../services/secrets.js"; import { @@ -361,15 +362,12 @@ function mcpSseResponse(payload: unknown): Response { } function mockToolsList(tools: unknown[]) { - return vi - .spyOn(globalThis, "fetch") - .mockResolvedValue( - mcpHttpResponse({ - jsonrpc: "2.0", - id: "paperclip-catalog-refresh", - result: { tools }, - }), - ); + return vi.spyOn(globalThis, "fetch").mockImplementation(async (_url, init) => { + const body = JSON.parse(String(init?.body)); + if (body.method === "notifications/initialized") return new Response(null, { status: 202 }); + return mcpHttpResponse({ jsonrpc: "2.0", id: body.id, + result: body.method === "initialize" ? { protocolVersion: "2025-06-18" } : { tools } }); + }); } const PUBLIC_MCP_FIXTURE_URL = "https://8.8.8.8/api/mcp"; @@ -852,12 +850,14 @@ describeEmbeddedPostgres("tool access service", () => { process.env.PAPERCLIP_TOOL_ACCESS_TEST_DATABASE_URL?.trim(); if (externalDatabaseUrl) { db = createDb(externalDatabaseUrl); + await instanceSettingsService(db).updateExperimental({ enableMcpAggregators: true }); return; } tempDb = await startEmbeddedPostgresTestDatabase( "paperclip-tool-access-service-", ); db = createDb(tempDb.connectionString); + await instanceSettingsService(db).updateExperimental({ enableMcpAggregators: true }); }, 20_000); afterEach(async () => { @@ -3382,10 +3382,10 @@ describeEmbeddedPostgres("tool access service", () => { priority: 100, selectors: { connectionId: connection.id }, }); - const fetchMock = vi.spyOn(globalThis, "fetch").mockResolvedValue( + const fetchMock = vi.spyOn(globalThis, "fetch").mockImplementation(async (_url, init) => mcpHttpResponse({ jsonrpc: "2.0", - id: "paperclip-tool-test", + id: JSON.parse(String(init?.body)).id, result: { content: [{ type: "text", text: "sent" }] }, }), ); @@ -3674,10 +3674,10 @@ describeEmbeddedPostgres("tool access service", () => { expect(waiting.body.result).toBeUndefined(); // 3. Approving from the review queue is what runs the parked test call. - const fetchMock = vi.spyOn(globalThis, "fetch").mockResolvedValue( + const fetchMock = vi.spyOn(globalThis, "fetch").mockImplementation(async (_url, init) => mcpHttpResponse({ jsonrpc: "2.0", - id: "paperclip-tool-test", + id: JSON.parse(String(init?.body)).id, result: { content: [{ type: "text", text: "sent" }] }, }), ); @@ -3741,10 +3741,10 @@ describeEmbeddedPostgres("tool access service", () => { .send(body) .expect(200); - vi.spyOn(globalThis, "fetch").mockResolvedValue( + vi.spyOn(globalThis, "fetch").mockImplementation(async (_url, init) => mcpHttpResponse({ jsonrpc: "2.0", - id: "paperclip-tool-test", + id: JSON.parse(String(init?.body)).id, result: { content: [{ type: "text", text: "sent" }] }, }), ); @@ -5101,7 +5101,7 @@ describeEmbeddedPostgres("tool access service", () => { "youcom", ]), ); - expect(res.body.apps).toHaveLength(48); + expect(res.body.apps).toHaveLength(51); expect( res.body.apps.find((app: { slug: string }) => app.slug === "gmail") .ownershipAvailability, diff --git a/server/src/__tests__/tool-gateway-service.test.ts b/server/src/__tests__/tool-gateway-service.test.ts index 111251fe71..6d8420561f 100644 --- a/server/src/__tests__/tool-gateway-service.test.ts +++ b/server/src/__tests__/tool-gateway-service.test.ts @@ -1177,9 +1177,9 @@ describeEmbeddedPostgres("tool gateway service", () => { selectors: { riskLevel: "read" }, }); const originalFetch = globalThis.fetch; - globalThis.fetch = async () => new Response(JSON.stringify({ + globalThis.fetch = async (_url, init) => new Response(JSON.stringify({ jsonrpc: "2.0", - id: "paperclip-tool-test", + id: JSON.parse(String(init?.body)).id, result: { _meta: { elicitation: { @@ -1765,9 +1765,9 @@ describeEmbeddedPostgres("tool gateway service", () => { selectors: { riskLevel: "read" }, }); const originalFetch = globalThis.fetch; - globalThis.fetch = async () => new Response(JSON.stringify({ + globalThis.fetch = async (_url, init) => new Response(JSON.stringify({ jsonrpc: "2.0", - id: "paperclip-tool-test", + id: JSON.parse(String(init?.body)).id, result: { elicitation: { message: "Need input" }, content: [] }, }), { status: 200, headers: { "content-type": "application/json" } }); try { diff --git a/server/src/__tests__/tool-gateway.test.ts b/server/src/__tests__/tool-gateway.test.ts index 9c200f79b2..92ac265675 100644 --- a/server/src/__tests__/tool-gateway.test.ts +++ b/server/src/__tests__/tool-gateway.test.ts @@ -585,10 +585,10 @@ describeEmbeddedPostgres("tool gateway acceptance", () => { it("exposes a named gateway with scoped bearer-token auth and revocation", async () => { const company = await createCompany(db); - const remote = await startFakeRemoteMcpServer(async () => ({ + const remote = await startFakeRemoteMcpServer(async ({ body }) => ({ body: { jsonrpc: "2.0", - id: "test", + id: body?.id, result: { content: [{ type: "text", text: "read ok" }], structuredContent: { ok: true } }, }, })); @@ -1127,10 +1127,10 @@ describeEmbeddedPostgres("tool gateway acceptance", () => { it("rate limits public named gateway session setup, discovery, and calls with redacted audits", async () => { const company = await createCompany(db); - const remote = await startFakeRemoteMcpServer(async () => ({ + const remote = await startFakeRemoteMcpServer(async ({ body }) => ({ body: { jsonrpc: "2.0", - id: "test", + id: body?.id, result: { content: [{ type: "text", text: "read ok" }], structuredContent: { ok: true } }, }, })); @@ -1561,7 +1561,7 @@ describeEmbeddedPostgres("tool gateway acceptance", () => { if (upstreamCalls === 1) return new Response("unauthorized", { status: 401 }); return new Response(JSON.stringify({ jsonrpc: "2.0", - id: "fixture", + id: JSON.parse(String(init.body)).id, result: { content: [{ type: "text", text: "repo-a" }], structuredContent: { repositories: ["repo-a"] } }, }), { status: 200, headers: { "content-type": "application/json" } }); }, diff --git a/server/src/redaction.ts b/server/src/redaction.ts index f475bacc8f..34ed09ca27 100644 --- a/server/src/redaction.ts +++ b/server/src/redaction.ts @@ -1,4 +1,5 @@ import { redactCommandText } from "@paperclipai/adapter-utils"; +import { isPublicExecutorToolSelector } from "@paperclipai/adapter-utils/command-redaction"; const SECRET_FIELD_NAME_PATTERN = String.raw`[A-Za-z0-9_-]*(?:api[-_]?key|access[-_]?token|auth(?:_?token)?|token|authorization|bearer|secret|passwd|password|credential|jwt|private[-_]?key|cookie|connectionstring|browser[-_]?code|login[-_]?url)[A-Za-z0-9_-]*`; @@ -735,7 +736,7 @@ function sanitizeValue(value: unknown): unknown { // string leaf after validated protocol discriminators have had a chance to // opt in above in sanitizeRecord. if (typeof value === "string") { - return JWT_VALUE_RE.test(value) + return JWT_VALUE_RE.test(value) && !isPublicExecutorToolSelector(value) ? REDACTED_EVENT_VALUE : redactSensitiveText(value); } @@ -926,6 +927,7 @@ export function sanitizeRecord( if ( typeof value === "string" && JWT_VALUE_RE.test(value) && + !isPublicExecutorToolSelector(value) && !isPaperclipSchemaDiscriminator(key, value) ) { redacted[key] = REDACTED_EVENT_VALUE; diff --git a/server/src/routes/tool-access.ts b/server/src/routes/tool-access.ts index 34184f288e..e4a0c1b0a9 100644 --- a/server/src/routes/tool-access.ts +++ b/server/src/routes/tool-access.ts @@ -4,6 +4,7 @@ import { agents, companies, connectionGrants, issueThreadInteractions, toolConne import { and, eq, or } from "drizzle-orm"; import { APP_STORE_DEFINITIONS, + isRemoteMcpConnectorId, GITHUB_CONNECTOR_PROFILES, GOOGLE_WORKSPACE_CONNECTOR_PROFILES, isAgentStatusAssignableToWork, @@ -67,6 +68,7 @@ import { paperclipCloudConnectorCapabilitiesFromEnv, } from "../services/paperclip-cloud-connector.js"; import { runtimeCanonicalOrigin } from "../services/cloud-runtime-identity.js"; +import { instanceSettingsService } from "../services/instance-settings.js"; import { completePaperclipCloudConnectorEnrollment, loadPaperclipCloudConnectorIdentity, @@ -815,6 +817,7 @@ function connectorEnrollmentPrincipal(req: Request): string { ? await options.paperclipCloudConnector.getCapabilities() : []; const vercelConnect = vercelConnectIntegrationStatus(); + const { enableMcpAggregators } = await instanceSettingsService(db).getExperimental(); res.json({ capabilities: await describeConnectionCreateCapabilities(req, companyId), credentialSources: { @@ -830,7 +833,7 @@ function connectorEnrollmentPrincipal(req: Request): string { : "Vercel Connect setup is disabled on this Paperclip instance.", }, }, - apps: APP_STORE_DEFINITIONS.map((app) => + apps: APP_STORE_DEFINITIONS.filter((app) => enableMcpAggregators || !isRemoteMcpConnectorId(app.slug)).map((app) => appWithPaperclipCloudConnectorAvailability(app, advertisedProfiles) ), }); diff --git a/server/src/services/instance-settings.ts b/server/src/services/instance-settings.ts index b3d5749d86..19691b2590 100644 --- a/server/src/services/instance-settings.ts +++ b/server/src/services/instance-settings.ts @@ -234,6 +234,7 @@ export function normalizeExperimentalSettings(raw: unknown): InstanceExperimenta // continuing to accept the compatibility key in stored settings. enableApps: true, enableChatConnectors: parsed.data.enableChatConnectors ?? false, + enableMcpAggregators: parsed.data.enableMcpAggregators ?? false, enablePipelines: parsed.data.enablePipelines ?? false, enableCases: parsed.data.enableCases ?? false, enableAgentChat: parsed.data.enableAgentChat ?? false, @@ -275,6 +276,7 @@ export function normalizeExperimentalSettings(raw: unknown): InstanceExperimenta enableStreamlinedUi: true, enableApps: true, enableChatConnectors: false, + enableMcpAggregators: false, enablePipelines: false, enableCases: false, enableAgentChat: false, diff --git a/server/src/services/mcp-http.ts b/server/src/services/mcp-http.ts index 0f699018c7..3ae11aec37 100644 --- a/server/src/services/mcp-http.ts +++ b/server/src/services/mcp-http.ts @@ -1,3 +1,4 @@ +import { createHash } from "node:crypto"; // Helpers for talking to remote MCP servers over the Streamable HTTP transport. // // The MCP Streamable HTTP spec requires the client to advertise that it accepts @@ -15,6 +16,13 @@ export const MCP_HTTP_ACCEPT = "application/json, text/event-stream"; export const MCP_PROTOCOL_VERSION = "2025-06-18"; +export class McpHttpResponseError extends Error { + constructor(readonly reason: "invalid_json" | "malformed_response" | "too_large", message: string) { + super(message); + this.name = "McpHttpResponseError"; + } +} + /** * Default headers for an MCP Streamable HTTP JSON-RPC POST. Caller-supplied * headers (e.g. resolved credentials) are preserved, while the required @@ -33,6 +41,7 @@ export class McpHttpInitializationError extends Error { message: string, readonly stage: "initialize" | "initialized_notification", readonly status: number | null, + readonly response?: Response, ) { super(message); this.name = "McpHttpInitializationError"; @@ -40,9 +49,8 @@ export class McpHttpInitializationError extends Error { } /** - * Establish the short-lived Streamable HTTP session needed by stateful MCP - * servers. The returned headers belong only to the caller's next request; no - * session id is persisted with the connection or shared across operations. + * Establish a Streamable HTTP session. Callers may retain protocol headers in + * the in-memory cache scoped to the connection and effective credential identity. */ export async function initializeMcpHttpSession(input: { send: (init: RequestInit) => Promise; @@ -68,14 +76,12 @@ export async function initializeMcpHttpSession(input: { `Remote MCP initialization returned HTTP ${initializeResponse.status}`, "initialize", initializeResponse.status, + initializeResponse, ); } let payload: unknown; try { - payload = parseMcpHttpResponseBody( - await initializeResponse.text(), - initializeResponse.headers.get("content-type"), - ); + payload = await readMcpHttpResponse(initializeResponse, `${input.requestId}-initialize`); } catch { throw new McpHttpInitializationError("Remote MCP initialization returned an invalid response", "initialize", null); } @@ -83,6 +89,7 @@ export async function initializeMcpHttpSession(input: { ? (payload as { result?: unknown }).result : null; const resultRecord = result && typeof result === "object" ? result as Record : null; + if (!resultRecord) throw new McpHttpInitializationError("Remote MCP initialization failed", "initialize", null); const protocolVersion = typeof resultRecord?.protocolVersion === "string" && resultRecord.protocolVersion ? resultRecord.protocolVersion : MCP_PROTOCOL_VERSION; @@ -166,3 +173,97 @@ export function parseMcpHttpResponseBody(bodyText: string, contentType: string | if (lastError) throw lastError; throw new SyntaxError("MCP SSE response contained no data events"); } + +/** Read until the response for this request arrives, without waiting for an SSE + * connection to close. Notifications and responses for other IDs are ignored. */ +export async function readMcpHttpResponse( + response: Response, + requestId: string | number, + options: { maxBytes?: number; onRequest?: (message: Record) => Promise } = {}, +): Promise { + const maxBytes = options.maxBytes ?? 8 * 1024 * 1024; + const isStream = response.headers.get("content-type")?.toLowerCase().includes("text/event-stream"); + const reader = response.body?.getReader(); + // Injected HTTP transports can expose a buffered text response rather than a + // Web ReadableStream. Keep the same size and message-ID checks for both forms. + if (!reader) { + const body = await response.text(); + if (Buffer.byteLength(body, "utf8") > maxBytes) throw new McpHttpResponseError("too_large", "MCP response exceeded the size limit"); + return readMcpHttpResponse(new Response(body, { + headers: { "content-type": response.headers.get("content-type") ?? "application/json" }, + }), requestId, options); + } + const decoder = new TextDecoder(); + let buffer = ""; + let bytes = 0; + const parse = (text: string): unknown => { + try { return JSON.parse(text); } + catch { throw new McpHttpResponseError("invalid_json", "MCP response contained invalid JSON"); } + }; + const inspect = async (message: unknown): Promise => { + if (!message || typeof message !== "object") return undefined; + const record = message as Record; + if (record.id === requestId && ("result" in record || "error" in record)) return record; + if ("method" in record && "id" in record) await options.onRequest?.(record); + return undefined; + }; + const event = async (value: string) => { + const data = value.split("\n").filter((line) => line.startsWith("data:")).map((line) => line.slice(5).replace(/^ /, "")).join("\n"); + if (!data) return undefined; + return inspect(parse(data)); + }; + try { + while (true) { + const { value, done } = await reader.read(); + bytes += value?.byteLength ?? 0; + if (bytes > maxBytes) throw new McpHttpResponseError("too_large", "MCP response exceeded the size limit"); + buffer += decoder.decode(value, { stream: !done }); + if (isStream) { + // Normalize CRLF after concatenating chunks, including split CR/LF pairs. + buffer = buffer.replace(/\r\n/g, "\n"); + let boundary: number; + while ((boundary = buffer.indexOf("\n\n")) >= 0) { + const result = await event(buffer.slice(0, boundary)); + buffer = buffer.slice(boundary + 2); + if (result !== undefined) return result; + } + } + if (done) break; + } + const result = isStream ? await event(buffer) : await inspect(parse(buffer)); + if (result !== undefined) return result; + throw new McpHttpResponseError("malformed_response", "MCP response did not contain the requested message ID"); + } finally { + await reader.cancel().catch(() => undefined); + reader.releaseLock(); + } +} + +const sessions = new Map; expiresAt: number }>(); +const initializing = new Map>>(); +const SESSION_TTL_MS = 30 * 60_000; + +/** Cache only protocol headers; credential hashes and scope separate every + * connection and effective identity. Never cache a tool call or replay a write. */ +export async function getMcpHttpSession(input: Parameters[0] & { scope: string }) { + const key = `${input.scope}:${createHash("sha256").update(JSON.stringify(Object.entries(input.headers ?? {}).sort())).digest("hex")}`; + const cached = sessions.get(key); + if (cached && cached.expiresAt > Date.now()) return { ...input.headers, ...cached.headers }; + const pending = initializing.get(key); + if (pending) return pending; + const promise = initializeMcpHttpSession(input).then((headers) => { + if (initializing.get(key) !== promise) throw new Error("MCP connection changed while initializing; reconnect before calling tools"); + for (const [id, value] of sessions) if (value.expiresAt <= Date.now()) sessions.delete(id); + if (sessions.size >= 1000) sessions.delete(sessions.keys().next().value!); + const protocolHeaders = Object.fromEntries(Object.entries(headers).filter(([name]) => ["mcp-session-id", "mcp-protocol-version"].includes(name.toLowerCase()))); + sessions.set(key, { headers: protocolHeaders, expiresAt: Date.now() + SESSION_TTL_MS }); + return headers; + }).finally(() => { if (initializing.get(key) === promise) initializing.delete(key); }); + initializing.set(key, promise); + return promise; +} + +export function forgetMcpHttpSessions(connectionId: string) { + for (const key of sessions.keys()) if (key.startsWith(`${connectionId}:`)) sessions.delete(key); + for (const key of initializing.keys()) if (key.startsWith(`${connectionId}:`)) initializing.delete(key); +} diff --git a/server/src/services/remote-mcp-pending.ts b/server/src/services/remote-mcp-pending.ts new file mode 100644 index 0000000000..3b65bdb6ce --- /dev/null +++ b/server/src/services/remote-mcp-pending.ts @@ -0,0 +1,78 @@ +import { redactEventPayload, redactSensitiveText } from "../redaction.js"; +import { checkOAuthEndpointUrl, type ToolUpstreamPending } from "@paperclipai/shared"; + +function record(value: unknown): Record | null { + return value && typeof value === "object" && !Array.isArray(value) ? value as Record : null; +} + +/** Recognize protocol/provider envelopes, never generic app-data statuses. + * Links are navigation targets kept outside durable result/audit storage. */ +export function extractRemoteMcpPending(value: unknown, provider?: string | null, toolName?: string): ToolUpstreamPending | null { + const root = record(value); + if (!root) return null; + const links = new Map(); + let pending: ToolUpstreamPending | null = null; + const addLink = (url: unknown, id?: string) => { + const checked = checkOAuthEndpointUrl(url); + if (checked.ok && links.size < 8) links.set(checked.url, { url: checked.url, host: checked.host, ...(id ? { elicitationId: id } : {}) }); + }; + const urlElicitation = (value: unknown) => { + const item = record(value); + if (item?.mode !== "url" || typeof item.elicitationId !== "string") return; + pending ??= { kind: "authorization", links: [] }; + const id = item.elicitationId.slice(0, 512); + addLink(item.url, id); + if (links.size && !pending.elicitationId) pending.elicitationId = id; + }; + if (root.method === "elicitation/create") urlElicitation(root.params); + const error = record(root.error); + const elicitations = record(error?.data)?.elicitations; + if (error?.code === -32042 && Array.isArray(elicitations)) elicitations.slice(0, 8).forEach(urlElicitation); + + const result = record(root.result) ?? root; + const payloads: Record[] = [result]; + const structured = record(result.structuredContent); + if (structured) payloads.push(structured); + if (Array.isArray(result.content)) { + for (const item of result.content.slice(0, 100)) { + const content = record(item); + if (content?.type !== "text" || typeof content.text !== "string" || content.text.length > 512_000) continue; + try { const parsed = record(JSON.parse(content.text)); if (parsed) payloads.push(parsed); } catch { /* Ordinary text. */ } + } + } + for (const payload of payloads) { + if (provider === "executor" && payload.status === "waiting_for_interaction" + && typeof payload.executionId === "string" && payload.executionId) { + const interaction = record(payload.interaction); + if (interaction?.kind !== "form" && interaction?.kind !== "url") continue; + pending = { kind: "approval", links: [], executionId: payload.executionId.slice(0, 512), resumeTool: "resume" }; + if (typeof payload.expiresAt === "string" && Number.isFinite(Date.parse(payload.expiresAt))) pending.expiresAt = payload.expiresAt; + if (typeof interaction.message === "string") pending.message = redactSensitiveText(interaction.message).slice(0, 4000); + const schema = record(interaction.requestedSchema); + if (schema) pending.requestedSchema = redactEventPayload(schema) ?? undefined; + if (interaction.kind === "url") addLink(interaction.url); + } + if (provider === "arcade" && (result.isError === true || /(?:^|[._])ManageAuthorization$/.test(toolName ?? "")) + && typeof payload.authorization_url === "string") { + addLink(payload.authorization_url); + pending ??= { kind: "authorization", links: [] }; + } + // This tool returns connection handoffs keyed by app. Other Composio tools + // may return arbitrary application data with redirect_url/status fields. + if (provider === "composio" && toolName === "COMPOSIO_MANAGE_CONNECTIONS") { + let visited = 0; + const visit = (item: unknown, depth: number) => { + if (++visited > 500 || depth > 10 || !item || typeof item !== "object") return; + const entry = record(item); + const checked = checkOAuthEndpointUrl(entry?.redirect_url); + if (checked.ok && checked.host === "connect.composio.dev" && new URL(checked.url).pathname.startsWith("/link/")) { + addLink(checked.url); + pending ??= { kind: "authorization", links: [] }; + } + for (const child of Object.values(item).slice(0, 100)) visit(child, depth + 1); + }; + visit(payload, 0); + } + } + return pending ? { ...pending, links: [...links.values()] } : null; +} diff --git a/server/src/services/tool-access-policy.ts b/server/src/services/tool-access-policy.ts index 5eb0cf2d36..e09d1b2872 100644 --- a/server/src/services/tool-access-policy.ts +++ b/server/src/services/tool-access-policy.ts @@ -1183,6 +1183,11 @@ export function toolAccessPolicyService(db: Db) { const { ctx, redaction } = loaded; const profileState = await effectiveProfiles(ctx); const effectiveProfileIds = profileState.profiles.map((profile) => profile.id); + const permittedByProfile = profileState.profiles.some((profile) => { + const matchingEntries = profileState.entries.filter((entry) => entry.profileId === profile.id && profileEntryMatches(entry, ctx)); + return !matchingEntries.some((entry) => entry.effect === "exclude") + && (profile.defaultAction === "allow" || matchingEntries.some((entry) => entry.effect === "include")); + }); const policies = await db.select().from(toolPolicies).where(and(eq(toolPolicies.companyId, ctx.companyId), eq(toolPolicies.enabled, true))).orderBy(asc(toolPolicies.priority), asc(toolPolicies.createdAt)); for (const policy of policies) { const conditions = policyConditions(policy); @@ -1276,6 +1281,9 @@ export function toolAccessPolicyService(db: Db) { return decision("allow", "allow_trust_rule", policy.description ?? "Tool access allowed by trust rule.", effectiveProfileIds, [policy.id], { redactionPlan: redaction.redactionPlan, policyExplanation }); } if (policy.policyType === "require_approval") { + // The connection's Ask first control restricts an existing action grant; + // it must never grant access to agents outside that connection's profile. + if (policy.config?.source === "app_gallery_finish" && !permittedByProfile) continue; return decision("require_approval", "requires_approval_policy", policy.description ?? "Tool access requires approval.", effectiveProfileIds, [policy.id], { redactionPlan: redaction.redactionPlan, policyExplanation }); } if (policy.policyType === "allow") { diff --git a/server/src/services/tool-access.ts b/server/src/services/tool-access.ts index e48dbec1cd..bf07307ce0 100644 --- a/server/src/services/tool-access.ts +++ b/server/src/services/tool-access.ts @@ -1,4 +1,5 @@ -import { connectionPurposeTransportSchema } from "@paperclipai/shared"; +import { isRemoteMcpConnectorMethod, connectionPurposeTransportSchema } from "@paperclipai/shared"; +import { instanceSettingsService } from "./instance-settings.js"; import { syncConnectionCredentialBindings } from "./connection-credential-bindings.js"; import { canBrowseProjectRepositoryGrant, mergeProjectRepository } from "./project-repositories.js"; import { captureRunIdentity } from "./run-identity.js"; @@ -186,6 +187,10 @@ import { logger } from "../middleware/logger.js"; import { logActivity } from "./activity-log.js"; import { initializeMcpHttpSession, + McpHttpInitializationError, + getMcpHttpSession, + forgetMcpHttpSessions, + readMcpHttpResponse, mcpHttpRequestHeaders, parseMcpHttpResponseBody, } from "./mcp-http.js"; @@ -2316,6 +2321,17 @@ function normalizedProviderToolName(toolName: string): string { .replace(/[:._-]+/g, "-"); } +// Aggregators can add arbitrarily powerful tools without changing their MCP +// endpoint. Only these reviewed, exact capabilities are read-only. Unknown or +// renamed actions remain enabled by the usual access rules, but have write risk. +// Legacy Composio child connections do not use these gallery template keys. +const AGGREGATOR_READ_TOOLS = new Map>([ + ["executor", new Set(["skills"])], + ["composio", new Set(["COMPOSIO_SEARCH_TOOLS", "COMPOSIO_GET_TOOL_SCHEMAS", "COMPOSIO_SEARCH_SKILLS", "COMPOSIO_USE_SKILL"])], + ["arcade", new Set(["Github.GetRepository"])], + ["zapier", new Set()], +]); + export function classifyRisk( tool: McpToolDescriptor, sourceTemplateKey?: string | null, @@ -2324,6 +2340,12 @@ export function classifyRisk( if (annotations.destructiveHint === true || annotations.destructive === true) return "destructive"; const normalizedToolName = normalizedProviderToolName(tool.name); + const reviewedReads = AGGREGATOR_READ_TOOLS.get(sourceTemplateKey ?? ""); + if (reviewedReads) { + if (verbMatches(tool.name, "delete|remove|destroy|unpublish")) return "destructive"; + if (annotations.readOnlyHint === false || annotations.writeHint === true) return "write"; + return reviewedReads.has(tool.name) ? "read" : "write"; + } if (sourceTemplateKey === "railway") { const reviewed = railwayRisk(normalizedToolName); return reviewed === "read" && (annotations.readOnlyHint === false || annotations.writeHint === true) ? "write" : reviewed; @@ -6068,6 +6090,7 @@ export function toolAccessService( removalOptions: { confirmComposioChildren?: boolean } = {}, ): Promise { const connection = await getConnectionRow(connectionId, companyId); + forgetMcpHttpSessions(connection.id); const now = new Date(); const binding = actorBinding(actor); @@ -6765,31 +6788,26 @@ export function toolAccessService( // Pinned to the address the guard approved: `config.url` is operator-supplied, // so a second DNS resolution here would reopen the rebinding window that // PAP-17098 closed for the OAuth endpoints. - const listRequestBody = JSON.stringify({ - jsonrpc: "2.0", - id: "paperclip-catalog-refresh", - method: "tools/list", - params: {}, - }); - const sendRemote = (init: RequestInit) => - requestRemoteHttpEndpoint(new URL(endpoint), init); - const sendToolsList = (requestHeaders: Record) => - sendRemote({ - method: "POST", - // MCP Streamable HTTP requires advertising that we accept both a JSON body - // and an SSE stream; spec-compliant servers 406 without it (see mcp-http.ts). - headers: mcpHttpRequestHeaders(requestHeaders), - body: listRequestBody, - }); + let listRequestId = "paperclip-catalog-refresh"; + let sessionHeaders = headers; + const sendRemote = (init: RequestInit) => requestRemoteHttpEndpoint(new URL(endpoint), init); + const sendToolsList = (requestHeaders: Record, cursor?: string) => { + sessionHeaders = requestHeaders; + return sendRemote({ method: "POST", headers: mcpHttpRequestHeaders(requestHeaders), + body: JSON.stringify({ jsonrpc: "2.0", id: listRequestId, method: "tools/list", params: cursor ? { cursor } : {} }) }); + }; let usedInitializedSession = connection.config.mcpSessionRequired === true; let response: Response; if (usedInitializedSession) { - const sessionHeaders = await initializeMcpHttpSession({ - send: sendRemote, - headers, - requestId: "paperclip-catalog-refresh", - }); - response = await sendToolsList(sessionHeaders); + try { + sessionHeaders = await getMcpHttpSession({ send: sendRemote, headers, + scope: `${connection.id}:catalog:${actor?.actorType}:${actor?.actorId}:${endpoint}`, + requestId: listRequestId }); + response = await sendToolsList(sessionHeaders); + } catch (error) { + if (!(error instanceof McpHttpInitializationError) || !error.response) throw error; + response = error.response; + } } else { response = await sendToolsList(headers); // `tools/list` is read-only, so a 400 can safely be retried after the MCP @@ -6810,6 +6828,17 @@ export function toolAccessService( } } } + if (response.status === 404 && new Headers(sessionHeaders).has("mcp-session-id")) { + // MCP uses 404 for an expired server session. Discovery is read-only, so + // discard the stale session and retry it once with a new handshake. + forgetMcpHttpSessions(connection.id); + await response.body?.cancel().catch(() => undefined); + sessionHeaders = await getMcpHttpSession({ send: sendRemote, headers, + scope: `${connection.id}:catalog:${actor?.actorType}:${actor?.actorId}:${endpoint}`, + requestId: listRequestId }); + response = await sendToolsList(sessionHeaders); + if (response.status === 404) forgetMcpHttpSessions(connection.id); + } if ( usedInitializedSession && connection.config.mcpSessionRequired !== true @@ -6833,6 +6862,8 @@ export function toolAccessService( const refreshed = await composioSessions.ensureSession(connection.id, { force: true, }); + listRequestId = "paperclip-catalog-refresh-retry"; + sessionHeaders = refreshed.headers; response = await requestRemoteHttpEndpoint(new URL(refreshed.url), { method: "POST", headers: mcpHttpRequestHeaders(refreshed.headers), @@ -6954,20 +6985,23 @@ export function toolAccessService( status: response.status, }); } - const payload = parseMcpHttpResponseBody( - await response.text(), - response.headers.get("content-type"), - ); - const result = asRecord(asRecord(payload).result); - const payloadTools = asRecord(payload).tools; - const tools: unknown[] = Array.isArray(result.tools) - ? result.tools - : Array.isArray(payloadTools) - ? payloadTools - : []; - const descriptors = tools - .map((tool) => normalizeToolDescriptor(tool)) - .filter((tool): tool is McpToolDescriptor => Boolean(tool)); + const descriptors: McpToolDescriptor[] = []; + const seenCursors = new Set(); + for (let page = 0; ; page += 1) { + const payload = await readMcpHttpResponse(response, listRequestId); + const record = asRecord(payload); + if (record.error) throw new HttpError(502, "Remote MCP tool discovery failed", { code: "mcp_catalog_error" }); + const result = asRecord(record.result); + if (!Array.isArray(result.tools)) throw new HttpError(502, "Remote MCP returned an invalid tool catalog", { code: "mcp_catalog_invalid" }); + descriptors.push(...result.tools.map((tool) => normalizeToolDescriptor(tool)).filter((tool): tool is McpToolDescriptor => Boolean(tool))); + const cursor = typeof result.nextCursor === "string" ? result.nextCursor : null; + if (!cursor) break; + if (seenCursors.has(cursor) || page >= 99 || descriptors.length > 20_000) throw new HttpError(502, "Remote MCP tool catalog pagination did not finish", { code: "mcp_catalog_pagination" }); + seenCursors.add(cursor); + listRequestId = `paperclip-catalog-refresh-${page + 1}`; + response = await sendToolsList(sessionHeaders, cursor); + if (!response.ok) throw new HttpError(502, "Remote MCP catalog page could not be read", { status: response.status }); + } if (!isRailwayConnection(connection)) return descriptors; if (descriptors.some((tool) => normalizeRailwayToolName(tool.name).startsWith(RAILWAY_TOOL_PREFIX))) { throw unprocessable("Railway advertised a reserved Paperclip action name. Refresh is blocked pending review.", { code: "railway_tool_name_collision" }); @@ -7015,7 +7049,7 @@ export function toolAccessService( connection: typeof toolConnections.$inferSelect, ): boolean { return ( - asRecord(connection.config).sourceTemplateKey === COMPOSIO_GALLERY_KEY + asRecord(connection.config).sourceTemplateKey === COMPOSIO_GALLERY_KEY && connection.transport === "rest_api" ); } @@ -7772,6 +7806,13 @@ export function toolAccessService( const existingByName = new Map( existingRows.map((entry) => [entry.toolName, entry]), ); + if (isRemoteMcpConnectorMethod(connection.config.sourceTemplateKey, connection.config.connectionMethodKey)) { + const discoveredNames = new Set(descriptors.map((descriptor) => descriptor.name)); + const removedIds = existingRows.filter((entry) => !discoveredNames.has(entry.toolName) && entry.status !== "disabled").map((entry) => entry.id); + if (removedIds.length) await db.update(toolCatalogEntries) + .set({ status: "disabled", quarantineReason: "mcp_tool_removed", updatedAt: refreshedAt }) + .where(and(eq(toolCatalogEntries.companyId, connection.companyId), eq(toolCatalogEntries.connectionId, connection.id), inArray(toolCatalogEntries.id, removedIds))); + } // Retired native actions are absent from discovery, but old catalog rows // still need to show as disabled. Gateway denial also applies before refresh. const blockedRailwayEntryIds = isRailwayEndpoint(connection.config.url) @@ -7835,7 +7876,7 @@ export function toolAccessService( ? "disabled" : shouldQuarantine ? "quarantined" - : existing?.status === "disabled" + : existing?.status === "disabled" && existing.quarantineReason !== "mcp_tool_removed" ? "disabled" : quarantineOnRefresh && existing?.status === "quarantined" ? "quarantined" @@ -7950,10 +7991,12 @@ export function toolAccessService( const activeEntries = updatedEntries.filter( (entry) => entry.status === "active", ); - if (!refreshOptions.skipDefaultProfileSync) { + const preserveMcpAccess = connection.config.mcpPreserveAccess === true + && isRemoteMcpConnectorMethod(connection.config.sourceTemplateKey, connection.config.connectionMethodKey); + if (!refreshOptions.skipDefaultProfileSync || preserveMcpAccess) { await enableCatalogEntriesByDefault({ connection: updatedConnection, - newCatalogEntryIds: refreshOptions.enableAllByDefault + newCatalogEntryIds: refreshOptions.enableAllByDefault && !isRemoteMcpConnectorMethod(connection.config.sourceTemplateKey, connection.config.connectionMethodKey) ? activeEntries.map((entry) => entry.id) : activeEntries .filter((entry) => { @@ -7962,7 +8005,7 @@ export function toolAccessService( }) .map((entry) => entry.id), activeCatalogEntryIds: activeEntries.map((entry) => entry.id), - restoreDraftDefaults: refreshOptions.restoreDraftDefaults, + restoreDraftDefaults: refreshOptions.restoreDraftDefaults || preserveMcpAccess, actor, }); } @@ -12231,6 +12274,15 @@ export function toolAccessService( return `${base.slice(0, 151).trimEnd()} (${randomUUID().slice(0, 6)})`; } + async function assertMcpAggregatorSetupEnabled(provider: unknown, method: unknown) { + if (isRemoteMcpConnectorMethod(provider, method) + && !(await instanceSettingsService(db).getExperimental()).enableMcpAggregators) { + throw forbidden("Enable MCP aggregators in Settings → Experimental to set up this connection", { + code: "mcp_aggregators_disabled", + }); + } + } + async function connectGalleryApp( companyId: string, input: ConnectToolApp, @@ -12377,18 +12429,20 @@ export function toolAccessService( const method = galleryEntry ? connectionMethodFor(galleryEntry, inferredMethodKey) : null; + const remoteMcpConnector = isRemoteMcpConnectorMethod(galleryEntry?.slug, method?.key); + await assertMcpAggregatorSetupEnabled(galleryEntry?.slug, method?.key); if (galleryEntry && input.link) { const acceptsProviderGeneratedUrl = method?.transport === "mcp_remote" && method.auth === "none" && !method.defaults?.serverUrl && !method.defaults?.serverUrlTemplate; - if (!acceptsProviderGeneratedUrl) { + if (!acceptsProviderGeneratedUrl && !remoteMcpConnector) { throw badRequest( `${galleryEntry.name} does not accept a provider-generated connection URL`, ); } - if (!getAppDefinitionForUrl(input.link, [galleryEntry])) { + if (!(remoteMcpConnector && galleryEntry.slug === "executor") && !getAppDefinitionForUrl(input.link, [galleryEntry])) { throw badRequest( `That connection URL does not belong to ${galleryEntry.name}`, ); @@ -12699,6 +12753,7 @@ export function toolAccessService( transport === "mcp_remote" ? { url: + (remoteMcpConnector ? remoteUrlCredential?.publicUrl : undefined) ?? normalizedMethodConfig?.url ?? method?.defaults?.serverUrl ?? remoteUrlCredential?.publicUrl ?? @@ -12716,6 +12771,11 @@ export function toolAccessService( // the wizard projects the app's action defaults into policies at // finish time instead of using catalog quarantine as access state. quarantineNewEntries: galleryEntry.slug === "railway", + ...(remoteMcpConnector ? { + mcpSessionRequired: true, + mcpAuthMode: input.authMode ?? "auto", + mcpPreserveAccess: Boolean(retainedConnection && (retainedConnection.status === "active" || asRecord(retainedConnection.config).mcpPreserveAccess === true)), + } : {}), ...(galleryEntry.slug === "posthog" ? { safeDefault: true } : {}), } : { ...baseConfig, quarantineNewEntries: false, unverifiedServer: true }; @@ -12734,7 +12794,7 @@ export function toolAccessService( config.quarantineNewEntries = true; } const acceptsCustomerOAuthClient = - method?.auth === "oauth" && method.ownershipModes.includes("customer"); + remoteMcpConnector || (method?.auth === "oauth" && method.ownershipModes.includes("customer")); if (galleryEntry && input.oauthClient && !acceptsCustomerOAuthClient) { throw badRequest( `${galleryEntry.name} does not accept customer-owned OAuth client credentials`, @@ -12784,7 +12844,7 @@ export function toolAccessService( // operator supplied and is upgraded to `oauth` when discovery proves the // endpoint needs sign-in (see `remoteTools` and `startOAuth`). const genericAuthKind: ToolConnectionAuthKind = - method?.auth ?? + (remoteMcpConnector ? undefined : method?.auth) ?? (input.authMode === "oauth" || input.oauthClient ? "oauth" : input.authMode === "bearer" || input.authMode === "custom_headers" @@ -12834,7 +12894,8 @@ export function toolAccessService( previousGrantKind === requestedGrantKind && galleryEntry && retainedSource === galleryEntry.slug && - retainedMethodKey === method?.key, + retainedMethodKey === method?.key && + (!remoteMcpConnector || (baseConfig.url === retainedConfig.url && genericAuthKind === retainedConnection.authKind)), ); const retainedCredentialSecretRefs = canRetainCredentialMaterial ? (retainedPersonalIdentity?.grant?.credentialSecretRefs ?? @@ -12866,9 +12927,9 @@ export function toolAccessService( const credentialFields = credentialSource === "vercel_connect" ? [] - : galleryEntry + : galleryEntry && !remoteMcpConnector ? credentialFieldsFor(galleryEntry, method?.key) - : linkCredentialFields(credentialValues); + : linkCredentialFields({ ...Object.fromEntries(retainedCredentialSecretRefs.filter((ref) => ref.configPath.startsWith("headers.") || ref.configPath === "credentials.authorization").map((ref) => [ref.configPath, "retained"])), ...credentialValues }); for (const field of credentialFields) { const value = credentialValues[field.configPath]; const retainedSecretRef = retainedCredentialSecretRefs.find( @@ -12923,6 +12984,13 @@ export function toolAccessService( } } + if (!remoteUrlCredential?.secretUrl && canRetainCredentialMaterial && baseConfig.url === retainedConfig.url) { + const retainedUrl = retainedCredentialSecretRefs.find((ref) => ref.configPath === REMOTE_URL_SECRET_CONFIG_PATH); + if (retainedUrl) { + credentialSecretRefs.push(retainedUrl); + credentialRefs.push({ name: REMOTE_URL_SECRET_CONFIG_PATH, secretId: retainedUrl.secretId, version: retainedUrl.versionSelector ?? "latest", placement: "url", key: "url", prefix: null }); + } + } if (remoteUrlCredential?.secretUrl) { const secret = await secrets.create( companyId, @@ -13072,6 +13140,7 @@ export function toolAccessService( const connectionCredentialSecretRefs = personalIdentityUserId || dedicatedAgentId ? [] : credentialSecretRefs; if (revivedConnectionPrevious) { + forgetMcpHttpSessions(revivedConnectionPrevious.id); [connectionRow] = await db .update(toolConnections) .set({ @@ -13292,6 +13361,10 @@ export function toolAccessService( ); await ensureRuntimeSlot(connectionRow); + if (input.saveDraft && remoteMcpConnector) { + return { connectionId: connectionRow.id, application: toApplication(applicationRow), connection: toConnection(connectionRow), + catalog: [], actions: { readOnly: [], canMakeChanges: [] }, suggestedDefaults: { access: "all_agents", askFirstRiskLevels: [] } }; + } if (galleryEntry && method?.auth === "oauth") { const suggestedDefaults = recommendedDefaultsForApp( galleryEntry, @@ -13317,7 +13390,7 @@ export function toolAccessService( // empty personal grant below so later catalog refreshes use the same // identity policy. const unauthenticatedPersonalProbe = Boolean( - !galleryEntry && + (!galleryEntry || remoteMcpConnector) && genericAuthKind === "none" && credentialSecretRefs.length === 0 && personalIdentityUserId && @@ -13330,7 +13403,10 @@ export function toolAccessService( }); } catch (error) { if ( - !galleryEntry && + (!galleryEntry || remoteMcpConnector) && + input.authMode !== "none" && + input.authMode !== "bearer" && + input.authMode !== "custom_headers" && error instanceof HttpError && asRecord(error.details).code === "oauth_challenge" ) { @@ -13421,7 +13497,7 @@ export function toolAccessService( // later fails: another retry may already be using the committed grant. personalPublicSetupEstablished = true; } - if (galleryEntry?.slug === COMPOSIO_GALLERY_KEY) { + if (galleryEntry?.slug === COMPOSIO_GALLERY_KEY && transport === "rest_api") { const [application] = await db .select() .from(toolApplications) @@ -13440,7 +13516,7 @@ export function toolAccessService( } const restoreDraftDefaults = Boolean(revivedConnectionPrevious); const refreshOptions = { - enableAllByDefault: restoreDraftDefaults, + enableAllByDefault: restoreDraftDefaults && !remoteMcpConnector, restoreDraftDefaults, }; const catalogConnectionId = connectionRow.id; @@ -13802,6 +13878,23 @@ export function toolAccessService( const connection = await getConnectionRow(connectionId, companyId); if (connection.status === "archived") throw conflict("Archived app connections cannot be finished"); + if (connection.config.mcpPreserveAccess === true && isRemoteMcpConnectorMethod(connection.config.sourceTemplateKey, connection.config.connectionMethodKey)) { + const [profile] = await db.select().from(toolProfiles).where(and( + eq(toolProfiles.companyId, companyId), eq(toolProfiles.profileKey, `app:${connection.id}`), + )).limit(1); + if (!profile) throw conflict("The saved connection permissions could not be found"); + const config = { ...connection.config }; + delete config.mcpPreserveAccess; + const updated = await db.transaction(async (tx) => { + const [row] = await tx.update(toolConnections).set({ config, transportConfig: config, status: "active", enabled: true, updatedAt: new Date() }) + .where(and(eq(toolConnections.id, connection.id), eq(toolConnections.companyId, companyId))).returning(); + await tx.update(toolApplications).set({ status: "active", updatedAt: new Date() }).where(and(eq(toolApplications.id, connection.applicationId), eq(toolApplications.companyId, companyId))); + return row; + }); + const details = await profileDetails(profile.id, companyId); + const policies = (await db.select().from(toolPolicies).where(eq(toolPolicies.companyId, companyId))).filter((policy) => asRecord(policy.config).connectionId === connection.id); + return { connection: toConnection(updated), profile: toProfile(profile), profileEntries: details.entries, profileBindings: details.bindings, policies: policies.map(toPolicy) }; + } const enabledIds = [ ...new Set([ ...input.enabledCatalogEntryIds, @@ -13825,7 +13918,7 @@ export function toolAccessService( connection.id, input.askFirstCatalogEntryIds, ); - if (enabledRows.some((entry) => entry.status === "disabled")) { + if (enabledRows.some((entry) => entry.status === "disabled" && !(entry.quarantineReason === "mcp_tool_removed" && isRemoteMcpConnectorMethod(connection.config.sourceTemplateKey, connection.config.connectionMethodKey)))) { throw badRequest("Disabled actions cannot be enabled"); } if (reviewedIds.length > 0) { @@ -14032,6 +14125,28 @@ export function toolAccessService( } const profileBindings: ToolProfileBinding[] = []; + // These connectors expose one agent-access choice. Commit installation + // reach and permission bindings together, including an empty selection. + if (isRemoteMcpConnectorMethod(connection.config.sourceTemplateKey, connection.config.connectionMethodKey)) { + const existingInstalls = await tx.select().from(toolConnectionInstalls).where(and( + eq(toolConnectionInstalls.companyId, companyId), eq(toolConnectionInstalls.connectionId, connection.id), + )); + const desired = new Map(bindingInputs.flatMap((binding) => binding.targetType === "company" || binding.targetType === "agent" + ? [[`${binding.targetType}:${binding.targetId}`, { targetType: binding.targetType, targetId: binding.targetId }] as const] : [])); + const removed = existingInstalls.filter((install) => !desired.has(`${install.targetType}:${install.targetId}`)); + const added = [...desired.values()].filter((binding) => !existingInstalls.some((install) => install.targetType === binding.targetType && install.targetId === binding.targetId)); + if (removed.length) await tx.delete(toolConnectionInstalls).where(inArray(toolConnectionInstalls.id, removed.map((install) => install.id))); + if (added.length) await tx.insert(toolConnectionInstalls).values(added.map((binding) => ({ + companyId, connectionId: connection.id, targetType: binding.targetType, targetId: binding.targetId, + createdByAgentId: actor?.actorType === "agent" ? (actor.actorId ?? null) : null, + createdByUserId: actor?.actorType === "user" ? (actor.actorId ?? null) : null, + }))); + if (added.length || removed.length) await tx.insert(toolAccessAuditEvents).values({ + companyId, connectionId: connection.id, actorType: actor?.actorType ?? "system", actorId: actor?.actorId ?? null, + action: "connection_installs.changed", outcome: "success", reasonCode: "installs_changed", + details: { added: added.map(({ targetType, targetId }) => ({ targetType, targetId })), removed: removed.map(({ targetType, targetId }) => ({ targetType, targetId })) }, + }); + } for (const bindingInput of bindingInputs) { const [binding] = await tx .insert(toolProfileBindings) @@ -14094,6 +14209,7 @@ export function toolAccessService( eq(toolCatalogEntries.companyId, companyId), inArray(toolCatalogEntries.id, enabledIds), ne(toolCatalogEntries.status, "quarantined"), + ne(toolCatalogEntries.status, "disabled"), ), ); } @@ -14228,6 +14344,7 @@ export function toolAccessService( actor?: ActorInfo, ): Promise { const connection = await getConnectionRow(connectionId, companyId); + await assertMcpAggregatorSetupEnabled(connection.config.sourceTemplateKey, connection.config.connectionMethodKey); if (connection.status === "archived") throw conflict("Archived app connections cannot be reconnected"); if (connection.credentialSource === "vercel_connect") { @@ -14409,6 +14526,7 @@ export function toolAccessService( }, ): Promise { let connection = await getConnectionRow(connectionId, companyId); + await assertMcpAggregatorSetupEnabled(connection.config.sourceTemplateKey, connection.config.connectionMethodKey); if (connection.status === "archived") throw conflict("Archived app connections cannot start sign in"); const sourceTemplateKey = @@ -15143,10 +15261,11 @@ export function toolAccessService( : suggestedAgentIds.length > 0 ? { agentIds: suggestedAgentIds } : "all_agents"; + const remoteMcpAccess = isRemoteMcpConnectorMethod(input.connection.config.sourceTemplateKey, input.connection.config.connectionMethodKey); const access: FinishToolApp["access"] = deferTaskAccess ? { agentIds: [] } : installs.length === 0 - ? normalizedSuggestedAccess + ? remoteMcpAccess ? { agentIds: [] } : normalizedSuggestedAccess : companyInstall ? "all_agents" : { agentIds }; @@ -15181,7 +15300,7 @@ export function toolAccessService( }, input.actor, ); - if (!deferTaskAccess && installs.length === 0) { + if (!deferTaskAccess && !remoteMcpAccess && installs.length === 0) { const installTargets = access === "all_agents" ? [ @@ -16856,6 +16975,7 @@ export function toolAccessService( if (!app || app.availability?.available === false) throw notFound("App not found"); const method = connectionMethodFor(app, methodKey); + await assertMcpAggregatorSetupEnabled(app.slug, method.key); if (method.transport !== "mcp_remote" || !method.defaults?.serverUrl) { throw unprocessable( "This app method does not use a hosted remote MCP endpoint", diff --git a/server/src/services/tool-gateway.ts b/server/src/services/tool-gateway.ts index 9c0c67e85c..cdb44da236 100644 --- a/server/src/services/tool-gateway.ts +++ b/server/src/services/tool-gateway.ts @@ -1,6 +1,7 @@ import { runIdentityContexts } from "@paperclipai/db"; import { captureRunIdentity } from "./run-identity.js"; import { resolveManagedGitHubIdentitySelection } from "./git-credentials.js"; +import { extractRemoteMcpPending } from "./remote-mcp-pending.js"; import { logger } from "../middleware/logger.js"; import { spawn } from "node:child_process"; import { createHash, randomBytes, randomUUID } from "node:crypto"; @@ -63,6 +64,7 @@ import type { ToolAccessDecision, ToolAccessDecisionInput, ToolConnectionTestCallStatus, + ToolUpstreamPending, ToolConnectionTestCallStatusPhase, ToolCredentialSecretRef, ToolMcpGateway, @@ -89,6 +91,10 @@ import { railwayCommandBudgetMs, createRailwayClient, isRailwayConnection, isRai import { RAILWAY_SSH_SECRET_PATH, runRailwaySshCommand } from "./railway-ssh.js"; import { initializeMcpHttpSession, + getMcpHttpSession, + forgetMcpHttpSessions, + readMcpHttpResponse, + McpHttpResponseError, mcpHttpRequestHeaders, parseMcpHttpResponseBody, } from "./mcp-http.js"; @@ -1049,6 +1055,28 @@ export function createToolGatewayService( now?: () => number; } = {}, ) { + // Authorization links can contain one-time codes. Keep them briefly in memory; + // persist only the redacted request and execution identifiers for recovery. + const upstreamHandoffs = new Map(); + async function retainUpstreamHandoff(invocationId: string, pending: ToolUpstreamPending) { + for (const [id, value] of upstreamHandoffs) if (value.expires <= Date.now()) upstreamHandoffs.delete(id); + while (upstreamHandoffs.size >= 256) upstreamHandoffs.delete(upstreamHandoffs.keys().next().value!); + upstreamHandoffs.set(invocationId, { pending, expires: Date.now() + 15 * 60_000 }); + const summary = validateToolContent({ + value: { upstreamPending: { ...pending, links: [] } }, direction: "result", sensitiveMode: "redact", promptInjectionMode: "ignore", + }).summary; + await db.update(toolInvocations).set({ resultSummary: summary }).where(eq(toolInvocations.id, invocationId)); + } + function recoverUpstreamHandoff(invocation: typeof toolInvocations.$inferSelect): ToolUpstreamPending | undefined { + if (invocation.errorCode !== "provider_interaction_required") return undefined; + const cached = upstreamHandoffs.get(invocation.id); + if (cached && cached.expires > Date.now()) return cached.pending; + upstreamHandoffs.delete(invocation.id); + const stored = asRecord(storedInvocationResult(invocation)); + const pending = asRecord(stored?.upstreamPending); + if (!pending || !["approval", "authorization"].includes(String(pending.kind))) return undefined; + return { ...pending, links: [] } as unknown as ToolUpstreamPending; + } const runtimeSupervisor = createToolRuntimeSupervisor(db, { deploymentMode: options.deploymentMode, deploymentExposure: options.deploymentExposure, @@ -5843,7 +5871,8 @@ export function createToolGatewayService( } let requestHeaders = headers; if (connection.config.mcpSessionRequired === true) { - requestHeaders = await initializeMcpHttpSession({ + requestHeaders = await getMcpHttpSession({ + scope: `${connection.id}:grant:${grant.id}:actor:${session.agentId}:${endpoint}`, send: (init) => dispatchRemote(endpoint, { ...init, @@ -6004,7 +6033,32 @@ export function createToolGatewayService( headers: mcpHttpRequestHeaders(headers), }); } - const body = await readBoundedRemoteResponse(response); + const sessionExpired = response.status === 404 && new Headers(requestHeaders).has("mcp-session-id"); + if (sessionExpired) { + // The next explicit call initializes again. Never replay a tools/call + // automatically: the failed call may have changed app data. + forgetMcpHttpSessions(connection.id); + } + const body = response.ok + ? JSON.stringify(await readMcpHttpResponse(response, requestId, { + maxBytes: MAX_REMOTE_MCP_RESPONSE_BYTES, + onRequest: async (message) => { + const pending = extractRemoteMcpPending(message); + if (pending) { + await retainUpstreamHandoff(invocationId, pending); + // Defer this interaction to the user. Do not claim that consent + // was granted or repeat a potentially state-changing tool call. + await dispatchRemote(endpoint, { + method: "POST", headers: mcpHttpRequestHeaders(requestHeaders), + body: JSON.stringify({ jsonrpc: "2.0", id: message.id, result: { action: "cancel" } }), + }); + throw new ToolGatewayHttpError(409, "This tool needs authorization in the provider.", "provider_interaction_required", { upstreamPending: pending, invocationId }); + } + const request = extractMcpElicitationRequest(message); + if (request) await requestElicitationForRecordedToolCall({ session, tool, invocationId, request }); + }, + })) + : await readBoundedRemoteResponse(response); execution.response = { httpStatus: response.status, contentType: response.headers.get("content-type"), @@ -6015,17 +6069,18 @@ export function createToolGatewayService( response.headers.get("traceparent"), }; if (!response.ok) { - await markRemoteConnectionHealth( - connection, - "error", - "Remote MCP server returned an HTTP error.", - ); + // Session expiration is recoverable on an explicit retry. Marking the + // connection unhealthy here would hide every tool and prevent it. + if (!sessionExpired) { + await markRemoteConnectionHealth(connection, "error", "Remote MCP server returned an HTTP error."); + } throw new ToolGatewayHttpError( 502, - "Remote MCP server returned an HTTP error", + sessionExpired ? "Remote MCP session expired. Retry the action explicitly to start a new session." : "Remote MCP server returned an HTTP error", "mcp_remote_status", { status: response.status, + ...(sessionExpired ? { sessionExpired: true } : {}), connectionId: connection.id, catalogEntryId: entry.id, execution, @@ -6034,10 +6089,7 @@ export function createToolGatewayService( } let payload: unknown; try { - payload = parseMcpHttpResponseBody( - body, - response.headers.get("content-type"), - ); + payload = JSON.parse(body); } catch { await markRemoteConnectionHealth( connection, @@ -6057,6 +6109,11 @@ export function createToolGatewayService( } const payloadRecord = asRecord(payload); if (!payloadRecord) throw malformedRemoteMcpResponse(); + const upstreamPending = extractRemoteMcpPending(payloadRecord, String(connection.config.sourceTemplateKey ?? ""), entry.toolName); + if (upstreamPending) { + await retainUpstreamHandoff(invocationId, upstreamPending); + throw new ToolGatewayHttpError(409, "Complete the provider's authorization or approval before continuing. The original call has not been replayed.", "provider_interaction_required", { upstreamPending, invocationId }); + } const topLevelElicitation = extractMcpElicitationRequest(payloadRecord); if (topLevelElicitation) { await requestElicitationForRecordedToolCall({ @@ -6117,6 +6174,15 @@ export function createToolGatewayService( ); return { result, headerSummary, execution }; } catch (error) { + if (error instanceof McpHttpResponseError) { + const failure = error.reason === "too_large" ? responseTooLargeError() + : error.reason === "malformed_response" ? malformedRemoteMcpResponse() + : new ToolGatewayHttpError(502, "Remote MCP server returned invalid JSON", "mcp_remote_invalid_json"); + await markRemoteConnectionHealth(connection, "error", failure.message); + throw new ToolGatewayHttpError(failure.status, failure.message, failure.reasonCode, { + connectionId: connection.id, catalogEntryId: entry.id, execution, + }); + } if (error instanceof RailwayError) { throw new ToolGatewayHttpError(error.status, error.message, error.code, { connectionId: connection.id, catalogEntryId: entry.id, execution }); } @@ -7181,6 +7247,9 @@ export function createToolGatewayService( decision: "allowed" as const, invocationId: args.invocationId, error: { message, reasonCode }, + ...(err instanceof ToolGatewayHttpError && err.reasonCode === "provider_interaction_required" + ? { upstreamPending: err.details.upstreamPending as ToolUpstreamPending } + : {}), }; } } @@ -7977,12 +8046,14 @@ export function createToolGatewayService( "executing", "rejected", "executed", + "failed", ]), ), ) .orderBy(desc(toolActionRequests.createdAt)) .limit(1); if (!match) return null; + if (match.actionRequest.status === "failed" && match.invocation.errorCode !== "provider_interaction_required") return null; // The gateway builds an ask-first request in two steps inside one call: it // inserts the row with a null signature and a null expiry, then signs the // row and sets the expiry. A concurrent matching call can observe the row in @@ -8041,6 +8112,14 @@ export function createToolGatewayService( const match = await matchingAgentActionRequest(input); if (!match) return null; const { actionRequest, invocation } = match; + if (actionRequest.status === "failed") { + throw new ToolGatewayHttpError(409, + "The provider is waiting for authorization or approval. Continue the existing execution; do not repeat the original call.", + "provider_interaction_required", { + invocationId: invocation.id, actionRequestId: actionRequest.id, + upstreamPending: recoverUpstreamHandoff(invocation), + }); + } if (actionRequest.status === "pending") { await throwApprovalRequired({ invocationId: invocation.id, @@ -8126,7 +8205,7 @@ export function createToolGatewayService( phase = "expired"; } else if ( actionRequest.status === "approved" || - actionRequest.status === "executed" + actionRequest.status === "executed" || actionRequest.status === "failed" ) { phase = invocationDone ? "done" : "running"; } else { @@ -8200,6 +8279,7 @@ export function createToolGatewayService( parameters, ...(result !== undefined ? { result } : {}), ...(error ? { error } : {}), + ...(recoverUpstreamHandoff(invocation) ? { upstreamPending: recoverUpstreamHandoff(invocation) } : {}), durationMs, requestedAt: actionRequest.createdAt.toISOString(), resolvedAt: actionRequest.resolvedAt diff --git a/tests/storybook-visual/remote-mcp-connections.config.ts b/tests/storybook-visual/remote-mcp-connections.config.ts new file mode 100644 index 0000000000..041b689c72 --- /dev/null +++ b/tests/storybook-visual/remote-mcp-connections.config.ts @@ -0,0 +1,8 @@ +import { defineConfig } from "@playwright/test"; + +export default defineConfig({ + testDir: ".", testMatch: "remote-mcp-connections.spec.ts", workers: 1, fullyParallel: false, + timeout: 180_000, retries: 0, outputDir: "./test-results/remote-mcp", reporter: [["list"]], + use: { browserName: "chromium", baseURL: "http://127.0.0.1:6138", reducedMotion: "reduce", actionTimeout: 10_000, trace: "retain-on-failure" }, + webServer: { command: "node ../../scripts/serve-storybook-static.mjs --port 6138", url: "http://127.0.0.1:6138/index.json", reuseExistingServer: false }, +}); diff --git a/tests/storybook-visual/remote-mcp-connections.spec.ts b/tests/storybook-visual/remote-mcp-connections.spec.ts new file mode 100644 index 0000000000..55ae7ba165 --- /dev/null +++ b/tests/storybook-visual/remote-mcp-connections.spec.ts @@ -0,0 +1,200 @@ +import { test, expect, type Page } from "@playwright/test"; + +const providers = ["zapier", "arcade", "composio", "executor"] as const; +const go = async (page: Page, provider: string, story: string) => { + await page.goto(`/iframe.html?id=apps-connections-${provider}--${story}&viewMode=story`); + await expect(page.locator(`[data-remote-mcp-provider="${provider}"]`)).toBeVisible(); +}; + +for (const provider of providers) { + test(`${provider}: setup finishes at discovery; regular permissions, testing and lifecycle`, async ({ page, context }) => { + const escapedRequests: string[] = []; + await context.route("**/*", async (route) => { + const url = new URL(route.request().url()); + if ((["http:", "https:"].includes(url.protocol) && url.hostname !== "127.0.0.1") || url.pathname.includes(`/tool-connections/review-${provider}/`)) { + escapedRequests.push(url.origin + url.pathname); await route.abort(); + } else await route.continue(); + }); + await go(page, provider, "complete-setup-journey"); + await expect(page.getByText("Step 1 of 2", { exact: true })).toBeVisible(); + await expect(page.getByRole("radio", { name: "Any human in the organization", exact: true })).toBeChecked(); + await expect(page.getByRole("radio", { name: "Any agent", exact: true })).toBeChecked(); + await page.getByRole("radio", { name: "Just me", exact: true }).click(); + await page.getByRole("radio", { name: "Just agents I pick", exact: true }).click(); + await expect(page.getByRole("button", { name: "Continue", exact: true })).toBeDisabled(); + await page.getByRole("button", { name: "Select agents", exact: true }).click(); + await page.getByRole("checkbox", { name: "Allow Researcher", exact: true }).check(); + await page.getByRole("checkbox", { name: "Allow Operator", exact: true }).check(); + await page.getByRole("button", { name: "Done", exact: true }).click(); + await page.getByRole("button", { name: "Continue", exact: true }).click(); + await expect(page.getByText("Step 2 of 2", { exact: true })).toBeVisible(); + await expect(page.getByLabel("Connection name", { exact: true })).toHaveCount(0); + await expect(page.getByRole("button", { name: /Test/ })).toHaveCount(0); + await page.getByRole("button", { name: "Back", exact: true }).click(); + await expect(page.getByRole("radio", { name: "Just me", exact: true })).toBeChecked(); + await page.getByRole("button", { name: "Continue", exact: true }).click(); + await page.getByRole("button", { name: "Use example configuration" }).click(); + await page.getByRole("button", { name: "Connect", exact: true }).click(); + if (provider !== "zapier") { + await expect(page.getByText(/Finish signing in to/)).toBeVisible(); + await page.getByRole("button", { name: "Complete sign-in (simulation)" }).click(); + } + await expect(page.getByRole("heading", { name: "Actions", exact: true })).toBeVisible(); + await expect(page.getByText("Simulated action calls: 0")).toBeVisible(); + await expect(page.getByText(/Step \d of/)).toHaveCount(0); + await expect(page.getByRole("button", { name: /Finish setup|Skip test|Run test/ })).toHaveCount(0); + for (const radio of await page.getByRole("radio", { name: /: Allowed$/ }).all()) await expect(radio).toBeChecked(); + const rows = page.locator("[data-action-id]"); + const firstId = await rows.first().getAttribute("data-action-id"); + const secondId = await rows.nth(1).getAttribute("data-action-id"); + const first = page.locator(`[data-action-id="${firstId}"]`); + const second = page.locator(`[data-action-id="${secondId}"]`); + await first.getByRole("button", { name: "Test", exact: true }).click(); + const dialog = page.getByRole("dialog"); + await expect(dialog.getByRole("heading", { name: /^Test / })).toBeVisible(); + await dialog.getByRole("button", { name: "Choose which agent to test as", exact: true }).click(); + await page.getByRole("button", { name: "Unassigned agent engineer", exact: true }).click(); + await expect(dialog.getByRole("button", { name: "Run", exact: true })).toHaveCount(0); + await dialog.getByRole("button", { name: "Choose which agent to test as", exact: true }).click(); + await page.getByRole("button", { name: "Researcher engineer", exact: true }).click(); + await dialog.getByRole("button", { name: "Run", exact: true }).click(); + await expect(dialog.getByText(/^Worked\./)).toBeVisible(); + await page.keyboard.press("Escape"); + await expect(page.getByText("Simulated action calls: 1")).toBeVisible(); + await first.getByRole("radio", { name: /: Ask first$/ }).click(); + await second.getByRole("radio", { name: /: Off$/ }).click(); + await page.getByRole("button", { name: "Refresh actions", exact: true }).click(); + await expect(page.getByRole("radio", { name: "Newly discovered tool: Allowed", exact: true })).toBeChecked(); + await expect(first.getByRole("radio", { name: /: Ask first$/ })).toBeChecked(); + await expect(second.getByRole("radio", { name: /: Off$/ })).toBeChecked(); + await page.getByRole("button", { name: "Connection settings", exact: true }).click(); + await page.getByRole("button", { name: "Who can use this connection", exact: true }).click(); + await expect(page.getByRole("radio", { name: "Just me", exact: true })).toBeChecked(); + await expect(page.getByRole("radio", { name: "Just agents I pick", exact: true })).toBeChecked(); + await page.getByRole("button", { name: "Done", exact: true }).click(); + await page.getByRole("button", { name: "Reconnect", exact: true }).click(); + await page.getByRole("button", { name: "Connect", exact: true }).click(); + if (provider !== "zapier") await page.getByRole("button", { name: "Complete sign-in (simulation)" }).click(); + await expect(first.getByRole("radio", { name: /: Ask first$/ })).toBeChecked(); + await expect(second.getByRole("radio", { name: /: Off$/ })).toBeChecked(); + await second.getByRole("button", { name: "Test", exact: true }).click(); + await expect(dialog.getByRole("button", { name: "Run", exact: true })).toHaveCount(0); + await page.keyboard.press("Escape"); + await page.getByRole("button", { name: "Connection settings", exact: true }).click(); + await page.getByRole("button", { name: "Disconnect", exact: true }).click(); + await page.getByRole("button", { name: "Disconnect connection", exact: true }).click(); + await expect(page.getByText("Disconnected", { exact: true })).toBeVisible(); + await expect(page.getByRole("button", { name: "Permissions", exact: true })).toBeDisabled(); + expect(escapedRequests).toEqual([]); + }); + + test(`${provider}: state matrix renders at desktop and narrow widths`, async ({ page, request }, testInfo) => { + const index = await (await request.get("/index.json")).json(); + const ids = Object.keys(index.entries).filter((id) => id.startsWith(`apps-connections-${provider}--`)); + expect(ids.length).toBeGreaterThanOrEqual(18); + expect(ids.some((id) => /empty-catalog|--test-|--paperclip-approval|--provider-approval/.test(id))).toBe(false); + const pageErrors: string[] = []; + page.on("pageerror", (error) => pageErrors.push(error.message)); + for (const width of [1280, 390]) { + await page.setViewportSize({ width, height: 900 }); + for (const id of ids) { + await test.step(`${width}: ${id}`, async () => { + await page.goto(`/iframe.html?id=${id}&viewMode=story&globals=theme:${width === 1280 ? "dark" : "light"}`); + await expect(page.locator(`[data-remote-mcp-provider="${provider}"]`)).toBeVisible(); + await expect(page.locator(".sb-errordisplay")).toBeHidden(); + expect(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth)).toBe(true); + if (/--(initial-setup|connection-details|manage-tool-permissions)$/.test(id)) { + await page.evaluate(() => document.fonts.ready); + await page.screenshot({ path: testInfo.outputPath(`${id}-${width}.png`), fullPage: true, animations: "disabled" }); + } + }); + } + } + expect(pageErrors).toEqual([]); + }); + + test(`${provider}: save and resume preserves credentials in memory and skips OAuth for tokens`, async ({ page }) => { + await go(page, provider, "advanced-authentication"); + await page.getByRole("button", { name: "Use example configuration" }).click(); + const originalUrl = await page.getByLabel("MCP server URL", { exact: true }).inputValue(); + await page.getByRole("button", { name: "Save & exit", exact: true }).click(); + await page.getByRole("button", { name: "Resume setup", exact: true }).click(); + await expect(page.getByLabel("MCP server URL", { exact: true })).toHaveValue(originalUrl); + const storage = await page.evaluate(() => JSON.stringify({ local: { ...localStorage }, session: { ...sessionStorage } })); + expect(storage).not.toContain("review-only-not-a-secret"); + expect(storage).not.toContain(originalUrl); + await page.getByRole("button", { name: "Connect", exact: true }).click(); + await expect(page.getByRole("heading", { name: "Actions", exact: true })).toBeVisible(); + await expect(page.getByText(/Finish signing in to/)).toHaveCount(0); + await expect(page.getByText("Simulated action calls: 0")).toBeVisible(); + }); +} + +test("Cancel, invalid URL and retry keep the form usable by keyboard", async ({ page }) => { + await go(page, "arcade", "connection-details"); + const url = page.getByLabel("MCP server URL", { exact: true }); + await url.fill("not-a-url"); + await url.press("Enter"); + await expect(page.getByText("Enter a valid MCP URL")).toBeVisible(); + await url.fill("https://arcade.example.invalid/review/mcp"); + await page.getByRole("button", { name: "Try again", exact: true }).click(); + await expect(page.getByText("Finish signing in to Arcade")).toBeVisible(); + await page.getByRole("button", { name: "Cancel sign-in", exact: true }).click(); + await expect(url).toHaveValue("https://arcade.example.invalid/review/mcp"); + await page.getByRole("button", { name: "Try again", exact: true }).press("Enter"); + await page.getByRole("button", { name: "Complete sign-in (simulation)" }).click(); + await expect(page.getByRole("heading", { name: "Arcade", exact: true })).toBeFocused(); + await page.keyboard.press("Tab"); + await expect(page.getByRole("button", { name: "Connection settings", exact: true })).toBeFocused(); +}); + +test("Zapier has no browser sign-in state or OAuth option", async ({ page, request }) => { + const index = await (await request.get("/index.json")).json(); + const ids = Object.keys(index.entries).filter((id) => id.startsWith("apps-connections-zapier--")); + expect(ids.some((id) => /sign-in/.test(id))).toBe(false); + await go(page, "zapier", "advanced-authentication"); + await expect(page.getByRole("option", { name: "Automatic (sign in if required)" })).toHaveCount(0); +}); + +for (const [action, headline] of [["Approve and resume", /^Worked\./], ["Decline", "Request declined"], ["Cancel request", "Request cancelled"]] as const) { + test(`Executor provider handoff: ${action} continues the same execution`, async ({ page }) => { + await go(page, "executor", "manage-tool-permissions"); + await page.getByLabel("Test response", { exact: true }).selectOption("provider"); + await page.getByRole("button", { name: "Test", exact: true }).first().click(); + const dialog = page.getByRole("dialog"); + await dialog.getByRole("button", { name: "Run", exact: true }).click(); + await expect(dialog.getByText("review-execution-001", { exact: true })).toBeVisible(); + await expect(dialog.getByRole("button", { name: "Run again", exact: true })).toBeDisabled(); + await dialog.getByRole("button", { name: action, exact: true }).click(); + await expect(dialog.getByText(headline)).toBeVisible(); + await expect(dialog.getByText("Approval needed in Executor", { exact: true })).toHaveCount(0); + await expect(page.getByText("Simulated action calls: 2")).toBeVisible(); + }); +} + +test("Regular permissions filter actions and open the shared test error/approval states", async ({ page }, testInfo) => { + await go(page, "zapier", "manage-tool-permissions"); + await page.getByRole("button", { name: "Write 1", exact: true }).click(); + await expect(page.locator("[data-action-id]")).toHaveCount(1); + await page.getByRole("button", { name: "All 3", exact: true }).click(); + await page.getByRole("textbox", { name: "Find an action", exact: true }).fill("spreadsheet rows"); + await expect(page.locator("[data-action-id]")).toHaveCount(1); + await page.getByLabel("Test response", { exact: true }).selectOption("error"); + await page.getByRole("button", { name: "Test", exact: true }).click(); + const dialog = page.getByRole("dialog"); + await dialog.getByRole("button", { name: "Run", exact: true }).click(); + await expect(dialog.getByText("Review resource was not found. Check the arguments.", { exact: true })).toBeVisible(); + await page.keyboard.press("Escape"); + await page.getByRole("radio", { name: /: Ask first$/ }).click(); + await page.getByRole("button", { name: "Test", exact: true }).click(); + await dialog.getByRole("button", { name: "Run", exact: true }).click(); + await expect(dialog.getByText("Sent for your OK.", { exact: true })).toBeVisible(); + await page.keyboard.press("Escape"); + await expect(page.getByText("Simulated action calls: 1")).toBeVisible(); + await page.setViewportSize({ width: 390, height: 844 }); + await page.getByRole("radio", { name: /: Allowed$/ }).click(); + await page.getByRole("button", { name: "Test", exact: true }).click(); + await page.evaluate(() => document.fonts.ready); + await page.screenshot({ path: testInfo.outputPath("canonical-test-dialog-narrow.png"), animations: "disabled" }); + expect(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth)).toBe(true); +}); diff --git a/ui/public/brands/apps/arcade.png b/ui/public/brands/apps/arcade.png new file mode 100644 index 0000000000..73721aa025 Binary files /dev/null and b/ui/public/brands/apps/arcade.png differ diff --git a/ui/public/brands/apps/executor.png b/ui/public/brands/apps/executor.png new file mode 100644 index 0000000000..ae9c3fd973 Binary files /dev/null and b/ui/public/brands/apps/executor.png differ diff --git a/ui/public/brands/apps/manifest.json b/ui/public/brands/apps/manifest.json index 502752b825..d2e05c527d 100644 --- a/ui/public/brands/apps/manifest.json +++ b/ui/public/brands/apps/manifest.json @@ -20,6 +20,12 @@ "localAsset": "/brands/apps/anthropic.svg", "darkAsset": "/brands/apps/anthropic-dark.svg" }, + { + "slug": "arcade", + "provider": "Arcade", + "catalogVisible": true, + "localAsset": "/brands/apps/arcade.png" + }, { "slug": "asana", "provider": "Asana", @@ -86,7 +92,7 @@ { "slug": "composio", "provider": "Composio", - "catalogVisible": false, + "catalogVisible": true, "localAsset": "/brands/apps/composio.svg", "darkAsset": "/brands/apps/composio-dark.svg" }, @@ -116,6 +122,12 @@ "catalogVisible": false, "localAsset": "/brands/apps/embat.svg" }, + { + "slug": "executor", + "provider": "Executor", + "catalogVisible": true, + "localAsset": "/brands/apps/executor.png" + }, { "slug": "github", "provider": "GitHub", diff --git a/ui/src/components/JsonSchemaForm.remote-mcp.test.tsx b/ui/src/components/JsonSchemaForm.remote-mcp.test.tsx new file mode 100644 index 0000000000..2b3485e456 --- /dev/null +++ b/ui/src/components/JsonSchemaForm.remote-mcp.test.tsx @@ -0,0 +1,42 @@ +// @vitest-environment jsdom +import { act, useState } from "react"; +import { createRoot } from "react-dom/client"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { JsonSchemaForm, validateJsonSchemaForm, type JsonSchemaNode } from "./JsonSchemaForm"; +vi.mock("./SecretBindingPicker", () => ({ SecretBindingPicker: () => null })); +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); +const schema: JsonSchemaNode = { type: "object", properties: { tools: { type: "array", items: { type: "object", properties: { arguments: { type: "object", additionalProperties: true } } } } } }; +let dispose: (() => void) | undefined; +afterEach(async () => { await act(async () => dispose?.()); document.body.innerHTML = ""; }); +describe("remote MCP open-ended argument inputs", () => { + it("edits nested tool arguments and rejects malformed JSON instead of submitting the previous object", async () => { + const container = document.createElement("div"); document.body.append(container); + const root = createRoot(container); dispose = () => root.unmount(); + let actual: Record = {}; + function Form() { + const [values, setValues] = useState({ tools: [{ arguments: {} }] } as Record); + actual = values; + return ; + } + await act(async () => root.render(
)); + const input = container.querySelector('textarea[aria-label="Arguments JSON"]')!; + expect(input).not.toBeNull(); + const fill = async (value: string) => act(async () => { + Object.getOwnPropertyDescriptor(HTMLTextAreaElement.prototype, "value")!.set!.call(input, value); + input.dispatchEvent(new Event("input", { bubbles: true })); + }); + await fill('{"repoName":"paperclipai/paperclip"}'); + expect(actual).toEqual({ tools: [{ arguments: { repoName: "paperclipai/paperclip" } }] }); + expect(validateJsonSchemaForm(schema, actual)).toEqual({}); + await fill('{"repoName":'); + expect(validateJsonSchemaForm(schema, actual)).toEqual({ "/tools/0/arguments": "Enter a valid JSON object" }); + await fill("[]"); + expect(validateJsonSchemaForm(schema, actual)["/tools/0/arguments"]).toBeTruthy(); + await fill("{}"); + expect(validateJsonSchemaForm(schema, actual)).toEqual({}); + }); + it("does not permit JSON null or array values for a required object", () => { + const args: JsonSchemaNode = { type: "object", required: ["arguments"], properties: { arguments: { type: "object" } } }; + for (const value of [null, [], "broken"]) expect(Object.keys(validateJsonSchemaForm(args, { arguments: value }))).toHaveLength(1); + }); +}); diff --git a/ui/src/components/JsonSchemaForm.tsx b/ui/src/components/JsonSchemaForm.tsx index d1466eb8fc..d07242b4d6 100644 --- a/ui/src/components/JsonSchemaForm.tsx +++ b/ui/src/components/JsonSchemaForm.tsx @@ -1,4 +1,4 @@ -import React, { useCallback, useEffect, useMemo, useState } from "react"; +import React, { useCallback, useEffect, useMemo, useRef, useState } from "react"; import { ChevronDown, ChevronRight, @@ -197,6 +197,9 @@ export function validateField( if (type === "secret-ref" && typeof value === "object") { return "Invalid secret reference"; } + if (type === "object" && (typeof value !== "object" || Array.isArray(value))) { + return "Enter a valid JSON object"; + } if (type === "string" || type === "secret-ref") { const str = String(value); @@ -524,7 +527,7 @@ const EnumField = React.memo(({ onValueChange={handleChange} disabled={disabled} > - + @@ -620,6 +623,8 @@ const SecretField = React.memo(({
{isVisible ? (