feat(apps): add experimental MCP aggregator connectors (#13755)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its tool gateway applies company access rules and approval controls
to connected apps.
> - MCP aggregators expose many apps through one provider endpoint.
> - Each aggregator needs its own credential, catalog, grants, and
lifecycle in Paperclip.
> - This pull request adds independent Zapier, Arcade, Composio Connect,
and Executor setup with a common Access → Connect layout.
> - A default-off MCP aggregators flag lets operators opt in while we
complete provider acceptance tests.
> - Agents use the normal Paperclip permissions, Test screen, and
gateway after setup.

## Linked Issues or Issue Description

**Subsystem affected**

Apps, connection setup, shared contracts, and the remote MCP gateway.

**Problem or motivation**

Aggregator endpoints need clear provider setup and correct MCP sessions.
Generic setup does not explain each provider's authentication or broad
execution tools. Provider approval must preserve the original execution
instead of replaying a write.

**Proposed solution**

Add four separate connectors behind Settings → Experimental → MCP
aggregators. Start with human and agent access, then connect the
endpoint and read its tools. Enable tools by default. Use the existing
Permissions and Test screens after setup. Keep legacy Composio API-key
and child connections intact.

**Alternatives considered**

A shared connection for all providers would mix credentials and access
rules. Separate provider-specific permission and test screens would
duplicate existing controls. Vercel Connect is outside this change.

**Roadmap alignment**

Extends the existing MCP Tool Gateway & Apps capability and the
Connected Apps roadmap area. This work was requested and reviewed by the
maintainer.

Related work: #11894, #12630, #12632, #12634, and #12906 concern the
legacy Composio broker. #13102 also covers remote MCP pagination. This
change preserves the broker path and adds initialized sessions, response
matching, and provider resume handling alongside pagination.

## What Changed

- Add branded setup and interactive Storybooks for Zapier, Arcade,
Composio Connect, and Executor. Use the existing access controls and
normal action tests. Do not request a connection name or action choices
during setup.
- Add the default-off `enableMcpAggregators` flag to settings, managed
feature metadata, the catalog, and setup guards. Hidden connections keep
running. Legacy Composio connections remain unchanged.
- Reuse the vault, grants, policy, and catalog models. Support OAuth
discovery, bearer tokens, custom headers, and credential-bearing URLs.
Add no database tables or migrations.
- Initialize and retain Streamable HTTP sessions by connection and
effective credentials. Read paginated catalogs and match streaming
responses to request IDs.
- Classify unfamiliar aggregator tools as writes despite upstream
read-only hints; only exact reviewed read capabilities enter the
read-only allowlist. Legacy Composio child behavior is preserved.
- Preserve provider authorization links and execution IDs. Support
Executor approve/resume, decline, and cancel without automatic replay of
uncertain writes.
- Preserve Off and Ask first choices during refresh and reconnect. Allow
new tools and retire removed tools. Keep agent access updates atomic and
preserve an empty agent selection.
- Document connector UX rules, provider branding sources, and live
acceptance results.
- Stabilize the existing Sentry release fixture after its repeated CI
failure by reusing one module mock; production Sentry behavior is
unchanged.

## Verification

- Final head `d11781970`: [CI
run](https://github.com/paperclipai/paperclip/actions/runs/35633534900)
passed, including broad typecheck, test shards, build, and E2E. All 54
checks pass; 2 optional checks are skipped. Greptile is 5/5, Security
Scan passes, and all review threads are resolved.

- Passed 27 focused connector Vitest checks and 18 connector-only
Storybook browser checks before the flag change. All 85 stories rendered
at desktop and narrow widths.
- Passed 5 connector lifecycle/server checks and 7 selected flag checks
after adding the flag. The latter cover settings, managed defaults,
cached catalog visibility, and all four setup routes.
- Review fixes passed 13 risk/handoff/lifecycle checks, dedicated
session-expiration and transport regressions, 13 selected
connector/gateway CI cases, and 10 selected setup/reconnect UI cases. A
real Composio connection-list call also succeeded through the refreshed
UI on `9ab115f71`.
- UI and server TypeScript checks passed. UI build, Storybook build,
token gates, and diff whitespace checks passed during implementation.
- Real browser and real Paperclip agent tests passed for Arcade,
Composio, and Executor. Tested action permissions, denied agent access,
reconnect, disconnect, and isolation. Tested Arcade catalog
additions/removal and Executor provider approve/resume, decline, and
cancel.
- Zapier live acceptance is incomplete. Its dedicated provider server is
configured, but its credential-copy dialog returned an empty clipboard
through browser automation. No live Zapier action is claimed.
- The three isolated Sentry release cases pass after the CI fixture fix.
- Local verification is deliberately narrow at the maintainer's request.
The full local suite, recursive typecheck, and repository-wide build
were not run. CI provides the broader checks.

## Risks

- Shared MCP transport changes affect other remote MCP servers. Protocol
fixtures cover initialized sessions, streaming response matching,
pagination, and isolation.
- Broad execution tools remain broad permissions. The provider governs
actions inside those tools.
- Provider handoff links are retained briefly in memory. After a server
restart, a one-time link may require reopening the provider dashboard.
Paperclip does not replay the original call.
- Zapier remains unproven live. Custom-header imports and self-hosted
endpoints have fixture coverage rather than a separate live account for
every variant.
- Turning the experimental flag off hides setup; it does not revoke
existing credentials or stop existing connections.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools, shell
execution, and browser automation. The exact runtime model ID and
context-window size are not exposed in this session. A separate
Anthropic-backed Paperclip agent performed live gateway acceptance
tasks.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-09-21 12:53:11 -05:00
1 parent 3790ca2f13
commit e8c8ba3c19
74 files changed
+2988 -240

No files matched your search

@@ -0,0 +1,8 @@
import { defineConfig } from "@playwright/test";
export default defineConfig({
testDir: ".", testMatch: "remote-mcp-connections.spec.ts", workers: 1, fullyParallel: false,
timeout: 180_000, retries: 0, outputDir: "./test-results/remote-mcp", reporter: [["list"]],
use: { browserName: "chromium", baseURL: "http://127.0.0.1:6138", reducedMotion: "reduce", actionTimeout: 10_000, trace: "retain-on-failure" },
webServer: { command: "node ../../scripts/serve-storybook-static.mjs --port 6138", url: "http://127.0.0.1:6138/index.json", reuseExistingServer: false },
});
@@ -0,0 +1,200 @@
import { test, expect, type Page } from "@playwright/test";
const providers = ["zapier", "arcade", "composio", "executor"] as const;
const go = async (page: Page, provider: string, story: string) => {
await page.goto(`/iframe.html?id=apps-connections-${provider}--${story}&viewMode=story`);
await expect(page.locator(`[data-remote-mcp-provider="${provider}"]`)).toBeVisible();
};
for (const provider of providers) {
test(`${provider}: setup finishes at discovery; regular permissions, testing and lifecycle`, async ({ page, context }) => {
const escapedRequests: string[] = [];
await context.route("**/*", async (route) => {
const url = new URL(route.request().url());
if ((["http:", "https:"].includes(url.protocol) && url.hostname !== "127.0.0.1") || url.pathname.includes(`/tool-connections/review-${provider}/`)) {
escapedRequests.push(url.origin + url.pathname); await route.abort();
} else await route.continue();
});
await go(page, provider, "complete-setup-journey");
await expect(page.getByText("Step 1 of 2", { exact: true })).toBeVisible();
await expect(page.getByRole("radio", { name: "Any human in the organization", exact: true })).toBeChecked();
await expect(page.getByRole("radio", { name: "Any agent", exact: true })).toBeChecked();
await page.getByRole("radio", { name: "Just me", exact: true }).click();
await page.getByRole("radio", { name: "Just agents I pick", exact: true }).click();
await expect(page.getByRole("button", { name: "Continue", exact: true })).toBeDisabled();
await page.getByRole("button", { name: "Select agents", exact: true }).click();
await page.getByRole("checkbox", { name: "Allow Researcher", exact: true }).check();
await page.getByRole("checkbox", { name: "Allow Operator", exact: true }).check();
await page.getByRole("button", { name: "Done", exact: true }).click();
await page.getByRole("button", { name: "Continue", exact: true }).click();
await expect(page.getByText("Step 2 of 2", { exact: true })).toBeVisible();
await expect(page.getByLabel("Connection name", { exact: true })).toHaveCount(0);
await expect(page.getByRole("button", { name: /Test/ })).toHaveCount(0);
await page.getByRole("button", { name: "Back", exact: true }).click();
await expect(page.getByRole("radio", { name: "Just me", exact: true })).toBeChecked();
await page.getByRole("button", { name: "Continue", exact: true }).click();
await page.getByRole("button", { name: "Use example configuration" }).click();
await page.getByRole("button", { name: "Connect", exact: true }).click();
if (provider !== "zapier") {
await expect(page.getByText(/Finish signing in to/)).toBeVisible();
await page.getByRole("button", { name: "Complete sign-in (simulation)" }).click();
}
await expect(page.getByRole("heading", { name: "Actions", exact: true })).toBeVisible();
await expect(page.getByText("Simulated action calls: 0")).toBeVisible();
await expect(page.getByText(/Step \d of/)).toHaveCount(0);
await expect(page.getByRole("button", { name: /Finish setup|Skip test|Run test/ })).toHaveCount(0);
for (const radio of await page.getByRole("radio", { name: /: Allowed$/ }).all()) await expect(radio).toBeChecked();
const rows = page.locator("[data-action-id]");
const firstId = await rows.first().getAttribute("data-action-id");
const secondId = await rows.nth(1).getAttribute("data-action-id");
const first = page.locator(`[data-action-id="${firstId}"]`);
const second = page.locator(`[data-action-id="${secondId}"]`);
await first.getByRole("button", { name: "Test", exact: true }).click();
const dialog = page.getByRole("dialog");
await expect(dialog.getByRole("heading", { name: /^Test / })).toBeVisible();
await dialog.getByRole("button", { name: "Choose which agent to test as", exact: true }).click();
await page.getByRole("button", { name: "Unassigned agent engineer", exact: true }).click();
await expect(dialog.getByRole("button", { name: "Run", exact: true })).toHaveCount(0);
await dialog.getByRole("button", { name: "Choose which agent to test as", exact: true }).click();
await page.getByRole("button", { name: "Researcher engineer", exact: true }).click();
await dialog.getByRole("button", { name: "Run", exact: true }).click();
await expect(dialog.getByText(/^Worked\./)).toBeVisible();
await page.keyboard.press("Escape");
await expect(page.getByText("Simulated action calls: 1")).toBeVisible();
await first.getByRole("radio", { name: /: Ask first$/ }).click();
await second.getByRole("radio", { name: /: Off$/ }).click();
await page.getByRole("button", { name: "Refresh actions", exact: true }).click();
await expect(page.getByRole("radio", { name: "Newly discovered tool: Allowed", exact: true })).toBeChecked();
await expect(first.getByRole("radio", { name: /: Ask first$/ })).toBeChecked();
await expect(second.getByRole("radio", { name: /: Off$/ })).toBeChecked();
await page.getByRole("button", { name: "Connection settings", exact: true }).click();
await page.getByRole("button", { name: "Who can use this connection", exact: true }).click();
await expect(page.getByRole("radio", { name: "Just me", exact: true })).toBeChecked();
await expect(page.getByRole("radio", { name: "Just agents I pick", exact: true })).toBeChecked();
await page.getByRole("button", { name: "Done", exact: true }).click();
await page.getByRole("button", { name: "Reconnect", exact: true }).click();
await page.getByRole("button", { name: "Connect", exact: true }).click();
if (provider !== "zapier") await page.getByRole("button", { name: "Complete sign-in (simulation)" }).click();
await expect(first.getByRole("radio", { name: /: Ask first$/ })).toBeChecked();
await expect(second.getByRole("radio", { name: /: Off$/ })).toBeChecked();
await second.getByRole("button", { name: "Test", exact: true }).click();
await expect(dialog.getByRole("button", { name: "Run", exact: true })).toHaveCount(0);
await page.keyboard.press("Escape");
await page.getByRole("button", { name: "Connection settings", exact: true }).click();
await page.getByRole("button", { name: "Disconnect", exact: true }).click();
await page.getByRole("button", { name: "Disconnect connection", exact: true }).click();
await expect(page.getByText("Disconnected", { exact: true })).toBeVisible();
await expect(page.getByRole("button", { name: "Permissions", exact: true })).toBeDisabled();
expect(escapedRequests).toEqual([]);
});
test(`${provider}: state matrix renders at desktop and narrow widths`, async ({ page, request }, testInfo) => {
const index = await (await request.get("/index.json")).json();
const ids = Object.keys(index.entries).filter((id) => id.startsWith(`apps-connections-${provider}--`));
expect(ids.length).toBeGreaterThanOrEqual(18);
expect(ids.some((id) => /empty-catalog|--test-|--paperclip-approval|--provider-approval/.test(id))).toBe(false);
const pageErrors: string[] = [];
page.on("pageerror", (error) => pageErrors.push(error.message));
for (const width of [1280, 390]) {
await page.setViewportSize({ width, height: 900 });
for (const id of ids) {
await test.step(`${width}: ${id}`, async () => {
await page.goto(`/iframe.html?id=${id}&viewMode=story&globals=theme:${width === 1280 ? "dark" : "light"}`);
await expect(page.locator(`[data-remote-mcp-provider="${provider}"]`)).toBeVisible();
await expect(page.locator(".sb-errordisplay")).toBeHidden();
expect(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth)).toBe(true);
if (/--(initial-setup|connection-details|manage-tool-permissions)$/.test(id)) {
await page.evaluate(() => document.fonts.ready);
await page.screenshot({ path: testInfo.outputPath(`${id}-${width}.png`), fullPage: true, animations: "disabled" });
}
});
}
}
expect(pageErrors).toEqual([]);
});
test(`${provider}: save and resume preserves credentials in memory and skips OAuth for tokens`, async ({ page }) => {
await go(page, provider, "advanced-authentication");
await page.getByRole("button", { name: "Use example configuration" }).click();
const originalUrl = await page.getByLabel("MCP server URL", { exact: true }).inputValue();
await page.getByRole("button", { name: "Save & exit", exact: true }).click();
await page.getByRole("button", { name: "Resume setup", exact: true }).click();
await expect(page.getByLabel("MCP server URL", { exact: true })).toHaveValue(originalUrl);
const storage = await page.evaluate(() => JSON.stringify({ local: { ...localStorage }, session: { ...sessionStorage } }));
expect(storage).not.toContain("review-only-not-a-secret");
expect(storage).not.toContain(originalUrl);
await page.getByRole("button", { name: "Connect", exact: true }).click();
await expect(page.getByRole("heading", { name: "Actions", exact: true })).toBeVisible();
await expect(page.getByText(/Finish signing in to/)).toHaveCount(0);
await expect(page.getByText("Simulated action calls: 0")).toBeVisible();
});
}
test("Cancel, invalid URL and retry keep the form usable by keyboard", async ({ page }) => {
await go(page, "arcade", "connection-details");
const url = page.getByLabel("MCP server URL", { exact: true });
await url.fill("not-a-url");
await url.press("Enter");
await expect(page.getByText("Enter a valid MCP URL")).toBeVisible();
await url.fill("https://arcade.example.invalid/review/mcp");
await page.getByRole("button", { name: "Try again", exact: true }).click();
await expect(page.getByText("Finish signing in to Arcade")).toBeVisible();
await page.getByRole("button", { name: "Cancel sign-in", exact: true }).click();
await expect(url).toHaveValue("https://arcade.example.invalid/review/mcp");
await page.getByRole("button", { name: "Try again", exact: true }).press("Enter");
await page.getByRole("button", { name: "Complete sign-in (simulation)" }).click();
await expect(page.getByRole("heading", { name: "Arcade", exact: true })).toBeFocused();
await page.keyboard.press("Tab");
await expect(page.getByRole("button", { name: "Connection settings", exact: true })).toBeFocused();
});
test("Zapier has no browser sign-in state or OAuth option", async ({ page, request }) => {
const index = await (await request.get("/index.json")).json();
const ids = Object.keys(index.entries).filter((id) => id.startsWith("apps-connections-zapier--"));
expect(ids.some((id) => /sign-in/.test(id))).toBe(false);
await go(page, "zapier", "advanced-authentication");
await expect(page.getByRole("option", { name: "Automatic (sign in if required)" })).toHaveCount(0);
});
for (const [action, headline] of [["Approve and resume", /^Worked\./], ["Decline", "Request declined"], ["Cancel request", "Request cancelled"]] as const) {
test(`Executor provider handoff: ${action} continues the same execution`, async ({ page }) => {
await go(page, "executor", "manage-tool-permissions");
await page.getByLabel("Test response", { exact: true }).selectOption("provider");
await page.getByRole("button", { name: "Test", exact: true }).first().click();
const dialog = page.getByRole("dialog");
await dialog.getByRole("button", { name: "Run", exact: true }).click();
await expect(dialog.getByText("review-execution-001", { exact: true })).toBeVisible();
await expect(dialog.getByRole("button", { name: "Run again", exact: true })).toBeDisabled();
await dialog.getByRole("button", { name: action, exact: true }).click();
await expect(dialog.getByText(headline)).toBeVisible();
await expect(dialog.getByText("Approval needed in Executor", { exact: true })).toHaveCount(0);
await expect(page.getByText("Simulated action calls: 2")).toBeVisible();
});
}
test("Regular permissions filter actions and open the shared test error/approval states", async ({ page }, testInfo) => {
await go(page, "zapier", "manage-tool-permissions");
await page.getByRole("button", { name: "Write 1", exact: true }).click();
await expect(page.locator("[data-action-id]")).toHaveCount(1);
await page.getByRole("button", { name: "All 3", exact: true }).click();
await page.getByRole("textbox", { name: "Find an action", exact: true }).fill("spreadsheet rows");
await expect(page.locator("[data-action-id]")).toHaveCount(1);
await page.getByLabel("Test response", { exact: true }).selectOption("error");
await page.getByRole("button", { name: "Test", exact: true }).click();
const dialog = page.getByRole("dialog");
await dialog.getByRole("button", { name: "Run", exact: true }).click();
await expect(dialog.getByText("Review resource was not found. Check the arguments.", { exact: true })).toBeVisible();
await page.keyboard.press("Escape");
await page.getByRole("radio", { name: /: Ask first$/ }).click();
await page.getByRole("button", { name: "Test", exact: true }).click();
await dialog.getByRole("button", { name: "Run", exact: true }).click();
await expect(dialog.getByText("Sent for your OK.", { exact: true })).toBeVisible();
await page.keyboard.press("Escape");
await expect(page.getByText("Simulated action calls: 1")).toBeVisible();
await page.setViewportSize({ width: 390, height: 844 });
await page.getByRole("radio", { name: /: Allowed$/ }).click();
await page.getByRole("button", { name: "Test", exact: true }).click();
await page.evaluate(() => document.fonts.ready);
await page.screenshot({ path: testInfo.outputPath("canonical-test-dialog-narrow.png"), animations: "disabled" });
expect(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth)).toBe(true);
});