feat(apps): add experimental MCP aggregator connectors (#13755)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its tool gateway applies company access rules and approval controls
to connected apps.
> - MCP aggregators expose many apps through one provider endpoint.
> - Each aggregator needs its own credential, catalog, grants, and
lifecycle in Paperclip.
> - This pull request adds independent Zapier, Arcade, Composio Connect,
and Executor setup with a common Access → Connect layout.
> - A default-off MCP aggregators flag lets operators opt in while we
complete provider acceptance tests.
> - Agents use the normal Paperclip permissions, Test screen, and
gateway after setup.

## Linked Issues or Issue Description

**Subsystem affected**

Apps, connection setup, shared contracts, and the remote MCP gateway.

**Problem or motivation**

Aggregator endpoints need clear provider setup and correct MCP sessions.
Generic setup does not explain each provider's authentication or broad
execution tools. Provider approval must preserve the original execution
instead of replaying a write.

**Proposed solution**

Add four separate connectors behind Settings → Experimental → MCP
aggregators. Start with human and agent access, then connect the
endpoint and read its tools. Enable tools by default. Use the existing
Permissions and Test screens after setup. Keep legacy Composio API-key
and child connections intact.

**Alternatives considered**

A shared connection for all providers would mix credentials and access
rules. Separate provider-specific permission and test screens would
duplicate existing controls. Vercel Connect is outside this change.

**Roadmap alignment**

Extends the existing MCP Tool Gateway & Apps capability and the
Connected Apps roadmap area. This work was requested and reviewed by the
maintainer.

Related work: #11894, #12630, #12632, #12634, and #12906 concern the
legacy Composio broker. #13102 also covers remote MCP pagination. This
change preserves the broker path and adds initialized sessions, response
matching, and provider resume handling alongside pagination.

## What Changed

- Add branded setup and interactive Storybooks for Zapier, Arcade,
Composio Connect, and Executor. Use the existing access controls and
normal action tests. Do not request a connection name or action choices
during setup.
- Add the default-off `enableMcpAggregators` flag to settings, managed
feature metadata, the catalog, and setup guards. Hidden connections keep
running. Legacy Composio connections remain unchanged.
- Reuse the vault, grants, policy, and catalog models. Support OAuth
discovery, bearer tokens, custom headers, and credential-bearing URLs.
Add no database tables or migrations.
- Initialize and retain Streamable HTTP sessions by connection and
effective credentials. Read paginated catalogs and match streaming
responses to request IDs.
- Classify unfamiliar aggregator tools as writes despite upstream
read-only hints; only exact reviewed read capabilities enter the
read-only allowlist. Legacy Composio child behavior is preserved.
- Preserve provider authorization links and execution IDs. Support
Executor approve/resume, decline, and cancel without automatic replay of
uncertain writes.
- Preserve Off and Ask first choices during refresh and reconnect. Allow
new tools and retire removed tools. Keep agent access updates atomic and
preserve an empty agent selection.
- Document connector UX rules, provider branding sources, and live
acceptance results.
- Stabilize the existing Sentry release fixture after its repeated CI
failure by reusing one module mock; production Sentry behavior is
unchanged.

## Verification

- Final head `d11781970`: [CI
run](https://github.com/paperclipai/paperclip/actions/runs/35633534900)
passed, including broad typecheck, test shards, build, and E2E. All 54
checks pass; 2 optional checks are skipped. Greptile is 5/5, Security
Scan passes, and all review threads are resolved.

- Passed 27 focused connector Vitest checks and 18 connector-only
Storybook browser checks before the flag change. All 85 stories rendered
at desktop and narrow widths.
- Passed 5 connector lifecycle/server checks and 7 selected flag checks
after adding the flag. The latter cover settings, managed defaults,
cached catalog visibility, and all four setup routes.
- Review fixes passed 13 risk/handoff/lifecycle checks, dedicated
session-expiration and transport regressions, 13 selected
connector/gateway CI cases, and 10 selected setup/reconnect UI cases. A
real Composio connection-list call also succeeded through the refreshed
UI on `9ab115f71`.
- UI and server TypeScript checks passed. UI build, Storybook build,
token gates, and diff whitespace checks passed during implementation.
- Real browser and real Paperclip agent tests passed for Arcade,
Composio, and Executor. Tested action permissions, denied agent access,
reconnect, disconnect, and isolation. Tested Arcade catalog
additions/removal and Executor provider approve/resume, decline, and
cancel.
- Zapier live acceptance is incomplete. Its dedicated provider server is
configured, but its credential-copy dialog returned an empty clipboard
through browser automation. No live Zapier action is claimed.
- The three isolated Sentry release cases pass after the CI fixture fix.
- Local verification is deliberately narrow at the maintainer's request.
The full local suite, recursive typecheck, and repository-wide build
were not run. CI provides the broader checks.

## Risks

- Shared MCP transport changes affect other remote MCP servers. Protocol
fixtures cover initialized sessions, streaming response matching,
pagination, and isolation.
- Broad execution tools remain broad permissions. The provider governs
actions inside those tools.
- Provider handoff links are retained briefly in memory. After a server
restart, a one-time link may require reopening the provider dashboard.
Paperclip does not replay the original call.
- Zapier remains unproven live. Custom-header imports and self-hosted
endpoints have fixture coverage rather than a separate live account for
every variant.
- Turning the experimental flag off hides setup; it does not revoke
existing credentials or stop existing connections.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools, shell
execution, and browser automation. The exact runtime model ID and
context-window size are not exposed in this session. A separate
Anthropic-backed Paperclip agent performed live gateway acceptance
tasks.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-09-21 12:53:11 -05:00
1 parent 3790ca2f13
commit e8c8ba3c19
74 files changed
+2988 -240

No files matched your search

@@ -39,6 +39,7 @@ import {
startEmbeddedPostgresTestDatabase,
} from "./helpers/embedded-postgres.js";
import { toolAccessService } from "../services/tool-access.js";
import { instanceSettingsService } from "../services/instance-settings.js";
import { ComposioApiError, type ComposioClient } from "../services/composio.js";
import { createComposioSessionManager } from "../services/composio-session-manager.js";
import { toolAccessPolicyService } from "../services/tool-access-policy.js";
@@ -468,6 +469,7 @@ describeEmbeddedPostgres("generic remote MCP connections", () => {
});
it("keeps a generated Zapier URL attached to the curated Zapier identity", async () => {
await instanceSettingsService(db).updateExperimental({ enableMcpAggregators: true });
const secretUrl = "https://mcp.zapier.com/api/v1/connect?token=zapier-secret";
const publicUrl = "https://mcp.zapier.com/api/v1/connect";
const company = await createCompany(db);
@@ -477,10 +479,12 @@ describeEmbeddedPostgres("generic remote MCP connections", () => {
remoteHttpEndpointLookup: async () => [{ address: "8.8.8.8", family: 4 }],
remoteHttpRequest: async (url, init) => {
if (url === secretUrl && (init.method ?? "GET").toUpperCase() === "POST") {
const body = JSON.parse(String(init.body));
if (body.method === "notifications/initialized") return new Response(null, { status: 202 });
return jsonResponse({
jsonrpc: "2.0",
id: "paperclip-catalog-refresh",
result: { tools: FIXTURE_TOOLS },
id: body.id,
result: body.method === "initialize" ? { protocolVersion: "2025-06-18" } : { tools: FIXTURE_TOOLS },
});
}
return jsonResponse({ error: "not_found" }, 404);
@@ -43,6 +43,7 @@ describe("instance settings service", () => {
enableApps: true,
enableAgentChat: false,
enableChatConnectors: false,
enableMcpAggregators: false,
enableConferenceRoomChat: false,
enableClassicTaskInterface: false,
enableExternalObjects: false,
@@ -0,0 +1,251 @@
import { randomUUID } from "node:crypto";
import { mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
import { agents, heartbeatRuns, issues, toolCatalogEntries, toolConnectionInstalls, toolInvocations, companies, companyMemberships, createDb, toolConnections, toolPolicies, toolProfileEntries } from "@paperclipai/db";
import { eq } from "drizzle-orm";
import { startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
import { toolAccessService } from "../services/tool-access.js";
import { createToolGatewayService } from "../services/tool-gateway.js";
import { instanceSettingsService, normalizeExperimentalSettings } from "../services/instance-settings.js";
import express from "express";
import request from "supertest";
import { toolAccessRoutes } from "../routes/tool-access.js";
const actor = { actorType: "user" as const, actorId: "mcp-test-user", actorSource: "local_implicit" as const };
const tool = (name: string) => ({ name, description: name, inputSchema: { type: "object", properties: {} }, annotations: { readOnlyHint: true } });
describe("remote connector lifecycle", () => {
let fixture: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
let db: ReturnType<typeof createDb>;
let keyDir: string;
beforeAll(async () => {
keyDir = await mkdtemp(join(tmpdir(), "mcp-connector-secrets-"));
vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY_FILE", join(keyDir, "key"));
fixture = await startEmbeddedPostgresTestDatabase("mcp-connectors-test-");
db = createDb(fixture.connectionString);
await instanceSettingsService(db).updateExperimental({ enableMcpAggregators: true });
});
afterAll(async () => { await fixture?.cleanup(); vi.unstubAllEnvs(); if (keyDir) await rm(keyDir, { recursive: true, force: true }); });
async function company() {
const [row] = await db.insert(companies).values({ name: `MCP ${randomUUID()}`, issuePrefix: `M${randomUUID().slice(0, 5)}` }).returning();
await db.insert(companyMemberships).values({ companyId: row.id, principalType: "user", principalId: actor.actorId, status: "active", membershipRole: "admin" });
return row;
}
function remoteFixture() {
const requests: { url: string; headers: Headers }[] = [];
let added = false;
let removed = false;
const service = toolAccessService(db, { deploymentMode: "local_trusted", deploymentExposure: "private",
remoteHttpEndpointLookup: async () => [{ address: "8.8.8.8", family: 4 }],
remoteHttpRequest: async (url, init) => {
const body = JSON.parse(String(init.body)); const headers = new Headers(init.headers);
requests.push({ url, headers });
if (body.method === "initialize") return Response.json({ id: body.id, jsonrpc: "2.0", result: { protocolVersion: "2025-06-18", capabilities: {} } }, { headers: { "Mcp-Session-Id": "session" } });
expect(headers.get("mcp-session-id")).toBe("session");
if (body.method === "notifications/initialized") return new Response(null, { status: 202 });
expect(body.method).toBe("tools/list");
return Response.json({ id: body.id, jsonrpc: "2.0", result: body.params?.cursor
? { tools: [tool("ask"), ...(added ? [tool("new_tool")] : [])] }
: { tools: [tool("read"), ...(!removed ? [tool("off")] : [])], nextCursor: "page-two" } });
},
});
return { service, requests, add: () => { added = true; }, remove: () => { removed = true; }, restore: () => { removed = false; } };
}
it("defaults MCP aggregators off and rejects direct setup without provider requests or credential writes", async () => {
expect(normalizeExperimentalSettings({}).enableMcpAggregators).toBe(false);
const org = await company(); const remote = remoteFixture();
await instanceSettingsService(db).updateExperimental({ enableMcpAggregators: false });
try {
const app = express();
app.use((req, _res, next) => { req.actor = { type: "board", userId: actor.actorId, source: "local_implicit", isInstanceAdmin: true }; next(); });
app.use("/api", toolAccessRoutes(db, { paperclipCloudConnector: null }));
const gallery = await request(app).get(`/api/companies/${org.id}/tools/gallery`);
expect(gallery.status).toBe(200);
expect(gallery.body.apps.some((entry: { slug: string }) => ["zapier", "arcade", "composio", "executor"].includes(entry.slug))).toBe(false);
expect(gallery.body.apps.some((entry: { slug: string }) => entry.slug === "notion")).toBe(true);
for (const [galleryKey, connectionMethodKey] of [["zapier", "generated-url"], ["arcade", "mcp"], ["composio", "mcp"], ["executor", "mcp"]]) {
await expect(remote.service.connectGalleryApp(org.id, { galleryKey, connectionMethodKey, saveDraft: true }, actor))
.rejects.toMatchObject({ status: 403, details: { code: "mcp_aggregators_disabled" } });
}
await expect(remote.service.preflightGalleryAppMetadata("composio", "mcp"))
.rejects.toMatchObject({ status: 403 });
expect(remote.requests).toHaveLength(0);
expect(await db.select().from(toolConnections).where(eq(toolConnections.companyId, org.id))).toHaveLength(0);
} finally {
await instanceSettingsService(db).updateExperimental({ enableMcpAggregators: true });
}
});
it("vaults generated URLs, paginates discovery, preserves Off/Ask during refresh and reconnect, and isolates companies", async () => {
const org = await company(); const other = await company(); const remote = remoteFixture();
const input = { galleryKey: "zapier", connectionMethodKey: "generated-url", link: "https://mcp.zapier.com/api/v1/connect?token=fixture-secret", authMode: "none" as const };
const connected = await remote.service.connectGalleryApp(org.id, input, actor);
expect(connected.catalog).toHaveLength(3);
expect(JSON.stringify(connected)).not.toContain("fixture-secret");
expect(remote.requests.every((r) => r.url.includes("token=fixture-secret"))).toBe(true);
const ids = Object.fromEntries(connected.catalog.map((entry) => [entry.toolName, entry.id]));
const finished = await remote.service.finishGalleryAppConnection(org.id, connected.connectionId, { enabledCatalogEntryIds: [ids.read, ids.ask], askFirstCatalogEntryIds: [ids.ask], access: "all_agents" }, actor);
remote.add();
const refreshed = await remote.service.refreshCatalog(connected.connectionId, actor, { enableAllByDefault: true });
const newId = refreshed.catalog.find((entry) => entry.toolName === "new_tool")!.id;
const entries = await db.select().from(toolProfileEntries).where(eq(toolProfileEntries.profileId, finished.profile.id));
expect(entries.map((entry) => entry.catalogEntryId)).toEqual(expect.arrayContaining([ids.read, ids.ask, newId]));
expect(entries.map((entry) => entry.catalogEntryId)).not.toContain(ids.off);
await expect(remote.service.connectGalleryApp(other.id, { ...input, reconnectConnectionId: connected.connectionId }, actor)).rejects.toThrow("not found");
const reconnected = await remote.service.connectGalleryApp(org.id, { ...input, reconnectConnectionId: connected.connectionId }, actor);
const restored = await remote.service.finishGalleryAppConnection(org.id, connected.connectionId, { enabledCatalogEntryIds: reconnected.catalog.map((entry) => entry.id), askFirstCatalogEntryIds: [], access: "all_agents" }, actor);
expect(restored.connection.status).toBe("active");
expect(restored.profileEntries.map((entry) => entry.catalogEntryId)).not.toContain(ids.off);
const policies = await db.select().from(toolPolicies).where(eq(toolPolicies.companyId, org.id));
expect(policies.some((policy) => policy.enabled && policy.config.catalogEntryId === ids.ask)).toBe(true);
remote.remove();
const afterRemoval = await remote.service.refreshCatalog(connected.connectionId, actor);
expect(afterRemoval.catalog.find((entry) => entry.toolName === "off")?.status).not.toBe("active");
const [retired] = await db.select().from(toolCatalogEntries).where(eq(toolCatalogEntries.id, ids.off));
expect(retired.status).toBe("disabled"); // Check persisted discovery, not just this refresh response.
remote.restore();
const afterReturn = await remote.service.refreshCatalog(connected.connectionId, actor);
expect(afterReturn.catalog.find((entry) => entry.toolName === "off")?.status).toBe("active");
const retainedEntries = await db.select().from(toolProfileEntries).where(eq(toolProfileEntries.profileId, finished.profile.id));
expect(retainedEntries.map((entry) => entry.catalogEntryId)).not.toContain(ids.off);
await remote.service.archiveConnection(connected.connectionId, org.id, actor);
const [removed] = await db.select().from(toolConnections).where(eq(toolConnections.id, connected.connectionId));
expect(removed.enabled).toBe(false);
expect(removed.status).toBe("archived");
});
it("renews expired discovery sessions and requires an explicit retry after an expired execution session", async () => {
const org = await company();
const [agent] = await db.insert(agents).values({ companyId: org.id, name: "Session tester", role: "engineer", adapterType: "process", adapterConfig: {}, runtimeConfig: {} }).returning();
let initialized = 0;
const expired = new Set<string>();
const calls: string[] = [];
const send = async (_url: string, init: RequestInit) => {
const body = JSON.parse(String(init.body));
if (body.method === "initialize") return Response.json({ id: body.id, result: { protocolVersion: "2025-06-18" } }, { headers: { "Mcp-Session-Id": `session-${++initialized}` } });
if (body.method === "notifications/initialized") return new Response(null, { status: 202 });
const session = new Headers(init.headers).get("mcp-session-id")!;
if (body.method === "tools/call") calls.push(session);
if (expired.has(session)) return new Response(null, { status: 404 });
return Response.json({ id: body.id, result: body.method === "tools/list" ? { tools: [tool("read")] } : { content: [{ type: "text", text: "ok" }] } });
};
const service = toolAccessService(db, { deploymentMode: "local_trusted", deploymentExposure: "private", remoteHttpEndpointLookup: async () => [{ address: "8.8.8.8", family: 4 }], remoteHttpRequest: send });
const connected = await service.connectGalleryApp(org.id, { galleryKey: "arcade", connectionMethodKey: "mcp", link: "https://api.arcade.dev/mcp/expiration", authMode: "none" }, actor);
const beforeRefresh = initialized;
expired.add(`session-${initialized}`);
expect((await service.refreshCatalog(connected.connectionId, actor)).catalog).toHaveLength(1);
expect(initialized).toBe(beforeRefresh + 1);
await service.finishGalleryAppConnection(org.id, connected.connectionId, { enabledCatalogEntryIds: connected.catalog.map((entry) => entry.id), askFirstCatalogEntryIds: [], access: "all_agents" }, actor);
const gateway = createToolGatewayService(db, { deploymentMode: "local_trusted", deploymentExposure: "private", remoteHttpRequest: send });
const call = () => gateway.executeTestCall({ companyId: org.id, connectionId: connected.connectionId, agentId: agent.id, userId: actor.actorId, toolName: "read", parameters: {} });
expect(await call()).toMatchObject({ decision: "allowed", result: { data: { isError: false } } });
const executionSession = calls[0];
expired.add(executionSession);
const failed = await call();
expect(failed).toMatchObject({ error: { reasonCode: "mcp_remote_status" } });
expect(calls).toEqual([executionSession, executionSession]);
expect(await call()).toMatchObject({ decision: "allowed", result: { data: { isError: false } } });
expect(calls).toHaveLength(3);
expect(calls[2]).not.toBe(executionSession);
});
it("saves a vaulted draft without contacting the provider and resumes with custom headers", async () => {
const org = await company(); const remote = remoteFixture();
const input = { galleryKey: "arcade", connectionMethodKey: "mcp", link: "https://api.arcade.dev/mcp/fixture", authMode: "bearer" as const };
const draft = await remote.service.connectGalleryApp(org.id, { ...input, saveDraft: true, credentialValues: { "credentials.authorization": "fixture-key", "headers.Arcade-User-ID": "test-user" } }, actor);
expect(remote.requests).toHaveLength(0);
expect(JSON.stringify(draft)).not.toContain("fixture-key");
const connected = await remote.service.connectGalleryApp(org.id, { ...input, resumeConnectionId: draft.connectionId }, actor);
expect(connected.catalog).toHaveLength(3);
expect(remote.requests.every((request) => request.headers.get("authorization") === "Bearer fixture-key" && request.headers.get("arcade-user-id") === "test-user")).toBe(true);
});
it("does not widen an empty agent selection after an OAuth callback or reconnect", async () => {
const org = await company();
const endpoint = "https://api.arcade.dev/mcp/fixture-oauth";
const send = async (url: string, init: RequestInit) => {
if (url.includes(".well-known/oauth-protected-resource")) return Response.json({ resource: endpoint, authorization_servers: ["https://auth.arcade.dev"] });
if (url.includes(".well-known/")) return Response.json({ issuer: "https://auth.arcade.dev", authorization_endpoint: "https://auth.arcade.dev/authorize", token_endpoint: "https://auth.arcade.dev/token", response_types_supported: ["code"], code_challenge_methods_supported: ["S256"], token_endpoint_auth_methods_supported: ["none"] });
if (url === "https://auth.arcade.dev/token") return Response.json({ access_token: "fixture-access", token_type: "Bearer", expires_in: 3600 });
if (url !== endpoint) throw new Error(`Unexpected fixture URL: ${url}`);
if (!new Headers(init.headers).has("authorization")) return new Response(null, { status: 401, headers: { "WWW-Authenticate": 'Bearer resource_metadata="https://api.arcade.dev/.well-known/oauth-protected-resource"' } });
const body = JSON.parse(String(init.body));
if (body.method === "notifications/initialized") return new Response(null, { status: 202 });
return Response.json({ id: body.id, result: body.method === "initialize" ? { protocolVersion: "2025-06-18" } : { tools: [tool("read")] } });
};
const service = toolAccessService(db, { deploymentMode: "local_trusted", deploymentExposure: "private", remoteHttpEndpointLookup: async () => [{ address: "8.8.8.8", family: 4 }], remoteHttpRequest: send });
const input = { galleryKey: "arcade", connectionMethodKey: "mcp", link: endpoint, authMode: "auto" as const, oauthClient: { clientId: "fixture-client" } };
const connected = await service.connectGalleryApp(org.id, input, actor);
const callback = async () => {
const redirectUri = "http://127.0.0.1:3116/api/tools/oauth/callback";
const started = await service.startOAuth(org.id, connected.connectionId, { redirectUri, actor });
return service.completeOAuthCallback({ state: new URL(started.authorizationUrl).searchParams.get("state")!, code: "fixture-code", redirectUri, actor });
};
await service.putConnectionInstalls(connected.connectionId, { installs: [] }, actor);
const first = await callback();
expect(first.connection.status).toBe("active");
expect((await service.listConnectionInstalls(connected.connectionId))).toHaveLength(0);
await service.connectGalleryApp(org.id, { ...input, reconnectConnectionId: connected.connectionId }, actor);
await callback();
expect((await service.listConnectionInstalls(connected.connectionId))).toHaveLength(0);
});
it("enforces agent and action permissions before dispatch, and preserves provider resume after Paperclip approval", async () => {
const org = await company();
const [allowed, denied] = await db.insert(agents).values(["Allowed", "Denied"].map((name) => ({ companyId: org.id, name, role: "engineer", adapterType: "process", adapterConfig: {}, runtimeConfig: {} }))).returning();
const calls: { name: string; arguments: unknown; session: string | null }[] = [];
const send = async (_url: string, init: RequestInit) => {
const body = JSON.parse(String(init.body));
if (body.method === "initialize") return Response.json({ id: body.id, result: { protocolVersion: "2025-06-18" } }, { headers: { "Mcp-Session-Id": "execution-session" } });
if (body.method === "notifications/initialized") return new Response(null, { status: 202 });
if (body.method === "tools/list") return Response.json({ id: body.id, result: { tools: [tool("execute"), tool("resume"), tool("off")] } });
calls.push({ ...body.params, session: new Headers(init.headers).get("mcp-session-id") });
return Response.json({ id: body.id, result: { structuredContent: body.params.name === "execute"
? { status: "waiting_for_interaction", executionId: "fixture-execution", interaction: { kind: "form", message: "Approve read?", requestedSchema: { type: "object", properties: {} } } }
: { ok: true, resumed: body.params.arguments.executionId }, content: [] } });
};
const service = toolAccessService(db, { deploymentMode: "local_trusted", deploymentExposure: "private", remoteHttpEndpointLookup: async () => [{ address: "8.8.8.8", family: 4 }], remoteHttpRequest: send });
const connection = await service.connectGalleryApp(org.id, { galleryKey: "executor", connectionMethodKey: "mcp", link: "https://executor.sh/test/mcp", authMode: "none" }, actor);
const ids = Object.fromEntries(connection.catalog.map((entry) => [entry.toolName, entry.id]));
await service.finishGalleryAppConnection(org.id, connection.connectionId, { enabledCatalogEntryIds: [ids.execute, ids.resume], askFirstCatalogEntryIds: [ids.execute], access: { agentIds: [allowed.id] } }, actor);
const gateway = createToolGatewayService(db, { deploymentMode: "local_trusted", deploymentExposure: "private", remoteHttpRequest: send, toolActionSigningSecret: "fixture-signing-key" });
const call = (agentId: string, toolName: string, parameters: Record<string, unknown> = {}) => gateway.executeTestCall({ companyId: org.id, connectionId: connection.connectionId, agentId, userId: actor.actorId, toolName, parameters });
expect((await call(denied.id, "execute")).decision).toBe("off");
expect((await call(allowed.id, "off")).decision).toBe("off");
expect(calls).toHaveLength(0);
const asked = await call(allowed.id, "execute");
expect(asked.decision).toBe("ask_first");
expect(calls).toHaveLength(0);
if (!("actionRequestId" in asked)) throw new Error("Missing approval request");
await gateway.approveActionRequest({ companyId: org.id, actionRequestId: asked.actionRequestId!, actor: { userId: actor.actorId } });
const status = await gateway.getTestCallStatus({ companyId: org.id, connectionId: connection.connectionId, actionRequestId: asked.actionRequestId! });
expect(status.phase).toBe("done");
expect(status.upstreamPending).toMatchObject({ executionId: "fixture-execution", resumeTool: "resume" });
const [invocation] = await db.select().from(toolInvocations).where(eq(toolInvocations.id, asked.invocationId));
expect(invocation.resultSummary?.summary).toContain("fixture-execution");
const resumed = await call(allowed.id, "resume", { executionId: status.upstreamPending!.executionId, action: "accept", content: "{}" });
expect(resumed.decision).toBe("allowed");
expect(calls.map((call) => call.name)).toEqual(["execute", "resume"]);
expect(calls.every((call) => call.session === "execution-session")).toBe(true);
const [issue] = await db.insert(issues).values({ companyId: org.id, title: "Provider approval", status: "in_progress", assigneeAgentId: allowed.id }).returning();
const [run] = await db.insert(heartbeatRuns).values({ companyId: org.id, agentId: allowed.id, invocationSource: "assignment", status: "running", contextSnapshot: { issueId: issue.id } }).returning();
const session = await gateway.createSession({ companyId: org.id, agentId: allowed.id, runId: run.id });
const execute = (await gateway.listToolsForSession(session.token)).find((entry) => entry.upstreamToolName === "execute")!;
expect(execute).toBeDefined();
let approvalId = "";
try { await gateway.executeTool({ sessionToken: session.token, tool: execute.name, parameters: {} }); }
catch (error) {
expect(error).toMatchObject({ reasonCode: "approval_required" });
approvalId = (error as { details: { actionRequestId: string } }).details.actionRequestId;
}
expect(approvalId).not.toBe("");
await gateway.approveActionRequest({ companyId: org.id, actionRequestId: approvalId, actor: { userId: actor.actorId } });
await expect(gateway.executeTool({ sessionToken: session.token, tool: execute.name, parameters: {} })).rejects.toMatchObject({
reasonCode: "provider_interaction_required", details: { upstreamPending: { executionId: "fixture-execution" } },
});
expect(calls).toHaveLength(3); // Retrying the approved action did not start another execution.
await service.finishGalleryAppConnection(org.id, connection.connectionId, { enabledCatalogEntryIds: [ids.execute, ids.resume], askFirstCatalogEntryIds: [ids.execute], access: { agentIds: [] } }, actor);
expect(await db.select().from(toolConnectionInstalls).where(eq(toolConnectionInstalls.connectionId, connection.connectionId))).toHaveLength(0);
expect((await call(allowed.id, "execute")).decision).toBe("off");
expect((await gateway.listToolsForSession(session.token)).filter((entry) => entry.connectionId === connection.connectionId)).toHaveLength(0);
await service.archiveConnection(connection.connectionId, org.id, actor);
await expect(call(allowed.id, "resume")).rejects.toThrow("not found");
expect(calls).toHaveLength(3);
});
});
@@ -0,0 +1,59 @@
import { describe, expect, it } from "vitest";
import { extractRemoteMcpPending } from "../services/remote-mcp-pending.js";
import { classifyRisk } from "../services/tool-access.js";
describe("remote MCP provider handoffs", () => {
it("recognizes Arcade's JSON text authorization response before redaction", () => {
const result = { result: { isError: true, content: [{ type: "text", text: JSON.stringify({ authorization_url: "https://github.com/login/oauth/authorize?state=test", message: "Not executed; authorize first" }) }] } };
expect(extractRemoteMcpPending(result, "arcade")).toMatchObject({ kind: "authorization", links: [{ host: "github.com", url: "https://github.com/login/oauth/authorize?state=test" }] });
});
it("recognizes nested Composio connection links without treating ordinary results as handoffs", () => {
expect(extractRemoteMcpPending({ data: { results: [{ redirect_url: "https://connect.composio.dev/link/test" }] } }, "composio", "COMPOSIO_MANAGE_CONNECTIONS")?.links).toHaveLength(1);
expect(extractRemoteMcpPending({ url: "https://example.com/article", execution_id: "completed" }, "executor")).toBeNull();
expect(extractRemoteMcpPending({ redirect_url: "https://example.com/article" }, "unrelated")).toBeNull();
});
it("preserves URL elicitation identities and drops unsafe navigation targets", () => {
const elicitations = [
{ mode: "url", elicitationId: "consent-1", url: "https://provider.example/approve" },
{ mode: "url", elicitationId: "consent-2", url: "javascript:alert(1)" },
{ mode: "url", elicitationId: "consent-3", url: "https://user:password@example.com" },
];
const pending = extractRemoteMcpPending({ error: { code: -32042, data: { elicitations } } });
expect(pending?.links).toEqual([{ url: "https://provider.example/approve", host: "provider.example", elicitationId: "consent-1" }]);
expect(pending?.elicitationId).toBe("consent-1");
});
it("keeps an Executor execution identity for manual resume, without inventing a retry", () => {
expect(extractRemoteMcpPending({ status: "waiting_for_interaction", executionId: "run-42", interaction: { kind: "form", message: "Approve read?", requestedSchema: { type: "object", properties: {} } } }, "executor")).toMatchObject({ kind: "approval", executionId: "run-42", resumeTool: "resume" });
});
it("does not turn ordinary successful app data into an approval or authorization handoff", () => {
for (const provider of ["arcade", "composio", "executor"]) {
for (const status of ["suspended", "pending_approval", "awaiting_approval", "waiting_for_interaction"]) {
expect(extractRemoteMcpPending({ result: { structuredContent: { records: [{ status, executionId: "app-job", interaction: { kind: "form" }, authorization_url: "https://example.com/auth", redirect_url: "https://connect.composio.dev/link/test" }] } } }, provider, "get_records")).toBeNull();
expect(extractRemoteMcpPending({ result: { structuredContent: { status } } }, provider, "get_record")).toBeNull();
}
expect(extractRemoteMcpPending({ result: { structuredContent: { mode: "url", elicitationId: "app-field", url: "https://example.com" } } }, provider)).toBeNull();
}
});
it("classifies broad execution and resume as writes even without annotations", () => {
for (const name of ["execute", "resume", "edit-artifact"]) expect(classifyRisk({ name }, "executor")).toBe("write");
expect(classifyRisk({ name: "skills", annotations: { readOnlyHint: true } }, "executor")).toBe("read");
expect(classifyRisk({ name: "COMPOSIO_MULTI_EXECUTE_TOOL", annotations: { readOnlyHint: true } }, "composio")).toBe("write");
});
it("defaults unfamiliar and namespaced aggregator capabilities to writes despite read-only hints", () => {
for (const provider of ["executor", "composio", "arcade", "zapier"]) {
for (const name of ["vendor.execute", "custom_resume", "code", "workbench", "new_capability", "get_and_run_action"]) {
for (const annotations of [undefined, { readOnlyHint: true }, { readOnlyHint: false }]) {
expect(classifyRisk({ name, annotations }, provider)).toBe("write");
}
}
expect(classifyRisk({ name: "delete_everything", annotations: { readOnlyHint: true } }, provider)).toBe("destructive");
}
for (const [provider, name] of [["executor", "skills"], ["composio", "COMPOSIO_SEARCH_TOOLS"], ["arcade", "Github.GetRepository"]]) {
expect(classifyRisk({ name }, provider)).toBe("read");
expect(classifyRisk({ name, annotations: { readOnlyHint: false } }, provider)).toBe("write");
expect(classifyRisk({ name, annotations: { destructiveHint: true } }, provider)).toBe("destructive");
expect(classifyRisk({ name: `custom.${name}`, annotations: { readOnlyHint: true } }, provider)).toBe("write");
}
expect(classifyRisk({ name: "GITHUB_LIST_REPOSITORIES", annotations: { readOnlyHint: true } })).toBe("read");
});
});
@@ -0,0 +1,79 @@
import { describe, expect, it, vi } from "vitest";
import { forgetMcpHttpSessions, getMcpHttpSession, initializeMcpHttpSession, McpHttpInitializationError, readMcpHttpResponse } from "../services/mcp-http.js";
const encoder = new TextEncoder();
function stream(chunks: string[], close = true) {
const cancel = vi.fn();
const response = new Response(new ReadableStream({ start(controller) { for (const chunk of chunks) controller.enqueue(encoder.encode(chunk)); if (close) controller.close(); }, cancel }), { headers: { "content-type": "text/event-stream" } });
return { response, cancel };
}
function event(message: unknown) { return `data: ${JSON.stringify(message)}\r\n\r\n`; }
describe("remote connector Streamable HTTP", () => {
it("matches the requested ID after progress and unrelated messages without waiting for stream closure", async () => {
const fixture = stream([event({ jsonrpc: "2.0", method: "notifications/progress", params: { progress: 1 } }), event({ jsonrpc: "2.0", id: "other", result: {} }), event({ jsonrpc: "2.0", id: "call", result: { content: [] } })], false);
expect(await readMcpHttpResponse(fixture.response, "call")).toEqual({ jsonrpc: "2.0", id: "call", result: { content: [] } });
expect(fixture.cancel).toHaveBeenCalledOnce();
});
it("decodes split CRLF and UTF-8 chunks", async () => {
const data = event({ id: 7, result: { text: "café" } });
expect(await readMcpHttpResponse(stream([...data]).response, 7)).toMatchObject({ result: { text: "café" } });
});
it("rejects an unrelated JSON result and bounds streamed responses", async () => {
await expect(readMcpHttpResponse(Response.json({ id: "wrong", result: {} }), "call")).rejects.toThrow("message ID");
await expect(readMcpHttpResponse(stream([event({ id: 1, result: "too much" })]).response, 1, { maxBytes: 8 })).rejects.toThrow("size limit");
});
it("applies matching, parse errors, and size limits to buffered HTTP transports", async () => {
const buffered = (body: string) => ({ headers: new Headers(), text: async () => body }) as Response;
expect(await readMcpHttpResponse(buffered('{"id":"call","result":{}}'), "call")).toMatchObject({ id: "call" });
await expect(readMcpHttpResponse(buffered('{"id":"other","result":{}}'), "call")).rejects.toMatchObject({ reason: "malformed_response" });
await expect(readMcpHttpResponse(buffered("not json"), "call")).rejects.toMatchObject({ reason: "invalid_json" });
await expect(readMcpHttpResponse(buffered("too large"), "call", { maxBytes: 2 })).rejects.toMatchObject({ reason: "too_large" });
});
it("delivers server requests before the matching response", async () => {
const onRequest = vi.fn(async () => {});
const request = { jsonrpc: "2.0", id: "auth", method: "elicitation/create", params: { mode: "url", url: "https://example.com/auth", elicitationId: "consent" } };
await readMcpHttpResponse(stream([event(request), event({ id: "call", result: {} })]).response, "call", { onRequest });
expect(onRequest).toHaveBeenCalledWith(request);
});
it("preserves an initialization authentication challenge for OAuth discovery", async () => {
const response = new Response(null, { status: 401, headers: { "www-authenticate": 'Bearer resource_metadata="https://example.com/.well-known/oauth-protected-resource"' } });
try { await initializeMcpHttpSession({ requestId: "a", send: async () => response }); throw new Error("Expected challenge"); }
catch (error) { expect(error).toBeInstanceOf(McpHttpInitializationError); expect((error as McpHttpInitializationError).response).toBe(response); }
});
it("initializes once per connection and identity and resets after revocation", async () => {
let count = 0;
const send = vi.fn(async (init: RequestInit) => {
const body = JSON.parse(String(init.body));
if (body.method === "initialize") { count++; return Response.json({ jsonrpc: "2.0", id: body.id, result: { protocolVersion: "2025-06-18", capabilities: {} } }, { headers: { "Mcp-Session-Id": `session-${count}` } }); }
expect(body.method).toBe("notifications/initialized");
return new Response(null, { status: 202 });
});
const input = { send, requestId: "a", scope: "connection-a:grant-a", headers: { Authorization: "Bearer secret-a" } };
const [a, repeated] = await Promise.all([getMcpHttpSession(input), getMcpHttpSession(input)]);
expect(a).toEqual(repeated);
expect((await getMcpHttpSession(input))["Mcp-Session-Id"]).toBe("session-1");
expect((await getMcpHttpSession({ ...input, scope: "connection-b:grant-a" }))["Mcp-Session-Id"]).toBe("session-2");
expect((await getMcpHttpSession({ ...input, scope: "connection-a:grant-b" }))["Mcp-Session-Id"]).toBe("session-3");
expect((await getMcpHttpSession({ ...input, headers: { Authorization: "Bearer rotated" } }))["Mcp-Session-Id"]).toBe("session-4");
forgetMcpHttpSessions("connection-a");
expect((await getMcpHttpSession(input))["Mcp-Session-Id"]).toBe("session-5");
expect(count).toBe(5);
});
it("does not resurrect an in-flight session after disconnect", async () => {
let finish: (() => void) | undefined;
const gate = new Promise<void>((resolve) => { finish = resolve; });
const send = async (init: RequestInit) => {
const body = JSON.parse(String(init.body));
if (body.method === "initialize") {
await gate;
return Response.json({ id: body.id, result: { protocolVersion: "2025-06-18" } }, { headers: { "Mcp-Session-Id": "revoked" } });
}
return new Response(null, { status: 202 });
};
const pending = getMcpHttpSession({ send, requestId: "revoking", scope: "revoking-connection:agent" });
forgetMcpHttpSessions("revoking-connection");
finish!();
await expect(pending).rejects.toThrow("connection changed");
});
});
@@ -0,0 +1,18 @@
import { describe, expect, it } from "vitest";
import { redactEventPayload, redactSensitiveText } from "../redaction.js";
describe("public Executor selectors and secret redaction", () => {
it("keeps only exact known public helper addresses readable in MCP results", () => {
const path = "executor.coreTools.integrations.list";
expect(redactEventPayload({ path, text: `Call tools.${path}({})` })).toEqual({ path, text: `Call tools.${path}({})` });
expect(redactSensitiveText(`${path}.privateSecretSuffix`)).toContain("REDACTED");
expect(redactEventPayload({ path: "arbitrary.provider.path" })?.path).toContain("REDACTED");
});
it("still redacts secret fields, bearer values, and JWT-shaped strings", () => {
const value = "executor.coreTools.integrations.list";
expect(redactEventPayload({ token: value })?.token).toContain("REDACTED");
expect(redactSensitiveText(`Authorization: Bearer ${value}`)).not.toContain(value);
expect(redactSensitiveText(`TOKEN=${value}`)).not.toContain(value);
expect(redactSensitiveText("eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.signature12345678")).toContain("REDACTED");
});
});
+8 -2
View File
@@ -546,6 +546,7 @@ describe("buildSentryInitOptions serverName", () => {
describe("buildSentryInitOptions release", () => {
const commit = "0123456789abcdef0123456789abcdef01234567";
const readBuildCommit = vi.fn<() => string | null>();
const integrations = {
httpIntegration: () => ({ name: "Http" }),
onUnhandledRejectionIntegration: () => ({ name: "OnUnhandledRejection" }),
@@ -553,12 +554,14 @@ describe("buildSentryInitOptions release", () => {
beforeEach(() => {
vi.stubEnv("SENTRY_RELEASE", "");
vi.doMock("../build-commit.js", () => ({ readBuildCommit: () => commit }));
readBuildCommit.mockReturnValue(commit);
vi.doMock("../build-commit.js", () => ({ readBuildCommit }));
});
afterEach(() => {
vi.unstubAllEnvs();
vi.doUnmock("../build-commit.js");
readBuildCommit.mockReset();
});
it("uses the server build commit", async () => {
@@ -573,9 +576,12 @@ describe("buildSentryInitOptions release", () => {
});
it("leaves an unknown build unattributed", async () => {
vi.doMock("../build-commit.js", () => ({ readBuildCommit: () => null }));
// Keep one module factory and change its return value explicitly for this
// case, rather than depending on a second factory replacing the first.
readBuildCommit.mockReturnValue(null);
const { buildSentryInitOptions } = await importFreshSentry();
expect(buildSentryInitOptions("test-dsn", integrations).release).toBeUndefined();
expect(readBuildCommit).toHaveBeenCalled();
});
});
@@ -72,6 +72,7 @@ import {
toolAccessService,
} from "../services/tool-access.js";
import { accessService } from "../services/access.js";
import { instanceSettingsService } from "../services/instance-settings.js";
import { toolAccessPolicyService } from "../services/tool-access-policy.js";
import { secretService } from "../services/secrets.js";
import {
@@ -361,15 +362,12 @@ function mcpSseResponse(payload: unknown): Response {
}
function mockToolsList(tools: unknown[]) {
return vi
.spyOn(globalThis, "fetch")
.mockResolvedValue(
mcpHttpResponse({
jsonrpc: "2.0",
id: "paperclip-catalog-refresh",
result: { tools },
}),
);
return vi.spyOn(globalThis, "fetch").mockImplementation(async (_url, init) => {
const body = JSON.parse(String(init?.body));
if (body.method === "notifications/initialized") return new Response(null, { status: 202 });
return mcpHttpResponse({ jsonrpc: "2.0", id: body.id,
result: body.method === "initialize" ? { protocolVersion: "2025-06-18" } : { tools } });
});
}
const PUBLIC_MCP_FIXTURE_URL = "https://8.8.8.8/api/mcp";
@@ -852,12 +850,14 @@ describeEmbeddedPostgres("tool access service", () => {
process.env.PAPERCLIP_TOOL_ACCESS_TEST_DATABASE_URL?.trim();
if (externalDatabaseUrl) {
db = createDb(externalDatabaseUrl);
await instanceSettingsService(db).updateExperimental({ enableMcpAggregators: true });
return;
}
tempDb = await startEmbeddedPostgresTestDatabase(
"paperclip-tool-access-service-",
);
db = createDb(tempDb.connectionString);
await instanceSettingsService(db).updateExperimental({ enableMcpAggregators: true });
}, 20_000);
afterEach(async () => {
@@ -3382,10 +3382,10 @@ describeEmbeddedPostgres("tool access service", () => {
priority: 100,
selectors: { connectionId: connection.id },
});
const fetchMock = vi.spyOn(globalThis, "fetch").mockResolvedValue(
const fetchMock = vi.spyOn(globalThis, "fetch").mockImplementation(async (_url, init) =>
mcpHttpResponse({
jsonrpc: "2.0",
id: "paperclip-tool-test",
id: JSON.parse(String(init?.body)).id,
result: { content: [{ type: "text", text: "sent" }] },
}),
);
@@ -3674,10 +3674,10 @@ describeEmbeddedPostgres("tool access service", () => {
expect(waiting.body.result).toBeUndefined();
// 3. Approving from the review queue is what runs the parked test call.
const fetchMock = vi.spyOn(globalThis, "fetch").mockResolvedValue(
const fetchMock = vi.spyOn(globalThis, "fetch").mockImplementation(async (_url, init) =>
mcpHttpResponse({
jsonrpc: "2.0",
id: "paperclip-tool-test",
id: JSON.parse(String(init?.body)).id,
result: { content: [{ type: "text", text: "sent" }] },
}),
);
@@ -3741,10 +3741,10 @@ describeEmbeddedPostgres("tool access service", () => {
.send(body)
.expect(200);
vi.spyOn(globalThis, "fetch").mockResolvedValue(
vi.spyOn(globalThis, "fetch").mockImplementation(async (_url, init) =>
mcpHttpResponse({
jsonrpc: "2.0",
id: "paperclip-tool-test",
id: JSON.parse(String(init?.body)).id,
result: { content: [{ type: "text", text: "sent" }] },
}),
);
@@ -5101,7 +5101,7 @@ describeEmbeddedPostgres("tool access service", () => {
"youcom",
]),
);
expect(res.body.apps).toHaveLength(48);
expect(res.body.apps).toHaveLength(51);
expect(
res.body.apps.find((app: { slug: string }) => app.slug === "gmail")
.ownershipAvailability,
@@ -1177,9 +1177,9 @@ describeEmbeddedPostgres("tool gateway service", () => {
selectors: { riskLevel: "read" },
});
const originalFetch = globalThis.fetch;
globalThis.fetch = async () => new Response(JSON.stringify({
globalThis.fetch = async (_url, init) => new Response(JSON.stringify({
jsonrpc: "2.0",
id: "paperclip-tool-test",
id: JSON.parse(String(init?.body)).id,
result: {
_meta: {
elicitation: {
@@ -1765,9 +1765,9 @@ describeEmbeddedPostgres("tool gateway service", () => {
selectors: { riskLevel: "read" },
});
const originalFetch = globalThis.fetch;
globalThis.fetch = async () => new Response(JSON.stringify({
globalThis.fetch = async (_url, init) => new Response(JSON.stringify({
jsonrpc: "2.0",
id: "paperclip-tool-test",
id: JSON.parse(String(init?.body)).id,
result: { elicitation: { message: "Need input" }, content: [] },
}), { status: 200, headers: { "content-type": "application/json" } });
try {
+5 -5
View File
@@ -585,10 +585,10 @@ describeEmbeddedPostgres("tool gateway acceptance", () => {
it("exposes a named gateway with scoped bearer-token auth and revocation", async () => {
const company = await createCompany(db);
const remote = await startFakeRemoteMcpServer(async () => ({
const remote = await startFakeRemoteMcpServer(async ({ body }) => ({
body: {
jsonrpc: "2.0",
id: "test",
id: body?.id,
result: { content: [{ type: "text", text: "read ok" }], structuredContent: { ok: true } },
},
}));
@@ -1127,10 +1127,10 @@ describeEmbeddedPostgres("tool gateway acceptance", () => {
it("rate limits public named gateway session setup, discovery, and calls with redacted audits", async () => {
const company = await createCompany(db);
const remote = await startFakeRemoteMcpServer(async () => ({
const remote = await startFakeRemoteMcpServer(async ({ body }) => ({
body: {
jsonrpc: "2.0",
id: "test",
id: body?.id,
result: { content: [{ type: "text", text: "read ok" }], structuredContent: { ok: true } },
},
}));
@@ -1561,7 +1561,7 @@ describeEmbeddedPostgres("tool gateway acceptance", () => {
if (upstreamCalls === 1) return new Response("unauthorized", { status: 401 });
return new Response(JSON.stringify({
jsonrpc: "2.0",
id: "fixture",
id: JSON.parse(String(init.body)).id,
result: { content: [{ type: "text", text: "repo-a" }], structuredContent: { repositories: ["repo-a"] } },
}), { status: 200, headers: { "content-type": "application/json" } });
},