mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-30 21:49:18 +01:00
## Thinking Path > - Paperclip connects agents to external tools through saved accounts and per-action permissions. > - Notebook exposes artifact management through an ordinary remote MCP server with OAuth. > - A private pilot needs a hidden, default-off catalog and execution gate. > - Local file publishing belongs to Notebook, which can accept bounded file bytes through ordinary MCP tools. > - Paperclip therefore needs the connector registration and generic availability checks, with no file-transfer architecture. ## Linked Issues or Issue Description **Subsystem affected** Apps catalog, experimental feature metadata, connection setup, and connector availability. **Problem or motivation** Users need to connect Notebook as their Cloud account, assign its tools to agents, and retain Paperclip's normal Allowed / Ask first / Off rules. The private pilot must remain hidden until an operator enables it. Notebook should own its upload protocol and local publishing script. **Proposed solution** Register a personal remote MCP OAuth connection at Notebook's canonical endpoint. Reuse existing OAuth discovery, credential storage, refresh, reconnect, and removal. Enforce availability through generic catalog feature metadata and saved application identity. Keep Notebook's standalone uploader in its own service repository; it calls the ordinary gateway and receives no saved OAuth credential. **Alternatives considered** A gallery-only filter permits retained tools and deep links while disabled. A shared workspace publisher and private upload-ticket transport add runtime behavior that a normal MCP connector does not require. Notebook can receive file bytes directly through its own tools instead. **Roadmap alignment** This extends Connected Apps and supports Artifacts and Work Products. ROADMAP.md and related PRs were checked. This PR is standalone on master. The separate OAuth lifecycle and file-transfer proposals (#17085 and #17086) are closed and are not dependencies. ## What Changed - Register Notebook's personal MCP OAuth method, canonical endpoint, scopes, catalog metadata, and branding. - Declare managed `enablePaperclipNotebook` with false defaults and no user-facing settings toggle or staging opt-in. - Use generic availability metadata for gallery/setup, retained connection identity, OAuth completion, health, tool/context discovery, and dispatch. Resources/prompts recheck before credentials, including retained handles. Removal remains available while disabled. - Resolve catalog application identity from the saved application key, so editable connection labels cannot bypass the hidden gate. - Select a method's sole supported credential kind when no kind is supplied. - Keep UI changes to metadata-driven availability and existing setup/reconnect controls. - Document Notebook's service-owned upload script, inline publication, ordinary action policies, and credential behavior. - Add ordinary MCP OAuth, assignment, action-policy, disabled-state, settings, and catalog tests. No workspace contribution registry, private transfer endpoint, native publisher, Runner file-reader change, or Paperclip CLI upload command is included. The entire native-runtime and CLI trees match master. Upload code lives in `paperclip-cloud/notebook/`. ## Verification Source head: `65ef741e4f476c67425b57d2fb2d6d4c00e99b06`, directly on master `53732c811bb69655eb43d3d12f2063ae4ca5ad67`. The diff is 42 files. Shared execution changes only enforce generic catalog availability; tool access also resolves saved application identity and the sole supported credential kind. Neither module contains Notebook upload logic. - Catalog/shared/settings/UI suites: 592 tests passed across 12 files. Shared/server/UI typechecks passed using installed compilers. - Fresh real PostgreSQL generic MCP suite: 84/84 passed at final source; server typecheck and module boundaries pass. Coverage includes personal OAuth, ordinary refresh/reconnect, assignments, Allowed / Ask first / Off, hidden feature enforcement, and local credential removal. Four new resources/prompts cases failed against the prior code and pass after filtering saved application identity and rechecking before credentials. They prove missing/relabelled setup markers and stale eligibility cannot dispatch or read secrets while disabled; an ordinary provider remains available with the flag off. - The full tool-access service suite passed 463/464 cases. One unchanged Google Workspace `slides.write` case timed out at 15 seconds; it passed on one isolated retry without source or timeout changes. Its exact test block matches master. The cause is not established; final-head CI must qualify the complete suite. - Maps, module boundaries, token gates, and whitespace checks passed. Byte comparisons confirm the former transfer/runtime/CLI surfaces match master. - Fresh exact-head Greptile is 5/5 with zero unresolved findings after an exhaustive feedback audit. Current-head CI workflow [39625691137](https://github.com/paperclipai/paperclip/actions/runs/39625691137) succeeded, including both required checks (`ci / verify`, `ci / e2e`), Build, and browser E2E. Supplementary security/advisory scan status is recorded in the PR Checks panel and qualification receipt; prior-head scans are not counted. The superseded `4bd7af9b2e` completed CI successfully, but Greptile found the valid resources/prompts availability gap described above. It was fixed, replied to, and resolved; earlier-head results do not qualify this revision. Preserved environment failures: the first socket-restricted UI/route run had 93 listener setup failures, and the same 93 cases passed with local-socket access. The package-manager launcher could not fetch its signature metadata; installed compilers passed without dependency or configuration changes. The full service timeout and isolated retry remain recorded separately. The superseded `585e870e73` CI also failed an unchanged agent-chat project-plan revision assertion after task creation; the complete spec and relevant runtime sources match master, and a same-master prior run passed. Its cause is not established. Logs are preserved; no test or timeout was weakened and no old-head rerun was requested. The subsequent heads run full CI independently; no prior-head pass substitutes for current qualification. The Notebook uploader requires a separate service release. This public PR adds no uploader runtime to Paperclip and claims no newly deployed browser/agent acceptance. The Notebook-owned upload path is qualified separately. Code-owner review remains required before merge. ## Risks - The feature flag gates the curated connector; it does not create a Notebook service-side team allowlist or block arbitrary standard MCP URLs. - Publishing, sharing, and deletion retain the existing assignment and action-policy boundaries. - The hidden feature must be checked for retained tools and saved identity, including when connection labels change. - Disconnect removes local credentials. It does not revoke the remote Notebook authorization; the owner can revoke it in Notebook. - Ordinary refresh uses the existing client. A lost response after refresh-token rotation can require reconnect. - The standalone Notebook script and its bounded upload tools must ship together in the Notebook service before that documented convenience path is available. - No database schema, workflow, lockfile, shared transfer, or Runner change is included. No merge, deployment, or rollout is performed by this update. ## Model Used OpenAI GPT-6 in Codex, with reasoning, repository inspection, code execution, and tool use. The runtime did not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
292 lines
9.9 KiB
TypeScript
292 lines
9.9 KiB
TypeScript
import { APP_DEFINITIONS } from "./app-definitions.generated.js";
|
|
import { SELF_SERVE_MCP_CANDIDATES } from "./self-serve-mcp-research.js";
|
|
import type { AppDefinition, ConnectionMethodDef, FieldDef } from "./types/app-definition.js";
|
|
import type { ToolConnectionOwnership } from "./types/tool-access.js";
|
|
|
|
export const CONNECTABLE_APP_SLUGS = new Set([
|
|
"anthropic", "openai", "openrouter", "xai", "google", "bedrock",
|
|
"responses-api", "messages-api", "chat-completions-api", "local",
|
|
"agentmail",
|
|
"browser-use-cloud",
|
|
"daytona",
|
|
"cognee",
|
|
...SELF_SERVE_MCP_CANDIDATES.map((entry) => entry.slug),
|
|
"monday",
|
|
"enterpret",
|
|
"zapier",
|
|
"arcade",
|
|
"executor",
|
|
"slack",
|
|
"notion",
|
|
"paperclip-notebook",
|
|
"railway",
|
|
"posthog",
|
|
"linear",
|
|
"google-sheets",
|
|
"context7",
|
|
"shopify",
|
|
"composio",
|
|
"gmail",
|
|
"google-drive",
|
|
"google-docs",
|
|
"google-slides",
|
|
"google-calendar",
|
|
"google-chat",
|
|
"google-people",
|
|
"google-workspace-search",
|
|
"google-workspace",
|
|
"github",
|
|
"github-code-review-bot",
|
|
"discord",
|
|
"speko",
|
|
"microsoft-teams",
|
|
"telegram",
|
|
"imessage-photon",
|
|
]);
|
|
|
|
export const CONNECTABLE_APP_DEFINITIONS = APP_DEFINITIONS.filter((app) =>
|
|
CONNECTABLE_APP_SLUGS.has(app.slug)
|
|
);
|
|
|
|
/**
|
|
* Definitions retained for existing connections and later verification, but
|
|
* intentionally withheld from the customer-facing store. Keeping visibility
|
|
* separate from recognition avoids breaking saved connections when a provider
|
|
* is pulled from Browse or reserved for a future first-party experience.
|
|
*/
|
|
export const APP_STORE_HIDDEN_SLUGS = new Set([
|
|
"beehiiv",
|
|
"brex",
|
|
"candid",
|
|
"clickup",
|
|
"coda",
|
|
"egnyte",
|
|
"embat",
|
|
"kernel",
|
|
"klaviyo",
|
|
"local-falcon",
|
|
"manufact",
|
|
"monday",
|
|
"oreilly",
|
|
"planetscale",
|
|
"razorpay",
|
|
"sanity",
|
|
"similarweb",
|
|
"ticket-tailor",
|
|
"ticktick",
|
|
"xero",
|
|
]);
|
|
|
|
export const APP_STORE_DEFINITIONS = CONNECTABLE_APP_DEFINITIONS.filter((app) =>
|
|
!APP_STORE_HIDDEN_SLUGS.has(app.slug)
|
|
);
|
|
|
|
export const DEFAULT_OWNERSHIP_AVAILABILITY: Record<ToolConnectionOwnership, boolean> = {
|
|
platform_shared: false,
|
|
platform_provisioned: false,
|
|
customer: true,
|
|
dcr: true,
|
|
};
|
|
|
|
export function getConnectableAppDefinition(slug: string): AppDefinition | null {
|
|
return CONNECTABLE_APP_DEFINITIONS.find((app) => app.slug === slug) ?? null;
|
|
}
|
|
|
|
export function getAppStoreDefinition(slug: string): AppDefinition | null {
|
|
return APP_STORE_DEFINITIONS.find((app) => app.slug === slug) ?? null;
|
|
}
|
|
|
|
export function isAppStoreVisibleSlug(slug: string | null | undefined): boolean {
|
|
return Boolean(slug && !APP_STORE_HIDDEN_SLUGS.has(slug) && CONNECTABLE_APP_SLUGS.has(slug));
|
|
}
|
|
|
|
function wildcardPatternToRegExp(pattern: string): RegExp {
|
|
const escaped = pattern.replace(/[.+?^${}()|[\]\\]/g, "\\$&").replace(/\*/g, ".*");
|
|
return new RegExp(`^${escaped}$`, "i");
|
|
}
|
|
|
|
export function getAppDefinitionForUrl(
|
|
link: string,
|
|
definitions: readonly AppDefinition[] = CONNECTABLE_APP_DEFINITIONS,
|
|
): AppDefinition | null {
|
|
let normalized: string;
|
|
try {
|
|
normalized = new URL(link.trim()).toString();
|
|
} catch {
|
|
return null;
|
|
}
|
|
return definitions.find((app) =>
|
|
app.urlPatterns.some((pattern) => wildcardPatternToRegExp(pattern).test(normalized))
|
|
) ?? null;
|
|
}
|
|
|
|
export function getAvailableConnectionMethods(app: AppDefinition): ConnectionMethodDef[] {
|
|
const availability = app.ownershipAvailability ?? DEFAULT_OWNERSHIP_AVAILABILITY;
|
|
return app.methods.filter((method) =>
|
|
method.ownershipModes.some((ownership) => availability[ownership] !== false)
|
|
);
|
|
}
|
|
|
|
/**
|
|
* Pick the method that gives a new connection the app's useful write surface.
|
|
*
|
|
* Google Workspace publishes separate read and write capability groups. The
|
|
* read method is intentionally listed first for documentation, but treating
|
|
* array order as a product default silently created read-only connections and
|
|
* left every write action Off. Capability metadata is the durable signal; apps
|
|
* without an explicit write/draft capability retain their declared order.
|
|
*/
|
|
export function getRecommendedConnectionMethod(
|
|
methods: readonly ConnectionMethodDef[],
|
|
): ConnectionMethodDef | null {
|
|
const recommendedCapability = (candidates: readonly ConnectionMethodDef[]) => candidates.find((method) => {
|
|
const capabilityKey = method.capabilityProfile?.key;
|
|
return capabilityKey === "write" || capabilityKey === "draft";
|
|
});
|
|
const managedMethods = methods.filter((method) =>
|
|
method.oauthStrategy === "paperclip_cloud_connector"
|
|
|| method.oauthStrategy === "paperclip_id_connector"
|
|
);
|
|
|
|
// Prefer the permissions needed for agent work, then the simplest sign-in.
|
|
// Explicit read-only selections pass their own capability group here.
|
|
return recommendedCapability(managedMethods)
|
|
?? recommendedCapability(methods)
|
|
?? managedMethods[0]
|
|
?? methods[0]
|
|
?? null;
|
|
}
|
|
|
|
export function getAvailableConnectionMethod(
|
|
app: AppDefinition,
|
|
methodKey?: string | null,
|
|
): ConnectionMethodDef | null {
|
|
const methods = getAvailableConnectionMethods(app);
|
|
return methodKey
|
|
? methods.find((method) => method.key === methodKey) ?? null
|
|
: getRecommendedConnectionMethod(methods);
|
|
}
|
|
|
|
export function connectionMethodSupportsAutomaticOAuth(method: ConnectionMethodDef | null | undefined): boolean {
|
|
return method?.auth === "oauth" && (
|
|
(method.oauthStrategy === "paperclip_cloud_connector" || method.oauthStrategy === "paperclip_id_connector")
|
|
|| method.ownershipModes.includes("dcr")
|
|
);
|
|
}
|
|
|
|
export function connectionMethodAcceptsCustomerOAuthClient(method: ConnectionMethodDef | null | undefined): boolean {
|
|
return method?.auth === "oauth"
|
|
&& !method.oauthStrategy
|
|
&& method.ownershipModes.includes("customer");
|
|
}
|
|
|
|
export function connectionMethodSupportsCatalogSetup(method: ConnectionMethodDef | null | undefined): boolean {
|
|
if (!method) return false;
|
|
if (method.transport === "runtime_auth") return Boolean(method.ai);
|
|
if (method.auth === "none" || method.auth === "api_key") return true;
|
|
return connectionMethodSupportsAutomaticOAuth(method)
|
|
|| connectionMethodAcceptsCustomerOAuthClient(method);
|
|
}
|
|
|
|
export function connectionMethodRequiresConfiguration(method: ConnectionMethodDef | null | undefined): boolean {
|
|
if (!method) return false;
|
|
const visibleTenantFields = method.tenantFields?.filter((field) => !field.hidden) ?? [];
|
|
const visibleExtensionFields = method.extensionFields?.filter((field) => !field.hidden) ?? [];
|
|
return Boolean(
|
|
method.credentialFields?.length
|
|
|| visibleTenantFields.length
|
|
|| visibleExtensionFields.length
|
|
|| method.configRequirements?.atLeastOneOf?.length
|
|
// "Use your own OAuth app" is an advanced alternative when DCR/CIMD is
|
|
// available, not a required setup field. Only customer-client-only methods
|
|
// must stop on the configuration screen.
|
|
|| (
|
|
connectionMethodAcceptsCustomerOAuthClient(method)
|
|
&& !connectionMethodSupportsAutomaticOAuth(method)
|
|
),
|
|
);
|
|
}
|
|
|
|
export function appSupportsCatalogSetup(app: AppDefinition | null | undefined): boolean {
|
|
return Boolean(app && getAvailableConnectionMethods(app).some(connectionMethodSupportsCatalogSetup));
|
|
}
|
|
|
|
export function isConnectableAppSlug(slug: string | null | undefined): boolean {
|
|
return Boolean(slug && CONNECTABLE_APP_SLUGS.has(slug));
|
|
}
|
|
|
|
export function appSupportsAutomaticOAuth(app: AppDefinition | null | undefined): boolean {
|
|
return Boolean(app && getAvailableConnectionMethods(app).some(connectionMethodSupportsAutomaticOAuth));
|
|
}
|
|
|
|
export function appAcceptsCustomerOAuthClient(app: AppDefinition | null | undefined): boolean {
|
|
return Boolean(app && getAvailableConnectionMethods(app).some(connectionMethodAcceptsCustomerOAuthClient));
|
|
}
|
|
|
|
export function credentialConfigPath(field: FieldDef, method?: ConnectionMethodDef | null): string {
|
|
if (method?.transport === "local_stdio" && method.keyPlacement?.location === "env") return `env.${field.key}`;
|
|
return `credentials.${field.key}`;
|
|
}
|
|
|
|
/** Older credential references used bare names; current references use paths. */
|
|
export function connectionCredentialConfigPath(ref: { name: string }): string {
|
|
return /^(credentials|headers|oauth|remote)\./.test(ref.name)
|
|
? ref.name
|
|
: `credentials.${ref.name}`;
|
|
}
|
|
|
|
export function resolveConnectionMethodServerUrl(
|
|
method: ConnectionMethodDef,
|
|
configValues: Record<string, string | boolean>,
|
|
): string | null {
|
|
const template = method.defaults?.serverUrlTemplate;
|
|
if (!template) return method.defaults?.serverUrl ?? null;
|
|
|
|
let missingValue = false;
|
|
const resolved = template.replace(/\{([a-zA-Z0-9_-]+)\}/g, (_placeholder, key: string) => {
|
|
const value = configValues[key];
|
|
if (value === undefined || String(value).trim().length === 0) {
|
|
missingValue = true;
|
|
return "";
|
|
}
|
|
return encodeURIComponent(String(value).trim());
|
|
});
|
|
if (missingValue) return null;
|
|
|
|
try {
|
|
return new URL(resolved).toString();
|
|
} catch {
|
|
return null;
|
|
}
|
|
}
|
|
|
|
export function recommendedDefaultsForApp(app: AppDefinition, methodKey?: string | null): Record<string, unknown> {
|
|
// Keep the parameters in the public contract: callers resolve defaults for a
|
|
// concrete app/method. Connection finalization remains a configure-authorized,
|
|
// audited operation, and Ask first stays available as an operator-selected
|
|
// policy for any action after the connection is made.
|
|
const method = methodKey
|
|
? app.methods.find((candidate) => candidate.key === methodKey)
|
|
: null;
|
|
if (app.slug === "gmail" && method?.capabilityProfile?.key === "archive") {
|
|
return {
|
|
access: "all_agents",
|
|
askFirstRiskLevels: ["write", "destructive"],
|
|
};
|
|
}
|
|
// Enterpret's run_graph_query self-reports readOnlyHint but executes Cypher
|
|
// and has not been established as read-only. Classify it as write (server
|
|
// classifyRisk) and require Ask first so new token connections do not ship
|
|
// that action as Allowed.
|
|
if (app.slug === "enterpret") {
|
|
return {
|
|
access: "all_agents",
|
|
askFirstRiskLevels: ["write", "destructive"],
|
|
};
|
|
}
|
|
return {
|
|
access: "all_agents",
|
|
askFirstRiskLevels: [],
|
|
};
|
|
}
|