mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-24 18:19:05 +02:00
## Thinking Path > - Paperclip manages AI agents and their work. > - Connections give agents governed access to external tools. > - Agents need durable memory across tasks and execution environments. > - Mem0, Zep, Supermemory, Cognee, and Honcho provide hosted memory tools. > - This pull request adds their setup flows behind an experimental toggle. > - It also delivers assigned MCP tools through the native remote Codex runner. > - Operators can connect a provider once and use the same governed tools locally or in Daytona. ## Linked Issues or Issue Description **Problem or motivation** The Apps catalog lacks a complete set of memory providers. Remote native Codex agents also need access to assigned managed MCP tools without receiving provider credentials. **Proposed solution** Add five memory connectors behind the disabled-by-default Experimental memory connectors setting. Use the existing connection setup and permissions UI. Default their tools to Allowed. Preserve company boundaries, operator permission changes, provider scopes, and audit attribution. **Alternatives considered** Direct provider credentials in each sandbox would duplicate setup and bypass the managed gateway. The remote runner instead uses its existing protocol channel to call the gateway on the server. **Roadmap alignment** This maintainer-requested experiment supports the Memory / Knowledge and Connected Apps roadmap areas. It adds provider connections without introducing a separate memory UI. Related connector authoring documentation is tracked in #13692; no duplicate memory-provider implementation was found. ## What Changed - Add provider definitions, official branding, and the experimental setting for all five providers. - Use OAuth for Zep and Supermemory, API credentials for Mem0 and Honcho, and a bundled Cloud API bridge for Cognee with no runtime downloads or subprocesses. - Default memory tools to Allowed and classify destructive actions explicitly. - Fix personal remote credential resolution and propagate provider tool errors. - Relay assigned managed MCP tools to remote native Codex through the runner protocol. Recheck current authority for each call and rotate stale tool contracts. - Add 21 Storybook states and complete OAuth walkthrough fixtures. - Document provider research, sanitized tool inventory, and live local and Daytona proof. ## Verification - Passed workspace typecheck: `pnpm -r typecheck`. - Passed production build: `pnpm build`. - Full local `pnpm test:run`: 13,273 passed, with failures from process/readiness timeouts under parallel load. Reran all 16 affected suites with one worker: 456 passed, leaving two macOS `/var` versus `/private/var` path assertions. Both passed with `TMPDIR=/private/tmp`. No test failures remain unverified. Latest-head remote CI passes all 54 checks (two optional Storybook jobs skipped). Greptile is 5/5 with no unresolved findings. - Latest Cognee gateway regression: 71 passed, including public deployment without a runtime host and immediate recovery after a provider error. Bundled bridge tests: 25 passed. - Browser setup and real agent tasks exercised all five providers. Mem0, Cognee, Zep, and Honcho have successful store/retrieve proof. - Supermemory now has scoped read/write consent. Local storage and real Daytona write, document read, and semantic recall passed; indexing completion was verified before claiming success. - All five providers were exercised through a real Daytona sandbox and its native runner MCP relay. Provider credentials remained on the server. The final bundled Cognee bridge also passed a fresh Daytona store/recall run. All disposable sandboxes were removed and verified absent after testing. - Storybook is rebuilt and contains the experimental toggle, catalog, setup, permissions, and error states. The Zep and Supermemory access steps advance correctly. ## Risks - Provider OAuth scopes and plan limits remain independent of Paperclip tool permissions. An Allowed tool can still be rejected by the provider. - Providers can queue memory indexing; save acceptance does not prove that semantic recall is ready. - Remote tool contracts must stay synchronized with current connection authority. Regression tests cover revocation and stale contracts. - This change has no database migration. Existing connections remain usable when the experimental catalog toggle is disabled. ## Model Used OpenAI Codex, GPT-6 Astra (`gpt-6-astra`), with reasoning, repository edits, code execution, and browser automation. The exact context window size is not exposed in this session. Live acceptance agents used `gpt-5.6-sol`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
262 lines
9.0 KiB
TypeScript
262 lines
9.0 KiB
TypeScript
import { APP_DEFINITIONS } from "./app-definitions.generated.js";
|
|
import { SELF_SERVE_MCP_CANDIDATES } from "./self-serve-mcp-research.js";
|
|
import type { AppDefinition, ConnectionMethodDef, FieldDef } from "./types/app-definition.js";
|
|
import type { ToolConnectionOwnership } from "./types/tool-access.js";
|
|
|
|
export const CONNECTABLE_APP_SLUGS = new Set([
|
|
"anthropic", "openai", "openrouter", "xai",
|
|
"agentmail",
|
|
"cognee",
|
|
...SELF_SERVE_MCP_CANDIDATES.map((entry) => entry.slug),
|
|
"zapier",
|
|
"arcade",
|
|
"executor",
|
|
"slack",
|
|
"notion",
|
|
"railway",
|
|
"posthog",
|
|
"linear",
|
|
"google-sheets",
|
|
"context7",
|
|
"shopify",
|
|
"composio",
|
|
"gmail",
|
|
"google-drive",
|
|
"google-docs",
|
|
"google-slides",
|
|
"google-calendar",
|
|
"google-chat",
|
|
"google-people",
|
|
"google-workspace-search",
|
|
"github",
|
|
"discord",
|
|
"microsoft-teams",
|
|
"telegram",
|
|
"imessage-photon",
|
|
]);
|
|
|
|
export const CONNECTABLE_APP_DEFINITIONS = APP_DEFINITIONS.filter((app) =>
|
|
CONNECTABLE_APP_SLUGS.has(app.slug)
|
|
);
|
|
|
|
/**
|
|
* Definitions retained for existing connections and later verification, but
|
|
* intentionally withheld from the customer-facing store. Keeping visibility
|
|
* separate from recognition avoids breaking saved connections when a provider
|
|
* is pulled from Browse or reserved for a future first-party experience.
|
|
*/
|
|
export const APP_STORE_HIDDEN_SLUGS = new Set([
|
|
"beehiiv",
|
|
"bitly",
|
|
"brex",
|
|
"candid",
|
|
"coda",
|
|
"context7",
|
|
"egnyte",
|
|
"embat",
|
|
"kernel",
|
|
"local-falcon",
|
|
"make",
|
|
"manufact",
|
|
"oreilly",
|
|
"planetscale",
|
|
"razorpay",
|
|
"sanity",
|
|
"similarweb",
|
|
"ticket-tailor",
|
|
"ticktick",
|
|
"xero",
|
|
]);
|
|
|
|
export const APP_STORE_DEFINITIONS = CONNECTABLE_APP_DEFINITIONS.filter((app) =>
|
|
!APP_STORE_HIDDEN_SLUGS.has(app.slug)
|
|
);
|
|
|
|
export const DEFAULT_OWNERSHIP_AVAILABILITY: Record<ToolConnectionOwnership, boolean> = {
|
|
platform_shared: false,
|
|
platform_provisioned: false,
|
|
customer: true,
|
|
dcr: true,
|
|
};
|
|
|
|
export function getConnectableAppDefinition(slug: string): AppDefinition | null {
|
|
return CONNECTABLE_APP_DEFINITIONS.find((app) => app.slug === slug) ?? null;
|
|
}
|
|
|
|
export function getAppStoreDefinition(slug: string): AppDefinition | null {
|
|
return APP_STORE_DEFINITIONS.find((app) => app.slug === slug) ?? null;
|
|
}
|
|
|
|
export function isAppStoreVisibleSlug(slug: string | null | undefined): boolean {
|
|
return Boolean(slug && !APP_STORE_HIDDEN_SLUGS.has(slug) && CONNECTABLE_APP_SLUGS.has(slug));
|
|
}
|
|
|
|
function wildcardPatternToRegExp(pattern: string): RegExp {
|
|
const escaped = pattern.replace(/[.+?^${}()|[\]\\]/g, "\\$&").replace(/\*/g, ".*");
|
|
return new RegExp(`^${escaped}$`, "i");
|
|
}
|
|
|
|
export function getAppDefinitionForUrl(
|
|
link: string,
|
|
definitions: readonly AppDefinition[] = CONNECTABLE_APP_DEFINITIONS,
|
|
): AppDefinition | null {
|
|
let normalized: string;
|
|
try {
|
|
normalized = new URL(link.trim()).toString();
|
|
} catch {
|
|
return null;
|
|
}
|
|
return definitions.find((app) =>
|
|
app.urlPatterns.some((pattern) => wildcardPatternToRegExp(pattern).test(normalized))
|
|
) ?? null;
|
|
}
|
|
|
|
export function getAvailableConnectionMethods(app: AppDefinition): ConnectionMethodDef[] {
|
|
const availability = app.ownershipAvailability ?? DEFAULT_OWNERSHIP_AVAILABILITY;
|
|
return app.methods.filter((method) =>
|
|
method.ownershipModes.some((ownership) => availability[ownership] !== false)
|
|
);
|
|
}
|
|
|
|
/**
|
|
* Pick the method that gives a new connection the app's useful write surface.
|
|
*
|
|
* Google Workspace publishes separate read and write capability groups. The
|
|
* read method is intentionally listed first for documentation, but treating
|
|
* array order as a product default silently created read-only connections and
|
|
* left every write action Off. Capability metadata is the durable signal; apps
|
|
* without an explicit write/draft capability retain their declared order.
|
|
*/
|
|
export function getRecommendedConnectionMethod(
|
|
methods: readonly ConnectionMethodDef[],
|
|
): ConnectionMethodDef | null {
|
|
const recommendedCapability = (candidates: readonly ConnectionMethodDef[]) => candidates.find((method) => {
|
|
const capabilityKey = method.capabilityProfile?.key;
|
|
return capabilityKey === "write" || capabilityKey === "draft";
|
|
});
|
|
const managedMethods = methods.filter((method) =>
|
|
method.oauthStrategy === "paperclip_cloud_connector"
|
|
|| method.oauthStrategy === "paperclip_id_connector"
|
|
);
|
|
|
|
// When a managed pilot advertises only read access, defaulting to a
|
|
// customer-owned write method would turn the available one-click path into
|
|
// an OAuth client setup form. Capability-specific callers pass only the
|
|
// selected group, so explicit write/draft choices keep their own fallback.
|
|
return recommendedCapability(managedMethods)
|
|
?? managedMethods[0]
|
|
?? recommendedCapability(methods)
|
|
?? methods[0]
|
|
?? null;
|
|
}
|
|
|
|
export function getAvailableConnectionMethod(
|
|
app: AppDefinition,
|
|
methodKey?: string | null,
|
|
): ConnectionMethodDef | null {
|
|
const methods = getAvailableConnectionMethods(app);
|
|
return methodKey
|
|
? methods.find((method) => method.key === methodKey) ?? null
|
|
: getRecommendedConnectionMethod(methods);
|
|
}
|
|
|
|
export function connectionMethodSupportsAutomaticOAuth(method: ConnectionMethodDef | null | undefined): boolean {
|
|
return method?.auth === "oauth" && (
|
|
(method.oauthStrategy === "paperclip_cloud_connector" || method.oauthStrategy === "paperclip_id_connector")
|
|
|| method.ownershipModes.includes("dcr")
|
|
);
|
|
}
|
|
|
|
export function connectionMethodAcceptsCustomerOAuthClient(method: ConnectionMethodDef | null | undefined): boolean {
|
|
return method?.auth === "oauth"
|
|
&& !method.oauthStrategy
|
|
&& method.ownershipModes.includes("customer");
|
|
}
|
|
|
|
export function connectionMethodSupportsCatalogSetup(method: ConnectionMethodDef | null | undefined): boolean {
|
|
if (!method) return false;
|
|
if (method.transport === "runtime_auth") return Boolean(method.ai);
|
|
if (method.auth === "none" || method.auth === "api_key") return true;
|
|
return connectionMethodSupportsAutomaticOAuth(method)
|
|
|| connectionMethodAcceptsCustomerOAuthClient(method);
|
|
}
|
|
|
|
export function connectionMethodRequiresConfiguration(method: ConnectionMethodDef | null | undefined): boolean {
|
|
if (!method) return false;
|
|
const visibleTenantFields = method.tenantFields?.filter((field) => !field.hidden) ?? [];
|
|
const visibleExtensionFields = method.extensionFields?.filter((field) => !field.hidden) ?? [];
|
|
return Boolean(
|
|
method.credentialFields?.length
|
|
|| visibleTenantFields.length
|
|
|| visibleExtensionFields.length
|
|
|| method.configRequirements?.atLeastOneOf?.length
|
|
// "Use your own OAuth app" is an advanced alternative when DCR/CIMD is
|
|
// available, not a required setup field. Only customer-client-only methods
|
|
// must stop on the configuration screen.
|
|
|| (
|
|
connectionMethodAcceptsCustomerOAuthClient(method)
|
|
&& !connectionMethodSupportsAutomaticOAuth(method)
|
|
),
|
|
);
|
|
}
|
|
|
|
export function appSupportsCatalogSetup(app: AppDefinition | null | undefined): boolean {
|
|
return Boolean(app && getAvailableConnectionMethods(app).some(connectionMethodSupportsCatalogSetup));
|
|
}
|
|
|
|
export function isConnectableAppSlug(slug: string | null | undefined): boolean {
|
|
return Boolean(slug && CONNECTABLE_APP_SLUGS.has(slug));
|
|
}
|
|
|
|
export function appSupportsAutomaticOAuth(app: AppDefinition | null | undefined): boolean {
|
|
return Boolean(app && getAvailableConnectionMethods(app).some(connectionMethodSupportsAutomaticOAuth));
|
|
}
|
|
|
|
export function appAcceptsCustomerOAuthClient(app: AppDefinition | null | undefined): boolean {
|
|
return Boolean(app && getAvailableConnectionMethods(app).some(connectionMethodAcceptsCustomerOAuthClient));
|
|
}
|
|
|
|
export function credentialConfigPath(field: FieldDef, method?: ConnectionMethodDef | null): string {
|
|
if (method?.transport === "local_stdio" && method.keyPlacement?.location === "env") return `env.${field.key}`;
|
|
return `credentials.${field.key}`;
|
|
}
|
|
|
|
export function resolveConnectionMethodServerUrl(
|
|
method: ConnectionMethodDef,
|
|
configValues: Record<string, string | boolean>,
|
|
): string | null {
|
|
const template = method.defaults?.serverUrlTemplate;
|
|
if (!template) return method.defaults?.serverUrl ?? null;
|
|
|
|
let missingValue = false;
|
|
const resolved = template.replace(/\{([a-zA-Z0-9_-]+)\}/g, (_placeholder, key: string) => {
|
|
const value = configValues[key];
|
|
if (value === undefined || String(value).trim().length === 0) {
|
|
missingValue = true;
|
|
return "";
|
|
}
|
|
return encodeURIComponent(String(value).trim());
|
|
});
|
|
if (missingValue) return null;
|
|
|
|
try {
|
|
return new URL(resolved).toString();
|
|
} catch {
|
|
return null;
|
|
}
|
|
}
|
|
|
|
export function recommendedDefaultsForApp(app: AppDefinition, methodKey?: string | null): Record<string, unknown> {
|
|
// Keep the parameters in the public contract: callers resolve defaults for a
|
|
// concrete app/method even though the initial policy is now uniform. This is
|
|
// an open default, not an approval bypass: connection finalization remains a
|
|
// configure-authorized, audited operation, and Ask first stays available as
|
|
// an operator-selected policy for any action after the connection is made.
|
|
void app;
|
|
void methodKey;
|
|
return {
|
|
access: "all_agents",
|
|
askFirstRiskLevels: [],
|
|
};
|
|
}
|