mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-29 21:07:40 +01:00
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies and their governed access to external systems. > - Connected Apps build on the existing Apps and MCP gateway substrate so companies can configure reusable, auditable integrations. > - The current connection record does not yet have a stable public address, explicit ownership/auth method fields, or subject-specific credential grants. > - Without that schema core, later OAuth, per-user authorization, token brokering, triggers, and connector-service phases cannot enforce tenant and subject boundaries consistently. > - This pull request adds the forward-compatible Connections v3 schema core while preserving the existing connection lifecycle and directly migrating the remote MCP transport name. > - The benefit is a company-scoped, least-privilege foundation for one-click integrations without bypassing Paperclip secrets, profiles, rules, or audit controls. ## Linked Issues or Issue Description No matching public issue was found. **Problem** Paperclip's current app connections need a durable identity and authorization substrate before Connected Apps can safely support multiple setup methods, per-user credentials, provider tenants, and managed connector services. The existing schema only models a single connection-level credential set and uses legacy transport terminology. **Proposed solution** Add a stable company-scoped connection UID, explicit ownership/auth/transport fields, a subject-aware `connection_grants` table, and multi-key credential annotations. Backfill existing connections and workspace grants in a reversible migration, then update shared/server/UI contracts to the new `mcp_remote` transport name. **Related work** - Related foundation: #9534 - Roadmap: Connected Apps (one-click integrations) ## What Changed - Added company-scoped connection `uid`, `ownership`, `authKind`, and canonical transport fields across database, shared contracts, validators, services, and UI fixtures. - Added `connection_grants` with workspace/user subject rules, provider tenant metadata, credential secret refs, revocation state, company scoping, and uniqueness constraints. - Added migration `0182_connections_v3_schema_core` to backfill stable UIDs, rename `remote_http` to `mcp_remote`, infer auth kinds, create default workspace grants, and support rollback coverage. - Added multi-key credential annotations and updated gateway/access services without changing the existing lifecycle behavior. - Updated the connection glossary, connector playbook, and security threat model for the new identity, grant, and relay boundaries. - Added explicit test UIDs to direct database fixtures so the new non-null invariant is exercised across affected server suites. ## Verification - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/db typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/ui typecheck` - `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts server/src/__tests__/tool-gateway-service.test.ts server/src/__tests__/tool-gateway.test.ts server/src/__tests__/heartbeat-runtime-skills.test.ts server/src/__tests__/tool-oauth-legacy-backfill.test.ts server/src/__tests__/tool-access-policy-service.test.ts server/src/__tests__/heartbeat-runtime-mcp-servers.test.ts packages/db/src/connections-v3-schema-core-migration.test.ts packages/shared/src/validators/tool-access.test.ts --config vitest.config.ts` — 9 files, 218 tests passed. - Latest-head GitHub Actions: build, typecheck, general/serialized suites, backup/worktree restore coverage, both e2e shards, canary, policy, and security scans pass. - Greptile: 5/5 with zero unresolved threads. - `pnpm check:token-gates` remains red only on five pre-existing `#9627` color literals outside this change. ## Risks - **Migration risk:** UID backfill and default-grant creation touch every existing connection. The migration uses company-scoped uniqueness, deterministic legacy UIDs with ID suffixes, and seeded up/rollback coverage. - **Authorization risk:** Grant rows carry credential references. Constraints enforce workspace-vs-user subject shape, company/connection lookup indexes, one default grant per connection, and one user grant per connection/subject. Security review is requested specifically for this design. - **Compatibility risk:** `remote_http` is renamed directly to `mcp_remote`; all repository call sites and fixtures are updated in the same change. - **Future-phase risk:** Subject-bound token issuance, triggers, and connector-service relay verification remain fail-closed requirements documented for later phases; this PR does not expose those capabilities. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex CLI coding agent. The runtime did not expose an exact underlying model ID or context-window size; capabilities used include repository inspection, code editing, shell execution, test execution, Git/GitHub CLI operations, and structured reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
162 lines
5.9 KiB
TypeScript
162 lines
5.9 KiB
TypeScript
import { lookup as dnsLookup } from "node:dns/promises";
|
|
import { isIP } from "node:net";
|
|
|
|
const DEFAULT_DNS_TIMEOUT_MS = 5_000;
|
|
|
|
type LookupResult = { address: string; family: number };
|
|
|
|
export type RemoteHttpEndpointLookup = (hostname: string) => Promise<LookupResult[]>;
|
|
|
|
export type RemoteHttpEndpointGuardOptions = {
|
|
allowPrivateNetwork?: boolean;
|
|
dnsTimeoutMs?: number;
|
|
lookup?: RemoteHttpEndpointLookup;
|
|
};
|
|
|
|
export type RemoteHttpEndpointErrorFactory = (message: string, code: string) => Error;
|
|
|
|
export function parseRemoteHttpEndpoint(
|
|
value: unknown,
|
|
error: RemoteHttpEndpointErrorFactory,
|
|
): URL {
|
|
if (typeof value !== "string" || value.trim().length === 0) {
|
|
throw error("Remote MCP connection requires config.url", "mcp_remote_url_missing");
|
|
}
|
|
let parsed: URL;
|
|
try {
|
|
parsed = new URL(value);
|
|
} catch {
|
|
throw error("Remote MCP connection URL is invalid", "mcp_remote_url_invalid");
|
|
}
|
|
if (parsed.protocol !== "http:" && parsed.protocol !== "https:") {
|
|
throw error("Remote MCP connection URL must use http or https", "mcp_remote_url_invalid");
|
|
}
|
|
return parsed;
|
|
}
|
|
|
|
export async function assertPublicRemoteHttpEndpoint(
|
|
endpoint: URL,
|
|
options: RemoteHttpEndpointGuardOptions,
|
|
error: RemoteHttpEndpointErrorFactory,
|
|
): Promise<void> {
|
|
if (options.allowPrivateNetwork) return;
|
|
|
|
const hostname = endpoint.hostname.replace(/^\[|\]$/g, "").toLowerCase();
|
|
if (hostname === "localhost" || hostname.endsWith(".localhost")) {
|
|
throw error("Remote MCP connection URL cannot target private or reserved network addresses", "remote_http_private_endpoint");
|
|
}
|
|
|
|
const literalVersion = isIP(hostname);
|
|
if (literalVersion !== 0) {
|
|
if (isPrivateOrReservedIp(hostname)) {
|
|
throw error("Remote MCP connection URL cannot target private or reserved network addresses", "remote_http_private_endpoint");
|
|
}
|
|
return;
|
|
}
|
|
|
|
let results: LookupResult[];
|
|
try {
|
|
results = await lookupWithTimeout(
|
|
hostname,
|
|
options.lookup ?? defaultLookup,
|
|
options.dnsTimeoutMs ?? DEFAULT_DNS_TIMEOUT_MS,
|
|
);
|
|
} catch {
|
|
throw error("Remote MCP connection hostname could not be resolved", "remote_http_dns_failed");
|
|
}
|
|
if (results.length === 0) {
|
|
throw error("Remote MCP connection hostname did not resolve", "remote_http_dns_failed");
|
|
}
|
|
if (results.some((result) => isPrivateOrReservedIp(result.address))) {
|
|
throw error("Remote MCP connection URL cannot resolve to private or reserved network addresses", "remote_http_private_endpoint");
|
|
}
|
|
}
|
|
|
|
function defaultLookup(hostname: string): Promise<LookupResult[]> {
|
|
return dnsLookup(hostname, { all: true, verbatim: true });
|
|
}
|
|
|
|
async function lookupWithTimeout(hostname: string, lookup: RemoteHttpEndpointLookup, timeoutMs: number) {
|
|
let timeout: NodeJS.Timeout | undefined;
|
|
try {
|
|
return await Promise.race([
|
|
lookup(hostname),
|
|
new Promise<never>((_, reject) => {
|
|
timeout = setTimeout(() => {
|
|
reject(new Error(`DNS lookup timed out for ${hostname}`));
|
|
}, timeoutMs);
|
|
timeout.unref?.();
|
|
}),
|
|
]);
|
|
} finally {
|
|
if (timeout) clearTimeout(timeout);
|
|
}
|
|
}
|
|
|
|
function isPrivateOrReservedIp(address: string): boolean {
|
|
const lower = address.toLowerCase();
|
|
const mappedIpv4 = lower.match(/^::ffff:(\d{1,3}(?:\.\d{1,3}){3})$/);
|
|
if (mappedIpv4?.[1]) return isPrivateOrReservedIpv4(mappedIpv4[1]);
|
|
const mappedIpv4Hex = parseMappedIpv4Hex(lower);
|
|
if (mappedIpv4Hex) return isPrivateOrReservedIpv4(mappedIpv4Hex);
|
|
if (isIP(address) === 4) return isPrivateOrReservedIpv4(address);
|
|
if (isIP(address) === 6) return isPrivateOrReservedIpv6(lower);
|
|
return true;
|
|
}
|
|
|
|
function isPrivateOrReservedIpv4(address: string): boolean {
|
|
const octets = parseIpv4Address(address);
|
|
if (!octets) return true;
|
|
const [a, b, c] = octets;
|
|
if (a === 0) return true;
|
|
if (a === 10) return true;
|
|
if (a === 100 && b >= 64 && b <= 127) return true;
|
|
if (a === 127) return true;
|
|
if (a === 169 && b === 254) return true;
|
|
if (a === 172 && b >= 16 && b <= 31) return true;
|
|
if (a === 192 && b === 0 && c === 0) return true;
|
|
if (a === 192 && b === 168) return true;
|
|
if (a === 192 && b === 0 && c === 2) return true;
|
|
if (a === 192 && b === 88 && c === 99) return true;
|
|
if (a === 198 && (b === 18 || b === 19)) return true;
|
|
if (a === 198 && b === 51 && c === 100) return true;
|
|
if (a === 203 && b === 0 && c === 113) return true;
|
|
if (a >= 224) return true;
|
|
return false;
|
|
}
|
|
|
|
function parseIpv4Address(address: string): [number, number, number, number] | null {
|
|
const parts = address.split(".");
|
|
if (parts.length !== 4) return null;
|
|
const parsed = parts.map((part) => {
|
|
if (!/^\d+$/.test(part)) return NaN;
|
|
return Number(part);
|
|
});
|
|
if (parsed.some((part) => !Number.isInteger(part) || part < 0 || part > 255)) return null;
|
|
return parsed as [number, number, number, number];
|
|
}
|
|
|
|
function parseMappedIpv4Hex(address: string): string | null {
|
|
const match = address.match(/^::ffff:([0-9a-f]{1,4}):([0-9a-f]{1,4})$/);
|
|
if (!match) return null;
|
|
const hi = Number.parseInt(match[1]!, 16);
|
|
const lo = Number.parseInt(match[2]!, 16);
|
|
if (!Number.isInteger(hi) || !Number.isInteger(lo)) return null;
|
|
return `${hi >> 8}.${hi & 0xff}.${lo >> 8}.${lo & 0xff}`;
|
|
}
|
|
|
|
function isPrivateOrReservedIpv6(address: string): boolean {
|
|
if (address === "::" || address === "::1") return true;
|
|
if (address.startsWith("fc") || address.startsWith("fd")) return true;
|
|
if (/^fe[89ab]/.test(address)) return true;
|
|
if (address.startsWith("ff")) return true;
|
|
if (address === "100::" || address.startsWith("100:")) return true;
|
|
if (/^2001:(?:0{0,4}:|:)/.test(address)) return true;
|
|
if (address.startsWith("2001:db8:") || address === "2001:db8::") return true;
|
|
if (address.startsWith("2001:2:") || address === "2001:2::") return true;
|
|
if (/^2001:0?2[0-9a-f]:/.test(address)) return true;
|
|
if (address.startsWith("2002:")) return true;
|
|
if (address.startsWith("64:ff9b:")) return true;
|
|
return false;
|
|
}
|