mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-19 16:29:42 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents use outside services through connections, and the control plane holds each connection's credentials > - Companies want agents to "do things on AWS" with exactly the permissions they choose, without Paperclip storing long-lived AWS keys or rebuilding the AWS API > - AWS hosts an MCP server that runs any AWS API call in its own sandbox, and it accepts a one-hour bearer token that can be minted from IAM credentials > - This pull request adds an AWS connection that mints that token for each connection at call time. The role method proves the connection with the company's workload identity issuer (#22083), so no AWS key is stored > - The benefit is that each AWS connection carries its own role and permissions, AWS enforces them, and agents never see any AWS credential ## Linked Issues or Issue Description Refs #22083 (workload identity issuer) and #22191 (spike findings). **Subsystem affected** Connections (tool access), tool gateway, shared app definitions, instance settings. **Problem or motivation** Paperclip has no way to connect AWS. AWS's hosted MCP server needs a bearer token from AWS Sign-in, minted from IAM credentials. No existing credential path mints a header at call time from a role or key. **Proposed solution** Methods can declare `credentialMinter: "aws_mcp"`. The gateway and connection health checks mint the `Authorization` header at call time, and they share one in-memory cache per connection and grant (about 55 minutes). There are two methods: - Role (`AssumeRoleWithWebIdentity` with a Paperclip-signed connection token). - Access key (the key directly, `GetFederationToken`, or `AssumeRole`). Read-only connections add `ReadOnlyAccess` and `AWSMCPSignInOAuthAccessPolicy` as session policies, so AWS refuses changes. The feature is behind the `enableAwsConnector` experimental setting. **Alternatives considered** - Run AWS's `mcp-proxy-for-aws` process on the control plane. Rejected: it adds a Python runtime and processes, and gives nothing the token exchange does not. - Put credentials in the agent sandbox and use the AWS CLI. Deferred: there is no per-call governance, and every sandbox would need the CLI. - Add a new `credentialSource`. Rejected: it needs a migration, and the `vercel_connect` database rules do not fit stored access keys. ## What Changed - `packages/shared`: - `credentialMinter` on connection methods. The validator requires `mcp_remote` and `api_key`, and does not allow `keyPlacement` or another credential source. - The `aws` app definition (two methods, reviewed tools, `mcpSessionRequired`, experimental key). - The `enableAwsConnector` setting in the catalog, types and validators. - `aws` added to the connectable apps. - `server/src/services/connectors/aws/mcp-credentials.ts`: - Settings parsing and checks. The URL must be AWS's endpoint for the saved region, and the role ARN must be valid. - The minter: role or key session, read-only session policies, Sign-in exchange, cache keyed on settings, key references and status, in-flight de-duplication, eviction. - A plain-language error mapping that never shows AWS's own text, because that text can contain account IDs and ARNs. - Gateway (`resolveCredentialHeadersUnrecorded`), credential access (`resolveCredentialHeaders`), catalog discovery and health-failure mapping: - The AWS branch. - After a 401, the gateway evicts the token and mints again, then requires an explicit retry (no replay). Discovery lists again. - Setup: - Minted methods may save a draft without contacting the provider. The role must trust a connection that already exists. - Saving an AWS role draft publishes the company's issuer, so AWS can validate it when the customer registers it. - `workload-identity.ts`: a draft connection can get a token, so setup can test it. Disabled, archived or switched-off connections still get none. - `server/package.json`: `@aws-sdk/client-sts` and `@aws-sdk/client-signin`. The lockfile is left to the refresh workflow. - Artwork: AWS light and dark marks from LobeHub's MIT icon set (already licensed in the repo). The manifest entry passes the brand checks. - `scripts/ingest-app-definitions.mjs`: minted methods are exempt from the `keyPlacement` requirement. The override source is `scripts/app-definition-overrides/aws.json`. Permission reviews are added. - UI: the experimental settings toggle, in alphabetical position. - Docs: `doc/connections/AWS.md`, the connections index, the issuer API doc (draft rule), and the plan's phase table. ## Verification - `server/src/services/connectors/aws/mcp-credentials.test.ts` (7 tests) covers these items: - Both methods and the read-only policies. - Caching, revision, forced refresh, expiry and eviction. - Concurrent de-duplication. - The error mapping, with no AWS text leaking. - Settings validation, including the URL and region check and the rule that an unknown access level means read only. - `server/src/__tests__/aws-mcp-connector.test.ts` (3 tests, real setup, catalog and gateway code with a fake AWS and a session-requiring fake MCP server) covers these items: - The setting gates setup. - The access key is never sent as a header or stored in config. - A cached token is reused. - After a 401, a fresh token is minted and the call is not replayed. - Role draft-then-resume publishes the issuer. - The signed token has the right `iss`, `sub` and `aud`. - An AWS refusal shows a plain health message without the account ID. - Red/green: - Removing the gateway hook fails the access-key test. - Turning off `mcpSessionRequired` fails both integration tests. - Removing draft support for minted methods fails both integration tests. - Restoring each change passes again. - Nearby suites: `tool-gateway-service`, `tool-access-service`, `remote-mcp-connectors`, `instance-settings-service`, `workload-identity`, plus the new files. 775 of 775 pass. - Shared `app-definitions.test.ts` passes (42), including new `credentialMinter` schema tests. UI `InstanceExperimentalSettings` (63) and the brand asset tests pass. `check-app-brand-assets` validates 112 identities. - Typecheck is clean for `packages/shared`, `server` and `ui`. The only UI errors are in Storybook's generated catalog files, which this worktree has not built. - **Real AWS** (Paperclip dev account, integrated code path, issuer behind a temporary public tunnel): - Access key, read only: 8 tools listed. `run_script` `ListBuckets` succeeded in 8.9 s on a cold sandbox. AWS refused `CreateBucket` ("no session policy allows"). - Role: the draft was saved without contacting AWS, and the issuer was published. After an IAM provider and a role trusting only that connection were created, resuming setup listed 8 tools. `ListBuckets` succeeded in 12.8 s. The role's own permissions refused `CreateBucket`. - This run found two bugs the fakes did not: AWS needs MCP sessions for calls, and the role method needs draft-first setup with the issuer published. Both are fixed and both now have tests. - All IAM users, keys, roles, providers and buckets were deleted, and a listing confirmed it. - Not run: Paperclip Cloud staging (the setting is off by default), and `AssumeRole` from an access key with a role ARN against real AWS (unit tests only). ## Risks - The feature is off by default, behind `enableAwsConnector`. No migration. - New dependencies `@aws-sdk/client-sts` and `@aws-sdk/client-signin` are in the existing `@aws-sdk` family (the server already uses `client-s3`). - Minted tokens are held only in process memory. A restart mints again. A cached token can outlive a deactivated key by up to an hour, unless AWS returns 401, which evicts it. - `aws___run_script` is classified as a write, so the normal approval and ask-first policy applies. Read-only connections rely on AWS session policies. - The guided setup screen and the CloudFormation template are not in this PR. Setup uses the generic form and the documented draft and resume steps. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, AWS SDK for JavaScript against the Paperclip dev AWS account, a Cloudflare quick tunnel, an exploration subagent). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing>
176 lines
8.4 KiB
TypeScript
176 lines
8.4 KiB
TypeScript
import { z } from "zod";
|
|
import { DEFAULT_FEEDBACK_DATA_SHARING_PREFERENCE } from "../types/feedback.js";
|
|
import {
|
|
DAILY_RETENTION_PRESETS,
|
|
WEEKLY_RETENTION_PRESETS,
|
|
MONTHLY_RETENTION_PRESETS,
|
|
DEFAULT_BACKUP_RETENTION,
|
|
} from "../types/instance.js";
|
|
import { feedbackDataSharingPreferenceSchema } from "./feedback.js";
|
|
import { shapeWithoutDefaults } from "./partial.js";
|
|
|
|
function presetSchema<T extends readonly number[]>(presets: T, label: string) {
|
|
return z.number().refine(
|
|
(v): v is T[number] => (presets as readonly number[]).includes(v),
|
|
{ message: `${label} must be one of: ${presets.join(", ")}` },
|
|
);
|
|
}
|
|
|
|
export const backupRetentionPolicySchema = z.object({
|
|
dailyDays: presetSchema(DAILY_RETENTION_PRESETS, "dailyDays").default(DEFAULT_BACKUP_RETENTION.dailyDays),
|
|
weeklyWeeks: presetSchema(WEEKLY_RETENTION_PRESETS, "weeklyWeeks").default(DEFAULT_BACKUP_RETENTION.weeklyWeeks),
|
|
monthlyMonths: presetSchema(MONTHLY_RETENTION_PRESETS, "monthlyMonths").default(DEFAULT_BACKUP_RETENTION.monthlyMonths),
|
|
});
|
|
|
|
export const instanceGeneralSettingsSchema = z.object({
|
|
censorUsernameInLogs: z.boolean().default(false),
|
|
feedbackDataSharingPreference: feedbackDataSharingPreferenceSchema.default(
|
|
DEFAULT_FEEDBACK_DATA_SHARING_PREFERENCE,
|
|
),
|
|
backupRetention: backupRetentionPolicySchema.default(DEFAULT_BACKUP_RETENTION),
|
|
// Execution policy. Absent/"any" = unrestricted; "kubernetes" forces the
|
|
// Kubernetes sandbox provider and denies local/ssh execution (cloud_tenant).
|
|
executionMode: z.enum(["kubernetes", "any"]).optional(),
|
|
}).strict();
|
|
|
|
export const patchInstanceGeneralSettingsSchema = z
|
|
.object(shapeWithoutDefaults(instanceGeneralSettingsSchema.shape))
|
|
.partial()
|
|
.strict();
|
|
|
|
export const instanceExperimentalSettingsSchema = z.object({
|
|
enableEnvironments: z.boolean().default(false),
|
|
enableNativeRunner: z.boolean().default(true),
|
|
enableAiConnectionRouters: z.boolean().default(false),
|
|
enableManagedSandboxOnly: z.boolean().default(false),
|
|
enableIsolatedWorkspaces: z.boolean().default(false),
|
|
remoteWorkspaceStorageEnabled: z.boolean().default(true),
|
|
enableIsolatedWorkspacesByDefault: z.boolean().default(false),
|
|
enableStreamlinedLeftNavigation: z.boolean().default(true),
|
|
enableStreamlinedUi: z.boolean().default(true),
|
|
// Deprecated compatibility key. Apps is a standard product surface and is
|
|
// always enabled; this remains accepted so older stored rows and managed
|
|
// configs continue to load during upgrades.
|
|
enableApps: z.boolean().default(true),
|
|
// Deprecated compatibility key. Assistant connections (MCP) graduated from
|
|
// Experimental and are always enabled; older stored rows, managed configs
|
|
// and clients may still send it.
|
|
enablePublicMcp: z.boolean().default(true),
|
|
enableOpenAiDot: z.boolean().default(false),
|
|
enableGrokBot: z.boolean().default(false),
|
|
enableMuse: z.boolean().default(false),
|
|
enableChatConnectors: z.boolean().default(false),
|
|
enableGitHubReviewBots: z.boolean().default(false),
|
|
// Compatibility only: old stored and managed values must still parse.
|
|
enableMcpAggregators: z.boolean().default(true),
|
|
enableMemoryConnectors: z.boolean().default(false),
|
|
enablePaperclipNotebook: z.boolean().default(false),
|
|
enableGoogleWorkspaceConnector: z.boolean().default(false),
|
|
enableAwsConnector: z.boolean().default(false),
|
|
enableCases: z.boolean().default(false),
|
|
enableAgentChat: z.boolean().default(true),
|
|
enableCombinedInboxTasks: z.boolean().default(false),
|
|
enableArtifactsV1: z.boolean().default(true),
|
|
enableConferenceRoomChat: z.boolean().default(false),
|
|
enableClassicTaskInterface: z.boolean().default(false),
|
|
enableLegacyComposerSubmit: z.boolean().default(false),
|
|
enableIssuePlanDecompositions: z.boolean().default(false),
|
|
enableExperimentalFileViewer: z.boolean().default(false),
|
|
enableExternalObjects: z.boolean().default(false),
|
|
enableSmokeLab: z.boolean().default(false),
|
|
enableBuiltInAgents: z.boolean().default(false),
|
|
enableBetaSkills: z.boolean().default(false),
|
|
enableSummaries: z.boolean().default(false),
|
|
enableStatusCards: z.boolean().default(false),
|
|
enableResponsibilities: z.boolean().default(false),
|
|
enableDecisions: z.boolean().default(false),
|
|
enableFastResponses: z.boolean().default(false),
|
|
enableGoalsSidebarLink: z.boolean().default(false),
|
|
enableServerInfoDebugView: z.boolean().default(false),
|
|
enablePaperclipDeveloperMode: z.boolean().default(false),
|
|
enableSimplifiedEnglishInteractions: z.boolean().default(false),
|
|
enableFirstTaskPlanProposal: z.boolean().default(false),
|
|
autoRestartDevServerWhenIdle: z.boolean().default(false),
|
|
enableWorkspaceBranchReconcileForward: z.boolean().default(true),
|
|
enableWorkspaceDirtyQuarantineRepair: z.boolean().default(true),
|
|
enableOwnerInstanceAdmin: z.boolean().default(false),
|
|
// Kill switch for the sandbox duplex command-stream bridge. Default off. When
|
|
// off the host keeps the file bridge for every run with no manifest change and
|
|
// no redeploy. The host reads this per run before it selects the transport.
|
|
enableSandboxDuplexBridge: z.boolean().default(false),
|
|
// MCP Tasks (io.modelcontextprotocol/tasks): durable long provider calls.
|
|
// Off: Paperclip never declares the extension, so servers never return tasks.
|
|
enableMcpTasks: z.boolean().default(false),
|
|
// Deprecated compatibility key. Runner ingress follows enableNativeRunner;
|
|
// this remains accepted so older stored rows and managed configs keep loading.
|
|
enableRunnerPreviewIngress: z.boolean().default(false),
|
|
enableWorktreeRunExecution: z.boolean().default(false),
|
|
worktreeRunExecutionActivatedAt: z.string().datetime().nullable().default(null),
|
|
worktreeRunExecutionActivationInstanceId: z.string().min(1).nullable().default(null),
|
|
}).strict();
|
|
|
|
export const patchInstanceExperimentalSettingsSchema = z
|
|
.object(
|
|
shapeWithoutDefaults(
|
|
instanceExperimentalSettingsSchema
|
|
.omit({
|
|
worktreeRunExecutionActivatedAt: true,
|
|
worktreeRunExecutionActivationInstanceId: true,
|
|
})
|
|
.shape,
|
|
),
|
|
)
|
|
.partial()
|
|
.strip();
|
|
|
|
export const managedSettingMetadataSchema = z.object({
|
|
managed: z.literal(true),
|
|
managedBy: z.literal("paperclip-cloud"),
|
|
}).strict();
|
|
|
|
// Response shape of the experimental settings endpoints: on cloud-managed
|
|
// instances every overlaid key is listed in `managedKeys`; self-hosted
|
|
// responses omit the field entirely.
|
|
export const instanceExperimentalSettingsWithManagedSchema = instanceExperimentalSettingsSchema.extend({
|
|
managedKeys: z.record(z.string(), managedSettingMetadataSchema).optional(),
|
|
}).strict();
|
|
|
|
export const patchInstanceSettingsSchema = z.object({
|
|
defaultEnvironmentId: z.string().guid().nullable().optional(),
|
|
}).strict();
|
|
|
|
// The longest time a task drain can run before it expires on its own. A
|
|
// caller can send a shorter `ttlMs`, but not a longer one — the request must
|
|
// fail instead of the server silently clamping the value.
|
|
export const MAX_TASK_DRAIN_TTL_MS = 24 * 60 * 60 * 1000;
|
|
|
|
export const startTaskDrainRequestSchema = z.object({
|
|
purpose: z.literal("idle").optional(),
|
|
ttlMs: z.number().int().positive().max(MAX_TASK_DRAIN_TTL_MS).nullable().optional(),
|
|
}).strict().refine(value => value.purpose !== "idle" ||
|
|
(typeof value.ttlMs === "number" && value.ttlMs >= 5_000 && value.ttlMs <= 300_000),
|
|
{ message: "Idle holds require ttlMs between 5000 and 300000", path: ["ttlMs"] });
|
|
|
|
export type InstanceGeneralSettings = z.infer<typeof instanceGeneralSettingsSchema>;
|
|
// The patch schema removes each default so an absent key stays absent. Declare
|
|
// the type from the full settings type, so every field keeps its precise type.
|
|
export type PatchInstanceGeneralSettings = Partial<InstanceGeneralSettings>;
|
|
export type InstanceExperimentalSettings = z.infer<typeof instanceExperimentalSettingsSchema>;
|
|
export type PatchInstanceExperimentalSettings = Partial<
|
|
Omit<
|
|
InstanceExperimentalSettings,
|
|
"worktreeRunExecutionActivatedAt" | "worktreeRunExecutionActivationInstanceId"
|
|
>
|
|
>;
|
|
export type PatchInstanceSettings = z.infer<typeof patchInstanceSettingsSchema>;
|
|
export type StartTaskDrainRequest = z.infer<typeof startTaskDrainRequestSchema>;
|
|
|
|
export const instanceSettingsSchema = z.object({
|
|
id: z.string().guid(),
|
|
defaultEnvironmentId: z.string().guid().nullable(),
|
|
general: instanceGeneralSettingsSchema,
|
|
experimental: instanceExperimentalSettingsWithManagedSchema,
|
|
createdAt: z.union([z.date(), z.string().datetime()]),
|
|
updatedAt: z.union([z.date(), z.string().datetime()]),
|
|
}).strict();
|