mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 22:56:58 +01:00
## Thinking Path > - Paperclip manages AI agents and their work. > - AI Connections separate account access from models and harnesses. > - A pool must act as one connection while retaining each task’s account. > - Core must enforce member access and preserve session and recovery rules. > - A plugin supplies rotation policy without receiving credentials. > - This change adds durable routing and native connector setup and management. ## Linked Issues or Issue Description **Subsystem affected** AI Connections, Connectors, plugins, run dispatch, and session compatibility. **Problem or motivation** Operators need to rotate new tasks across saved accounts while each task keeps its account and session. Pool setup must fit the existing connector catalog and account workflow. **Proposed solution** Add an experimental router binding, a capability-gated plugin hook, and transactional task pins. Plugins declare native pooled connectors through `aiConnectionRouter`. Core hosts the existing-account picker, ordering step, and account settings. Related usage contract: #14936. Companion private plugin: https://github.com/paperclipai/paperclip-cloud/pull/643. **Roadmap alignment** This extends Apps and AI Connections. Core supplies generic enforcement and native connector UI; the private plugin owns rotation and quota policy. The prior duplicate search found no matching router implementation. ## What Changed - Add a router binding without changing existing concrete bindings. Keep the instance flag and new pools disabled by default. Require manual operator configuration. Show no routing toggle in Experimental settings on either open-source or Cloud installs, even after routing is enabled. - Persist company-scoped pools, one shared cursor per pool, and pins keyed by company, pool, agent, and task. Commit pins and cursor advances together with revision checks and bounded retries. Persist run-ID affinity before allocation. - Pass only authorized metadata and normalized usage to plugins. Core retains credential handling, member access checks, runtime qualification, and recovery evidence. Probe outside locks with a shared 15-second budget and freshness cache. - Resolve routing before credential preparation and backend selection. Preserve pins through turns, session resets, removed members, and quota waits. Retain admitted recovery after disable or uninstall. - Separate credential session epochs from token generations. Verified refresh preserves the epoch; reconnect and manual replacement change it. Include the credential slot ID in session and usage-cache identity, so reconnecting an indexed legacy account invalidates its old session even when both epochs are zero. - Validate pool member installations before accepting saved-agent bindings and recheck compatibility when the harness changes. Install only authorized members in the new-agent transaction and record their IDs in local activity. Pool membership cannot install a restricted shared connection. - Preserve pool bindings when agents hire teammates through either creation API or native caller runtime inheritance. Block stale manager credential references; retain explicit child authentication precedence and reject incompatible inherited pools. - Add native connector registration through plugin metadata. Reuse the Connectors catalog, setup header, account header, sidebar, dialogs, and usage display. Setup selects and orders saved connections. Advanced settings hold usage rules and member runtime defaults. New-account setup opens in another tab. - Use revision-checked pool archival from the Connectors catalog and account page. Keep task pins, cursors, recovery evidence, and underlying connections. Reject ordinary connection updates or removals that bypass pool revisions. - Add pool selectors, composer models, override notes, quota status, run details, activity records, and local run-log records. Keep session-adoption copy minimal. - Show **Used by** below the pool connections. List current company agents with shared avatars and profile links. Include paused agents; exclude terminated agents and agents using another pool. - Add Core stories for the generic connector workflow and runtime surfaces. Cloud stories reuse these production routes and tokens through a preview-only alias. ## Verification - Final head `73cb953bca30ed83e4505dd820edd9b5edffd28b`: full workspace `pnpm -r typecheck`, `pnpm build`, and `pnpm check:token-gates` pass locally. - All 422 focused connector/settings/shared-contract/migration tests and all 156 database-backed AI connection, hiring, reconnect, and durable-routing cases pass (69 hiring cases rerun after the final auth-precedence fix). The merged shared contract retains connection instructions and pool metadata. The pool migration is generated at sequence 0299 after the latest upstream migrations; this PR makes no lockfile changes. - All four full-app Playwright tests pass on the final head after a cold restart and migration, against the installed private plugin and isolated database, with no route or pool-API mocks. They cover hidden routing controls after manual opt-in, native pool creation, ordering, membership edits, rename, paused defaults, enabling/save/refresh persistence, stale edits, cancellation/removal, preserved underlying accounts, unavailable routers, and Used by avatars and profile links. Exact command: `PAPERCLIP_CONNECTION_POOL_E2E=1 AI_CONNECTIONS_TEST_COMPANY_ID=a37b9625-5ecf-4e29-8081-04df3d6e7d6f AI_CONNECTIONS_TEST_URL=http://127.0.0.1:3108 pnpm exec playwright test --config tests/ai-connections-app/playwright.config.ts connection-pools.spec.ts`. - [Native setup, ordering, and management screenshots](https://github.com/paperclipai/paperclip/pull/14967#issuecomment-6006976278) address the review follow-up. [Earlier selector, quota, and run-detail screenshots](https://github.com/paperclipai/paperclip/pull/14967#issuecomment-5971537316) show the runtime surfaces. Core previews: `pnpm --filter @paperclipai/ui storybook`, then **Connectors / Pool host** or **AI Connections / Connection pools**. Cloud owns its host-backed plugin stories; both repositories’ Operator Setup Required story assertions pass. - Live acceptance used OpenAI/Codex and Anthropic/Claude ACPX, resumed both exact sessions after restart, preserved pinned accounts through explicit reset and controlled quota deferral/recovery, and committed only two allocations across fourteen runs. A later UI-created task test again rotated OpenAI then Anthropic and resumed OpenAI through follow-up/restart/quota recovery. That later Anthropic execution was blocked by its saved OAuth token expiring (provider 401). No live usage probes ran. - The full local `pnpm test:run` was attempted earlier and did not complete because of macOS embedded PostgreSQL bootstrap/shared-memory failures and the 40,000-file Git fixture timeout. The focused database suites above now pass; full-suite verification is provided by the split CI lanes. The preceding CI run had one runtime readiness timeout; it passes locally both alone and inside the larger runtime suite. That larger local suite also encountered an embedded PostgreSQL setup failure and two macOS temporary-path alias assertions; those two assertions pass with canonical TMPDIR=/private/tmp. All final-head CI checks are terminal green, including full general/serialized server suites, Runner checks, browser E2E shards, canary verification, build, and typecheck. Greptile is 5/5 on that exact head with no unresolved threads. ## Risks - The migration adds routing tables and a credential epoch column. Install the private plugin only with the compatible Core contract. - Routing and each pool require opt-in. Production distribution and fleet defaults remain unchanged. - Unknown usage stays eligible. Known pinned exhaustion waits; revoked access requires operator repair. - Legacy adapters require compatible members. Runner model and effort overrides remain limited by qualified backend support. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository editing, code execution, and browser testing. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes:` / `Refs:` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted suites; full-suite limitations are reported above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
385 lines
14 KiB
TypeScript
385 lines
14 KiB
TypeScript
import { z } from "zod";
|
|
import { instanceExperimentalSettingsSchema } from "./validators/instance.js";
|
|
|
|
/**
|
|
* Feature catalog for cloud-managed instances.
|
|
*
|
|
* The instance-settings zod schema is the feature manifest; this module adds
|
|
* only metadata about the flags the schema already declares. Keys are derived
|
|
* from the schema type, so adding, removing, or renaming a boolean flag in
|
|
* `instanceExperimentalSettingsSchema` without updating the metadata map is a
|
|
* compile error (and vice versa).
|
|
*
|
|
* Tiers:
|
|
* - `preference`: tenant-controllable taste setting; the cloud harness does
|
|
* not manage it.
|
|
* - `managed`: the cloud harness may set this per fleet/stack via
|
|
* `PAPERCLIP_MANAGED_CONFIG`.
|
|
* - `floor`: pinned by code on managed instances; no flag value may widen it.
|
|
*/
|
|
export const FEATURE_TIERS = ["preference", "managed", "floor"] as const;
|
|
|
|
export type FeatureTier = (typeof FEATURE_TIERS)[number];
|
|
|
|
type ExperimentalSettings = z.infer<typeof instanceExperimentalSettingsSchema>;
|
|
|
|
/**
|
|
* The boolean flag keys of the experimental settings schema. Non-flag keys
|
|
* (activation timestamps, numeric tuning values) are excluded.
|
|
*/
|
|
export type InstanceFeatureKey = {
|
|
[K in keyof ExperimentalSettings]: ExperimentalSettings[K] extends boolean ? K : never;
|
|
}[keyof ExperimentalSettings];
|
|
|
|
export interface FeatureCatalogEntry {
|
|
title: string;
|
|
description: string;
|
|
tier: FeatureTier;
|
|
/** Desired default on cloud-managed instances. */
|
|
cloudDefault: boolean;
|
|
/** Must match the schema default; enforced by test. */
|
|
selfHostedDefault: boolean;
|
|
}
|
|
|
|
export const INSTANCE_FEATURE_CATALOG: Record<InstanceFeatureKey, FeatureCatalogEntry> = {
|
|
enableEnvironments: {
|
|
title: "Environments",
|
|
description:
|
|
"Show environment management in company settings and allow project and agent environment assignment controls.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableAiConnectionRouters: {
|
|
title: "AI connection routers",
|
|
description: "Allow experimental plugin connections to choose task-pinned accounts and harnesses.",
|
|
tier: "managed", cloudDefault: false, selfHostedDefault: false,
|
|
},
|
|
enableNativeRunner: {
|
|
title: "Paperclip Runner",
|
|
description:
|
|
"Allow explicitly configured local Codex, OpenCode, and qualified ACPX agents to use the experimental Rust Paperclip Runner, including authenticated sandbox ingress when required. Onboarding remains on legacy adapters.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
// On by default for self-hosted instances. Requires a Rust toolchain (or
|
|
// PAPERCLIP_RUNNER_BINARY) for `pnpm dev`, which builds runnerd whenever
|
|
// this is on.
|
|
selfHostedDefault: true,
|
|
},
|
|
enableManagedSandboxOnly: {
|
|
title: "Managed Environment Only",
|
|
description:
|
|
"Hide the local environment and run all agents in the platform-managed environment.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableIsolatedWorkspaces: {
|
|
title: "Isolated Workspaces",
|
|
description:
|
|
"Show execution workspace controls in project configuration and allow isolated workspace behavior for task runs.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableIsolatedWorkspacesByDefault: {
|
|
title: "Isolated Workspaces By Default",
|
|
description:
|
|
"Treat a project that has no execution workspace policy of its own as if it selected isolated workspaces, so its tasks get a per-task worktree instead of sharing the project checkout. Requires Isolated Workspaces. A project that carries its own policy keeps it.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableStreamlinedLeftNavigation: {
|
|
title: "Streamlined Left Navigation",
|
|
description: "Use the streamlined main sidebar navigation layout.",
|
|
tier: "preference",
|
|
cloudDefault: true,
|
|
selfHostedDefault: true,
|
|
},
|
|
enableStreamlinedUi: {
|
|
title: "Streamlined UI",
|
|
description:
|
|
"Use the streamlined application shell, shared task collections, focused task detail layout, contextual navigation, and simplified main sidebar.",
|
|
tier: "preference",
|
|
cloudDefault: true,
|
|
selfHostedDefault: true,
|
|
},
|
|
enableApps: {
|
|
title: "Apps (compatibility)",
|
|
description:
|
|
"Deprecated compatibility key. Apps is always enabled; stored and managed values are ignored.",
|
|
tier: "managed",
|
|
cloudDefault: true,
|
|
selfHostedDefault: true,
|
|
},
|
|
enableChatConnectors: {
|
|
title: "Chat connectors",
|
|
description:
|
|
"Show experimental chat connector setup and Board surfaces. Existing connections keep running when hidden; AgentMail, GitHub tools, and other tool connectors are unaffected.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableMemoryConnectors: {
|
|
title: "Memory connectors",
|
|
description: "Show experimental Mem0, Zep, Supermemory, Cognee, and Honcho setup. Existing connections keep running when hidden.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableMcpAggregators: {
|
|
title: "MCP aggregators (compatibility)",
|
|
description: "Deprecated compatibility key. MCP aggregators are always enabled; stored and managed values are ignored.",
|
|
tier: "managed",
|
|
cloudDefault: true,
|
|
selfHostedDefault: true,
|
|
},
|
|
enablePipelines: {
|
|
title: "Pipelines",
|
|
description: "Enable pipeline definitions and pipeline-driven case production surfaces.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableCases: {
|
|
title: "Cases",
|
|
description:
|
|
"Durable work products that tasks create and iterate on. Adds the Cases tab and the agent case API.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableAgentChat: {
|
|
title: "Agent Chat",
|
|
description:
|
|
"Persistent task-backed conversations that clarify goals and hand work off to tasks. Chat leads the Work group with an agent rail, and each chat's side panel shows the agent's tasks and artifacts as cards.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableCombinedInboxTasks: {
|
|
title: "Combined Inbox + Task List",
|
|
description:
|
|
"Inbox becomes a set of views inside Tasks: one Tasks row in the nav carries the unread badge, and a Views menu reaches every inbox and task view.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableConferenceRoomChat: {
|
|
title: "Conference Room Chat",
|
|
description:
|
|
"Add the Conference Room team chat, the live activity feed, and the redesigned onboarding; restyles task threads as chat bubbles.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableClassicTaskInterface: {
|
|
title: "Classic Task Interface",
|
|
description:
|
|
"Restore the pre-chat task detail page: the page-level header with inline description editor, the plain comment thread, and the fixed Properties sidebar. Chat-only features (streaming activity folding, inline plan/question cards, the three-mode composer) are unavailable in the classic view.",
|
|
tier: "preference",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableIssuePlanDecompositions: {
|
|
title: "Task Plan Decomposition",
|
|
description: "Show accepted-plan decomposition history on task detail pages.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableExperimentalFileViewer: {
|
|
title: "Experimental File Viewer",
|
|
description:
|
|
"Show task detail controls for browsing and previewing workspace files relative to a task.",
|
|
tier: "preference",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableStatusCards: {
|
|
title: "Status Cards",
|
|
description:
|
|
"Enable the experimental shared status-card board, update engine, and gated API.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableExternalObjects: {
|
|
title: "External Objects",
|
|
description:
|
|
"Detect external URLs in issues and show resolved status for pull requests, tickets, and other referenced work objects.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableSmokeLab: {
|
|
title: "Smoke Lab",
|
|
description:
|
|
"Add the Smoke Lab tab and dashboard card for exercising integration paths against deterministic local fixtures. Private deployments only.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableBuiltInAgents: {
|
|
title: "Built-in Agents",
|
|
description:
|
|
"Show Paperclip-managed built-in agent surfaces, including roster badges, the Built-in agents tab, and setup controls.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableBetaSkills: {
|
|
title: "Beta skills",
|
|
description: "Allow agents to pin beta releases of the Paperclip core skill.",
|
|
tier: "preference",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableSummaries: {
|
|
title: "Summaries",
|
|
description:
|
|
"Show Summarizer-generated status slots on project and workspace pages, with on-demand refresh and revision history.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableDecisions: {
|
|
title: "Decisions",
|
|
description:
|
|
"Show the Decisions item in the main sidebar — the attention home that surfaces tasks awaiting input.",
|
|
tier: "preference",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableGoalsSidebarLink: {
|
|
title: "Goals Sidebar Link",
|
|
description: "Restore the Goals item in the main sidebar while the goals surface is being evaluated.",
|
|
tier: "preference",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableSimplifiedEnglishInteractions: {
|
|
title: "Simplified English Interactions",
|
|
description:
|
|
"Instruct agents to write user interactions (confirmations, questions, suggested tasks) in ASD-STE100 Simplified Technical English with brief decision context.",
|
|
tier: "preference",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableServerInfoDebugView: {
|
|
title: "Server Info Debug View",
|
|
description:
|
|
"Show a Server section in the account drawer with the current server restart time and running commit.",
|
|
tier: "preference",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enablePaperclipDeveloperMode: {
|
|
title: "Paperclip Developer Mode",
|
|
description:
|
|
"Show internal Paperclip maintainer tools and observability links, including Honeycomb trace queries on run pages.",
|
|
tier: "preference",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
autoRestartDevServerWhenIdle: {
|
|
title: "Auto-Restart Dev Server When Idle",
|
|
description:
|
|
"In local development, wait for queued and running agent runs to finish, then restart the server automatically when backend changes make the current boot stale.",
|
|
tier: "preference",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableWorkspaceBranchReconcileForward: {
|
|
title: "Workspace Branch Reconcile Forward",
|
|
description:
|
|
"Let execution workspaces reconcile a diverged recorded branch forward instead of failing branch containment.",
|
|
tier: "managed",
|
|
cloudDefault: true,
|
|
selfHostedDefault: true,
|
|
},
|
|
enableWorkspaceDirtyQuarantineRepair: {
|
|
title: "Workspace Dirty Quarantine Repair",
|
|
description:
|
|
"Let workspace runtime recovery quarantine and repair dirty execution workspaces before runs.",
|
|
tier: "managed",
|
|
cloudDefault: true,
|
|
selfHostedDefault: true,
|
|
},
|
|
enableOwnerInstanceAdmin: {
|
|
title: "Owner Instance Admin",
|
|
description:
|
|
"On cloud-managed instances, grant the stack owner instance-admin access to their own dedicated instance. Elevation is computed at the trusted-header auth boundary; no instance admin role rows are created. Inert on self-hosted instances.",
|
|
tier: "managed",
|
|
cloudDefault: true,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableSandboxDuplexBridge: {
|
|
title: "Sandbox Duplex Bridge",
|
|
description:
|
|
"Let a run open the sandbox duplex command-stream bridge when the provider grants the capability. The host reads this per run before it selects the transport. Off keeps the file bridge for every run.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableRunnerPreviewIngress: {
|
|
title: "Runner Preview Ingress (Deprecated)",
|
|
description:
|
|
"Compatibility-only key retained for older managed configs. Runner ingress follows the Paperclip Runner setting.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableWorktreeRunExecution: {
|
|
title: "Worktree Run Execution",
|
|
description:
|
|
"Let the scheduler execute runs inside an isolated git-worktree preview instance for tasks created after activation.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableFirstTaskPlanProposal: {
|
|
title: "First task: propose with a plan document",
|
|
description:
|
|
"When the user's first request is a single task, the chief of staff writes a short plan document and a checkbox card instead of a one-card confirmation. Applies to organizations created after the toggle is flipped.",
|
|
tier: "preference",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
};
|
|
|
|
export const INSTANCE_FEATURE_KEYS = Object.keys(INSTANCE_FEATURE_CATALOG).sort() as InstanceFeatureKey[];
|
|
|
|
/**
|
|
* Shape of the `feature-catalog.json` release artifact the cloud harness
|
|
* imports per app release and validates feature writes against.
|
|
*/
|
|
export const featureCatalogArtifactSchema = z
|
|
.object({
|
|
catalogVersion: z.string().min(1),
|
|
features: z.record(
|
|
z.string().min(1),
|
|
z.object({ tier: z.enum(FEATURE_TIERS) }).strict(),
|
|
),
|
|
})
|
|
.strict();
|
|
|
|
export type FeatureCatalogArtifact = z.infer<typeof featureCatalogArtifactSchema>;
|
|
|
|
export function buildFeatureCatalogArtifact(catalogVersion: string): FeatureCatalogArtifact {
|
|
if (catalogVersion.trim().length === 0) {
|
|
throw new Error("catalogVersion must be a non-empty string");
|
|
}
|
|
const features: FeatureCatalogArtifact["features"] = {};
|
|
for (const key of INSTANCE_FEATURE_KEYS) {
|
|
features[key] = { tier: INSTANCE_FEATURE_CATALOG[key].tier };
|
|
}
|
|
return { catalogVersion, features };
|
|
}
|
|
|
|
/** Deterministic serialization (sorted keys, trailing newline) for the artifact file. */
|
|
export function renderFeatureCatalogArtifact(catalogVersion: string): string {
|
|
return `${JSON.stringify(buildFeatureCatalogArtifact(catalogVersion), null, 2)}\n`;
|
|
}
|