Files
PaperClipAI/server/src/app.ts
T
DottaandClaude Opus 5.5 2d1c2db64a feat(mcp): host the MCP Skills extension for connected apps (#21219)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents use connected apps through MCP. Paperclip already hosts MCP
Apps views and speaks stateless MCP 2026-07-28.
> - The MCP Skills extension (SEP-2640, Final) lets an app publish
skills (instructions and files) next to its tools. Hugging Face already
serves 25.
> - Paperclip did not read them, so agents missed each app's own
workflow guidance.
> - Skills are a prompt-injection surface. The spec makes manifest
binding, verification, origin tagging and approval revocation host
requirements.
> - This pull request makes Paperclip a Skills host: discover, review,
verify, and deliver app skills to the agents that use the app.
> - The benefit is that connected apps can teach agents their workflows,
under board control.

## Linked Issues or Issue Description

**Problem**
Connected MCP servers can publish skills with the
`io.modelcontextprotocol/skills` extension. Paperclip ignored them. A
board could not see, review or approve them, and agents never received
them.

**Proposed solution**
Implement the host side of SEP-2640. List skills during catalog refresh,
record them per connection and company, bind board approval to the full
file manifest, and load approved skills into agent runs with verified,
lazily fetched content.

**Why it matters**
Apps ship workflow guidance with their tools. Without the extension,
each team must copy that guidance into company skills by hand, and it
goes stale.

Refs #21119, Refs #21140 (the MCP Apps and stateless MCP client work
this builds on).

## What Changed

- **Client capability.** Every stateless request declares
`io.modelcontextprotocol/skills`. Legacy handshake sessions do not. The
capabilities that `server/discover` advertises are remembered beside the
protocol era.
- **Discovery.** Catalog refresh calls `skills/list` (paged) only when
the server declared the extension. A listing inside its `ttlMs` is
reused. A skills failure never fails the tool refresh. Nothing runs
while a run starts.
- **Schema.** New `tool_mcp_skills` (company, connection, skill URI;
frontmatter, sorted manifest, manifest digest, review state) and
`tool_mcp_skill_files` (verified bytes keyed by connection, URI and
digest). Both cascade with the connection.
- **Governance.** New skills wait for review. Approval binds to the
order-free digest of every `{uri, digest, size}`; a request for any
other digest gets 409. A changed manifest or frontmatter revokes
approval and logs `tool_mcp_skill.approval_revoked`. Dynamic, oversized
and invalid skills are declined with a reason. Approve and decline use
the company skill policy (`skills.install`, `external_package`,
`mcp-connection:<id>`).
- **Run time.** Approved skills of connections the agent can use load in
parallel with other skill preparation, inside a 3 s budget. `SKILL.md`
is read with the agent's grant and checked for size, SHA-256 and
frontmatter. A failure calls `skills/get` and takes the skill out of
use. Each skill becomes a per-run bundle on the connector skill path,
named in its app's namespace, tagged with its origin. Paperclip writes
the frontmatter: `allowed-tools` only if the board approved it.
- **Supporting files.** `GET
/api/agents/me/mcp-skills/:skillId/files?path=` serves manifest-listed
files to an agent in an active run, through the skill's own connection.
The sandbox bridge allows the route.
- **Board API and UI.** `GET/POST /api/companies/:companyId/mcp-skills…`
(list, read instructions, approve, decline). The Skills page shows
"Skills from connected apps" with app, state, size, requested tools and
plain-language notes. It renders nothing when no app publishes skills.
- **Docs.** `doc/connections/MCP-SKILLS.md`, linked from
`doc/connections/README.md`.
- **Shared frontmatter parser.** It now decodes YAML single-quoted
scalars. Before, 5 of Hugging Face's 25 skills kept their quotes and
would fail the frontmatter check; now all 25 match. The generated runner
copy is regenerated.

File size notes: `server/src/services/tool-gateway.ts` grows by about
200 lines. The run-time reads must use its grant resolution and
`callAssignedConnectionProtocol`, like MCP Apps view reads. Splitting
the gateway is a separate refactor. `CompanySkills.tsx` and
`CompanySkills.production.tsx` gain 2–4 lines each; the section lives in
its own component.

## Verification

- `server/src/__tests__/mcp-skills.test.ts` (15 tests) uses a fake
stateless skills server and embedded Postgres. It covers: discovery only
after the server declares the extension; the capability in `_meta`;
`ttlMs` reuse; lazy fetch (no reads at listing or approval; `SKILL.md`
at load; supporting files on read); a stale-digest approval refused; a
reorder that keeps approval; a changed or added file that revokes it;
dynamic skills declined; a same-size tampered `SKILL.md` rejected,
refreshed with `skills/get`, and taken out of use; mismatched
frontmatter rejected; reads outside the manifest refused; the same URI
and name from two servers kept apart; company scoping; a slow app
skipped within the budget and cached later; HTTP authorization (board
only, agent needs an active run, run header must match) and activity
logging.
- Red/green: with the digest check disabled, the tamper test fails; with
the manifest comparison disabled, the revocation test fails. Both pass
with the code.
- `server/src/__tests__/mcp-protocol-versions.test.ts`,
`mcp-apps.test.ts`, `connection-instructions.test.ts`,
`tool-gateway-discovery-http.test.ts`: 47 pass (two capability
assertions updated for the new extension).
- `packages/shared/src/frontmatter.test.ts` adds a single-quoted scalar
case; the runner contract test passes (23 tests).
- `ui/src/components/skills/ConnectedAppSkills.test.tsx` (3) and
`ui/src/pages/CompanySkills.test.tsx` (44) pass.
`packages/adapter-utils/src/sandbox-callback-bridge.test.ts` (89)
passes.
- `pnpm --filter @paperclipai/server typecheck` passes. UI `tsc` passes
except the known missing generated Storybook catalog files.
- `pnpm verify` returned exit 2 (doc change is a global CI input);
required CI supplies broad coverage.
- Live check against `https://huggingface.co/mcp` without credentials,
through Paperclip's own services and the real egress guard: 25 skills
discovered as pending; three approved; cold load 135 ms, cached load 13
ms; a supporting file read through the manifest; the bundle tagged with
its origin.
- Storybook: `Agents and reusable capabilities/Skills and Skill
Studio/Connected app skills` (needs review, all states, tool request,
name collision, reading instructions).

## Risks

- **Migration:** two new tables only. No changes to existing tables.
Safe with the previous release running.
- **Run start:** companies without an approved app skill pay one indexed
read. With approved skills the wait is capped at 3 s; late or failing
skills are left out with a `skills.mcp_unavailable` run warning and
never fail the run.
- **Agent harness enforcement:** Paperclip cannot gate each
code-execution tool call an agent makes while following an app's skill.
Approval is explicit and per skill, the approval note says agents may
run commands the skill describes, and `allowed-tools` is never granted
without a separate tick.
- **Shared-home adapters** receive app skills like company skills. A
revoked skill leaves on the next skill sync.
- Not in this PR: per-connection opt-in for dynamic skills; serving
Paperclip's own skills from its public MCP endpoint;
`resources/directory/read`; app skills on an agent's skills tab.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, local test runs, live HTTP checks).
Extended reasoning enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 19:18:06 -06:00

1466 lines
68 KiB
TypeScript

import type { IdleDeadlineOwners } from "./services/idle-deadline-types.js";
import { startMcpClientMirrorSync } from "./services/public-mcp/client-mirror-sync.js";
import { registerIdleMcpOwners } from "./services/public-mcp/idle-sleep-proof.js";
import { startMcpRevocationIndexSync } from "./services/public-mcp/revocation-index-sync.js";
import { startMcpIngressHandoffExpiry } from "./services/public-mcp/ingress-handoff-expiry.js";
import { runnerAttachmentDownloadRoutes } from "./services/native-runtime/runner-attachment-download.js";
import { nativeRunMcpRoutes } from "./routes/native-run-mcp.js";
import { connectionEventWakeRoutes } from "./routes/connection-event-wake.js";
import { registerGitHubReviewProjectionWork } from "./services/chat-channels/github/review-projection-work.js";
import { finishServiceShutdown } from "./shutdown.js";
import { promptInjectionPreventionRoutes } from "./routes/prompt-injection-prevention.js";
import { voiceSessionRoutes, voiceWebhookRoutes } from "./routes/voice-sessions.js";
import { eq } from "drizzle-orm";
import { chatEndpoints } from "@paperclipai/db";
import { fastResponseRoutes } from "./routes/fast-responses.js";
import { fastResponseService } from "./services/fast-responses.js";
import { createDeliveryWorkCoordinator } from "./services/delivery-work-coordinator.js";
import { DELIVERY_QUEUES } from "./services/delivery-work-notifications.js";
import { createLifecycleDriver } from "./services/agent-lifecycle-driver.js";
import { startAgentLifecycle } from "./services/agent-lifecycle.js";
import { idleAdmissionMiddleware, trackIdleRequestHandlers } from "./middleware/idle-admission.js";
import { databaseOperationMiddleware } from "./middleware/database-operation.js";
import { staticUiFiles } from "./middleware/static-ui.js";
import { publishRetainedUiAssetsAtStartup, retainedUiAssets } from "./retained-ui-assets.js";
import { createStorageProviderFromConfig } from "./storage/provider-registry.js";
import { loadStorageConfig } from "./storage/config.js";
import { isIdleTaskDrainActive, trackIdleWork } from "./services/task-admission.js";
import { customerSuccessRoutes } from "./routes/customer-success.js";
import { cloudWarmStandbyMiddleware } from "./middleware/cloud-warm-standby.js";
import type { CloudWarmStandby } from "./services/cloud-warm-standby.js";
import { browserUseRoutes } from "./routes/browser-use.js";
import { mcpAppsRoutes } from "./routes/mcp-apps.js";
import { mcpSkillsRoutes } from "./routes/mcp-skills.js";
import { registerChatActionWork } from "./services/chat-action-work.js";
import { registerChatDeliveryWork } from "./services/chat-delivery-work.js";
import { registerBrowserUseCleanup } from "./services/browser-use-work.js";
import { browserUseService } from "./services/browser-use.js";
import { slackToolRoutes } from "./routes/slack-tools.js";
import { createPublicMcpOAuth, publicMcpConfig } from "./services/public-mcp/oauth.js";
import { createDotPrincipalHttp } from "./modules/external-agents/index.js";
import { createDotRunnerMcpTools, dotRunnerBroker } from "./services/dot-runner-broker.js";
import { agentProfileAvatarRoutes } from "./routes/agent-profile-avatar.js";
import { museTransportRoutes, museBoardRoutes } from "./routes/muse.js";
import { museQualificationRoutes } from "./routes/muse-qualification.js";
import { startMuseMaintenance } from "./services/muse-maintenance.js";
import { grokBotRoutes } from "./routes/grok-bot.js";
import { createGrokRoutineDispatcher } from "./services/grok-bot/dispatcher.js";
import { dotRunnerRoutes } from "./routes/dot-runner.js";
import { createPublicMcpTransfers } from "./services/public-mcp/file-transfers.js";
import { createMcpApiDispatch, createPublicMcpExecutor } from "./services/public-mcp/capabilities.js";
import { createPublicMcpEvents, type PublicMcpEvents } from "./services/public-mcp/events.js";
import { publicMcpIngressRoutes, publicMcpManagementRoutes } from "./routes/public-mcp.js";
import { agentAvatarRoutes } from "./routes/agent-avatars.js";
import { decisionModelRoutes } from "./routes/decision-models.js";
import { agentObservabilityRoutes } from "./routes/agent-observability.js";
import { aiConnectionRoutes } from "./routes/ai-connections.js";
import { taskWorkspaceRoutes } from "./routes/task-workspaces.js";
import { projectToolRoutes } from "./routes/project-tools.js";
import { emailChannelService } from "./services/email-channels.js";
import { emailRoutes, emailWebhookRoutes } from "./routes/email.js";
import { toolActionDeliveryService } from "./services/tool-action-delivery.js";
import { registerAssignedMcpGateway } from "./services/native-runtime/assigned-mcp-tools.js";
import express, { Router, type Request as ExpressRequest } from "express";
import {
createServer as createHttpServer,
type Server as HttpServer,
} from "node:http";
import path from "node:path";
import fs from "node:fs";
import { fileURLToPath } from "node:url";
import type { Db } from "@paperclipai/db";
import {
derivePaperclipViteHmrPort,
type DeploymentExposure,
type DeploymentMode,
} from "@paperclipai/shared";
import type { InspectDatabaseBackupHealthOptions } from "./services/database-backup-health.js";
import type { StorageService } from "./storage/types.js";
import { httpLogger, errorHandler } from "./middleware/index.js";
import { skillAccessRequestContext } from "./services/skill-access.js";
import { actorMiddleware } from "./middleware/auth.js";
import { boardMutationGuard } from "./middleware/board-mutation-guard.js";
import {
privateHostnameGuard,
resolvePrivateHostnameAllowSet,
} from "./middleware/private-hostname-guard.js";
import {
applyTrustProxy,
parseTrustProxyEnv,
} from "./middleware/trust-proxy.js";
import {
resolveDefaultImportTransferSpoolRoot,
} from "./services/company-import-transfers.js";
import { registerImportSpoolCleanupWork } from "./services/import-spool-cleanup-work.js";
import { companyTransferRunService } from "./services/company-transfer-runs.js";
import { healthRoutes } from "./routes/health.js";
import { readinessRoutes } from "./routes/ready.js";
import { livenessRoutes } from "./routes/liveness.js";
import { cloudRuntimeIdentityMiddleware } from "./middleware/cloud-runtime-identity.js";
import { cloudControlMiddleware } from "./middleware/cloud-control.js";
import { cloudRoutes } from "./routes/cloud.js";
import { companyRoutes } from "./routes/companies.js";
import { companySkillRoutes } from "./routes/company-skills.js";
import { companySkillPolicyRoutes } from "./routes/company-skill-policy.js";
import { inboxAgentPolicyRoutes } from "./routes/inbox-agent-policy.js";
import { builtInAgentRoutes } from "./routes/built-in-agents.js";
import { folderRoutes } from "./routes/folders.js";
import { summarySlotRoutes } from "./routes/summary-slots.js";
import { statusCardRoutes } from "./routes/status-cards.js";
import { teamsCatalogRoutes } from "./routes/teams-catalog.js";
import { agentRoutes } from "./routes/agents.js";
import { agentInteractionPermissionRoutes } from "./routes/agent-interaction-permissions.js";
import { agentPermissionGrantRoutes } from "./routes/agent-permission-grants.js";
import { resolveAgentRunnerTargetForCompany } from "./services/agent-runner-selection.js";
import type { SetupTokenSessionService } from "./services/setup-token-session.js";
import {
buildSetupTokenLoginTransport,
createProductionSetupTokenSandboxProvider,
createProductionSetupTokenCleanupStore,
createSetupTokenSecretWriter,
createWorkerBoundLoginPtyOpener,
} from "./services/setup-token-transport-binding.js";
import { environmentService } from "./services/environments.js";
import { environmentRuntimeService } from "./services/environment-runtime.js";
import { projectRoutes } from "./routes/projects.js";
import { issueRoutes } from "./routes/issues.js";
import { linkPreviewRoutes } from "./routes/link-previews.js";
import { issueTreeControlRoutes } from "./routes/issue-tree-control.js";
import { caseRoutes } from "./routes/cases.js";
import { caseHandoffRoutes } from "./routes/case-handoffs.js";
import { workflowRoutes } from "./routes/workflows.js";
import { fileResourceRoutes } from "./routes/file-resources.js";
import { createRemoteWorkspaceFiles } from "./services/remote-workspace-files.js";
import { createSavedWorkspaceLoader } from "./services/remote-workspaces.js";
import { createSavedWorkspaceArchiveOpener } from "./services/saved-workspace-archive.js";
import { routineRoutes } from "./routes/routines.js";
import { environmentRoutes } from "./routes/environments.js";
import { executionWorkspaceRoutes } from "./routes/execution-workspaces.js";
import { goalRoutes } from "./routes/goals.js";
import { onboardingSeedRoutes } from "./routes/onboarding-seed.js";
import { boardChatRoutes } from "./routes/board-chat.js";
import { approvalRoutes } from "./routes/approvals.js";
import { secretRoutes } from "./routes/secrets.js";
import { toolAccessRoutes } from "./routes/tool-access.js";
import {
chatChannelRoutes,
chatWebhookRoutes,
} from "./routes/chat-channels.js";
import { smokeLabRoutes } from "./routes/smoke-lab.js";
import { costRoutes } from "./routes/costs.js";
import { agentCommentaryRoutes } from "./routes/agent-commentary.js";
import { agentChatMessageRoutes } from "./routes/agent-chat-messages.js";
import { activityRoutes } from "./routes/activity.js";
import { dashboardRoutes } from "./routes/dashboard.js";
import { attentionRoutes } from "./routes/attention.js";
import { decisionTrainingRoutes } from "./routes/decision-training.js";
import { decisionRoutes } from "./routes/decisions.js";
import { decisionQueueRoutes } from "./routes/decision-queues.js";
import type { DecisionServiceOptions } from "./services/decisions.js";
import { userProfileRoutes } from "./routes/user-profiles.js";
import { sidebarBadgeRoutes } from "./routes/sidebar-badges.js";
import { sidebarPreferenceRoutes } from "./routes/sidebar-preferences.js";
import { announcementRoutes } from "./routes/announcements.js";
import { serverVersion } from "./version.js";
import { workspacePreferenceRoutes } from "./routes/workspace-preferences.js";
import { primaryAgentRoutes } from "./routes/primary-agent.js";
import { resourceMembershipRoutes } from "./routes/resource-memberships.js";
import { inboxDismissalRoutes } from "./routes/inbox-dismissals.js";
import { instanceSettingsRoutes } from "./routes/instance-settings.js";
import { ephemeralEnrollmentRoutes } from "./routes/ephemeral-enrollment.js";
import { instanceSettingsService } from "./services/instance-settings.js";
import { openApiRoutes } from "./routes/openapi.js";
import {
instanceDatabaseBackupRoutes,
type InstanceDatabaseBackupService,
} from "./routes/instance-database-backups.js";
import { llmRoutes } from "./routes/llms.js";
import { authRoutes } from "./routes/auth.js";
import { assetRoutes } from "./routes/assets.js";
import { accessRoutes } from "./routes/access.js";
import { pluginRoutes } from "./routes/plugins.js";
import {
mcpGatewayProtocolRoutes,
toolGatewayRoutes,
} from "./routes/tool-gateway.js";
import {
connectionIntentBoardRoutes,
runtimeConnectionIntentRoutes,
} from "./routes/connection-intents.js";
import { adapterRoutes } from "./routes/adapters.js";
import { managedAgentProfileRoutes } from "./routes/managed-agent-profiles.js";
import { remoteAgentProfileRoutes } from "./routes/remote-agent-profiles.js";
import { pluginUiStaticRoutes } from "./routes/plugin-ui-static.js";
import { readBrandedStaticIndexHtml } from "./static-index-html.js";
import { staticUiCacheControl } from "./static-ui-cache.js";
import { staticUiCompression } from "./middleware/static-ui-compression.js";
import { applyUiBranding } from "./ui-branding.js";
import { logger } from "./middleware/logger.js";
import {
DEFAULT_LOCAL_PLUGIN_DIR,
pluginLoader,
type PluginLoader,
} from "./services/plugin-loader.js";
import {
SELF_HOSTED_AUTO_INSTALL_KEYS,
BUNDLED_PLUGIN_CATALOG,
ensureBundledPlugins,
resolveBundledCatalogRoot,
resolveBundledPluginInstalls,
resolveManagedBundledPluginKeys,
} from "./services/bundled-plugins.js";
import { createBundledPluginPackageResolver } from "./services/bundled-plugin-package.js";
import { readDistributionPluginCatalog, distributionPluginActivationGuard } from "./services/distribution-plugin-catalog.js";
import {
createPluginWorkerManager,
type PluginWorkerManager,
} from "./services/plugin-worker-manager.js";
import { createPluginJobScheduler } from "./services/plugin-job-scheduler.js";
import { pluginJobStore } from "./services/plugin-job-store.js";
import { createPluginToolDispatcher } from "./services/plugin-tool-dispatcher.js";
import { createToolGatewayService } from "./services/tool-gateway.js";
import { toolAccessService } from "./services/tool-access.js";
import { chatChannelService } from "./services/chat-channels.js";
import type { BudgetResumeDelivery } from "./services/budget-resume-continuation.js";
import { deliverNativeQuestionResponse } from "./services/native-runtime/native-question-bridge.js";
import { nativeChatProgressWork } from "./services/chat-native-progress-work.js";
import { chatInboundWakeNoticeWork } from "./services/chat-inbound-wake-notice-work.js";
import { chatFailedRetryNoticeWork } from "./services/chat-failed-retry-notice-work.js";
import { chatRunMilestoneWork } from "./services/chat-run-milestone-work.js";
import { heartbeatService } from "./services/heartbeat.js";
import { registerInterruptedTurnChatRetry } from "./services/heartbeat/restart-interrupted-turn.js";
import { pluginLifecycleManager } from "./services/plugin-lifecycle.js";
import { registerPluginLifecycleDelivery } from "./services/plugin-lifecycle-delivery.js";
import { createPluginJobCoordinator } from "./services/plugin-job-coordinator.js";
import {
buildHostServices,
flushPluginLogBuffer,
} from "./services/plugin-host-services.js";
import { createPluginEventBus } from "./services/plugin-event-bus.js";
import { setPluginEventBus } from "./services/activity-log.js";
import { createPluginDevWatcher } from "./services/plugin-dev-watcher.js";
import { createPluginHostServiceCleanup } from "./services/plugin-host-service-cleanup.js";
import { pluginRegistryService } from "./services/plugin-registry.js";
import { createHostClientHandlers } from "@paperclipai/plugin-sdk";
import type { BetterAuthSessionResult } from "./auth/better-auth.js";
import { createCachedViteHtmlRenderer } from "./vite-html-renderer.js";
import {
DEFAULT_JSON_BODY_LIMIT,
PORTABLE_JSON_BODY_LIMIT,
} from "./http/body-limits.js";
import { COMPANY_IMPORT_API_PATH } from "./routes/company-import-paths.js";
import { apiCompression } from "./middleware/api-compression.js";
import { apiServerTiming } from "./middleware/api-server-timing.js";
import { chatWebhookBodyParser } from "./middleware/chat-webhook-body.js";
import { createChatWebhookDiagnostics } from "./services/chat-webhook-diagnostics.js";
type UiMode = "none" | "static" | "vite-dev";
const FEEDBACK_EXPORT_FLUSH_INTERVAL_MS = 5_000;
const VITE_DEV_ASSET_PREFIXES = [
"/@fs/",
"/@id/",
"/@react-refresh",
"/@vite/",
"/assets/",
"/node_modules/",
"/src/",
];
const VITE_DEV_STATIC_PATHS = new Set([
"/apple-touch-icon.png",
"/favicon-16x16.png",
"/favicon-32x32.png",
"/favicon.ico",
"/favicon.svg",
"/site.webmanifest",
"/sw.js",
]);
export function isDatabaseConnectionUnavailableError(err: unknown): boolean {
const error = err as { code?: unknown; message?: unknown; cause?: unknown };
if (error?.code === "ECONNREFUSED") return true;
return Boolean(
error?.cause && isDatabaseConnectionUnavailableError(error.cause),
);
}
export function resolveViteHmrPort(serverPort: number): number {
return derivePaperclipViteHmrPort(serverPort);
}
export function resolveViteHmrHost(bindHost: string): string | undefined {
const normalized = bindHost.trim().toLowerCase();
if (
normalized === "0.0.0.0" ||
normalized === "::" ||
normalized === "127.0.0.1" ||
normalized === "::1" ||
normalized === "localhost"
)
return undefined;
return bindHost;
}
export function resolveViteHmrProtocol(
value: string | undefined,
): "ws" | "wss" | undefined {
if (!value) return undefined;
if (value === "ws" || value === "wss") return value;
throw new Error("PAPERCLIP_VITE_HMR_PROTOCOL must be ws or wss");
}
export function listenViteHmrServer(
server: HttpServer,
port: number,
bindHost: string,
): Promise<void> {
return new Promise((resolve, reject) => {
const onError = (error: Error) => {
server.off("listening", onListening);
reject(error);
};
const onListening = () => {
server.off("error", onError);
resolve();
};
server.once("error", onError);
server.once("listening", onListening);
server.listen(port, bindHost);
});
}
export function shouldServeViteDevHtml(req: ExpressRequest): boolean {
const pathname = req.path;
if (VITE_DEV_STATIC_PATHS.has(pathname)) return false;
if (VITE_DEV_ASSET_PREFIXES.some((prefix) => pathname.startsWith(prefix)))
return false;
return req.accepts(["html"]) === "html";
}
export function shouldEnablePrivateHostnameGuard(opts: {
deploymentMode: DeploymentMode;
deploymentExposure: DeploymentExposure;
}): boolean {
return (
opts.deploymentExposure === "private" &&
(opts.deploymentMode === "local_trusted" ||
opts.deploymentMode === "authenticated")
);
}
export function createManagedBundledPluginWorkerRecovery(input: {
managedBundledPluginKeys: readonly string[];
workerManager: Pick<
PluginWorkerManager,
"getWorker" | "isRunning" | "stopWorker"
>;
getLoader: () => Pick<PluginLoader, "loadSingle"> | null;
}): (plugin: { id: string; pluginKey: string }) => Promise<boolean> {
const recoverablePluginKeys = new Set(input.managedBundledPluginKeys);
const inFlightStarts = new Map<string, Promise<boolean>>();
// A failed attempt can leave behind the dead handle it registered (e.g. the
// worker process died during initialize, which kills the process without
// scheduling a restart). No pre-existing handle survives to a recovery
// attempt — recovery only starts when getWorker() was empty — so discarding
// the dead handle lets a later capability request retry instead of being
// blocked by the handle-presence gate until the process restarts. Handles
// in starting/running/backoff states belong to the worker manager's own
// lifecycle and are left alone.
const discardDeadRecoveryHandle = async (plugin: {
id: string;
pluginKey: string;
}) => {
const handle = input.workerManager.getWorker(plugin.id);
if (!handle || (handle.status !== "crashed" && handle.status !== "stopped"))
return;
try {
await input.workerManager.stopWorker(plugin.id);
} catch (err) {
logger.warn(
{
pluginId: plugin.id,
pluginKey: plugin.pluginKey,
err: err instanceof Error ? err.message : String(err),
},
"failed to discard dead plugin worker handle after recovery failure",
);
}
};
return async (plugin) => {
if (!recoverablePluginKeys.has(plugin.pluginKey)) return false;
const inFlight = inFlightStarts.get(plugin.id);
if (inFlight) return inFlight;
const startPromise = (async () => {
if (input.workerManager.getWorker(plugin.id)) {
return input.workerManager.isRunning(plugin.id);
}
const loader = input.getLoader();
if (!loader) return false;
try {
const result = await loader.loadSingle(plugin.id, {
markErrorOnFailure: false,
});
if (
result.success === true ||
input.workerManager.isRunning(plugin.id)
) {
return true;
}
await discardDeadRecoveryHandle(plugin);
return false;
} catch (err) {
logger.warn(
{
pluginId: plugin.id,
pluginKey: plugin.pluginKey,
err: err instanceof Error ? err.message : String(err),
},
"managed bundled plugin lazy worker recovery failed",
);
await discardDeadRecoveryHandle(plugin);
throw err;
}
})();
inFlightStarts.set(plugin.id, startPromise);
try {
return await startPromise;
} finally {
if (inFlightStarts.get(plugin.id) === startPromise) {
inFlightStarts.delete(plugin.id);
}
}
};
}
export async function createApp(
db: Db,
opts: {
cloudWarmStandby?: CloudWarmStandby;
uiMode: UiMode;
serverPort: number;
storageService: StorageService;
feedbackExportService?: {
hasPendingFeedbackTraces(): Promise<boolean>;
flushPendingFeedbackTraces(input?: {
companyId?: string;
traceId?: string;
limit?: number;
now?: Date;
signal?: AbortSignal;
}): Promise<unknown>;
};
databaseBackupService?: InstanceDatabaseBackupService;
prepareIdleDatabaseBackup?: () => Promise<boolean>;
idleDeadlineOwners?: IdleDeadlineOwners;
/** Filled with the chat service's Retry callback for background budget-resume delivery. */
budgetResumeDelivery?: BudgetResumeDelivery;
databaseBackupHealth?: InspectDatabaseBackupHealthOptions;
deploymentMode: DeploymentMode;
deploymentExposure: DeploymentExposure;
allowedHostnames: string[];
bindHost: string;
authPublicBaseUrl?: string;
chatWebhookPublicBaseUrl?: string;
authReady: boolean;
companyDeletionEnabled: boolean;
announcements?: { enabled: boolean; feedUrl: string };
instanceId?: string;
hostVersion?: string;
localPluginDir?: string;
pluginMigrationDb?: Db;
pluginWorkerManager?: PluginWorkerManager;
decisionServiceOptions: DecisionServiceOptions;
betterAuthHandler?: express.RequestHandler;
resolveSession?: (
req: ExpressRequest,
) => Promise<BetterAuthSessionResult | null>;
/**
* `plugins.autoInstall` from the managed config (PAPERCLIP_MANAGED_CONFIG).
* `null`/absent ⇒ self-hosted: only the built-in kubernetes bundle is
* ensured, exactly as before. A managed list is resolved against the
* bundled catalog fail-to-start (see services/bundled-plugins.ts).
*/
managedPluginAutoInstall?: readonly string[] | null;
/** Test override for the bundled plugin catalog root. */
bundledPluginCatalogRoot?: string;
},
) {
const app = express();
// First, so the reported server time includes admission and database waits.
app.use("/api", apiServerTiming());
app.use(databaseOperationMiddleware);
app.use(idleAdmissionMiddleware);
const staticUi = express.Router();
app.locals.paperclipDb = db;
const isWarmStandby = opts.cloudWarmStandby ?? (() => false);
const health = healthRoutes(db, {
deploymentMode: opts.deploymentMode,
deploymentExposure: opts.deploymentExposure,
authReady: opts.authReady,
companyDeletionEnabled: opts.companyDeletionEnabled,
databaseBackupHealth: opts.databaseBackupHealth,
isWarmStandby,
});
const captureRawBody = (
req: express.Request,
_res: express.Response,
buf: Buffer,
) => {
(req as unknown as { rawBody: Buffer }).rawBody = buf;
};
// Respect the operator's `TRUST_PROXY` env var (see middleware/trust-proxy.ts).
// Default is unset → Express trusts nothing, which is the only safe choice
// when the server may be reachable without a known reverse proxy in front.
applyTrustProxy(app, parseTrustProxyEnv(process.env.TRUST_PROXY));
app.use(
COMPANY_IMPORT_API_PATH,
express.json({
limit: PORTABLE_JSON_BODY_LIMIT,
verify: captureRawBody,
}),
);
// Chat providers sign the exact request bytes. Capture every webhook media
// type before the global JSON parser so JSON events and form-encoded action
// callbacks are verified against the provider's original body.
app.use(
"/api/chat-webhooks",
createChatWebhookDiagnostics(),
chatWebhookBodyParser,
);
let dotPrincipalOAuth: ReturnType<typeof createPublicMcpOAuth> | null = null;
const dotPrincipalHttp = createDotPrincipalHttp(() => dotPrincipalOAuth);
app.use(dotPrincipalHttp.bodyParser);
const jsonBodyParser = express.json({ limit: DEFAULT_JSON_BODY_LIMIT, verify: captureRawBody });
// File tickets authenticate before parsing bytes; JSON attachments must remain bytes too.
app.use((req, res, next) => (dotPrincipalHttp.matchesPost(req) || ["/mcp/files/upload", "/mcp/runner/files/upload"].includes(req.path)) ? next() : jsonBodyParser(req, res, next));
app.use("/api", apiCompression());
app.use(httpLogger);
const privateHostnameGateEnabled = shouldEnablePrivateHostnameGuard({
deploymentMode: opts.deploymentMode,
deploymentExposure: opts.deploymentExposure,
});
const privateHostnameAllowSet = resolvePrivateHostnameAllowSet({
allowedHostnames: opts.allowedHostnames,
bindHost: opts.bindHost,
});
app.use(
privateHostnameGuard({
enabled: privateHostnameGateEnabled,
allowedHostnames: opts.allowedHostnames,
bindHost: opts.bindHost,
}),
);
let mcpConfig: ReturnType<typeof publicMcpConfig> = null;
try { mcpConfig = publicMcpConfig(process.env, opts.authPublicBaseUrl); }
catch { logger.warn("Assistant connections require an HTTPS public URL (HTTP loopback is allowed for development)."); }
const publicMcpOAuth = mcpConfig ? createPublicMcpOAuth(db, mcpConfig) : null;
const publicMcpIngress = Router();
publicMcpIngress.use(dotPrincipalHttp.router);
app.use(cloudRuntimeIdentityMiddleware(db));
// Routine probes stop before session/actor resolution, even after claim.
// Keep hostname validation and the signed-bootstrap endpoint guard above.
app.use("/api/liveness", livenessRoutes());
app.use("/api/ready", readinessRoutes());
// A signed claim above commits identity before any normal request can seed
// company data. Unclaimed probes bypass session resolution as well as SQL.
app.use(cloudWarmStandbyMiddleware(isWarmStandby, health, staticUi));
const workerManager = opts.pluginWorkerManager ?? createPluginWorkerManager();
app.use("/api/customer-success/v1", customerSuccessRoutes(db, opts.storageService, {
pluginWorkerManager: workerManager,
}));
app.use(museTransportRoutes(db, opts.authPublicBaseUrl));
app.use(publicMcpIngress);
// Connection-intent tools carry their own short-lived, run-bound bearer and
// must be reachable by remote adapters that intentionally do not receive an
// agent API key. Every request revalidates the active heartbeat row.
app.use(runtimeConnectionIntentRoutes(db));
app.use(nativeRunMcpRoutes());
app.use(runnerAttachmentDownloadRoutes(db, opts.storageService));
const hostServicesDisposers = new Map<string, () => void>();
let lifecyclePluginsReady = false;
const agentLifecycle = startAgentLifecycle(db, createLifecycleDriver(db, workerManager), () => lifecyclePluginsReady && !isWarmStandby() && !isIdleTaskDrainActive());
const connectionIntentHeartbeat = heartbeatService(db, {
pluginWorkerManager: workerManager,
});
const chatChannels = chatChannelService(db, {
allowLocalVoiceBoard: opts.deploymentMode === "local_trusted",
deferWebhookProcessing: true,
heartbeat: connectionIntentHeartbeat,
publicBaseUrl: opts.authPublicBaseUrl,
githubWizardOrigin: opts.deploymentMode === "local_trusted"
&& ["127.0.0.1", "localhost", "::1"].includes(opts.bindHost ?? "")
&& Number.isInteger(opts.serverPort) && opts.serverPort! > 0
? `http://${opts.bindHost === "::1" ? "[::1]" : opts.bindHost}:${opts.serverPort}` : null,
webhookPublicBaseUrl: opts.chatWebhookPublicBaseUrl,
resolveNativeQuestion: (interaction) =>
deliverNativeQuestionResponse(db, interaction),
storage: opts.storageService,
});
if (opts.budgetResumeDelivery) opts.budgetResumeDelivery.prepareChatRetry = chatChannels.prepareFailedChatRunRetry;
// A restart's automatic Retry of a connected conversation needs the same callback.
registerInterruptedTurnChatRetry(chatChannels.prepareFailedChatRunRetry);
// Voice capabilities are verified by the signed callback route, not by
// board/agent bearer authentication. Mount before actorMiddleware so its
// per-session Bearer cannot be mistaken for an agent API key.
app.use(voiceWebhookRoutes(db, chatChannels.voice));
app.use(
actorMiddleware(db, {
deploymentMode: opts.deploymentMode,
resolveSession: opts.resolveSession,
}),
);
// After the actor middleware on purpose: a valid Cloud control assertion
// REPLACES whatever actor the request otherwise resolved to, and only on
// the one endpoint it authorizes (see the middleware for the contract).
app.use(cloudControlMiddleware());
app.use(connectionEventWakeRoutes(db));
app.use(skillAccessRequestContext);
app.use("/api/auth", authRoutes(db));
if (opts.betterAuthHandler) {
app.all("/api/auth/{*authPath}", opts.betterAuthHandler);
}
app.use(llmRoutes(db));
// Provider-authenticated ingress is intentionally outside the board
// mutation guard. The Chat SDK adapter verifies the provider signature
// before Paperclip persists or acts on any event.
const emailChannels = emailChannelService(db, {
isReconciliationEnabled: () => !isWarmStandby(),
isBackgroundWorkEnabled: () => !isWarmStandby() && !isIdleTaskDrainActive(),
heartbeat: connectionIntentHeartbeat,
storage: opts.storageService,
publicBaseUrl: opts.chatWebhookPublicBaseUrl ?? opts.authPublicBaseUrl,
});
app.use(emailWebhookRoutes(emailChannels));
app.locals.fastResponses = fastResponseService(db, {
authorizeExternal: async (tx, request) => {
const [endpoint] = await tx.select({ provider: chatEndpoints.provider }).from(chatEndpoints).where(eq(chatEndpoints.id, request.endpointId!));
return endpoint?.provider === "agentmail" ? emailChannels.authorizeFastResponse(tx, request) : chatChannels.authorizeFastResponse(tx, request);
},
publishExternal: async (tx, request, commentId, text) => {
const [endpoint] = await tx.select({ provider: chatEndpoints.provider }).from(chatEndpoints).where(eq(chatEndpoints.id, request.endpointId!));
if (endpoint?.provider !== "agentmail") return false;
await emailChannels.publishFastResponse(tx, request, commentId, text);
return true;
},
budgetHooks: { cancelWorkForScope: connectionIntentHeartbeat.cancelBudgetScopeWork },
});
app.use(chatWebhookRoutes(chatChannels));
// The instance validates single-use registration state and its trusted
// current origin. These exact callback routes are the public setup returns.
app.get("/api/chat-github/manifest/callback", async (req, res) => {
res.set("Cache-Control", "no-store");
res.set("Referrer-Policy", "no-referrer");
const redirect = await chatChannels.githubWizard.directCallback(String(req.query.state ?? ""), String(req.query.code ?? ""), req.actor);
res.redirect(303, redirect);
});
app.get("/api/chat-github/cloud/callback", async (req, res) => {
res.set("Cache-Control", "no-store"); res.set("Referrer-Policy", "no-referrer");
res.redirect(303, await chatChannels.githubWizard.cloudCallback(String(req.query.state ?? ""), String(req.query.registration ?? ""), typeof req.query.claim === "string" ? req.query.claim : undefined, req.actor));
});
app.get("/api/chat-github/identity/callback", async (req, res) => {
res.set("Cache-Control", "no-store"); res.set("Referrer-Policy", "no-referrer");
res.redirect(303, await chatChannels.githubWizard.identityCallback(String(req.query.state ?? ""), String(req.query.code ?? ""), req.actor));
});
const managedAutoInstallKeys = resolveManagedBundledPluginKeys(opts.managedPluginAutoInstall);
const bundledCatalogRoot =
opts.bundledPluginCatalogRoot ?? resolveBundledCatalogRoot(process.env);
const distributionPlugins = readDistributionPluginCatalog(bundledCatalogRoot, BUNDLED_PLUGIN_CATALOG);
const bundledPluginInstalls = resolveBundledPluginInstalls(
managedAutoInstallKeys ?? SELF_HOSTED_AUTO_INSTALL_KEYS,
{
catalogRoot: bundledCatalogRoot,
env: process.env,
enforceCatalogRoot: managedAutoInstallKeys !== null,
distributionPlugins,
},
);
const bundledPackageResolver = managedAutoInstallKeys === null ? undefined : createBundledPluginPackageResolver(bundledPluginInstalls);
const managedBundledPluginKeys =
managedAutoInstallKeys !== null
? bundledPluginInstalls.map((install) => install.pluginKey)
: [];
let runtimePluginLoader: Pick<PluginLoader, "loadSingle"> | null = null;
// A sibling process can install a managed bundled plugin while this process
// skips the mid-install row, then finish the row after this process's
// loadAll() pass. The capabilities route may recover only those managed
// bundles by starting their ready-but-unstarted worker lazily.
const recoverManagedBundledPluginWorker =
managedAutoInstallKeys !== null
? createManagedBundledPluginWorkerRecovery({
managedBundledPluginKeys,
workerManager,
getLoader: () => runtimePluginLoader,
})
: undefined;
// Mount API routes
const api = Router();
const agentAvatars = agentAvatarRoutes();
api.use(agentAvatars.router);
api.use(boardMutationGuard());
api.use("/health", health);
api.use(openApiRoutes());
api.use("/cloud", cloudRoutes());
api.use("/companies", companyRoutes(db, opts.storageService));
api.use(llmRoutes(db));
api.use(folderRoutes(db));
api.use(companySkillRoutes(db, { pluginWorkerManager: workerManager }));
api.use(companySkillPolicyRoutes(db));
api.use(inboxAgentPolicyRoutes(db));
api.use(agentCommentaryRoutes(db));
api.use(agentChatMessageRoutes(db));
api.use(builtInAgentRoutes(db));
api.use(summarySlotRoutes(db));
api.use(statusCardRoutes(db));
api.use(teamsCatalogRoutes(db));
// The setup-token login session service. The router builds it and hands it
// back through the callback below, so the shutdown hook can cancel every live
// session (SR-4).
let setupTokenLoginService: SetupTokenSessionService | null = null;
// The dedicated proxy IP or CIDR allowlist for the confidential setup-token
// login responses (SR-7). The global `TRUST_PROXY` setting does not satisfy
// the guard; an operator sets this allowlist to the real TLS-terminating
// proxy addresses. An empty value keeps the confidential responses on direct
// TLS (or a `local_trusted` loopback peer) only.
const setupTokenLoginProxyAllowlist = (
process.env.CLAUDE_LOGIN_TRUSTED_PROXIES ?? ""
)
.split(",")
.map((entry) => entry.trim())
.filter((entry) => entry.length > 0);
// The explicit operator declaration that a platform edge terminates TLS for
// every client request (SR-7). This complements the allowlist for managed
// platforms (Railway, Render, Fly, and the like) where the app socket is
// always plain HTTP and the edge-proxy peer addresses are not stable or
// documented, so `CLAUDE_LOGIN_TRUSTED_PROXIES` cannot express them. It is a
// dedicated, single-purpose setting; the guard still never reads the global
// `TRUST_PROXY` value.
const setupTokenLoginEdgeTlsTerminated = /^(1|true|yes|on)$/i.test(
(process.env.CLAUDE_LOGIN_EDGE_TLS_TERMINATED ?? "").trim(),
);
// Bind the production setup-token login transport. It carries the live lease
// manager, the login-process factory over the sandbox pseudo-terminal, and the
// durable cleanup store. The factory passes only the fixed command
// `CLAUDE_SETUP_TOKEN_COMMAND`; it never reads a command from a
// route, a request body, or an adapter configuration. The durable store and the
// startup reaper are live now, so a restart reaps a leftover lease.
//
// The live sandbox pseudo-terminal opener binds inside the sandbox provider
// worker, so the server process does not hold the raw sandbox process. The
// opener drives the worker through the plugin worker manager route gate.
// The manager mints a host-owned route identifier, permits one
// active credential pseudo-terminal per worker, binds the worker session
// identifier one time for output only, and terminalizes the route on every open
// failure path. With the opener supplied, the provider acquires a lease and the
// start route drives a live login instead of the fixed 503.
const setupTokenLoginTransport = buildSetupTokenLoginTransport({
sandbox: createProductionSetupTokenSandboxProvider({
db,
environments: environmentService(db),
environmentRuntime: environmentRuntimeService(db, {
pluginWorkerManager: workerManager,
}),
openLivePtySession: createWorkerBoundLoginPtyOpener({
workerManager,
environments: environmentService(db),
log: (line) => logger.info(line),
}),
log: (line) => logger.info(line),
}),
store: createProductionSetupTokenCleanupStore(db),
// Bind the atomic credential-claim writer, so a completed login transitions
// the durable row to `stored` and stores the minted token in one control-plane
// transaction. The writer reads the company and the owner only from the
// immutable session scope. The confirm-replacement flow owns rotation. Without
// this writer the router falls back to the deferred, fail-closed 503 and never
// stores the token.
completeCredential: createSetupTokenSecretWriter({ db }),
// Forward the login runner diagnostic lines to the server logger. The
// runner is the sole producer, and every line is a fixed, non-secret
// literal. Without this sink the diagnostics fall back to a no-op in
// production, so a failed login leaves no log trail.
log: (line) => logger.info(line),
});
api.use(
agentRoutes(db, {
chatRunRetries: chatChannels,
pluginWorkerManager: workerManager,
deploymentMode: opts.deploymentMode,
confidentialProxyAllowlist: setupTokenLoginProxyAllowlist,
confidentialEdgeTlsTerminated: setupTokenLoginEdgeTlsTerminated,
setupTokenLogin: setupTokenLoginTransport,
onSetupTokenLoginService: (service) => {
// Capture the service, so the graceful-shutdown hook cancels every live
// session and releases each lease. The standalone scheduled reaper owns
// the startup and interval lease cleanup now (SR-4).
setupTokenLoginService = service;
},
}),
);
api.use(agentPermissionGrantRoutes(db));
api.use(agentInteractionPermissionRoutes(db));
api.use(assetRoutes(db, opts.storageService));
api.use(agentProfileAvatarRoutes(db, opts.storageService));
api.use(projectToolRoutes(db));
api.use(taskWorkspaceRoutes(db));
api.use(projectRoutes(db));
api.use(caseRoutes(db, opts.storageService));
api.use(caseHandoffRoutes(db));
api.use(workflowRoutes(db));
api.use(issueTreeControlRoutes(db, { pluginWorkerManager: workerManager }));
api.use(fileResourceRoutes(db, { remoteFiles: createRemoteWorkspaceFiles(db, {
environmentRuntime: environmentRuntimeService(db, { pluginWorkerManager: workerManager }),
loadSavedWorkspace: createSavedWorkspaceLoader(db, opts.storageService),
openSavedWorkspaceArchive: createSavedWorkspaceArchiveOpener(db, opts.storageService),
}) }));
api.use(routineRoutes(db, { pluginWorkerManager: workerManager }));
api.use(
environmentRoutes(db, {
pluginWorkerManager: workerManager,
retryPendingEnvironmentCleanup: input => connectionIntentHeartbeat.sweepPendingCleanupLeases({ environmentRetry: input }),
recoverMissingPluginWorker: recoverManagedBundledPluginWorker
? {
pluginKeys: managedBundledPluginKeys,
startWorker: recoverManagedBundledPluginWorker,
}
: undefined,
}),
);
api.use(executionWorkspaceRoutes(db, { pluginWorkerManager: workerManager }));
api.use(emailRoutes(db, emailChannels));
api.use(goalRoutes(db));
api.use(onboardingSeedRoutes(db));
api.use(boardChatRoutes(db, { deploymentMode: opts.deploymentMode }));
api.use(voiceSessionRoutes(db, chatChannels.voice));
api.use(approvalRoutes(db, { pluginWorkerManager: workerManager }));
api.use(secretRoutes(db));
api.use(managedAgentProfileRoutes(db));
api.use(remoteAgentProfileRoutes(db));
api.use(
chatChannelRoutes(db, {
heartbeat: connectionIntentHeartbeat,
publicBaseUrl: opts.authPublicBaseUrl,
storage: opts.storageService,
service: chatChannels,
}),
);
const trustedLocalStdioRuntimeHost =
process.env.PAPERCLIP_TRUSTED_MCP_RUNTIME_HOST ??
process.env.PAPERCLIP_TOOL_RUNTIME_TRUSTED_HOST ??
null;
api.use(costRoutes(db, { pluginWorkerManager: workerManager }));
api.use(activityRoutes(db));
api.use(dashboardRoutes(db));
api.use(attentionRoutes(db, {
pluginWorkerManager: workerManager,
decisionServiceOptions: opts.decisionServiceOptions,
}));
api.use(decisionTrainingRoutes(db));
api.use(decisionRoutes(db, opts.decisionServiceOptions));
api.use(decisionQueueRoutes(db));
api.use(userProfileRoutes(db));
api.use(sidebarBadgeRoutes(db));
api.use(sidebarPreferenceRoutes(db));
api.use(announcementRoutes(db, { ...opts.announcements, version: opts.hostVersion ?? serverVersion }));
api.use(resourceMembershipRoutes(db));
api.use(primaryAgentRoutes(db));
api.use(workspacePreferenceRoutes(db));
api.use(inboxDismissalRoutes(db));
api.use(instanceSettingsRoutes(db, workerManager, opts.prepareIdleDatabaseBackup, opts.idleDeadlineOwners,
input => resourceLifecycleDelivery.prepareForIdleSleep(input)));
api.use(ephemeralEnrollmentRoutes(db));
if (opts.databaseBackupService) {
api.use(instanceDatabaseBackupRoutes(opts.databaseBackupService));
}
const pluginRegistry = pluginRegistryService(db);
const eventBus = createPluginEventBus();
setPluginEventBus(eventBus);
const jobStore = pluginJobStore(db);
const lifecycle = pluginLifecycleManager(db, { workerManager });
const scheduler = createPluginJobScheduler({
isBackgroundWorkEnabled: () => !isWarmStandby() && !isIdleTaskDrainActive(),
db,
jobStore,
workerManager,
});
const toolDispatcher = createPluginToolDispatcher({
workerManager,
lifecycleManager: lifecycle,
db,
});
const gatewayOAuthAccess = toolAccessService(db, {
deploymentMode: opts.deploymentMode,
deploymentExposure: opts.deploymentExposure,
trustedLocalStdioRuntimeHost,
});
const toolActionDeliveries = toolActionDeliveryService(db, heartbeatService(db, { pluginWorkerManager: workerManager }));
const toolGateway = createToolGatewayService(db, {
budgetHooks: { cancelWorkForScope: heartbeatService(db, { pluginWorkerManager: workerManager }).cancelBudgetScopeWork },
onToolActionSettled: (id) => toolActionDeliveries.deliver(id),
pluginToolDispatcher: toolDispatcher,
deploymentMode: opts.deploymentMode,
deploymentExposure: opts.deploymentExposure,
trustedLocalStdioRuntimeHost,
oauthGrantRefresher: (input) =>
gatewayOAuthAccess.refreshOAuthGrantCredentials(input),
});
// Issue routes are intentionally mounted after the gateway is constructed because
// issue approval endpoints delegate to it. The intervening routers use distinct
// route prefixes, so this dependency does not change issue-route precedence.
api.use(issueRoutes(db, opts.storageService, {
chatRunRetries: chatChannels,
pluginWorkerManager: workerManager,
approveToolActionRequest: (input) => toolGateway.approveActionRequest(input),
declineToolActionRequest: (input) => toolGateway.declineActionRequest(input),
}));
api.use(slackToolRoutes(db, opts.authPublicBaseUrl));
app.locals.toolGateway = toolGateway;
registerAssignedMcpGateway(db, toolGateway);
app.locals.toolActionDeliveries = toolActionDeliveries;
app.use(mcpGatewayProtocolRoutes(toolGateway));
api.use(decisionModelRoutes(db));
api.use(agentObservabilityRoutes(db, { deploymentMode: opts.deploymentMode, deploymentExposure: opts.deploymentExposure }));
api.use(fastResponseRoutes(db));
api.use(linkPreviewRoutes(db));
api.use(promptInjectionPreventionRoutes(db));
api.use(aiConnectionRoutes(db, { deploymentMode: opts.deploymentMode, deploymentExposure: opts.deploymentExposure, trustedLocalStdioRuntimeHost, pluginWorkerManager: workerManager }));
api.use(
toolAccessRoutes(db, {
deploymentMode: opts.deploymentMode,
deploymentExposure: opts.deploymentExposure,
authPublicBaseUrl: opts.authPublicBaseUrl,
trustedLocalStdioRuntimeHost,
toolGateway,
connectionIntentHeartbeat,
}),
);
api.use(connectionIntentBoardRoutes(db, connectionIntentHeartbeat));
api.use(
smokeLabRoutes(db, {
deploymentMode: opts.deploymentMode,
deploymentExposure: opts.deploymentExposure,
}),
);
const jobCoordinator = createPluginJobCoordinator({
db,
lifecycle,
scheduler,
jobStore,
});
const hostServiceCleanup = createPluginHostServiceCleanup(
lifecycle,
hostServicesDisposers,
);
let viteHtmlRenderer: ReturnType<typeof createCachedViteHtmlRenderer> | null =
null;
let viteDevServer: { close(): Promise<void> } | null = null;
let viteHmrServer: HttpServer | null = null;
const loader = pluginLoader(
db,
{
localPluginDir: opts.localPluginDir ?? DEFAULT_LOCAL_PLUGIN_DIR,
migrationDb: opts.pluginMigrationDb,
assertPackageActivation: bundledPackageResolver?.assertActivation ?? distributionPluginActivationGuard(bundledCatalogRoot, distributionPlugins, managedAutoInstallKeys),
resolvePackageRoot: bundledPackageResolver?.resolve,
prebuiltOnly: managedAutoInstallKeys !== null,
enableLocalFilesystem: managedAutoInstallKeys === null,
enableNpmDiscovery: managedAutoInstallKeys === null,
},
{
workerManager,
eventBus,
jobScheduler: scheduler,
jobStore,
toolDispatcher,
lifecycleManager: lifecycle,
instanceInfo: {
instanceId: opts.instanceId ?? "default",
hostVersion: opts.hostVersion ?? serverVersion,
deploymentMode: opts.deploymentMode,
deploymentExposure: opts.deploymentExposure,
},
buildHostHandlers: (pluginId, manifest) => {
const notifyWorker = (method: string, params: unknown) => {
const handle = workerManager.getWorker(pluginId);
if (handle) handle.notify(method, params);
};
const services = buildHostServices(
db,
pluginId,
manifest.id,
eventBus,
notifyWorker,
{
pluginWorkerManager: workerManager,
manifest,
},
);
hostServicesDisposers.set(pluginId, () => services.dispose());
return createHostClientHandlers({
pluginId,
capabilities: manifest.capabilities,
services,
});
},
},
);
runtimePluginLoader = loader;
workerManager.setActivationTimeoutRecovery?.(plugin => loader.requestActivationTimeoutRecovery(plugin));
const browserUse = browserUseService(db, undefined,
{ cancelWorkForScope: heartbeatService(db, { pluginWorkerManager: workerManager }).cancelBudgetScopeWork },
(session, run) => toolGateway.browserUseSessionAuthorized({ ...session, runId: run.heartbeatRunId, invocationId: run.invocationId }));
api.use(browserUseRoutes(db, browserUse));
api.use(toolGatewayRoutes(db, toolGateway));
api.use(mcpAppsRoutes(db, toolGateway));
api.use(mcpSkillsRoutes(db, toolGateway));
api.use(
pluginRoutes(
db,
loader,
{ scheduler, jobStore },
{ workerManager },
{ toolDispatcher },
{ workerManager },
{ toolGateway },
),
);
api.use(
adapterRoutes({
db,
resolveRunnerTarget: (companyId, environmentId) => resolveAgentRunnerTargetForCompany(db, companyId, environmentId),
getNativeRunnerEnabled: async () =>
(await instanceSettingsService(db).getExperimental())
.enableNativeRunner === true,
getOpenAiDotEnabled: async () =>
(await instanceSettingsService(db).getExperimental())
.enableOpenAiDot === true,
}),
);
api.use(
accessRoutes(db, {
deploymentMode: opts.deploymentMode,
deploymentExposure: opts.deploymentExposure,
bindHost: opts.bindHost,
allowedHostnames: opts.allowedHostnames,
authPublicBaseUrl: opts.authPublicBaseUrl,
}),
);
let publicMcpEvents: PublicMcpEvents | null = null;
const grokDispatcher = createGrokRoutineDispatcher(db, () => !isWarmStandby() && !isIdleTaskDrainActive());
let dotMcpEvents: PublicMcpEvents | null = null;
if (publicMcpOAuth) {
const dispatch = createMcpApiDispatch(api, app.locals);
publicMcpEvents = createPublicMcpEvents(db, publicMcpOAuth, dispatch, {
isBackgroundWorkEnabled: () => !isWarmStandby() && !isIdleTaskDrainActive(),
});
const transfers = createPublicMcpTransfers(db, publicMcpOAuth, dispatch, opts.storageService);
publicMcpIngress.use(transfers.router);
publicMcpIngress.use(publicMcpIngressRoutes(publicMcpOAuth, createPublicMcpExecutor(db, publicMcpOAuth, dispatch, transfers), publicMcpEvents));
const dotOAuth = createPublicMcpOAuth(db, { ...publicMcpOAuth.config, resource: publicMcpOAuth.config.origin + "/mcp/runner" });
dotPrincipalOAuth = dotOAuth;
dotMcpEvents = createPublicMcpEvents(db, dotOAuth, dispatch, { enableDotRunner: true, isBackgroundWorkEnabled: () => !isWarmStandby() && !isIdleTaskDrainActive() });
const grokTransfers = createPublicMcpTransfers(db, dotOAuth, dispatch, opts.storageService, { pathPrefix: "/mcp/runner/files",
agentSession: { authorize: (principal, sessionId, taskId, write, database) => dotRunnerBroker(database ?? db).authorizeFileSession(principal, sessionId, taskId, write) } });
publicMcpIngress.use(grokTransfers.router);
publicMcpIngress.use(publicMcpIngressRoutes(dotOAuth, createPublicMcpExecutor(db, dotOAuth, dispatch), dotMcpEvents, createDotRunnerMcpTools(db, connectionIntentHeartbeat, grokTransfers)));
api.use(publicMcpManagementRoutes(publicMcpOAuth, dotOAuth));
api.use(dotRunnerRoutes(db, publicMcpOAuth.config.origin + "/mcp/runner"));
api.use(grokBotRoutes(db, publicMcpOAuth.config.origin + "/mcp/runner"));
}
api.use(museBoardRoutes(db, opts.authPublicBaseUrl));
api.use(museQualificationRoutes(db));
app.use("/api", api);
app.use("/api", (_req, res) => {
res.status(404).json({ error: "API route not found" });
});
app.use(
pluginUiStaticRoutes(db, {
localPluginDir: opts.localPluginDir ?? DEFAULT_LOCAL_PLUGIN_DIR,
resolvePackageRoot: bundledPackageResolver?.resolve,
}),
);
const __dirname = path.dirname(fileURLToPath(import.meta.url));
if (opts.uiMode === "static") {
// Try published location first (server/ui-dist/), then monorepo dev location (../../ui/dist)
const candidates = [
path.resolve(__dirname, "../ui-dist"),
path.resolve(__dirname, "../../ui/dist"),
];
const uiDist = candidates.find((p) =>
fs.existsSync(path.join(p, "index.html")),
);
if (uiDist) {
staticUi.use(staticUiCompression());
// Hashed asset files (Vite emits them under /assets/<name>.<hash>.<ext>)
// never change once built, so they can be cached aggressively.
staticUi.use(
"/assets",
staticUiFiles(path.join(uiDist, "assets"), {
maxAge: "1y",
immutable: true,
}),
);
if (process.env.PAPERCLIP_SHARED_UI_ASSETS === "true") {
const retained = retainedUiAssets(createStorageProviderFromConfig(loadStorageConfig()));
// Publish while the routes below are built; the outcome is awaited
// before the first background work starts. The promise settles to the
// failure instead of rejecting, so it is never an unhandled rejection.
app.locals.retainedUiAssetsPublication = trackIdleWork(publishRetainedUiAssetsAtStartup(retained, path.join(uiDist, "assets"), error =>
logger.error({ err: error }, "Shared UI assets could not be published because storage is unavailable; serving this build from local files"))
.then(() => null, (error: unknown) => ({ error })));
staticUi.use("/assets", retained.serve);
}
// Serve root/index through the same runtime HTML transform as SPA routes.
staticUi.get(["/", "/index.html"], (_req, res) => {
res.type("html").set("Cache-Control", "no-cache").send(readBrandedStaticIndexHtml(uiDist));
});
// Non-hashed static files (favicon.ico, manifest, robots.txt, etc.):
// short cache so operators who swap them out see the new version
// reasonably fast, with must-revalidate overrides for index.html and
// sw.js (see staticUiCacheControl for why those two).
staticUi.use(
staticUiFiles(uiDist, {
maxAge: "1h",
setHeaders(res, filePath) {
const override = staticUiCacheControl(filePath);
if (override) {
res.set("Cache-Control", override);
}
},
}),
);
// SPA fallback. Only for non-asset routes — if the browser asks for
// /assets/something.js that doesn't exist, we must NOT serve the HTML
// shell: the browser would try to load it as a JavaScript module, fail
// with a MIME-type error, and cache that broken response. Return 404
// instead. The index.html response itself is no-cache so a subsequent
// deploy's updated asset hashes are picked up on next load.
staticUi.get(/.*/, (req, res) => {
if (req.path.startsWith("/assets/")) {
res.status(404).end();
return;
}
res
.status(200)
.set("Content-Type", "text/html")
.set("Cache-Control", "no-cache")
.end(readBrandedStaticIndexHtml(uiDist));
});
} else {
console.warn("[paperclip] UI dist not found; running in API-only mode");
}
if (process.env.PAPERCLIP_MANAGED_RUNTIME_EXPOSURE === "tailscale_https") {
// The managed-runtime supervisor waits for the app port AND its derived
// Vite HMR companion port to bind before publishing the service. Static
// mode has no Vite, so bind the same placeholder listener dev mode uses
// or the supervisor kills a healthy server at the readiness deadline
// (PAP-18043).
const hmrServer = createHttpServer((_req, res) => {
res.writeHead(426, { "Content-Type": "text/plain" });
res.end("Upgrade Required");
});
await listenViteHmrServer(
hmrServer,
resolveViteHmrPort(opts.serverPort),
opts.bindHost,
);
viteHmrServer = hmrServer;
}
}
if (opts.uiMode === "vite-dev") {
const uiRoot = path.resolve(__dirname, "../../ui");
const publicUiRoot = path.resolve(uiRoot, "public");
const hmrPort = resolveViteHmrPort(opts.serverPort);
const hmrHost = resolveViteHmrHost(opts.bindHost);
const hmrProtocol = resolveViteHmrProtocol(
process.env.PAPERCLIP_VITE_HMR_PROTOCOL,
);
const hmrServer = createHttpServer((_req, res) => {
res.writeHead(426, { "Content-Type": "text/plain" });
res.end("Upgrade Required");
});
const { createServer: createViteServer } = await import("vite");
const configuredViteCacheDir = process.env.PAPERCLIP_VITE_CACHE_DIR?.trim();
const vite = await createViteServer({
root: uiRoot,
...(configuredViteCacheDir
? { cacheDir: path.resolve(configuredViteCacheDir) }
: {}),
appType: "custom",
// Vite otherwise discovers every HTML entry below the UI root. Generated
// Storybook output can reference dependencies that are intentionally not
// part of the application install, poisoning a clean embedded dev-server
// cache before the browser opens. The embedded UI has one real entry.
optimizeDeps: { entries: [path.resolve(uiRoot, "index.html")] },
server: {
// Listener binding and browser HMR hostname are deliberately separate:
// exposed branch runtimes stay loopback-only while the browser uses the
// current MagicDNS hostname through the broker's HTTPS listener.
host: opts.bindHost,
middlewareMode: true,
hmr: {
server: hmrServer,
...(hmrHost ? { host: hmrHost } : {}),
...(hmrProtocol ? { protocol: hmrProtocol } : {}),
port: hmrPort,
clientPort: hmrPort,
},
allowedHosts: privateHostnameGateEnabled
? Array.from(privateHostnameAllowSet)
: undefined,
},
});
try {
await listenViteHmrServer(hmrServer, hmrPort, opts.bindHost);
} catch (error) {
await vite.close();
throw error;
}
viteDevServer = vite;
viteHmrServer = hmrServer;
viteHtmlRenderer = createCachedViteHtmlRenderer({
vite,
uiRoot,
brandHtml: applyUiBranding,
});
const renderViteHtml = viteHtmlRenderer;
if (fs.existsSync(publicUiRoot)) {
staticUi.use(staticUiFiles(publicUiRoot, { index: false }));
}
staticUi.get(/.*/, async (req, res, next) => {
if (!shouldServeViteDevHtml(req)) {
next();
return;
}
try {
const html = await renderViteHtml.render(req.originalUrl);
res.status(200).set({ "Content-Type": "text/html" }).end(html);
} catch (err) {
next(err);
}
});
staticUi.use(vite.middlewares);
}
app.use(staticUi);
app.use(errorHandler);
// A build whose asset names collide with different published bytes refuses
// to start, so the outcome is known before any job, queue, or run starts.
const retainedUiAssetsPublication = await (app.locals.retainedUiAssetsPublication as Promise<{ error: unknown } | null> | undefined);
if (retainedUiAssetsPublication) throw retainedUiAssetsPublication.error;
jobCoordinator.start();
const deliveryWork = createDeliveryWorkCoordinator({
owner: db,
canRun: () => !isWarmStandby() && !isIdleTaskDrainActive(),
canReconcile: () => !isWarmStandby(),
onError: (err, queue) => logger.error({ err, queue }, "Delivery reconciliation failed"),
});
app.locals.deliveryWork = deliveryWork;
await scheduler.start(deliveryWork);
const agentLifecycleDelivery = deliveryWork.register(DELIVERY_QUEUES.agentLifecycle, {
retryMs: 1_000,
run: () => agentLifecycle.sweep(),
hasPending: () => agentLifecycle.hasPending(),
nextRunAt: () => agentLifecycle.nextRunAt(),
});
agentLifecycle.setWake(agentLifecycleDelivery.wake);
const clientMirror = startMcpClientMirrorSync(db, deliveryWork, mcpConfig?.origin,{authorizationOrigin:mcpConfig?.authorizationOrigin});
const unregisterMcpOwners = registerIdleMcpOwners(db, { config: mcpConfig, personal: publicMcpEvents, dot: dotMcpEvents, clientMirrorInstanceId:clientMirror.instanceId });
startMcpRevocationIndexSync(db, deliveryWork);
startMcpIngressHandoffExpiry(db, deliveryWork);
publicMcpEvents?.start(deliveryWork);
dotMcpEvents?.start(deliveryWork);
if (publicMcpOAuth) grokDispatcher.start(deliveryWork);
const stopMuseMaintenance = startMuseMaintenance(db, deliveryWork, async () => {
const state = await connectionIntentHeartbeat.resolveSchedulingSuppression(true);
return !state.suppressed || state.reason === "task_drain";
});
const resourceLifecycleDelivery = registerPluginLifecycleDelivery(deliveryWork, db, workerManager, lifecycle);
await resourceLifecycleDelivery.ready;
if (opts.feedbackExportService) {
deliveryWork.register(DELIVERY_QUEUES.feedback, {
retryMs: FEEDBACK_EXPORT_FLUSH_INTERVAL_MS,
run: (signal) => opts.feedbackExportService!.flushPendingFeedbackTraces({ signal }),
hasPending: () => opts.feedbackExportService!.hasPendingFeedbackTraces(),
});
}
registerChatDeliveryWork(deliveryWork, chatChannels, () => !isIdleTaskDrainActive());
registerChatActionWork(deliveryWork, chatChannels, () => !isIdleTaskDrainActive());
chatChannels.voice.registerReportWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
chatChannels.voice.registerRecoveryWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
chatChannels.voice.registerReplyWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
chatChannels.voice.inbound.registerCleanupWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
chatChannels.voice.inbound.registerRetentionWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
chatChannels.registerReceiverWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
chatChannels.registerTeamsConsentExpiryWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
chatChannels.registerTeamsAttemptRecoveryWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
chatChannels.registerInteractionContinuationWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
registerGitHubReviewProjectionWork(db, deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
chatChannels.registerGitHubReviewCheckWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
chatChannels.registerGitHubTerminalResponseWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
chatChannels.registerGitHubAcknowledgementWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
chatChannels.registerTerminalConfirmationWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
chatChannels.registerCommandWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
nativeChatProgressWork(db).register(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
chatInboundWakeNoticeWork(chatChannels).register(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
chatFailedRetryNoticeWork(chatChannels).register(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
chatRunMilestoneWork(db, opts.authPublicBaseUrl).register(
deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive(),
);
emailChannels.start();
chatChannels.registerGitHubWebhookRecoveryWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
const importSpoolCleanup = registerImportSpoolCleanupWork(db, deliveryWork,
resolveDefaultImportTransferSpoolRoot(), () => !isWarmStandby() && !isIdleTaskDrainActive());
registerBrowserUseCleanup(deliveryWork, browserUse, () => !isIdleTaskDrainActive());
void companyTransferRunService
.recoverStrandedApplyingRuns(db)
.then((recovered) => {
if (recovered.length > 0) {
logger.warn(
{ count: recovered.length, runIds: recovered },
"failed company transfer runs stranded in applying by a restart",
);
}
})
.catch((err) => {
logger.error({ err }, "stranded company transfer apply recovery failed");
});
void toolDispatcher.initialize().catch((err) => {
logger.error({ err }, "Failed to initialize plugin tool dispatcher");
});
const devWatcher = managedAutoInstallKeys !== null ? null : createPluginDevWatcher(
lifecycle,
async (pluginId) =>
(await pluginRegistry.getById(pluginId))?.packagePath ?? null,
undefined,
{ unwatchedRoots: [bundledCatalogRoot] },
);
// Auto-provision bundled plugins so their providers are registered for
// agent runs. Bundles are excluded from the pnpm
// workspace and built standalone into the image (see Dockerfile), then
// installed here from their local paths. This runs BEFORE loadAll() so
// loadAll() can activate them in the same startup pass.
//
// Workers are started exactly once, by loadAll(): the `lifecycle` manager
// above is constructed without a runtime-capable loader
// (pluginLifecycleManager(db, { workerManager }) — no `loader` option), so
// the lifecycle.load() that ensureBundledPlugins performs per newly
// installed bundle only records the `ready` status and does not spawn a
// worker (see activateReadyPlugin in services/plugin-lifecycle.ts).
//
// Managed instances (`plugins.autoInstall` from PAPERCLIP_MANAGED_CONFIG)
// drive the key list from the control plane; self-hosted instances keep
// the pre-existing behavior of ensuring only the kubernetes bundle.
//
// Resolution is deliberately synchronous and NOT fail-safe: an
// unknown key or a path escaping the bundled catalog root throws out of
// createApp so a managed instance refuses to start (positive allowlist,
// fail closed).
// SAFETY: installation is fully fail-safe. Any failure
// (missing bundle, install error, load error) is caught, logged, and
// swallowed per plugin so the server ALWAYS finishes booting. A degraded
// boot (a provider unavailable, some agents cannot run) is strictly
// preferable to a crash loop.
//
// The chain is not awaited here (createApp stays fast), but the settled
// promise is exposed via `app.locals.bundledPluginsStartup` so boot steps
// that must not outrun plugin availability — managed sandbox environments
// (`applyManagedEnvironments`) run before the heartbeat resumes queued
// runs — can sequence on it. It never rejects.
const bundledPluginsStartup = ensureBundledPlugins(
bundledPluginInstalls,
{ registry: pluginRegistry, loader, lifecycle, logger },
// Managed mode reinstalls soft-uninstalled bundles (the control plane
// owns provisioning); self-hosted leaves an operator's uninstall alone.
// Disabled plugins never start automatically. Added distribution permissions
// still enter upgrade_pending so enabling them requires an operator decision.
{ reinstallUninstalled: managedAutoInstallKeys !== null, preservePackagePath: managedAutoInstallKeys !== null },
)
.then(() => loader.loadAll())
.then((result) => {
if (!result) return;
for (const loaded of result.results) {
if (devWatcher && loaded.success && loaded.plugin.packagePath) {
devWatcher.watch(loaded.plugin.id, loaded.plugin.packagePath);
}
}
})
.catch((err) => {
logger.error({ err }, "Failed to load ready plugins on startup");
});
app.locals.bundledPluginsStartup = trackIdleWork(bundledPluginsStartup);
// Manifest ids the managed auto-install list resolves to; the managed
// environments bootstrap checks plugin entries against them.
app.locals.managedBundledPluginKeys = managedBundledPluginKeys;
void bundledPluginsStartup.then(() => { lifecyclePluginsReady = true; resourceLifecycleDelivery.wake(); agentLifecycleDelivery.wake(); }).catch(() => logger.warn("Agent lifecycle recovery failed; retrying."));
// The shutdown hook runs at most once. It caches the in-flight promise, so a
// second caller (for example the `exit` handler) awaits the same completion
// instead of starting a second teardown.
let appServicesShutdown: Promise<void> | null = null;
const shutdownAppServices = (): Promise<void> => {
if (appServicesShutdown) return appServicesShutdown;
appServicesShutdown = finishServiceShutdown(() => scheduler.stop(), async () => {
await loader.stopActivationTimeoutRecovery();
await agentLifecycle.stop();
unregisterMcpOwners();
await publicMcpEvents?.stop();
await dotMcpEvents?.stop();
jobCoordinator.stop();
await finishServiceShutdown(() => resourceLifecycleDelivery.dispose(), async () => {
importSpoolCleanup.unsubscribe();
await deliveryWork.stop();
devWatcher?.close();
viteHtmlRenderer?.dispose();
void viteDevServer?.close().catch(() => undefined);
viteHmrServer?.close();
hostServiceCleanup.disposeAll();
hostServiceCleanup.teardown();
await stopMuseMaintenance();
await emailChannels.shutdown();
await chatChannels.shutdown();
// End the avatar worker pool, if a request ever started one, so no
// render outlives the HTTP teardown.
await agentAvatars.close();
// Cancel every live setup-token login session and AWAIT the cancellation,
// so each direct child stops and the server releases each lease before the
// caller stops the database and the provider. A lease release that
// fails stays a durable record for the startup reaper.
await setupTokenLoginService?.shutdown();
});
});
return appServicesShutdown;
};
app.locals.paperclipShutdown = shutdownAppServices;
// The `exit` event is synchronous. It cannot await the teardown, so it runs
// the best-effort cleanup and drops the returned promise. The orderly signal
// path awaits `shutdownAppServices` in full before the process exits.
process.once("exit", () => {
void shutdownAppServices();
});
process.once("beforeExit", () => {
void flushPluginLogBuffer();
});
trackIdleRequestHandlers(app);
return app;
}