Files
PaperClipAI/packages/shared/src/feature-catalog.ts
T
DottaandClaude Opus 5.5 944b1c64d7 feat(mcp): keep long provider calls durable with the MCP Tasks extension (#21263)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents use connected apps through Paperclip's MCP client in the tool
gateway
> - Some provider actions run for minutes: CI runs, deploys, batch jobs
> - Today such a call is lost if it outlives the tool timeout or if
Paperclip restarts or deploys mid-call, and a retry can start the work a
second time
> - Recovery must just work, and the target rate of runs that need a
manual retry is 0%
> - This pull request implements the MCP Tasks extension
(`io.modelcontextprotocol/tasks`, 2026-07-28) in the MCP client
> - The benefit is that a long provider call becomes a durable task: it
survives timeouts, restarts, and deploys, and it is never started twice

## Linked Issues or Issue Description

**Problem**
A long provider tool call holds an HTTP request open. When it passes the
tool timeout, or when Paperclip restarts or deploys during the call, the
result is lost. The agent then often repeats the call, and the provider
does the work again.

**Proposed solution**
Implement the client side of the MCP Tasks extension. Declare it on
stateless tool calls. Store each returned task durably, wait on it
during the call, keep polling it in the background after the call ends,
and let the same call attach to the stored task instead of sending
`tools/call` again.

**Why it matters**
CI, deploy, and batch actions are where agents lose the most work today.
A durable task handle makes these actions safe across timeouts,
restarts, and deploys, without a person.

Refs #21119, Refs #21140, Refs #21219 (the MCP Apps, stateless MCP
client, and MCP Skills work this builds on).

## What Changed

- **Off by default:** MCP Tasks are behind the new instance experimental
setting `enableMcpTasks` ("Durable app actions (MCP Tasks)"), default
off. When it is off, Paperclip never declares the extension, nothing
attaches to or polls tasks, pending rows do not block idle sleep, and
behavior matches master. A test covers the off state.

- `mcp-http.ts`: the tasks extension is declared per request, only on
HTTP `tools/call` and `tasks/*`. Discovery, listings, and stdio servers
do not declare it. `tasks/*` requests put the task ID in `Mcp-Name`.
- New `mcp-tasks.ts`: parses task results, bounds the server's timing
hints, sends `tasks/get`, `tasks/update`, and `tasks/cancel`, and seals
task IDs at rest (AES-256-GCM, bound to company and connection; a keyed
hash gives uniqueness).
- New company-scoped table `tool_mcp_tasks` (migration `0475`,
online-compatible). It links the task to the call, run, task,
connection, and grant. It stores no provider result.
- `tool-gateway.ts`:
- A returned task is recorded, then polled up to the normal tool
timeout. Its result goes through the same validation, redaction, and
recording as a direct result.
- Past the timeout, or when the caller disconnects, the agent gets
`provider_task_pending` ("call again with the same arguments"). The task
stays durable.
- A call with the same endpoint, tool, full arguments, grant, agent,
task, and actor attaches to the undelivered task. A retry with the same
idempotency key also attaches, and it can never start different work
under that key.
- Input: URL requests show the provider link (Paperclip never answers
them). Form requests become a question card; only the person's answer is
sent with `tasks/update`. Other requests stop with a clear "needs input"
error.
- Stopping the run sends `tasks/cancel` and marks the task cancelled. A
caller that only disconnects keeps the task.
- `failed` becomes a tool error with the provider's message. An unknown
task or a passed TTL becomes `provider_task_expired`. Every task stops
being tracked at its TTL, or after 7 days at most.
- Audit entries: `tool_gateway.task_started`, `task_input_sent`,
`task_cancelled`.
- New `mcp-task-work.ts`, registered at startup with the shared deadline
scheduler: it resumes polling pending tasks after a restart, cancels
tasks whose task closed or run was stopped, and wakes the assignee once
when a task settles after its run ended.
- Docs: `doc/connections/MCP-TASKS.md`, linked from
`doc/connections/README.md`.

Background work (per `rules/sleep-aware-background-work.mdc`): trigger
is a task row write or release (`mcp-tasks` queue) and one startup pass.
The deadline is the earliest `next_poll_at`. Cadence is the server's
`pollIntervalMs` within 5 s – 5 min, with backoff; 5 min while waiting
for a person. It is quiet when no row has a next poll. This is the
narrow external-source exception: providers give no wake Paperclip can
receive, and polling stops with the task.

## Verification

- `server/src/__tests__/mcp-tasks.test.ts`: 20 tests with a fake
2026-07-28 server that returns tasks only to a client that declares the
extension and checks `Mcp-Name`. They cover:
- polling to completion, with the result identical to a synchronous
call;
  - timeout, then an attaching call with no second `tools/call`;
  - a keyed retry, and a reused key with different arguments (refused);
- restart mid-task: a fresh gateway with no in-memory state polls the
row, wakes the agent once, and the recovered run gets the result once
with no second `tools/call`;
  - no wake while the run is still going;
- URL input (link, never answered), and form input (question card, then
`tasks/update` with the answer);
- cancellation by the run, by closing the task, and no cancellation when
only the caller leaves;
- failure, an unknown task, and TTL expiry (ends in under 3 s, no hang);
  - another agent never attaches;
- synchronous calls unchanged and older servers never see the extension.
- Red/green: with attach disabled, the timeout, keyed-retry, and restart
tests fail (3). With `tasks/cancel` disabled, both cancellation tests
fail (2). Before `tools/call` declared the extension, 13 end-to-end
tests failed. All pass with the change.
- Existing suites pass: `mcp-protocol-versions`, `mcp-http`, `mcp-apps`,
`mcp-skills`, `server-startup-feedback-export`, `tool-gateway`,
`tool-gateway-discovery-http`, `remote-mcp-connectors` (342 tests).
- `tsc --noEmit -p server` is clean. `pnpm --filter @paperclipai/db
check:migrations` passes (the migration is classified
online-compatible). Snapshots pruned to five.
- Agent boot and time-to-first-token: no change in the boot or
first-turn path. The only new per-call cost is one indexed lookup on
stateless tool calls.
- Not run: a live provider that returns tasks. No public MCP server that
returns tasks was available without new credentials.

## Risks

- Shipped off by default. Review findings that only affect the
setting-on path after this merge are tracked as follow-ups.

- Migration: a new table with indexes and foreign keys to `companies`,
`tool_invocations`, and `tool_connections`. Online-compatible.
- Behaviour change only for servers that return tasks, which they may do
only to clients that declare the extension. Older servers and stdio
servers are unchanged.
- A task that settles while nobody is waiting stays attachable for 1
hour, so a deliberate repeat of the same call within that hour collects
the earlier result instead of starting new work.
- The wake after a background task settles uses the generic task wake;
the agent must call the tool again to collect the result.
- Follow-ups: `notifications/tasks` via `subscriptions/listen`; sampling
and roots input requests; showing a long action's progress in the task
chat.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with
tool use (shell, file edits, local test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 08:32:06 -06:00

463 lines
18 KiB
TypeScript

import { z } from "zod";
import { instanceExperimentalSettingsSchema } from "./validators/instance.js";
/**
* Feature catalog for cloud-managed instances.
*
* The instance-settings zod schema is the feature manifest; this module adds
* only metadata about the flags the schema already declares. Keys are derived
* from the schema type, so adding, removing, or renaming a boolean flag in
* `instanceExperimentalSettingsSchema` without updating the metadata map is a
* compile error (and vice versa).
*
* Tiers:
* - `preference`: tenant-controllable taste setting; the cloud harness does
* not manage it.
* - `managed`: the cloud harness may set this per fleet/stack via
* `PAPERCLIP_MANAGED_CONFIG`.
* - `floor`: pinned by code on managed instances; no flag value may widen it.
*/
export const FEATURE_TIERS = ["preference", "managed", "floor"] as const;
export type FeatureTier = (typeof FEATURE_TIERS)[number];
type ExperimentalSettings = z.infer<typeof instanceExperimentalSettingsSchema>;
/**
* The boolean flag keys of the experimental settings schema. Non-flag keys
* (activation timestamps, numeric tuning values) are excluded.
*/
export type InstanceFeatureKey = {
[K in keyof ExperimentalSettings]: ExperimentalSettings[K] extends boolean ? K : never;
}[keyof ExperimentalSettings];
export interface FeatureCatalogEntry {
title: string;
description: string;
tier: FeatureTier;
/** Desired default on cloud-managed instances. */
cloudDefault: boolean;
/** Must match the schema default; enforced by test. */
selfHostedDefault: boolean;
}
export const INSTANCE_FEATURE_CATALOG: Record<InstanceFeatureKey, FeatureCatalogEntry> = {
enableEnvironments: {
title: "Environments",
description:
"Show environment management in company settings and allow project and agent environment assignment controls.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableAiConnectionRouters: {
title: "AI connection routers",
description: "Allow experimental plugin connections to choose task-pinned accounts and harnesses.",
tier: "managed", cloudDefault: false, selfHostedDefault: false,
},
enableNativeRunner: {
title: "Paperclip Runner",
description:
"Allow explicitly configured local Codex, OpenCode, and qualified ACPX agents to use the experimental Rust Paperclip Runner, including authenticated sandbox ingress when required. Onboarding remains on legacy adapters.",
tier: "managed",
cloudDefault: false,
// On by default for self-hosted instances. Requires a Rust toolchain (or
// PAPERCLIP_RUNNER_BINARY) for `pnpm dev`, which builds runnerd whenever
// this is on.
selfHostedDefault: true,
},
enableManagedSandboxOnly: {
title: "Managed Environment Only",
description:
"Hide the local environment and run all agents in the platform-managed environment.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableIsolatedWorkspaces: {
title: "Isolated Workspaces",
description:
"Show execution workspace controls in project configuration and allow isolated workspace behavior for task runs.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
remoteWorkspaceStorageEnabled: {
title: "Remote workspace storage",
description: "Keep eligible new native sandbox workspaces remote when durable storage is configured, and save verified snapshots in object storage.",
tier: "managed",
cloudDefault: true,
selfHostedDefault: true,
},
enableIsolatedWorkspacesByDefault: {
title: "Isolated Workspaces By Default",
description:
"Treat a project that has no execution workspace policy of its own as if it selected isolated workspaces, so its tasks get a per-task worktree instead of sharing the project checkout. Requires Isolated Workspaces. A project that carries its own policy keeps it.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableStreamlinedLeftNavigation: {
title: "Streamlined Left Navigation",
description: "Use the streamlined main sidebar navigation layout.",
tier: "preference",
cloudDefault: true,
selfHostedDefault: true,
},
enableStreamlinedUi: {
title: "Streamlined UI",
description:
"Use the streamlined application shell, shared task collections, focused task detail layout, contextual navigation, and simplified main sidebar.",
tier: "preference",
cloudDefault: true,
selfHostedDefault: true,
},
enableApps: {
title: "Apps (compatibility)",
description:
"Deprecated compatibility key. Apps is always enabled; stored and managed values are ignored.",
tier: "managed",
cloudDefault: true,
selfHostedDefault: true,
},
enablePublicMcp: {
title: "Assistant connections (MCP)",
description: "Let external assistants connect as a person to review work, delegate tasks, add feedback, and follow results.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableGitHubReviewBots: {
title: "GitHub review bots",
description: "Show GitHub review bot setup and management independently of chat connectors. Existing bots keep running when hidden; GitHub tool connections are unaffected.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableOpenAiDot: {
title: "OpenAI Dot",
description: "Add OpenAI Dot as a standalone agent choice with its own experimental opt-in. Requires Assistant connections (MCP) and an authenticated instance with a public HTTPS URL.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableGrokBot: {
title: "Grok Bot",
description: "Connect an existing Grok Bot as an experimental Paperclip agent. Requires Assistant connections (MCP), Paperclip Runner, and an authenticated public HTTPS instance. Provider spending is not reported.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableMuse: {
title: "Muse — Personal agent",
description: "Connect your personal Meta Muse through its managed background receiver and the external Paperclip Runner. Experimental; requires authenticated public HTTPS and Paperclip Runner.",
tier: "managed", cloudDefault: false, selfHostedDefault: false,
},
enableChatConnectors: {
title: "Chat connectors",
description:
"Show experimental chat connector setup and Board surfaces. Existing connections keep running when hidden; AgentMail, GitHub review bots, and tool connectors are unaffected.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableMemoryConnectors: {
title: "Memory connectors",
description: "Show experimental Mem0, Zep, Supermemory, Cognee, and Honcho setup. Existing connections keep running when hidden.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enablePaperclipNotebook: {
title: "Paperclip Notebook",
description: "Allow Paperclip Notebook connections and publishing through an explicitly enabled instance rollout.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableGoogleWorkspaceConnector: {
title: "Google Workspace connector (Paperclip)",
description: "Show and allow the built-in Google Workspace connector while its Google app verification is pending. When off, Google apps are offered through Composio and Arcade instead.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableMcpAggregators: {
title: "MCP aggregators (compatibility)",
description: "Deprecated compatibility key. MCP aggregators are always enabled; stored and managed values are ignored.",
tier: "managed",
cloudDefault: true,
selfHostedDefault: true,
},
enableCases: {
title: "Cases",
description:
"Durable work products that tasks create and iterate on. Adds the Cases tab and the agent case API.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableAgentChat: {
title: "Agent Chat (compatibility)",
description: "Deprecated compatibility key. Agent Chat is always enabled; stored and managed values are ignored.",
tier: "managed",
cloudDefault: true,
selfHostedDefault: true,
},
enableCombinedInboxTasks: {
title: "Combined Inbox + Task List",
description:
"Inbox becomes a set of views inside Tasks: one Tasks row in the nav carries the unread badge, and a Views menu reaches every inbox and task view.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableArtifactsV1: {
title: "Artifacts V1",
description:
"The artifact presentation from the artifact directions lab: paper stacks and tiles on the Artifacts page, compact rows in the panel's Artifacts tab, and grouped rows in the task thread.",
tier: "managed",
cloudDefault: true,
selfHostedDefault: true,
},
enableConferenceRoomChat: {
title: "Conference Room Chat",
description:
"Add the Conference Room team chat, the live activity feed, and the redesigned onboarding; restyles task threads as chat bubbles.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableClassicTaskInterface: {
title: "Classic Task Interface",
description:
"Restore the pre-chat task detail page: the page-level header with inline description editor, the plain comment thread, and the fixed Properties sidebar. Chat-only features (streaming activity folding, inline plan/question cards, the three-mode composer) are unavailable in the classic view.",
tier: "preference",
cloudDefault: false,
selfHostedDefault: false,
},
enableLegacyComposerSubmit: {
title: "Command/Ctrl+Enter to Submit",
description:
"Require Command/Ctrl+Enter to submit task replies and new tasks. Enter inserts a new line.",
tier: "preference",
cloudDefault: false,
selfHostedDefault: false,
},
enableIssuePlanDecompositions: {
title: "Task Plan Decomposition",
description: "Show accepted-plan decomposition history on task detail pages.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableExperimentalFileViewer: {
title: "Experimental File Viewer",
description:
"Show task detail controls for browsing and previewing workspace files relative to a task.",
tier: "preference",
cloudDefault: false,
selfHostedDefault: false,
},
enableStatusCards: {
title: "Status Cards",
description:
"Enable the experimental shared status-card board, update engine, and gated API.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableExternalObjects: {
title: "External Objects",
description:
"Detect external URLs in issues and show resolved status for pull requests, tickets, and other referenced work objects.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableSmokeLab: {
title: "Smoke Lab",
description:
"Add the Smoke Lab tab and dashboard card for exercising integration paths against deterministic local fixtures. Private deployments only.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableBuiltInAgents: {
title: "Built-in Agents",
description:
"Show Paperclip-managed built-in agent surfaces, including roster badges, the Built-in agents tab, and setup controls.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableBetaSkills: {
title: "Beta skills",
description: "Allow agents to pin beta releases of the Paperclip core skill.",
tier: "preference",
cloudDefault: false,
selfHostedDefault: false,
},
enableSummaries: {
title: "Summaries",
description:
"Show Summarizer-generated status slots on project and workspace pages, with on-demand refresh and revision history.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableDecisions: {
title: "Decisions",
description:
"Show the Decisions item in the main sidebar — the attention home that surfaces tasks awaiting input.",
tier: "preference",
cloudDefault: false,
selfHostedDefault: false,
},
enableFastResponses: {
title: "Experimental fast responses",
description:
"Allow short acknowledgements while agents begin work. Configure them in Settings → Connections.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableGoalsSidebarLink: {
title: "Goals Sidebar Link",
description: "Restore the Goals item in the main sidebar while the goals surface is being evaluated.",
tier: "preference",
cloudDefault: false,
selfHostedDefault: false,
},
enableSimplifiedEnglishInteractions: {
title: "Simplified English Interactions",
description:
"Instruct agents to write user interactions (confirmations, questions, suggested tasks) in ASD-STE100 Simplified Technical English with brief decision context.",
tier: "preference",
cloudDefault: false,
selfHostedDefault: false,
},
enableServerInfoDebugView: {
title: "Server Info Debug View",
description:
"Show a Server section in the account drawer with the current server restart time and running commit.",
tier: "preference",
cloudDefault: false,
selfHostedDefault: false,
},
enablePaperclipDeveloperMode: {
title: "Paperclip Developer Mode",
description:
"Show internal Paperclip maintainer tools and observability links, including Honeycomb trace queries on run pages.",
tier: "preference",
cloudDefault: false,
selfHostedDefault: false,
},
autoRestartDevServerWhenIdle: {
title: "Auto-Restart Dev Server When Idle",
description:
"In local development, wait for queued and running agent runs to finish, then restart the server automatically when backend changes make the current boot stale.",
tier: "preference",
cloudDefault: false,
selfHostedDefault: false,
},
enableWorkspaceBranchReconcileForward: {
title: "Workspace Branch Reconcile Forward",
description:
"Let execution workspaces reconcile a diverged recorded branch forward instead of failing branch containment.",
tier: "managed",
cloudDefault: true,
selfHostedDefault: true,
},
enableWorkspaceDirtyQuarantineRepair: {
title: "Workspace Dirty Quarantine Repair",
description:
"Let workspace runtime recovery quarantine and repair dirty execution workspaces before runs.",
tier: "managed",
cloudDefault: true,
selfHostedDefault: true,
},
enableOwnerInstanceAdmin: {
title: "Owner Instance Admin",
description:
"On cloud-managed instances, grant the stack owner instance-admin access to their own dedicated instance. Elevation is computed at the trusted-header auth boundary; no instance admin role rows are created. Inert on self-hosted instances.",
tier: "managed",
cloudDefault: true,
selfHostedDefault: false,
},
enableMcpTasks: {
title: "Durable app actions (MCP Tasks)",
description:
"Let connected apps run long actions, such as CI runs or deploys, as tasks that Paperclip keeps track of across timeouts, restarts, and deploys, so the work is never started twice.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableSandboxDuplexBridge: {
title: "Sandbox Duplex Bridge",
description:
"Let a run open the sandbox duplex command-stream bridge when the provider grants the capability. The host reads this per run before it selects the transport. Off keeps the file bridge for every run.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableRunnerPreviewIngress: {
title: "Runner Preview Ingress (Deprecated)",
description:
"Compatibility-only key retained for older managed configs. Runner ingress follows the Paperclip Runner setting.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableWorktreeRunExecution: {
title: "Worktree Run Execution",
description:
"Let the scheduler execute runs inside an isolated git-worktree preview instance for tasks created after activation.",
tier: "managed",
cloudDefault: false,
selfHostedDefault: false,
},
enableFirstTaskPlanProposal: {
title: "First task: propose with a plan document",
description:
"When the user's first request is a single task, the chief of staff writes a short plan document and a checkbox card instead of a one-card confirmation. Applies to organizations created after the toggle is flipped.",
tier: "preference",
cloudDefault: false,
selfHostedDefault: false,
},
};
export const INSTANCE_FEATURE_KEYS = Object.keys(INSTANCE_FEATURE_CATALOG).sort() as InstanceFeatureKey[];
/**
* Shape of the `feature-catalog.json` release artifact the cloud harness
* imports per app release and validates feature writes against.
*/
export const featureCatalogArtifactSchema = z
.object({
catalogVersion: z.string().min(1),
features: z.record(
z.string().min(1),
z.object({ tier: z.enum(FEATURE_TIERS) }).strict(),
),
})
.strict();
export type FeatureCatalogArtifact = z.infer<typeof featureCatalogArtifactSchema>;
export function buildFeatureCatalogArtifact(catalogVersion: string): FeatureCatalogArtifact {
if (catalogVersion.trim().length === 0) {
throw new Error("catalogVersion must be a non-empty string");
}
const features: FeatureCatalogArtifact["features"] = {};
for (const key of INSTANCE_FEATURE_KEYS) {
features[key] = { tier: INSTANCE_FEATURE_CATALOG[key].tier };
}
return { catalogVersion, features };
}
/** Deterministic serialization (sorted keys, trailing newline) for the artifact file. */
export function renderFeatureCatalogArtifact(catalogVersion: string): string {
return `${JSON.stringify(buildFeatureCatalogArtifact(catalogVersion), null, 2)}\n`;
}