Commit Graph
2 Commits
Author SHA1 Message Date
DottaandClaude Opus 5.5 904c01d1a3 feat(mcp): graduate Assistant connections (MCP) out of experimental (#22015)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Assistant connections (MCP) let an outside assistant, or another
Paperclip instance, sign in as a person and use the `/mcp/paperclip`
endpoint
> - The feature was behind the experimental `enablePublicMcp` setting.
The setting was off by default and in practice off on Cloud stacks
> - Instance-to-instance work (one person's agent files a task on a
colleague's instance) fails when the receiving instance has the setting
off
> - This pull request moves the feature out of Experimental. It is now
always on, and a stored or Cloud-managed "off" value is ignored
> - The benefit is that every authenticated instance with a public URL,
including Cloud workspaces, accepts assistant connections with no setup
step

## Linked Issues or Issue Description

**Subsystem affected**
Public MCP (`server/src/services/public-mcp/`), instance settings,
Connectors → Assistant Connection (MCP) setup page, Experimental
settings page.

**Problem or motivation**
A remote MCP connector on one instance cannot use another instance's
`/mcp/paperclip` unless an admin on the other instance finds and turns
on an experimental toggle. On Cloud, the toggle is off and is managed by
the fleet. Instance-to-instance delegation therefore does not work by
default.

**Proposed solution**
Remove the gate. Keep `enablePublicMcp` only as a compatibility key, the
same way `enableApps` was graduated: old stored rows, PATCH bodies and
Cloud managed configs still parse, but the effective value is always
`true`.

**Alternatives considered**
An instance-admin kill switch that defaults ON. I did not keep one. Each
person already controls access: a connection exists only after that
person approves OAuth consent for one company, and the person or an
admin can revoke it. Deployment policy (authenticated mode and a valid
public URL) still decides whether the endpoint exists. A second switch
would add a state that can silently break instance-to-instance work
again. A schema migration was also not needed: the value lives in the
`experimental` JSON column, and the stored key becomes inert.

**Roadmap alignment**
Requested by the maintainer to enable instance-to-instance
communication.

## What Changed

- `packages/shared`: `enablePublicMcp` is a deprecated compatibility
key. The schema default is `true`. The feature catalog entry is
"Assistant connections (compatibility)" with both defaults `true`.
- `server` instance settings: normalization always returns
`enablePublicMcp: true`. The Cloud managed overlay skips the key, so a
fleet value of `false` cannot disable it. Removed the key from the Cloud
staging experimental-key list. Removed the client-mirror policy step
that ran only when the toggle changed.
- `server` public MCP: personal discovery, OAuth, tools, setup and event
delivery no longer read instance settings (`isEnabled()` returns `true`
for the personal endpoint without a DB read). Dot, Grok and Muse agent
connections keep their own experimental settings.
- `server` Cloud paths: the client-mirror sync, the hosted event
authority, the ingress-handoff expiry worker and the idle sleep proof
treat the front door as enabled. The warm-standby `503
workspace_unclaimed` behavior is unchanged.
- `server` Dot/Grok gates: they no longer also require
`enablePublicMcp`. Error messages no longer tell people to enable
Assistant connections.
- `ui`: removed the Experimental toggle, the "Enable assistant
connections / Open Experimental settings" block on the Assistant
Connection page, and the "Enable Assistant connections (MCP)" hint in
Dot setup. Updated the Dot and Grok copy.
- Storybook: deleted the Experimental-setting story and removed the
"enable the experiment" step from the journey and presenter.
- Docs: `doc/public-mcp.md` ("Enable an instance", "Start from
Connections", `GET /api/mcp/setup`), `doc/openai-dot-runner.md`,
`doc/idle-grok-dispatch.md`, and the agent-access map.
- Tests: new red/green tests for "enabled with no setting stored" and "a
stored or managed off value is ignored". Removed the tests for the
"disabled" behavior, which no longer exists.

## Verification

- Red/green: with the old source and the new tests,
`instance-settings-readonly.test.ts`,
`instance-settings-service.test.ts` and `public-mcp.test.ts` fail (4
failures, and the public-MCP suite fails in `beforeAll` because
`isEnabled()` was `false` with no setting stored). With the new source
they pass.
- `server`: `public-mcp.test.ts`, instance-settings (readonly, service,
managed overlay), `managed-config`, `cloud-event-authority`,
`events-work`, `dot-runner`, `dot-oauth-onboarding` and
`agent-runtime-assignments`. 354 passed.
- Cloud client-mirror lifecycle
(`client-mirror-lifecycle.postgres.test.ts`, skipped in CI) passed
against a local Postgres 17 with
`PAPERCLIP_MCP_SLEEP_TEST_DATABASE_URL`. 6/6, plus `events-work` 28/28.
- Idle and sleep: `idle-incoming-wake` (service and route),
`idle-deadline-proof`, `idle-incoming-wake-capability`,
`idle-sleep-proof`, `client-mirror-ack`, `recovery-deadline-acceptance`
and `idle-completed-history` pass. The warm-standby test now also
asserts that discovery after the claim does no SQL read.
- `ui`: `InstanceExperimentalSettings`, `AssistantConnection`,
`NewAgentDialog`, `NewAgent` and `ExternalAgentInviteDialog.muse` pass.
267 passed.
- Typecheck: `packages/shared`, `server` and `ui` pass. `node
scripts/check-maps.mjs` passes.
- `pnpm verify`: see the CI checks on this PR.
- After the deploy: confirm
`/.well-known/oauth-protected-resource/mcp/paperclip` and `/mcp/setup`
return 200 on paperclip.staging without changing any setting.

## Risks

- Behavior change: instances that had the setting off now serve
discovery, OAuth and `/mcp/paperclip`. Security invariants are
unchanged: the endpoint is still mounted only in authenticated mode with
a valid public URL, OAuth still needs a real signed-in user's consent,
local implicit board authority cannot approve, and revocation works the
same way.
- Cloud sleep: on a managed stack, the MCP sleep proof blocks sleep
until the client mirror publishes the tenant front door to Cloud. The
client-mirror sync runs at boot and does this once. Stacks that already
had the flag on use this path today. A stack whose Cloud connector
cannot publish the mirror would stay awake. Watch fleet sleep after the
deploy.
- Personal public-MCP requests now read instance settings one time less.
- Fleet admin: the Cloud fleet catalog can still show and set
`enablePublicMcp`. The value is ignored. Removing it from the fleet
catalog is a separate change in the Cloud repo.
- No migration. The `0354` trigger still compares the stored key. That
causes at most one extra connector-runtime invalidation if an old client
writes the key.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 11:42:36 -06:00
DottaandPaperclip 2adcde162e fix(server): wake Grok dispatch from committed reservations (#17490)
## Thinking Path

> - Paperclip manages agent work and should become idle when no work is
owed.
> - Grok routine dispatch already persists each outbound intent before
delivery.
> - Its dispatcher starts a one-second timer whenever public MCP is
configured.
> - That timer queries settings even when Grok is disabled, and scans
dispatches when enabled but idle.
> - The reservation transaction and feature-setting transaction already
define when work can become available.
> - This PR connects those writes to the shared coordinator and retains
the Grok service's claim and delivery policy.
> - The worker becomes quiet without adding a queue table or replaying
uncertain sends.

## Linked Issues or Issue Description

Stacked on #17486. This PR owns only Grok routine dispatch. It does not
include decision retention or other remaining maintenance owners.
Related open Grok PR #17278 concerns Runner credential-lease fixtures,
not this dispatcher. The duplicate search found no overlapping
dispatcher migration.

**What existing behavior does this improve?**
Grok routine dispatch recovery in the server.

**Current behavior**
With public MCP configured, the dispatcher reads instance settings every
second, including when Grok is disabled. With Grok enabled, it scans for
prepared dispatches every second. The baseline regression observed three
queries in 3.2 seconds with Grok disabled.

**Proposed behavior**
Reconcile once at startup. Wake from committed dispatch reservations and
experimental-setting changes. Schedule another pass only while prepared
dispatches remain or reconciliation failed. Disabled and settled workers
perform no recurring SQL or provider calls.

**Reason and benefit**
Remove unnecessary tenant database activity while preserving dispatch
recovery, current authorization, provider outcome semantics, and
shutdown ownership. This follows the existing sleep-aware maintenance
direction; ROADMAP review found no competing dispatcher redesign.

**Breaking changes**
No public API or provider payload change. The internal dispatcher start
method receives the shared coordinator; that coordinator now owns
shutdown. Sending, accepted, unknown, and terminal dispatches remain
non-replayable. This does not extend Cloud wake coverage.

## What Changed

- Register the Grok worker with the existing coordinator; remove its
independent timer.
- Signal reservation and experimental-setting writes before commit.
Nested writes follow their outer transaction.
- Preserve ten-row batches, durable send claims, current authority
checks, and existing cancellation/uncertainty policy.
- Join admitted work during shutdown and leave later rows for restart.
- Add a concurrent, replay-safe partial index for prepared dispatch
lookup. Keep only the five newest generated snapshots.
- Reuse the existing real OAuth/agent/routine fixture for lifecycle
tests and add a validated external PostgreSQL test option.
- Document producers, recovery, quiet conditions, and remaining
host-sleep limitations.

## Verification

Test revision: `6d81746fe260ee879442e7e0c879cfe88c250c8e`, followed by
docs-only head `679f523d78`, which received both 5/5 reviews and passing
CI. This candidate normally integrates landed target `8dfed893d8`. Its
entire runtime and acceptance-test sources match that reviewed/tested
tree; only four independently merged Runner E2E accounting files and the
ledger differ, preserved exactly. Migration 0391 and the parent journal
prefix are unchanged. Local checks were not repeated on unchanged
runtime; fresh final-head CI and reviews are required.

- **273 cases pass** in nine complete suites: real-PostgreSQL dispatch
lifecycle, Grok qualification, server startup, scoped deadlines and all
five instance-settings suites. Server typecheck and production build
pass with the updated Runner vendor.
- The baseline disabled-feature regression failed with three SQL reads.
Existing lifecycle cases cover first reservation, restart,
non-replayable history, re-enable, outer commit/rollback, drain,
failure/retry, concurrent enqueue, paging, foreground send race, expiry,
settings commit/rollback, and shutdown.
- Generated migration `0391_third_roulette.sql` adds only the
concurrent/idempotent prepared-dispatch index. All 389 parent journal
entries are unchanged; five generated snapshots remain (0387–0391).
Migration safety/numbering, schema drift and actual fresh apply/replay
on two isolated PostgreSQL databases pass.
- The earlier 259-case selection and 40,000-row generic-plan probe
remain historical evidence: both unchanged queries used the partial
index, six shared buffers including the routine join, 0.032ms / 0.025ms
local execution. No new deployed suspension or cost claim is made.
- Earlier browser CI failed during agent-chat fixture restoration (lock
timeout/HTTP 500), followed by a test seeing the experiment left
enabled. Those failures remain recorded in the owning audit; they were
not dispatcher assertions. Later pre-reconciliation `a74305e2b4` checks
were terminal and green. Fresh CI and both 5/5 bot reviews are required
for this head.

Commands: server Vitest for `grok-dispatch-work`, `grok-bot-routine`,
`server-startup-feedback-export`, `scoped-deadline-work` and the five
`instance-settings-*` suites; server typecheck/build; DB migration
safety/numbering, snapshot drift and fresh migration apply/replay.

## Risks

- Signals are local to the shared database owner and its child pools.
Independent SQL writers and other replicas need explicit wake
integration. Startup recovers committed prepared rows.
- Disabled Grok retains prepared rows and resumes after a committed
settings change. This does not relax Cloud's binding/assignment/hold
guards.
- Existing send-error cancellation policy is preserved. Sending or
unknown outcomes are not reset or automatically replayed; this is not an
exactly-once delivery change.
- Shutdown awaits the current send within existing transport timeouts
and stops before the next row.
- The index adds write/storage cost but bounds queries independently of
completed history. Existing migration execution runs concurrent index
creation outside a transaction.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning and code execution. The exact
serving model identifier and context window were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 19:07:53 -05:00