mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-20 18:26:40 +01:00
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Assistant connections (MCP) let an outside assistant, or another
Paperclip instance, sign in as a person and use the `/mcp/paperclip`
endpoint
> - The feature was behind the experimental `enablePublicMcp` setting.
The setting was off by default and in practice off on Cloud stacks
> - Instance-to-instance work (one person's agent files a task on a
colleague's instance) fails when the receiving instance has the setting
off
> - This pull request moves the feature out of Experimental. It is now
always on, and a stored or Cloud-managed "off" value is ignored
> - The benefit is that every authenticated instance with a public URL,
including Cloud workspaces, accepts assistant connections with no setup
step
## Linked Issues or Issue Description
**Subsystem affected**
Public MCP (`server/src/services/public-mcp/`), instance settings,
Connectors → Assistant Connection (MCP) setup page, Experimental
settings page.
**Problem or motivation**
A remote MCP connector on one instance cannot use another instance's
`/mcp/paperclip` unless an admin on the other instance finds and turns
on an experimental toggle. On Cloud, the toggle is off and is managed by
the fleet. Instance-to-instance delegation therefore does not work by
default.
**Proposed solution**
Remove the gate. Keep `enablePublicMcp` only as a compatibility key, the
same way `enableApps` was graduated: old stored rows, PATCH bodies and
Cloud managed configs still parse, but the effective value is always
`true`.
**Alternatives considered**
An instance-admin kill switch that defaults ON. I did not keep one. Each
person already controls access: a connection exists only after that
person approves OAuth consent for one company, and the person or an
admin can revoke it. Deployment policy (authenticated mode and a valid
public URL) still decides whether the endpoint exists. A second switch
would add a state that can silently break instance-to-instance work
again. A schema migration was also not needed: the value lives in the
`experimental` JSON column, and the stored key becomes inert.
**Roadmap alignment**
Requested by the maintainer to enable instance-to-instance
communication.
## What Changed
- `packages/shared`: `enablePublicMcp` is a deprecated compatibility
key. The schema default is `true`. The feature catalog entry is
"Assistant connections (compatibility)" with both defaults `true`.
- `server` instance settings: normalization always returns
`enablePublicMcp: true`. The Cloud managed overlay skips the key, so a
fleet value of `false` cannot disable it. Removed the key from the Cloud
staging experimental-key list. Removed the client-mirror policy step
that ran only when the toggle changed.
- `server` public MCP: personal discovery, OAuth, tools, setup and event
delivery no longer read instance settings (`isEnabled()` returns `true`
for the personal endpoint without a DB read). Dot, Grok and Muse agent
connections keep their own experimental settings.
- `server` Cloud paths: the client-mirror sync, the hosted event
authority, the ingress-handoff expiry worker and the idle sleep proof
treat the front door as enabled. The warm-standby `503
workspace_unclaimed` behavior is unchanged.
- `server` Dot/Grok gates: they no longer also require
`enablePublicMcp`. Error messages no longer tell people to enable
Assistant connections.
- `ui`: removed the Experimental toggle, the "Enable assistant
connections / Open Experimental settings" block on the Assistant
Connection page, and the "Enable Assistant connections (MCP)" hint in
Dot setup. Updated the Dot and Grok copy.
- Storybook: deleted the Experimental-setting story and removed the
"enable the experiment" step from the journey and presenter.
- Docs: `doc/public-mcp.md` ("Enable an instance", "Start from
Connections", `GET /api/mcp/setup`), `doc/openai-dot-runner.md`,
`doc/idle-grok-dispatch.md`, and the agent-access map.
- Tests: new red/green tests for "enabled with no setting stored" and "a
stored or managed off value is ignored". Removed the tests for the
"disabled" behavior, which no longer exists.
## Verification
- Red/green: with the old source and the new tests,
`instance-settings-readonly.test.ts`,
`instance-settings-service.test.ts` and `public-mcp.test.ts` fail (4
failures, and the public-MCP suite fails in `beforeAll` because
`isEnabled()` was `false` with no setting stored). With the new source
they pass.
- `server`: `public-mcp.test.ts`, instance-settings (readonly, service,
managed overlay), `managed-config`, `cloud-event-authority`,
`events-work`, `dot-runner`, `dot-oauth-onboarding` and
`agent-runtime-assignments`. 354 passed.
- Cloud client-mirror lifecycle
(`client-mirror-lifecycle.postgres.test.ts`, skipped in CI) passed
against a local Postgres 17 with
`PAPERCLIP_MCP_SLEEP_TEST_DATABASE_URL`. 6/6, plus `events-work` 28/28.
- Idle and sleep: `idle-incoming-wake` (service and route),
`idle-deadline-proof`, `idle-incoming-wake-capability`,
`idle-sleep-proof`, `client-mirror-ack`, `recovery-deadline-acceptance`
and `idle-completed-history` pass. The warm-standby test now also
asserts that discovery after the claim does no SQL read.
- `ui`: `InstanceExperimentalSettings`, `AssistantConnection`,
`NewAgentDialog`, `NewAgent` and `ExternalAgentInviteDialog.muse` pass.
267 passed.
- Typecheck: `packages/shared`, `server` and `ui` pass. `node
scripts/check-maps.mjs` passes.
- `pnpm verify`: see the CI checks on this PR.
- After the deploy: confirm
`/.well-known/oauth-protected-resource/mcp/paperclip` and `/mcp/setup`
return 200 on paperclip.staging without changing any setting.
## Risks
- Behavior change: instances that had the setting off now serve
discovery, OAuth and `/mcp/paperclip`. Security invariants are
unchanged: the endpoint is still mounted only in authenticated mode with
a valid public URL, OAuth still needs a real signed-in user's consent,
local implicit board authority cannot approve, and revocation works the
same way.
- Cloud sleep: on a managed stack, the MCP sleep proof blocks sleep
until the client mirror publishes the tenant front door to Cloud. The
client-mirror sync runs at boot and does this once. Stacks that already
had the flag on use this path today. A stack whose Cloud connector
cannot publish the mirror would stay awake. Watch fleet sleep after the
deploy.
- Personal public-MCP requests now read instance settings one time less.
- Fleet admin: the Cloud fleet catalog can still show and set
`enablePublicMcp`. The value is ignored. Removing it from the fleet
catalog is a separate change in the Cloud repo.
- No migration. The `0354` trigger still compares the stored key. That
causes at most one extra connector-runtime invalidation if an old client
writes the key.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, test execution).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
122 lines
7.1 KiB
Markdown
122 lines
7.1 KiB
Markdown
# Idle Grok routine dispatch
|
||
|
||
The Grok dispatcher previously started a one-second timer whenever public MCP
|
||
was configured. Each tick read experimental settings, even when Grok was
|
||
otherwise disabled. Enabled ticks scanned prepared dispatches regardless of
|
||
whether any work existed. The disabled baseline regression observed three
|
||
settings queries in 3.2 seconds.
|
||
|
||
The dispatcher now registers with the existing delivery coordinator and shared
|
||
scheduler. It performs one startup reconciliation. The worker becomes quiet
|
||
when Grok is disabled or when no prepared dispatch remains. Quiet means no
|
||
recurring worker SQL or provider calls, including with retained terminal history.
|
||
This does not establish whole-instance or database suspension.
|
||
|
||
## Ownership and producers
|
||
|
||
`grokRoutineService.reserve` is the only supported creator of prepared dispatches.
|
||
It signals intent before its insert in the existing transaction. Route verification,
|
||
manual retry of a confirmed rejection, and native-broker assignment admission all
|
||
use this service. The native broker passes its transaction, so the wake follows
|
||
the outer commit. Savepoint release is not publication; rollback does not send.
|
||
|
||
`instanceSettingsService.updateExperimental` signals the same queue before its
|
||
existing transactional update. This covers enableNativeRunner
|
||
and enableGrokBot changes, including caller-owned outer transactions. It permits
|
||
prepared work retained while disabled to resume when enabled. Managed settings
|
||
and environment overlays are process configuration and are reconstructed on restart.
|
||
Other settings writers update general/default-environment fields, not these flags.
|
||
|
||
All other dispatch writers move rows away from prepared: send claims, delivery
|
||
settlement, acknowledgement, explicit recovery, assignment completion, and
|
||
revocation. None reopens a row as prepared. They cannot introduce hidden new work.
|
||
A previously scheduled backlog pass may observe cancellation and disarm.
|
||
|
||
Signals cover the current process's shared database owner and child pools.
|
||
Independent SQL writers or another process need explicit ingress integration;
|
||
this change does not claim cross-process notifications. Startup reconstructs
|
||
committed prepared rows from the existing dispatch table. No queue table is added.
|
||
|
||
## Delivery and recovery
|
||
|
||
The Grok service continues to own current authority checks, binding locks,
|
||
prepared-to-sending compare-and-set, secret access, provider delivery, and outcome
|
||
settlement. The worker does not interpret provider payloads or relocate policy
|
||
into app wiring. A foreground send and background recovery share the durable
|
||
claim; only the winner contacts the provider.
|
||
|
||
Each pass processes at most ten prepared rows. The next lookup uses the same
|
||
predicate and schedules the existing one-second cadence only while prepared work
|
||
remains, or after a reconciliation failure. New commits wake the shared worker;
|
||
its deadline adapter preserves requested generations through concurrent changes
|
||
and drain. The partial created_at/id index keeps both selection and pending-work
|
||
lookup independent of retained completed history. The migration builds it
|
||
concurrently, is replay-safe, and uses the normal snapshot-retention workflow.
|
||
|
||
The existing send-error policy is preserved: if sending throws and the row is
|
||
still prepared, the dispatcher cancels it. A row already claimed as sending is
|
||
never reset or replayed. Accepted, sending, unknown, acknowledged, rejected, and
|
||
cancelled rows do not become automatic retry work. Explicit operator retry still
|
||
creates a new intent only for confirmed rejection. This migration does not claim
|
||
exactly-once delivery or resolve uncertain provider outcomes automatically.
|
||
|
||
## Lifecycle and sleep boundary
|
||
|
||
The app registers only when public MCP is configured, after the shared coordinator
|
||
exists. Registration does not delay readiness for a provider send. The coordinator
|
||
tracks startup work and joins admitted sends on shutdown. Abort prevents the next
|
||
row from starting; existing HTTP timeouts bound a current transport operation.
|
||
Later prepared rows remain durable for restart.
|
||
|
||
Drain and warm standby retain pending requests without passive worker queries.
|
||
Resume uses the same coordinator retry path; a first write or failed sweep cannot
|
||
be forgotten merely because dispatch was paused. A disabled feature may retain
|
||
prepared rows without scheduling a scan; a committed settings change reactivates
|
||
it. Disabled feature state is not permission to discard durable work or relax
|
||
Cloud sleep guards. Existing binding/assignment/hold blockers remain unchanged.
|
||
|
||
## Verification
|
||
|
||
- The baseline failed the disabled-feature quiet assertion with three SQL reads.
|
||
- 21 real-PostgreSQL lifecycle cases cover disabled/configured idle, first real
|
||
reservation, prepared startup recovery, all six non-prepared states, re-enable,
|
||
outer commit/rollback, drain/startup, failed scan through drain, concurrent
|
||
enqueue, two-page draining, foreground claim overlap, expired unsent work,
|
||
settings transaction rollback/commit, and shutdown joining a send.
|
||
- The existing 45 Grok qualification cases and 39 startup cases pass together
|
||
with the new suite: 105 cases total.
|
||
- The four affected settings suites pass 154 cases (259 selected cases total).
|
||
- Workspace typecheck, production build, migration checks and replay pass. A
|
||
generic-plan probe with 40,000 completed dispatches and one prepared row uses
|
||
the partial index for both queries (six shared buffers including the routine
|
||
join; 0.032ms batch and 0.025ms next-work execution in this local probe).
|
||
- Remote PR review and CI must pass before the slice is ready.
|
||
|
||
The tests use the existing real OAuth/agent/routine fixture and a controlled
|
||
provider response. A validated disposable external PostgreSQL option permits
|
||
local execution where embedded PostgreSQL is unavailable; CI retains embedded
|
||
PostgreSQL by default. Provider-host suspension is outside this local proof.
|
||
|
||
|
||
## Current parent reconciliation
|
||
|
||
The normal merge of target parent `85ce6545bf` preserves its complete tree outside
|
||
this duty's explicit files. Grok dispatch, reservation, settings and app wiring
|
||
are unchanged from the preceding candidate. The index is regenerated as
|
||
`0391_third_roulette.sql`, with concurrent/idempotent execution. All 389 parent
|
||
journal entries are unchanged; five generated snapshots remain (0387–0391).
|
||
Schema drift, migration safety/numbering and fresh apply/replay on two isolated
|
||
PostgreSQL databases pass.
|
||
|
||
All 273 cases in nine complete suites pass at `6d81746fe2`: dispatch lifecycle,
|
||
Grok qualification, server startup, scoped deadlines and all five instance-settings
|
||
suites. Server typecheck and production build pass with the current Runner vendor.
|
||
The earlier 259-case and populated-plan measurements above remain historical
|
||
evidence; fresh exact-head reviews and CI are required.
|
||
|
||
An earlier browser CI run failed during agent-chat fixture restoration with a
|
||
lock timeout/HTTP 500; the subsequent case observed the experiment left enabled.
|
||
Those failures were not dispatcher assertions and are retained in the evidence.
|
||
The later pre-reconciliation `a74305e2b4` checks were terminal and green. Neither
|
||
that older result nor local acceptance substitutes for this candidate's CI.
|