2 Commits

Include renames

Author	SHA1	Message	Date
DottaandPaperclip	0726a00e80	feat(apps): qualify Bitly, Context7, Hugging Face and Resend MCP (#15686) ... ## Thinking Path > - Paperclip lets people manage AI agents and govern their tools. > - Apps already supports provider-hosted MCP servers. > - Useful providers still need real setup and read proof before Browse exposure. > - Browser testing found a Bitly token-format failure and an OAuth callback that replaced an explicit empty agent selection. > - This PR qualifies four provider methods and fixes those shared OAuth paths. > - Providers that still need login or approval remain withheld from new Browse listings. ## Linked Issues or Issue Description Refs: #15621, the merged hosted MCP qualification batch. Searches found no duplicate Bitly or Context7 implementation PR. **What existing behavior does this improve?** Apps setup, provider MCP discovery, token exchange, and saved agent access through an OAuth callback. **Subsystem affected** Shared Apps definitions, the server tool-access service, and the UI setup/recovery route. **Current behavior** Bitly's callback can fail with `oauth_access_token_missing` when its token response is not JSON. Curated MCP callbacks can treat an explicitly saved empty install set as access for all agents. Context7 is hidden and Hugging Face lacks a curated anonymous method. Saved drafts for hidden providers cannot finish setup through their catalog wizard. **Proposed behavior** Request JSON token responses. Record explicit remote MCP install choices in the install transaction and preserve them at callback. Keep managed subject/default access when no choice was saved. Expose the qualified Bitly and Context7 methods, add anonymous Hugging Face Hub access, and record Resend qualification. Resume hidden providers only through an exact matching company-visible saved draft. **Reason and benefit** People can connect useful hosted tools through the existing Apps flow. An explicit zero-agent selection survives authorization. Public documentation lookups need no OAuth app, account, or key. **Breaking changes** Explicitly saved empty MCP install choices now remain empty. Existing managed defaults without a saved choice retain their behavior. No schema migration or API shape change. ## What Changed - Qualify Bitly OAuth, Context7 public MCP, Hugging Face public MCP, and Resend OAuth with real browser setup, useful UI reads, and reload checks. - Negotiate JSON OAuth token responses and preserve explicitly saved MCP agent reach with a transactional config marker. - Add durable provider overrides, permission reviews, dated text-only evidence, and official branding provenance. Remove all seven evidence images and their report references. - Restore Finish setup for hidden-provider drafts with company/application/provider checks; keep fresh setup withheld. Preserve success after the exact resumed draft becomes active, and restore valid completed URLs to that exact saved Permissions page; reject initially active, disabled, or archived resume targets. Align the Context7 route assertion with its new qualified availability. - Retain hidden ClickUp, Klaviyo, and monday.com research with exact login/approval gates. monday.com requires provider callback allowlisting. ## Verification - Connector service suite: 420 passed, including default/empty/agent/company reach and managed OAuth lifecycle coverage. - Shared catalog: 36 passed. Override tests: 3 passed. Brand checks: 101 identities passed. - Focused UI setup/recovery, Browse, and OAuth-helper suites: 306 passed. Actual hidden-draft resume/reload passed; a public Context7 draft seeded in the isolated app completed through the active-state refetch and persisted with zero agents after reload. The completed URL refresh and View connection open the exact saved Permissions page. Fresh hidden setup, normal initially active resume, disabled/archived status, and provider mismatch are rejected; a draft at the completion stage stays in setup. - Recursive typecheck and build passed again after rebase. The full local Vitest run began before rebase and was cancelled as superseded while still in the general-server phase; it is not a completed pass. Exact-head CI supplies broad coverage. The focused connector service suite passed against the final code. UI typecheck, build, and token gates passed again after the draft-recovery fix. - Targeted mock MCP Playwright journey: 1 passed (34.9s). It now verifies View connection reaches the exact saved Permissions route, both discovered actions, and reload persistence. This replaces the stale Browse expectation found by CI. - All CI gates passed on final commit `32bb4c6422411eee632486950e45e2735102d1b3`. The inherited-pause test timed out in the cleanup run and passed on one retry without code changes; the entire file also passed locally, 36/36. Its exact timing cause remains unresolved. A signoff timeout on the preceding production-code commit also passed on its single retry. Greptile reviewed the final commit at 5/5 with no open findings. - Actual UI reads: Bitly current-user read; Resend List Domains including a fresh setup after the final callback refinement; Context7 React docs; Hugging Face public docs search. No provider writes or real agent runtimes ran. - [Text-only qualification report](https://github.com/paperclipai/paperclip/blob/32bb4c6422411eee632486950e45e2735102d1b3/doc/connections/easy-mcp-batch-2-2026-10-09.md) include evidence limits and exact blockers. Account/profile/domain results are omitted. ## Risks - Bitly `mcp.all` and Resend `full_access` remain broad provider grants. Qualification kept writes Off and selected no agents. Read proof does not establish write behavior or refresh rotation. - The explicit-choice marker applies to new install saves; older unmarked flows keep their existing defaults. Its JSONB merge avoids overwriting concurrent config fields. - Public provider catalogs and rate limits can change. Unqualified providers remain withheld from new Browse entries; retained connections remain available for recovery. ## Model Used OpenAI Codex, based on GPT-6, plus OpenAI `gpt-6-luna` subagents for provider research, browser testing, and review. The exact primary serving variant and context-window size are not exposed by this runtime. Used reasoning, code execution, and browser tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-09 11:52:33 -05:00
DottaandPaperclip	2b688a8fe3	feat(connections): add verified MCP providers and setup fixes (#15621) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Apps give agents governed access to external tools. > - Several provider MCP servers lacked a supported catalog entry or failed during setup. > - Real browser tests identified specific registration, session, and form defects. > - This pull request adds seven catalog entries and fixes the shared paths used by eleven verified providers. > - Users can connect these providers through the existing Apps flow and control each action. ## Linked Issues or Issue Description **What existing behavior does this improve?** The Apps catalog and remote MCP setup, discovery, and action tester. **Subsystem affected** Shared app definitions, the server tool services, and the Apps UI. **Current behavior** Seven providers lack a catalog entry. Airtable can select an advertised client-metadata flow that fails. Calendly rejects the registration name. Tavily needs an initialized session before a call. Firecrawl exposes invalid defaults for optional nested form objects. HTTPS setup can display a callback that differs from the server callback. **Proposed behavior** Add Calendly, Exa, Firecrawl, GSC Wizard, Parallel Search, Tavily, and Windsor.ai. Preserve Airtable, Linear, Make, and PostHog. Use reviewed provider options through the shared MCP and OAuth paths. Display the callback supplied by the server. Omit untouched optional object inputs. **Reason and benefit** Eleven providers passed bounded reads through the real Paperclip browser action tester. This PR includes that verified set and its required shared fixes. Unfinished providers remain outside this change. AgentMail keeps its existing integration. **Breaking changes** No database migration. Existing OAuth credentials and action policies keep their ownership and access rules. Airtable's reviewed method re-registers a retained client-metadata binding through DCR. Tavily's reviewed method initializes sessions before dispatch. The existing customer, managed, and Vercel OAuth gateway paths now refresh on upstream 401 and return `oauth_refreshed_retry_required` (409), requiring an explicit caller retry. They do not replay the rejected call automatically; the next invocation initializes a fresh credential-scoped session when required. Related catalog work: #15545. This branch preserves current master catalog entries and does not add another Google Workspace integration. Open and closed provider PRs and public MCP issues were searched. No duplicate for this verified set was found. The change extends the shipped Connected Apps roadmap item. ## What Changed - Add seven catalog entries with official provider branding and reviewed permissions. - Update Airtable, Linear, and Make metadata and show Make in Apps. - Add authoritative provider source overrides that use the existing generation and review checks. - Add the reviewed Airtable DCR option and compatible Calendly registration names. - Initialize Tavily sessions before discovery and calls, including retained connections. Keep credential-scoped session caching and single call dispatch. - Use the server's callback URL in the OAuth setup UI. - Omit untouched optional object defaults; keep supplied empty strings, false, zero, and object values intact in the action tester, with strict required-child validation. - Require an explicit retry after OAuth refresh instead of replaying a tools/call with missing session headers. - Record all eleven successful reads, write policies, auth-method limits, and qualification caveats. Omit credentials and private account data. - Find chat connector cards through catalog search in browser tests, so pagination does not hide Slack or other later entries. ## Verification - Real browser qualification: eleven bounded reads passed. Catalog refresh and reload proof are recorded in `doc/connections/verified-mcp-qualification-2026-10-08.md`. - Provider writes and actual agent-adapter sessions were not run. Alternative documented auth methods and expiry refresh remain unqualified. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - Focused shared/catalog tests: 41 passed. Source override tests: 3 passed. Catalog regeneration fixture suite: 9 passed. Latest form regression suites: 31 passed. Gateway suite: 38 passed. Other focused UI suites passed before these last fixes. - UI token gates and both token sync checks: passed. - The catalog regeneration fixture includes the authoritative provider overrides; its full suite passes. The focused OAuth socket case passed on isolated rerun. - Complete local UI suite: 7,866 passed. CLI suite: 511 passed, 6 skipped. Complete shared suite: 892 passed. - The unchanged AgentMail/ClickUp discovery fallback suite passes all 45 cases. An OpenAI login test passed on isolated rerun. - Local broad database coverage is limited by macOS PostgreSQL shared-memory exhaustion (`shmget: No space left on device`, not disk space). The broad run was interrupted after diagnosis; no host settings or other running services were changed. - CI found that the Slack browser test assumed its catalog card was on the first page. The test now uses catalog search; the Slack case passes locally. Adjacent GitHub and iMessage cases could not start locally because embedded PostgreSQL initialization failed before browser assertions. - Final commit [`febe4520e`](https://github.com/paperclipai/paperclip/commit/febe4520e13d4b3a4121eafc3d22540c2b11379d): all 54 checks passed, with none pending or failed. [CI run](https://github.com/paperclipai/paperclip/actions/runs/37847182065) passed typecheck, build, all general and serialized test suites, all eight browser shards, runner checks, canary dry run, and the aggregate verification gate. - Greptile reviewed that exact final commit at 2026-10-08 21:33 UTC and returned 5/5 with no outstanding findings or unresolved threads. - PR UI preview against the existing local test server: Calendly catalog/setup observed; Firecrawl read passed after expanding More options with nested optional inputs untouched. This retest proves frontend behavior, not the revised OAuth backend against a live provider. <details> <summary>Browser evidence</summary>    </details> ## Risks - Provider registration and consent behavior can change. Airtable's DCR option is explicit and keeps the existing issuer, resource, redirect, and PKCE checks. - Tavily uses initialized sessions without automatic call retries. After a successful OAuth refresh following 401, callers now receive a retry-required result. A later explicit retry can still require reauthorization if the provider rejects the refreshed token. Provider writes are not live-qualified. - Optional object cleanup affects the shared action tester. Regression tests cover absent objects, required children, defaults, and populated values. - GSC Wizard's underlying Google data scopes and Google flow completion were not independently verified. Its account reported paid/trial metadata of unknown origin. No purchase was performed. - No database migration, new AgentMail integration, or change to existing action grants is included. ## Model Used OpenAI GPT-6 through Codex assisted with implementation, research, tool use, and code execution. The root backend model ID and context window are not exposed in this session. The cheaper subagents used OpenAI `gpt-6-luna`. No model context size is inferred. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass — targeted checks and complete UI/CLI/shared suites passed; the broad local database run was blocked by the macOS PostgreSQL startup limitation documented above. The full database/workspace suite passed in CI. - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-09 08:10:46 -05:00