mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-06 10:48:12 +02:00
feat(connections): connect services from native task feeds (#13058)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents use connections to reach external services. > - A fresh native task can have no service tools installed. > - The agent needs a way to discover services and ask the responsible person for access. > - This pull request brings the existing connection-intent flow into native task execution. > - The person can connect from the task, and the agent can continue with updated tools. ## Linked Issues or Issue Description **Subsystem affected** Native runner tool authority, connection intents, task interactions, and shared connection setup. **Problem or motivation** A task that needs an unconnected service cannot finish its work. Leaving the task to configure access also loses context. A resolved request must survive a restart and resume the correct agent once. **Proposed solution** Expose connection discovery and access requests as server-owned native tools. Render a durable task card and use the shared setup dialog. Persist outcome delivery and start a fresh provider session after access is ready. **Alternatives considered** Sending the person to the Connections page adds navigation and does not solve continuation. Polling for authorization consumes runs and can create duplicate requests. **Roadmap alignment** This extends the existing connection-intent runtime and setup experience. It reuses the shared access model and the native runner. Related: #12345, #12347. The service-slug fix in #12906 is related but separate. Companion evaluation PR: https://github.com/paperclipai/paperclip-evals/pull/21. ## What Changed - Expose `connections_search` and `connection_request` with server-bound company, task, agent, and responsible user. Preserve the legacy entry points. - Discover catalog services and authorized custom connections. Check installation, identity, health, and executable permissions before reporting ready. - Keep pending cards through ordinary messages. Reuse requests and retire stale ownership. Put Connect at the right of Not now. - Reuse the shared setup flow in a task dialog. Keep access additive and default to the requesting agent. Recover from cancelled or blocked OAuth windows with a new-tab fallback. - Persist outcome delivery with an idempotent wake key. Resume in a fresh session and recheck ownership before dispatch. - Add native browser fixtures, offline Storybook states, server contracts, and evaluation fixtures. Update guidance and documentation. ## Verification - `pnpm build`: passed after replaying the change on current master. - `pnpm -r typecheck`: passed. - `pnpm check:token-gates`: passed. - `pnpm --filter @paperclipai/ui build-storybook`: passed. - New continuation-policy regression cases: 16 passed. - Docker-backed PostgreSQL regressions passed for requester-only OAuth access, assignment-only expiry, terminal expiry, and credential-free setup metadata. - Shared setup and task-card UI tests: 121 passed, including configured MCP reconnect URL recovery and preserving user edits across refetch. - Storybook browser checks: all 119 passed on the latest reconnect fix. - `pnpm test:run`: 4,734 tests passed in the first server group, but embedded PostgreSQL startup failures and resulting cleanup errors prevented a complete local pass. All Linux CI lanes passed on the latest reviewed commit. One external-object route test returned an unexplained 500 on the first run; it passed twice locally and the failed shard passed on retry without code changes. - Earlier feature-checkout evidence: three deterministic native browser journeys passed, including restart delivery and an actual fixture tool result. Legacy scripted coverage also passed. All 59 added stories were inspected in light and dark themes. - Live Notion testing recorded successful provider reads. The manual test used a local-trusted instance. It does not prove authenticated/cloud deployment or every provider journey. - Native browser rerun reached the embedded PostgreSQL startup limit before bootstrap, so the latest checkout’s full native browser journey remains unverified. Both OAuth page/task regression cases passed against isolated Docker-backed PostgreSQL 17. They verify no premature task access, requester-only completion, additive retries, and reconnect preservation. - Applied both new migrations twice to isolated PostgreSQL 17. Foreign keys remained intact, duplicate active delivery keys were rejected, and failed delivery records did not block retries. Reviewer path: start a fresh test drive, enable the native runner, use an agent that can perform work directly, and ask it to summarize a Notion page. Connect from the card, then verify the resumed provider call and source-linked answer. The default test-drive CEO is instructed to delegate, so it can introduce an unrelated hiring step. ## Risks - Two additive migrations create durable deliveries and a partial unique wake index. They are idempotent. The wake index can require a maintenance window on large tables because migrations run in a transaction. - OAuth and continuation cross asynchronous boundaries. Tests cover ownership changes, retries, additive access, and restart delivery; live provider behavior still varies. - The latest requester-scope fix has not yet been exercised through live OAuth. GitHub, API-key, authenticated-user, and all recovery journeys are not claimed as verified. ## Model Used OpenAI GPT-6-based Codex assisted with implementation, tests, and review using tools and code execution. The runtime does not expose the exact model version, context window, or reasoning setting. Live evaluation used `gpt-5.6-luna`; manual native testing used `gpt-5.6-sol`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used and disclosed unavailable runtime details - [x] I have checked ROADMAP.md and confirmed this extends existing connection work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the feature issue template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [ ] I have run all required tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
5128b4f323
commit
e095b84dab
67 files changed
+88330
-327
No files matched your search
@@ -15,10 +15,12 @@ Provider-specific setup must stay in the shared feature and `AppDefinition` meta
|
||||
|
||||
Every active heartbeat with a responsible user receives two run-bound tools:
|
||||
|
||||
- `connections_search({ query })` searches first-party connectable definitions and returns `ready`, `needs_user_action`, `available`, or `unavailable` from the requesting agent's perspective.
|
||||
- `connection_request({ service })` returns immediately when the service is already usable. Otherwise it creates or reuses a `connection_intent` and instructs the agent to end the run pending continuation.
|
||||
- `connections_search({ query })` searches catalog names and descriptions, plus authorized configured MCP connections and indexed tool descriptions and returns `ready`, `needs_user_action`, `available`, or `unavailable` from the requesting agent's perspective.
|
||||
- `connection_request({ service })` returns immediately when the service is already usable. Otherwise it creates or reuses a `connection_intent` and instructs the agent to finish independent work, then yield pending continuation.
|
||||
|
||||
Claude and Codex receive the tools through a native managed MCP server. Local/process adapters receive `PAPERCLIP_RUNTIME_TOOLS_*` environment variables and CLI guidance. Cloud, HTTP, gateway, and external adapters receive the typed runtime descriptor in their invocation context; compatible adapters may also project it into their remote environment.
|
||||
The native Paperclip Runner advertises both tools through its server-owned tool authority even with an empty MCP assignment. It captures the current responsible identity at each call. Legacy Claude and Codex receive the tools through a managed MCP server. Local/process adapters receive `PAPERCLIP_RUNTIME_TOOLS_*` environment variables and CLI guidance. Cloud, HTTP, gateway, and external adapters receive the typed runtime descriptor in their invocation context; compatible adapters may also project it into their remote environment.
|
||||
|
||||
Legacy delivery uses the same intent service, setup card, and fresh-session resolution wake. Environment and descriptor delivery require the receiving harness to consume them; they do not establish support in every third-party runtime. The default legacy prompt includes the canonical discovery guidance. A custom `promptTemplate` replaces that default and should retain the connection guidance if proactive discovery is desired.
|
||||
|
||||
The equivalent CLI helpers are:
|
||||
|
||||
@@ -31,14 +33,46 @@ The manually configured Paperclip MCP server also advertises `connections_search
|
||||
|
||||
## Security and lifecycle
|
||||
|
||||
- Company, agent, run, task, and responsible user come only from the signed runtime token and stored heartbeat context.
|
||||
- Company, agent, run, task, and responsible user come only from the signed legacy runtime token or the native server-owned binding and stored execution identity.
|
||||
- Tokens are scoped to connection intents, expire after one hour, and are rejected when the heartbeat is no longer running.
|
||||
- The thread payload contains only service identity, requesting-agent identity, and a safe phase. It never contains credentials or authorization URLs.
|
||||
- OAuth state is linked to the interaction. The same-origin callback finalizes the existing connection pipeline, posts only interaction ID/outcome to its opener, and redirects back to the task if there is no opener.
|
||||
- Personal OAuth defaults to the addressed user and creates an explicit delegation to the requesting agent. Reuse and installs are additive.
|
||||
- Task-hosted setup locks install reach to the requesting agent; the store host retains its normal broader access choices.
|
||||
- The intent resolves only after the connection, grant/delegation, profile access, and install succeed. Failures remain pending with `needs_retry`.
|
||||
- Closing the task, a newer run requesting the same service, or a newer human task comment expires the intent and deletes linked OAuth state.
|
||||
- Success and decline wake the assignee once using an interaction-and-status idempotency key and force a fresh continuation session.
|
||||
- Closing or reassigning the task expires pending intents and deletes linked OAuth state. Ordinary comments and later runs preserve the pending card. Requests reuse the same task, requester, addressed user and service; a different addressed user supersedes an older request for that service.
|
||||
- Success and explicit decline atomically persist a continuation delivery with resolution. A leased startup/periodic worker dispatches through heartbeat with a unique `connection-intent:<interaction>:<outcome>` wake key. It checks assignment, status, membership and current executable access, retries paused/suppressed delivery, and recovers a crash after enqueue without creating a second wake. The continuation forces a fresh provider session; heartbeat queues it behind active execution.
|
||||
|
||||
Legacy `request_confirmation.payload.connectionAuthorization` interactions remain readable and resolvable. New agent requests use `connection_intent` exclusively.
|
||||
|
||||
## Model evaluations
|
||||
|
||||
The companion `paperclip-evals` repository owns the connection cases in
|
||||
`evals/runner-api-tools/connection-cases.json`. They use the existing real-server
|
||||
API-tool eval controller and `scripts/runner-api-eval-worker.ts`, with this
|
||||
checkout's production tool definitions, connection guidance, and authority.
|
||||
Natural prompts measure discovery and request selection; explicit contract probes
|
||||
measure deduplication, readiness, and denied targets. The fixture helper supplies
|
||||
isolated company records and retains initial and final connection interaction
|
||||
state. Scoring requires observed calls/results and persisted state, not an
|
||||
assistant's claim of having connected.
|
||||
|
||||
These tool evals do not perform live provider OAuth or establish browser quality.
|
||||
The native and legacy connection browser suites separately cover setup and
|
||||
continuation, while the live-provider journey report records actual authorization
|
||||
and data-read coverage.
|
||||
|
||||
|
||||
## Custom targets, readiness and recovery
|
||||
|
||||
Catalog slugs remain stable. Search also returns `connection:<uuid>` for configured custom connections whose active identity grants authorize the responsible person, their company, or the requesting agent. Identifiers are never interpreted as URLs. Configured metadata and tool descriptions, including catalog-provider descriptions, are read only after the company and identity audience checks. Setup choices expose only display and selection metadata; they never include connection configuration, transport settings, or credential fields. Discovery reads the stored index without refreshing providers. Search is ranked with exact provider matches first and capped at 20 results.
|
||||
|
||||
`ready` requires an installed, enabled, healthy executable connection, permitted catalog tools, and a usable runtime identity. An installed connection with denied actions is administrative denial rather than a request to reauthenticate. Runtime calls continue enforcing access after a historical card resolves. If access is ready but the native provider's pinned tool snapshot is older, `connection_request` queues a fresh session without another authorization card.
|
||||
|
||||
Task setup defaults to personal identity when supported and the requesting agent's install reach. Existing installs are additive. An OAuth callback from a task prepares the catalog without adding access. Intent completion validates the current task and identity, then adds the requesting agent’s binding and install in the resolution transaction. Callback messages do not establish authorization: the card reloads the durable server result. A blocked popup offers a new-tab fallback; closing or declining provider sign-in keeps the request retryable. Only the card's **Not now** action declines the request. Decline continuations are told to pursue alternatives and cannot immediately request the same service again.
|
||||
|
||||
## Verification
|
||||
|
||||
Service and native-authority tests cover discovery, current identity, company boundaries, cross-run deduplication, additive grants and installs, permission denial, resolution atomicity, restart delivery and stale assignment. `tests/e2e/in-feed-native/playwright.config.ts` starts source `test-drive` instances with fresh data directories and a deterministic fake Codex provider plus MCP server. It exercises the real native runner and gateway; it is fixture proof, not live Notion or GitHub proof. Run with `pnpm exec playwright test -c tests/e2e/in-feed-native/playwright.config.ts`.
|
||||
|
||||
Offline Storybook examples live in `ui/storybook/stories/in-feed-connections.stories.tsx`. Build with `pnpm --filter @paperclipai/ui build-storybook`, then run `pnpm exec playwright test -c tests/storybook-visual/in-feed-connections.config.ts`. The suite checks every independently addressable story in both themes, catches play-function failures, and saves screenshots. Live provider acceptance additionally requires a model credential and a test workspace/account; do not describe fixture results or local-trusted testing as authenticated/cloud acceptance.
|
||||
@@ -404,3 +404,16 @@ Per-component rationale:
|
||||
### 7.3 Interactive-card affordance (Run 3 review feedback)
|
||||
|
||||
`Card` gained an `interactive` prop — pointer cursor, quiet hover (border→foreground/20 + shadow-md lift), focus-visible ring — used when the whole card is a click target (e.g. Companies selector). Skills tiles (CompanySkills `SkillCard`) and artifact cards (`ArtifactCard`/`ArtifactGroupCard`) apply the same recipe verbatim since they cannot render through Card (button/Link semantics). Static container Cards stay affordance-free by design.
|
||||
|
||||
|
||||
## In-task connections — 2026-09-07
|
||||
|
||||
| Reusable surface | Production owner | Hosts / coverage |
|
||||
|---|---|---|
|
||||
| Connection request card | `ui/src/features/connections/ConnectionIntentInteractionBody.tsx` | Task timeline, interaction card, design guide; pending, reuse, authorizing, retry, resolved, audience and error stories |
|
||||
| Connection setup flow | `ui/src/features/connections/ConnectionSetupFlow.tsx` | Connections page and task dialog share provider forms, OAuth, validation and additive installs |
|
||||
| OAuth handoff | `OAuthConnectStateScreen` in the shared setup module | Entry, starting, open window, blocked popup, closure, callback failure, retry and new-tab fallback |
|
||||
| Identity and agent access | `AccessStep` in the shared setup module | Personal, organization, dedicated agent, unavailable identity and loading; task host fixes install reach to the requester |
|
||||
| Setup completion | `ConnectionSetupCompletionScreen` in the shared setup module | Page and dialog; identity, granted agent access and enabled actions |
|
||||
|
||||
Independently addressable examples live under `Connections/In-task connections` in Storybook. The task composer remains available while a card is pending. These components use the existing token and primitive layers.
|
||||
Reference in new issue
Block a user