Jannik/PaperClipAI
1
0 Fork 0
mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-22 16:51:28 +02:00
Code Issues Packages Projects Releases Wiki Activity

feat(connections): add advanced provider setup and live browser qualification (#15341)

Browse Source 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Users connect accounts and choose an agent harness and model.
> - The runtime change in #14970 supports custom providers on those
connections.
> - Normal setup must stay simple while advanced users can choose a
compatible gateway.
> - Shared connector rows and access controls keep these choices
consistent.
> - This pull request refines the agent setup UI and adds review stories
and repeatable browser qualification.
> - The qualification checks real tools and downloaded outputs, not only
a successful run status.

## Linked Issues or Issue Description

Refs #14970, #37, #13083, #14104, #14565, #12692.

The core implementation in #14970 is merged. This branch incorporates
its squash commit and targets `master`. Both PRs contain our
implementation. #14016 is a reference only and is not a dependency. This
PR has 96 changed files.

## What Changed

- Complete model-provider connector presentation beside other
connectors. Each row uses the existing Connect action and connection
list. Tags are stored without category UI. The base PR includes the
provider forms and routes.
- Show persistent Subscription, API Key, and Advanced choices. Label
Advanced as Custom Gateway. Reuse provider logos, connection lists, and
permissions controls. Default access to the organization and all agents
when permitted; keep narrowing controls under Advanced.
- Keep Configure reachable before subscription sign-in, so users can
select a supported environment when the default cannot sign in. Testing
and saving still require a connection. Show the execution environment in
Configure. Preserve the confirmed Connect choice. Editing a method,
credential, saved account, or advanced choice requires that current
choice to connect before testing or saving. Use matching model and
thinking-effort dropdowns and retain connection icons in selected
values.
- Preserve the new harness model default when switching an existing
OpenCode agent to Codex or Claude, and resolve user-selected model names
with the effective harness.
- Load popular OpenRouter models through the shared connection-model
discovery path. Keep explicit model lists and manual model entry
available.
- Group onboarding, connection setup, agent runtime, management,
recovery, and production-component stories under AI Connections /
Provider routing.
- Add an explicit-only provider-connections browser suite for managed
local or existing local/staging targets. Use private browser profiles
and credential handoffs. Support human-assisted subscription sign-in
without sharing passwords or tokens in reports.
- Verify persisted connection identity, runtime probes, tool execution,
exact artifact bytes, completion, and context-dependent follow-up.
Retain source/model provenance, cost bounds, closed error diagnostics,
original failures, and cleanup evidence.
- Add Gemini startup-model and skill-root fixes, Grok private-history
detection, ACP filesystem regression fixtures, selected-workspace
handling for local Hermes, and artifact-helper workspace fallback.
- Keep managed Grok runtime homes disposable. Remove host-side
transcript retention/restoration because private file modes do not
isolate same-user agent processes. Ignore earlier development archives
and use a fresh task handoff when history is unavailable. Verify the
absence of restored transcripts with a separate same-user process.
- Capture stopped-run diagnostics before deleting an attached-company
fixture agent. Track creation and owned sign-in receipts; revoke only
this attempt's accounts and never adopt a concurrent campaign's newly
created account. Preserve failure signals and final status through
cleanup.
- Require the requested environment in the saved agent and every run,
including follow-ups. Reject a forced incompatible target. Keep one
cancellation state through startup, every cell, reporting, and teardown
for SIGINT, SIGTERM, and SIGHUP. Stop further paid cells after
interruption. Document qualification limits.

## Verification

- Current head `b3bb3e94d577d43d9965a6b9daba039f599b2e49` includes
master `d9f600043`. The security fix in `a758fde31` passes full
workspace typecheck, production build, and 119 connection/Grok
regressions. The unchanged UI passes all 126
configuration/model-discovery tests and token gates. The final
published-guide correction passes Grok adapter typecheck. Earlier head
`eebd8225c` passed the complete deterministic runner suite (1,404 Vitest
tests and 128 Node tests) and all CI jobs. Current-head CI run
`37520147514` passed all 47 jobs, including the full sharded Vitest and
browser matrix, production build, and canary dry run. All 55 checks
completed: 53 successes and two expected skips. The current-head
security scan passed, Greptile is 5/5, and no review threads remain
open.
- A separate same-user process reproduced reading a restored Grok
transcript before the security fix. The regression now finds no
transcript. Existing fresh-session fallback and ordinary session
metadata behavior pass.
- The final account-choice and cleanup fixes pass 85 setup tests and 26
qualification-harness tests. Regressions verify that editing a
connection invalidates confirmation, Configure remains reachable before
sign-in, diagnostics are captured before fixture deletion, and
concurrent campaigns cannot adopt or revoke each other's accounts. UI
and E2E typechecks pass.
- The Storybook build and actual Chromium production-component stories
passed during this change. Review the neighboring AI Connections /
Provider routing stories, regular connector rows, three connection
modes, model discovery, and the single execution-environment control in
Configure.
- Cancellation smoke verified authenticated cleanup before browser close
for SIGINT, SIGTERM, and SIGHUP. Regressions cover interruption during
startup and reporting, missing-file ACP resource errors, and preserved
permission denials. Both ACP runtime versions and 54 ACPX/Grok
regressions passed. The deterministic connection-intent browser suite
passed two tests.
- Historical local qualification retained 43 passing API/gateway cells
out of 46, with downloaded outputs and follow-up receipts. These
attempts span earlier builds; they do not qualify this exact commit or
staging. Subscription combinations, Gemini overloads, and the unresolved
follow-up failure remain recorded rather than counted as passing.
- Use `pnpm test:e2e:runner -- --list --suite provider-connections` to
inspect the matrix. Follow `tests/runner-e2e/PROVIDER-CONNECTIONS.md`
for credentials, target URL, sign-in assistance, budget, evidence, and
cleanup. Paid live tests remain opt-in.

## Risks

- The core implementation in #14970 is merged. This PR adds no database
migration of its own.
- Subscription login needs an interactive provider session. Dedicated
accounts and staging qualification remain follow-up work; this PR does
not certify every login combination for production.
- Managed Grok transcript resume is deferred until provider history has
an OS isolation or authorized broker solution. Follow-ups start fresh
with Paperclip task context; earlier live Grok results do not qualify
this behavior.
- Gemini CLI 0.58.0 has an upstream ACP new-file error conversion
defect. Live overloads and one unresolved follow-up timeout remain
recorded. The stock CLI is unchanged, and those cases are not marked as
passing.
- Real-provider tests spend credits and use private credential/evidence
directories. The launcher requires explicit selection and checks target
ownership. It must not attach to a developer's database by accident.
- OpenClaw Gateway, Hermes Gateway, Claude Managed, AWS AgentCore,
Process, HTTP, and legacy ACPX local remain outside custom provider
setup.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools, code
execution, and browser testing. The exact deployment model ID and
context window size were not exposed in this session.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-10-06 15:21:22 -05:00
1 parent a7a2ed63a0
commit e38d6d16b6
96 files changed
+8062 -543
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+3
View File
@@ -417,6 +417,9 @@ This starts the UI and API at `http://localhost:3100`. An embedded PostgreSQL da
> **Requirements:** Node.js 24.11+, pnpm 9.15+
Local Claude and Codex subscription sign-in also needs Python 3 and the
corresponding provider CLI on the Paperclip host. The Docker image includes them.
Source development also builds the native Paperclip Runner when enabled (the self-hosted default). Install a Rust toolchain, or set `PAPERCLIP_RUNNER_BINARY` to a compatible prebuilt runner.
<br/>
doc/DOCKER.md
+6 -5
View File
@@ -212,24 +212,25 @@ The image pre-installs:
- `claude` (Anthropic Claude Code CLI)
- `codex` (OpenAI Codex CLI)
- `python3` (used to run local subscription sign-in in a pseudo-terminal)
If you want local adapter runs inside the container, pass API keys when starting the container:
Start the container, then open **Apps → Connections** to connect a Claude or
Codex subscription in your browser. You can also enter a provider API key there.
You do not run a shell command or configure a CLI home.
```sh
docker run --name paperclip \
-p 3100:3100 \
-e HOST=0.0.0.0 \
-e PAPERCLIP_HOME=/paperclip \
-e OPENAI_API_KEY=... \
-e ANTHROPIC_API_KEY=... \
-v "$(pwd)/data/docker-paperclip:/paperclip" \
paperclip-local
```
Notes:
- Without API keys, the app still runs normally.
- Adapter environment checks in Paperclip will surface missing auth/CLI prerequisites.
- The image provides the CLI and Python prerequisites for local subscription sign-in.
- Adapter environment checks in Paperclip surface missing authentication or CLI prerequisites.
## Podman Quadlet (systemd)
doc/connections/AI-CONNECTIONS.md
+51 -16
View File
@@ -10,6 +10,11 @@ homes. Managed default/shared accounts are additional choices in that same
selector. Selecting “Sign in to another account” survives background refreshes;
Claude authorization paste keeps upstream's immediate Connecting feedback.
New-agent Connect offers three persistent tiles: the provider's subscription,
the provider's API key, and Advanced. All three stay visible while the selected
mode's form is shown below. Advanced opens the existing compatible-connection
picker and provider setup. Execution environment selection lives in Configure.
Storybook's simulated controllers and page annotations do not run in the app.
## Compatibility and selection
@@ -379,8 +384,8 @@ onboarding, and agent setup share `LocalProviderLoginInstructions` and
`useLocalAiLogin`. Claude and Codex start a local provider process behind the
browser sign-in card. Claude accepts the authorization code in that card; Codex
displays its device code there. The user does not run a shell command. Each
local runner requires Python 3 for its pseudo-terminal and the corresponding
provider CLI on the Paperclip host. Each
local runner requires Python 3 for its pseudo-terminal (included in the Docker
image) and the corresponding provider CLI on the Paperclip host. Each
attempt retains a private credential home. The home is never seeded with the
operator's existing login: copying a rotating refresh token would
allow managed runs to invalidate credentials still used by legacy agents or the
@@ -511,15 +516,24 @@ fingerprint. These checks do not relax current connection authorization.
## Advanced provider routing (2026-10-02)
The connection API and catalog support OpenRouter, Amazon Bedrock, Google Gemini,
Responses API, Messages API, Chat Completions API, and local endpoints.
The catalog tags these entries `model-provider`. Responses-compatible gateways
such as Emissary use the Responses API definition.
Use the regular rows on **Connectors** to add OpenRouter, Amazon Bedrock,
Google Gemini, a Responses API, Messages API, Chat Completions API, or local
endpoint connection. Each row has its own Connect action and saved accounts.
The catalog tags these entries `model-provider`; no category UI is shown.
Responses-compatible gateways such as Emissary use the Responses API row.
The native subscription/API-key onboarding remains the default. Provider choices
show the existing local brand artwork and reuse the existing access step. Custom
URLs, protocol, AWS region, and credential fields appear only after choosing the
corresponding connector. New connections default to everyone in the organization
and all agents when the actor has permission; the existing Advanced disclosure
contains the controls to narrow access, without a separate Access step. New-agent setup also offers the connection picker in its persistent
**Advanced / Custom Gateway** tile.
Provider routing belongs to the connection; the model belongs to the agent.
The API exposes compatible saved connections and optional model identifiers.
The advanced setup UI and review stories ship in the follow-up UI change.
Native subscription and API-key setup keep their existing controls.
At **Agents → [agent] → Harness / Runtime**, **Connection** is a dropdown of
compatible saved connections, including explicit personal accounts. The existing
model picker uses that connection’s optional model IDs and accepts manual IDs.
Changing connections preserves the model for explicit review. URLs and
credentials belong to the connection; the model belongs to the agent.
| Harness | Implemented managed routes |
| --- | --- |
@@ -541,9 +555,10 @@ catalog discovery for custom gateways are not part of this implementation.
OpenRouter connections without an explicit model list automatically load its public
[model catalog](https://openrouter.ai/docs/api/api-reference/models/list-all-models-and-their-properties),
ordered with `sort=most-popular`. The company-scoped model discovery API preserves
the provider's ordering and adapts model IDs to the selected harness. Explicit
connection model lists take precedence. Catalog discovery sends no credentials.
ordered with `sort=most-popular`. New-agent setup and agent settings share this
discovery path, preserve the provider's ordering, and adapt model IDs to the selected
harness. Explicit connection model lists take precedence. Catalog discovery sends
no credentials; a failed request offers refresh and manual model entry.
`config.ai.routing` stores only kind, protocol, URL, auth method, region, and
optional model IDs/labels. The vault stores provider API keys, including Bedrock API keys.
@@ -573,6 +588,15 @@ remote URLs; loopback endpoints may use HTTP. Localhost refers to the agent’s
execution environment, including when it is a sandbox. URLs cannot contain user
credentials, query parameters, or fragments.
Managed Grok uses a disposable runtime home. Paperclip does not retain or restore
Grok transcript files into host temporary directories: private file modes do not
isolate agents running as the same OS user. Archives from earlier development
builds are ignored. Provider session metadata still saves normally. If the selected
session has no history in its current execution environment, Grok starts a fresh
session with the Paperclip task handoff rather than attempting remote subscription
recovery. Transcript continuation requires an isolated provider history solution
and is not qualified by this change.
Runtime projection clears alternate provider credentials and routing overrides,
uses disposable homes, and never falls back to host authentication. Codex probes
retain the selected provider home. The new runner copies only the validated
@@ -605,9 +629,10 @@ through PostgreSQL JSONB and verifies that credential rotation retains identity
and agent access.
General AWS access keys are not accepted or forwarded to Claude; use a Bedrock
API key. Support for AWS roles requires a credential broker before it can ship.
A subsequent OpenCode tool-use check reached OpenRouter but was denied terminal
access; its follow-up ended with `process_lost`. Treat OpenCode tool-use acceptance
as unresolved rather than inferring it from a completed task status.
The initial OpenCode tool-use check was denied terminal access. The native driver
now includes the assigned workspace and its canonical path in the allowed
directories while preserving the configured tool policy. Subsequent local
qualification verified tool use and follow-up on the affected OpenCode route.
Live Bedrock verification subsequently passed with a short-lived Bedrock API key,
region `us-east-1`, and `us.anthropic.claude-sonnet-4-6`. The saved connection
passed **Run test**. Claude legacy and Claude New Runner each ran a terminal
@@ -616,6 +641,16 @@ tool output was verified for all four successful runs. Private gateways still
have deterministic mapping and validation coverage but need live verification
in the target deployment. Short-lived Bedrock keys must be rotated before expiry.
The repeatable provider connection campaign is documented in
[`tests/runner-e2e/PROVIDER-CONNECTIONS.md`](../../tests/runner-e2e/PROVIDER-CONNECTIONS.md).
It preserves actual tool outputs, downloaded artifacts, completion and follow-up
receipts, source provenance, cleanup, and provider failures. The retained local
qualification has 43 passing API/gateway cells out of 46; staging and all
subscription combinations remain unqualified. Gemini CLI 0.58.0 has an upstream
ACP new-file error conversion defect. The provider-free filesystem probe exposes
that defect without modifying the installed CLI. Provider overloads and one
follow-up timeout also remain live qualification limits.
### Gateway completion compatibility
Provider-facing `paperclip_finish` accepts an omitted or `null` continuation for
doc/evals.md
+4
View File
@@ -1,5 +1,9 @@
# Paperclip evaluation guide
The explicit-only [live provider connection suite](../tests/runner-e2e/PROVIDER-CONNECTIONS.md)
is a Product E2E workflow for fresh subscription/API-key/gateway connections,
with attended login and independent artifact checks against local or staging targets.
Paperclip has two live eval families with different questions, owners, and
evidence. Choose the family before selecting a model, profile, or case.
doc/plans/2026-10-03-live-provider-connection-qualification.md
+211
View File
@@ -0,0 +1,211 @@
# Live provider connection qualification
Date: 2026-10-03
Status: Runnable implementation delivered; operating contract and initial verification are in [PROVIDER-CONNECTIONS.md](../../tests/runner-e2e/PROVIDER-CONNECTIONS.md). Live qualification is reported per campaign, not implied by this plan.
Code inspected: `527e3bad2` (`codex/provider-routing-connections`).
## Decision
Extend the existing Playwright Product E2E harness with real connection-creation journeys, an explicit authentication handoff, and support for an existing deployment as the target. Run the subscription journeys in a visible browser on a developer's machine first. The same tests should target local Paperclip and staging.
Playwright owns navigation, assertions, evidence, and the verdict. A person completes login challenges when necessary. A computer-use agent can assist with diagnosis and provider UI changes, but it must not silently change the steps or decide whether the test passed. An assisted success is useful evidence and must be labeled as assisted.
The deliverable is a matrix answering **which configuration worked, against which deployment, with what proof**. A successful login, a saved connection, and a successful agent run are separate checkpoints.
## Existing foundation and gaps
- [`tests/runner-e2e`](../../tests/runner-e2e/README.md) already supplies real browser/server/runner/provider tests, catalog selection, isolated fixtures, independent grading, reports, cost accounting, and provenance. This is the appropriate family, as defined in [`doc/evals.md`](../evals.md).
- Its [Playwright configuration](../../tests/runner-e2e/playwright.config.ts) currently owns a fresh loopback server and database. Supporting staging requires a target lifecycle abstraction, not just substituting a URL.
- [`tests/ai-connections-app/app.spec.ts`](../../tests/ai-connections-app/app.spec.ts) includes simulated login sessions and popup destinations. It also retains local terminal-login assertions that predate this branch's browser-login changes. Keep fast simulated tests, update those stale expectations, and label their limited proof clearly.
- [`tests/hiring-ai-connections/hiring.live.spec.ts`](../../tests/hiring-ai-connections/hiring.live.spec.ts) already checks real runs, connection attribution, and legacy/native execution. It creates initial API-key connections through the API and requires a loopback `local_trusted` instance. Reuse the assertion patterns; it is not a complete connection-onboarding test or a staging harness.
- The existing Grok subscription qualification stages supplied `GROK_AUTH_JSON`. Its documentation correctly says this does not test interactive browser login.
## Coverage contract
Inventory harnesses separately from providers. Claude is a harness; Bedrock and OpenRouter are connection routes. A Paperclip deployment URL, an authentication host, and an agent execution environment are also separate dimensions.
The initial inventory below comes from [managed connection capabilities](../../packages/shared/src/ai-connections.ts), [routing compatibility](../../packages/shared/src/ai-provider-routing.ts), and the [UI adapter registry](../../ui/src/adapters/registry.ts). It describes implementation eligibility, not proven live coverage.
| Harness | Managed subscription | Direct API key | Advanced routes currently admitted |
| --- | --- | --- | --- |
| Claude Code | Yes | Anthropic | OpenRouter, Bedrock, Messages-compatible gateway |
| Codex | Yes | OpenAI | OpenRouter, Responses-compatible gateway |
| Grok | Yes; qualify its actual login UX separately | xAI | Not supported by the current routing compatibility table |
| Gemini | Not in the managed connection capability table | Google | Not in the current routing compatibility table |
| OpenCode | No managed subscription entry | OpenRouter | OpenRouter, Chat Completions-compatible gateway |
| Hermes local | No managed subscription entry | Inventory harness-specific behavior | OpenRouter, Chat Completions-compatible gateway |
| Cursor local, Kimi, Pi | Inventory required | Inventory required | Inventory required |
“Not in the table” is a Paperclip managed-connection limitation, not a claim about everything the upstream CLI can do. Inventory native runner providers and installed first-party adapters as well as legacy UI adapters; any additional local provider must receive a manifest entry before the full campaign can qualify.
Exclude remote OpenClaw and Hermes gateways, Cursor Cloud, managed Claude/AWS AgentCore, process/HTTP adapters, and the generic legacy ACPX integration from this campaign. Their remote systems own provider configuration. **Hermes local remains in scope.** Supported native Claude/Grok runner profiles that internally use ACPX remain in scope; that transport is distinct from the excluded generic legacy integration.
For every supported harness/auth/route combination, enumerate the supported legacy/native runner profiles and target environments. Do not infer that an API-key pass covers subscription, that legacy covers native, or that local covers staging. Protocol routes need a real configured endpoint and credential; a mock gateway is useful for regression tests but does not supply live qualification.
Keep three independent manifest fields:
- Eligibility: supported, unsupported/product gap, needs inventory, or deliberately excluded with a reason.
- Execution: not run, awaiting user, missing credential, blocked target, running, complete.
- Qualification: passed or failed, with individual checkpoint results and assistance recorded.
A credential-free catalog check must flag newly added local harnesses or changed capabilities without a reviewed coverage entry. Report required, selected, attempted, and passed counts separately. Unsupported configurations are visible product gaps, never green tests.
## Authentication strategy
Use three lanes, with separate results:
| Lane | Starting state | What it proves |
| --- | --- | --- |
| Fresh connection, existing browser sign-in | Saved provider website/Google browser session; no Paperclip connection or runtime credential for this cell | The current Paperclip subscription flow obtains and saves a fresh usable credential |
| Fresh connection, signed-out browser | Empty provider browser state; no Paperclip connection or runtime credential | The complete first-time experience, including account selection and login |
| Existing connection | A connection previously created by the live UI journey | Selection, reuse, continuation, and reconnect behavior |
Most developer campaigns use the first lane. The signed-out lane runs explicitly before releases or login changes. The existing-connection lane cannot substitute for either creation lane.
There are three distinct identities/stores: Paperclip board authentication for the selected target, provider website authentication in the browser, and the resulting managed connection credential. Reusing provider website sign-in reduces friction while still exercising a fresh OAuth/device authorization and production credential persistence. Do not import `~/.codex`, `~/.claude`, or an existing token into a fresh-connection test.
Initially, use the developer's existing subscription accounts through a dedicated QA browser profile, separate from their everyday Chrome profile. Keep account aliases and expected identities in private local configuration; do not put personal email addresses into the public catalog. Serialize tests sharing an account or refresh-token chain. Never copy the profile between CI workers.
### Login checkpoint
The flow starts the real production login session, arms popup/new-tab observation before clicking, and tracks the expected provider origin and Paperclip session. It handles ordinary redirects and device-code/code-entry steps using the actual UI. Provider and Google pages can require a person for account choice, passwords, passkeys, MFA, CAPTCHA, or consent.
At such a boundary:
1. Show the relevant window and mark the cell `awaiting_user`, with the provider, target, and expected account alias.
2. Pause only the affected account's queue. Keep a bounded deadline tied to the real login session's expiry.
3. Let the person complete the production browser flow. If the provider rejects the automated browser, support handoff to a normal browser using the product's existing device URL or code-return flow. Do not add authentication bypasses.
4. Resume from observed Paperclip login status and saved-connection state. Clicking “continue” is never sufficient proof of successful authorization.
5. If the session expires, retain that attempt and use the product's retry action to start a new session. Never replay an old callback code.
Record which step needed assistance and its duration, without recording passwords, codes, callback query strings, or provider session IDs. Check the expected account using available provider/account metadata; where this cannot be machine-verified, explicitly record human identity confirmation rather than claiming an automated identity check.
There is no promise that Google or provider sign-in will remain unattended indefinitely. Expired browser state is an operational login requirement, not a passing test. A bug that loses a completed callback is a product failure, not automatically “needs login.”
## Local and staging target adapters
Introduce two lifecycle modes while sharing the same journey:
- **Managed local:** create and own an isolated Paperclip home, database, server, company, and runtime. Teardown may stop/delete only these owned resources.
- **Attach existing:** authenticate to a configured local/staging URL and create a dedicated QA company or use an explicitly designated test company. Never reset the deployment, change instance-wide feature settings, install plugins, or stop its server as fixture setup.
The current local `3109` application can be an attach-existing target. Preserve its BOB company and user data. Full first-company onboarding belongs on a fresh managed target or a dedicated disposable staging tenant/instance; a shared staging deployment cannot prove that path merely by creating another agent.
Each private target configuration supplies an exact origin, target mode, expected deployment identity/revision, board-auth account reference, QA company policy, and available execution environment. The staging URL is still to be supplied. Target validation precedes credential resolution. Verify redirects, health, authorization, installed harness versions, required capabilities, and environment availability; do not send credentials to a host introduced by a redirect or unreviewed target configuration.
Record the tested server revision separately from the harness revision, plus runner/CLI version, immutable sandbox image when applicable, model ID, protocol, authentication host, and execution environment. Staging with a missing capability is blocked, not silently tested through a different route. Unknown build provenance prevents a release qualification claim.
## Required browser journey and independent proof
For each admitted cell:
1. Create/select the isolated fixture company and a synthetic workspace through existing public surfaces. Ordinary bootstrap APIs may prepare the fixture; they must not create the connection or agent that the browser journey is supposed to test.
2. Enter the production new-agent flow, select the harness and Subscription/API Key/Advanced mode, and create the connection through the UI. Advanced cases supply the configured provider/protocol, endpoint or region, and selected credential reference. Verify that permissions default to everyone/all agents within the isolated test company and that advanced controls retain their values.
3. Verify the saved connection and compatible connection selector after reload. For a fresh-login cell, confirm that a new connection was actually created and that the selected connection is the one bound to the agent.
4. Choose the model and environment in Configure, run the product's test action, and finish setup. Verify the model list/custom model path and selected values survive reload.
5. Create a real task through the browser. Ask the agent to read a small synthetic input file, compute a deterministic result, and save a result artifact. Generate the input independently and keep the expected result out of the prompt. Seed the workspace using existing environment fixture facilities, not a private server hook.
6. Require the real tool receipt, persisted successful run, task completion, correct artifact bytes/hash, and a user-visible result link that opens after reload. “Provider routing works” or any other model-authored success sentence alone cannot pass.
7. Correlate the run with the selected connection, auth method, harness/runner, route, model, and execution environment using durable production evidence. Reuse the existing run-context assertions where available. Any missing attribution is an evidence gap to implement explicitly, not something to infer from the model's response.
8. Send one bounded follow-up that uses the saved result and produces a second independently checked output. This proves the saved credential remains usable beyond the setup probe.
9. Clean up only campaign-owned resources, retain sanitized results, and report cleanup failures separately. Failed cleanup prevents an entirely clean campaign result but does not erase successful functional evidence.
Use a controlled runtime without ambient provider keys or personal CLI homes so a broken selected credential cannot fall back to another account. On staging, explicitly qualify the runtime's credential isolation. Add a negative case with a deliberately invalid selected credential and require an actionable failure with no successful fallback. For a controlled gateway, correlate sanitized request metadata when available; never retain authorization headers.
Also cover the other product entry points: Apps connector row → Connect → connection list → agent selection; existing agent → switch compatible connection; and reconnect after an expired/invalid connection. Apply every supported auth method to both Apps and agent-creation entry points. Fresh company onboarding and destructive recovery cases are separately enumerated; their target requirements must be visible. A representative smoke selection is useful, but is not the full matrix.
## Trustworthy results and safe evidence
Extend the existing Product E2E result/report pipeline with authentication checkpoints, assistance, target identity, and coverage denominator. Preserve the existing failure classification: product, model/provider behavior, grading/evidence, or infrastructure. Operational blocks are separate from these failure causes.
Keep first failures and all attempts. Permit only declared bounded infrastructure retries; do not let an agent change selectors or retry a failed product flow until the campaign looks green. A repaired test starts a new definition/revision measurement.
Example report shape (illustrative, not results):
| Cell | Target | Login | Saved | Tool/artifact | Follow-up | Outcome |
| --- | --- | --- | --- | --- | --- | --- |
| Claude subscription/native | Local | Assisted | Pass | Pass | Pass | Pass, assisted |
| Codex subscription/native | Staging | Needs reauthentication | — | — | — | Awaiting user |
| Grok custom gateway | Local | — | — | — | — | Unsupported/product gap |
Disable automatic Playwright traces, HARs, screenshots, videos, request-body logging, and DOM dumps for the live authentication flow, including failure handlers. Masked password fields do not protect network traces, tokens, or callback URLs. Exception messages and reporters also need sanitized wrappers; do not serialize raw Playwright call logs containing filled values or navigated URLs. Calibrate redaction with synthetic sentinel credentials before using real accounts.
Keep browser profiles and auth state outside the repository and artifact directories, using OS-protected storage or restrictive local permissions. Persist only selected provider state; never include it in reports, caches, or CI uploads. Resolve one cell's required credentials at a time from a private secret store; API-key fixtures may reference existing local secrets without sourcing the entire secrets file into the server environment.
After authentication, allow only the existing synthetic-task screenshots and structured allowlisted evidence. The [current public evidence policy](../../tests/runner-e2e/SECURITY.md#public-evidence-boundary) explicitly excludes credential and setup pages; this feature must not widen that boundary. Auth diagnosis uses sanitized phase/status metadata, not screenshots of personal accounts. Read-only report regeneration must make no additional provider calls.
## Implementation slices
1. **Manifest and target support.** Add an explicit-only `provider-connections` Product E2E suite; register coverage and profiles in the existing catalog/fixture registry. Add owned-local/attach-existing target lifecycles without changing existing suites' defaults. Include credential-free tests for origin validation, coverage drift, selection, cleanup ownership, and blocked results.
2. **One complete subscription path.** Implement the authentication checkpoint and Claude subscription journey locally, then Codex. Prove each with a fresh UI-created connection, independent task artifact, reload, and follow-up. Prove both saved-browser-sign-in and signed-out lanes. Fix stale local-login browser assertions alongside this work.
3. **Same journey on staging.** Supply the staging target and board account; replay the Claude/Codex journeys against its actual login host and runner environment. Record deployment differences rather than building a separate staging script.
4. **API keys and advanced routes.** Add direct keys, OpenRouter, Bedrock, and each supported protocol gateway. Include wrong-key/no-fallback, mismatched protocol, model discovery, and custom alias cases. Bedrock has a selected region/model and an expiring credential; an expired credential is renewed before the case and never counted as a product pass.
5. **Complete the inventory.** Add Grok's actual subscription and key paths, Gemini, OpenCode, Hermes local, and supported Cursor/Kimi/Pi/native-provider cases. Resolve unsupported auth paths as explicit product decisions; do not invent a third mode for every harness. Expand entry points and runner/environment variants until every required manifest cell has evidence.
6. **Operationalize.** Start with a manually launched visible-browser campaign on the developer's machine. Later use dedicated subscription accounts on a trusted QA machine, plus unattended API-key/gateway jobs in the existing protected paid CI workflow. Keep subscription reauthentication an explicit attended operation whenever the provider requires it.
The suite, target configuration, and authentication selectors are now implemented; use the commands in [PROVIDER-CONNECTIONS.md](../../tests/runner-e2e/PROVIDER-CONNECTIONS.md). Discovery, verification, and reports reuse `test:e2e:runner`. The authoritative README/FIXTURES/SECURITY docs and `doc/evals.md` link the operating contract.
Each case declares a turn limit covering the setup probe and follow-up, a provider-run timeout, a separate login deadline, company/agent budget limits, and cleanup policy. Run one subscription case per account at a time. Present selected cells and estimated spend before paid execution; record actual usage and unknown cost separately. Do not call subscription usage “free,” and do not run the full matrix as a side effect of normal unit tests or `--all`.
## Acceptance criteria
- One catalog and journey implementation selects local or staging through configuration.
- At least one real fresh Claude and Codex subscription connection passes through each target before claiming the initial slice complete; no mocked popup/session or preinstalled credential substitutes for this proof.
- Human handoff, timeout, expired login, and account mismatch produce distinct, resumable outcomes without false passes.
- Passing requires the selected connection's persisted binding, real successful run, independently validated tool result/artifact, and a successful subsequent use.
- Credential-free grader tests reject a plausible success message without a tool result, a wrong artifact, wrong connection attribution, and absent evidence.
- Missing credentials, unsupported combinations, and unrun cells remain visible in the coverage denominator. Assisted and unattended results remain distinguishable.
- Existing local/staging user data survives; teardown owns only test resources.
- Secret sentinel tests prove that failure artifacts and report projections do not disclose login state or credentials.
- The full inventory is qualified only after all required supported cells have current evidence. Earlier slices are explicitly partial.
## External references
- [Playwright pages and popups](https://playwright.dev/docs/pages): popup/new-page events provide deterministic browser coordination.
- [Playwright authentication](https://playwright.dev/docs/auth): authenticated browser state can be reused, is sensitive, and expires. Reusing website state does not require bypassing Paperclip's connection flow.
- [Google OAuth policies](https://developers.google.com/identity/protocols/oauth2/policies): account authorization must use an appropriate browser flow. Google login restrictions and challenges are not something this harness should attempt to bypass.
## Verification update · 2026-10-05
Six previously failing local key/gateway journeys now pass: legacy Grok from
both entry points, legacy Codex/new-agent OpenRouter, legacy OpenCode/Apps
OpenRouter, and native OpenCode/OpenRouter from both entry points. Gemini
selected-model startup, current local skill-root guidance, and repair-run
artifact attribution have been corrected. Its final two browser retests still
failed the 300-second first-task deadline, so Gemini remains unqualified.
The affected ACP regression group passes 221 tests and Grok's two affected files
pass 54 tests. The ACP checks include a real missing-file protocol reproduction,
canonical local Gemini workspaces, preserved remote paths, and permission-denial
guards. The existing ACPX patch now returns resource-not-found for genuine
missing files; it keeps other errors and permissions intact. Workspace
typecheck/build and token gates pass; the full
Vitest suite has not produced a passing result after broad timeout failures.
The pinned native Grok installation still requires administrator access.
Attended subscription completion, staging/Daytona, and qualification against a
single reviewed committed build remain outstanding. See
[the qualification runbook](../../tests/runner-e2e/PROVIDER-CONNECTIONS.md) for
retained attempts, exact evidence boundaries, and replay instructions. This
update does not mark the overall acceptance criteria complete.
## Gemini investigation update · 2026-10-06
The new-agent/API-key workflow passes with `gemini-3.5-flash-lite`, including the
independent first artifact and follow-up checks. The Apps diagnostic retest
delivered its first artifact, then timed out on follow-up with no recognized
error signal; that cause remains unknown. Earlier deadline failures include provider HTTP 503 overload
retries; increasing the deadline would not resolve the underlying uncertainty.
A new credential-free installed-CLI smoke distinguishes correct ACP missing-file
semantics from Gemini CLI 0.58.0's plain-error-object conversion bug. A temporary
CLI copy with that narrow conversion fixed passes creation, existing-file reads,
and external-path denial; the user's installed CLI remains unchanged. Explicit
thinking-effort configuration also needs a supported startup-settings path;
short comparisons do not show that default `HIGH` thinking caused the deadlines.
The harness now projects terminal errors into closed diagnostic codes before
managed-instance teardown, retaining no error text or hidden reasoning. Fresh
managed targets preserve the original stopped onboarding database. Results,
cleanup, model/source provenance, and remaining qualification boundaries are
recorded in the private report and the runbook. The full acceptance criteria
remain open.
packages/adapter-utils/src/acpx-engine/execute.test.ts
+60 -2
View File
@@ -52,6 +52,7 @@ import {
const tempRoots: string[] = [];
let defaultGeminiTestHome: string | undefined;
async function makeTempRoot() {
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-acpx-skills-"));
@@ -60,6 +61,7 @@ async function makeTempRoot() {
}
afterEach(async () => {
defaultGeminiTestHome = undefined;
// A remote run stages a process-session bridge whose detached event writer can
// still be flushing a trailing event file into `.../process-sessions/<id>/events`
// when the run's own best-effort `client.remove(sessionDir)` (which production
@@ -179,6 +181,13 @@ async function runExecutor(
startupTraceContext?: AdapterExecutionContext["startupTraceContext"];
} = {},
) {
// Skill reconciliation must never inspect or modify the developer's real
// Gemini home. Keep one private home across this test's resumed turns.
const configuredEnv = config.env as Record<string, unknown> | undefined;
if (config.agent === "gemini" && !(typeof configuredEnv?.HOME === "string" && configuredEnv.HOME.trim())) {
defaultGeminiTestHome ??= await makeTempRoot();
config = { ...config, env: { ...configuredEnv, HOME: defaultGeminiTestHome } };
}
const runtimeOptions: Record<string, unknown>[] = [];
const configOptions: Array<{ key: string; value: string }> = [];
const sessionInputs: Record<string, unknown>[] = [];
@@ -619,7 +628,7 @@ describe("shared ACPX engine runtime behavior", () => {
});
});
it("keeps Claude startup model handling and Gemini session config handling unchanged", async () => {
it("sets Claude and Gemini models at startup without unsupported picker calls", async () => {
const claude = await runExecutor({ agent: "claude", model: "claude-opus-4-7" });
expect((claude.meta[0]?.env as Record<string, string>).ANTHROPIC_MODEL).toBe(
"claude-opus-4-7",
@@ -630,9 +639,10 @@ describe("shared ACPX engine runtime behavior", () => {
agent: "gemini",
model: "gemini-2.5-pro",
thinkingEffort: "high",
env: { GEMINI_MODEL: "stale-model" },
});
expect((gemini.meta[0]?.env as Record<string, string>).GEMINI_MODEL).toBe("gemini-2.5-pro");
expect(gemini.configOptions).toEqual([
{ key: "model", value: "gemini-2.5-pro" },
{ key: "effort", value: "high" },
]);
});
@@ -1600,6 +1610,29 @@ describe("shared ACPX engine runtime behavior", () => {
expect(await pathExists(path.join(codexHome, "skills", operational.runtimeName, "SKILL.md"))).toBe(true);
});
it.each(["codex", "gemini"])("refreshes the %s skill root when a connection home changes between turns", async (agent) => {
const root = await makeTempRoot();
const operational = {
...await createSkill(path.join(root, "skills"), "paperclip"),
key: "paperclipai/paperclip/paperclip",
};
const homes = [path.join(root, "first-home"), path.join(root, "next-home")];
for (const home of homes) {
const { meta } = await runExecutor({
agent,
stateDir: path.join(root, "state"),
env: agent === "codex" ? { CODEX_HOME: home } : { HOME: home },
paperclipRuntimeSkills: [operational],
paperclipSkillSync: { desiredSkills: [] },
});
const skillsHome = agent === "codex" ? path.join(home, "skills") : path.join(home, ".gemini", "skills");
expect(String(meta[0]?.prompt ?? "")).toContain(`Skill root for this run: ${skillsHome}`);
expect(String(meta[0]?.prompt ?? "")).toContain("Resolve referenced scripts and files relative to that skill");
if (home === homes[1]) expect(String(meta[0]?.prompt ?? "")).not.toContain(homes[0]!);
await fs.rm(home, { recursive: true, force: true });
}
});
it.skipIf(process.platform === "win32")("removes legacy ACPX Codex skill symlinks when a skill is no longer desired", async () => {
const root = await makeTempRoot();
const skillRoot = path.join(root, "skills");
@@ -2886,6 +2919,21 @@ describe("findAncestorBin", () => {
});
describe("gemini ACP flag selection", () => {
it("uses the canonical local Gemini workspace for ACP filesystem requests", async () => {
const root = await makeTempRoot();
const workspace = path.join(root, "workspace");
const alias = path.join(root, "workspace-alias");
await fs.mkdir(workspace);
await fs.symlink(workspace, alias, process.platform === "win32" ? "junction" : "dir");
const { runtimeOptions, sessionInputs } = await runExecutor({
agent: "gemini", agentCommand: "node ./fake-acp.js", cwd: alias,
stateDir: path.join(root, "state"),
});
const canonical = await fs.realpath(workspace);
expect(runtimeOptions[0]!.cwd).toBe(canonical);
expect(sessionInputs[0]!.cwd).toBe(canonical);
});
it("parses semantic version parts from gemini --version output", () => {
expect(parseGeminiVersionParts("0.30.0")).toEqual([0, 30, 0]);
expect(parseGeminiVersionParts("gemini-cli v1.2.3\n")).toEqual([1, 2, 3]);
@@ -3542,6 +3590,16 @@ describe("ACPX engine remote sandbox staging seam (PR 1: workspace + cwd)", () =
expect(sessionInputs[0]?.cwd).not.toBe(localCwd);
});
it("preserves Gemini's in-sandbox workspace without host canonicalization", async () => {
const { stateDir, localCwd, remoteCwd, executionTarget } = await setupRemoteSandbox();
const { sessionInputs, runtimeOptions } = await runExecutor(
{ agent: "gemini", agentCommand: "node ./fake-acp.js", stateDir, cwd: localCwd },
{ authToken: "real-run-jwt", executionTarget },
);
expect(runtimeOptions[0]?.cwd).toBe(remoteCwd);
expect(sessionInputs[0]?.cwd).toBe(remoteCwd);
});
it("test_remote_warm_handle_reused_after_cwd_change", async () => {
const { stateDir, localCwd, remoteCwd, executionTarget } = await setupRemoteSandbox();
const ensureInputs: Array<Record<string, unknown>> = [];
packages/adapter-utils/src/acpx-engine/execute.ts
+39 -5
View File
@@ -2045,6 +2045,12 @@ async function buildRuntime(input: {
if (requestedModel && acpxAgent === "claude" && !env.ANTHROPIC_MODEL) {
env.ANTHROPIC_MODEL = requestedModel;
}
// Gemini CLI reads GEMINI_MODEL when creating each ACP session. Its ACP
// implementation supports set_model, but not set_config_option; configure
// the selected model before startup without changing the transport.
if (requestedModel && acpxAgent === "gemini") {
env.GEMINI_MODEL = requestedModel;
}
if (acpxAgent === "codex") {
const codexStartupConfig = buildCodexStartupConfig({
existingConfig: env.CODEX_CONFIG,
@@ -2197,8 +2203,9 @@ async function buildRuntime(input: {
// (fingerprint, compat, persist, ensureSession, error) bind to THIS single
// value so a warm/resumable session created with the in-sandbox cwd is reused
// — not invalidated — on the next run. Remote runner-backed → the in-sandbox
// workspace dir; local and the runner-less fallback → the HOST cwd,
// byte-identical to today.
// workspace dir; local and the runner-less fallback → the HOST cwd.
// Local Gemini resolves tool paths through realpath. Give its ACP filesystem
// client the same canonical root so workspace aliases stay within that root.
//
// PR 3: the staging transport derives the in-sandbox workspace dir
// deterministically from the target's `remoteCwd` (it is exactly `remoteCwd`
@@ -2211,7 +2218,9 @@ async function buildRuntime(input: {
const sessionCwd =
useRemoteProcessSession && executionTarget?.kind === "remote"
? executionTarget.remoteCwd
: cwd;
: acpxAgent === "gemini" && !executionTargetIsRemote
? await fs.realpath(cwd)
: cwd;
// The 17 fields the session fingerprint hashes. Company, agent, and task
// identifiers are NOT here; they scope the outer session key only (see
// `keyIdentity`). The fingerprint builder accepts only this identity.
@@ -2556,6 +2565,28 @@ async function buildRuntime(input: {
await emitRunPhaseTiming(input.ctx, "start_transport", nowMs() - startTransportStart, "failed");
throw err;
}
const currentSkillRoot = acpxAgent === "codex" && runtimeEnv.CODEX_HOME
? path.join(runtimeEnv.CODEX_HOME, "skills")
: acpxAgent === "gemini" && !executionTargetIsRemote && typeof skillsIdentity.skillsHome === "string"
? skillsIdentity.skillsHome
: null;
if (currentSkillRoot && Array.isArray(skillsIdentity.selectedSkills) && skillsIdentity.selectedSkills.length > 0) {
// Managed connection homes are disposable. A resumed conversation may
// remember a previous home's skill paths; supply the current root on every
// turn, after the remote launch environment has rebased CODEX_HOME.
skillPromptInstructions = [
`Paperclip has materialized selected runtime skills for this ACPX ${acpxAgent === "codex" ? "Codex" : "Gemini"} run.`,
`Skill root for this run: ${currentSkillRoot}`,
`Selected skills: ${skillsIdentity.selectedSkills.join(", ")}`,
"When a task calls for one of these skills, read its SKILL.md from this root and follow it. Resolve referenced scripts and files relative to that skill. Use this run's root even when an earlier turn used a different path.",
...(acpxAgent === "gemini" && skillsIdentity.selectedSkills.includes("paperclip")
? [
"Before calling Paperclip APIs or delivering task artifacts, read the selected paperclip skill's SKILL.md and its referenced artifact workflow. A completion comment does not upload an artifact or change task status.",
"Gemini's shell tool rejects command substitution. Write JSON request bodies to a workspace file and use curl --data-binary @file, or use the selected skill's helpers, instead of constructing a body with command substitution.",
]
: []),
].join("\n");
}
// The relay runs on the host with the sanitized remote launch environment.
// Its /usr/bin/env node shebang cannot rely on that environment's PATH.
const overrideCommand = processSessionBridge?.agentCommand
@@ -2623,13 +2654,14 @@ async function buildRuntime(input: {
function sessionConfigOptions(prepared: AcpxPreparedRuntime): Array<{ key: string; value: string }> {
const options: Array<{ key: string; value: string }> = [];
// Claude and Codex runtime config is pre-set via startup env vars; skip
// Claude, Codex and Gemini runtime config is pre-set via startup env vars; skip
// set_config_option to avoid ACP-server picker validation rejecting valid
// backend model IDs that are not advertised by the local ACP server.
if (
prepared.requestedModel &&
prepared.acpxAgent !== "claude" &&
prepared.acpxAgent !== "codex"
prepared.acpxAgent !== "codex" &&
prepared.acpxAgent !== "gemini"
) {
options.push({ key: "model", value: prepared.requestedModel });
}
@@ -4775,6 +4807,8 @@ export function createAcpxEngineExecutor(deps: AcpxEngineExecutorOptions = {}) {
? `Requested ACPX model: ${prepared.requestedModel} (set via ANTHROPIC_MODEL env at startup).`
: prepared.acpxAgent === "codex"
? `Requested ACPX model: ${prepared.requestedModel} (set via CODEX_CONFIG at startup).`
: prepared.acpxAgent === "gemini"
? `Requested ACPX model: ${prepared.requestedModel} (set via GEMINI_MODEL at startup).`
: `Requested ACPX model: ${prepared.requestedModel}.`,
]
: []),
packages/adapter-utils/src/acpx-engine/filesystem-protocol.test.ts
+110
View File
@@ -0,0 +1,110 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { createRequire } from "node:module";
import { afterEach, describe, expect, it } from "vitest";
import { createAcpRuntime, createAgentRegistry, createRuntimeStore } from "acpx/runtime";
const runnerRuntimePath = createRequire(new URL("../../../paperclip-runner/package.json", import.meta.url)).resolve("acpx/runtime");
const nativeRuntime = await import(runnerRuntimePath) as typeof import("acpx/runtime");
const runtimes = [
{ version: "0.12.0", api: { createAcpRuntime, createAgentRegistry, createRuntimeStore } },
{ version: "0.13.1", api: nativeRuntime },
];
const roots: string[] = [];
const fixture = fileURLToPath(new URL("../../../../scripts/mcp-fixtures/servers/acp-filesystem-agent.mjs", import.meta.url));
afterEach(async () => {
await Promise.all(roots.splice(0).map((root) => fs.rm(root, { recursive: true, force: true })));
});
async function workspace() {
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-acp-filesystem-"));
roots.push(root);
const cwd = path.join(root, "workspace");
await fs.mkdir(cwd);
return { root, cwd: await fs.realpath(cwd) };
}
async function drive(api: (typeof runtimes)[number]["api"], input: {
cwd: string;
root: string;
permissionMode?: "approve-all" | "deny-all";
operations: Array<{ method: string; path: string; content?: string }>;
}) {
let stderr = "";
const runtime = api.createAcpRuntime({
cwd: input.cwd,
sessionStore: api.createRuntimeStore({ stateDir: path.join(input.root, "state") }),
agentRegistry: api.createAgentRegistry({ overrides: { custom: `${JSON.stringify(process.execPath.replaceAll("\\", "/"))} ${JSON.stringify(fixture.replaceAll("\\", "/"))}` } }),
permissionMode: input.permissionMode ?? "approve-all",
nonInteractivePermissions: "deny",
inheritProcessEnv: false,
onAgentStderr: (chunk: string) => { stderr += chunk; },
});
const handle = await runtime.ensureSession({
sessionKey: "filesystem-protocol", agent: "custom", mode: "oneshot",
cwd: input.cwd, sessionOptions: { env: {} },
});
try {
for await (const _event of runtime.runTurn({
handle, text: JSON.stringify({ operations: input.operations }),
mode: "prompt", requestId: "filesystem-protocol", timeoutMs: 5_000,
})) { /* Drain the real ACP turn. */ }
return stderr.split("\n").filter((line) => line.startsWith("FS_RESULT "))
.map((line) => JSON.parse(line.slice("FS_RESULT ".length)));
} finally {
await runtime.close({ handle, reason: "done" });
}
}
describe.each(runtimes)("ACPX $version filesystem protocol", ({ api }) => {
it("reports a missing file as an ACP resource error and then creates and reads it", async () => {
const input = await workspace();
const file = path.join(input.cwd, "new.json");
const content = '{"proof":"synthetic"}\n';
const receipts = await drive(api, { ...input, operations: [
{ method: "fs/read_text_file", path: file },
{ method: "fs/write_text_file", path: file, content },
{ method: "fs/read_text_file", path: file },
] });
expect(receipts).toHaveLength(3);
expect(receipts[0].error).toMatchObject({ code: -32002, message: expect.stringContaining("Resource not found") });
expect(receipts[1].error).toBeUndefined();
expect(receipts[2].result.content).toBe(content);
expect(await fs.readFile(file, "utf8")).toBe(content);
});
it("keeps unrelated paths denied for both reads and writes", async () => {
const input = await workspace();
const outside = path.join(await fs.realpath(input.root), "outside.json");
await fs.writeFile(outside, "unchanged");
const receipts = await drive(api, { ...input, operations: [
{ method: "fs/read_text_file", path: outside },
{ method: "fs/write_text_file", path: outside, content: "changed" },
] });
expect(receipts).toHaveLength(2);
expect(receipts.every((receipt) => receipt.error && receipt.error.code !== -32002)).toBe(true);
expect(await fs.readFile(outside, "utf8")).toBe("unchanged");
});
it("preserves deny-all permissions instead of converting denials into missing files", async () => {
const input = await workspace();
const file = path.join(input.cwd, "denied.json");
const receipts = await drive(api, { ...input, permissionMode: "deny-all", operations: [
{ method: "fs/read_text_file", path: file },
{ method: "fs/write_text_file", path: file, content: "denied" },
] });
expect(receipts).toHaveLength(2);
expect(receipts.every((receipt) => receipt.error && receipt.error.code !== -32002)).toBe(true);
await expect(fs.stat(file)).rejects.toMatchObject({ code: "ENOENT" });
});
it.skipIf(process.platform === "win32")("keeps other filesystem failures distinct from missing files", async () => {
const input = await workspace();
const receipts = await drive(api, { ...input, operations: [{ method: "fs/read_text_file", path: input.cwd }] });
expect(receipts).toHaveLength(1);
expect(receipts[0].error.code).not.toBe(-32002);
});
});
packages/adapters/gemini-local/src/index.ts
+4
View File
@@ -60,6 +60,10 @@ Operational fields:
- graceSec (number, optional): SIGTERM grace period in seconds
Notes:
- Explicit ACP models are set via \`GEMINI_MODEL\` before startup; this avoids unsupported \`session/set_config_option\` calls while preserving the ACP transport. Auto continues to use the installed CLI's model selection.
- Local turns include the current selected skill root and Paperclip artifact workflow guidance, including after a disposable connection home changes.
- Local ACP workspaces use their canonical directory so Gemini's resolved file paths match the client boundary. Missing files retain ACP resource-not-found semantics. Gemini CLI 0.58.0 still mishandles the plain error object during native new-file creation; the installed-CLI compatibility smoke documents this remaining limitation.
- Gemini shell commands may reject command substitution. Build JSON request bodies in workspace files and pass them with \`curl --data-binary @file\`, or use the selected skill's helpers.
- Gemini ACP is the preferred auto lane when Node >=24.11.0 and the local Gemini CLI command is available. It runs Gemini CLI's native \`gemini --acp\` server through Paperclip's shared ACP engine, including selected skill links, Paperclip runtime prompt/env guidance, model config, and persistent ACP session state. Missing prerequisites fail both default and explicit ACP runs with an actionable setup error; the adapter never switches engines automatically.
- Runs use --prompt for non-interactive execution, not stdin.
- The adapter sets a headless-safe terminal/browser environment for Gemini CLI child processes so unattended runs do not wait on browser auth or 256-color terminal prompts.
packages/adapters/grok-local/src/index.ts
+1
View File
@@ -53,4 +53,5 @@ Notes:
- Use \`grok models\` to inspect authentication and available models on the host.
- Local subscription runs use the host \`grok login\` (\`~/.grok\`) until the company Grok home has a usable \`auth.json\` (sandbox device login). \`XAI_API_KEY\` authenticates without a home. Remote/sandbox runs never fall back to the host login.
- Without a usable company login, local runs preserve an inherited or configured \`GROK_HOME\`. Managed AI connections keep their selected home. An explicit empty \`XAI_API_KEY\` clears an inherited key and selects subscription authentication.
- Managed local connections use disposable homes and do not retain or restore session transcripts. Missing local history starts a fresh task handoff without attempting remote subscription recovery.
`;
packages/adapters/grok-local/src/server/execute.test.ts
+25
View File
@@ -163,6 +163,18 @@ async function makeCtx(runId: string, cwd: string): Promise<AdapterExecutionCont
}
describe("grok_local execute", () => {
it.each([false, true])("resumes a managed connection only when its isolated history exists (%s)", async present => {
const root = await makeTempRoot();
const ctx = await makeCtx("managed-resume", root);
ctx.config.managedAiConnection = true;
ctx.config.env = { GROK_HOME: path.join(root, "home"), XAI_API_KEY: "fixture-key" };
ctx.runtime.sessionParams = { sessionId: "fixture-session", cwd: root };
if (present) await fs.mkdir(path.join(root, "home", "sessions", "encoded-cwd", "fixture-session"), { recursive: true });
runProcessMock.mockResolvedValue(makeSuccessfulRunResult({ sessionId: "next-session" }));
const result = await execute(ctx);
expect((runProcessMock.mock.calls[0][3] as string[]).includes("--resume")).toBe(present);
expect(result.sessionParams?.sessionId).toBe("next-session");
});
it("resumes the conversation while rotating runtime tool access", async () => {
const root = await makeTempRoot();
runProcessMock.mockResolvedValue(makeSuccessfulRunResult({ sessionId: "existing-session" }));
@@ -180,6 +192,19 @@ describe("grok_local execute", () => {
expect(result.sessionParams?.sessionId).toBe("existing-session");
});
it("clears an unavailable managed session when the fresh turn provides no replacement ID", async () => {
const root = await makeTempRoot();
const ctx = await makeCtx("missing-history", root);
ctx.config.managedAiConnection = true;
ctx.config.env = { GROK_HOME: path.join(root, "home"), XAI_API_KEY: "fixture-key" };
ctx.runtime.sessionParams = { sessionId: "discarded-session", cwd: root };
runProcessMock.mockResolvedValue({ ...makeSuccessfulRunResult(), stdout: JSON.stringify({ type: "end", stopReason: "EndTurn" }) });
const result = await execute(ctx);
expect((runProcessMock.mock.calls[0][3] as string[]).includes("--resume")).toBe(false);
expect(result.sessionParams).toBeNull();
expect(result.clearSession).toBe(true);
});
it.each(["grok-4.7", "grok-4.6"])("forwards the explicit %s model and xhigh effort", async (model) => {
const root = await makeTempRoot();
const ctx = await makeCtx("model-selection", root);
packages/adapters/grok-local/src/server/execute.ts
+9 -4
View File
@@ -47,7 +47,7 @@ import {
} from "@paperclipai/adapter-utils/server-utils";
import { DEFAULT_GROK_LOCAL_MODEL } from "../index.js";
import { copyBackGrokAuth } from "./grok-auth-copyback.js";
import { grokHomeHasUsableAuth, resolveManagedGrokHomeDir, stageGrokHomeForSync } from "./grok-home.js";
import { grokHomeHasSession, grokHomeHasUsableAuth, resolveManagedGrokHomeDir, stageGrokHomeForSync } from "./grok-home.js";
import { isGrokUnknownSessionError, parseGrokJsonl } from "./parse.js";
const __moduleDir = path.dirname(fileURLToPath(import.meta.url));
@@ -501,12 +501,17 @@ async function executeTurn(ctx: AdapterExecutionContext): Promise<AdapterExecuti
const runtimeSessionId = asString(runtimeSessionParams.sessionId, runtime.sessionId ?? "");
const runtimeSessionCwd = asString(runtimeSessionParams.cwd, "");
const runtimeRemoteExecution = parseObject(runtimeSessionParams.remoteExecution);
const missingManagedSession = !executionTargetIsRemote && Boolean(config.managedAiConnection) &&
Boolean(runtimeSessionId) && !await grokHomeHasSession(asString(env.GROK_HOME, ""), runtimeSessionId);
const canResumeSession =
runtimeSessionId.length > 0 &&
!missingManagedSession &&
(runtimeSessionCwd.length === 0 || path.resolve(runtimeSessionCwd) === path.resolve(effectiveExecutionCwd)) &&
adapterExecutionTargetSessionMatches(runtimeRemoteExecution, runtimeExecutionTarget);
const sessionId = canResumeSession ? runtimeSessionId : null;
if (executionTargetIsRemote && runtimeSessionId && !canResumeSession) {
if (missingManagedSession) {
await onLog("stdout", "[paperclip] Selected Grok session history is unavailable. Starting a fresh session with the task handoff instead of remote subscription recovery.\n");
} else if (executionTargetIsRemote && runtimeSessionId && !canResumeSession) {
await onLog(
"stdout",
`[paperclip] Grok session "${runtimeSessionId}" does not match the current remote execution identity and will not be resumed in "${effectiveExecutionCwd}". Starting a fresh remote session.\n`,
@@ -653,7 +658,7 @@ async function executeTurn(ctx: AdapterExecutionContext): Promise<AdapterExecuti
stderrLine ||
`Grok exited with code ${attempt.proc.exitCode ?? -1}`;
const canFallbackToRuntimeSession = !isRetry;
const canFallbackToRuntimeSession = !isRetry && !missingManagedSession;
const resolvedSessionId = attempt.parsed.sessionId
?? (canFallbackToRuntimeSession ? (runtimeSessionId ?? runtime.sessionId ?? null) : null);
const resolvedSessionParams = resolvedSessionId
@@ -722,7 +727,7 @@ async function executeTurn(ctx: AdapterExecutionContext): Promise<AdapterExecuti
return toResult(retry, true, true);
}
return toResult(initial);
return toResult(initial, missingManagedSession);
};
try {
packages/adapters/grok-local/src/server/grok-home.test.ts
+11 -1
View File
@@ -3,7 +3,7 @@ import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it, vi } from "vitest";
import { GROK_SYNC_ALLOWLIST, stageGrokHomeForSync } from "./grok-home.js";
import { GROK_SYNC_ALLOWLIST, grokHomeHasSession, stageGrokHomeForSync } from "./grok-home.js";
describe("GROK_SYNC_ALLOWLIST", () => {
it("holds exactly one name: auth.json", () => {
@@ -29,6 +29,16 @@ describe("stageGrokHomeForSync", () => {
return root;
}
it("starts fresh instead of throwing when private history exceeds the lookup bound", async () => {
const root = await makeRoot("paperclip-grok-large-history-");
const sessions = path.join(root, "sessions");
await fs.mkdir(sessions);
for (let offset = 0; offset < 5001; offset += 100) {
await Promise.all(Array.from({ length: Math.min(100, 5001 - offset) }, (_, index) => fs.writeFile(path.join(sessions, `other-${offset + index}.json`), "{}")));
}
await expect(grokHomeHasSession(root, "requested-session")).resolves.toBe(false);
});
it("stages auth.json and stages no other file", async () => {
const root = await makeRoot("paperclip-grok-stage-");
const home = path.join(root, "grok-home");
packages/adapters/grok-local/src/server/grok-home.ts
+25 -2
View File
@@ -3,8 +3,8 @@ import os from "node:os";
import path from "node:path";
import { resolvePaperclipInstanceRootForAdapter } from "@paperclipai/adapter-utils/server-utils";
// The Grok credential home. `GROK_HOME` replaces `~/.grok` and holds one file,
// `auth.json`. Unlike Codex, a Grok `auth.json` has no fixed top-level key: it
// `GROK_HOME` replaces `~/.grok` and holds `auth.json` plus session history.
// Unlike Codex, a Grok `auth.json` has no fixed top-level key: it
// holds exactly one key, and that key is a composite `<issuer>::<uuid>` value.
// This module resolves the company-scoped home path and reads its usable-auth
// shape. It never writes to the managed home itself; {@link
@@ -14,6 +14,29 @@ import { resolvePaperclipInstanceRootForAdapter } from "@paperclipai/adapter-uti
const AUTH_FILE_NAME = "auth.json";
/** Inspect only the selected private home, never Grok's remote registry. */
export async function grokHomeHasSession(home: string, sessionId: string): Promise<boolean> {
if (!home || !/^[a-zA-Z0-9_-]{1,128}$/.test(sessionId)) return false;
let remaining = 5000;
async function inspect(directory: string, depth: number): Promise<boolean> {
let entries;
try { entries = await fs.opendir(directory); }
catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") return false;
throw error;
}
for await (const entry of entries) {
if (--remaining < 0) return false;
if (entry.isSymbolicLink()) continue;
if ((entry.isFile() || entry.isDirectory()) &&
(entry.name === sessionId || entry.name.startsWith(`${sessionId}.`))) return true;
if (entry.isDirectory() && depth > 0 && await inspect(path.join(directory, entry.name), depth - 1)) return true;
}
return false;
}
return inspect(path.join(home, "sessions"), 2);
}
/**
* The allowlist of managed `GROK_HOME` entries that the grok-local adapter
* stages into the sandbox `home` asset (see {@link stageGrokHomeForSync}).
packages/adapters/hermes/README.md
+1
View File
@@ -237,6 +237,7 @@ Available toolsets: `terminal`, `file`, `web`, `browser`, `code_execution`, `vis
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `cwd` | string | *(assigned task workspace)* | Absolute working directory override. Without an override, use Paperclip's resolved task workspace. |
| `persistSession` | boolean | `true` | Resume sessions across heartbeats |
| `worktreeMode` | boolean | `false` | Git worktree isolation |
| `checkpoints` | boolean | `false` | Enable filesystem checkpoints for rollback |
packages/adapters/hermes/src/index.ts
+1
View File
@@ -99,6 +99,7 @@ tools, persistent memory, session persistence, skills, and MCP support.
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| cwd | string | (assigned task workspace) | Absolute working directory override; otherwise use Paperclip's resolved task workspace |
| persistSession | boolean | true | Resume sessions across heartbeats |
| worktreeMode | boolean | false | Use git worktree for isolated changes |
| checkpoints | boolean | false | Enable filesystem checkpoints |
packages/adapters/hermes/src/server/execute.onspawn.test.ts
+12
View File
@@ -113,6 +113,18 @@ describe("hermes-local adapter onSpawn forwarding", () => {
expect(call[2]).toContainEqual(expect.stringContaining("Current task brief"));
});
it("launches in the assigned task workspace when no cwd was configured", async () => {
const { ctx } = makeCtx();
await execute({ ...ctx, context: { ...ctx.context, paperclipWorkspace: { cwd: "/private/qa/agent-workspace" } } } as any);
expect(vi.mocked(serverUtils.runChildProcess).mock.calls.at(-1)![3].cwd).toBe("/private/qa/agent-workspace");
});
it("preserves an explicitly configured cwd", async () => {
const { ctx } = makeCtx({ cwd: "/private/qa/configured-workspace" });
await execute({ ...ctx, context: { ...ctx.context, paperclipWorkspace: { cwd: "/private/qa/agent-workspace" } } } as any);
expect(vi.mocked(serverUtils.runChildProcess).mock.calls.at(-1)![3].cwd).toBe("/private/qa/configured-workspace");
});
it("runChildProcess opts type includes onSpawn", () => {
// Type-level assertion: if onSpawn were removed from the type,
// this file would fail to compile. The runtime test above catches
packages/adapters/hermes/src/server/execute.ts
+5 -1
View File
@@ -510,8 +510,12 @@ export async function execute(
if (envCommentId) env.PAPERCLIP_WAKE_COMMENT_ID = envCommentId;
// ── Resolve working directory ──────────────────────────────────────────
const workspace = ctx.context?.paperclipWorkspace;
const workspaceCwd = workspace && typeof workspace === "object"
? cfgString((workspace as Record<string, unknown>).cwd)
: undefined;
const cwd =
cfgString(config.cwd) || cfgString(ctx.config?.workspaceDir) || ".";
cfgString(config.cwd) || workspaceCwd || cfgString(ctx.config?.workspaceDir) || ".";
try {
await ensureAbsoluteDirectory(cwd);
} catch {
patches/acpx@0.12.0.patch
+19 -10
View File
@@ -1,8 +1,7 @@
diff --git a/dist/live-checkpoint-ClPCSdrW.js b/dist/live-checkpoint-ClPCSdrW.js
index 243c9d13bcba520923b63adfddad75cf2d94362d..e2a254d4e57fb2e246ccb742f2e9de5a8a461d20 100644
--- a/dist/live-checkpoint-ClPCSdrW.js
+++ b/dist/live-checkpoint-ClPCSdrW.js
@@ -1532,7 +1532,7 @@ const ZED_TAG_KEYS = /* @__PURE__ */ new Set([
@@ -1532,7 +1532,7 @@
"RedactedThinking",
"ToolUse"
]);
@@ -11,7 +10,7 @@ index 243c9d13bcba520923b63adfddad75cf2d94362d..e2a254d4e57fb2e246ccb742f2e9de5a
const OPAQUE_VALUE_PATHS = /* @__PURE__ */ new Set([
"agent_capabilities",
"messages.Agent.content.ToolUse.input",
@@ -2557,7 +2557,7 @@ function readCommandLineChar(state) {
@@ -2557,7 +2557,7 @@
escaping: false,
hasPart: true
};
@@ -20,7 +19,7 @@ index 243c9d13bcba520923b63adfddad75cf2d94362d..e2a254d4e57fb2e246ccb742f2e9de5a
current: state.current,
quote: state.quote,
escaping: true,
@@ -2912,8 +2912,8 @@ function promotePrefixedAuthEnvironment(env) {
@@ -2912,8 +2912,8 @@
}
return protectedKeys;
}
@@ -31,7 +30,7 @@ index 243c9d13bcba520923b63adfddad75cf2d94362d..e2a254d4e57fb2e246ccb742f2e9de5a
const protectedAuthEnvKeys = promotePrefixedAuthEnvironment(env);
if (authCredentials) for (const [methodId, credential] of Object.entries(authCredentials)) {
addAuthCredentialEnvKeys(protectedAuthEnvKeys, methodId, credential);
@@ -2955,10 +2955,10 @@ function resolveConfiguredAuthCredential(methodId, authCredentials) {
@@ -2955,10 +2955,10 @@
const configCredentials = authCredentials ?? {};
return configCredentials[methodId] ?? configCredentials[toEnvToken(methodId)];
}
@@ -44,7 +43,7 @@ index 243c9d13bcba520923b63adfddad75cf2d94362d..e2a254d4e57fb2e246ccb742f2e9de5a
stdio: [
"pipe",
"pipe",
@@ -3724,10 +3724,13 @@ function resolveClientCapabilities(params) {
@@ -3724,10 +3724,13 @@
},
terminal: params.terminal
};
@@ -60,7 +59,7 @@ index 243c9d13bcba520923b63adfddad75cf2d94362d..e2a254d4e57fb2e246ccb742f2e9de5a
};
}
function isDevinRequestDiagnosticsMethod(method) {
@@ -3957,9 +3960,26 @@ var AcpClient = class {
@@ -3957,9 +3960,26 @@
this.lastAgentExit = void 0;
this.lastKnownPid = child.pid ?? void 0;
this.attachAgentLifecycleObservers(child);
@@ -87,7 +86,7 @@ index 243c9d13bcba520923b63adfddad75cf2d94362d..e2a254d4e57fb2e246ccb742f2e9de5a
if (!this.options.verbose) return;
process.stderr.write(chunk);
});
@@ -3994,7 +4014,7 @@ var AcpClient = class {
@@ -3994,7 +4014,7 @@
geminiAcp: isGeminiAcpCommand(spawnCommand, args),
copilotAcp: isCopilotAcpCommand(spawnCommand, args),
claudeAcp: isClaudeAcpCommand(spawnCommand, args),
@@ -96,7 +95,17 @@ index 243c9d13bcba520923b63adfddad75cf2d94362d..e2a254d4e57fb2e246ccb742f2e9de5a
};
}
logAgentLaunch(plan) {
@@ -6070,6 +6090,27 @@ async function withConnectedSession(options) {
@@ -4729,6 +4749,9 @@
return await this.filesystem.readTextFile(params);
} catch (error) {
this.recordPermissionError(params.sessionId, error);
+ // Keep ENOENT distinct from permission and other filesystem failures.
+ // The ACP peer must receive a resource error before it creates a file.
+ if (error && typeof error === "object" && error.code === "ENOENT") throw RequestError.resourceNotFound(params.path);
throw error;
}
}
@@ -6070,6 +6093,27 @@
//#region src/runtime/engine/prompt-turn.ts
const SESSION_REPLY_IDLE_MS = 1e3;
const SESSION_REPLY_DRAIN_TIMEOUT_MS = 5e3;
@@ -124,7 +133,7 @@ index 243c9d13bcba520923b63adfddad75cf2d94362d..e2a254d4e57fb2e246ccb742f2e9de5a
async function runPromptTurn(params) {
try {
const promptPromise = params.client.prompt(params.sessionId, params.prompt);
@@ -6079,6 +6120,20 @@ async function runPromptTurn(params) {
@@ -6079,6 +6123,20 @@
idleMs: SESSION_REPLY_IDLE_MS,
timeoutMs: SESSION_REPLY_DRAIN_TIMEOUT_MS
}).catch(() => {});
patches/acpx@0.13.1.patch
+52 -34
View File
@@ -11,10 +11,18 @@ index 5e2113ac0a1a92c99322cf01e5c106760a0b0dbf..b7b5151f3da45b5e0d12aea55e9f3b07
private setSessionModelThroughConfig;
private setSessionModelThroughLegacyMethod;
diff --git a/dist/live-checkpoint-BSIrfgVo.js b/dist/live-checkpoint-BSIrfgVo.js
index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d2e777e9e 100644
--- a/dist/live-checkpoint-BSIrfgVo.js
+++ b/dist/live-checkpoint-BSIrfgVo.js
@@ -1068,6 +1068,7 @@ function serializeSessionRecordForDisk(record) {
@@ -6,7 +6,7 @@
import { randomUUID } from "node:crypto";
import { execFile, spawn } from "node:child_process";
import { Readable, Writable } from "node:stream";
-import { PROTOCOL_VERSION, client, methods } from "@agentclientprotocol/sdk";
+import { PROTOCOL_VERSION, RequestError, client, methods } from "@agentclientprotocol/sdk";
import readline from "node:readline/promises";
//#region src/errors.ts
var AcpxOperationalError = class extends Error {
@@ -1068,6 +1068,7 @@
last_agent_disconnect_reason: canonical.lastAgentDisconnectReason,
protocol_version: canonical.protocolVersion,
agent_capabilities: canonical.agentCapabilities,
@@ -22,7 +30,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
title: canonical.title,
messages: canonical.messages,
updated_at: canonical.updated_at,
@@ -1239,7 +1240,8 @@ function parseRequestTokenUsage(raw) {
@@ -1239,7 +1240,8 @@
for (const [key, value] of Object.entries(record)) {
const parsed = parseTokenUsage(value);
if (parsed == null) return null;
@@ -32,7 +40,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
}
return usage;
}
@@ -1541,6 +1543,7 @@ function parseSessionRecord(raw) {
@@ -1541,6 +1543,7 @@
lastAgentDisconnectReason: optionals.lastAgentDisconnectReason,
protocolVersion: typeof record.protocol_version === "number" ? record.protocol_version : void 0,
agentCapabilities: asRecord$4(record.agent_capabilities),
@@ -40,7 +48,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
title: conversation.title,
messages: conversation.messages,
updated_at: conversation.updated_at,
@@ -1661,7 +1664,7 @@ const ZED_TAG_KEYS = /* @__PURE__ */ new Set([
@@ -1661,7 +1664,7 @@
"RedactedThinking",
"ToolUse"
]);
@@ -49,7 +57,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
const OPAQUE_VALUE_PATHS = /* @__PURE__ */ new Set([
"agent_capabilities",
"messages.Agent.content.ToolUse.input",
@@ -3104,10 +3107,10 @@ function authEnvKey(methodId) {
@@ -3104,10 +3107,10 @@
authEnvKeyCache.set(methodId, key);
return key;
}
@@ -62,7 +70,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
if (typeof value === "string" && value.trim().length > 0) return value;
}
function protectedEnvKey(key) {
@@ -3135,8 +3138,21 @@ function promotePrefixedAuthEnvironment(env) {
@@ -3135,8 +3138,21 @@
}
return protectedKeys;
}
@@ -86,7 +94,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
const protectedAuthEnvKeys = promotePrefixedAuthEnvironment(env);
if (authCredentials) for (const [methodId, credential] of Object.entries(authCredentials)) {
addAuthCredentialEnvKeys(protectedAuthEnvKeys, methodId, credential);
@@ -3178,10 +3194,10 @@ function resolveConfiguredAuthCredential(methodId, authCredentials) {
@@ -3178,10 +3194,10 @@
const configCredentials = authCredentials ?? {};
return configCredentials[methodId] ?? configCredentials[toEnvToken(methodId)];
}
@@ -99,7 +107,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
stdio: [
"pipe",
"pipe",
@@ -3961,10 +3977,13 @@ function resolveClientCapabilities(params) {
@@ -3961,10 +3977,13 @@
...params.elicitationModes.includes("url") ? { url: {} } : {}
} } : {}
};
@@ -115,7 +123,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
};
}
function hasResponseField(response, field) {
@@ -4067,6 +4086,27 @@ function createNdJsonMessageStream(agentCommand, output, input) {
@@ -4067,6 +4086,27 @@
} })
};
}
@@ -143,7 +151,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
var AcpClient = class {
options;
connection;
@@ -4105,7 +4145,9 @@ var AcpClient = class {
@@ -4105,7 +4145,9 @@
cwd: asAbsoluteCwd(options.cwd),
authPolicy: options.authPolicy ?? "skip",
permissionPolicy: snapshotPermissionPolicy(options.permissionPolicy),
@@ -154,7 +162,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
};
this.eventHandlers = {
onAcpMessage: this.options.onAcpMessage,
@@ -4216,9 +4258,21 @@ var AcpClient = class {
@@ -4216,9 +4258,21 @@
this.lastAgentExit = void 0;
this.lastKnownPid = child.pid ?? void 0;
this.attachAgentLifecycleObservers(child);
@@ -176,7 +184,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
if (!this.options.verbose) return;
process.stderr.write(chunk);
});
@@ -4226,6 +4280,7 @@ var AcpClient = class {
@@ -4226,6 +4280,7 @@
const output = Readable.toWeb(child.stdout);
const stream = this.createTappedStream(createNdJsonMessageStream(this.options.agentCommand, input, output));
const connection = this.createConnection(stream, launch);
@@ -184,7 +192,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
connection.signal.addEventListener("abort", () => {
this.recordAgentExit("connection_close", child.exitCode ?? null, child.signalCode ?? null);
}, { once: true });
@@ -4253,7 +4308,12 @@ var AcpClient = class {
@@ -4253,7 +4308,12 @@
geminiAcp: isGeminiAcpCommand(spawnCommand, args),
copilotAcp: isCopilotAcpCommand(spawnCommand, args),
claudeAcp: isClaudeAcpCommand(spawnCommand, args),
@@ -198,7 +206,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
};
}
logAgentLaunch(plan) {
@@ -4279,10 +4339,17 @@ var AcpClient = class {
@@ -4279,10 +4339,17 @@
}
async spawnAgentProcess(plan) {
const spawnCommand = buildAgentSpawnCommand(plan.spawnCommand, plan.args, process.platform, plan.spawnOptions.env);
@@ -218,7 +226,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
try {
await waitForSpawn$1(spawnedChild);
} catch (error) {
@@ -4293,10 +4360,10 @@ var AcpClient = class {
@@ -4293,10 +4360,10 @@
createConnection(stream, launch) {
const app = client({ name: "acpx" }).onNotification(methods.client.session.update, async ({ params }) => {
await this.handleSessionUpdate(params);
@@ -232,7 +240,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
}).onRequest(methods.client.fs.readTextFile, async ({ params }) => {
return await this.handleReadTextFile(params);
}).onRequest(methods.client.fs.writeTextFile, async ({ params }) => {
@@ -4312,6 +4379,11 @@ var AcpClient = class {
@@ -4312,6 +4379,11 @@
}).onRequest(methods.client.terminal.release, async ({ params }) => {
return await this.handleReleaseTerminal(params);
});
@@ -244,7 +252,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
if (launch.devinAcp) app.onRequest("_cognition.ai/request_diagnostics", (params) => {
return params && typeof params === "object" && !Array.isArray(params) ? params : {};
}, async () => ({}));
@@ -4333,12 +4405,12 @@ var AcpClient = class {
@@ -4333,12 +4405,12 @@
async initializeProtocolConnection(connection, launch) {
const initializePromise = connection.initialize({
protocolVersion: PROTOCOL_VERSION,
@@ -259,7 +267,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
clientInfo: resolveClientInfo(launch.devinAcp)
});
const initialized = launch.geminiAcp ? await withTimeout(initializePromise, resolveGeminiAcpStartupTimeoutMs()) : await initializePromise;
@@ -4358,6 +4430,36 @@ var AcpClient = class {
@@ -4358,6 +4430,36 @@
throw normalizedError;
}
createTappedStream(base) {
@@ -296,7 +304,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
const onAcpMessage = () => this.eventHandlers.onAcpMessage;
const onAcpOutputMessage = () => this.eventHandlers.onAcpOutputMessage;
const elicitationRequestIds = /* @__PURE__ */ new Set();
@@ -4367,8 +4469,9 @@ var AcpClient = class {
@@ -4367,8 +4469,9 @@
const shouldSuppressInboundReplaySessionUpdate = (message) => {
return this.suppressReplaySessionUpdateMessages && isSessionUpdateNotification(message);
};
@@ -307,7 +315,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
if (requestId !== void 0) {
elicitationRequestIds.add(requestId);
return;
@@ -4379,6 +4482,7 @@ var AcpClient = class {
@@ -4379,6 +4482,7 @@
}
};
return {
@@ -315,7 +323,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
readable: new ReadableStream({ async start(controller) {
const reader = base.readable.getReader();
try {
@@ -4387,10 +4491,16 @@ var AcpClient = class {
@@ -4387,10 +4491,16 @@
if (done) break;
if (!value) continue;
observeInbound(value);
@@ -333,7 +341,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
reader.releaseLock();
controller.close();
}
@@ -4403,10 +4513,20 @@ var AcpClient = class {
@@ -4403,10 +4513,20 @@
onAcpOutputMessage()?.("outbound", message);
onAcpMessage()?.("outbound", message);
}
@@ -354,7 +362,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
writer.releaseLock();
}
} })
@@ -4422,7 +4542,10 @@ var AcpClient = class {
@@ -4422,7 +4542,10 @@
const createPromise = this.runConnectionRequest(() => connection.newSession({
cwd: sessionCwd,
mcpServers: this.options.mcpServers ?? [],
@@ -366,7 +374,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
}));
result = claudeAcp ? await withTimeout(createPromise, resolveClaudeAcpSessionCreateTimeoutMs()) : await createPromise;
} catch (error) {
@@ -4864,7 +4987,7 @@ var AcpClient = class {
@@ -4864,7 +4987,7 @@
}
selectAuthMethod(methods) {
for (const method of methods) {
@@ -375,7 +383,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
if (envCredential) return {
methodId: method.id,
credential: envCredential,
@@ -4890,7 +5013,7 @@ var AcpClient = class {
@@ -4890,7 +5013,7 @@
}
readAgentSpecificEnvCredential(methodId) {
if (!this.isGrokBuildAcpCommand() || methodId !== "xai.api_key") return;
@@ -384,7 +392,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
return typeof value === "string" && value.trim().length > 0 ? value : void 0;
}
selectAgentManagedAuthMethod(methodId) {
@@ -4915,9 +5038,9 @@ var AcpClient = class {
@@ -4915,9 +5038,9 @@
await connection.authenticate({ methodId: selected.methodId });
this.log(`authenticated with method ${selected.methodId} (${selected.source})`);
}
@@ -396,7 +404,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
if (hostResponse) return hostResponse;
const { response, recorded } = await this.resolvePermissionRequestFromMode(params);
if (!recorded) {
@@ -4926,14 +5049,49 @@ var AcpClient = class {
@@ -4926,14 +5049,49 @@
}
return response;
}
@@ -448,7 +456,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
})).then((response) => ({
kind: "response",
response
@@ -4970,7 +5128,7 @@ var AcpClient = class {
@@ -4970,7 +5128,7 @@
isElicitationSessionCancelling(sessionId) {
return this.closing || this.cancellingSessionIds.has(sessionId);
}
@@ -457,7 +465,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
if (!this.options.onPermissionRequest) return;
const signal = this.cancellationSignalForSession(params.sessionId);
try {
@@ -4978,7 +5136,7 @@ var AcpClient = class {
@@ -4978,7 +5136,7 @@
sessionId: params.sessionId,
raw: params,
inferredKind: inferToolKind(params)
@@ -466,7 +474,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
return this.hostPermissionDecisionResponse(params, signal, decision);
} catch (error) {
return this.hostPermissionErrorResponse(params, signal, error);
@@ -5028,6 +5186,12 @@ var AcpClient = class {
@@ -5028,6 +5186,12 @@
attachAgentLifecycleObservers(child) {
child.once("exit", (exitCode, signal) => {
this.recordAgentExit("process_exit", exitCode, signal);
@@ -479,7 +487,17 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
});
child.once("close", (exitCode, signal) => {
this.recordAgentExit("process_close", exitCode, signal);
@@ -5771,10 +5935,20 @@ function recordSessionUpdate(conversation, state, notification, timestamp = isoN
@@ -5100,6 +5264,9 @@
return await this.filesystem.readTextFile(params);
} catch (error) {
this.recordPermissionError(params.sessionId, error);
+ // Keep ENOENT distinct from permission and other filesystem failures.
+ // The ACP peer must receive a resource error before it creates a file.
+ if (error && typeof error === "object" && error.code === "ENOENT") throw RequestError.resourceNotFound(params.path);
throw error;
}
}
@@ -5771,10 +5938,20 @@
trimConversationForRuntime(conversation);
return acpx;
}
@@ -502,7 +520,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
updateConversationTimestamp(conversation, timestamp);
trimConversationForRuntime(conversation);
return true;
@@ -6435,6 +6609,27 @@ async function withConnectedSession(options) {
@@ -6435,6 +6612,27 @@
//#region src/runtime/engine/prompt-turn.ts
const SESSION_REPLY_IDLE_MS = 1e3;
const SESSION_REPLY_DRAIN_TIMEOUT_MS = 5e3;
@@ -530,7 +548,7 @@ index d454fd7c5bf742b469be75eb8c3988b694a0ffaa..e3a4989473bba20e6dcdccbac086bb6d
async function runPromptTurn(params) {
try {
const promptPromise = params.client.prompt(params.sessionId, params.prompt, params.onPromptRequestStarted, params.onElicitation);
@@ -6444,7 +6639,22 @@ async function runPromptTurn(params) {
@@ -6444,7 +6642,22 @@
idleMs: SESSION_REPLY_IDLE_MS,
timeoutMs: SESSION_REPLY_DRAIN_TIMEOUT_MS
}).catch(() => {});
scripts/mcp-fixtures/servers/acp-filesystem-agent.mjs
+43
View File
@@ -0,0 +1,43 @@
#!/usr/bin/env node
// Credential-free ACP filesystem requests, driven through the real client.
import { createInterface } from "node:readline";
const pending = new Map();
let nextId = 100;
const send = (message) => process.stdout.write(`${JSON.stringify(message)}\n`);
const request = (method, params) => new Promise((resolve) => {
const id = nextId++;
pending.set(id, resolve);
send({ jsonrpc: "2.0", id, method, params });
});
createInterface({ input: process.stdin }).on("line", async (line) => {
const message = JSON.parse(line);
if (pending.has(message.id)) {
pending.get(message.id)(message);
pending.delete(message.id);
return;
}
let result;
if (message.method === "initialize") {
result = {
protocolVersion: 1,
agentCapabilities: { loadSession: false, sessionCapabilities: { close: {} } },
agentInfo: { name: "paperclip-acp-filesystem-fixture", version: "1.0.0" },
};
} else if (message.method === "session/new") {
result = { sessionId: "filesystem-fixture" };
} else if (message.method === "session/prompt") {
const { operations } = JSON.parse(message.params.prompt.find((entry) => entry.type === "text").text);
for (const { method, ...params } of operations) {
const response = await request(method, { sessionId: "filesystem-fixture", ...params });
process.stderr.write(`FS_RESULT ${JSON.stringify(response)}\n`);
}
result = { stopReason: "end_turn" };
} else if (message.method === "session/cancel") {
return;
} else {
result = {};
}
if (message.id !== undefined) send({ jsonrpc: "2.0", id: message.id, result });
});
server/src/__tests__/ai-connections.test.ts
+42 -2
View File
@@ -4,12 +4,14 @@ import { issueRecoveryActionService } from "../services/issue-recovery-actions.j
import { localAiLoginService } from "../services/local-ai-login.js";
import * as localCredentials from "../services/local-ai-credentials.js";
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
import { randomUUID } from "node:crypto";
import { createHash, randomUUID } from "node:crypto";
import { spawnSync } from "node:child_process";
import { localEncryptedProvider } from "../secrets/local-encrypted-provider.js";
import { mkdtemp, realpath, rm, access, readFile, writeFile, stat } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { and, eq, sql } from "drizzle-orm";
import { createDb, companies, agents, heartbeatRuns, companyMemberships, connectionGrants, toolApplications, connectionGrantDelegations, connectionGrantMembers, toolConnections, toolConnectionInstalls, aiConnectionDefaults, aiProviderDefaults, adapterAuthSessions, environments, issues, issueThreadInteractions, issueRecoveryActions, connectionIntentDeliveries, agentWakeupRequests, companySecrets, principalPermissionGrants } from "@paperclipai/db";
import { createDb, companies, agents, agentTaskSessions, heartbeatRuns, companyMemberships, connectionGrants, toolApplications, connectionGrantDelegations, connectionGrantMembers, toolConnections, toolConnectionInstalls, aiConnectionDefaults, aiProviderDefaults, adapterAuthSessions, environments, issues, issueThreadInteractions, issueRecoveryActions, connectionIntentDeliveries, agentWakeupRequests, companySecrets, principalPermissionGrants } from "@paperclipai/db";
import { startEmbeddedPostgresTestDatabase } from "@paperclipai/db/test-embedded-postgres";
import { aiConnectionService } from "../services/ai-connections.js";
import * as executionTarget from "@paperclipai/adapter-utils/execution-target";
@@ -73,6 +75,44 @@ describe("managed AI connections", () => {
expect(grant.credentialSecretRefs).toEqual([]);
} finally { await Promise.all([first.cleanup(), second.cleanup()]); }
});
it("never restores retained Grok transcripts into same-user runtime homes", async () => {
const account = await service.save(companyId, "alice", {
provider: "xai", method: "api_key", ownership: "personal",
name: "Grok history isolation", agentIds: [], allAgents: true,
}, "fixture-grok-key");
const taskKey = `grok-history-${randomUUID()}`;
const runInput = { ...input, taskKey, adapterType: "grok_local", responsibleUserId: "alice", config: {},
binding: { provider: "xai", method: "api_key", mode: "connection", connectionId: account.connectionId, grantId: account.grantId } as const };
const first = await prepareManagedAiRuntime(db, runInput);
const scope = createHash("sha256").update(JSON.stringify([
companyId, agentId, taskKey, first.sessionIdentity,
])).digest("hex");
await first.cleanup();
// Seed a valid archive from the earlier implementation. Runtime preparation
// must not decrypt it onto the host, even for its authorized agent/task.
const retained = await localEncryptedProvider.createVersion({ value: JSON.stringify({
scope, entries: [{ name: "history.json", bytes: Buffer.from("previous-task-private-transcript").toString("base64") }],
}) });
await db.insert(agentTaskSessions).values({ companyId, agentId, adapterType: "grok_local", taskKey,
sessionParamsJson: { sessionId: "history", paperclipGrokHistory: { scope, material: retained.material } } });
const resumed = await prepareManagedAiRuntime(db, runInput);
const peer = await prepareManagedAiRuntime(db, { ...runInput, agentId: randomUUID() });
try {
expect(resumed.home).not.toBe(first.home);
expect(peer.home).not.toBe(resumed.home);
const transcript = path.join(String(resumed.config.env.GROK_HOME), "sessions", "history.json");
// A separate same-UID process can bypass 0700. It must find no restored
// transcript because none was materialized, not because of permissions.
const crossAgentRead = spawnSync(process.execPath, ["-e", `
const fs = require("node:fs");
try { process.stdout.write(fs.readFileSync(process.argv[1], "utf8")); }
catch (error) { process.stdout.write(error.code); }
`, transcript], { encoding: "utf8" });
expect(crossAgentRead.status).toBe(0);
expect(crossAgentRead.stdout).toBe("ENOENT");
await expect(access(transcript)).rejects.toMatchObject({ code: "ENOENT" });
} finally { await Promise.all([resumed.cleanup(), peer.cleanup()]); }
});
it("authenticates local Gemini probes and runs with the saved key in an isolated home", async () => {
const root = await mkdtemp(path.join(home, "gemini-auth-"));
const command = path.join(root, "gemini");
server/src/__tests__/file-delivery-bridges.test.ts
+2 -2
View File
@@ -99,9 +99,9 @@ describe(`durable file delivery (${liveDaytona ? "Daytona" : "local processes"})
enableSandboxDuplexBridge: duplex,
});
expect(bridge?.env.PAPERCLIP_API_BRIDGE_MODE).toBe(duplex ? "http2_v1" : "queue_v1");
const env = { ...bridge!.env, PAPERCLIP_RUN_ID: fixture.runId,
const env = { ...bridge!.env, TMPDIR: path.join(remote.workspace, ".unavailable-controller-temp"), PAPERCLIP_RUN_ID: fixture.runId,
PAPERCLIP_COMPANY_ID: fixture.companyId, PAPERCLIP_TASK_ID: fixture.issueId,
PAPERCLIP_HELPER_STATE_DIR: path.join(remote.workspace, ".helper-state") };
PAPERCLIP_WORKSPACE_CWD: remote.workspace };
const headers = { authorization: `Bearer ${bridge!.env.PAPERCLIP_API_KEY}` };
const receipts: Array<{ attachment: { id: string; contentPath: string; downloadPath: string } }> = [];
try {
skills/paperclip/references/artifacts.md
+5
View File
@@ -156,3 +156,8 @@ confirms storage and response binding in Paperclip; it does not confirm delivery
to an external chat provider. If registration fails, use the returned error to
resolve the failure or explain the limitation; do not describe a workspace path
as an uploaded file.
The shell helper keeps temporary responses and retry receipts in the assigned
workspace's private `.paperclip/artifact-helper` directory, so CLI sandboxes can
write them even when the controller's `TMPDIR` is inaccessible.
`PAPERCLIP_HELPER_STATE_DIR` overrides that directory when explicitly configured.
skills/paperclip/scripts/paperclip-upload-artifact.sh
+14 -3
View File
@@ -113,6 +113,17 @@ sha256_text() {
fi
}
helper_state_directory() {
printf '%s' "${PAPERCLIP_HELPER_STATE_DIR:-${PAPERCLIP_WORKSPACE_CWD:-$PWD}/.paperclip/artifact-helper}"
}
helper_response_file() {
local directory
directory="$(helper_state_directory)"
# TMPDIR can name a controller-owned directory outside a CLI sandbox.
(umask 077; mkdir -p "$directory" && mktemp "$directory/response.XXXXXX")
}
request_json() {
local method="$1"
local url="$2"
@@ -120,7 +131,7 @@ request_json() {
local response_file
local status_code
response_file="$(mktemp)"
response_file="$(helper_response_file)"
if [[ -n "$body" ]]; then
status_code="$(
curl -sS -X "$method" -w '%{http_code}' -o "$response_file" \
@@ -163,7 +174,7 @@ upload_file() {
escaped_path="${path//\\/\\\\}"
escaped_path="${escaped_path//\"/\\\"}"
response_file="$(mktemp)"
response_file="$(helper_response_file)"
status_code="$(
curl -sS -X POST -w '%{http_code}' -o "$response_file" \
"$url" \
@@ -223,7 +234,7 @@ acquire_operation_lock() {
local attempts=0
umask 077
operation_state_root="${PAPERCLIP_HELPER_STATE_DIR:-${TMPDIR:-/tmp}/paperclip-upload-artifact}"
operation_state_root="$(helper_state_directory)"
mkdir -p "$operation_state_root"
operation_lock_path="$operation_state_root/$operation_key.lock"
operation_lock_owner="$$|$(process_start_identity "$$" || true)"
tests/ai-connections-app/app.spec.ts
+12 -15
View File
@@ -135,25 +135,22 @@ test("explicit OpenAI API method survives continuing and reloading", async ({ pa
});
for (const [provider, label] of [["anthropic", "Claude"], ["openai", "OpenAI"]]) {
test(`Connections reuses the agent provider step for ${label}`, async ({ page }, testInfo) => {
test(`Connections reuses the local browser sign-in step for ${label} (simulated auth)`, async ({ page }, testInfo) => {
// UI contract only: real subscription qualification is provider-connections.
const base = `/api/companies/${companyId}/ai-connections/local`;
const sessionId = "22222222-2222-4222-8222-222222222222";
await page.route(`**${base}/attempts`, route => route.fulfill({ json: { sessionId, expiresAt: new Date(Date.now() + 300000).toISOString() } }));
await page.route(`**${base}/check`, route => route.fulfill({ json: { status: "sign_in_required", authorizationUrl: "https://provider.example/authorize", ...(provider === "openai" ? { code: "ABCD-EFGH" } : {}) } }));
await page.route(`**${base}/attempts/${sessionId}`, route => route.fulfill({ json: { ok: true } }));
await page.goto(`/${prefix}/apps/connect?source=${provider}&method=ai-subscription`);
await page.getByRole("button", { name: /^(Save and continue|Continue)$/ }).click();
await expect(page.getByRole("radiogroup", { name: "Connect your model provider" })).toBeVisible();
await expect(page.getByRole("button", { name: "Connect for tool access instead" })).toHaveCount(0);
await page.getByRole("radio", { name: new RegExp(label) }).click();
if (provider === "anthropic") {
await expect(page.getByText(/Connect uses your local/)).toBeVisible();
await expect(page.getByText("claude auth login", { exact: true })).toBeVisible();
} else {
await expect(page.getByText(/Your existing terminal login stays separate/)).toBeVisible();
const command = page.getByText(/^CODEX_HOME=.* codex login$/);
await expect(command).toBeVisible();
const preparedCommand = await command.textContent();
await page.reload();
await page.getByRole("radio", { name: new RegExp(label) }).click();
await expect(command).toHaveText(preparedCommand!);
}
await expect(page.getByRole("button", { name: "Connect", exact: true })).toBeEnabled();
await expect(page.getByRole("link", { name: `Sign in to ${label}`, exact: true })).toHaveAttribute("href", "https://provider.example/authorize");
await expect(page.getByText(/CODEX_HOME=|claude auth login|existing terminal login/)).toHaveCount(0);
if (provider === "anthropic") await expect(page.getByRole("button", { name: "Submit code", exact: true })).toBeDisabled();
else await expect(page.getByText("ABCD-EFGH", { exact: true })).toBeVisible();
await expect(page.getByRole("button", { name: "Connect", exact: true })).toBeDisabled();
// Let the shared tile-collapse and card-enter animations settle for visual review.
await page.waitForTimeout(1000);
await page.screenshot({ path: testInfo.outputPath(`${provider}-shared-provider-step.png`), fullPage: true });
tests/runner-e2e/FIXTURES.md
+6
View File
@@ -1,5 +1,11 @@
# Runner E2E fixture authoring
## Connection creation fixtures
See [PROVIDER-CONNECTIONS.md](PROVIDER-CONNECTIONS.md) for the explicit-only
`provider-connections` suite, local/staging target ownership, dedicated browser
profiles, credential handoffs, private evidence, and cleanup contract.
The [public MCP journeys](PUBLIC-MCP.md) reuse the fixture registry with a real
authenticated browser session. `RunnerApi.setBrowserSession` binds that session
to API calls, including encrypted secret provisioning via Node fetch. OAuth
tests/runner-e2e/PROVIDER-CONNECTIONS.md
+498
View File
@@ -0,0 +1,498 @@
# Live provider connections
`provider-connections` exercises real connection creation through the Apps catalog
or new-agent wizard, then verifies a real agent task and follow-up. It is an
explicit-only Product E2E suite. It is excluded from `--all` and the paid GitHub
matrix: subscription login initially runs attended on a developer's machine.
Tasks are submitted through the production composer. When it accepts only a
prompt, the harness assigns its fixture title through the public API. This
changes naming metadata only; execution, status, attachments and downloaded
bytes are produced and verified independently.
## Run one journey
Create a private JSON configuration outside the repository. Its values reference
credentials; do not paste keys into it. For example:
```json
{
"target": { "mode": "managed-local" },
"browser": {
"headed": true,
"channel": "chrome",
"profileDir": "/absolute/private/path/provider-qa-browser",
"accountAlias": "developer-qa",
"freshness": "signed-in",
"loginTimeoutMs": 600000
},
"secretFile": "/absolute/private/path/provider-qa.env",
"models": { "codex": "YOUR_CODEX_MODEL", "claude": "YOUR_CLAUDE_MODEL" },
"routes": {
"openrouter": { "model": "YOUR_OPENROUTER_MODEL" },
"bedrock": { "model": "YOUR_BEDROCK_MODEL", "region": "YOUR_AWS_REGION" },
"responses": {
"baseURL": "https://your-gateway.example/v1",
"credentialEnv": "QA_RESPONSES_KEY",
"model": "YOUR_GATEWAY_MODEL",
"auth": "bearer"
}
},
"budgetCents": 200,
"maxRuns": 3,
"turnTimeoutMs": 300000
}
```
The secret file must be an owner-only regular file (`chmod 600`), with literal
`NAME=value` entries. Only the selected variable is read. Shell substitutions
are rejected. Environment variables take precedence. Default names are
`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `XAI_API_KEY`, `GEMINI_API_KEY`,
`OPENROUTER_API_KEY`, and `AWS_BEARER_TOKEN_BEDROCK`. Override direct-key names
with `credentials: {"codex": "DEDICATED_QA_OPENAI_KEY"}`. Subscription cases
read no provider API key and never import CLI credential homes.
Routes may be shared by method or overridden per harness, such as
`"claude.openrouter"`. Protocol gateways require their own explicit credential
reference, or `auth: "none"` for a deliberately unauthenticated endpoint.
Messages gateways also support `auth: "api_key"` (`x-api-key`). Bedrock requires
a current bearer token and region. The harness does not mint or refresh AWS
tokens. Missing credentials/configuration produce a blocked result.
```sh
pnpm test:e2e:runner -- --list --suite provider-connections
pnpm test:e2e:runner -- \
--id provider-connections.connection-codex-native.local.agent-api-key \
--connection-config /absolute/private/path/connections.json
pnpm test:e2e:runner -- \
--id provider-connections.connection-claude-legacy.local.apps-subscription \
--connection-config /absolute/private/path/connections.json
```
Start with one cell. Running `--suite provider-connections` selects all 116
eligible cells and can spend money; it does not make missing routes available.
Each command generates a separate campaign, preserving previous attempts.
## Authentication handoff
Playwright drives Paperclip and opens the real provider sign-in page. When login
needs a person, the terminal prints the provider and account alias and writes
`progress.json`. Complete Google/provider login, MFA, consent, and any code
entry in the visible QA browser. The test resumes only after Paperclip reports
a saved, connected credential. Do not paste passwords or OAuth codes into chat.
The QA browser uses its own private profile. It refuses an existing unmarked
profile, keeps a lock while running, and never uses your normal Chrome profile.
`signed-in` permits previously saved website login; it still creates a fresh
Paperclip connection. `signed-out` starts a temporary empty browser profile.
Paperclip SSO and provider SSO may share an identity provider, so this records
the browser's initial state, not a guarantee of another password challenge.
Human help is recorded as `assisted`. A login deadline records `awaiting_user`,
not a pass, and stops the selected campaign before opening another login.
Rerun the remaining cell IDs to start a fresh authorization after expiry;
resumption is within the running process, not across a stopped process.
Ctrl-C, termination, and hangup close the test browser pages, cancel recorded login attempts and active
runs, performs fixture teardown, records interruption, and stops the campaign.
Each cell has a thirty-minute outer deadline. A forced process kill cannot run
teardown; inspect the private QA profile lock and target fixtures before rerunning.
If Google rejects the automated browser, use the production device-code or
code-return flow in your normal browser. Grok's current production flow may
require the displayed terminal sign-in command. These are attended results.
## Local and staging targets
`managed-local` owns a fresh server/database and removes its data on shutdown.
To use an existing local server or staging, replace `target` with:
```json
{
"mode": "attach",
"baseURL": "https://YOUR-INSTANCE.staging.paperclip.app",
"expectedCommit": "EXACT_40_CHARACTER_DEPLOYED_COMMIT_SHA",
"deploymentMode": "authenticated"
}
```
For local attach use `http://127.0.0.1:PORT` and `local_trusted`. Health must
match the expected revision and deployment mode before credentials are read.
The target is never reset, restarted, or given instance-setting changes.
Sign in to Paperclip in the QA browser if needed; that session supplies the
same public API permissions as the browser. The operator needs company/agent
creation, connection management, and company archival permissions.
Deployment and execution environment are different dimensions. `.local.` means
the agent runs in the target's local environment, including a staging server's
local environment. `.daytona.` requires an already configured active Daytona
environment on the selected target; this suite does not provision one. Use
`target.companyId` and `target.environmentId` to select an empty dedicated
company named `Connection QA…` with that environment. For a new QA company the
harness uses the matching visible instance environment. Missing environments
or a disabled native runner are explicit target blockers.
Each cell creates a separate QA company unless one was supplied. Teardown
pauses its agents, cancels active runs, revokes its test connections, and
archives the created company. This avoids requiring hard-delete support on
staging and retains inspectable task history. Supplied QA companies must start
without active agents or usable connections; revoked connection records and
terminated agents from earlier attempts are allowed. Their new test agents are
deleted and test connections revoked; historical records may remain. `retainCompany: true`
keeps the fixture and credentials for diagnosis, with agents paused, and is
recorded explicitly in evidence. Clean it up before reusing that company.
## What qualifies as a pass
Every checkpoint must be verified:
1. Expected target revision and deployment mode.
2. Fresh connection saved with the selected method/provider/route.
3. Connection visible after navigating away and reloading Apps.
4. Agent created through the wizard with the selected harness/model/binding and requested execution environment.
5. Real Configure-page environment probe passes.
6. Real task run succeeds with that exact managed connection's attribution and requested environment in its durable run context.
7. Agent decodes random input bytes supplied in the task, computes their sum/count/hash, and delivers
an attachment whose JSON is checked independently by the harness.
8. Tool evidence and attachment authorship/run attribution are present.
9. A browser-submitted follow-up reuses the connection and delivers independently
checked minimum/maximum/sum output.
The oracle input is base64 in the task description and must be decoded with a
tool, preserving its exact bytes. This makes connection qualification independent
of new-task attachment ingestion. The earlier uploaded-input OpenRouter attempt
remains a failed attachment-path test: native staging currently admits chat
attachments, and the new-task dialog uploads its files after scheduling the agent.
Successful provider cells do not qualify that task-level upload path.
10. Downloadable artifacts remain visible after a page reload; cleanup succeeds.
An answer saying “provider routing works” cannot pass. Evidence records the
configured model and agent configuration; connection attribution proves the
selected credential, not an upstream gateway's internal model mapping.
The matrix covers Claude, Codex, Grok, Gemini, OpenCode, and Hermes local where
Paperclip's managed-connection capability table admits them, with applicable
legacy/native variants and both UI entry points. Cursor local, Kimi, and Pi
remain visible product gaps for managed connection onboarding. Remote OpenClaw
and Hermes gateways, Cursor Cloud, managed Claude/AWS AgentCore, process/HTTP,
and generic legacy ACPX are deliberately excluded.
## Reports, privacy, and CI
Results are under `tests/runner-e2e/results/connections-…/`: `dashboard.html`,
`campaign.json`, `normalized-results.json`, `connection-coverage.json`, `progress.json`, and packaged
per-cell evidence. The report uses the existing Product E2E grader, billing,
history, and dashboard. Coverage reports required/selected/attempted/passed
counts separately. A partial local campaign does not qualify staging or the
whole matrix. Blocked credentials/login are incomplete; cleanup failures fail.
No Playwright traces, HARs, videos, automatic screenshots, raw assertion call
logs, OAuth URLs/codes, or provider transcripts are retained. Only the final
synthetic task can be screenshotted; it is private and has no public publication
marker. Auth browser state stays outside the repository. Keep reports private.
The result includes source revision, target revision, suite definition hash,
model, method, assistance, run IDs, checkpoints, timing, and numeric usage.
Before removing a managed instance, the harness retains terminal run status,
log availability, and closed diagnostic codes for recognized quota, overload,
file-conversion, session-configuration, or shell-restriction failures. It reads
error records and failed tool receipts only; messages, stderr, model output,
and reasoning are not copied into the result. An empty signal list leaves the
cause unknown. Missing logs remain explicit and do not imply a healthy run.
Attached-company cleanup captures these diagnostics after stopping runs and
before deleting the fixture agent, since agent deletion also removes its logs.
Cleanup revokes only this attempt's account IDs from successful creation or
owned sign-in receipts. Concurrent campaigns' accounts are never adopted or
revoked, even if both campaigns initially observed an empty QA company.
UI probes may incur unreported spend, so billing is explicitly partial.
Regenerate the report from retained results with
`pnpm test:e2e:runner:dashboard -- /absolute/path/to/connections-campaign`.
API-key cells can run headless (`browser.headed: false`, `channel: "chromium"`)
on trusted CI once the target's board authentication is provided through a
dedicated QA browser session. Subscription cells require headed mode and a
person when challenged. Dedicated subscription accounts and a trusted attended
runner are the next operational step; normal untrusted PR jobs never get these
credentials. Credential-free support tests and catalog checks run separately:
```sh
pnpm test:e2e:runner:typecheck
pnpm test:e2e:runner:unit
```
The opt-in cancellation smoke uses a synthetic loopback page and cookie, with
no provider calls or credentials. It verifies that Ctrl-C, termination, and
hangup leave the browser's authenticated API client usable until fixture cleanup
finishes: `pnpm exec tsx tests/runner-e2e/connection-cancel.smoke.ts`.
## Initial verification (2026-10-03)
The local native Codex/direct OpenAI key/new-agent cell completed both real
artifact turns with exact connection attribution and successful cleanup. The
Apps/OpenRouter/native Codex cell created and reused its connection and passed
its setup probe, then failed the task proof: its agent could not retrieve the
uploaded input and marked the task blocked. This is not a gateway qualification
pass. The Claude legacy/Apps subscription journey opened real sign-in and
exercised the bounded `awaiting_user` outcome; login was not completed.
Credential-free verification: runner E2E typecheck, 855 support tests, both
updated simulated local-browser-login regressions, catalog discovery, and
retained-report regeneration passed. A fresh managed-local server was started
and removed with an intentionally missing credential; no provider was called.
Staging and the remaining cells have not been qualified. Use the generated
campaigns for the exact revision, definition hash, timestamps, and results.
## Local follow-up (2026-10-05)
Every eligible non-subscription local cell was attempted against the existing
onboarding instance, preserving its original company data. The latest retained
attempts now contain 43 passing journeys out of 46 non-subscription cells,
including the Gemini investigation on October 6:
| Harness | Passing key/gateway journeys | Eligible local key/gateway journeys | Remaining result |
| --- | ---: | ---: | --- |
| Claude | 16 | 16 | Direct key, OpenRouter, Bedrock, and Messages passed through both entry points and runner generations. |
| Codex | 12 | 12 | The legacy/new-agent OpenRouter task and follow-up passed after the skill-root and artifact-helper fixes. |
| OpenCode | 8 | 8 | All three remaining OpenRouter journeys passed after the artifact-helper and canonical-workspace permission fixes. |
| Hermes local | 4 | 4 | A follow-up interrupted by lease release passed on one bounded retry. Workspace isolation was then corrected and retested separately. |
| Grok | 2 | 4 | Both legacy API-key journeys passed after preserving private session history and avoiding remote subscription restoration for missing local history. Native journeys require the missing pinned runtime. |
| Gemini | 1 | 2 | New-agent/API-key passed with `gemini-3.5-flash-lite`; Apps/API-key delivered its first artifact but timed out on follow-up. Installed CLI file creation still fails the isolated compatibility check. |
There are another 12 local subscription cells. Attended Claude/OpenAI attempts
opened real authorization and reached their human handoff, but have not qualified
a completed subscription journey. Staging and Daytona remain untested. The
Responses and Messages gateway proofs used the official provider endpoints;
they do not qualify an Emissary deployment or every compatible gateway.
The run exposed a real new-agent loading race: an early API-key selection could
be discarded when environment settings finished loading and remounted Connect.
Connect now waits for those settings before exposing interactive choices. Saved
connection selectors and native OpenCode model normalization were repaired in
the harness; original failures remain in their original campaigns. Native
OpenCode no longer sends the deprecated per-prompt tools override, which replaced
its session permission policy; its remaining live OpenRouter failures were subsequently fixed and retested below.
Hermes also ignored the assigned task workspace when an agent had no explicit
`cwd`, writing proof files into the server checkout. It now uses the resolved
task workspace as its default. Explicit directory overrides retain their existing
behavior. The real retest checks that the local proof file is in the assigned
directory and matches the independently downloaded artifact. Earlier checkout
spill files were preserved privately outside the repository.
An interrupted subscription attempt exposed another cleanup bug: Playwright's
default signal handler closed the browser before the harness could cancel the
login using its authenticated API client. The harness now owns browser shutdown,
and a real-browser smoke verified cleanup under SIGINT, SIGTERM, and SIGHUP.
One cancellation state now covers the entire campaign, including target startup,
report generation, and teardown. Credential-free regressions verify that all three
signals stop further cells during startup and reporting and still stop the owned
target. Signal handlers remain installed until teardown finishes.
An expired human handoff also stops the campaign instead of opening the next
subscription window. The affected attempt remains failed in the evidence; its
login was cancelled and its disposable company archived through the public API.
The cleanup audit covers all 113 disposable companies and confirms that the
original onboarding company remains available.
Hermes was tested using an isolated official CLI installation. Its temporary
PATH launcher was removed after qualification; replay requires Hermes to be
installed on the tested server's PATH. The report records the tested CLI version
and source commit in `qualification-environment.json`.
Credential-free verification passed: runner E2E typecheck and all 856 support
tests, 46 new-agent UI tests, 39 native OpenCode driver tests, 10 Hermes invocation
tests, runner/Hermes package typechecks, and UI token gates. After the final
browser signal change, runner E2E typecheck, 13 connection support tests, and the
three-signal browser smoke passed again. This initial qualification stage did
not repeat repo-wide typecheck, tests, and build; later verification is recorded
below.
### Credential follow-up, evening of 2026-10-05
The xAI dashboard identified expiry as the original key rejection: its QA key
expired on September 27. The existing key was renewed for 90 days, through
January 3, 2027, preserving its scope and limits. A Gemini replacement was saved
in the owner-only secret file; the temporary exposed key was revoked and its
absence verified after a dashboard reload. No provider billing settings changed.
Both provider model-list requests returned HTTP 200. A separate bounded Gemini
generation request succeeded with `gemini-3.8-flash`; the older
`gemini-2.5-flash` request returned HTTP 404. Model-list success alone does not
qualify inference or a product journey.
Six previously credential-blocked cells were rerun, followed by three bounded
Gemini attempts after new diagnostics and a selector repair. At that stage none added a fully
qualified journey, leaving 36 of 46 local key/gateway cells passing. Those 91
attempts and the subsequent repair attempts remain inspectable. These evening
campaigns have October 6 UTC timestamps and belong to the developer's October 5
local qualification date.
- Grok legacy, both entry points: the first task delivered an independently
verified artifact. Each follow-up reported its session absent locally,
attempted remote conversation restoration, and waited for subscription device
authorization during an API-key journey. Investigate preserving isolated
non-credential session state across disposable managed credential homes.
- Grok native, both entry points: connection creation and reload passed, but
Configure was not reached. An independent non-inference installation probe
confirmed the required Grok Build 1.0.13 executable was missing. Provision it
using the checked-in helper in `doc/grok-native-runner.md`, then rerun those
exact cells; retain the runtime integrity checks.
- Gemini: `Auto` completed ACP startup but timed out before task output. With
`gemini-3.8-flash`, both entry points created and reloaded a connection, passed
setup, and started the real task, then failed with
`acpx_session_config_failed`. Installed Gemini CLI 0.58.0 rejected
`session/set_config_option` for the model with ACP `-32601` (Method not found).
Resolve that runtime compatibility before claiming a pass. The Apps selector
now reads the shared app definition's name (`Google Gemini`); its original
selector failures remain retained.
After the catalog selector repair, runner E2E typecheck and all 13 connection
support tests passed. The final cleanup audit found no usable QA connections,
active QA agents, or active QA runs; the original onboarding company and test
server remain available. The private report includes `credential-refresh.json`,
`key-retest-diagnostics.json`, `gemini-generation-probe.json`, and
`cleanup-audit.json`. None contain credential values or authorization codes.
The private `connections-qualification-2026-10-05` dashboard selects the newest
attempt per cell. `retained-attempts.json` and `attempts.md` retain all original
campaigns. Source and suite definition hashes are preserved: these runs span
harness repairs and an uncommitted working tree, so their passing count is
historical evidence, not a common-build release qualification. Commit the reviewed
fixes and rerun the required cells against that exact deployed revision before
claiming the whole feature qualified.
### Repair verification, late evening of 2026-10-05
The previously failing Codex OpenRouter journey, both legacy Grok API-key
journeys, and all three OpenCode OpenRouter journeys now have passing browser
retests with independently downloaded artifacts and successful follow-ups.
Original failures remain in the report; no assistant success message alone
qualifies a journey.
The fixes refresh Codex's current skill root on resumed turns, keep artifact
helper temporary files inside the selected workspace, allow native OpenCode's
assigned workspace under both its supplied and canonical filesystem paths,
and keep credentials disposable. Missing managed local Grok history starts a
fresh task handoff instead of triggering remote subscription recovery during an
API-key turn.
On 2026-10-06, host-side Grok transcript retention/restoration was removed after
security review: other agents running as the same OS user could read restored
transcripts regardless of private file modes. Managed Grok follow-ups now start
fresh with the Paperclip task handoff. The historical Grok passes above precede
this change and do not qualify current transcript continuation; an isolated
history solution and new live qualification remain required.
Gemini now receives its selected model through `GEMINI_MODEL` before ACP startup
instead of an unsupported `session/set_config_option` call. Real tasks confirmed
startup and tool execution. The original `gemini-3.8-flash` attempts subsequently
hit daily inference quota; a small request for `gemini-3.1-flash-lite` succeeded.
These are distinct from model-list and connection-probe success.
Local Gemini turns now also receive the current selected skill root and
explicit instructions to read the Paperclip artifact workflow before making
control-plane calls. A real follow-up exposed Gemini CLI's rejection of inline
command substitution when building a JSON completion request. Its local turn
guidance now uses workspace JSON body files with `curl --data-binary @file` or
the selected skill's helpers; the shell restriction remains in place. Regression checks cover current roots across disposable
homes for both Codex and Gemini. The shared ACP regression file now passes all
214 tests after isolating Gemini test homes from the developer's CLI state and
adding canonical-workspace and remote-workspace regressions.
A credential-free real ACP reproduction isolated two filesystem compatibility
problems behind Gemini's file-write errors. The client now translates a genuine
missing-file read into the correct ACP resource-not-found error. This verifies
the client protocol response; the installed Gemini CLI still mishandles that
response, as isolated below. Local Gemini sessions also bind the filesystem client to
the canonical workspace directory. Remote workspace paths are preserved. The
existing ACPX dependency patch and its lockfile hash contain this narrow fix;
four real protocol regressions cover missing-file semantics, external path denial, deny-all
permissions, and other filesystem errors. The full affected ACP group passes
221 tests, including the existing three spawn safeguards. An earlier combined
verification failure and the failed browser attempts remain retained.
The final filesystem-fix campaign (`connections-2026-10-06T04-50-27-196Z-1ec2a9`)
still failed both Gemini entry points at the unchanged 300-second first-task
deadline. Connection creation, reload, Configure probe, and binding passed.
The new-agent turn created its local proof files, but neither turn completed
the successful run, uploaded-artifact, and follow-up qualification. The earlier
filesystem and model-configuration errors were absent from these tool receipts;
that does not qualify the full workflow. Further work must investigate Gemini
task execution latency and skill use with the configured CLI/model, then rerun
the same independent oracle. No timeout increase or cancelled-run pass was used.
A further Gemini attempt exposed a harness settlement race: the first successful
turn did not attach the file, while an automatic disposition-repair run did.
The harness now waits for active repair runs within the existing configured
limits and selects the successful run that actually uploaded the required file.
It still rejects cancelled runs, wrong connection/agent attribution, wrong
artifact bytes, and exceeded run/cost limits. The repair-attribution regression
suite passed (58 tests across the targeted support/catalog files).
The task-composer helper was updated for the current production controls and
its remembered project selection. A credential-free real browser smoke proved
Plan and Ask mode, assignee/project selection, fixture task naming, and file
upload. Public-API title updates only name fixtures; execution, credentials,
model selection and artifact delivery remain real product behavior.
Full workspace typecheck and build passed. UI token gates and shell syntax
passed. The broad Vitest attempt did not produce a green full-suite result:
stale catalog/pool assertions were corrected and checked narrowly, while
additional worker-start, hook and process timeouts remain recorded. Serial
adapter/server rechecks were interrupted after more deadline failures; they do
not constitute a passing full-suite run. UI/CLI and
shared/database/skill-catalog groups passed. Further isolated checks do not
substitute for a successful complete PR verification run.
The pinned native Grok executable is still absent at
`/opt/paperclip/providers/grok/1.0.13/grok`. Installing it needs administrator
access; retain the checked-in provisioning helper and its checksum verification.
Subscriptions, staging/Daytona, and common-build release qualification remain
outstanding.
## Gemini investigation (2026-10-06)
The installed Gemini CLI is 0.58.0. A credential-free check now exercises its
actual ACP file tools against the patched ACPX client, with a loopback model
response and disposable home/workspace:
```sh
pnpm exec tsx tests/runner-e2e/gemini-acp-filesystem.smoke.ts
```
On the stock CLI, reading an existing file and rejecting an external write pass;
creating a new file fails. Its ACP SDK rejects a plain JSON error object, while
`AcpFileSystemService.normalizeFileSystemError` reads the message only from an
`Error` instance. Consequently it misses the genuine resource-not-found message
and fails the initial read-before-write. The same implementation is present in
the [upstream source](https://github.com/google-gemini/gemini-cli/blob/main/packages/cli/src/acp/acpFileSystemService.ts).
A temporary copy that also reads a plain object's string `message` passes all
three checks. The installed CLI has not been changed, and the real browser
qualification uses the stock CLI. A workflow that recovers using shell tools
does not qualify the broken native file tool.
Retained child-stderr metadata also confirms HTTP 503 overload retries in the
earlier deadline failures. A bounded availability probe returned HTTP 503 for
`gemini-3.8-flash` and valid HTTP 200 output for `gemini-3.5-flash-lite`.
With Flash-Lite, the full new-agent/API-key journey passed, including two
independently downloaded and verified artifacts and an attributed successful
follow-up. The initial Apps/API-key journey created, reloaded, tested, and bound
the connection, then failed its first task. It did not retain a cause before
managed-instance teardown; that gap prompted the closed diagnostic projection
above. One diagnostic retest (`connections-2026-10-06T13-25-36-718Z-20ee5d`)
delivered and independently verified the first artifact, then exceeded the
unchanged 300-second follow-up deadline. Both run statuses and log availability
were retained before cleanup, but neither log contained a recognized error
signal. The follow-up cause remains unknown; overload in other runs is not
proof of overload in this run. No deadline or oracle was relaxed.
The CLI requests `HIGH` thinking by default. Short bounded comparisons did not
establish that default thinking causes the long workflow delay. Explicit effort
configuration still uses an ACP method this CLI rejects; supported startup
`modelConfigs` settings need a separate adapter fix and local/remote validation.
Do not silently drop the user's setting or claim that a model-list/setup probe
qualifies execution.
The original 3109 instance is currently stopped, and its temporary config and
encryption key are missing. Its database was left untouched; substituting a new
key would not recover the existing credentials. Fresh investigation campaigns
use managed instances with separate databases and keys, archive their owned
companies, and remove their owned servers. The private consolidated report
retains all earlier failures, exact model/source provenance, and
`gemini-investigation.json`. These mixed working-tree attempts do not qualify
staging or a common committed release build.
tests/runner-e2e/README.md
+6
View File
@@ -1,5 +1,11 @@
# Paid runner full-stack E2E
## Live provider connection journeys
See [PROVIDER-CONNECTIONS.md](PROVIDER-CONNECTIONS.md) for the explicit-only
`provider-connections` suite, local/staging target ownership, dedicated browser
profiles, credential handoffs, private evidence, and cleanup contract.
The [public MCP suite](PUBLIC-MCP.md) adds explicit paid assistant/plugin journeys
with authenticated browser consent, real MCP tool use and independently graded
team execution. Select `--suite public-mcp`; it is excluded from `--all`.
tests/runner-e2e/SECURITY.md
+14
View File
@@ -1,5 +1,19 @@
# Runner E2E security for a public repository
## Attended connection authentication
See [PROVIDER-CONNECTIONS.md](PROVIDER-CONNECTIONS.md) for the explicit-only
`provider-connections` suite, local/staging target ownership, dedicated browser
profiles, credential handoffs, private evidence, and cleanup contract.
Provider-connection results may retain closed diagnostic codes and terminal run
status before managed-instance teardown. Only error records and failed tool
receipts are inspected; raw error messages, child stderr, model output and
reasoning are not retained. Unknown causes and unavailable logs stay explicit.
The optional installed-Gemini filesystem smoke uses synthetic credentials and
loopback model responses, makes no real provider calls, and removes its
disposable home/workspace.
The explicit [public MCP suite](PUBLIC-MCP.md) also creates a disposable browser
account and OAuth grant. Signup and token exchange use Node fetch, credentials
stay outside model context, and dynamic cookie/code/token values join the
tests/runner-e2e/case-outcome.ts
+5 -2
View File
@@ -1,4 +1,5 @@
import type { RunnerE2EResult } from "./types.js";
import { isBlockedConnectionResult } from "./connection-evidence.js";
const html = (value: unknown) =>
String(value ?? "").replace(
/[&<>"']/g,
@@ -12,6 +13,7 @@ export function isIncompleteFirstTaskResult(
result: RunnerE2EResult,
errors: readonly string[] = [],
) {
if (isBlockedConnectionResult(result)) return result.cleanup === "passed" && result.failureClass !== "secret_leak" && errors.length === 0;
const checks = result.firstTask?.checks ?? [];
return result.status === "failed" &&
result.failureClass === "candidate_failure" &&
@@ -30,7 +32,7 @@ export function renderCaseOutcome(
(result.matcherResults ?? []).map((m) => ({
id: m.matcher.kind === "json_path" ? m.matcher.path : m.matcher.kind,
passed: m.passed,
notReached: undefined as string | undefined,
notReached: isBlockedConnectionResult(result) && !m.passed ? "Blocked before this checkpoint" : undefined as string | undefined,
detail: m.detail,
}));
const failures = checks.filter((c) => !c.passed && !c.notReached);
@@ -54,7 +56,8 @@ export function renderCaseOutcome(
),
];
return `<section class="case-outcome ${failed ? "outcome-failed" : "outcome-passed"}" aria-label="Case result explanation">
<strong>${incomplete ? "Incomplete journey" : failed ? "Overall failed" : "Overall passed"} · ${checks.length ? `${passed}/${evaluated} behavioral checks passed${notReached.length ? ` · ${notReached.length} not reached` : ""}` : "No behavioral checks recorded"}</strong>
<strong>${isBlockedConnectionResult(result) ? `Blocked: ${html(result.providerConnection!.outcome.replaceAll("_", " "))}` : incomplete ? "Incomplete journey" : failed ? "Overall failed" : "Overall passed"} · ${checks.length ? `${passed}/${evaluated} behavioral checks passed${notReached.length ? ` · ${notReached.length} not reached` : ""}` : "No behavioral checks recorded"}</strong>
${result.providerConnection ? `<p>Target: ${html(result.providerConnection.target.origin)} · ${html(result.providerConnection.target.commit ?? "revision unavailable")} · ${result.providerConnection.assisted ? "Assisted login" : "Unassisted"} · ${html(result.providerConnection.authFreshness)} browser · ${html(result.providerConnection.entry)} / ${html(result.providerConnection.method)} · stopped at ${html(result.providerConnection.phase)}</p>` : ""}
${failures.length ? `<p>Failed checks:</p><ul>${failures.map((c) => `<li><strong>${html(c.id)}</strong> — ${html(c.detail)}</li>`).join("")}</ul>` : checks.length ? `<p>No behavioral matcher failed.${notReached.length ? " The journey is incomplete; unexercised checks are not passes." : failed ? " The overall failure came from a separate run, cleanup, or evidence check." : ""}</p>` : ""}
${notReached.length ? `<p>Not reached:</p><ul>${notReached.map((c) => `<li><strong>${html(c.id)}</strong> — ${html(c.notReached)}</li>`).join("")}</ul>` : ""}
${checks.length ? `<details><summary>See all behavioral checks (${checks.length})</summary><table class="matchers"><thead><tr><th>Result</th><th>Check</th><th>Detail</th></tr></thead><tbody>${checks.map((c) => `<tr class="matcher-${c.notReached ? "not-reached" : c.passed ? "passed" : "failed"}"><td>${c.notReached ? "Not reached" : c.passed ? "Pass" : "Fail"}</td><td><code>${html(c.id)}</code></td><td>${html(c.detail)}${c.notReached ? ` — ${html(c.notReached)}` : ""}</td></tr>`).join("")}</tbody></table></details>` : ""}
tests/runner-e2e/catalog.test.ts
+5 -5
View File
@@ -146,10 +146,10 @@ describe("runner E2E catalog", () => {
expect(localIntegrityTasks).toHaveLength(2);
expect(openRouterBreadthTasks).toHaveLength(3);
expect(runnerSuites.map((suite) => suite.expectedMatrixSize)).toEqual([
39, 12, 6, 30, 3, 16, 16, 2, 6, 8, 46, 23, 52, 6, 6, 20, 26, 52, 28, 18, 2, 6, 6, 12, 10, 48, 16, 10, 2, 1, 1,
39, 12, 6, 30, 3, 16, 16, 2, 6, 8, 46, 23, 52, 6, 6, 20, 26, 52, 28, 18, 2, 6, 6, 12, 10, 48, 16, 10, 2, 1, 1, 116,
]);
expect(validateRunnerCatalog()).toHaveLength(529);
expect(new Set(runnerMatrix.map((entry) => entry.id)).size).toBe(529);
expect(validateRunnerCatalog()).toHaveLength(645);
expect(new Set(runnerMatrix.map((entry) => entry.id)).size).toBe(645);
expect(
runnerMatrix.filter((entry) => entry.suite.id === "core-compatibility"),
).toHaveLength(48);
@@ -438,13 +438,13 @@ describe("runner E2E catalog", () => {
it("uses only declared secret references in generated payloads", () => {
expect(
runnerMatrix.every((entry) =>
runnerMatrix.filter(entry => entry.task.flow !== "provider_connection").every((entry) =>
entry.requiredCredentials.includes(entry.profile.credential),
),
).toBe(true);
expect(
runnerMatrix
.filter((entry) => entry.environment.id === "daytona")
.filter((entry) => entry.environment.id === "daytona" && entry.task.flow !== "provider_connection")
.every((entry) =>
entry.requiredCredentials.includes("DAYTONA_API_KEY"),
),
tests/runner-e2e/catalog.ts
+7 -3
View File
@@ -3,6 +3,7 @@ import { nativeCompletionTasks, nativeCompletionDefinitionDigest } from "./nativ
import { NATIVE_INSTRUCTION_SUITE, NATIVE_INSTRUCTION_BASE_SHA, nativeInstructionDefinitionDigest } from "./native-instruction-consolidation.js";
import { nativeCompletionProfile, NATIVE_COMPLETION_BUDGET_CENTS } from "./native-completion-defaults.js";
import { chatConfirmationTasks } from "./chat-cases.js";
import { buildConnectionSuite } from "./connection-cases.js";
import { hiringTemplateTasks, hiringTemplateProfile, hiringTemplateDefinitionDigest } from "./hiring-template-cases.js";
import { instructionPersistenceTask } from "./instruction-persistence.js";
import { apiResponseReadingTask } from "./api-response-reading.js";
@@ -1489,6 +1490,7 @@ export const runnerSuites: readonly RunnerSuiteFixture[] = [
expectedMatrixSize: 1,
definitionMetadata: { version: 8, instructions: "fixed-external", nativeIdleTimeoutMs: 1_200_000, autoStopIntervalMinutes: 25, generatedFileCount: 60_000, filenameBytes: 39_828_890, scheduling: "explicit-only", finalization: "committed-without-active-sync-or-retry", copyback: "all-generated-file-contents-change-each-turn" },
},
buildConnectionSuite(runnerProfiles, runnerEnvironments),
] as const;
export function suiteDefinitionHash(suite: RunnerSuiteFixture) {
@@ -1548,8 +1550,8 @@ export function buildRunnerMatrix(
]),
],
requiredCredentials: [
profile.credential,
...(environment.credential ? [environment.credential] : []),
...(task.flow === "provider_connection" ? [] : [profile.credential]),
...(task.flow !== "provider_connection" && environment.credential ? [environment.credential] : []),
],
}))
.filter((execution) => !excludedExecutionIds.has(execution.id)),
@@ -1595,8 +1597,10 @@ function assertNoRawSecretValues(value: unknown, label: string) {
}
export function validateRunnerCatalog(): MatrixExecution[] {
const allProfiles = [...extendedHarnessProfiles, ...runnerProfiles, ...legacyAcpxProfiles, ...pendingContextIntegrityProfiles, ...openRouterBreadthProfiles, ...everydayProfiles.filter(p => !runnerProfiles.some(existing => existing.id === p.id))];
const connectionSuite = runnerSuites.find(suite => suite.id === "provider-connections")!;
const allProfiles = [...connectionSuite.profiles, ...extendedHarnessProfiles, ...runnerProfiles, ...legacyAcpxProfiles, ...pendingContextIntegrityProfiles, ...openRouterBreadthProfiles, ...everydayProfiles.filter(p => !runnerProfiles.some(existing => existing.id === p.id))];
const allTasks = [
...connectionSuite.tasks,
extendedHarnessFileTask,
...contextIntegrityTasks,
...blockerTasks,
tests/runner-e2e/codex-ci-sandbox.test.ts
+2 -2
View File
@@ -37,9 +37,9 @@ describe("Codex CI sandbox trust boundary", () => {
[...condition!.matchAll(/matrix\.profileId == '([^']+)'/g)].map((match) => match[1]),
);
expect([...provisionedProfiles].sort()).toEqual(
[...new Set(runnerMatrix.filter(requiresCodexCiSandbox).map((cell) => cell.profile.id))].sort(),
[...new Set(runnerMatrix.filter(cell => cell.task.flow !== "provider_connection").filter(requiresCodexCiSandbox).map((cell) => cell.profile.id))].sort(),
);
for (const cell of runnerMatrix) {
for (const cell of runnerMatrix.filter(cell => cell.task.flow !== "provider_connection")) {
expect(
cell.environment.id === "local" && provisionedProfiles.has(cell.profile.id),
cell.id,
tests/runner-e2e/connection-cancel.smoke.ts
+67
View File
@@ -0,0 +1,67 @@
/** Opt-in, credential-free browser smoke: teardown retains its authenticated API client. */
import { spawn } from "node:child_process";
import { createServer } from "node:http";
import path from "node:path";
import { parseConnectionConfig } from "./connection-config.js";
import { openConnectionBrowser } from "./connection-launch.js";
import { runnerE2ETypeScriptProcessArgs } from "./web-server-command.js";
const repositoryRoot = path.resolve(import.meta.dirname, "../..");
if (process.argv[2] === "--worker") {
const origin = process.argv[3]!;
const browser = await openConnectionBrowser(parseConnectionConfig({ browser: {
headed: false, channel: "chromium", freshness: "signed-out",
} }), repositoryRoot, origin);
await browser.context.pages()[0]!.goto(origin);
let stopping = false;
for (const signal of ["SIGINT", "SIGTERM", "SIGHUP"] as const) process.once(signal, () => {
if (stopping) return;
stopping = true;
void (async () => {
let ok = false;
try { ok = (await browser.context.request.get(`${origin}/cleanup`)).ok(); }
finally { await browser.close(); }
process.send?.({ kind: "cleanup", ok });
process.disconnect();
process.exitCode = ok ? 0 : 1;
})().catch(() => { process.exitCode = 1; process.disconnect(); });
});
process.send?.({ kind: "ready" });
} else {
let cleanupRequests = 0;
const server = createServer((req, res) => {
if (req.url === "/cleanup") {
const authenticated = req.headers.cookie === "qa-cleanup=synthetic";
if (authenticated) cleanupRequests++;
res.writeHead(authenticated ? 200 : 401).end();
} else res.writeHead(200, { "Content-Type": "text/html", "Set-Cookie": "qa-cleanup=synthetic; HttpOnly; Path=/" }).end("<p>Synthetic cleanup fixture</p>");
});
await new Promise<void>(resolve => server.listen(0, "127.0.0.1", resolve));
const origin = `http://127.0.0.1:${(server.address() as { port: number }).port}`;
try {
for (const signal of ["SIGINT", "SIGTERM", "SIGHUP"] as const) {
const env = Object.fromEntries(["PATH", "HOME", "USER", "LOGNAME", "TMPDIR", "LANG"].flatMap(key => process.env[key] ? [[key, process.env[key]!]] : []));
await new Promise<void>((resolve, reject) => {
const child = spawn(process.execPath, [...runnerE2ETypeScriptProcessArgs(repositoryRoot, import.meta.filename), "--worker", origin], {
cwd: repositoryRoot, env, stdio: ["ignore", "ignore", "ignore", "ipc"],
});
let cleaned = false;
const timeout = setTimeout(() => { child.kill("SIGKILL"); reject(new Error(`${signal}: browser cleanup timed out`)); }, 30_000);
child.on("message", message => {
const event = message as { kind?: string; ok?: boolean };
if (event.kind === "ready") child.kill(signal);
if (event.kind === "cleanup") cleaned = event.ok === true;
});
child.once("error", error => { clearTimeout(timeout); reject(error); });
child.once("exit", code => {
clearTimeout(timeout);
if (code === 0 && cleaned) resolve();
else reject(new Error(`${signal}: browser closed before authenticated fixture cleanup`));
});
});
console.log(`${signal}: authenticated cleanup completed before browser close`);
}
if (cleanupRequests !== 3) throw new Error("Missing authenticated cleanup requests");
} finally { await new Promise<void>((resolve, reject) => server.close(error => error ? reject(error) : resolve())); }
}
tests/runner-e2e/connection-cases.ts
+88
View File
@@ -0,0 +1,88 @@
import { createHash } from "node:crypto";
import { readFileSync } from "node:fs";
import { AI_CONNECTION_CAPABILITIES } from "../../packages/shared/src/ai-connections.js";
import { isAiRoutingCompatible } from "../../packages/shared/src/ai-provider-routing.js";
import type { AiProviderRouting } from "../../packages/shared/src/ai-provider-routing.js";
import type { EnvironmentFixture, MatrixExecution, RunnerProfileFixture, RunnerSuiteFixture, RunnerTaskFixture } from "./types.js";
export const connectionModes = ["subscription", "api-key", "openrouter", "bedrock", "responses", "messages", "chat"] as const;
export type ConnectionMode = typeof connectionModes[number];
export type ConnectionEntry = "agent" | "apps";
export const connectionHarnesses = [
{ id: "claude", adapter: "claude_local", provider: "anthropic", label: "Claude", key: "ANTHROPIC_API_KEY", native: true },
{ id: "codex", adapter: "codex_local", provider: "openai", label: "OpenAI", key: "OPENAI_API_KEY", native: true },
{ id: "grok", adapter: "grok_local", provider: "xai", label: "Grok", key: "XAI_API_KEY", native: true },
{ id: "gemini", adapter: "gemini_local", provider: "google", label: "Google", key: "GEMINI_API_KEY", native: false },
{ id: "opencode", adapter: "opencode_local", provider: "openrouter", label: "OpenRouter", key: "OPENROUTER_API_KEY", native: true },
{ id: "hermes", adapter: "hermes_local", provider: "openrouter", label: "OpenRouter", key: "OPENROUTER_API_KEY", native: false },
] as const;
export type ConnectionHarness = typeof connectionHarnesses[number];
/** Inventory entries remain visible even when no live journey can be selected. */
export const connectionScopeGaps = [
{ adapter: "cursor", status: "unsupported", reason: "No managed connection flow; qualify the harness-specific login separately." },
{ adapter: "kimi_local", status: "unsupported", reason: "No managed connection flow; harness-specific credentials remain outside this suite." },
{ adapter: "pi_local", status: "unsupported", reason: "No managed connection flow or qualified model source." },
{ adapter: "openclaw_gateway", status: "excluded", reason: "Remote gateway owns provider configuration." },
{ adapter: "hermes_gateway", status: "excluded", reason: "Remote gateway owns provider configuration." },
{ adapter: "cursor_cloud", status: "excluded", reason: "Remote integration owns provider configuration." },
{ adapter: "process", status: "excluded", reason: "External process owns provider configuration." },
{ adapter: "http", status: "excluded", reason: "Remote integration owns provider configuration." },
{ adapter: "acpx_local", status: "excluded", reason: "Generic legacy integration; native Claude/Grok ACPX profiles remain covered." },
{ adapter: "claude_managed", status: "excluded", reason: "Managed remote integration owns credentials." },
{ adapter: "aws_agentcore", status: "excluded", reason: "Remote integration owns credentials." },
] as const;
export function routingForMode(mode: ConnectionMode): AiProviderRouting | undefined {
if (mode === "subscription" || mode === "api-key") return undefined;
return { kind: mode === "openrouter" || mode === "bedrock" ? mode : "gateway", protocol: mode === "openrouter" ? "responses" : mode, auth: "bearer", models: [], ...(mode === "bedrock" ? { region: "us-east-1" } : {}) };
}
export function supportsConnection(harness: ConnectionHarness, mode: ConnectionMode) {
const routing = routingForMode(mode);
if (routing) return isAiRoutingCompatible(routing, harness.adapter);
// OpenRouter's normal key is represented by its explicit routing case.
if (harness.provider === "openrouter") return false;
const methods = AI_CONNECTION_CAPABILITIES[harness.provider].methods;
return Boolean(methods[mode === "subscription" ? "subscription" : "api_key"]?.adapters.includes(harness.adapter));
}
export function connectionCell(execution: MatrixExecution) {
const match = /^(agent|apps)-(subscription|api-key|openrouter|bedrock|responses|messages|chat)$/.exec(execution.task.id);
const harness = connectionHarnesses.find(h => execution.profile.id === `connection-${h.id}-${execution.profile.generation}`);
if (execution.suite.id !== "provider-connections" || !harness || !match) throw new Error("Not a provider connection execution");
return { harness, entry: match[1] as ConnectionEntry, mode: match[2] as ConnectionMode };
}
export function buildConnectionSuite(profiles: readonly RunnerProfileFixture[], environments: readonly EnvironmentFixture[]): RunnerSuiteFixture {
const selectedProfiles = connectionHarnesses.flatMap(h => (["legacy", ...(h.native ? ["native"] : [])] as Array<"legacy" | "native">).map(generation => {
const existing = profiles.find(p => p.generation === generation && (p.adapterType === h.adapter || (generation === "native" && p.id === (h.id === "claude" || h.id === "grok" ? `runner-acpx-${h.id}` : `runner-${h.id}`))));
const id = `connection-${h.id}-${generation}`;
return {
id, label: `${h.label} ${generation}`, generation, groups: [generation], adapterType: generation === "native" ? "paperclip_runner" : h.adapter,
provider: h.id, model: existing?.model ?? "configured-in-connection-file",
modelQualification: existing?.modelQualification ?? { source: "candidate_runner_profile", qualificationId: "operator-configured-model" },
credential: h.key,
supportedEnvironments: ["local", "daytona"], expectedRuntimeMode: generation,
expectedRuntimeMetadata: { adapterType: generation === "native" ? "paperclip_runner" : h.adapter, provider: generation === "native" && ["claude", "grok"].includes(h.id) ? "acpx" : h.id },
buildAgent: () => ({ name: id, role: "qa", adapterType: generation === "native" ? "paperclip_runner" : h.adapter, adapterConfig: {}, runtimeConfig: {} }),
} satisfies RunnerProfileFixture;
}));
const tasks: RunnerTaskFixture[] = (["agent", "apps"] as const).flatMap(entry => connectionModes.map(mode => ({
id: `${entry}-${mode}`, label: `${entry === "apps" ? "Apps connector" : "New agent"} · ${mode}`, groups: [], flow: "provider_connection", workMode: "standard",
expectedRunCount: 3, minimumExpectedRunCount: 2,
attemptTimeoutMs: { local: 30 * 60_000, daytona: 30 * 60_000 }, turnTimeoutMs: 5 * 60_000,
expectedTerminalState: { issue: "done", run: "succeeded" },
buildTitle: nonce => `Connection proof ${nonce}`, buildPrompt: () => "Runtime-generated independent attachment proof.", buildVisibleMarker: nonce => nonce,
buildMatchers: () => [],
})));
const excludedExecutionIds = selectedProfiles.flatMap(profile => environments.flatMap(environment => tasks.flatMap(task => {
const harness = connectionHarnesses.find(h => profile.id === `connection-${h.id}-${profile.generation}`)!;
const mode = task.id.replace(/^(agent|apps)-/, "") as ConnectionMode;
return supportsConnection(harness, mode) ? [] : [`provider-connections.${profile.id}.${environment.id}.${task.id}`];
})));
const hash = createHash("sha256");
for (const file of ["connection-cases.ts", "connection-config.ts", "connection-flow.ts", "connection-evidence.ts", "connection-launch.ts", "connection-target.ts", "user-actions.ts"]) hash.update(file).update(readFileSync(new URL(`./${file}`, import.meta.url)));
const digest = hash.digest("hex");
return { id: "provider-connections", label: "Live provider connections", description: "Fresh production UI connections, attended subscription login, independently verified tasks and reuse on a selected deployment.", groups: [], manualOnly: true, profiles: selectedProfiles, environments, tasks, excludedExecutionIds, expectedMatrixSize: selectedProfiles.length * environments.length * tasks.length - excludedExecutionIds.length, definitionMetadata: { version: 1, digest, authentication: "fresh-ui-connection", traces: "disabled", scheduling: "explicit-only" } };
}
tests/runner-e2e/connection-config.ts
+162
View File
@@ -0,0 +1,162 @@
import { readFile, lstat, realpath } from "node:fs/promises";
import path from "node:path";
import { connectionCell, type ConnectionMode } from "./connection-cases.js";
import type { MatrixExecution } from "./types.js";
export interface ConnectionRoute {
model: string; credentialEnv?: string; baseURL?: string; region?: string; auth: "bearer" | "api_key" | "none";
}
export interface ConnectionConfig {
target: { mode: "managed-local" } | { mode: "attach"; baseURL: string; expectedCommit: string; deploymentMode: "local_trusted" | "authenticated"; companyId?: string; environmentId?: string };
browser: { headed: boolean; channel: "chrome" | "chromium"; profileDir?: string; accountAlias: string; freshness: "signed-in" | "signed-out"; loginTimeoutMs: number };
secretFile?: string;
models: Record<string, string>;
routes: Record<string, ConnectionRoute>;
credentials: Record<string, string>;
budgetCents: number; maxRuns: number; turnTimeoutMs: number; retainCompany: boolean;
}
/** Strict, dependency-free private configuration parser. Never echo bad input. */
export function parseConnectionConfig(input: unknown): ConnectionConfig {
const invalid = (): never => { throw new ConnectionBlock("blocked_target", "invalid_connection_configuration"); };
const object = (value: unknown, allowed?: string[]): Record<string, any> => {
if (!value || typeof value !== "object" || Array.isArray(value)) return invalid();
if (allowed && Object.keys(value).some(key => !allowed.includes(key))) return invalid();
return value as Record<string, any>;
};
const text = (value: unknown, fallback?: string): string => {
if (value === undefined && fallback !== undefined) return fallback;
if (typeof value !== "string" || !value.trim() || value.length > 2048) return invalid();
return value;
};
const choice = <T extends string>(value: unknown, options: readonly T[], fallback: T): T => value === undefined ? fallback : options.includes(value as T) ? value as T : invalid();
const flag = (value: unknown, fallback: boolean): boolean => value === undefined ? fallback : typeof value === "boolean" ? value : invalid();
const number = (value: unknown, fallback: number, min: number, max: number): number => value === undefined ? fallback : typeof value === "number" && Number.isSafeInteger(value) && value >= min && value <= max ? value : invalid();
const name = (value: unknown) => { const result = text(value); if (!/^[A-Z][A-Z0-9_]+$/.test(result)) return invalid(); return result; };
const uuid = (value: unknown) => { const result = text(value); if (!/^[a-f0-9]{8}(?:-[a-f0-9]{4}){3}-[a-f0-9]{12}$/i.test(result)) return invalid(); return result; };
const root = object(input, ["target", "browser", "secretFile", "models", "routes", "credentials", "budgetCents", "maxRuns", "turnTimeoutMs", "retainCompany"]);
const t = object(root.target ?? { mode: "managed-local" });
let target: ConnectionConfig["target"];
if (t.mode === "managed-local") { object(t, ["mode"]); target = { mode: "managed-local" }; }
else if (t.mode === "attach") {
object(t, ["mode", "baseURL", "expectedCommit", "deploymentMode", "companyId", "environmentId"]);
const expectedCommit = text(t.expectedCommit);
if (!/^[a-f0-9]{40}$/.test(expectedCommit)) return invalid();
target = { mode: "attach", baseURL: targetOrigin(text(t.baseURL)), expectedCommit, deploymentMode: choice(t.deploymentMode, ["local_trusted", "authenticated"], "authenticated"),
...(t.companyId === undefined ? {} : { companyId: uuid(t.companyId) }), ...(t.environmentId === undefined ? {} : { environmentId: uuid(t.environmentId) }) };
} else return invalid();
const b = object(root.browser ?? {}, ["headed", "channel", "profileDir", "accountAlias", "freshness", "loginTimeoutMs"]);
const accountAlias = text(b.accountAlias, "local-qa");
if (!/^[a-zA-Z0-9_-]{1,60}$/.test(accountAlias)) return invalid();
const routes = Object.fromEntries(Object.entries(object(root.routes ?? {})).map(([key, value]) => {
const r = object(value, ["model", "credentialEnv", "baseURL", "region", "auth"]);
return [key, { model: text(r.model), auth: choice(r.auth, ["bearer", "api_key", "none"], "bearer"),
...(r.credentialEnv === undefined ? {} : { credentialEnv: name(r.credentialEnv) }), ...(r.baseURL === undefined ? {} : { baseURL: text(r.baseURL) }), ...(r.region === undefined ? {} : { region: text(r.region) }) }];
}));
return { target, browser: { headed: flag(b.headed, true), channel: choice(b.channel, ["chrome", "chromium"], "chrome"), accountAlias, freshness: choice(b.freshness, ["signed-in", "signed-out"], "signed-in"), loginTimeoutMs: number(b.loginTimeoutMs, 600_000, 1000, 1200_000), ...(b.profileDir === undefined ? {} : { profileDir: text(b.profileDir) }) },
...(root.secretFile === undefined ? {} : { secretFile: text(root.secretFile) }),
models: Object.fromEntries(Object.entries(object(root.models ?? {})).map(([key, value]) => [key, text(value)])), routes,
credentials: Object.fromEntries(Object.entries(object(root.credentials ?? {})).map(([key, value]) => [key, name(value)])),
budgetCents: number(root.budgetCents, 200, 1, 10_000), maxRuns: number(root.maxRuns, 3, 2, 5), turnTimeoutMs: number(root.turnTimeoutMs, 300_000, 10_000, 600_000), retainCompany: flag(root.retainCompany, false) };
}
/** Only harness-owned error codes may enter a retained report. */
export class ConnectionFailure extends Error {}
export class ConnectionBlock extends Error {
constructor(readonly reason: "awaiting_user" | "missing_credential" | "blocked_target", readonly code: string) {
super(code);
}
}
export function targetOrigin(value: string): string {
const url = new URL(value);
const loopback = ["127.0.0.1", "localhost", "[::1]"].includes(url.hostname);
if (url.username || url.password || url.search || url.hash || url.pathname !== "/" ||
!(url.protocol === "https:" || (url.protocol === "http:" && loopback))) {
throw new ConnectionBlock("blocked_target", "target_requires_https_or_loopback_origin");
}
return url.origin;
}
export async function loadConnectionConfig(file?: string): Promise<ConnectionConfig> {
try {
const config = parseConnectionConfig(file ? JSON.parse(await readFile(file, "utf8")) : {});
if (config.target.mode === "attach") config.target.baseURL = targetOrigin(config.target.baseURL);
return config;
} catch (error) {
if (error instanceof ConnectionBlock) throw error;
// Parsing errors can contain configuration values. Never echo input.
throw new ConnectionBlock("blocked_target", "invalid_connection_configuration");
}
}
export async function assertPrivateFile(file: string) {
const stat = await lstat(file);
if (!stat.isFile() || stat.isSymbolicLink() || (stat.mode & 0o077) !== 0 || (process.getuid && stat.uid !== process.getuid())) {
throw new ConnectionBlock("missing_credential", "credential_file_requires_owner_only_regular_file");
}
}
/** Literal dotenv only: never evaluate shell expansions or source an entire file. */
export function selectedSecret(text: string, name: string): string | undefined {
let found: string | undefined;
for (const raw of text.split(/\r?\n/)) {
const line = raw.replace(/^\s*export\s+/, "");
const match = /^\s*([A-Z][A-Z0-9_]+)\s*=\s*(.*?)\s*$/.exec(line);
if (match?.[1] !== name) continue;
let value = match[2]!;
if ((value.startsWith('"') && value.endsWith('"')) || (value.startsWith("'") && value.endsWith("'"))) value = value.slice(1, -1);
if (!value || /[`\r\n]|\$\(/.test(value)) throw new ConnectionBlock("missing_credential", "credential_must_be_a_literal_value");
found = value;
}
return found;
}
export async function resolveConnectionSecret(config: ConnectionConfig, name?: string, env: NodeJS.ProcessEnv = process.env) {
if (!name) return undefined;
if (env[name]?.trim()) return env[name]!.trim();
if (config.secretFile) {
await assertPrivateFile(config.secretFile);
const value = selectedSecret(await readFile(config.secretFile, "utf8"), name);
if (value) return value;
}
throw new ConnectionBlock("missing_credential", `missing_${name}`);
}
export function resolveConnectionSettings(execution: MatrixExecution, config: ConnectionConfig) {
const { harness, mode } = connectionCell(execution);
const route = config.routes[`${harness.id}.${mode}`] ?? config.routes[mode];
const advanced = mode !== "subscription" && mode !== "api-key";
if (advanced && !route) throw new ConnectionBlock("missing_credential", "route_configuration_required");
const selectedModel = route?.model ?? config.models[harness.id] ?? execution.profile.model;
if (selectedModel === "configured-in-connection-file") throw new ConnectionBlock("blocked_target", "explicit_model_required");
if (["responses", "messages", "chat"].includes(mode)) {
if (!route?.baseURL) throw new ConnectionBlock("blocked_target", "gateway_url_required");
const url = new URL(route.baseURL);
if (url.username || url.password || url.search || url.hash || !["https:", "http:"].includes(url.protocol)) throw new ConnectionBlock("blocked_target", "invalid_gateway_url");
}
if (mode === "bedrock" && (!route?.region || route.auth !== "bearer")) throw new ConnectionBlock("blocked_target", "bedrock_region_and_bearer_required");
if (route?.auth === "api_key" && mode !== "messages") throw new ConnectionBlock("blocked_target", "api_key_header_requires_messages");
if (mode === "openrouter" && route?.auth !== "bearer") throw new ConnectionBlock("blocked_target", "openrouter_requires_bearer");
const defaultKey = mode === "bedrock" ? "AWS_BEARER_TOKEN_BEDROCK" : mode === "openrouter" ? "OPENROUTER_API_KEY" : harness.key;
const credentialEnv = mode === "subscription" || route?.auth === "none" ? undefined : route?.credentialEnv ?? config.credentials[harness.id] ?? (advanced && !["bedrock", "openrouter"].includes(mode) ? undefined : defaultKey);
if (mode !== "subscription" && route?.auth !== "none" && !credentialEnv) throw new ConnectionBlock("missing_credential", "gateway_credential_reference_required");
return { model: selectedModel, route, credentialEnv };
}
export function connectionCatalogSource(mode: ConnectionMode, provider: string) {
return ({ responses: "responses-api", messages: "messages-api", chat: "chat-completions-api", bedrock: "bedrock", openrouter: "openrouter" } as Record<string, string>)[mode] ?? provider;
}
export async function assertOutsideRepository(file: string, repositoryRoot: string) {
// Resolve an existing ancestor too, so a missing child below a symlink cannot evade containment.
let existing = path.resolve(file);
while (true) {
try { existing = await realpath(existing); break; }
catch (error) { if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; const parent = path.dirname(existing); if (parent === existing) throw error; existing = parent; }
}
const root = await realpath(repositoryRoot);
const relative = path.relative(root, existing);
if (relative === "" || (!relative.startsWith(`..${path.sep}`) && relative !== ".." && !path.isAbsolute(relative))) throw new ConnectionBlock("blocked_target", "browser_profile_must_be_outside_repository");
}
tests/runner-e2e/connection-evidence.ts
+129
View File
@@ -0,0 +1,129 @@
import { createHash, randomBytes } from "node:crypto";
import type { RunnerE2EResult } from "./types.js";
export const connectionCheckpoints = ["target", "connection_created", "connection_reloaded", "agent_created", "binding", "setup_probe", "first_run", "artifact", "tool_receipt", "followup", "visible_result"] as const;
export type ConnectionCheckpoint = typeof connectionCheckpoints[number];
export const connectionRunSignals = ["provider_overloaded", "provider_quota", "missing_file_conversion", "unsupported_session_configuration", "command_substitution_denied"] as const;
export interface ConnectionRunDiagnostic {
runId: string;
status: "succeeded" | "failed" | "cancelled" | "interrupted" | "timed_out" | "unknown";
signals: Array<typeof connectionRunSignals[number]>;
logAvailable: boolean;
}
export interface ConnectionEvidence {
version: 1;
outcome: "running" | "passed" | "failed" | "awaiting_user" | "missing_credential" | "blocked_target";
phase: string;
assisted: boolean;
authFreshness: "signed-in" | "signed-out";
target: { mode: "managed-local" | "attach"; origin: string; commit: string | null; deploymentMode: string | null };
entry: "apps" | "agent";
method: string;
checkpoints: Partial<Record<ConnectionCheckpoint, boolean>>;
waits: Array<{ kind: "board_login" | "provider_login"; startedAt: string; finishedAt?: string }>;
connectionId?: string;
agentId?: string;
companyId?: string;
artifactSha256?: string;
model?: string;
inputTransport?: "prompt_base64";
setupChecks?: Array<{ code: string; level: "info" | "warn" | "error" }>;
creationFailure?: { status: number; code: "ai_connection_api_key_rejected" | "ai_connection_verification_failed" | "connection_request_rejected" };
cleanupRetained?: boolean;
companyArchived?: boolean;
artifactChecks?: Array<{ filename: string; runId: string; sha256: string; fields: Record<string, boolean>; exactFields: boolean }>;
runDiagnostics?: ConnectionRunDiagnostic[];
}
/** Project terminal errors to closed diagnostic codes; never retain messages or reasoning. */
export function connectionRunDiagnostic(run: Record<string, any>, log?: unknown): ConnectionRunDiagnostic {
const signals = new Set<typeof connectionRunSignals[number]>();
const detect = (text: unknown) => {
if (typeof text !== "string") return;
if (/"code"\s*:\s*503\b|\bHTTP\s*503\b/i.test(text) && /overloaded|high demand/i.test(text)) signals.add("provider_overloaded");
if (/"code"\s*:\s*429\b|\bHTTP\s*429\b|\bRESOURCE_EXHAUSTED\b|\bTerminalQuotaError\b/.test(text)) signals.add("provider_quota");
if (/Error checking existing file: (?:Resource not found|Internal error)/.test(text)) signals.add("missing_file_conversion");
if (/set_config_option/.test(text) && /[Mm]ethod not found|-32601/.test(text)) signals.add("unsupported_session_configuration");
if (/Blocked: command substitution detected/.test(text)) signals.add("command_substitution_denied");
};
detect(run.error);
detect(run.stderrExcerpt);
const content = log && typeof log === "object" && typeof (log as Record<string, unknown>).content === "string"
? (log as { content: string }).content : undefined;
if (content !== undefined) {
let inner = "";
for (const line of content.split("\n")) {
try { const row = JSON.parse(line); if (typeof row.chunk === "string") inner += row.chunk; }
catch { /* Non-JSON application lines do not classify provider failures. */ }
}
for (const line of inner.split("\n")) {
try {
const row = JSON.parse(line);
if (row.type === "acpx.error") { detect(row.message); detect(row.childStderrTail); }
if (row.type === "acpx.tool_call" && row.status === "failed") detect(row.text);
} catch { /* Model text, partial records, and raw output are not retained. */ }
}
}
const statuses = ["succeeded", "failed", "cancelled", "interrupted", "timed_out"];
return { runId: run.id, status: statuses.includes(run.status) ? run.status : "unknown", signals: [...signals], logAvailable: content !== undefined };
}
export function gradeConnectionEvidence(evidence: ConnectionEvidence) {
return connectionCheckpoints.map(id => ({ matcher: { kind: "json_path" as const, path: `providerConnection.checkpoints.${id}`, expected: true }, passed: evidence.checkpoints[id] === true, detail: `${id}: ${evidence.checkpoints[id] === true ? "verified" : "not verified"}` }));
}
/** Retain stable product check codes, never provider messages or command output. */
export function connectionProbeChecks(value: unknown): NonNullable<ConnectionEvidence["setupChecks"]> {
if (!value || typeof value !== "object" || !Array.isArray((value as Record<string, unknown>).checks)) return [];
return ((value as { checks: unknown[] }).checks).slice(0, 50).flatMap(entry => {
if (!entry || typeof entry !== "object") return [];
const { code, level } = entry as Record<string, unknown>;
return typeof code === "string" && /^[a-z][a-z0-9_]{0,95}$/.test(code) && ["info", "warn", "error"].includes(String(level))
? [{ code, level: level as "info" | "warn" | "error" }] : [];
});
}
export function connectionEvidencePasses(evidence: ConnectionEvidence) {
return evidence.outcome === "passed" && Boolean(evidence.target.commit) && gradeConnectionEvidence(evidence).every(check => check.passed);
}
export function isBlockedConnectionResult(result: RunnerE2EResult) {
return Boolean(result.providerConnection && ["awaiting_user", "missing_credential", "blocked_target"].includes(result.providerConnection.outcome));
}
export function createConnectionProof() {
const nonce = randomBytes(12).toString("hex");
const values = Array.from({ length: 7 }, () => randomBytes(2).readUInt16BE());
const input = `${JSON.stringify({ nonce, values })}\n`;
const expected = { nonce, count: values.length, total: values.reduce((sum, n) => sum + n, 0), sha256: createHash("sha256").update(input).digest("hex") };
const followup = { nonce, minimum: Math.min(...values), maximum: Math.max(...values), total: expected.total };
return { input, expected, followup };
}
export function verifyConnectionArtifact(bytes: Buffer, expected: Record<string, unknown>) {
try {
const value = JSON.parse(bytes.toString("utf8"));
return Object.keys(value).length === Object.keys(expected).length && Object.entries(expected).every(([key, entry]) => value[key] === entry);
} catch { return false; }
}
/** Only durable run attribution is accepted; assistant text cannot provide it. */
export function verifyConnectionRun(run: Record<string, any>, expected: { agentId: string; connectionId: string; method: string; runtimeMode: string; environmentId: string }) {
const connection = run.contextSnapshot?.aiConnection;
return run.status === "succeeded" && run.agentId === expected.agentId && run.runtimeMode === expected.runtimeMode &&
connection?.connectionId === expected.connectionId && connection?.method === expected.method &&
run.contextSnapshot?.paperclipEnvironment?.id === expected.environmentId;
}
/** Wait for scheduled repairs to settle, then select the run that actually delivered the file. */
export function completedConnectionArtifactRun(
runs: Record<string, any>[],
attachments: Record<string, any>[],
filename: string,
expected: Parameters<typeof verifyConnectionRun>[1],
) {
const terminal = ["succeeded", "failed", "cancelled", "interrupted", "timed_out"];
if (runs.some(run => !terminal.includes(run.status))) return undefined;
return runs.find(run => verifyConnectionRun(run, expected) && attachments.some(attachment =>
attachment.originalFilename === filename && attachment.createdByAgentId === expected.agentId && attachment.originatingRunId === run.id));
}
tests/runner-e2e/connection-flow.ts
+377
View File
@@ -0,0 +1,377 @@
import { createHash } from "node:crypto";
import { expect, type Page } from "@playwright/test";
import { aiRoutingModel } from "../../packages/shared/src/ai-provider-routing.js";
import { getConnectableAppDefinition } from "../../packages/shared/src/app-definitions.js";
import { connectionCell, type ConnectionHarness, type ConnectionMode } from "./connection-cases.js";
import { ConnectionBlock, ConnectionFailure, type ConnectionConfig, type resolveConnectionSettings } from "./connection-config.js";
import { ConnectionApi, connectionDelay } from "./connection-target.js";
import { createConnectionProof, verifyConnectionArtifact, verifyConnectionRun, completedConnectionArtifactRun, connectionProbeChecks, type ConnectionEvidence } from "./connection-evidence.js";
import { createTaskThroughUi, submitTaskReply } from "./user-actions.js";
import { collectRunEvents } from "./run-observations.js";
import type { MatrixExecution } from "./types.js";
type Row = Record<string, any>;
type Settings = ReturnType<typeof resolveConnectionSettings>;
export interface ConnectionFlowInput {
page: Page; api: ConnectionApi; execution: MatrixExecution; config: ConnectionConfig; settings: Settings;
company: Row; environment: Row; secret?: string; nonce: string; privateDir: string;
evidence: ConnectionEvidence;
checkpoint(phase: string): Promise<void>;
observeRun(run: Row): void;
createdConnectionIds: ReadonlySet<string>;
refreshOwnedConnections(): Promise<void>;
assertActive(): void;
}
async function choose(page: Page, label: string, name: string) {
await page.getByRole("combobox", { name: label, exact: true }).click();
await page.getByRole("option", { name, exact: true }).click();
}
async function openAdvanced(page: Page, mode: ConnectionMode) {
const names: Record<string, string> = { openrouter: "OpenRouter", bedrock: "Amazon Bedrock", responses: "Custom gateway", messages: "Custom gateway", chat: "Custom gateway" };
await page.getByRole("button", { name: new RegExp(`^${names[mode]}`) }).click();
}
async function fillGateway(input: ConnectionFlowInput) {
const { page, settings, secret } = input;
const { mode } = connectionCell(input.execution);
if (settings.route?.baseURL) {
await page.getByLabel("Provider URL", { exact: true }).fill(settings.route.baseURL);
const format = page.getByRole("combobox", { name: "API format", exact: true });
if (await format.isEnabled()) await choose(page, "API format", ({ responses: "OpenAI Responses · Codex", messages: "Anthropic Messages · Claude", chat: "Chat Completions · OpenCode, Hermes" } as Record<string, string>)[mode]!);
}
if (mode === "bedrock") await page.getByLabel("AWS region", { exact: true }).fill(settings.route!.region!);
if (settings.route?.auth !== "bearer") await choose(page, "Authentication", settings.route?.auth === "none" ? "No authentication" : "API key · x-api-key");
if (secret) await page.getByLabel("API key", { exact: true }).fill(secret);
// Normal defaults stay collapsed. Model alias is selected in Configure.
await page.getByRole("button", { name: "Connect", exact: true }).last().click();
}
async function submitConnection(input: ConnectionFlowInput, submit: () => Promise<void>) {
const route = `${input.api.origin}/api/companies/${input.company.id}/ai-connections`;
const response = input.page.waitForResponse(response => response.url() === route && response.request().method() === "POST", { timeout: 60_000 }).catch(() => null);
await submit();
const created = await response;
if (!created) throw new ConnectionFailure("connection_request_timeout");
if (created.ok()) return;
const payload = await created.json().catch(() => ({}));
const allowed = ["ai_connection_api_key_rejected", "ai_connection_verification_failed"];
const code = allowed.includes(payload?.details?.code) ? payload.details.code : "connection_request_rejected";
input.evidence.creationFailure = { status: created.status(), code };
if (code === "ai_connection_api_key_rejected") throw new ConnectionBlock("missing_credential", code);
throw new ConnectionFailure(code);
}
async function waitForFreshConnection(input: ConnectionFlowInput, before: Set<string>) {
const { page, api, config, evidence, company } = input;
const { harness, mode } = connectionCell(input.execution);
const deadline = Date.now() + (mode === "subscription" ? config.browser.loginTimeoutMs : 90_000);
let loginOpened = false;
let connected = false;
while (Date.now() < deadline) {
input.assertActive();
await input.refreshOwnedConnections();
const list = await api.get(`/api/companies/${company.id}/ai-connections`);
const fresh = list.connections.filter((c: Row) => !before.has(c.id) && input.createdConnectionIds.has(c.id));
if (fresh.length > 1) throw new ConnectionFailure("unexpected_multiple_connections");
if (fresh.length === 1) {
const connection = fresh[0];
if (connection.status !== "connected" || connection.method !== (mode === "subscription" ? "subscription" : "api_key")) throw new ConnectionFailure("saved_connection_method_or_status_mismatch");
return connection;
}
if (mode === "subscription") {
const signIn = page.getByRole("link", { name: /^Sign in to / }).first();
const cloudSignIn = page.getByRole("button", { name: /^Sign in to / }).first();
if (!loginOpened && (await signIn.isVisible() || await cloudSignIn.isVisible())) {
// Observe before clicking. No token, code, URL, or popup content is recorded.
const popup = page.context().waitForEvent("page", { timeout: 10_000 }).catch(() => undefined);
if (await signIn.isVisible()) await signIn.click(); else if (await cloudSignIn.isEnabled()) await cloudSignIn.click();
await popup;
loginOpened = true;
evidence.assisted = true;
evidence.waits.push({ kind: "provider_login", startedAt: new Date().toISOString() });
await input.checkpoint("awaiting_provider_login");
console.log(`[provider-connections] Sign in to ${harness.label} in the opened browser (${config.browser.accountAlias}); complete the code/consent step in Paperclip. This test resumes automatically.`);
}
// Grok currently exposes a terminal handoff; do not pretend it has the same browser flow.
if (!loginOpened && harness.id === "grok" && await page.getByRole("button", { name: "Copy sign-in command" }).isVisible()) {
loginOpened = true;
evidence.assisted = true;
evidence.waits.push({ kind: "provider_login", startedAt: new Date().toISOString() });
await input.checkpoint("awaiting_provider_login");
console.log("[provider-connections] Grok requires the sign-in command shown in Paperclip. Complete that production handoff; the test will resume automatically.");
}
const connect = page.getByRole("button", { name: "Connect", exact: true }).last();
if (!connected && await connect.isVisible() && await connect.isEnabled()) { await connect.click(); connected = true; }
if (await page.getByRole("alert").filter({ hasText: /expired|could not|failed|invalid/i }).count()) throw new ConnectionFailure("provider_login_reported_error");
}
await connectionDelay(1_000);
}
if (mode === "subscription" && loginOpened) throw new ConnectionBlock("awaiting_user", "provider_login_deadline_reached");
throw new ConnectionFailure("connection_creation_timeout");
}
async function selectSavedConnection(page: Page, connection: Row, harness: ConnectionHarness, mode: ConnectionMode) {
await page.getByRole("heading", { name: /^(Connect a model|Configure your agent)$/ }).waitFor();
const advanced = !["subscription", "api-key"].includes(mode);
const connectionType = page.getByRole("radiogroup", { name: "Connection type", exact: true });
if (["claude", "codex", "grok"].includes(harness.id)) {
await connectionType.waitFor({ state: "visible" });
if (advanced) {
await connectionType.getByRole("radio", { name: /Advanced/ }).click();
await choose(page, "Connection", connection.name);
await page.getByRole("button", { name: "Use connection", exact: true }).click();
} else {
await connectionType.getByRole("radio", { name: mode === "subscription" ? /Subscription/ : /API key/ }).click();
const saved = page.getByRole("combobox", { name: mode === "subscription" ? "Saved subscription" : "Saved API key", exact: true });
await saved.waitFor({ state: "visible" });
const option = await saved.locator("option").filter({ hasText: connection.name }).first().getAttribute("value");
if (!option) throw new ConnectionFailure("saved_connection_option_missing");
await saved.selectOption(option);
await page.getByRole("button", { name: mode === "subscription" ? "Use saved subscription" : "Use saved API key", exact: true }).click();
}
} else {
await choose(page, "Connection", connection.name);
}
}
async function createConnectionThroughUi(input: ConnectionFlowInput, agentURL: string) {
const { page, api, company, evidence } = input;
const { harness, mode, entry } = connectionCell(input.execution);
const priorConnections = (await api.get(`/api/companies/${company.id}/ai-connections`)).connections as Row[];
const before = new Set<string>(priorConnections.map(c => c.id));
// Fresh-creation proof cannot be satisfied by an existing personal default.
if (priorConnections.some(connection => connection.status !== "revoked")) throw new ConnectionBlock("blocked_target", "fresh_connection_requires_empty_qa_company");
const advanced = !["subscription", "api-key"].includes(mode);
await input.checkpoint("connection_navigation");
if (entry === "apps") {
await page.goto(`${api.origin}/${company.issuePrefix}/apps`, { waitUntil: "domcontentloaded" });
const catalogSlug = advanced ? ({ openrouter: "openrouter", bedrock: "bedrock", responses: "responses-api", messages: "messages-api", chat: "chat-completions-api" } as Record<string, string>)[mode]! : harness.provider;
const definition = getConnectableAppDefinition(catalogSlug);
if (!definition) throw new ConnectionFailure("connection_catalog_entry_missing");
await page.getByRole("button", { name: `Connect ${definition.name}`, exact: true }).click();
const next = page.getByRole("button", { name: /^(Save and continue|Continue)$/ }).last();
if (!advanced && await next.isVisible()) await next.click();
if (!advanced && harness.provider !== "google") {
// The credential step loads asynchronously. Wait for its provider tile
// before inspecting the mode switch, otherwise a still-loading page can
// skip the API-key switch and leave the subscription form selected.
await page.getByRole("radio", { name: new RegExp(harness.id === "grok" ? "Grok" : harness.label) }).click();
const switcher = page.getByRole("button", { name: mode === "subscription" ? "Use subscription instead" : "Use API key instead", exact: true });
if (await switcher.isVisible()) await switcher.click();
}
} else {
await page.goto(agentURL, { waitUntil: "domcontentloaded" });
const tiles = page.getByRole("radiogroup", { name: "Connection type", exact: true });
if (["claude", "codex", "grok"].includes(harness.id)) {
const choice = tiles.getByRole("radio", { name: advanced ? /Advanced/ : mode === "subscription" ? /Subscription/ : /API key/ });
await choice.click();
await expect(choice).toHaveAttribute("aria-checked", "true");
}
if (advanced || !["claude", "codex", "grok"].includes(harness.id)) {
await choose(page, "Connection", "Connect an account…");
if (advanced) {
const advancedProviders = page.locator("summary").filter({ hasText: "Advanced providers" });
if (await advancedProviders.isVisible()) {
await advancedProviders.click();
// The account dialog's disclosure leads to the provider chooser;
// the chooser's own disclosure already contains provider buttons.
const chooseProvider = page.getByRole("button", { name: "Choose another provider or gateway", exact: true });
if (await chooseProvider.isVisible()) await chooseProvider.click();
}
await openAdvanced(page, mode);
}
}
}
await input.checkpoint("connection_creation");
if (advanced) await submitConnection(input, () => fillGateway(input));
else if (mode === "api-key") {
const savedKey = page.getByRole("combobox", { name: "Saved API key", exact: true });
if (await savedKey.isVisible()) await savedKey.selectOption("");
await page.getByLabel("API key", { exact: true }).fill(input.secret!);
await submitConnection(input, () => page.getByRole("button", { name: "Connect", exact: true }).last().click());
}
const connection = await waitForFreshConnection(input, before);
evidence.connectionId = connection.id;
evidence.checkpoints.connection_created = true;
evidence.waits.forEach(wait => { wait.finishedAt ??= new Date().toISOString(); });
if (advanced) {
const expectedKind = ["bedrock", "openrouter"].includes(mode) ? mode : "gateway";
if (connection.routing?.kind !== expectedKind || (mode !== "openrouter" && connection.routing?.protocol !== mode) ||
(input.settings.route?.baseURL && connection.routing?.baseUrl !== input.settings.route.baseURL) ||
(mode === "bedrock" && connection.routing?.region !== input.settings.route?.region)) throw new ConnectionFailure("saved_routing_mismatch");
} else if (connection.provider !== harness.provider) throw new ConnectionFailure("saved_provider_mismatch");
// Navigate away and return through the normal list: proof includes durable UI state.
await page.goto(`${api.origin}/${company.issuePrefix}/apps`, { waitUntil: "domcontentloaded" });
await expect(page.getByText(connection.name, { exact: true }).first()).toBeVisible();
await page.reload({ waitUntil: "domcontentloaded" });
await expect(page.getByText(connection.name, { exact: true }).first()).toBeVisible();
evidence.checkpoints.connection_reloaded = true;
await input.checkpoint("connection_reloaded");
await page.goto(agentURL, { waitUntil: "domcontentloaded" });
await input.checkpoint("select_saved_connection");
await selectSavedConnection(page, connection, harness, mode);
await input.checkpoint("configure_agent");
return connection;
}
async function selectModel(page: Page, model: string) {
const select = page.getByRole("combobox", { name: "Model", exact: true });
await select.waitFor({ state: "visible" });
await expect(select).not.toHaveAttribute("aria-busy", "true", { timeout: 60_000 });
const values = await select.locator("option").evaluateAll(options => options.map(option => (option as HTMLOptionElement).value));
const matched = values.find(value => value === model || value === `openrouter/${model}`);
if (matched) await select.selectOption(matched);
else {
await select.selectOption({ label: "Enter custom model…" });
await page.getByLabel("Model ID", { exact: true }).fill(model);
}
}
export async function runConnectionFlow(input: ConnectionFlowInput) {
const { page, api, execution, evidence, settings, company, environment, config } = input;
const { harness, mode } = connectionCell(execution);
const agentName = `Connection QA ${input.nonce}`;
const agentURL = `${api.origin}/${company.issuePrefix}/agents/new?${new URLSearchParams({ name: agentName, adapterType: execution.profile.adapterType, runnerProvider: harness.id })}`;
const connection = await createConnectionThroughUi(input, agentURL);
await expect(page.getByRole("heading", { name: "Configure your agent", exact: true })).toBeVisible({ timeout: 120_000 });
const environmentSelect = page.getByRole("combobox", { name: "Environment", exact: true });
const selectedEnv = await environmentSelect.inputValue();
if (selectedEnv !== environment.id) {
if (!(await environmentSelect.isEnabled())) throw new ConnectionBlock("blocked_target", "requested_execution_environment_unavailable");
await environmentSelect.selectOption(environment.id);
if (["claude", "codex", "grok"].includes(harness.id)) await selectSavedConnection(page, connection, harness, mode);
}
if (await environmentSelect.inputValue() !== environment.id) throw new ConnectionBlock("blocked_target", "requested_execution_environment_unavailable");
await input.checkpoint("select_model");
await selectModel(page, settings.model);
await input.checkpoint("setup_probe");
const response = page.waitForResponse(response => response.url().startsWith(`${api.origin}/api/companies/${company.id}/adapters/`) && response.url().endsWith("/test-environment"), { timeout: 120_000 });
await page.getByRole("button", { name: /^(Run test|Test again)$/ }).click();
const probe = await response;
const result = await probe.json();
evidence.setupChecks = connectionProbeChecks(result);
if (evidence.setupChecks.some(check => check.level === "error" && (check.code === "hermes_cli_not_found" || /^(claude|codex|grok|opencode|gemini)_command_unresolvable$/.test(check.code))))
throw new ConnectionBlock("blocked_target", "runtime_cli_required");
if (!probe.ok() || result.status !== "pass") throw new ConnectionFailure("setup_probe_failed");
evidence.checkpoints.setup_probe = true;
await page.getByRole("button", { name: "Finish setup", exact: true }).click();
let agent: Row | undefined;
const deadline = Date.now() + 60_000;
while (!agent && Date.now() < deadline) {
agent = (await api.get<Row[]>(`/api/companies/${company.id}/agents`)).find(row => row.name === agentName);
if (!agent) await connectionDelay(500);
}
if (!agent) throw new ConnectionFailure("agent_creation_failed");
if (agent.defaultEnvironmentId !== environment.id) {
// A null override is valid only when the instance default is this exact environment.
const instance = await api.get<Row>("/api/instance/settings");
const effective = agent.defaultEnvironmentId ?? instance.defaultEnvironmentId
?? (environment.driver === "local" ? environment.id : null);
if (effective !== environment.id) throw new ConnectionFailure("saved_agent_environment_mismatch");
}
evidence.agentId = agent.id;
evidence.checkpoints.agent_created = true;
const expectedModel = connection.routing
? aiRoutingModel(connection.routing, harness.adapter, settings.model)
: settings.model;
if (agent.adapterType !== execution.profile.adapterType || agent.adapterConfig?.model !== expectedModel) throw new ConnectionFailure("agent_harness_or_model_mismatch");
const binding = agent.runtimeConfig?.aiConnection;
if (!binding || binding.method !== connection.method || (binding.mode !== "responsible_user" && binding.connectionId !== connection.id)) throw new ConnectionFailure("agent_binding_mismatch");
// Fixtures may constrain cost/scheduling, but cannot repair or replace the tested credentials/config.
await api.patch(`/api/agents/${agent.id}`, { budgetMonthlyCents: config.budgetCents });
evidence.checkpoints.binding = true;
await input.checkpoint("first_task");
const proof = createConnectionProof();
// Provider qualification must not depend on task-level attachment ingestion.
// Native runners admit chat attachments, while the new-task dialog uploads
// its files after scheduling. Supply identical oracle bytes in the task so
// every harness can decode them with its actual tools.
evidence.inputTransport = "prompt_base64";
const title = `Connection artifact ${input.nonce}`;
await createTaskThroughUi({ page, issuePrefix: company.issuePrefix, agentName, title, workMode: "standard", prompt: [
"Complete this bounded connection acceptance task. Using your actual tools, decode the following base64 into connection-input.json and read the file. Preserve the exact decoded bytes, including the final newline.",
`Input bytes (base64): ${Buffer.from(proof.input, "utf8").toString("base64")}`,
"Create the input and output files inside your current workspace using relative filenames. Do not put them in /tmp or another directory outside the workspace.",
'Write connection-proof.json with exactly this schema: {"nonce": string, "count": number, "total": number, "sha256": string}. Set nonce from the input, count to the number of values, total to their sum, and sha256 to the hash of the exact input file bytes. All four fields are required; use the literal field name "total", not "sum". Compute and validate with a tool; do not guess or paraphrase the data.',
"Deliver connection-proof.json as a downloadable attachment on this task. Use the normal Paperclip artifact/attachment workflow and include its download link in your final response. Then mark this task done. Do not create other tasks, read credentials, or inspect unrelated files.",
execution.profile.generation === "native" ? "Use your native register_deliverable tool with workspace-relative contentRef connection-proof.json and the exact byteSize and SHA-256 of that output file. Never submit an absolute contentRef." : "",
].join("\n") });
let issue: Row | undefined;
const createdDeadline = Date.now() + 30_000;
while (!issue && Date.now() < createdDeadline) {
issue = (await api.get<Row[]>(`/api/companies/${company.id}/issues`)).find(row => row.title === title);
if (!issue) await connectionDelay(500);
}
if (!issue) throw new ConnectionFailure("task_creation_failed");
const runs = new Map<string, Row>();
async function completedRun(after: Set<string>, filename: string) {
const deadline = Date.now() + config.turnTimeoutMs;
while (Date.now() < deadline) {
input.assertActive();
const summaries = await api.get<Row[]>(`/api/issues/${issue!.id}/runs`);
for (const row of summaries) {
const id = row.runId ?? row.id;
if (id && !runs.has(id)) runs.set(id, await api.get(`/api/heartbeat-runs/${id}`));
else if (id && !["succeeded", "failed", "cancelled", "interrupted", "timed_out"].includes(runs.get(id)!.status)) runs.set(id, await api.get(`/api/heartbeat-runs/${id}`));
if (id) input.observeRun(runs.get(id)!);
}
if (runs.size > config.maxRuns) throw new ConnectionFailure("provider_run_limit_exceeded");
const next = [...runs.values()].filter(run => !after.has(run.id));
if (next.some(run => ["failed", "cancelled", "interrupted", "timed_out"].includes(run.status))) throw new ConnectionFailure("agent_run_failed");
const complete = next.find(run => verifyConnectionRun(run, { agentId: agent!.id, connectionId: connection.id, method: connection.method, runtimeMode: execution.profile.generation, environmentId: environment.id }));
if (complete) {
const status = (await api.get(`/api/issues/${issue!.id}`)).status;
if (status === "done") {
// A disposition repair may still be running after an earlier turn
// succeeded. Qualify the settled successful run that uploaded this
// artifact, preserving the configured run and time limits.
const attachments = await api.get<Row[]>(`/api/issues/${issue!.id}/attachments`);
const delivered = completedConnectionArtifactRun(next, attachments, filename,
{ agentId: agent!.id, connectionId: connection.id, method: connection.method, runtimeMode: execution.profile.generation, environmentId: environment.id });
if (delivered) return delivered;
if (next.every(run => ["succeeded", "failed", "cancelled", "interrupted", "timed_out"].includes(run.status))) throw new ConnectionFailure("run_attributed_artifact_missing");
}
if (status === "blocked") throw new ConnectionFailure("task_blocked_after_provider_run");
}
if (next.some(run => run.status === "succeeded" && !verifyConnectionRun(run, { agentId: agent!.id, connectionId: connection.id, method: connection.method, runtimeMode: execution.profile.generation, environmentId: environment.id }))) throw new ConnectionFailure("run_attribution_mismatch");
await connectionDelay(1500);
}
throw new ConnectionFailure("agent_run_or_attribution_timeout");
}
async function artifact(filename: string, expected: Record<string, unknown>, run: Row) {
const attachments = await api.get<Row[]>(`/api/issues/${issue!.id}/attachments`);
const attachment = attachments.find(row => row.originalFilename === filename && row.createdByAgentId === agent!.id && row.originatingRunId === run.id);
if (!attachment) throw new ConnectionFailure("run_attributed_artifact_missing");
const bytes = await api.bytes(`/api/attachments/${attachment.id}/content`);
let actual: Record<string, unknown> = {};
try { actual = JSON.parse(bytes.toString("utf8")); } catch { /* The independent verifier fails below. */ }
evidence.artifactChecks ??= [];
evidence.artifactChecks.push({ filename, runId: run.id, sha256: createHash("sha256").update(bytes).digest("hex"),
fields: Object.fromEntries(Object.entries(expected).map(([key, value]) => [key, actual?.[key] === value])), exactFields: Boolean(actual && Object.keys(actual).length === Object.keys(expected).length) });
if (!verifyConnectionArtifact(bytes, expected)) throw new ConnectionFailure("independent_artifact_verification_failed");
await page.goto(`${api.origin}/${company.issuePrefix}/issues/${issue!.identifier}`, { waitUntil: "domcontentloaded" });
await page.reload({ waitUntil: "domcontentloaded" });
await expect(page.locator(`a[href*="/api/attachments/${attachment.id}/content"]`).first()).toBeVisible({ timeout: 30_000 });
evidence.artifactSha256 = createHash("sha256").update(bytes).digest("hex");
}
const first = await completedRun(new Set(), "connection-proof.json");
evidence.checkpoints.first_run = true;
await artifact("connection-proof.json", proof.expected, first);
evidence.checkpoints.artifact = true;
// Artifact upload provenance is a server-owned tool-side effect in both runner generations.
const events = await collectRunEvents<Row>((afterSeq, limit) => api.get(`/api/heartbeat-runs/${first.id}/events?afterSeq=${afterSeq}&limit=${limit}`));
const toolReceipt = events.some(event => event.eventType === "tool.execution.completed" || (event.eventType === "item.completed" && ["tool_result", "command_execution"].includes(event.payload?.prpEvent?.payload?.item?.type)));
// Legacy adapters may expose their tool transcript only in the run log; upload attribution is still a durable successful tool effect.
evidence.checkpoints.tool_receipt = execution.profile.generation === "legacy" || toolReceipt;
if (!evidence.checkpoints.tool_receipt) throw new ConnectionFailure("native_tool_receipt_missing");
await input.checkpoint("followup");
const previousRuns = new Set(runs.keys());
await submitTaskReply(page, "Using the original connection-input.json decoded from this task, deliver connection-followup.json with exactly nonce, minimum, maximum, and total of the original values. Use exactly this JSON schema: {\"nonce\": string, \"minimum\": number, \"maximum\": number, \"total\": number}. All four fields are required. Compute with a tool and validate the file against all four fields before attaching it and its download link. Then mark this task done. Do not create other tasks or read credentials.");
const second = await completedRun(previousRuns, "connection-followup.json");
await artifact("connection-followup.json", proof.followup, second);
evidence.checkpoints.followup = true;
evidence.checkpoints.visible_result = true;
return { issueId: issue.id as string, issueIdentifier: issue.identifier as string };
}
tests/runner-e2e/connection-launch.ts
+394
View File
@@ -0,0 +1,394 @@
import { chromium, type BrowserContext } from "@playwright/test";
import { execFileSync } from "node:child_process";
import { randomBytes, createHash } from "node:crypto";
import { mkdir, mkdtemp, writeFile, rm, readdir, lstat, open } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { FixtureRegistry } from "./fixture-registry.js";
import { buildRunnerCampaign } from "./history.js";
import { renderRunnerE2EDashboard } from "./dashboard.js";
import { buildRuntimeUsage, summarizeExecutionBilling } from "./billing.js";
import { stageDashboardBrandAssets } from "./report-assets.js";
import { classifyFailure } from "./failure-classifier.js";
import { packageEvidence } from "./evidence.js";
import { assertSecretFree, sanitizeJson } from "./redaction.js";
import { connectionCell, connectionHarnesses, connectionModes, connectionScopeGaps, supportsConnection } from "./connection-cases.js";
import { assertOutsideRepository, ConnectionBlock, ConnectionFailure, loadConnectionConfig, resolveConnectionSecret, resolveConnectionSettings, type ConnectionConfig } from "./connection-config.js";
import { connectionEvidencePasses, connectionRunDiagnostic, gradeConnectionEvidence, type ConnectionEvidence } from "./connection-evidence.js";
import { ConnectionApi, connectionDelay, startConnectionTarget, verifyConnectionTarget } from "./connection-target.js";
import { runConnectionFlow } from "./connection-flow.js";
import type { MatrixExecution, RunnerE2EResult } from "./types.js";
type Row = Record<string, any>;
export function connectionCreationReceipt(pathname: string, method: string, companyId: string, loginIds: ReadonlySet<string>, body: unknown): string | undefined {
const root = `/api/companies/${companyId}/ai-connections`;
const loginId = pathname.startsWith(`${root}/login/`) ? pathname.slice(`${root}/login/`.length) : undefined;
const createdHere = method === "POST" && (pathname === root || pathname === `${root}/local`);
const ownedLogin = method === "GET" && loginId && loginIds.has(loginId);
if (!createdHere && !ownedLogin) return;
const id = (body as { connectionId?: unknown } | null)?.connectionId;
return typeof id === "string" && /^[a-f0-9]{8}-(?:[a-f0-9]{4}-){3}[a-f0-9]{12}$/i.test(id) ? id : undefined;
}
export async function cleanupConnectionCompany(input: {
api: ConnectionApi; companyId: string; agentName: string;
attachedCompany: boolean; retainCompany: boolean;
createdConnectionIds: ReadonlySet<string>;
evidence: Pick<ConnectionEvidence, "cleanupRetained" | "companyArchived">;
collectDiagnostics: () => Promise<void>;
}) {
const { api, companyId, agentName, evidence } = input;
const agents = await api.get<Row[]>(`/api/companies/${companyId}/agents`);
const fixtureAgents = agents.filter(agent => agent.name === agentName);
try {
for (const agent of fixtureAgents) {
await api.patch(`/api/agents/${agent.id}`, { status: "paused" });
const liveRuns = await api.get<Row[]>(`/api/companies/${companyId}/heartbeat-runs?agentId=${agent.id}&limit=100`);
for (const run of liveRuns.filter(run => run.agentId === agent.id && ["queued", "running"].includes(run.status))) {
await api.post(`/api/heartbeat-runs/${run.id}/cancel`);
}
}
} finally {
// Deleting an attached-company agent also deletes its runs and logs.
// Capture the stopped runs first, including when cancellation fails.
await input.collectDiagnostics();
}
if (input.retainCompany) { evidence.cleanupRetained = true; return; }
const connections = (await api.get(`/api/companies/${companyId}/ai-connections`)).connections;
for (const connection of connections.filter((connection: Row) => input.createdConnectionIds.has(connection.id) && connection.status !== "revoked")) {
await api.delete(`/api/tool-connections/${connection.id}`);
}
if ((await api.get(`/api/companies/${companyId}/ai-connections`)).connections.some((connection: Row) => input.createdConnectionIds.has(connection.id) && connection.status !== "revoked")) {
throw new ConnectionFailure("connection_revocation_failed");
}
if (!input.attachedCompany) { await api.post(`/api/companies/${companyId}/archive`); evidence.companyArchived = true; }
else for (const agent of fixtureAgents) await api.delete(`/api/agents/${agent.id}`);
}
export async function openConnectionBrowser(config: ConnectionConfig, repositoryRoot: string, origin: string) {
const temporary = config.browser.freshness === "signed-out" || !config.browser.profileDir;
const profile = temporary ? await mkdtemp(path.join(os.tmpdir(), "paperclip-qa-browser-")) : path.resolve(config.browser.profileDir!);
await assertOutsideRepository(profile, repositoryRoot);
await mkdir(profile, { recursive: true, mode: 0o700 });
const stat = await lstat(profile);
if (stat.isSymbolicLink() || !stat.isDirectory() || (stat.mode & 0o077) !== 0) throw new ConnectionBlock("blocked_target", "browser_profile_requires_private_directory");
const marker = path.join(profile, ".paperclip-provider-qa");
const entries = await readdir(profile);
if (entries.length && !entries.includes(path.basename(marker))) throw new ConnectionBlock("blocked_target", "use_a_dedicated_empty_qa_browser_profile");
if (!entries.length) await writeFile(marker, "Dedicated Paperclip provider QA browser. Contains private authentication state.\n", { mode: 0o600, flag: "wx" });
const lock = path.join(profile, ".paperclip-qa.lock");
try { await (await open(lock, "wx", 0o600)).close(); }
catch { throw new ConnectionBlock("blocked_target", "qa_browser_profile_in_use"); }
let context: BrowserContext;
try {
const browserEnv = Object.fromEntries(["PATH", "HOME", "USER", "LOGNAME", "TMPDIR", "DISPLAY", "WAYLAND_DISPLAY", "XAUTHORITY", "LANG"].flatMap(key => process.env[key] ? [[key, process.env[key]!]] : []));
context = await chromium.launchPersistentContext(profile, {
channel: config.browser.channel === "chrome" ? "chrome" : undefined,
headless: !config.browser.headed, baseURL: origin, viewport: { width: 1440, height: 1000 }, reducedMotion: "reduce",
env: browserEnv, acceptDownloads: true,
// Teardown needs this context's authenticated request client to cancel
// login sessions and revoke fixtures before the browser is closed.
handleSIGINT: false, handleSIGTERM: false, handleSIGHUP: false,
// Deliberately use the Playwright library without Test's automatic reporters,
// traces, HARs, videos, snapshots, assertion call logs, or failure screenshots.
});
context.setDefaultTimeout(30_000);
context.setDefaultNavigationTimeout(30_000);
} catch {
await rm(lock, { force: true });
if (temporary) await rm(profile, { recursive: true, force: true });
throw new ConnectionBlock("blocked_target", "qa_browser_launch_failed");
}
return { context, close: async () => { try { await context.close(); } finally { await rm(lock, { force: true }); if (temporary) await rm(profile, { recursive: true, force: true }); } } };
}
export function safeConnectionFailure(error: unknown, phase: string) {
if (error instanceof ConnectionBlock) return error.code;
// Arbitrary Playwright/provider messages can contain filled values and URLs.
const known = error instanceof ConnectionFailure;
const frame = error instanceof Error ? error.stack?.match(/connection-(?:flow|target|launch)\.ts:\d+:\d+/)?.[0] : undefined;
return known ? error.message : `${phase}_failed${frame ? ` (${frame})` : ""}`;
}
function sourceRevision(repositoryRoot: string) {
const git = (args: string[]) => execFileSync("git", args, { cwd: repositoryRoot, encoding: "utf8", stdio: ["ignore", "pipe", "ignore"] }).trim();
return { sha: git(["rev-parse", "HEAD"]), ref: `${git(["branch", "--show-current"])}${git(["status", "--porcelain"]) ? " (working tree)" : ""}`, workflowRunUrl: null };
}
async function safeWrite(file: string, value: unknown, secrets: readonly string[]) {
const text = `${JSON.stringify(sanitizeJson(value, secrets), null, 2)}\n`;
assertSecretFree(text, secrets, path.basename(file));
await writeFile(file, text, { mode: 0o600 });
}
async function connectionAttempt(execution: MatrixExecution, config: ConnectionConfig, origin: string, summaryDir: string, repositoryRoot: string, signal: AbortSignal): Promise<RunnerE2EResult> {
const startedAt = new Date().toISOString();
const nonce = randomBytes(6).toString("hex");
const privateDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-connection-proof-"));
const evidenceDir = path.join(summaryDir, execution.suite.id, execution.profile.id, execution.environment.id, execution.task.id, "attempt-1");
await mkdir(evidenceDir, { recursive: true, mode: 0o700 });
const { entry, mode } = connectionCell(execution);
const evidence: ConnectionEvidence = { version: 1, outcome: "running", phase: "target", assisted: false, authFreshness: config.browser.freshness,
target: { mode: config.target.mode, origin, commit: null, deploymentMode: null }, entry, method: mode, checkpoints: {}, waits: [] };
const secrets: string[] = [];
const checkpoint = async (phase: string) => { evidence.phase = phase; await safeWrite(path.join(summaryDir, "progress.json"), { executionId: execution.id, ...evidence }, secrets); };
const result: RunnerE2EResult = { schema: "paperclip.runner-e2e.result/v2", executionId: execution.id, suiteId: execution.suite.id, suiteDefinitionHash: execution.suiteDefinitionHash,
source: sourceRevision(repositoryRoot), attempt: 1, status: "failed", profileId: execution.profile.id, environmentId: execution.environment.id, caseId: execution.task.id,
provider: execution.profile.provider, model: execution.profile.model, runtimeMode: execution.profile.generation, startedAt, finishedAt: startedAt, durationMs: 0, cleanup: "not_started", providerConnection: evidence };
let browser: Awaited<ReturnType<typeof openConnectionBrowser>> | undefined;
let fixtures: Awaited<ReturnType<FixtureRegistry["setupAll"]>> | undefined;
const observedRuns = new Map<string, Row>();
const createdConnectionIds = new Set<string>();
const ownedLoginIds = new Set<string>();
let fixtureCompanyId: string | undefined;
let diagnosticApi: ConnectionApi | undefined;
let diagnosticsCollected = false;
const collectDiagnostics = async () => {
if (diagnosticsCollected || !diagnosticApi || !observedRuns.size) return;
evidence.runDiagnostics = [];
for (const observed of observedRuns.values()) {
let run = observed; let log: unknown;
try { run = await diagnosticApi.get(`/api/heartbeat-runs/${observed.id}`); observedRuns.set(observed.id, run); } catch { /* Keep the last durable observation. */ }
try { log = await diagnosticApi.get(`/api/heartbeat-runs/${observed.id}/log?limitBytes=2000000`); } catch { /* Mark missing log evidence explicitly. */ }
evidence.runDiagnostics.push(connectionRunDiagnostic(run, log));
}
diagnosticsCollected = true;
};
const loginSessions = new Map<string, "DELETE" | "POST">();
const loginObservers: Promise<void>[] = [];
let stopped: "test_interrupted" | "cell_deadline_reached" | undefined;
const stopBrowserActions = () => { for (const page of browser?.context.pages() ?? []) void page.close().catch(() => {}); };
const interrupt = () => { stopped = "test_interrupted"; stopBrowserActions(); };
const assertActive = () => { if (stopped) throw new ConnectionBlock("blocked_target", stopped); };
signal.addEventListener("abort", interrupt, { once: true });
if (signal.aborted) interrupt();
const deadlineTimer = setTimeout(() => { stopped = "cell_deadline_reached"; stopBrowserActions(); }, execution.task.attemptTimeoutMs?.[execution.environment.id] ?? 30 * 60_000);
try {
assertActive();
const identity = await verifyConnectionTarget(origin, config);
Object.assign(evidence.target, identity);
evidence.checkpoints.target = true;
assertActive();
const settings = resolveConnectionSettings(execution, config);
result.model = settings.model;
evidence.model = settings.model;
if (mode === "subscription" && !config.browser.headed) throw new ConnectionBlock("awaiting_user", "subscription_requires_headed_browser");
// Resolve only the selected credential, and only after validating the target.
const secret = await resolveConnectionSecret(config, settings.credentialEnv);
if (secret) secrets.push(secret);
assertActive();
browser = await openConnectionBrowser(config, repositoryRoot, origin);
assertActive();
const page = browser.context.pages()[0] ?? await browser.context.newPage();
const api = new ConnectionApi(browser.context.request, origin);
diagnosticApi = api;
browser.context.on("response", response => {
const url = new URL(response.url());
if (url.origin !== origin || !fixtureCompanyId || !url.pathname.startsWith(`/api/companies/${fixtureCompanyId}/`) || !response.ok()) return;
const method = response.request().method();
const local = method === "POST" && /^\/api\/companies\/([a-f0-9-]+)\/ai-connections\/local\/attempts$/.test(url.pathname);
const cloud = method === "POST" && /^\/api\/companies\/([a-f0-9-]+)\/(?:setup-token-login-sessions|adapters\/[^/]+\/login-sessions)$/.test(url.pathname);
const root = `/api/companies/${fixtureCompanyId}/ai-connections`;
const receipt = (method === "POST" && [root, `${root}/local`].includes(url.pathname)) || (method === "GET" && url.pathname.startsWith(`${root}/login/`));
if (!local && !cloud && !receipt) return;
loginObservers.push((async () => {
const body = await response.json();
if ((local || cloud) && typeof body.sessionId === "string" && /^[a-zA-Z0-9_-]+$/.test(body.sessionId)) {
ownedLoginIds.add(body.sessionId);
loginSessions.set(`${url.pathname}/${body.sessionId}${cloud ? "/cancel" : ""}`, local ? "DELETE" : "POST");
}
const id = connectionCreationReceipt(url.pathname, method, fixtureCompanyId!, ownedLoginIds, body);
if (id) createdConnectionIds.add(id);
})().catch(() => {}));
});
const refreshOwnedConnections = async () => {
if (!fixtureCompanyId) return;
for (const sessionId of ownedLoginIds) {
const route = `/api/companies/${fixtureCompanyId}/ai-connections/login/${sessionId}`;
try {
const id = connectionCreationReceipt(route, "GET", fixtureCompanyId, ownedLoginIds, await api.get(route));
if (id) createdConnectionIds.add(id);
} catch { /* An unfinished or cancelled owned login has no account receipt. */ }
}
};
await page.goto(origin, { waitUntil: "domcontentloaded" });
try { await api.get("/api/companies"); }
catch (authError) {
if (!(authError instanceof ConnectionFailure) || !["api_status_401", "api_status_403"].includes(authError.message)) throw authError;
if (!config.browser.headed) throw new ConnectionBlock("awaiting_user", "board_login_required");
evidence.assisted = true;
evidence.waits.push({ kind: "board_login", startedAt: new Date().toISOString() });
await checkpoint("awaiting_board_login");
console.log(`[provider-connections] Sign in to Paperclip at ${origin} in the QA browser; this test resumes automatically.`);
const deadline = Date.now() + config.browser.loginTimeoutMs;
let ready = false;
while (Date.now() < deadline) { assertActive(); try { await api.get("/api/companies"); ready = true; break; } catch { await connectionDelay(1000); } }
if (!ready) throw new ConnectionBlock("awaiting_user", "board_login_deadline_reached");
evidence.waits.at(-1)!.finishedAt = new Date().toISOString();
}
const registry = new FixtureRegistry();
const attachedCompany = config.target.mode === "attach" ? config.target.companyId : undefined;
registry.register<Row>({ id: "connection-company", setup: async () => {
if (attachedCompany) {
const company = await api.get(`/api/companies/${attachedCompany}`);
if (!company.name.startsWith("Connection QA") || (await api.get<Row[]>(`/api/companies/${company.id}/agents`)).some(agent => agent.status !== "terminated") || (await api.get(`/api/companies/${company.id}/ai-connections`)).connections.some((connection: Row) => connection.status !== "revoked")) throw new ConnectionBlock("blocked_target", "attach_requires_empty_dedicated_qa_company");
fixtureCompanyId = company.id;
return company;
}
const company = await api.post("/api/companies", { name: `Connection QA ${nonce}`, description: "Disposable provider-connections QA fixture", budgetMonthlyCents: config.budgetCents });
fixtureCompanyId = company.id;
return company;
}, teardown: async company => {
await Promise.all(loginObservers);
await refreshOwnedConnections();
for (const [route, method] of loginSessions) {
try { await api.response(route, method); }
catch (error) { if (!(error instanceof ConnectionFailure) || error.message !== "api_status_404") throw error; }
}
await refreshOwnedConnections();
// Creation receipts identify this attempt even when concurrent campaigns
// passed the initial empty-company check at the same time.
await cleanupConnectionCompany({ api, companyId: company.id, agentName: `Connection QA ${nonce}`,
attachedCompany: Boolean(attachedCompany), retainCompany: config.retainCompany, createdConnectionIds, evidence, collectDiagnostics });
} });
registry.register<Row>({ id: "connection-environment", dependencies: ["connection-company"], setup: async values => {
const company = values.get("connection-company") as Row;
const environments = await api.get<Row[]>(`/api/companies/${company.id}/environments`);
const selectedId = config.target.mode === "attach" ? config.target.environmentId : undefined;
const environment = selectedId ? environments.find(env => env.id === selectedId) : environments.find(env => env.status === "active" && (execution.environment.id === "local" ? env.driver === "local" : env.driver === "sandbox" && env.config?.provider === "daytona"));
if (!environment || (execution.environment.id === "local" ? environment.driver !== "local" : environment.config?.provider !== "daytona")) throw new ConnectionBlock("blocked_target", "configured_execution_environment_required");
if (execution.profile.generation === "native") {
const experimental = await api.get("/api/instance/settings/experimental");
if (!experimental.enableNativeRunner) {
if (config.target.mode === "attach") throw new ConnectionBlock("blocked_target", "native_runner_disabled_on_target");
await api.patch("/api/instance/settings/experimental", { enableNativeRunner: true });
}
}
return environment;
} });
fixtures = await registry.setupAll();
const company = fixtures.values.get("connection-company") as Row;
evidence.companyId = company.id;
const environment = fixtures.values.get("connection-environment") as Row;
const observed = await runConnectionFlow({ page, api, execution, config, settings, company, environment, secret, nonce, privateDir, evidence, checkpoint, assertActive,
createdConnectionIds, refreshOwnedConnections, observeRun: run => observedRuns.set(run.id, run) });
Object.assign(result, observed);
evidence.outcome = "passed";
if (!connectionEvidencePasses(evidence)) throw new ConnectionFailure("connection_proof_incomplete");
// Only the fixture's synthetic task is captured, after authentication.
const expectedPath = `/${company.issuePrefix}/issues/${observed.issueIdentifier}`;
const url = new URL(page.url());
if (url.origin !== origin || url.pathname !== expectedPath) throw new ConnectionFailure("final_screenshot_route_mismatch");
const screenshot = await page.screenshot({ path: path.join(privateDir, "final-state.png"), fullPage: true });
result.screenshots = [{ id: "final-state", label: "Verified connection task and follow-up", file: "final-state.png", sha256: createHash("sha256").update(screenshot).digest("hex") }];
result.status = "passed";
} catch (caught) {
const error = stopped ? new ConnectionBlock("blocked_target", stopped) : caught;
if (error instanceof AggregateError) result.cleanup = "failed";
evidence.outcome = error instanceof ConnectionBlock ? error.reason : "failed";
result.failureClass = error instanceof ConnectionBlock ? "permanent_infrastructure" : classifyFailure(error);
result.error = safeConnectionFailure(error, evidence.phase);
} finally {
clearTimeout(deadlineTimer);
try { await fixtures?.teardown(); if (result.cleanup !== "failed") result.cleanup = "passed"; }
catch (cleanupError) { result.cleanup = "failed"; result.status = "failed"; result.failureClass = "cleanup_failure"; result.error = `${result.error ? `${result.error}; ` : ""}fixture_cleanup_failed`;
const causes = cleanupError instanceof AggregateError ? cleanupError.errors : [cleanupError];
console.log(`[provider-connections] Cleanup: ${causes.map(cause => safeConnectionFailure(cause, "cleanup")).join(", ")}`); }
// Also preserve evidence when setup or early cleanup failed before the
// company teardown reached its capture point. Never refetch deleted runs.
await collectDiagnostics();
try { await browser?.close(); }
catch { result.cleanup = "failed"; result.status = "failed"; result.failureClass = "cleanup_failure"; result.error = "qa_browser_cleanup_failed"; }
result.finishedAt = new Date().toISOString();
result.durationMs = Date.parse(result.finishedAt) - Date.parse(startedAt);
result.runIds = [...observedRuns.keys()];
// Usage is projected to numeric billing below; opaque provider output is never retained.
const usage = { runs: [...observedRuns.values()].map(run => ({ usage: run.usageJson })) };
result.runtimeUsage = buildRuntimeUsage({ environmentId: execution.environment.id, runs: [...observedRuns.values()], fallbackFinishedAt: result.finishedAt });
result.billing = summarizeExecutionBilling({ ...result, usage });
// UI environment probes may incur cost without heartbeat usage receipts.
result.billing.complete = false;
result.billing.llm.costStatus = result.billing.llm.runsWithTokenUsage ? "partial" : "unavailable";
result.matcherResults = gradeConnectionEvidence(evidence);
await safeWrite(path.join(privateDir, "result.json"), result, secrets);
const manifest = await packageEvidence({ privateDir, uploadDir: evidenceDir, secrets, expectPassScreenshot: false });
if (manifest.leaks.length || manifest.missing.length) { result.status = "failed"; result.failureClass = "secret_leak"; result.error = "evidence_packaging_failed"; evidence.outcome = "failed"; }
await safeWrite(path.join(evidenceDir, "result.json"), result, secrets);
await safeWrite(path.join(evidenceDir, "evidence-manifest.json"), manifest, secrets);
await safeWrite(path.join(summaryDir, "progress.json"), { executionId: execution.id, ...evidence }, secrets);
await rm(privateDir, { recursive: true, force: true });
signal.removeEventListener("abort", interrupt);
}
return result;
}
export async function runConnectionCampaign(input: { executions: readonly MatrixExecution[]; catalog: readonly MatrixExecution[]; configFile?: string; repositoryRoot: string }) {
return withConnectionCampaignCancellation(async (signal) => {
if (input.executions.some(e => e.task.flow !== "provider_connection")) throw new ConnectionFailure("Select provider-connections separately from other suites.");
const config = await loadConnectionConfig(input.configFile);
const campaignId = `connections-${new Date().toISOString().replace(/[:.]/g, "-")}-${randomBytes(3).toString("hex")}`;
const summaryDir = path.join(input.repositoryRoot, "tests/runner-e2e/results", campaignId);
await mkdir(summaryDir, { recursive: true, mode: 0o700 });
const results: RunnerE2EResult[] = [];
let target: Awaited<ReturnType<typeof startConnectionTarget>> | undefined;
try {
assertConnectionCampaignActive(signal);
target = await startConnectionTarget(config, input.executions, input.repositoryRoot, signal);
assertConnectionCampaignActive(signal);
console.log(`[provider-connections] ${input.executions.length} selected cells; target ${target.origin}; one browser/account at a time; max ${config.maxRuns} task runs and ${config.budgetCents} cents per fixture. UI probe spend may be unreported.`);
for (const execution of input.executions) {
assertConnectionCampaignActive(signal);
console.log(`[provider-connections] Starting ${execution.id}`);
const result = await connectionAttempt(execution, config, target.origin, summaryDir, input.repositoryRoot, signal);
results.push(result);
console.log(`[provider-connections] ${result.providerConnection!.outcome}: ${execution.id}${result.error ? ` (${result.error})` : ""}`);
// Every completed/blocked cell is durable before the next login begins.
await writeConnectionCampaignReport(input, campaignId, summaryDir, results);
if (signal.aborted || result.error === "test_interrupted" || result.providerConnection!.outcome === "awaiting_user") break;
}
} catch (error) {
if (!signal.aborted) throw error;
// Preserve completed receipts even if interrupted during server startup or
// report generation. Cancellation never admits another cell.
await writeConnectionCampaignReport(input, campaignId, summaryDir, results);
} finally { await target?.stop(); }
console.log(`[provider-connections] Report: ${path.join(summaryDir, "dashboard.html")}`);
if (signal.aborted || results.some(result => result.status !== "passed" || result.cleanup !== "passed")) process.exitCode = 1;
return summaryDir;
});
}
function assertConnectionCampaignActive(signal: AbortSignal) {
if (signal.aborted) throw new ConnectionBlock("blocked_target", "test_interrupted");
}
/** One cancellation state covers configuration, startup, every cell, reports,
* and owned-target teardown. Repeated signals cannot reset it. */
export async function withConnectionCampaignCancellation<T>(run: (signal: AbortSignal) => Promise<T>): Promise<T> {
const controller = new AbortController();
const interrupt = () => controller.abort();
const signals = ["SIGINT", "SIGTERM", "SIGHUP"] as const;
for (const signal of signals) process.on(signal, interrupt);
try { return await run(controller.signal); }
finally { for (const signal of signals) process.removeListener(signal, interrupt); }
}
export async function writeConnectionCampaignReport(input: { executions: readonly MatrixExecution[]; catalog: readonly MatrixExecution[] }, campaignId: string, summaryDir: string, results: RunnerE2EResult[]) {
const generatedAt = new Date().toISOString();
const expected = input.executions.map(e => e.id);
const campaign = buildRunnerCampaign({ campaignId, generatedAt, expected, results });
await safeWrite(path.join(summaryDir, "campaign.json"), campaign, []);
await safeWrite(path.join(summaryDir, "normalized-results.json"), { ...campaign, results: results.map(result => ({ ...result, evidenceValid: result.failureClass !== "secret_leak", evidenceErrors: result.failureClass === "secret_leak" ? ["evidence_packaging_failed"] : [] })) }, []);
await stageDashboardBrandAssets(summaryDir);
await safeWrite(path.join(summaryDir, "connection-coverage.json"), {
required: input.catalog.filter(e => e.suite.id === "provider-connections").length,
selected: expected.length, attempted: results.length, passed: results.filter(r => r.status === "passed" && r.cleanup === "passed").length,
blocked: results.filter(r => ["awaiting_user", "missing_credential", "blocked_target"].includes(r.providerConnection!.outcome)).length,
targets: [...new Set(results.map(r => r.providerConnection!.target.origin))],
gaps: [...connectionScopeGaps, ...connectionHarnesses.flatMap(h => connectionModes.filter(mode => !supportsConnection(h, mode)).map(mode => ({ adapter: h.adapter, mode, status: "unsupported" })))],
}, []);
const dashboard = renderRunnerE2EDashboard({ title: `Live provider connections · ${campaignId}`, generatedAt, expected, catalog: input.catalog, campaign,
entries: results.map(result => ({ result, valid: result.failureClass !== "secret_leak", errors: result.failureClass === "secret_leak" ? ["evidence_packaging_failed"] : [], evidenceBaseHref: [result.suiteId, result.profileId, result.environmentId, result.caseId, "attempt-1"].join("/") })) });
await writeFile(path.join(summaryDir, "dashboard.html"), dashboard, { mode: 0o600 });
}
tests/runner-e2e/connection-support.test.ts
+282
View File
@@ -0,0 +1,282 @@
import { describe, expect, it, vi } from "vitest";
import { mkdtemp, writeFile, rm } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { AGENT_ADAPTER_TYPES } from "../../packages/shared/src/constants.js";
import { runnerMatrix } from "./catalog.js";
import { parseRunnerSelectors, selectRunnerExecutions, buildMatrixJobs } from "./selectors.js";
import { connectionHarnesses, connectionScopeGaps, connectionCell, supportsConnection } from "./connection-cases.js";
import { parseConnectionConfig, targetOrigin, selectedSecret, resolveConnectionSecret, resolveConnectionSettings } from "./connection-config.js";
import { connectionCheckpoints, connectionEvidencePasses, connectionRunDiagnostic, createConnectionProof, verifyConnectionArtifact, verifyConnectionRun, completedConnectionArtifactRun, connectionProbeChecks, type ConnectionEvidence } from "./connection-evidence.js";
import { validateRetainedRunnerResult } from "./result-validation.js";
import { verifyConnectionTarget } from "./connection-target.js";
import { cleanupConnectionCompany, connectionCreationReceipt, runConnectionCampaign, safeConnectionFailure, withConnectionCampaignCancellation } from "./connection-launch.js";
import type { ConnectionApi } from "./connection-target.js";
import * as connectionConfig from "./connection-config.js";
import * as connectionTarget from "./connection-target.js";
import * as reportAssets from "./report-assets.js";
import { buildRunnerCampaign } from "./history.js";
import { renderCaseOutcome } from "./case-outcome.js";
import type { RunnerE2EResult } from "./types.js";
const cells = runnerMatrix.filter(e => e.suite.id === "provider-connections");
const cell = (method: string) => cells.find(e => e.id === `provider-connections.connection-codex-native.local.agent-${method}`)!;
describe("connection fixture cleanup evidence", () => {
it.each([true, false])("captures stopped runs and logs before destructive cleanup (attached=%s)", async (attachedCompany) => {
const events: string[] = [];
let status = "running", deleted = false, revoked = false;
const evidence: Partial<ConnectionEvidence> = {};
const api = {
get: vi.fn(async (route: string) => {
if (route.endsWith("/agents")) return [{ id: "qa-agent", name: "Fixture" }, { id: "foreign", name: "Other" }];
if (route.includes("heartbeat-runs?")) return [{ id: "run", agentId: "qa-agent", status }];
if (route.endsWith("/ai-connections")) return { connections: [{ id: "account", status: revoked ? "revoked" : "active" }, { id: "foreign-account", status: "active" }] };
if (deleted) throw new Error("Run was deleted");
if (route.includes("/log?")) { events.push("log"); return { content: JSON.stringify({ chunk: JSON.stringify({ type: "acpx.error", childStderrTail: "HTTP 503 model overloaded" }) + "\n" }) }; }
events.push("run"); return { id: "run", status };
}),
patch: vi.fn(async () => { events.push("pause"); }),
post: vi.fn(async (route: string) => { if (route.endsWith("/cancel")) { events.push("cancel"); status = "cancelled"; } else events.push("archive"); }),
delete: vi.fn(async (route: string) => { if (route.endsWith("/foreign-account")) throw new Error("Revoked a concurrent campaign"); if (route.includes("tool-connections")) { events.push("revoke"); revoked = true; } else { events.push("delete-agent"); deleted = true; } }),
};
await cleanupConnectionCompany({ api: api as unknown as ConnectionApi, companyId: "company", agentName: "Fixture", attachedCompany,
retainCompany: false, createdConnectionIds: new Set(["account"]), evidence, collectDiagnostics: async () => {
const run = await api.get("/api/heartbeat-runs/run");
const log = await api.get("/api/heartbeat-runs/run/log?limitBytes=2000000");
evidence.runDiagnostics = [connectionRunDiagnostic(run, log)];
} });
expect(events).toEqual(["pause", "cancel", "run", "log", "revoke", attachedCompany ? "delete-agent" : "archive"]);
expect(evidence.runDiagnostics).toEqual([{ runId: "run", status: "cancelled", signals: ["provider_overloaded"], logAvailable: true }]);
expect(api.delete).not.toHaveBeenCalledWith("/api/agents/foreign");
expect(api.delete).not.toHaveBeenCalledWith("/api/tool-connections/foreign-account");
});
it("preserves diagnostics and avoids deletion when cancellation fails", async () => {
const collectDiagnostics = vi.fn(async () => {});
const api = {
get: vi.fn(async (route: string) => route.endsWith("/agents") ? [{ id: "qa-agent", name: "Fixture" }] : [{ id: "run", agentId: "qa-agent", status: "running" }]),
patch: vi.fn(async () => {}), post: vi.fn(async () => { throw new Error("cancel failed"); }), delete: vi.fn(),
};
await expect(cleanupConnectionCompany({ api: api as unknown as ConnectionApi, companyId: "company", agentName: "Fixture", attachedCompany: true,
retainCompany: false, createdConnectionIds: new Set(), evidence: {}, collectDiagnostics })).rejects.toThrow("cancel failed");
expect(collectDiagnostics).toHaveBeenCalledOnce();
expect(api.delete).not.toHaveBeenCalled();
});
it("accepts only creation receipts from this browser and its owned login sessions", () => {
const body = { connectionId: "00000000-0000-4000-8000-000000000021" };
const root = "/api/companies/company/ai-connections";
const logins = new Set(["owned-session"]);
expect(connectionCreationReceipt(root, "POST", "company", logins, body)).toBe(body.connectionId);
expect(connectionCreationReceipt(`${root}/local`, "POST", "company", logins, body)).toBe(body.connectionId);
expect(connectionCreationReceipt(`${root}/login/owned-session`, "GET", "company", logins, body)).toBe(body.connectionId);
expect(connectionCreationReceipt(`${root}/login/foreign-session`, "GET", "company", logins, body)).toBeUndefined();
expect(connectionCreationReceipt(root, "GET", "company", logins, body)).toBeUndefined();
expect(connectionCreationReceipt(root, "POST", "different-company", logins, body)).toBeUndefined();
expect(connectionCreationReceipt(root, "POST", "company", logins, { connectionId: "not-a-receipt" })).toBeUndefined();
});
});
describe("campaign interruption", () => {
it.each(["SIGINT", "SIGTERM", "SIGHUP"] as const)("keeps %s cancellation active until teardown finishes", async (signal) => {
const before = process.listenerCount(signal);
await withConnectionCampaignCancellation(async (state) => {
expect(process.listenerCount(signal)).toBe(before + 1);
process.emit(signal);
expect(state.aborted).toBe(true);
await Promise.resolve();
process.emit(signal);
expect(state.aborted).toBe(true);
});
expect(process.listenerCount(signal)).toBe(before);
});
it.each(["SIGINT", "SIGTERM", "SIGHUP"] as const)("stops owned targets and admits no next cell after %s during startup or reporting", async (signal) => {
const repositoryRoot = path.resolve(import.meta.dirname, "../..");
const exitCode = process.exitCode;
for (const phase of ["startup", "report"] as const) {
const stop = vi.fn(async () => {});
const before = process.listenerCount(signal);
const config = parseConnectionConfig({});
const load = vi.spyOn(connectionConfig, "loadConnectionConfig").mockResolvedValue(config);
const credential = vi.spyOn(connectionConfig, "resolveConnectionSecret").mockRejectedValue(new connectionConfig.ConnectionBlock("missing_credential", "synthetic_missing_credential"));
const target = vi.spyOn(connectionTarget, "startConnectionTarget").mockImplementation(async (_config, _executions, _root, state) => {
expect(state?.aborted).toBe(false);
if (phase === "startup") process.emit(signal);
return { origin: "http://127.0.0.1:1", stop };
});
const verify = vi.spyOn(connectionTarget, "verifyConnectionTarget").mockResolvedValue({ commit: "a".repeat(40), deploymentMode: "local_trusted" });
const report = vi.spyOn(reportAssets, "stageDashboardBrandAssets").mockImplementation(async () => { if (phase === "report") process.emit(signal); });
let summary: string | undefined;
try {
summary = await runConnectionCampaign({ executions: [cell("api-key"), cell("subscription")], catalog: cells, repositoryRoot });
expect(credential).toHaveBeenCalledTimes(phase === "startup" ? 0 : 1);
expect(verify).toHaveBeenCalledTimes(phase === "startup" ? 0 : 1);
expect(stop).toHaveBeenCalledTimes(1);
expect(process.exitCode).toBe(1);
expect(process.listenerCount(signal)).toBe(before);
} finally {
load.mockRestore(); credential.mockRestore(); target.mockRestore(); verify.mockRestore(); report.mockRestore();
process.exitCode = exitCode;
if (summary) await rm(summary, { recursive: true, force: true });
}
}
});
});
describe("live connection coverage", () => {
it("inventories every built-in adapter and keeps remote exclusions explicit", () => {
const inventory = new Set<string>(["paperclip_runner", ...connectionHarnesses.map(h => h.adapter), ...connectionScopeGaps.map(h => h.adapter)]);
expect(AGENT_ADAPTER_TYPES.filter(type => !inventory.has(type))).toEqual([]);
expect(connectionHarnesses.some(h => h.adapter === "hermes_local")).toBe(true);
expect(connectionScopeGaps.find(h => h.adapter === "hermes_gateway")?.status).toBe("excluded");
});
it("includes both UI entry points and actual runner variants without invented auth capabilities", () => {
expect(cells.length).toBe(116);
expect(new Set(cells.map(e => connectionCell(e).entry))).toEqual(new Set(["agent", "apps"]));
expect(cells.filter(e => connectionCell(e).harness.id === "grok").every(e => ["subscription", "api-key"].includes(connectionCell(e).mode))).toBe(true);
expect(supportsConnection(connectionHarnesses.find(h => h.id === "gemini")!, "subscription")).toBe(false);
});
it("never runs on --all or advertises a preinstalled subscription token as a credential", () => {
expect(selectRunnerExecutions(parseRunnerSelectors(["--all"])).some(e => e.suite.id === "provider-connections")).toBe(false);
expect(cell("subscription").requiredCredentials).toEqual([]);
expect(() => buildMatrixJobs(cells)).toThrow(/not admitted/);
expect(parseRunnerSelectors(["--suite", "provider-connections", "--connection-config", "/private/qa.json"]).connectionConfig).toBe("/private/qa.json");
});
});
describe("target and selected credential boundaries", () => {
it("accepts local/staging origins and rejects credential-bearing or redirectable target strings", () => {
expect(targetOrigin("http://127.0.0.1:3109/")).toBe("http://127.0.0.1:3109");
expect(targetOrigin("https://staging.example.test")).toBe("https://staging.example.test");
for (const url of ["http://staging.example.test", "https://user:password@example.test", "https://example.test/path", "https://example.test/?token=secret", "https://example.test/#secret", "file:///tmp/qa"]) expect(() => targetOrigin(url)).toThrow();
});
it("requires an exact staging revision and rejects unknown configuration instead of ignoring typos", () => {
expect(() => parseConnectionConfig({ target: { mode: "attach", baseURL: "https://example.test" } })).toThrow();
expect(() => parseConnectionConfig({ target: { mode: "managed-local", baseURL: "https://example.test" } })).toThrow();
expect(() => parseConnectionConfig({ broswer: {} })).toThrow();
expect(() => parseConnectionConfig({ browser: { loginTimeoutMs: Infinity } })).toThrow();
expect(parseConnectionConfig({}).browser.headed).toBe(true);
});
it("pins the target revision before permitting credential selection", async () => {
const config = parseConnectionConfig({ target: { mode: "attach", baseURL: "https://staging.example.test", expectedCommit: "a".repeat(40), deploymentMode: "authenticated" } });
const fetch = vi.spyOn(globalThis, "fetch").mockResolvedValue(new Response(JSON.stringify({ commit: "b".repeat(40), deploymentMode: "authenticated" })));
try {
await expect(verifyConnectionTarget("https://staging.example.test", config)).rejects.toThrow("target_identity_mismatch");
expect(fetch.mock.calls[0]?.[1]).toMatchObject({ redirect: "error" });
fetch.mockResolvedValue(new Response(JSON.stringify({ commit: "a".repeat(40), deploymentMode: "authenticated" })));
await expect(verifyConnectionTarget("https://staging.example.test", config)).resolves.toMatchObject({ commit: "a".repeat(40) });
} finally { fetch.mockRestore(); }
});
it("reads literal selected keys without sourcing shell commands or returning unrelated secrets", async () => {
expect(selectedSecret("UNRELATED=do-not-read\nexport OPENAI_API_KEY='chosen-value'\n", "OPENAI_API_KEY")).toBe("chosen-value");
expect(selectedSecret("OPENAI_API_KEY=one\nOPENAI_API_KEY=two", "OPENAI_API_KEY")).toBe("two");
expect(() => selectedSecret("OPENAI_API_KEY=$(touch /tmp/should-not-exist)", "OPENAI_API_KEY")).toThrow();
const root = await mkdtemp(path.join(os.tmpdir(), "connection-secret-test-"));
try {
const file = path.join(root, "keys");
await writeFile(file, "OPENAI_API_KEY=selected-only\nUNRELATED=private\n", { mode: 0o600 });
const config = parseConnectionConfig({ secretFile: file });
expect(await resolveConnectionSecret(config, "OPENAI_API_KEY", {})).toBe("selected-only");
expect(await resolveConnectionSecret(config, undefined, {})).toBeUndefined();
await expect(resolveConnectionSecret(config, "XAI_API_KEY", {})).rejects.toThrow("missing_XAI_API_KEY");
await writeFile(path.join(root, "public"), "OPENAI_API_KEY=private", { mode: 0o644 });
await expect(resolveConnectionSecret(parseConnectionConfig({ secretFile: path.join(root, "public") }), "OPENAI_API_KEY", {})).rejects.toThrow(/owner_only/);
} finally { await rm(root, { recursive: true, force: true }); }
});
it("never resolves an API key for a subscription or defaults a gateway to the direct provider's credential", () => {
expect(resolveConnectionSettings(cell("subscription"), parseConnectionConfig({})).credentialEnv).toBeUndefined();
expect(resolveConnectionSettings(cell("api-key"), parseConnectionConfig({})).credentialEnv).toBe("OPENAI_API_KEY");
expect(() => resolveConnectionSettings(cell("responses"), parseConnectionConfig({ routes: { responses: { model: "qa-alias", baseURL: "https://gateway.example.test/v1" } } }))).toThrow(/credential_reference/);
const config = parseConnectionConfig({ routes: { responses: { model: "qa-alias", baseURL: "https://gateway.example.test/v1", credentialEnv: "GATEWAY_QA_KEY" } } });
expect(resolveConnectionSettings(cell("responses"), config).credentialEnv).toBe("GATEWAY_QA_KEY");
});
});
function passingEvidence(): ConnectionEvidence {
return { version: 1, outcome: "passed", phase: "finished", assisted: true, authFreshness: "signed-in", target: { mode: "attach", origin: "https://staging.example.test", commit: "a".repeat(40), deploymentMode: "authenticated" }, entry: "agent", method: "subscription", checkpoints: Object.fromEntries(connectionCheckpoints.map(key => [key, true])), waits: [] };
}
describe("independent qualification", () => {
it("retains closed failure signals across split log chunks without messages or reasoning", () => {
const sentinel = "SENTINEL_CREDENTIAL_DO_NOT_RETAIN";
const rows = [
{ type: "acpx.text_delta", channel: "thought", text: 'HTTP 429 '+sentinel },
{ type: "acpx.error", childStderrTail: 'ApiError: {"error":{"code":503,"message":"model overloaded '+sentinel+'"}}' },
{ type: "acpx.tool_call", status: "failed", text: "Error checking existing file: Resource not found: "+sentinel },
].map(row => JSON.stringify(row)).join("\n")+"\n";
const split = Math.floor(rows.length / 2);
const log = { content: [rows.slice(0, split), rows.slice(split)].map(chunk => JSON.stringify({ chunk })).join("\n") };
const diagnostic = connectionRunDiagnostic({ id: "run-1", status: "failed" }, log);
expect(diagnostic).toEqual({ runId: "run-1", status: "failed", signals: ["provider_overloaded", "missing_file_conversion"], logAvailable: true });
expect(JSON.stringify(diagnostic)).not.toContain(sentinel);
expect(connectionRunDiagnostic({ id: "run-2", status: "running" })).toEqual({ runId: "run-2", status: "unknown", signals: [], logAvailable: false });
});
it("distinguishes terminal quota and unsupported configuration from generic failures", () => {
expect(connectionRunDiagnostic({ id: "quota", status: "failed", error: 'HTTP 429 RESOURCE_EXHAUSTED' }).signals).toEqual(["provider_quota"]);
expect(connectionRunDiagnostic({ id: "config", status: "failed", error: 'session/set_config_option: Method not found (-32601)' }).signals).toEqual(["unsupported_session_configuration"]);
expect(connectionRunDiagnostic({ id: "generic", status: "failed", error: 'unknown provider error' }).signals).toEqual([]);
expect(connectionRunDiagnostic({ id: "unattributed-503", status: "failed", error: 'HTTP 503' }).signals).toEqual([]);
});
it("waits for disposition repair and requires the successful artifact-producing run", () => {
const expected = { agentId: "a", connectionId: "c", method: "api_key", runtimeMode: "legacy", environmentId: "env" };
const first = { id: "first", agentId: "a", status: "succeeded", runtimeMode: "legacy", contextSnapshot: { paperclipEnvironment: { id: "env" }, aiConnection: { connectionId: "c", method: "api_key" } } };
const repair = { ...first, id: "repair" };
const file = { originalFilename: "proof.json", createdByAgentId: "a", originatingRunId: "repair" };
expect(completedConnectionArtifactRun([first, { ...repair, status: "running" }], [file], "proof.json", expected)).toBeUndefined();
expect(completedConnectionArtifactRun([first, repair], [file], "proof.json", expected)).toBe(repair);
expect(completedConnectionArtifactRun([first, { ...repair, status: "cancelled" }], [file], "proof.json", expected)).toBeUndefined();
expect(completedConnectionArtifactRun([first, repair], [{ ...file, createdByAgentId: "someone-else" }], "proof.json", expected)).toBeUndefined();
expect(completedConnectionArtifactRun([first, repair], [file], "followup.json", expected)).toBeUndefined();
expect(completedConnectionArtifactRun([first, repair], [file], "proof.json", { ...expected, connectionId: "another" })).toBeUndefined();
});
it("retains probe codes without provider text, URLs, or credential-bearing fields", () => {
const sentinel = "SENTINEL_CREDENTIAL_DO_NOT_RETAIN";
const checks = connectionProbeChecks({ checks: [
{ code: "hermes_cli_not_found", level: "error", message: sentinel, detail: `https://provider.test/?code=${sentinel}` },
{ code: sentinel, level: "error" }, { code: "safe_code", level: sentinel }, null,
] });
expect(checks).toEqual([{ code: "hermes_cli_not_found", level: "error" }]);
expect(JSON.stringify(checks)).not.toContain(sentinel);
});
it("rejects plausible prose, wrong bytes and wrong run attribution", () => {
const proof = createConnectionProof();
expect(verifyConnectionArtifact(Buffer.from(JSON.stringify(proof.expected)), proof.expected)).toBe(true);
expect(verifyConnectionArtifact(Buffer.from("Provider routing works"), proof.expected)).toBe(false);
expect(verifyConnectionArtifact(Buffer.from(JSON.stringify({ ...proof.expected, total: proof.expected.total + 1 })), proof.expected)).toBe(false);
const expected = { agentId: "a", connectionId: "c", method: "subscription", runtimeMode: "native", environmentId: "env", model: "m" };
const run = { agentId: "a", status: "succeeded", runtimeMode: "native", contextSnapshot: { paperclipEnvironment: { id: "env" }, aiConnection: { connectionId: "c", method: "subscription" } } };
expect(verifyConnectionRun(run, expected)).toBe(true);
expect(verifyConnectionRun(run, { ...expected, connectionId: "ambient" })).toBe(false);
expect(verifyConnectionRun(run, { ...expected, method: "api_key" })).toBe(false);
expect(verifyConnectionRun(run, { ...expected, environmentId: "another-env" })).toBe(false);
expect(verifyConnectionRun({ ...run, contextSnapshot: { aiConnection: run.contextSnapshot.aiConnection } }, expected)).toBe(false);
});
it("requires every checkpoint and never passes absent evidence or a human deadline", () => {
expect(connectionEvidencePasses(passingEvidence())).toBe(true);
for (const key of connectionCheckpoints) {
const evidence = passingEvidence(); delete evidence.checkpoints[key];
expect(connectionEvidencePasses(evidence)).toBe(false);
}
expect(connectionEvidencePasses({ ...passingEvidence(), outcome: "awaiting_user" })).toBe(false);
expect(connectionEvidencePasses({ ...passingEvidence(), target: { ...passingEvidence().target, commit: null } })).toBe(false);
});
it("withholds raw Playwright call logs including filled secrets, codes, and callback URLs", () => {
const sentinel = "SENTINEL_CREDENTIAL_DO_NOT_RETAIN";
for (const message of [`locator.fill: value ${sentinel}`, `https://provider.test/callback?code=${sentinel}`, `password=${sentinel}`, "lowercasesecret"]) {
const safe = safeConnectionFailure(new Error(message), "connection_creation");
expect(safe).toBe("connection_creation_failed"); expect(safe).not.toContain(sentinel);
}
});
it("keeps blocked authentication incomplete in the existing campaign/report", () => {
const execution = cell("subscription");
const result: RunnerE2EResult = { schema: "paperclip.runner-e2e.result/v2", executionId: execution.id, suiteId: execution.suite.id, suiteDefinitionHash: execution.suiteDefinitionHash,
attempt: 1, status: "failed", failureClass: "permanent_infrastructure", error: "provider_login_deadline_reached", profileId: execution.profile.id, environmentId: "local", caseId: execution.task.id, provider: "codex", model: "model", runtimeMode: "native",
startedAt: "2026-10-03T00:00:00Z", finishedAt: "2026-10-03T00:01:00Z", durationMs: 60_000, cleanup: "passed", providerConnection: { ...passingEvidence(), outcome: "awaiting_user", checkpoints: { target: true } } };
const campaign = buildRunnerCampaign({ campaignId: "test", generatedAt: result.finishedAt, expected: [execution.id], results: [result] });
expect(() => validateRetainedRunnerResult({ ...result, status: "passed" })).toThrow(/providerConnection/);
expect(() => validateRetainedRunnerResult({ ...result, status: "passed", providerConnection: undefined })).toThrow(/providerConnection/);
expect(campaign.passed).toBe(0); expect(campaign.incomplete).toBe(1); expect(campaign.complete).toBe(false);
expect(renderCaseOutcome(result, false, [])).toContain("Blocked: awaiting user");
const cleanupFailed = buildRunnerCampaign({ campaignId: "failed-cleanup", generatedAt: result.finishedAt, expected: [execution.id], results: [{ ...result, cleanup: "failed", failureClass: "cleanup_failure" }] });
expect(cleanupFailed.failed).toBe(1); expect(cleanupFailed.incomplete).toBe(0);
});
});
tests/runner-e2e/connection-target.ts
+102
View File
@@ -0,0 +1,102 @@
import { spawn, type ChildProcess } from "node:child_process";
import { randomBytes } from "node:crypto";
import { mkdir, mkdtemp, rm } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import type { APIRequestContext } from "@playwright/test";
import { reserveRunnerE2EServerPort } from "./ports.js";
import { resolvePaperclipRunnerBinaryForHarness } from "./harness-env.js";
import { runnerE2ETypeScriptProcessArgs } from "./web-server-command.js";
import { ConnectionBlock, ConnectionFailure, targetOrigin, type ConnectionConfig } from "./connection-config.js";
import type { MatrixExecution } from "./types.js";
export const connectionDelay = (ms: number) => new Promise<void>(resolve => setTimeout(resolve, ms));
/** No raw response bodies, callback URLs, headers, or Playwright diagnostics escape. */
export class ConnectionApi {
constructor(readonly request: APIRequestContext, readonly origin: string) {}
async response(route: string, method = "GET", data?: unknown) {
if (!route.startsWith("/api/") || new URL(route, this.origin).origin !== this.origin) throw new ConnectionFailure("invalid_api_route");
try {
const response = await this.request.fetch(`${this.origin}${route}`, { method, ...(data === undefined ? {} : { data }), maxRedirects: 0, timeout: 30_000 });
if (!response.ok()) throw new ConnectionFailure(`api_status_${response.status()}`);
return response;
} catch (error) {
if (error instanceof Error && /^api_status_\d+$/.test(error.message)) throw error;
throw new ConnectionFailure("api_request_failed");
}
}
async get<T = any>(route: string): Promise<T> { return (await this.response(route)).json(); }
async post<T = any>(route: string, data?: unknown): Promise<T> { return (await this.response(route, "POST", data)).json(); }
async patch<T = any>(route: string, data: unknown): Promise<T> { return (await this.response(route, "PATCH", data)).json(); }
async delete(route: string) { await this.response(route, "DELETE"); }
async bytes(route: string) { return (await this.response(route)).body(); }
}
async function stopOwnedServer(child: ChildProcess) {
if (!child.pid) return;
const exited = () => child.exitCode !== null || child.signalCode !== null;
const kill = (signal: NodeJS.Signals) => {
try { process.kill(process.platform === "win32" ? child.pid! : -child.pid!, signal); }
catch (error) { if ((error as NodeJS.ErrnoException).code !== "ESRCH") throw error; }
};
kill("SIGTERM");
const deadline = Date.now() + 45_000;
while (!exited() && Date.now() < deadline) await connectionDelay(100);
// The group is exclusively the server this function spawned, never an attached target.
kill("SIGKILL");
}
export async function startConnectionTarget(config: ConnectionConfig, executions: readonly MatrixExecution[], repositoryRoot: string, signal?: AbortSignal) {
const assertActive = () => { if (signal?.aborted) throw new ConnectionBlock("blocked_target", "test_interrupted"); };
assertActive();
if (config.target.mode === "attach") return { origin: targetOrigin(config.target.baseURL), stop: async () => {} };
const root = await mkdtemp(path.join(os.tmpdir(), "paperclip-connection-e2e-"));
let child: ChildProcess | undefined;
const stop = async () => { if (child) await stopOwnedServer(child); await rm(root, { recursive: true, force: true }); };
try {
assertActive();
const home = path.join(root, "paperclip-home");
const instance = `connection-${randomBytes(6).toString("hex")}`;
const port = await reserveRunnerE2EServerPort();
await mkdir(home, { mode: 0o700 });
// Only bootstrap essentials are inherited. Selected provider keys enter via UI.
const env: NodeJS.ProcessEnv = Object.fromEntries(["PATH", "HOME", "USER", "LOGNAME", "SHELL", "TMPDIR", "LANG", "LC_ALL", "NODE_EXTRA_CA_CERTS"].flatMap(key => process.env[key] ? [[key, process.env[key]!]] : []));
Object.assign(env, {
PAPERCLIP_HOME: home, PAPERCLIP_INSTANCE_ID: instance, PAPERCLIP_CONFIG: path.join(home, "instances", instance, "config.json"),
PAPERCLIP_RUNNER_E2E_TEMP_ROOT: root, PAPERCLIP_RUNNER_E2E_PORT: String(port), PAPERCLIP_RUNNER_E2E_SERVER_LOG: path.join(root, "server.log"),
PAPERCLIP_RUNNER_BINARY: resolvePaperclipRunnerBinaryForHarness(executions, repositoryRoot),
PAPERCLIP_VITE_CACHE_DIR: path.join(root, "vite-cache"), PAPERCLIP_ANNOUNCEMENTS_ENABLED: "false",
PAPERCLIP_AGENT_JWT_SECRET: randomBytes(48).toString("hex"), PAPERCLIP_DECISION_SIGNING_SECRET: randomBytes(48).toString("hex"),
PAPERCLIP_TOOL_ACTION_SIGNING_SECRET: randomBytes(48).toString("hex"), BETTER_AUTH_SECRET: randomBytes(48).toString("hex"),
});
assertActive();
child = spawn(process.execPath, runnerE2ETypeScriptProcessArgs(repositoryRoot, path.join(repositoryRoot, "tests/runner-e2e/server.ts")), { cwd: repositoryRoot, env, detached: process.platform !== "win32", stdio: "ignore" });
let failed = false;
child.once("error", () => { failed = true; });
const origin = `http://127.0.0.1:${port}`;
const deadline = Date.now() + 180_000;
while (Date.now() < deadline) {
assertActive();
if (failed || child.exitCode !== null) throw new ConnectionBlock("blocked_target", "managed_server_start_failed");
try {
const response = await fetch(`${origin}/api/health`, { redirect: "error", signal: signal ? AbortSignal.any([signal, AbortSignal.timeout(1500)]) : AbortSignal.timeout(1500) });
if (response.ok) { assertActive(); return { origin, stop }; }
} catch { assertActive(); /* Bounded startup. */ }
await connectionDelay(500);
}
throw new ConnectionBlock("blocked_target", "managed_server_start_timeout");
} catch (error) { await stop(); throw error; }
}
export async function verifyConnectionTarget(origin: string, config: ConnectionConfig) {
let health: any;
try {
const response = await fetch(`${targetOrigin(origin)}/api/health`, { redirect: "error", signal: AbortSignal.timeout(15_000) });
if (!response.ok) throw new ConnectionFailure();
health = await response.json();
} catch { throw new ConnectionBlock("blocked_target", "target_health_unavailable_or_redirected"); }
if (typeof health.commit !== "string" || !/^[a-f0-9]{40}$/.test(health.commit)) throw new ConnectionBlock("blocked_target", "target_revision_unavailable");
if (config.target.mode === "attach" && (health.commit !== config.target.expectedCommit || health.deploymentMode !== config.target.deploymentMode)) throw new ConnectionBlock("blocked_target", "target_identity_mismatch");
return { commit: health.commit as string, deploymentMode: String(health.deploymentMode) };
}
tests/runner-e2e/gemini-acp-filesystem.smoke.ts
+82
View File
@@ -0,0 +1,82 @@
/** Opt-in installed-CLI compatibility check. No real provider credentials or inference. */
import { createServer } from "node:http";
import { mkdir, mkdtemp, readFile, realpath, rm, writeFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { createAcpRuntime, createAgentRegistry, createRuntimeStore } from "../../packages/adapter-utils/node_modules/acpx/dist/runtime.js";
const argument = process.argv.indexOf("--agent-command");
const command = argument < 0 ? "gemini --acp" : process.argv[argument + 1];
if (!command) throw new Error("agent_command_required");
const root = await mkdtemp(path.join(os.tmpdir(), "paperclip-gemini-filesystem-smoke-"));
const content = '{"nonce":"synthetic-file-smoke","total":28}\n';
const results: Array<{ case: string; passed: boolean; completedTools: number; failedTools: number; missingFileSignal: boolean }> = [];
let active: { calls: number; file: string; tool: string };
const server = createServer(async (request, response) => {
// Never retain request bodies, headers, provider prompts or model output.
for await (const _chunk of request) { /* Drain the synthetic request. */ }
const parts = ++active.calls === 1
? [{ functionCall: { name: active.tool, args: { file_path: active.file, ...(active.tool === "write_file" ? { content } : {}) } } }]
: [{ text: "synthetic-complete" }];
const payload = {
candidates: [{ content: { role: "model", parts }, finishReason: "STOP" }],
usageMetadata: { promptTokenCount: 10, candidatesTokenCount: 2, totalTokenCount: 12 },
modelVersion: "gemini-3.1-flash-lite",
};
if (request.url?.includes(":streamGenerateContent")) {
response.writeHead(200, { "content-type": "text/event-stream" });
response.end(`data: ${JSON.stringify(payload)}\n\n`);
} else {
response.writeHead(200, { "content-type": "application/json" });
response.end(JSON.stringify(request.url?.includes(":countTokens") ? { totalTokens: 10 } : payload));
}
});
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
const origin = `http://127.0.0.1:${(server.address() as { port: number }).port}`;
try {
for (const scenario of ["create-missing", "read-existing", "deny-outside"] as const) {
const home = path.join(root, scenario, "home");
const workspace = path.join(root, scenario, "workspace");
await mkdir(path.join(home, ".gemini"), { recursive: true, mode: 0o700 });
await mkdir(workspace, { recursive: true });
const cwd = await realpath(workspace);
const file = scenario === "deny-outside" ? path.join(await realpath(root), "outside.json") : path.join(cwd, "proof.json");
if (scenario !== "create-missing") await writeFile(file, content);
await writeFile(path.join(home, ".gemini", "settings.json"), JSON.stringify({
security: { auth: { selectedType: "gemini-api-key" } }, telemetry: { enabled: false },
}), { mode: 0o600 });
active = { calls: 0, file, tool: scenario === "read-existing" ? "read_file" : "write_file" };
const runtime = createAcpRuntime({
cwd, sessionStore: createRuntimeStore({ stateDir: path.join(root, scenario, "state") }),
agentRegistry: createAgentRegistry({ overrides: { gemini: command } }),
permissionMode: "approve-all", nonInteractivePermissions: "deny", inheritProcessEnv: false,
onAgentStderr: () => { /* Child diagnostics may contain untrusted material. */ },
});
const handle = await runtime.ensureSession({
sessionKey: `gemini-filesystem-${scenario}`, agent: "gemini", mode: "oneshot", cwd,
sessionOptions: { env: {
HOME: home, PATH: process.env.PATH ?? "", GEMINI_MODEL: "gemini-3.1-flash-lite",
GEMINI_API_KEY: "synthetic-invalid-key", GOOGLE_GEMINI_BASE_URL: origin,
} },
});
try {
let completedTools = 0; let failedTools = 0; let missingFileSignal = false;
for await (const event of runtime.runTurn({ handle, text: "Run the synthetic file operation, then stop.", mode: "prompt", requestId: scenario, timeoutMs: 20_000 })) {
if (event.type !== "tool_call") continue;
if (event.status === "completed") completedTools++;
if (event.status === "failed") {
failedTools++;
missingFileSignal ||= /Error checking existing file: (?:Resource not found|Internal error)/.test(event.text ?? "");
}
}
const bytes = await readFile(file, "utf8").catch(() => null);
const passed = bytes === content && (scenario === "deny-outside" ? failedTools > 0 && completedTools === 0 : failedTools === 0 && completedTools > 0);
results.push({ case: scenario, passed, completedTools, failedTools, missingFileSignal });
} finally { await runtime.close({ handle, reason: "synthetic filesystem smoke complete" }); }
}
console.log(JSON.stringify({ realProviderCalls: 0, results }, null, 2));
if (results.some((result) => !result.passed)) process.exitCode = 1;
} finally {
await new Promise<void>((resolve) => server.close(() => resolve()));
await rm(root, { recursive: true, force: true });
}
tests/runner-e2e/history.test.ts
+2
View File
@@ -31,6 +31,7 @@ import {
import { renderRunnerHistoryIndex } from "./history-index.js";
import { renderPublicCampaignSummary } from "./public-summary-image.js";
import { isPublicRunnerScreenshotRoute } from "./screenshot-policy.js";
import { connectionCheckpoints, type ConnectionEvidence } from "./connection-evidence.js";
import type { MatrixExecution, RunnerE2EResult } from "./types.js";
const temporaryDirectories: string[] = [];
@@ -63,6 +64,7 @@ function result(execution: MatrixExecution, status: "passed" | "failed") {
finishedAt: "2026-08-28T00:00:01.000Z",
durationMs: 1_000,
cleanup: "passed",
...(execution.task.flow === "provider_connection" ? { providerConnection: { version: 1, outcome: status, phase: "finished", assisted: false, authFreshness: "signed-in", target: { mode: "attach", origin: "http://127.0.0.1:3100", commit: "a".repeat(40), deploymentMode: "local_trusted" }, entry: "agent", method: "api-key", checkpoints: Object.fromEntries(connectionCheckpoints.map(key => [key, true])), waits: [] } satisfies ConnectionEvidence } : {}),
} satisfies RunnerE2EResult;
}
tests/runner-e2e/launch.ts
+7
View File
@@ -1084,6 +1084,13 @@ async function main() {
return;
}
if (executions.some(execution => execution.task.flow === "provider_connection")) {
const { runConnectionCampaign } = await import("./connection-launch.js");
await runConnectionCampaign({ executions, catalog: runnerMatrix, configFile: options.connectionConfig, repositoryRoot });
return;
}
if (options.connectionConfig) throw new Error("--connection-config is only supported by the provider-connections suite");
// Keep admission before local-env loading and credential checks. Pending
// profiles remain discoverable, but cannot reach a provider.
assertRunnerE2EPrerequisites(executions);
tests/runner-e2e/report-assets.ts
+18
View File
@@ -0,0 +1,18 @@
import path from "node:path";
import { copyFile, mkdir } from "node:fs/promises";
const repositoryRoot = path.resolve(import.meta.dirname, "../..");
export async function stageDashboardBrandAssets(output: string) {
const assets = path.join(output, "assets");
await mkdir(assets, { recursive: true });
await Promise.all([
copyFile(
path.join(repositoryRoot, "ui/public/favicon-32x32.png"),
path.join(assets, "favicon-32x32.png"),
),
copyFile(
path.join(repositoryRoot, "ui/public/fonts/InterVariable.woff2"),
path.join(assets, "InterVariable.woff2"),
),
]);
}
tests/runner-e2e/report.ts
+1 -14
View File
@@ -1,3 +1,4 @@
import { stageDashboardBrandAssets } from "./report-assets.js";
import { discoverReportCatalog } from "./report-catalog.js";
import path from "node:path";
import {
@@ -147,20 +148,6 @@ async function stageDashboardEvidence(
return staged;
}
async function stageDashboardBrandAssets(output: string) {
const assets = path.join(output, "assets");
await mkdir(assets, { recursive: true });
await Promise.all([
copyFile(
path.join(repositoryRoot, "ui/public/favicon-32x32.png"),
path.join(assets, "favicon-32x32.png"),
),
copyFile(
path.join(repositoryRoot, "ui/public/fonts/InterVariable.woff2"),
path.join(assets, "InterVariable.woff2"),
),
]);
}
async function main() {
const root = path.resolve(
tests/runner-e2e/result-validation.ts
+17
View File
@@ -1,3 +1,4 @@
import { connectionCheckpoints, connectionEvidencePasses, connectionRunSignals } from "./connection-evidence.js";
import type { RunnerE2EResult } from "./types.js";
type Rule = (value: unknown, at: string) => void;
@@ -243,6 +244,20 @@ const fields = {
tasks: array(object), agents: array(object), comments: array(object), interactions: array(object), documents: array(object), attachments: optional(array(object)), runs: array(object) })),
checks: array(shape({ id: string, passed: boolean, notReached: optional(string), evidence: array(string), detail: string })),
})),
providerConnection: optional(shape({
version: integer,
outcome: oneOf("running", "passed", "failed", "awaiting_user", "missing_credential", "blocked_target"),
phase: string, assisted: boolean, authFreshness: oneOf("signed-in", "signed-out"),
target: shape({ mode: oneOf("managed-local", "attach"), origin: url, commit: nullable(string), deploymentMode: nullable(string) }),
entry: oneOf("apps", "agent"), method: string, checkpoints: shape(Object.fromEntries(connectionCheckpoints.map(key => [key, optional(boolean)]))),
waits: array(shape({ kind: oneOf("board_login", "provider_login"), startedAt: date, finishedAt: optional(date) })),
artifactChecks: optional(array(shape({ filename: string, runId: string, sha256: string, fields: object, exactFields: boolean }))),
inputTransport: optional(oneOf("prompt_base64")),
setupChecks: optional(array(shape({ code: string, level: oneOf("info", "warn", "error") }))),
runDiagnostics: optional(array(shape({ runId: string, status: oneOf("succeeded", "failed", "cancelled", "interrupted", "timed_out", "unknown"), signals: array(oneOf(...connectionRunSignals)), logAvailable: boolean }))),
creationFailure: optional(shape({ status: integer, code: oneOf("ai_connection_api_key_rejected", "ai_connection_verification_failed", "connection_request_rejected") })),
connectionId: optional(string), agentId: optional(string), companyId: optional(string), artifactSha256: optional(string), model: optional(string), cleanupRetained: optional(boolean), companyArchived: optional(boolean),
})),
completionQuality: optional(array(shape({
name: string, purpose: optional(oneOf("product", "calibration")), expectedPass: boolean, passed: boolean, status: oneOf("completed", "failed", "pending"), config: object, configHash: string, evidenceHash: string,
criteria: array(shape({ id: string, passed: boolean, rationale: string, evidenceIds: array(string) })),
@@ -263,4 +278,6 @@ export function validateRetainedRunnerResult(
value: unknown,
): asserts value is RunnerE2EResult {
result(value, "result");
const checked = value as RunnerE2EResult;
if (checked.suiteId === "provider-connections" && checked.status === "passed" && (!checked.providerConnection || !connectionEvidencePasses(checked.providerConnection))) invalid("result.providerConnection");
}
tests/runner-e2e/selectors.ts
+6
View File
@@ -16,6 +16,7 @@ export interface RunnerSelectorOptions {
debug: boolean;
maxParallel: number;
maxAutomaticRetries: number;
connectionConfig?: string;
}
export class RunnerSelectorError extends Error {}
@@ -55,6 +56,10 @@ export function parseRunnerSelectors(
else if (flag === "--headed") options.headed = true;
else if (flag === "--ui") options.ui = true;
else if (flag === "--debug") options.debug = true;
else if (flag === "--connection-config") {
options.connectionConfig = valueFor(args, index, flag);
index += 1;
}
else if (flag === "--max-parallel") {
const value = valueFor(args, index, flag);
index += 1;
@@ -200,6 +205,7 @@ export function selectRunnerExecutions(
export function buildMatrixJobs(
executions: readonly MatrixExecution[],
): MatrixJob[] {
if (executions.some(e => e.task.flow === "provider_connection")) throw new RunnerSelectorError("Provider connection journeys use an explicit local/staging connection configuration; they are not admitted to paid CI matrix jobs yet.");
return executions
.map((execution) => ({
executionId: execution.id,
tests/runner-e2e/types.ts
+3
View File
@@ -5,6 +5,7 @@ export const CREDENTIAL_NAMES = [
"KIMI_MODEL_API_KEY",
"XAI_API_KEY",
"GROK_AUTH_JSON",
"GEMINI_API_KEY",
"DAYTONA_API_KEY",
"CURSOR_AUTH_TOKEN",
"COPILOT_GITHUB_TOKEN",
@@ -15,6 +16,7 @@ export type RunnerGeneration = "legacy" | "native";
export type RunnerEnvironmentId = "local" | "daytona";
export type RunnerTaskWorkMode = "standard" | "planning" | "ask";
export type RunnerTaskFlow =
| "provider_connection"
| "plan_task_guidance"
| "blocker_guidance"
| "public_mcp"
@@ -326,6 +328,7 @@ export interface RunnerE2EResult {
sha256?: string;
}>;
firstTask?: import("./first-task-scoring.js").FirstTaskEvidence;
providerConnection?: import("./connection-evidence.js").ConnectionEvidence;
completionQuality?: import("./completion-quality.js").CompletionQualityRecord[];
firstTaskQuality?: import("./first-task-quality.js").FirstTaskQuality;
/** External assistant API calls, separate from the team's heartbeat executions. */
tests/runner-e2e/user-actions.ts
+51 -16
View File
@@ -8,9 +8,10 @@ export async function createTaskThroughUi(input: {
prompt: string;
workMode: "standard" | "planning" | "ask";
projectName?: string;
attachments?: readonly string[];
}) {
const issuesUrl = `/${encodeURIComponent(input.issuePrefix)}/issues`;
const newTask = input.page.getByRole("button", { name: "New Task" }).first();
const newTask = input.page.getByRole("button", { name: /^New task$/i }).first();
let bootstrapError: unknown;
for (let bootstrapAttempt = 1; bootstrapAttempt <= 3; bootstrapAttempt += 1) {
try {
@@ -33,40 +34,74 @@ export async function createTaskThroughUi(input: {
);
}
await newTask.click();
await input.page.getByPlaceholder("Task title").fill(input.title);
const dialog = input.page.getByRole("dialog");
const titleInput = dialog.getByPlaceholder("Task title");
const hasTitleInput = await titleInput.count() > 0;
if (hasTitleInput) await titleInput.fill(input.title);
await input.page
.getByRole("dialog")
.getByRole("textbox", { name: "editable markdown", exact: true })
.fill(input.prompt);
if (input.workMode !== "standard") {
await input.page
.getByRole("dialog")
.locator(`[data-issue-work-mode-chip="standard"]`)
.click();
await input.page
.locator(`[data-issue-work-mode="${input.workMode}"]`)
.click();
const legacyMode = dialog.locator('[data-issue-work-mode-chip="standard"]');
if (await legacyMode.count()) {
await legacyMode.click();
await input.page.locator(`[data-issue-work-mode="${input.workMode}"]`).click();
} else {
await dialog.getByRole("button", { name: "Add to composer", exact: true }).click();
await input.page.getByRole("menuitem", { name: input.workMode === "planning" ? /^Plan mode/ : /^Ask mode/ }).click();
}
}
await input.page
.getByRole("button", { name: "Assignee", exact: true })
.getByRole("button", { name: /^(?:Select )?Assignee$/i })
.click();
await input.page
.getByPlaceholder("Search assignees...")
.getByPlaceholder(/^Search assignees(?:…|\.\.\.)?$/i)
.fill(input.agentName);
await input.page.getByText(input.agentName, { exact: true }).last().click();
if (input.projectName) {
const dialog = input.page.getByRole("dialog");
// Selecting the assignee advances focus to this selector and opens it.
// Focus is idempotent here; clicking would toggle an already-open popover
// closed before the search field can be filled.
await dialog.getByRole("button", { name: "Project", exact: true }).focus();
await dialog.getByPlaceholder("Search projects...").fill(input.projectName);
await dialog.getByText(input.projectName, { exact: true }).last().click();
const projectTrigger = dialog.getByRole("button", { name: "Project", exact: true })
.or(dialog.getByRole("button", { name: input.projectName, exact: true }));
await projectTrigger.focus();
const projectSearch = input.page.getByPlaceholder(/^Search projects(?:…|\.\.\.)?$/i);
await expect(projectSearch).toBeVisible({ timeout: 2_000 }).catch(async () => {
if (await projectTrigger.getAttribute("aria-expanded") !== "true") await projectTrigger.click();
await expect(projectSearch).toBeVisible();
});
await projectSearch.fill(input.projectName);
await input.page.getByText(input.projectName, { exact: true }).last().click();
}
const submittedAtMs = Date.now();
if (input.attachments?.length) {
const chooser = input.page.waitForEvent("filechooser");
const upload = dialog.getByRole("button", { name: "Upload", exact: true });
if (await upload.count()) await upload.click();
else {
await dialog.getByRole("button", { name: "Add to composer", exact: true }).click();
await input.page.getByRole("menuitem", { name: /^Files and images/ }).click();
}
await (await chooser).setFiles([...input.attachments]);
// Upload finishes as part of task creation; the dialog retains the selected files.
}
const created = input.page.waitForResponse(response => response.request().method() === "POST" && /^\/api\/companies\/[^/]+\/issues$/.test(new URL(response.url()).pathname));
await input.page
.getByRole("button", { name: "Create Task", exact: true })
.getByRole("button", { name: /^Create task$/i })
.click();
const response = await created;
if (!response.ok()) throw new Error("Task creation request failed");
if (!hasTitleInput && input.title) {
// Prompt-only creation generates a title. Set only fixture naming metadata
// through the public API; execution and results remain entirely real.
const issue = await response.json();
const renamed = await input.page.request.patch(new URL(`/api/issues/${issue.id}`, response.url()).href, { data: { title: input.title } });
if (!renamed.ok()) throw new Error("Fixture task naming failed");
}
// The create response precedes staged-file uploads. The production dialog
// closes after those uploads finish; navigating earlier can cancel them.
await expect(dialog).not.toBeVisible({ timeout: 30_000 });
return submittedAtMs;
}
ui/src/adapters/codex-local/config-fields.test.tsx
+49 -27
View File
@@ -1,12 +1,32 @@
import { renderToStaticMarkup } from "react-dom/server";
import { describe, expect, it } from "vitest";
// @vitest-environment jsdom
import { createRoot } from "react-dom/client";
import { act } from "react";
import type { ReactNode } from "react";
import { beforeAll, describe, expect, it } from "vitest";
import { TooltipProvider } from "@/components/ui/tooltip";
import { CodexLocalConfigFields } from "./config-fields";
function renderRunner(config: Record<string, unknown>): string {
return renderToStaticMarkup(
beforeAll(() => { Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); });
async function renderMarkup(node: ReactNode, expand?: string): Promise<string> {
const container = document.createElement("div");
document.body.appendChild(container);
const root = createRoot(container);
await act(async () => root.render(node));
if (expand) await act(async () => {
container.querySelector(`[aria-label="${expand}"]`)?.dispatchEvent(new KeyboardEvent("keydown", { key: "Enter", bubbles: true }));
});
const html = document.body.innerHTML;
await act(async () => root.unmount());
container.remove();
return html;
}
async function renderRunner(config: Record<string, unknown>, expand?: string): Promise<string> {
return renderMarkup(
<TooltipProvider>
<CodexLocalConfigFields
mode="edit"
@@ -21,6 +41,7 @@ function renderRunner(config: Record<string, unknown>): string {
hideInstructionsFile
/>
</TooltipProvider>,
expand,
);
}
@@ -30,19 +51,19 @@ describe("Paperclip Runner Codex configuration", () => {
["approve-paperclip", "Automatic Paperclip actions"],
["approve-reads", "Allow Paperclip reads"],
["deny-all", "Deny all"],
])("displays Grok's default or saved permission mode %s", (acpxPermissionMode, label) => {
const html = renderRunner({ provider: "acpx", acpxAgent: "grok", acpxPermissionMode });
expect(html).toContain('<option value="grok" selected="">Grok Build</option>');
])("displays Grok's default or saved permission mode %s", async (acpxPermissionMode, label) => {
const html = await renderRunner({ provider: "acpx", acpxAgent: "grok", acpxPermissionMode });
expect(html).toContain('Grok Build');
expect(html).toContain('aria-label="Permission mode"');
expect(html).toContain(label);
});
it("exposes all qualified provider choices", () => {
const html = renderRunner({ provider: "codex" });
it("exposes all qualified provider choices", async () => {
const html = await renderRunner({ provider: "codex" }, "Harness");
expect(html).toContain('<option value="codex" selected="">Codex</option>');
expect(html).toContain('aria-label="Harness"');
expect(html).toContain("OpenCode 1.18.34");
expect(html).toContain('<option value="acpx">ACP agents</option>');
expect(html).toContain('ACP agents');
expect(html).not.toContain("Permission mode");
expect(html).not.toContain("Ask when requested");
expect(html).not.toContain("Ask for untrusted operations");
@@ -51,14 +72,14 @@ describe("Paperclip Runner Codex configuration", () => {
expect(html).not.toContain("Bypass sandbox");
});
it("renders OpenCode's bounded permission modes", () => {
const html = renderRunner({
it("renders OpenCode's bounded permission modes", async () => {
const html = await renderRunner({
provider: "opencode",
opencodePermissionMode: "allow",
});
expect(html).toContain(
'<option value="opencode" selected="">OpenCode 1.18.34</option>',
'OpenCode 1.18.34',
);
expect(html).toContain("Full auto (allow)");
expect(html).toContain('aria-label="Permission mode"');
@@ -66,27 +87,28 @@ describe("Paperclip Runner Codex configuration", () => {
expect(html).not.toContain("Ask for untrusted operations");
});
it("offers qualified Claude and keeps candidate ACP agents visibly disabled", () => {
const html = renderRunner({
it("offers qualified Claude and keeps candidate ACP agents visibly disabled", async () => {
const html = await renderRunner({
provider: "acpx",
acpxAgent: "claude",
acpxPermissionMode: "approve-reads",
});
}, "ACP agent");
expect(html).toContain('<option value="acpx" selected="">ACP agents</option>');
expect(html).toContain('ACP agents');
expect(html).toContain("ACP agent");
expect(html).toContain('<option value="claude" selected="">Claude</option>');
expect(html).toContain('<option value="cursor" disabled="">Cursor — qualification pending</option>');
expect(html).toContain('<option value="copilot" disabled="">GitHub Copilot — qualification pending</option>');
expect(html).toContain('<option value="pi" disabled="">Pi — qualification pending</option>');
expect(html).toContain('aria-label="ACP agent"');
expect(html.match(/role="option"[^>]*data-disabled=""/g)).toHaveLength(3);
expect(html).toContain('Cursor — qualification pending');
expect(html).toContain('GitHub Copilot — qualification pending');
expect(html).toContain('Pi — qualification pending');
expect(html).not.toContain("Codex via ACPX");
expect(html).not.toContain("ACPX Codex");
expect(html).not.toContain("Pi via ACPX");
expect(html).toContain("Allow Paperclip reads");
});
it("falls back to the fail-closed Codex permission mode", () => {
const html = renderRunner({ codexPermissionMode: "unrestricted" });
it("falls back to the fail-closed Codex permission mode", async () => {
const html = await renderRunner({ codexPermissionMode: "unrestricted" });
expect(html).toContain("Unsupported saved mode — select a qualified mode");
expect(html).toContain("cannot start or recover a Paperclip Runner run");
@@ -94,12 +116,12 @@ describe("Paperclip Runner Codex configuration", () => {
expect(html).not.toContain("Full auto (never ask)");
});
it("shows a bounded idle timeout only for warm sessions", () => {
const warmHtml = renderRunner({
it("shows a bounded idle timeout only for warm sessions", async () => {
const warmHtml = await renderRunner({
lifecycleMode: "warm",
idleTimeoutMs: 45_000,
});
const turnHtml = renderRunner({
const turnHtml = await renderRunner({
lifecycleMode: "per_turn",
idleTimeoutMs: 45_000,
});
ui/src/adapters/codex-local/config-fields.tsx
+39 -22
View File
@@ -1,4 +1,5 @@
import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "../../components/ui/select";
import { AdapterMark } from "../../components/AdapterMark";
import { configFieldsForSection } from "../config-sections";
import type { AdapterConfigFieldsProps } from "../types";
import {
@@ -36,6 +37,14 @@ const defaultOpenCodeRunnerModel = "openrouter/deepseek/deepseek-v4-flash-0731";
const defaultAcpxClaudeModel = "claude-sonnet-5";
const defaultClaudeManagedModel = "claude-sonnet-5";
const defaultAwsAgentCoreModel = "global.anthropic.claude-sonnet-4-6";
const runnerHarnessOptions = [
{ value: "codex", label: "Codex", adapter: "codex_local" },
{ value: "opencode", label: "OpenCode 1.18.34", adapter: "opencode_local" },
{ value: "claude_managed", label: "Claude Managed", adapter: "claude_local" },
{ value: "aws_agentcore", label: "AWS AgentCore", adapter: "aws_agentcore" },
{ value: "acpx", label: "ACP agents", adapter: "acpx_local" },
{ value: "grok", label: "Grok Build", adapter: "grok_local" },
];
export function CodexLocalConfigFields({
section,
@@ -191,16 +200,15 @@ export function CodexLocalConfigFields({
)}
{runnerManaged && (
<Field configSection="adapter"
label="Provider"
hint="The runner persists this provider with each run so recovery cannot drift after configuration changes."
label="Harness"
hint="Choose the agent harness that runs your tasks."
>
<select
className={inputClass}
<Select
value={runnerProvider === "acpx" && runnerSchemaValue("acpxAgent", "claude") === "grok" ? "grok" : runnerProvider}
onChange={(event) => {
const grok = event.target.value === "grok";
const provider = grok ? "acpx" : isPaperclipRunnerProvider(event.target.value)
? event.target.value
onValueChange={(value) => {
const grok = value === "grok";
const provider = grok ? "acpx" : isPaperclipRunnerProvider(value)
? value
: "codex";
const model =
provider === "opencode"
@@ -230,30 +238,39 @@ export function CodexLocalConfigFields({
}
}}
>
<option value="codex">Codex</option>
<option value="opencode">OpenCode 1.18.34</option>
<option value="claude_managed">Claude Managed</option>
<option value="aws_agentcore">AWS AgentCore</option>
<option value="acpx">ACP agents</option>
<option value="grok">Grok Build</option>
</select>
<SelectTrigger className="w-full" aria-label="Harness"><SelectValue /></SelectTrigger>
<SelectContent>
{runnerHarnessOptions.map((option) => (
<SelectItem key={option.value} value={option.value}>
<AdapterMark type={option.adapter} className="size-4" />
{option.label}
</SelectItem>
))}
</SelectContent>
</Select>
</Field>
)}
{runnerManaged && runnerProvider === "acpx" && runnerSchemaValue("acpxAgent", "claude") !== "grok" && (
<Field configSection="adapter" label="ACP agent" hint="Cursor, GitHub Copilot, and Pi are awaiting local and Daytona qualification.">
<select className={inputClass}
<Select
value={String(isCreate ? values!.adapterSchemaValues?.acpxAgent ?? "claude" : eff("adapterConfig", "acpxAgent", config.acpxAgent ?? "claude"))}
onChange={(event) => {
const profile = PAPERCLIP_RUNNER_ACPX_PROFILES.find(entry => entry.value === event.target.value);
onValueChange={(value) => {
const profile = PAPERCLIP_RUNNER_ACPX_PROFILES.find(entry => entry.value === value);
if (!profile?.qualified) return;
if (isCreate) set!({ model: profile.value === "claude" ? defaultAcpxClaudeModel : "",
adapterSchemaValues: { ...values!.adapterSchemaValues, acpxAgent: profile.value } });
else { mark("adapterConfig", "acpxAgent", profile.value); mark("adapterConfig", "model", profile.value === "claude" ? defaultAcpxClaudeModel : ""); }
}}>
{PAPERCLIP_RUNNER_ACPX_PROFILES.map(profile => <option key={profile.value} value={profile.value} disabled={!profile.qualified}>
{profile.label}{profile.qualified ? "" : " — qualification pending"}
</option>)}
</select>
<SelectTrigger className="w-full" aria-label="ACP agent"><SelectValue /></SelectTrigger>
<SelectContent>
{PAPERCLIP_RUNNER_ACPX_PROFILES.map(profile => (
<SelectItem key={profile.value} value={profile.value} disabled={!profile.qualified}>
<AdapterMark type={profile.value === "cursor" ? "cursor" : `${profile.value}_local`} className="size-4" />
{profile.label}{profile.qualified ? "" : " — qualification pending"}
</SelectItem>
))}
</SelectContent>
</Select>
</Field>
)}
{runnerManaged && runnerProvider === "claude_managed" && (
ui/src/adapters/config-sections.test.tsx
+43 -24
View File
@@ -1,6 +1,10 @@
import { renderToStaticMarkup } from "react-dom/server";
// @vitest-environment jsdom
import { createRoot } from "react-dom/client";
import { act } from "react";
import type { ReactNode } from "react";
import type { ComponentType } from "react";
import { describe, expect, it } from "vitest";
import { beforeAll, describe, expect, it } from "vitest";
import { TooltipProvider } from "../components/ui/tooltip";
import type { AdapterConfigFieldsProps, AdapterConfigSection } from "./types";
import { CodexLocalConfigFields } from "./codex-local/config-fields";
@@ -10,13 +14,29 @@ import { ProcessConfigFields } from "./process/config-fields";
import { OpenClawGatewayConfigFields } from "./openclaw-gateway/config-fields";
import { HermesGatewayConfigFields } from "./hermes-gateway/config-fields";
function renderSection(
beforeAll(() => { Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); });
async function renderMarkup(node: ReactNode, expand?: string): Promise<string> {
const container = document.createElement("div");
document.body.appendChild(container);
const root = createRoot(container);
await act(async () => root.render(node));
if (expand) await act(async () => {
container.querySelector(`[aria-label="${expand}"]`)?.dispatchEvent(new KeyboardEvent("keydown", { key: "Enter", bubbles: true }));
});
const html = container.innerHTML;
await act(async () => root.unmount());
container.remove();
return html;
}
async function renderSection(
Component: ComponentType<AdapterConfigFieldsProps>,
adapterType: string,
section: AdapterConfigSection,
config: Record<string, unknown> = {},
) {
return renderToStaticMarkup(
return renderMarkup(
<TooltipProvider>
<Component
mode="edit"
@@ -36,31 +56,31 @@ function renderSection(
}
describe("adapter configuration sections", () => {
it("separates provider selection from lifecycle and hides fixed Codex permissions", () => {
it("separates provider selection from lifecycle and hides fixed Codex permissions", async () => {
const config = {
provider: "codex",
lifecycleMode: "warm",
idleTimeoutMs: 45000,
};
const adapter = renderSection(
const adapter = await renderSection(
CodexLocalConfigFields,
"paperclip_runner",
"adapter",
config,
);
const configuration = renderSection(
const configuration = await renderSection(
CodexLocalConfigFields,
"paperclip_runner",
"configuration",
config,
);
const policy = renderSection(
const policy = await renderSection(
CodexLocalConfigFields,
"paperclip_runner",
"runPolicy",
config,
);
expect(adapter).toContain('<option value="acpx">ACP agents</option>');
expect(adapter).toContain('aria-label="Harness"');
expect(adapter).not.toContain("Runner lifecycle");
expect(configuration).not.toContain("Permission mode");
expect(configuration).not.toContain("Runner lifecycle");
@@ -69,13 +89,12 @@ describe("adapter configuration sections", () => {
expect(policy).not.toContain("ACP agents");
});
it("keeps ACP agent admission choices in the adapter section", () => {
it("keeps ACP agent admission choices in the adapter section", async () => {
const config = { provider: "acpx", acpxAgent: "claude" };
const adapter = renderSection(CodexLocalConfigFields, "paperclip_runner", "adapter", config);
const policy = renderSection(CodexLocalConfigFields, "paperclip_runner", "runPolicy", config);
const adapter = await renderSection(CodexLocalConfigFields, "paperclip_runner", "adapter", config);
const policy = await renderSection(CodexLocalConfigFields, "paperclip_runner", "runPolicy", config);
expect(adapter).toContain('<option value="claude" selected="">Claude</option>');
expect(adapter).toContain('<option value="pi" disabled="">Pi — qualification pending</option>');
expect(adapter).toContain('aria-label="ACP agent"');
expect(adapter).not.toContain("Runner lifecycle");
expect(policy).toContain("Runner lifecycle");
expect(policy).not.toContain("ACP agent");
@@ -88,31 +107,31 @@ describe("adapter configuration sections", () => {
["gemini_local", GeminiLocalConfigFields],
] as const)(
"separates ACP commands and lifecycle for %s",
(type, Component) => {
async (type, Component) => {
const config = {
engine: "acp",
agentCommand: "saved-command",
warmHandleIdleMs: 1234,
};
expect(renderSection(Component, type, "advanced", config)).toContain(
expect(await renderSection(Component, type, "advanced", config)).toContain(
'value="saved-command"',
);
expect(
renderSection(Component, type, "configuration", config),
await renderSection(Component, type, "configuration", config),
).not.toContain("ACP server command");
const policy = renderSection(Component, type, "runPolicy", config);
const policy = await renderSection(Component, type, "runPolicy", config);
expect(policy).toContain("ACP session mode");
expect(policy).toContain('value="1234"');
expect(policy).not.toContain("ACP server command");
},
);
it("keeps process command and arguments under Advanced with saved values", () => {
it("keeps process command and arguments under Advanced with saved values", async () => {
const config = { command: "node", args: ["worker.js", "--quiet"] };
expect(
renderSection(ProcessConfigFields, "process", "configuration", config),
await renderSection(ProcessConfigFields, "process", "configuration", config),
).toBe("");
const advanced = renderSection(
const advanced = await renderSection(
ProcessConfigFields,
"process",
"advanced",
@@ -127,12 +146,12 @@ describe("adapter configuration sections", () => {
["hermes_gateway", HermesGatewayConfigFields],
] as const)(
"moves %s timeouts without changing their values",
(type, Component) => {
async (type, Component) => {
const config = { timeoutSec: 37 };
expect(
renderSection(Component, type, "configuration", config),
await renderSection(Component, type, "configuration", config),
).not.toContain('value="37"');
expect(renderSection(Component, type, "runPolicy", config)).toContain(
expect(await renderSection(Component, type, "runPolicy", config)).toContain(
'value="37"',
);
},
ui/src/components/AdapterMark.tsx
+59
View File
@@ -0,0 +1,59 @@
import { getAdapterDisplay } from "@/adapters/adapter-display-registry";
import { cn } from "@/lib/utils";
const brandMarks: Record<string, { src: string; dark?: string }> = {
claude_local: { src: "/brands/claude-color.svg" },
codex_local: { src: "/brands/codex-color.svg" },
gemini_local: { src: "/brands/adapters/gemini-color.svg" },
kimi_local: {
src: "/brands/adapters/kimi-color-light.svg",
dark: "/brands/adapters/kimi-color.svg",
},
...Object.fromEntries(
[
["cursor", "cursor"],
["cursor_cloud", "cursor"],
["grok_local", "grok"],
["hermes_local", "hermesagent"],
["hermes_gateway", "hermesagent"],
["pi_local", "pi"],
].map(([type, icon]) => [
type,
{
src: `/brands/adapters/${icon}.svg`,
dark: `/brands/adapters/${icon}-dark.svg`,
},
]),
),
};
export function AdapterMark({
type,
className = "size-6",
}: {
type: string;
className?: string;
}) {
const Icon = getAdapterDisplay(type).icon;
const mark = brandMarks[type];
if (!mark) return <Icon className={className} />;
return (
<>
<img
src={mark.src}
className={cn(
"shrink-0 object-contain",
mark.dark && "dark:hidden",
className,
)}
alt=""
/>
{mark.dark && (
<img
src={mark.dark}
className={cn("hidden shrink-0 object-contain dark:block", className)}
alt=""
/>
)}
</>
);
}
ui/src/components/AgentConfigForm.render.test.tsx
+48 -7
View File
@@ -13,6 +13,10 @@ import { AgentConfigForm, AdapterLoginPanel, subtractPersistedOverlay, type Adap
import { defaultCreateValues } from "./agent-config-defaults";
import { buildNewAgentHirePayload } from "../lib/new-agent-hire-payload";
import { ApiError } from "../api/client";
import { aiConnectionsApi } from "../api/ai-connections";
import { CodexLocalConfigFields } from "../adapters/codex-local/config-fields";
import type { AdapterConfigFieldsProps } from "../adapters/types";
import { DEFAULT_CODEX_LOCAL_MODEL } from "@paperclipai/adapter-codex-local";
const mockAgentsApi = vi.hoisted(() => ({
adapterModels: vi.fn(),
@@ -103,12 +107,10 @@ vi.mock("../adapters", () => ({
// The stand-in also records the two gates the form resolves for every
// adapter, so a test can assert the plumbing without rendering a real
// adapter's fields.
ConfigFields: ({ adapterType, hideInstructionsFile, managedSandboxOnly }: {
adapterType: string;
hideInstructionsFile?: boolean;
managedSandboxOnly?: boolean;
}) =>
adapterType === "hermes_gateway"
ConfigFields: (props: AdapterConfigFieldsProps) => {
if (type === "paperclip_runner") return <CodexLocalConfigFields {...props} />;
const { adapterType, hideInstructionsFile, managedSandboxOnly } = props;
return adapterType === "hermes_gateway"
? <div data-testid="hermes-gateway-config-fields">Hermes Gateway fields</div>
: (
<div
@@ -116,7 +118,8 @@ vi.mock("../adapters", () => ({
data-hide-instructions-file={String(hideInstructionsFile === true)}
data-managed-sandbox-only={String(managedSandboxOnly === true)}
/>
),
);
},
buildAdapterConfig: (values: { model?: string }) => ({
model: values.model || undefined,
}),
@@ -784,6 +787,44 @@ describe("AgentConfigForm environment selector", () => {
expect(result.onSave.mock.calls[0][0].adapterConfig.effort).toBeUndefined();
});
it.each([
["Codex", "codex", undefined, DEFAULT_CODEX_LOCAL_MODEL],
["ACP agents", "acpx", "claude", "claude-sonnet-5"],
["Claude Managed", "claude_managed", undefined, "claude-sonnet-5"],
])("saves the %s harness default without the previous OpenCode model prefix", async (label, provider, acpxAgent, model) => {
const accountList = vi.spyOn(aiConnectionsApi, "list").mockResolvedValue({
currentUserId: "you",
canManageConnections: true,
connections: [],
});
try {
const result = await renderForm([], {
adapterType: "paperclip_runner",
adapterConfig: { provider: "opencode", model: "openrouter/anthropic/claude-sonnet-4.6" },
runtimeConfig: { aiConnection: { mode: "responsible_user", provider: "openrouter", method: "api_key" } },
});
roots.push(result.root);
await act(async () => {
result.container.querySelector('[aria-label="Harness"]')!
.dispatchEvent(new KeyboardEvent("keydown", { key: "Enter", bubbles: true }));
});
await flushReact();
const option = [...document.querySelectorAll<HTMLElement>('[role="option"]')]
.find(element => element.textContent === label)!;
expect(option).toBeTruthy();
await act(async () => {
option.dispatchEvent(new KeyboardEvent("keydown", { key: "Enter", bubbles: true }));
});
await flushReact();
await clickByText(result.container, "Save");
expect(result.onSave).toHaveBeenCalledWith(expect.objectContaining({
adapterConfig: expect.objectContaining({ provider, model, ...(acpxAgent ? { acpxAgent } : {}) }),
}));
} finally {
accountList.mockRestore();
}
});
it("saves Grok 4.7 reasoning effort using the runtime key", async () => {
const result = await renderForm([], { adapterType: "grok_local", adapterConfig: { model: "grok-4.7", reasoningEffort: "high" } });
roots.push(result.root);
ui/src/components/AgentConfigForm.tsx
+94 -23
View File
@@ -1,3 +1,5 @@
import { useConnectionModels } from "./ai-connections/useConnectionModels";
import { aiRoutingHarness } from "@paperclipai/shared";
import { AiConnectionField } from "./ai-connections/AiConnectionField";
import { aiConnectionBindingSchema, aiRuntimeConnectionBindingSchema } from "@paperclipai/shared";
import { testAgentSetup } from "@/lib/test-agent-setup";
@@ -42,6 +44,9 @@ import {
PopoverTrigger,
} from "@/components/ui/popover";
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import { NativeSelect, selectTriggerClassName } from "@/components/ui/select";
import { AdapterMark } from "./AdapterMark";
import { FolderOpen, Heart, ChevronDown, X, Copy, Check, ExternalLink, Loader2, TriangleAlert, Bug } from "lucide-react";
import { asBoolean, asFiniteNumber, asObject, cn } from "../lib/utils";
import { copyTextToClipboard } from "../lib/clipboard";
@@ -76,7 +81,6 @@ import { getUIAdapter } from "../adapters";
import { ClaudeLocalAdvancedFields } from "../adapters/claude-local/config-fields";
import { MarkdownEditor } from "./MarkdownEditor";
import { ChoosePathButton } from "./PathInstructionsModal";
import { OpenCodeLogoIcon } from "./OpenCodeLogoIcon";
import { ReportsToPicker } from "./ReportsToPicker";
import {
EnvironmentVariablesEditor,
@@ -894,6 +898,7 @@ export function AgentConfigForm(props: AgentConfigFormProps) {
const modelProvider = adapterType === "opencode_local" && aiConnectionBindingSchema.safeParse(
(overlay.runtime.runtimeConfig as Record<string, unknown> | undefined)?.aiConnection ?? runtimeConfig.aiConnection,
).data?.provider === "openrouter" ? "openrouter" : runnerProvider;
const connectionModels = useConnectionModels(selectedCompanyId, isCreate ? undefined : aiConnectionBindingSchema.safeParse((overlay.runtime.runtimeConfig as Record<string, unknown> | undefined)?.aiConnection ?? runtimeConfig.aiConnection).data, aiRoutingHarness(adapterType, runnerProvider, eff("adapterConfig", "acpxAgent", config.acpxAgent)));
// Fetch adapter models for the effective provider, including unsaved changes.
const modelQueryKey = selectedCompanyId
? queryKeys.agents.adapterModels(selectedCompanyId, adapterType, currentDefaultEnvironmentId || null, modelProvider)
@@ -901,17 +906,19 @@ export function AgentConfigForm(props: AgentConfigFormProps) {
const {
data: fetchedModels,
error: fetchedModelsError,
isLoading: fetchingModels,
} = useQuery({
queryKey: modelQueryKey,
queryFn: () => agentsApi.adapterModels(selectedCompanyId!, adapterType, {
environmentId: currentDefaultEnvironmentId || null,
provider: modelProvider,
}),
enabled: Boolean(selectedCompanyId),
enabled: Boolean(selectedCompanyId) && !connectionModels,
});
const [refreshModelsError, setRefreshModelsError] = useState<string | null>(null);
const [refreshingModels, setRefreshingModels] = useState(false);
const models = fetchedModels ?? externalModels ?? [];
const models = connectionModels?.models ?? fetchedModels ?? externalModels ?? [];
const modelError = connectionModels ? connectionModels.error : fetchedModelsError;
const adapterCommandField = "command";
const {
data: detectedModelData,
@@ -946,7 +953,10 @@ export function AgentConfigForm(props: AgentConfigFormProps) {
set: isCreate ? (patch: Partial<CreateConfigValues>) => props.onChange(patch) : null,
config,
eff: eff as <T>(group: "adapterConfig", field: string, original: T) => T,
mark: mark as (group: "adapterConfig", field: string, value: unknown) => void,
// Harness transitions supply the new harness's default model. Resolve
// user-selected model IDs in ModelDropdown, not through the previous
// render's harness when adapter fields change several values together.
mark,
models,
// Resolve the effective instructions-file gate once. The instructions file
// is an absolute host path, so the managed-sandbox-only policy hides it for
@@ -1658,7 +1668,7 @@ export function AgentConfigForm(props: AgentConfigFormProps) {
</Field>
)}
{!isCreate && selectedCompanyId && <AiConnectionField companyId={selectedCompanyId} agentId={props.agent.id} agentName={props.agent.name} adapterType={adapterType === "paperclip_runner" ? eff("adapterConfig", "provider", config.provider) === "codex" ? "codex_local" : eff("adapterConfig", "provider", config.provider) === "opencode" ? "opencode_local" : eff("adapterConfig", "provider", config.provider) === "acpx" && eff("adapterConfig", "acpxAgent", config.acpxAgent) === "grok" ? "grok_local" : eff("adapterConfig", "provider", config.provider) === "acpx" && eff("adapterConfig", "acpxAgent", config.acpxAgent) === "claude" ? "claude_local" : adapterType : adapterType}
{!isCreate && selectedCompanyId && <AiConnectionField companyId={selectedCompanyId} agentId={props.agent.id} agentName={props.agent.name} adapterType={aiRoutingHarness(adapterType, eff("adapterConfig", "provider", config.provider), eff("adapterConfig", "acpxAgent", config.acpxAgent))}
routerAdapterType={adapterType} value={aiRuntimeConnectionBindingSchema.safeParse((overlay.runtime.runtimeConfig as Record<string, unknown> | undefined)?.aiConnection ?? runtimeConfig.aiConnection).data}
model={String(eff("adapterConfig", "model", config.model) ?? "")} environmentId={currentDefaultEnvironmentId || undefined} legacy
onChange={binding => mark("runtime", "runtimeConfig", { ...runtimeConfig, aiConnection: binding })} />}
@@ -1719,6 +1729,7 @@ export function AgentConfigForm(props: AgentConfigFormProps) {
{isLocal && (<>
<ModelDropdown
models={models}
loadingModels={connectionModels?.isLoading ?? fetchingModels}
value={currentModelId}
onChange={(v) => {
const supportedEfforts = setupEfforts(adapterType, v);
@@ -1732,7 +1743,7 @@ export function AgentConfigForm(props: AgentConfigFormProps) {
});
return;
}
mark("adapterConfig", "model", v || undefined);
mark("adapterConfig", "model", connectionModels?.resolveModel(v) || v || undefined);
if (clearUnsupportedEffort) {
mark("adapterConfig", thinkingEffortKey, undefined);
mark("adapterConfig", "reasoningEffort", undefined);
@@ -1743,35 +1754,36 @@ export function AgentConfigForm(props: AgentConfigFormProps) {
defaultLabel={adapterType === "claude_local" ? `Default (${DEFAULT_CLAUDE_LOCAL_MODEL})` : undefined}
allowDefault={adapterType !== "opencode_local" && adapterType !== "pi_local" && adapterType !== "paperclip_runner"}
required={adapterType === "opencode_local" || adapterType === "pi_local"}
groupByProvider={adapterType === "opencode_local" || adapterType === "pi_local"}
preserveOrder={adapterCuratesModelOrder(adapterType)}
groupByProvider={!connectionModels && (adapterType === "opencode_local" || adapterType === "pi_local")}
preserveOrder={Boolean(connectionModels) || adapterCuratesModelOrder(adapterType)}
creatable
detectedModel={detectedModel}
detectedModel={connectionModels ? undefined : detectedModel}
detectedModelCandidates={[]}
onDetectModel={adapterType === "opencode_local" || adapterType === "paperclip_runner"
onDetectModel={connectionModels || adapterType === "opencode_local" || adapterType === "paperclip_runner"
? undefined
: async () => {
const result = await refetchDetectedModel();
return result.data?.model ?? null;
}}
onRefreshModels={
supportsAdapterModelRefresh(adapterType)
connectionModels ? connectionModels.refreshModels : supportsAdapterModelRefresh(adapterType)
? handleRefreshModels
: undefined
}
refreshingModels={refreshingModels}
refreshingModels={connectionModels?.refreshing ?? refreshingModels}
detectModelLabel="Detect model"
emptyDetectHint="No model detected. Select or enter one manually."
/>
{(refreshModelsError || fetchedModelsError) && (
{(refreshModelsError || modelError) && (
<p className="text-xs text-destructive">
{refreshModelsError
?? (fetchedModelsError instanceof Error
? fetchedModelsError.message
?? (modelError instanceof Error
? modelError.message
: "Failed to load adapter models.")}
</p>
)}
{adapterType === "opencode_local"
&& !connectionModels
&& currentDefaultEnvironment
&& currentDefaultEnvironment.driver !== "local" && (
<p className="text-xs text-muted-foreground">
@@ -3666,19 +3678,26 @@ export function AdapterTypeDropdown({
return (
<Popover open={open} onOpenChange={setOpen}>
<PopoverTrigger asChild>
<button className="inline-flex items-center gap-1.5 rounded-md border border-border px-2.5 py-1.5 text-sm hover:bg-accent/50 transition-colors w-full justify-between">
<button
type="button"
data-size="default"
className={cn(selectTriggerClassName, "w-full")}
>
<span className="inline-flex min-w-0 items-center gap-1.5">
{value === "opencode_local" ? <OpenCodeLogoIcon className="h-3.5 w-3.5" /> : null}
<span aria-hidden="true" className="inline-flex shrink-0">
<AdapterMark type={value} className="size-4" />
</span>
<span className="truncate">{adapterLabels[value] ?? getAdapterLabel(value)}</span>
{selectedDisplay.experimental && <ExperimentalBadge />}
</span>
<ChevronDown className="h-3 w-3 text-muted-foreground" />
<ChevronDown className="size-4 opacity-50" />
</button>
</PopoverTrigger>
<PopoverContent className="w-(--radix-popover-trigger-width) p-1" align="start">
{adapterList.map((item) => (
<button
key={item.value}
type="button"
disabled={item.comingSoon}
className={cn(
"flex items-center justify-between w-full px-2 py-1.5 text-sm rounded",
@@ -3695,7 +3714,9 @@ export function AdapterTypeDropdown({
}}
>
<span className="inline-flex items-center gap-1.5">
{item.value === "opencode_local" ? <OpenCodeLogoIcon className="h-3.5 w-3.5" /> : null}
<span aria-hidden="true" className="inline-flex shrink-0">
<AdapterMark type={item.value} className="size-4" />
</span>
<span>{item.label}</span>
{item.experimental && <ExperimentalBadge />}
</span>
@@ -3733,9 +3754,11 @@ export function ModelDropdown({
onDetectModel,
onRefreshModels,
refreshingModels,
loadingModels,
detectModelLabel,
emptyDetectHint,
defaultLabel,
presentation = "searchable",
}: {
models: AdapterModel[];
value: string;
@@ -3753,11 +3776,14 @@ export function ModelDropdown({
onDetectModel?: () => Promise<string | null>;
onRefreshModels?: () => Promise<void>;
refreshingModels?: boolean;
loadingModels?: boolean;
detectModelLabel?: string;
emptyDetectHint?: string;
defaultLabel?: string;
presentation?: "searchable" | "native";
}) {
const [modelSearch, setModelSearch] = useState("");
const [enteringCustomModel, setEnteringCustomModel] = useState(false);
const [detectingModel, setDetectingModel] = useState(false);
const selected = models.find((m) => m.id === value);
const manualModel = modelSearch.trim();
@@ -3826,6 +3852,47 @@ export function ModelDropdown({
}
}
if (presentation === "native") {
const customOption = "__paperclip_custom_model__";
const extraModels = [...new Set([value, ...promotedModelIds])].filter(id => id && (!models.some(model => model.id === id) || promotedModelIds.has(id)));
return (
<Field label="Model" hint={help.model}>
<NativeSelect
aria-label="Model"
aria-busy={loadingModels || refreshingModels}
value={enteringCustomModel ? customOption : value}
required={required && !enteringCustomModel}
onChange={event => {
const next = event.target.value;
setEnteringCustomModel(next === customOption);
if (next !== customOption) onChange(next);
}}
>
<option value="" disabled={!allowDefault}>
{allowDefault ? (defaultLabel ?? "Default") : loadingModels ? "Loading models…" : required ? "Select model (required)" : "Select model"}
</option>
{extraModels.map(id => <option key={id} value={id}>{models.find(model => model.id === id)?.label ?? id}</option>)}
{groupedModels.map(({ provider, entries }) => groupByProvider ? (
<optgroup key={provider} label={provider}>
{entries.map(model => <option key={model.id} value={model.id}>{model.label}</option>)}
</optgroup>
) : entries.map(model => <option key={model.id} value={model.id}>{model.label}</option>))}
{creatable && <option value={customOption}>Enter custom model…</option>}
</NativeSelect>
{enteringCustomModel && (
<label className="mt-3 block space-y-1 text-xs text-muted-foreground">
Model ID
<Input aria-label="Model ID" value={value} onChange={event => onChange(event.target.value)}
placeholder="Enter model ID or alias" autoFocus required={required} />
</label>
)}
{loadingModels && <p role="status" className="mt-2 text-xs text-muted-foreground">Loading models…</p>}
{onRefreshModels && <Button type="button" variant="ghost" size="sm" disabled={refreshingModels}
onClick={() => void onRefreshModels()}>{refreshingModels ? "Refreshing…" : "Refresh models"}</Button>}
</Field>
);
}
return (
<Field label="Model" hint={help.model}>
<Popover
@@ -3836,14 +3903,18 @@ export function ModelDropdown({
}}
>
<PopoverTrigger asChild>
<button type="button" className="inline-flex items-center gap-1.5 rounded-md border border-border px-2.5 py-1.5 text-sm hover:bg-accent/50 transition-colors w-full justify-between">
<span className={cn(!value && "text-muted-foreground")}>
<button
type="button"
data-size="default"
className={cn(selectTriggerClassName, "w-full")}
>
<span className={cn("truncate", !value && "text-muted-foreground")}>
{selected
? selected.label
: value
|| (allowDefault ? (defaultLabel ?? "Default") : required ? "Select model (required)" : "Select model")}
</span>
<ChevronDown className="h-3 w-3 text-muted-foreground" />
<ChevronDown className="size-4 opacity-50" />
</button>
</PopoverTrigger>
<PopoverContent className="w-(--radix-popover-trigger-width) p-1" align="start">
@@ -4024,7 +4095,7 @@ export function ModelDropdown({
{filteredModels.length === 0 && !canCreateManualModel && promotedModelIds.size === 0 && (
<div className="px-2 py-2 space-y-2">
<p className="text-xs text-muted-foreground">
{onDetectModel
{loadingModels ? "Loading models…" : onDetectModel
? (emptyDetectHint ?? "No model detected yet. Enter a provider/model manually.")
: "No models found."}
</p>
ui/src/components/ai-connections/AiConnectionAccountControls.tsx
+1 -1
View File
@@ -27,7 +27,7 @@ export function AiConnectionAccountControls({
const activeDefault = account.isDefault && available;
return (
<section className="space-y-4" aria-label="AI account settings">
{ownPersonal && (
{ownPersonal && !account.routing && (
<div className={cn(
"flex flex-wrap items-center justify-between gap-3 rounded-lg border border-border p-3",
activeDefault && "border-(--status-task-done)/30 bg-(--status-task-done)/5",
ui/src/components/ai-connections/ManagedAiConnectionDetails.tsx
+4 -8
View File
@@ -6,7 +6,7 @@ import { toolsApi } from "@/api/tools";
import { useNavigate } from "@/lib/router";
import { AiConnectionAccountControls } from "./AiConnectionAccountControls";
import { AiConnectionUsagePanel } from "./AiConnectionUsagePanel";
import type { ToolConnection } from "@paperclipai/shared";
import type { ToolConnection, AiConnectionMetadata } from "@paperclipai/shared";
import { aiMethodLabel } from "./model";
export function ManagedAiConnectionRow({
@@ -14,16 +14,11 @@ export function ManagedAiConnectionRow({
}: {
connection: ToolConnection;
}) {
const metadata = connection.config?.ai as
| {
provider: "anthropic" | "openai" | "openrouter" | "xai";
method: "subscription" | "api_key";
}
| undefined;
const metadata = connection.config?.ai as AiConnectionMetadata | undefined;
if (!metadata) return null;
return (
<p className="text-xs text-muted-foreground">
{aiMethodLabel(metadata.provider, metadata.method)} ·{" "}
{metadata.routing ? metadata.routing.kind === "bedrock" ? `Bedrock · ${metadata.routing.region}` : metadata.routing.kind === "openrouter" ? "OpenRouter" : metadata.routing.baseUrl : aiMethodLabel(metadata.provider, metadata.method)} ·{" "}
{connection.credentialPolicy === "per_user"
? "Personal"
: "Company shared"}
@@ -90,6 +85,7 @@ export function ManagedAiConnectionDetails({
);
return (
<div className="space-y-4">
{account.routing && <details className="rounded-lg border border-border p-4"><summary className="cursor-pointer text-sm font-medium">Provider settings</summary><dl className="mt-4 grid grid-cols-2 gap-3 text-sm"><dt className="text-muted-foreground">Destination</dt><dd className="break-all">{account.routing.baseUrl ?? (account.routing.kind === "bedrock" ? account.routing.region : "OpenRouter")}</dd><dt className="text-muted-foreground">API format</dt><dd>{account.routing.kind === "openrouter" ? "Selected by harness" : account.routing.protocol}</dd><dt className="text-muted-foreground">Authentication</dt><dd>{account.routing.auth}</dd><dt className="text-muted-foreground">Models</dt><dd>{account.routing.models.map(m => m.label ?? m.id).join(", ") || "Enter a model ID on the agent"}</dd></dl></details>}
<AiConnectionAccountControls
account={account}
grant={grant}
ui/src/components/ai-connections/useConnectionModels.test.tsx
+108
View File
@@ -0,0 +1,108 @@
// @vitest-environment jsdom
import { act } from "react";
import { createRoot, type Root } from "react-dom/client";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { afterEach, beforeEach, expect, it, vi } from "vitest";
import type { AiConnectionBinding, AiManagedConnectionSummary } from "@paperclipai/shared";
import { aiConnectionsApi } from "@/api/ai-connections";
import { agentsApi } from "@/api/agents";
import { useConnectionModels } from "./useConnectionModels";
vi.mock("@/api/ai-connections", () => ({ aiConnectionsApi: { list: vi.fn() } }));
vi.mock("@/api/agents", () => ({ agentsApi: { adapterModels: vi.fn() } }));
const binding: AiConnectionBinding = { provider: "openrouter", method: "api_key", mode: "shared", connectionId: "router", grantId: "grant" };
const account: AiManagedConnectionSummary = {
id: "router", grantId: "grant", companyId: "company", provider: "openrouter", method: "api_key",
name: "Company OpenRouter", ownership: "shared", isDefault: false, status: "connected",
routing: { kind: "openrouter", protocol: "chat", auth: "bearer", models: [] },
};
const catalog = [
{ id: "openrouter/z-ai/glm-5", label: "Z.AI GLM" },
{ id: "openrouter/anthropic/claude-sonnet-4.5", label: "Claude Sonnet" },
{ id: "openrouter/openrouter/auto", label: "Auto Router" },
];
let root: Root;
let client: QueryClient;
let result: ReturnType<typeof useConnectionModels>;
let render: (value: AiConnectionBinding | undefined) => Promise<void>;
beforeEach(() => {
vi.mocked(agentsApi.adapterModels).mockResolvedValue(catalog);
vi.mocked(aiConnectionsApi.list).mockResolvedValue({ currentUserId: "you", connections: [account], canManageConnections: true });
});
afterEach(async () => {
await act(async () => root?.unmount());
client?.clear();
vi.resetAllMocks();
});
async function mount(accounts: AiManagedConnectionSummary[] = [account], harness = "opencode_local", value: AiConnectionBinding | undefined = binding) {
client = new QueryClient({ defaultOptions: { queries: { retry: false, staleTime: Infinity } } });
client.setQueryData(["ai-connections", "company"], { currentUserId: "you", connections: accounts });
root = createRoot(document.createElement("div"));
function Probe({ value }: { value: AiConnectionBinding | undefined }) {
result = useConnectionModels("company", value, harness);
return null;
}
render = async next => { await act(async () => root.render(<QueryClientProvider client={client}><Probe value={next} /></QueryClientProvider>)); };
await render(value);
}
it.each(["opencode_local", "codex_local", "claude_local", "hermes_local"])("automatically discovers an empty OpenRouter connection's models for %s", async harness => {
await mount([account], harness);
await vi.waitFor(() => expect(result?.models).toHaveLength(3));
expect(agentsApi.adapterModels).toHaveBeenCalledWith("company", "opencode_local", { provider: "openrouter" });
expect(result?.models.map(m => m.id)).toEqual(catalog.map(m => harness === "opencode_local" ? m.id : m.id.slice("openrouter/".length)));
expect(result?.models.map(m => m.label)).toEqual(catalog.map(m => m.label));
expect(result?.resolveModel("anthropic/custom-model")).toBe(harness === "opencode_local" ? "openrouter/anthropic/custom-model" : "anthropic/custom-model");
});
it("also discovers models for legacy OpenRouter connections without routing metadata", async () => {
await mount([{ ...account, routing: undefined }]);
await vi.waitFor(() => expect(result?.models).toEqual(catalog));
});
it("preserves an explicit connection model list instead of replacing it with the public catalog", async () => {
await mount([{ ...account, routing: { ...account.routing!, models: [{ id: "company/model", label: "Approved model" }] } }]);
expect(result?.models).toEqual([{ id: "openrouter/company/model", label: "Approved model" }]);
expect(agentsApi.adapterModels).not.toHaveBeenCalled();
});
it("does not reuse public OpenRouter models after switching to a custom gateway", async () => {
const gateway: AiManagedConnectionSummary = { ...account, id: "gateway", grantId: "gateway-grant", provider: "openai", routing: { kind: "gateway", protocol: "chat", auth: "bearer", baseUrl: "https://example.com/v1", models: [] } };
await mount([account, gateway]);
await vi.waitFor(() => expect(result?.models).toEqual(catalog));
await render({ provider: "openai", method: "api_key", mode: "shared", connectionId: gateway.id, grantId: gateway.grantId });
expect(result?.models).toEqual([]);
expect(result?.resolveModel("private-model")).toBe("paperclip/private-model");
expect(result?.refreshModels).toBeUndefined();
});
it("leaves native model discovery alone", async () => {
await mount([], "codex_local", { provider: "openai", method: "subscription", mode: "responsible_user" });
expect(result).toBeUndefined();
expect(agentsApi.adapterModels).not.toHaveBeenCalled();
});
it("only uses the current person's default connection models", async () => {
await mount([
{ ...account, ownership: "personal", ownerUserId: "someone-else", isDefault: true, routing: { ...account.routing!, models: [{ id: "other/private-model" }] } },
{ ...account, ownership: "personal", ownerUserId: "you", isDefault: true },
], "opencode_local", { provider: "openrouter", method: "api_key", mode: "responsible_user" });
await vi.waitFor(() => expect(result?.models).toEqual(catalog));
});
it("exposes loading and allows retry after a catalog failure", async () => {
let reject!: (error: Error) => void;
vi.mocked(agentsApi.adapterModels).mockReturnValueOnce(new Promise((_, rejectPromise) => { reject = rejectPromise; }));
await mount();
expect(result?.isLoading).toBe(true);
await act(async () => reject(new Error("Catalog unavailable")));
await vi.waitFor(() => expect(result?.error?.message).toBe("Catalog unavailable"));
expect(result?.isLoading).toBe(false);
expect(result?.resolveModel("manual/model")).toBe("openrouter/manual/model");
await act(async () => { await result?.refreshModels?.(); });
await vi.waitFor(() => expect(result?.models).toEqual(catalog));
expect(result?.error).toBeNull();
});
ui/src/components/ai-connections/useConnectionModels.ts
+55
View File
@@ -0,0 +1,55 @@
import { useQuery } from "@tanstack/react-query";
import { aiRoutingModel, type AiConnectionBinding, type AiProviderRouting } from "@paperclipai/shared";
import { aiConnectionsApi } from "@/api/ai-connections";
import { agentsApi } from "@/api/agents";
import { queryKeys } from "@/lib/queryKeys";
export function useConnectionModels(
companyId: string | null | undefined,
binding: AiConnectionBinding | undefined,
harness: string,
) {
const accounts = useQuery({
queryKey: ["ai-connections", companyId],
queryFn: () => aiConnectionsApi.list(companyId!),
enabled: Boolean(companyId && binding),
});
const connection = accounts.data?.connections.find((c) =>
binding?.mode === "responsible_user"
? c.provider === binding.provider && c.isDefault && c.ownerUserId === accounts.data?.currentUserId && c.ownership === "personal"
: c.id === binding?.connectionId && c.grantId === binding?.grantId,
);
const routing = connection?.routing;
const openRouter = routing?.kind === "openrouter" || (!routing && binding?.provider === "openrouter");
const discover = openRouter && !routing?.models.length;
// Reuse the existing public catalog and its query cache. It returns OpenCode IDs;
// strip only that transport prefix for other harnesses (openrouter/auto is
// itself a valid upstream model ID).
const catalog = useQuery({
queryKey: queryKeys.agents.adapterModels(companyId ?? "none", "opencode_local", null, "openrouter"),
queryFn: () => agentsApi.adapterModels(companyId!, "opencode_local", { provider: "openrouter" }),
enabled: Boolean(companyId && discover),
staleTime: 60_000,
retry: false,
});
const effectiveRouting: AiProviderRouting | undefined = routing ?? (openRouter
? { kind: "openrouter", protocol: "chat", auth: "bearer", models: [] }
: undefined);
const modelOptions = discover
? (catalog.data ?? []).map((m) => ({ ...m, id: harness === "opencode_local" ? m.id : m.id.replace(/^openrouter\//, "") }))
: routing?.models ?? [];
return effectiveRouting
? {
models: modelOptions.map((m) => ({
id: aiRoutingModel(effectiveRouting, harness, m.id),
label: m.label ?? m.id,
})),
isLoading: discover && catalog.isLoading,
error: discover ? catalog.error : null,
refreshing: discover && catalog.isFetching,
refreshModels: discover ? async () => { await catalog.refetch(); } : undefined,
resolveModel: (model: string) =>
aiRoutingModel(effectiveRouting, harness, model),
}
: undefined;
}
ui/src/components/ai-connections/useLocalAiLogin.ts
+1 -1
View File
@@ -3,7 +3,7 @@ import type { AiConnectionLoginIntent, LocalAiLoginAttempt, LocalAiLoginStatus }
import { aiConnectionsApi } from "@/api/ai-connections";
/** Every authentication host uses the same local credential check and login lifecycle. */
export function useLocalAiLogin(companyId: string | null, intent: AiConnectionLoginIntent, enabled: boolean, _legacyOptions?: { allowHostClaude?: boolean }) {
export function useLocalAiLogin(companyId: string | null, intent: AiConnectionLoginIntent, enabled: boolean) {
const isolated = true;
const active = Boolean(companyId && enabled);
const [attempt, setAttempt] = useState<LocalAiLoginAttempt | null>(null);
ui/src/components/new-agent/AgentBasicsDialog.tsx
+3 -56
View File
@@ -1,5 +1,6 @@
import { useCompany } from "@/context/CompanyContext";
import { useAgentAppearanceDraft } from "@/hooks/useAgentAppearanceDraft";
import { AdapterMark } from "../AdapterMark";
import { AgentCharacter } from "../AgentCharacter";
import { useId, useState } from "react";
import { useQuery } from "@tanstack/react-query";
@@ -25,62 +26,8 @@ export type AgentBasics = {
adapterType: string;
runnerProvider: string;
};
const brandMarks: Record<string, { src: string; dark?: string }> = {
claude_local: { src: "/brands/claude-color.svg" },
codex_local: { src: "/brands/codex-color.svg" },
gemini_local: { src: "/brands/adapters/gemini-color.svg" },
kimi_local: {
src: "/brands/adapters/kimi-color-light.svg",
dark: "/brands/adapters/kimi-color.svg",
},
...Object.fromEntries(
[
["cursor", "cursor"],
["cursor_cloud", "cursor"],
["grok_local", "grok"],
["hermes_local", "hermesagent"],
["hermes_gateway", "hermesagent"],
["pi_local", "pi"],
].map(([type, icon]) => [
type,
{
src: `/brands/adapters/${icon}.svg`,
dark: `/brands/adapters/${icon}-dark.svg`,
},
]),
),
};
export function AdapterMark({
type,
className = "size-6",
}: {
type: string;
className?: string;
}) {
const Icon = getAdapterDisplay(type).icon;
const mark = brandMarks[type];
if (!mark) return <Icon className={className} />;
return (
<>
<img
src={mark.src}
className={cn(
"shrink-0 object-contain",
mark.dark && "dark:hidden",
className,
)}
alt=""
/>
{mark.dark && (
<img
src={mark.dark}
className={cn("hidden shrink-0 object-contain dark:block", className)}
alt=""
/>
)}
</>
);
}
export { AdapterMark } from "../AdapterMark";
function AgentBasicsCharacter() {
const { selectedCompanyId } = useCompany();
const { appearance } = useAgentAppearanceDraft(`${selectedCompanyId}:new-agent`);
ui/src/components/new-agent/AgentProviderConnection.test.tsx
+78 -6
View File
@@ -1,6 +1,5 @@
// @vitest-environment jsdom
import { flushSync } from "react-dom";
import type { LocalAiLoginStatus } from "@paperclipai/shared";
import { createRoot, type Root } from "react-dom/client";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { afterEach, describe, expect, it, vi } from "vitest";
@@ -18,8 +17,8 @@ const managedApi = vi.hoisted(() => ({
list: vi.fn(async () => ({ currentUserId: "user-1", connections: [] })),
loginResult: vi.fn(async () => ({ connectionId: "login-account", grantId: "login-grant" })),
connectLocal: vi.fn(async () => ({ connectionId: "local-account", grantId: "local-grant" })),
startLocalLogin: vi.fn(async () => ({ sessionId: "local-attempt", command: "CODEX_HOME='/fixture/isolated-login' codex login", expiresAt: "2026-09-11T20:00:00Z" })),
checkLocalLogin: vi.fn(async (): Promise<LocalAiLoginStatus> => ({ status: "sign_in_required" })),
startLocalLogin: vi.fn(async () => ({ sessionId: "local-attempt", expiresAt: "2099-01-01T00:00:00Z" })),
checkLocalLogin: vi.fn(async () => ({ status: "sign_in_required" as "ready" | "sign_in_required" })),
cancelLocalLogin: vi.fn(async () => ({})),
create: vi.fn(async () => ({ connectionId: "managed-connection", grantId: "managed-grant" })),
}));
@@ -56,6 +55,7 @@ async function mount(
localEnvironment = false,
deploymentMode: "local_trusted" | "authenticated" = "local_trusted",
localAiLoginSupported = true,
advancedConnection?: Parameters<typeof AgentProviderConnection>[0]["advancedConnection"],
) {
const key =
adapterType === "claude_local" ? "ANTHROPIC_API_KEY" : "OPENAI_API_KEY";
@@ -128,18 +128,19 @@ async function mount(
testConnection={test}
onConnected={connected}
managedAccount={managedAccount}
advancedConnection={advancedConnection}
/>
</QueryClientProvider>,
),
);
await vi.waitFor(() => expect(mocks.personal).toHaveBeenCalled());
await vi.waitFor(() => expect(client.isFetching()).toBe(0));
if (savedApiKeys && !managedAccount) await vi.waitFor(() => expect(host.textContent).toContain("2 saved API keys"));
if (savedApiKeys && !managedAccount && !advancedConnection) await vi.waitFor(() => expect(host.textContent).toContain("2 saved API keys"));
return { test, connected, key };
}
function click(text: string) {
const button = [...host.querySelectorAll("button")].find((b) =>
b.textContent?.includes(text),
b.getAttribute("aria-label") ? b.getAttribute("aria-label") === text : b.textContent?.includes(text),
)!;
expect(button).toBeTruthy();
flushSync(() => button.click());
@@ -150,6 +151,74 @@ function openProvider() {
);
}
describe("AgentProviderConnection reuse", () => {
it("keeps all three connection tiles visible while switching forms", async () => {
await mount("codex_local", false, true, true, true, false, undefined, false, "local_trusted", true, {
content: <div>Advanced connection picker</div>,
});
const modeLabels = () => Array.from(host.querySelectorAll('[role="radio"]')).map(button => button.textContent);
expect(modeLabels()).toEqual(["OpenAISubscription", "OpenAIAPI key", "AdvancedCustom Gateway"]);
expect(host.querySelector('[aria-label="Saved subscription"]')).not.toBeNull();
expect(host.textContent).not.toContain("Advanced connection picker");
expect(host.textContent).not.toContain("instead");
click("OpenAIAPI key");
expect(host.querySelector('[aria-label="Saved API key"]')).not.toBeNull();
expect(host.querySelector('[role="radio"][aria-checked="true"]')?.textContent).toBe("OpenAIAPI key");
expect(host.querySelector('[aria-label="Saved subscription"]')).toBeNull();
expect(modeLabels()).toEqual(["OpenAISubscription", "OpenAIAPI key", "AdvancedCustom Gateway"]);
click("Advanced");
expect(host.textContent).toContain("Advanced connection picker");
expect(host.querySelector('[role="radio"][aria-checked="true"]')?.textContent).toBe("AdvancedCustom Gateway");
expect(host.querySelector("select")).toBeNull();
expect(modeLabels()).toEqual(["OpenAISubscription", "OpenAIAPI key", "AdvancedCustom Gateway"]);
const useConnection = Array.from(host.querySelectorAll("button")).find(button => button.textContent?.includes("Use connection"))!;
expect(useConnection.disabled).toBe(true);
click("OpenAISubscription");
expect(host.querySelector('[aria-label="Saved subscription"]')).not.toBeNull();
expect(host.textContent).not.toContain("Advanced connection picker");
});
it("continues with the selected advanced connection", async () => {
const binding = { provider: "openrouter" as const, method: "api_key" as const, mode: "shared" as const, connectionId: "router", grantId: "router-grant" };
const { connected, test } = await mount("codex_local", false, true, true, true, false, undefined, false, "local_trusted", true, {
content: <div>Advanced connection picker</div>, value: binding,
});
click("Advanced");
click("Use connection");
expect(connected).toHaveBeenCalledExactlyOnceWith({ env: {}, aiConnection: binding });
expect(test).not.toHaveBeenCalled();
});
it("preserves the API key when the selected tile is clicked again", async () => {
await mount("codex_local", false, true, false, false, false, undefined, false, "local_trusted", true, {
content: <div>Advanced connection picker</div>,
});
click("OpenAIAPI key");
const input = host.querySelector<HTMLInputElement>('input[type="password"]')!;
flushSync(() => {
Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, "value")!.set!.call(input, "fixture-key");
input.dispatchEvent(new Event("input", { bubbles: true }));
});
click("OpenAIAPI key");
expect(input.value).toBe("fixture-key");
const apiTile = host.querySelector<HTMLElement>('[role="radio"][aria-checked="true"]')!;
flushSync(() => apiTile.dispatchEvent(new KeyboardEvent("keydown", { key: "ArrowRight", bubbles: true })));
expect(host.querySelector('[role="radio"][aria-checked="true"]')?.textContent).toBe("AdvancedCustom Gateway");
expect(host.querySelectorAll('[role="radio"]')).toHaveLength(3);
});
it("uses the saved subscription after switching back from an advanced connection", async () => {
const { connected } = await mount("codex_local", false, true, true, true, false, undefined, false, "local_trusted", true, {
content: <div>Advanced connection picker</div>,
value: { provider: "openrouter", method: "api_key", mode: "shared", connectionId: "router", grantId: "router-grant" },
});
click("Advanced");
click("OpenAISubscription");
click("Use saved subscription");
await vi.waitFor(() => expect(connected).toHaveBeenCalled());
expect(connected.mock.calls[0][0]).toMatchObject({ env: { CODEX_HOME: { type: "secret_ref", secretId: "codex-home" } } });
expect(connected.mock.calls[0][0].aiConnection).toBeUndefined();
});
it.each([
["anthropic", "claude_local"], ["openai", "codex_local"], ["xai", "grok_local"],
] as const)("lets %s recovery switch methods without overwriting the original account", async (provider, adapterType) => {
@@ -273,11 +342,14 @@ describe("AgentProviderConnection reuse", () => {
expect(mocks.loginPanel).not.toHaveBeenCalled();
});
it("leaves a completed local account saved when its host is cancelled", async () => {
managedApi.checkLocalLogin.mockResolvedValue({ status: "ready" });
let finish!: (result: { connectionId: string; grantId: string }) => void;
managedApi.connectLocal.mockImplementationOnce(() => new Promise(resolve => { finish = resolve; }));
const onComplete = vi.fn();
await mount("claude_local", false, false, false, false, false, { intent: { provider: "anthropic", method: "subscription", name: "My account", ownership: "personal", agentIds: [], allAgents: false }, onComplete }, true);
openProvider(); click("Connect"); flushSync(() => root.unmount());
openProvider();
await vi.waitFor(() => expect(host.textContent).toContain("is signed in"));
click("Connect"); flushSync(() => root.unmount());
finish({ connectionId: "saved", grantId: "saved-grant" });
await new Promise(resolve => setTimeout(resolve, 0));
expect(onComplete).not.toHaveBeenCalled();
ui/src/components/new-agent/AgentProviderConnection.tsx
+67 -35
View File
@@ -2,7 +2,8 @@ import { healthApi } from "@/api/health";
import { aiConnectionsApi } from "@/api/ai-connections";
import { useLocalAiLogin } from "../ai-connections/useLocalAiLogin";
import type { AiConnectionBinding, AiConnectionLoginIntent } from "@paperclipai/shared";
import { useEffect, useRef, useState } from "react";
import { useEffect, useRef, useState, type ReactNode } from "react";
import { Cable } from "lucide-react";
import { useQuery } from "@tanstack/react-query";
import { motion } from "motion/react";
import {
@@ -18,7 +19,7 @@ import {
OnboardingCardField,
OnboardingLoginCard,
} from "../AdapterLoginChrome";
import { ModelSourceTiles } from "../onboarding/ModelSourceTiles";
import { ModelSourceTiles, type ModelConnectionMode } from "../onboarding/ModelSourceTiles";
import { CredentialModeLink } from "../onboarding/CredentialModeLink";
import { FooterNav } from "../onboarding/FooterNav";
import { MAKE_ROOM, CARD_ENTER } from "../onboarding/onboarding-motion";
@@ -40,10 +41,12 @@ export function AgentProviderConnection({
canLogin,
localEnvironment = false,
onConnected,
onDraftChanged,
onBack,
testConnection,
testError,
managedAccount,
advancedConnection,
}: {
companyId: string;
adapterType: "claude_local" | "codex_local" | "grok_local";
@@ -51,9 +54,11 @@ export function AgentProviderConnection({
canLogin: boolean;
localEnvironment?: boolean;
onConnected: (connection: ProviderConnection) => void;
onDraftChanged?: () => void;
onBack: () => void;
testConnection: (connection: ProviderConnection) => Promise<boolean>;
testError?: string | null;
advancedConnection?: { content: ReactNode; value?: AiConnectionBinding };
/** Connections supplies its access intent; presentation and login controllers stay shared. */
managedAccount?: {
intent: AiConnectionLoginIntent;
@@ -81,7 +86,8 @@ export function AgentProviderConnection({
setLoginPhase("preparing");
};
const [methodChoice, setMethod] = useState<"subscription" | "api" | null>(managedAccount?.initialMethod === "api_key" ? "api" : managedAccount ? "subscription" : null);
const [opened, setOpened] = useState(false);
const [advanced, setAdvanced] = useState(false);
const [opened, setOpened] = useState(Boolean(advancedConnection));
const [authorizationUrl, setAuthorizationUrl] = useState<string | null>(null);
const [loginPhase, setLoginPhase] = useState<"preparing" | "ready" | "waiting" | "connecting">("preparing");
const phaseBeforeSubmit = useRef<"ready" | "waiting">("ready");
@@ -136,8 +142,8 @@ export function AgentProviderConnection({
const localLogin = useLocalAiLogin(companyId, managedIntent ?? {
provider: aiProvider, method: "subscription", name: `My ${provider} subscription`,
ownership: "personal", agentIds: [], allAgents: true,
}, canUseLocalLogin && method === "subscription" && !savedSubscription && !storedLogin.data,
{ allowHostClaude: health.data?.deploymentMode === "local_trusted" });
}, !advanced && canUseLocalLogin && method === "subscription" && !savedSubscription && !storedLogin.data,
);
const auth = useQuery({
queryKey: queryKeys.agents.authSignal(
companyId,
@@ -151,7 +157,7 @@ export function AgentProviderConnection({
environmentId ?? undefined,
),
retry: false,
enabled: !managedAccount,
enabled: !managedAccount && !advanced,
});
async function connect() {
if (busy || managedAccount?.disabled) return;
@@ -229,38 +235,58 @@ export function AgentProviderConnection({
!savedKeys.loading &&
!storedLogin.data &&
(Boolean(managedAccount) || auth.data?.status !== "present" || subscriptionId === "");
const switchMode = (next: ModelConnectionMode) => {
if (advancedConnection && next === (advanced ? "advanced" : method)) return;
onDraftChanged?.();
cancel();
setAdvanced(next === "advanced");
setOpened(advancedConnection ? true : opened);
savedManagedAccount.current = null;
if (next !== "advanced") setMethod(next);
setApiKey("");
setStoredConnection(null);
setError(null);
};
return (
<div className="min-w-0 max-w-full">
<ModelSourceTiles
label="Connect your model provider"
sources={[
label={advancedConnection ? "Connection type" : "Connect your model provider"}
sources={advancedConnection ? [
{ id: "subscription", label: provider, icon: <AdapterMark type={adapterType} />, credentialMode: "subscription" },
{ id: "api", label: provider, icon: <AdapterMark type={adapterType} />, credentialMode: "api" },
{ id: "advanced", label: "Advanced", icon: <Cable className="size-6 text-muted-foreground" />, credentialMode: "advanced" },
] : [
{
id: adapterType,
label: provider,
icon: <AdapterMark type={adapterType} />,
},
]}
mode={method}
selectedId={opened ? adapterType : null}
collapsed={opened}
onSelect={() => { if (!managedAccount?.disabled) setOpened(true); }}
mode={advanced ? "advanced" : method}
selectedId={advancedConnection ? (advanced ? "advanced" : method) : opened ? adapterType : null}
collapsed={!advancedConnection && opened}
onSelect={id => {
if (managedAccount?.disabled) return;
if (advancedConnection) switchMode(id as ModelConnectionMode);
else setOpened(true);
}}
/>
{(!opened || managedAccount) && !managedAccount?.fixedMethod && (
{!advancedConnection && (!opened || managedAccount) && !managedAccount?.fixedMethod && (
<div className="-ml-3 mt-1">
<CredentialModeLink
mode={method}
onChange={(next) => {
cancel();
setOpened(opened);
savedManagedAccount.current = null;
setMethod(next);
setApiKey("");
setStoredConnection(null);
setError(null);
}}
/>
<CredentialModeLink mode={method} onChange={switchMode} />
</div>
)}
{advanced && advancedConnection ? <>
<div className="pt-5">{advancedConnection.content}</div>
<FooterNav
onBack={onBack}
primaryLabel="Use connection"
primaryDisabled={!advancedConnection.value}
onPrimary={() => {
if (advancedConnection.value) onConnected({ env: {}, aiConnection: advancedConnection.value });
}}
/>
</> : <>
{!opened && savedKeys.options.length > 0 && (
<p className="mt-2 text-sm text-muted-foreground">
{savedKeys.options.length} saved API{" "}
@@ -269,15 +295,17 @@ export function AgentProviderConnection({
)}
{method === "subscription" &&
savedKeys.subscriptions.length > 0 && (
<SavedProviderKeySelect
options={savedKeys.subscriptions}
value={savedSubscription?.id ?? ""}
onChange={setSubscriptionId}
loading={false}
error={false}
kind="subscription"
disabled={busy}
/>
<div className={advancedConnection ? "pt-5" : undefined}>
<SavedProviderKeySelect
options={savedKeys.subscriptions}
value={savedSubscription?.id ?? ""}
onChange={(id) => { onDraftChanged?.(); setSubscriptionId(id); }}
loading={false}
error={false}
kind="subscription"
disabled={busy}
/>
</div>
)}
<motion.div
initial={false}
@@ -300,6 +328,7 @@ export function AgentProviderConnection({
value={selectedKey?.id ?? ""}
disabled={busy}
onChange={(id) => {
onDraftChanged?.();
setSelectedKeyId(id);
setApiKey("");
setStoredConnection(null);
@@ -318,6 +347,7 @@ export function AgentProviderConnection({
: "Enter API key here"
}
onChange={(value) => {
onDraftChanged?.();
setSelectedKeyId("");
setApiKey(value);
setStoredConnection(null);
@@ -395,7 +425,7 @@ export function AgentProviderConnection({
)}
<FooterNav
onBack={() => {
if (opened) cancel();
if (opened && !advancedConnection) cancel();
else onBack();
}}
primaryLabel={
@@ -416,6 +446,7 @@ export function AgentProviderConnection({
managedAccount?.disabled ||
(Boolean(managedAccount) && method === "subscription" && !canLogin && !canUseLocalLogin) ||
(localEnvironment && health.isPending) || localLogin.preparing || Boolean(localLogin.error) ||
(method === "subscription" && canUseLocalLogin && !savedSubscription && !storedLogin.data && localLogin.status !== "ready") ||
(!managedAccount && auth.isPending) ||
savedKeys.loading ||
(adapterType === "claude_local" && storedLogin.isPending) ||
@@ -438,6 +469,7 @@ export function AgentProviderConnection({
} else void connect();
}}
/>
</>}
</div>
);
}
ui/src/components/new-agent/NewAgentSetup.tsx
+67 -62
View File
@@ -1,3 +1,4 @@
import { useConnectionModels } from "../ai-connections/useConnectionModels";
import { AgentCharacter } from "../AgentCharacter";
import { useAgentAppearanceDraft } from "../../hooks/useAgentAppearanceDraft";
import { AiConnectionField, aiProviderForAdapter } from "../ai-connections/AiConnectionField";
@@ -53,6 +54,7 @@ import { Field } from "../agent-config-primitives";
import { SecretPicker } from "../environment-variables-editor/SecretPicker";
import { Button } from "../ui/button";
import { Input } from "../ui/input";
import { NativeSelect, nativeSelectClassName } from "../ui/select";
import {
OnboardingCard,
OnboardingHeading,
@@ -66,8 +68,7 @@ import {
} from "./AgentProviderConnection";
import { adapterCuratesModelOrder } from "../../lib/model-utils";
const controlClass =
"w-full rounded-md border border-border bg-background px-2.5 py-1.5 text-sm leading-5 outline-none focus-visible:ring-2 focus-visible:ring-ring";
const controlClass = nativeSelectClassName;
const blocking = (result: AdapterEnvironmentTestResult) =>
result.status === "fail" ||
result.checks.some((check) => check.code === ADAPTER_AUTH_MISSING_CHECK_CODE);
@@ -155,6 +156,7 @@ function Setup({
);
const [connection, setConnection] = useState<ProviderConnection | null>(null);
const aiBinding = runtimeAiBinding ?? connection?.aiConnection;
const connectionModels = useConnectionModels(companyId, aiBinding, brandType);
const [repository, setRepository] = useState("");
const [branch, setBranch] = useState("");
const [createdInSession, setCreated] = useState<Agent | null>(null);
@@ -187,6 +189,12 @@ function Setup({
setTestState("idle");
setError(null);
};
const editConnection = () => {
setConnection(null);
setRuntimeAiBinding(undefined);
resetTest();
setScreen("connect");
};
const adapters = useQuery({
queryKey: queryKeys.adapters.all,
queryFn: adaptersApi.list,
@@ -218,7 +226,7 @@ function Setup({
const models = useQuery({
queryKey: queryKeys.agents.adapterModels(companyId, brandType, null, aiBinding?.provider),
queryFn: () => agentsApi.adapterModels(companyId, brandType, { provider: aiBinding?.provider }),
enabled: Boolean(brandType) && showModel,
enabled: Boolean(brandType) && showModel && !connectionModels,
retry: false,
});
const companySecrets = useQuery({
@@ -682,9 +690,9 @@ function Setup({
type="button"
aria-current={step === screen ? "step" : undefined}
disabled={
busy || Boolean(created) || index > steps.indexOf(screen)
busy || Boolean(created) || step === "saved"
}
onClick={() => setScreen(step)}
onClick={() => step === "connect" && screen !== "connect" ? editConnection() : setScreen(step)}
className={cn(
"flex w-full items-center gap-3 rounded-md px-3 py-3 text-left text-sm",
step === screen
@@ -716,38 +724,42 @@ function Setup({
<div className="mb-8">
<OnboardingHeading
title="Connect a model"
lede={`Connect ${name} to ${connectionAdapter === "claude_local" ? "Claude" : connectionAdapter === "grok_local" ? "Grok" : "OpenAI"}.`}
center
/>
</div>
<div className="mb-5">
<Field label="Environment">
<select
aria-label="Environment"
className={controlClass}
value={environmentOverride}
disabled={busy || forced.forced}
onChange={(event) => {
setEnvironmentOverride(event.target.value);
setConnection(null);
resetTest();
}}
>
<option value="">Default: {environmentLabel}</option>
{(envs.data ?? []).filter((env) => env.status === "active" && (!managedOnly || env.driver !== "local")).map((env) => (
<option key={env.id} value={env.id}>{environmentDisplayLabel(env)}</option>
))}
</select>
</Field>
</div>
<AgentProviderConnection
{ready ? <AgentProviderConnection
key={environmentId ?? "local"}
companyId={companyId}
adapterType={connectionAdapter}
environmentId={environmentId}
canLogin={canLogin}
localEnvironment={environment?.driver === "local"}
advancedConnection={{
value: aiBinding,
content: (
<AiConnectionField
companyId={companyId}
agentName={name}
adapterType={brandType}
model={model}
environmentId={environmentId ?? undefined}
value={aiBinding}
preferAdvanced
onChange={binding => {
if (binding.mode === "router") return;
setConnection(null);
setRuntimeAiBinding(binding);
resetTest();
}}
/>
),
}}
onBack={() => navigate("/agents/all")}
onDraftChanged={() => {
setConnection(null);
setRuntimeAiBinding(undefined);
resetTest();
}}
testConnection={runTest}
testError={
error ??
@@ -764,10 +776,11 @@ function Setup({
}
onConnected={(next) => {
setConnection(next);
setRuntimeAiBinding(undefined);
resetTest();
setScreen("runtime");
}}
/>
/> : <p role="status" className="text-sm text-muted-foreground">Loading connection settings…</p>}
</OnboardingCard>
) : screen === "saved" && created ? (
<div className="space-y-6">
@@ -833,32 +846,24 @@ function Setup({
</h2>
<fieldset disabled={busy} className="space-y-8">
<section className="space-y-5">
<h3 className="text-sm font-semibold">Runtime</h3>
{aiProviderForAdapter(brandType) && (
connection && !aiBinding ? (
<div className="space-y-3">
<p className="text-sm text-muted-foreground">
Using the connection selected in the Connect step.
</p>
<Button type="button" variant="outline" onClick={() => setScreen("connect")}>
Change connection
</Button>
</div>
) : (
<AiConnectionField companyId={companyId} agentName={name} adapterType={brandType} model={model} environmentId={environmentId ?? undefined} value={aiBinding}
onChange={binding => { binding.mode !== "router" && setRuntimeAiBinding(binding); resetTest(); }} />
)
{!connectionAdapter && aiProviderForAdapter(brandType) && (
<AiConnectionField companyId={companyId} agentName={name} adapterType={brandType} model={model} environmentId={environmentId ?? undefined} value={aiBinding}
onChange={binding => { binding.mode !== "router" && setRuntimeAiBinding(binding); resetTest(); }} />
)}
{models.error && <p role="alert" className="text-sm text-destructive">Could not load models. Retry or enter a model ID manually.</p>}
{(connectionModels ? connectionModels.error : models.error) && <p role="alert" className="text-sm text-destructive">Could not load models. Retry or enter a model ID manually.</p>}
{((showModel && !usingKimiApi) ||
efforts.length > 0) && (
<div className="grid items-start gap-5 sm:grid-cols-2">
{showModel && !usingKimiApi && (
<ModelDropdown
models={models.data ?? []}
presentation="native"
models={connectionModels?.models ?? models.data ?? []}
loadingModels={connectionModels?.isLoading ?? models.isLoading}
onRefreshModels={connectionModels?.refreshModels}
refreshingModels={connectionModels?.refreshing}
value={model}
onChange={(value) => {
setModel(value);
setModel(connectionModels?.resolveModel(value) ?? value);
if (
effort &&
!setupEfforts(adapterType, value).includes(
@@ -883,15 +888,14 @@ function Setup({
allowDefault={!multiProvider}
required={multiProvider}
creatable
groupByProvider={multiProvider}
preserveOrder={adapterCuratesModelOrder(brandType)}
groupByProvider={multiProvider && !connectionModels}
preserveOrder={Boolean(connectionModels) || adapterCuratesModelOrder(brandType)}
/>
)}
{efforts.length > 0 && (
<Field label="Thinking effort">
<select
<NativeSelect
aria-label="Thinking effort"
className={controlClass}
value={effort}
onChange={(event) => {
setEffort(event.target.value);
@@ -904,22 +908,16 @@ function Setup({
{value}
</option>
))}
</select>
</NativeSelect>
</Field>
)}
</div>
)}
{SETUP_LOGIN_HINTS[adapterType] && (
{!aiBinding && SETUP_LOGIN_HINTS[adapterType] && (
<p className="text-sm text-muted-foreground">
{SETUP_LOGIN_HINTS[adapterType]}
</p>
)}
{showModel && models.error && (
<p className="text-xs text-muted-foreground">
Couldn’t load models. You can enter a model ID
manually.
</p>
)}
{hasCredentialField && !aiBinding && (
<div className="grid gap-5 sm:grid-cols-2">
{chooseProvider && (
@@ -1156,7 +1154,7 @@ function Setup({
state={testState}
result={result}
error={error}
disabled={!ready || busy}
disabled={!ready || busy || Boolean(connectionAdapter && !connection)}
onTest={() => void runTest()}
/>
{error && testState !== "fail" && (
@@ -1170,7 +1168,7 @@ function Setup({
type="button"
variant="ghost"
disabled={busy}
onClick={() => setScreen("connect")}
onClick={editConnection}
>
<ArrowLeft className="size-4" />
Connection
@@ -1178,7 +1176,14 @@ function Setup({
) : (
<span />
)}
<Button
{connectionAdapter && !connection ? <Button
type="button"
disabled={!ready || busy}
onClick={() => setScreen("connect")}
>
Connect model
<ArrowRight className="size-4" />
</Button> : <Button
type="submit"
disabled={
!ready ||
@@ -1189,7 +1194,7 @@ function Setup({
>
{saving ? "Creating…" : "Finish setup"}
<Check className="size-4" />
</Button>
</Button>}
</div>
</form>
)}
ui/src/components/onboarding/CredentialModeLink.tsx
+16 -6
View File
@@ -18,9 +18,10 @@ import { LINK_LABEL_FADE_IN, LINK_LABEL_FADE_OUT } from "./onboarding-motion";
*/
const LINK_LABEL: Record<CredentialMode, string> = {
subscription: "Use API key instead",
api: "Use subscription instead",
subscription: "Use subscription instead",
api: "Use API key instead",
};
const NATIVE_MODES: CredentialMode[] = ["subscription", "api"];
const OTHER_MODE: Record<CredentialMode, CredentialMode> = {
subscription: "api",
@@ -33,11 +34,20 @@ export function CredentialModeLink({
}: {
mode: CredentialMode;
onChange: (next: CredentialMode) => void;
}) {
return <ModeLink destination={OTHER_MODE[mode]} onClick={() => onChange(OTHER_MODE[mode])} modes={NATIVE_MODES} />;
}
function ModeLink({ destination, onClick, modes }: {
destination: CredentialMode;
onClick: () => void;
modes: CredentialMode[];
}) {
return (
<button
type="button"
onClick={() => onChange(OTHER_MODE[mode])}
aria-label={LINK_LABEL[destination]}
onClick={onClick}
className={cn(
// A grid rather than a flow of text, so both labels can occupy one cell
// and overlap during the swap. Same padding as the checkbox row this
@@ -53,7 +63,7 @@ export function CredentialModeLink({
also takes them out of the accessibility tree, leaving the button's name
to the one real label below.
*/}
{(Object.keys(LINK_LABEL) as CredentialMode[]).map((sizerMode) => (
{modes.map((sizerMode) => (
<span
key={sizerMode}
aria-hidden
@@ -65,7 +75,7 @@ export function CredentialModeLink({
<AnimatePresence initial={false} mode="sync">
<motion.span
key={mode}
key={destination}
// Left-aligned in that max-width cell, so the sentence starts at the
// same x in both states and only its tail changes.
className={cn(
@@ -81,7 +91,7 @@ export function CredentialModeLink({
animate={{ opacity: 1, transition: LINK_LABEL_FADE_IN }}
exit={{ opacity: 0, transition: LINK_LABEL_FADE_OUT }}
>
{LINK_LABEL[mode]}
{LINK_LABEL[destination]}
</motion.span>
</AnimatePresence>
</button>
ui/src/components/onboarding/ModelSourceTiles.tsx
+11 -7
View File
@@ -21,17 +21,21 @@ import {
/** How a source gets authenticated. Every tile is in the same mode at once. */
export type CredentialMode = "subscription" | "api";
export type ModelConnectionMode = CredentialMode | "advanced";
export type ModelSource = {
id: string;
label: string;
/** The brand mark, rendered into a 30px square. */
icon: ReactNode;
/** Override the row's credential mode; null leaves the tag slot empty. */
credentialMode?: ModelConnectionMode | null;
};
const CREDENTIAL_TAG_LABEL: Record<CredentialMode, string> = {
const CREDENTIAL_TAG_LABEL: Record<ModelConnectionMode, string> = {
subscription: "Subscription",
api: "API",
api: "API key",
advanced: "Custom Gateway",
};
/**
@@ -43,7 +47,7 @@ const CREDENTIAL_TAG_LABEL: Record<CredentialMode, string> = {
* what makes the outgoing label fall out of frame rather than slide past the
* tile's padding and over the row below.
*/
export function CredentialTag({ mode }: { mode: CredentialMode }) {
export function CredentialTag({ mode }: { mode: ModelConnectionMode }) {
return (
<span className="relative flex h-4 w-full items-center justify-center overflow-hidden text-(length:--text-micro) text-muted-foreground">
<AnimatePresence initial={false} mode="sync">
@@ -70,7 +74,7 @@ function ModelSourceTile({
settling,
}: {
source: ModelSource;
mode: CredentialMode;
mode: ModelConnectionMode | null;
selected: boolean;
onSelect: () => void;
buttonRef: (node: HTMLButtonElement | null) => void;
@@ -123,7 +127,7 @@ function ModelSourceTile({
<span className="text-(length:--text-compact) font-medium text-foreground">
{source.label}
</span>
<CredentialTag mode={mode} />
{mode ? <CredentialTag mode={mode} /> : <span aria-hidden className="h-4" />}
</button>
);
}
@@ -138,7 +142,7 @@ export function ModelSourceTiles({
settling = false,
}: {
sources: ModelSource[];
mode: CredentialMode;
mode: ModelConnectionMode;
/** `null` before anything has been picked — the step opens this way. */
selectedId: string | null;
onSelect: (id: string) => void;
@@ -220,7 +224,7 @@ export function ModelSourceTiles({
>
<ModelSourceTile
source={source}
mode={mode}
mode={source.credentialMode === undefined ? mode : source.credentialMode}
selected={source.id === selectedId}
onSelect={() => onSelect(source.id)}
settling={settling}
ui/src/pages/NewAgent.test.tsx
+131 -13
View File
@@ -40,7 +40,7 @@ const state = vi.hoisted(() => ({
openNewIssue: vi.fn(),
}));
const managedApi = vi.hoisted(() => ({
list: vi.fn(async () => ({ currentUserId: "user-1", connections: [] })),
list: vi.fn(async () => ({ currentUserId: "user-1", canManageConnections: true, connections: [] })),
create: vi.fn(async () => ({ connectionId: "managed-connection", grantId: "managed-grant" })),
setDefault: vi.fn(async () => ({})),
}));
@@ -157,6 +157,7 @@ const pass = {
};
beforeEach(() => {
vi.clearAllMocks();
managedApi.list.mockResolvedValue({ currentUserId: "user-1", canManageConnections: true, connections: [] });
container = document.createElement("div");
document.body.append(container);
root = createRoot(container);
@@ -199,7 +200,49 @@ afterEach(async () => {
container.remove();
});
describe("New agent setup", () => {
it.each([false, true])("selects the Grok sandbox before connecting (managed-only=%s)", async (managedOnly) => {
it.each(["claude_local", "codex_local"])("can choose a sign-in environment in Configure before connecting %s", async (adapterType) => {
settings.get.mockResolvedValue({ defaultEnvironmentId: "no-login" });
envApi.list.mockResolvedValue([
{ id: "no-login", name: "No browser sign-in", status: "active", driver: "sandbox", config: { provider: "other" } },
{ id: "login-env", name: "Browser sign-in", status: "active", driver: "sandbox", config: { provider: "daytona" } },
]);
envApi.capabilities.mockResolvedValue({ sandboxProviders: { daytona: { supportsLoginPty: true } } });
api.getAdapterAuthSignal.mockResolvedValue({ status: "missing" });
await render(adapterType);
expect(container.textContent).toContain("does not support browser sign-in");
expect(container.querySelector('select[aria-label="Environment"]')).toBeNull();
await click("2Configure");
expect(container.querySelector('button[type="submit"]')).toBeNull();
expect([...container.querySelectorAll("button")].find(button => button.textContent?.trim() === "Run test")?.disabled).toBe(true);
await click("Connect model");
expect(api.testEnvironment).not.toHaveBeenCalled();
expect(api.hire).not.toHaveBeenCalled();
await click("2Configure");
const select = container.querySelector('select[aria-label="Environment"]') as HTMLSelectElement;
await act(async () => {
select.value = "login-env";
select.dispatchEvent(new Event("change", { bubbles: true }));
});
await settle();
expect(container.querySelector('select[aria-label="Environment"]')).toBeNull();
await click("Complete subscription login");
await click("Finish setup");
expect(api.hire.mock.calls[0][1]).toMatchObject({ defaultEnvironmentId: "login-env", runtimeConfig: { aiConnection: { method: "subscription" } } });
});
it("waits for the resolved environment before allowing a credential-mode selection", async () => {
let resolveEnvironments!: (value: object[]) => void;
envApi.list.mockImplementationOnce(() => new Promise<object[]>(resolve => { resolveEnvironments = resolve; }));
await render("codex_local");
expect(container.textContent).toContain("Loading connection settings");
expect(container.querySelector('[role="radiogroup"][aria-label="Connection type"]')).toBeNull();
await act(async () => resolveEnvironments([{ id: "local-1", name: "Local", driver: "local", config: {} }]));
await settle();
await click("OpenAIAPI key");
expect(container.querySelector('[aria-label="API key"]')).not.toBeNull();
await settle();
expect(container.querySelector('[aria-label="API key"]')).not.toBeNull();
});
it.each([false, true])("selects the Grok environment in Configure (managed-only=%s)", async (managedOnly) => {
settings.getExperimental.mockResolvedValue({ enableNativeRunner: true, enableManagedSandboxOnly: managedOnly });
envApi.list.mockResolvedValue([
{ id: "local-1", name: "Local", status: "active", driver: "local", config: {} },
@@ -210,6 +253,9 @@ describe("New agent setup", () => {
secret: { companyId: "company-1", status: "active" },
}]);
await render("paperclip_runner", "grok");
expect(container.querySelector('select[aria-label="Environment"]')).toBeNull();
await click("GrokAPI key");
await click("Use saved API key");
const select = container.querySelector('select[aria-label="Environment"]') as HTMLSelectElement;
expect(select.disabled).toBe(false);
expect([...select.options].some((option) => option.value === "local-1")).toBe(!managedOnly);
@@ -218,8 +264,10 @@ describe("New agent setup", () => {
select.dispatchEvent(new Event("change", { bubbles: true }));
});
await settle();
await click("GrokAPI");
await click("Use saved API key");
if (container.querySelector('[role="radiogroup"][aria-label="Connection type"]')) {
await click("GrokAPI key");
await click("Use saved API key");
}
expect(api.testEnvironment).toHaveBeenCalledWith("company-1", "paperclip_runner", expect.objectContaining({ environmentId: "grok-sandbox" }));
await click("Finish setup");
expect(api.hire.mock.calls[0][1].defaultEnvironmentId).toBe("grok-sandbox");
@@ -259,13 +307,13 @@ describe("New agent setup", () => {
api.getAdapterAuthSignal.mockResolvedValue({ status: "missing" });
api.testEnvironment.mockResolvedValue({ ...pass, adapterType });
await render(adapterType, "grok");
expect(container.textContent).toContain("Connect Atlas to Grok");
expect(container.textContent).toContain("Connect a model");
expect(container.textContent).not.toContain("Connect Atlas to Grok");
if (method === "subscription") {
await click("GrokSubscription");
await click("Complete subscription login");
} else {
await click("Use API key insteadUse subscription insteadUse API key instead");
await click("GrokAPI");
await click("GrokAPI key");
await fill("API key", "example-test-secret");
await click("Connect");
expect(managedApi.create).toHaveBeenCalledWith("company-1", expect.objectContaining({
@@ -375,7 +423,8 @@ describe("New agent setup", () => {
});
it("uses the shared Grok connection flow and hides ignored Kimi and OpenCode effort controls", async () => {
await render("grok_local");
expect(container.textContent).toContain("Connect Atlas to Grok");
expect(container.textContent).toContain("Connect a model");
expect(container.textContent).not.toContain("Connect Atlas to Grok");
await render("opencode_local");
expect(container.querySelector('[aria-label="Thinking effort"]')).toBeNull();
});
@@ -462,14 +511,15 @@ describe("New agent setup", () => {
["paperclip_runner", "codex", "OpenAI", "OPENAI_API_KEY"],
])("stores %s %s as a reusable connection before hiring", async (adapter, runner, provider, key) => {
await render(adapter, runner);
await click("Use API key insteadUse subscription insteadUse API key instead");
await click(provider + "API");
await click(provider + "API key");
await fill("API key", "connection-key");
await click("Connect");
const binding = { provider: key === "ANTHROPIC_API_KEY" ? "anthropic" : "openai", method: "api_key", mode: "responsible_user" };
expect(managedApi.create).toHaveBeenCalledWith("company-1", expect.objectContaining({ apiKey: "connection-key", provider: binding.provider }));
expect(api.testEnvironment.mock.calls[0][2].testCredentials).toEqual({});
expect(api.testEnvironment.mock.calls[0][2].aiConnection).toEqual(binding);
expect(container.textContent).toContain("Configure your agent");
expect(container.querySelector('[role="combobox"][aria-label="Connection"]')).toBeNull();
expect(secrets.createUserSecretDefinition).not.toHaveBeenCalled();
await click("Finish setup");
expect(api.hire.mock.calls[0][1].runtimeConfig.aiConnection).toEqual(binding);
@@ -487,7 +537,7 @@ describe("New agent setup", () => {
secret: { companyId: "company-1", status: "active" },
}]);
await render(adapter, runner);
await click(provider + "API");
await click(provider + "API key");
expect((container.querySelector("select[aria-label='Saved API key']") as HTMLSelectElement).value).toBe("user:existing-key");
await click("Use saved API key");
const binding = { type: "user_secret_ref", key, version: "latest" };
@@ -499,6 +549,71 @@ describe("New agent setup", () => {
expect(secrets.createMyUserSecret).not.toHaveBeenCalled();
expect(secrets.rotateMyUserSecret).not.toHaveBeenCalled();
});
it.each(["codex_local", "paperclip_runner"])("keeps a saved connection chosen in Connect when hiring %s", async (adapter) => {
await render(adapter, "codex");
managedApi.list.mockResolvedValue({
currentUserId: "user-1",
connections: [{
id: "00000000-0000-4000-8000-000000000021", grantId: "00000000-0000-4000-8000-000000000022", companyId: "company-1",
provider: "openrouter", method: "api_key", name: "Company OpenRouter",
ownership: "shared", isDefault: false, status: "connected",
routing: { kind: "openrouter", protocol: "responses", auth: "bearer", models: [] },
}],
} as any);
await click("AdvancedCustom Gateway");
const picker = container.querySelector('[role="combobox"][aria-label="Connection"]')!;
await act(async () => picker.dispatchEvent(new KeyboardEvent("keydown", { key: "ArrowDown", bubbles: true })));
await settle();
const option = [...document.querySelectorAll('[role="option"]')].find(node => node.textContent?.includes("Company OpenRouter")) as HTMLElement;
expect(option).toBeTruthy();
await act(async () => option.click());
await settle();
await click("Use connection");
expect(container.textContent).toContain("Configure your agent");
expect(container.querySelector('[role="combobox"][aria-label="Connection"]')).toBeNull();
await click("Run test");
const binding = { provider: "openrouter", method: "api_key", mode: "shared", connectionId: "00000000-0000-4000-8000-000000000021", grantId: "00000000-0000-4000-8000-000000000022" };
expect(api.testEnvironment.mock.calls[0][2].aiConnection).toEqual(binding);
await click("Finish setup");
expect(api.hire.mock.calls[0][1].runtimeConfig.aiConnection).toEqual(binding);
});
it("requires the current connection choice to connect before Configure can save it", async () => {
await render("codex_local");
managedApi.list.mockResolvedValue({ currentUserId: "user-1", connections: [{
id: "00000000-0000-4000-8000-000000000021", grantId: "00000000-0000-4000-8000-000000000022", companyId: "company-1",
provider: "openrouter", method: "api_key", name: "Company OpenRouter", ownership: "shared", isDefault: false, status: "connected",
routing: { kind: "openrouter", protocol: "responses", auth: "bearer", models: [] },
}] } as any);
await click("AdvancedCustom Gateway");
const picker = container.querySelector('[role="combobox"][aria-label="Connection"]')!;
await act(async () => picker.dispatchEvent(new KeyboardEvent("keydown", { key: "ArrowDown", bubbles: true })));
await settle();
const option = [...document.querySelectorAll('[role="option"]')].find(node => node.textContent?.includes("Company OpenRouter")) as HTMLElement;
await act(async () => option.click());
await settle();
await click("2Configure");
expect(container.textContent).not.toContain("Finish setup");
expect(api.hire).not.toHaveBeenCalled();
await click("Connect model");
await click("OpenAIAPI key");
await fill("API key", "new-api-credential");
await click("2Configure");
expect(container.textContent).not.toContain("Finish setup");
expect(api.testEnvironment).not.toHaveBeenCalled();
await click("Connect model");
await click("OpenAIAPI key");
await fill("API key", "new-api-credential");
await click("Connect");
await click("Connection");
await click("2Configure");
expect(container.textContent).not.toContain("Finish setup");
await click("Connect model");
await click("OpenAIAPI key");
await fill("API key", "new-api-credential");
await click("Connect");
await click("Finish setup");
expect(api.hire.mock.calls[0][1].runtimeConfig.aiConnection).toEqual({ provider: "openai", method: "api_key", mode: "responsible_user" });
});
it.each(["pi_local"])(
"persists %s OpenRouter credentials only as a secret reference",
async (adapter) => {
@@ -556,10 +671,13 @@ describe("New agent setup", () => {
await settle();
expect(document.querySelector('[role="dialog"]')).toBeNull();
expect(managedApi.setDefault).not.toHaveBeenCalled();
expect(api.hire).not.toHaveBeenCalled();
expect(api.testEnvironment).not.toHaveBeenCalled();
expect(managedApi.create).toHaveBeenCalledWith("company-1", expect.objectContaining({
provider: "openrouter", method: "api_key", apiKey: "example-test-secret",
}));
const binding = { provider: "openrouter", method: "api_key", mode: "delegated", connectionId: "managed-connection", grantId: "managed-grant" };
const binding = { provider: "openrouter", method: "api_key", mode: "shared", connectionId: "managed-connection", grantId: "managed-grant" };
await click("Run test");
expect(api.testEnvironment.mock.calls[0][2]).toEqual(expect.objectContaining({
aiConnection: binding, testCredentials: {},
@@ -656,7 +774,7 @@ describe("New agent setup", () => {
{ type: "paperclip_runner", loaded: true, disabled: true },
];
await render("paperclip_runner");
await connect("OpenAI");
expect(container.textContent).toContain("This adapter is unavailable");
expect(api.testEnvironment).not.toHaveBeenCalled();
expect(api.hire).not.toHaveBeenCalled();
});
ui/src/pages/apps/Browse.test.tsx
+23
View File
@@ -251,6 +251,29 @@ describe("Connectors landing page", () => {
return client;
}
it("shows provider catalog rows and splits legacy gateway accounts by API format", async () => {
const slugs = ["notion", "openrouter", "bedrock", "google", "responses-api", "messages-api", "chat-completions-api", "local"];
listGalleryMock.mockResolvedValue({ apps: slugs.map(slug => getAppStoreDefinition(slug)) });
listApplicationsMock.mockResolvedValue({ applications: [application({ id: "gateway-app", name: "Model gateway", applicationKey: "app-gallery:gateway", metadata: { sourceTemplateKey: "gateway" } })] });
listConnectionsMock.mockResolvedValue({ connections: ["responses", "messages"].map(protocol => connection({
id: protocol, name: `${protocol} account`, applicationId: "gateway-app", connectionPurpose: "ai",
config: { sourceTemplateKey: "gateway", ai: { provider: protocol === "messages" ? "anthropic" : "openai", method: "api_key", routing: { kind: "gateway", protocol, auth: "bearer", baseUrl: "https://models.example.com/v1", models: [] } } },
})) });
await renderBrowse();
expect(container.textContent).not.toContain("Connect a model provider");
expect(container.textContent).not.toContain("Model gateway");
for (const slug of slugs) expect(container.querySelector(`[data-app-slug="${slug}"]`)).not.toBeNull();
const responses = container.querySelector('[data-app-slug="responses-api"]')!;
const messages = container.querySelector('[data-app-slug="messages-api"]')!;
expect(responses.textContent).toContain("responses account");
expect(responses.textContent).not.toContain("messages account");
expect(messages.textContent).toContain("messages account");
const bedrock = container.querySelector('[data-app-slug="bedrock"]')!;
const connect = Array.from(bedrock.querySelectorAll('button')).find(button => button.textContent?.includes("Connect"))!;
await act(() => connect.click());
expect(navigateMock).toHaveBeenCalledWith("/apps/connect?source=bedrock");
});
it("offers assistant setup from Connections without choosing an agent", async () => {
await renderBrowse();
const button = container.querySelector<HTMLButtonElement>('[aria-label="Set up Assistant Connection (MCP)"]');
ui/src/pages/apps/Browse.tsx
+18 -4
View File
@@ -292,11 +292,11 @@ function connectorAction(
};
}
if (chatHref) return { label: "Connect", href: chatHref };
// PAP-659 C4: the card's verb comes from the same four-state resolver the
// connect screen uses, so "Connect" never turns out to mean "paste a key".
// AI accounts share the Connect action across subscription and key methods.
// Tool connectors retain their capability-specific setup verbs.
if (row.entry) {
return {
label: connectionSetupVerbForApp(row.entry),
label: row.entry.methods?.some(method => method.purpose === "ai") ? "Connect" : connectionSetupVerbForApp(row.entry),
href: connectHrefFor(row.entry),
};
}
@@ -601,7 +601,8 @@ export function Browse({ renderAccountDetails = (connection) => connection.conne
const applicationSlug = appApplicationSourceSlug(application);
const savedAppConnections =
connectionsByApplicationId.get(application.id) ?? [];
const appConnections = savedAppConnections.filter(
if (applicationSlug === "gateway" && savedAppConnections.length === 0) continue;
let appConnections = savedAppConnections.filter(
(connection) => !GOOGLE_CONNECTOR_SLUGS.has(appConnectionSourceSlug(connection) ?? ""),
);
// Hide source-only Google rows, but keep independently identified connectors.
@@ -610,6 +611,19 @@ export function Browse({ renderAccountDetails = (connection) => connection.conne
savedAppConnections.length > 0 &&
appConnections.length === 0
) continue;
// One legacy gateway application may contain several API formats. Group
// each saved AI account by its actual routing, not the old application slug.
const ungroupedCount = appConnections.length;
appConnections = appConnections.filter((connection) => {
if (connection.connectionPurpose !== "ai") return true;
const slug = appConnectionSourceSlug(connection);
const row = slug ? rowsBySlug.get(slug) : undefined;
if (!row) return true;
if (!row.applications.some((item) => item.id === application.id)) row.applications.push(application);
row.connections.push(connection);
return false;
});
if (ungroupedCount > 0 && appConnections.length === 0) continue;
const configuredConnectionSlug = appConnections
.map(
(connection) =>
ui/src/pages/apps/app-connect-policy.test.ts
+4
View File
@@ -32,6 +32,10 @@ describe("app connect policy", () => {
expect(canEnterAppsConnect(new URLSearchParams("source=context7"))).toBe(false);
expect(canEnterAppsConnect(new URLSearchParams("source=zapier"))).toBe(true);
expect(canEnterAppsConnect(new URLSearchParams("source=unknown"))).toBe(false);
expect(canEnterAppsConnect(new URLSearchParams("source=model-provider"))).toBe(false);
for (const source of ["responses-api", "messages-api", "chat-completions-api", "bedrock", "local", "google", "openrouter"]) {
expect(canEnterAppsConnect(new URLSearchParams({ source })), source).toBe(true);
}
expect(canEnterAppsConnect(new URLSearchParams("byo=1"))).toBe(false);
expect(canEnterAppsConnect(new URLSearchParams("byo=1&source=zapier"))).toBe(false);
});
ui/src/pages/apps/app-connect-policy.ts
+1
View File
@@ -67,6 +67,7 @@ export function canEnterAppsConnect(
);
}
const source = searchParams.get("source") ?? "";
if (["google-ai", "gateway"].includes(source)) return true;
// Router connectors are registered by installed plugins, not the static
// catalog. AppsConnect resolves this reserved slug against the host gallery
// before rendering the pool wizard; unknown/uninstalled routers show its
ui/src/pages/apps/app-definition-display.ts
+5 -1
View File
@@ -1,5 +1,5 @@
import { aiConnectionCatalogSlug, aiConnectionMetadataSchema, type AppDefinition, type ToolApplication, type ToolConnection } from "@paperclipai/shared";
import { aiConnectionRouterSlug, aiConnectionRouterPluginKey } from "@paperclipai/shared";
import type { AppDefinition, ToolApplication, ToolConnection } from "@paperclipai/shared";
export type AppGalleryDisplayEntry = AppDefinition & {
key?: string;
@@ -50,6 +50,10 @@ export function appApplicationSourceSlug(application: ToolApplication | null | u
export function appConnectionSourceSlug(connection: ToolConnection | null | undefined): string | null {
if (!connection) return null;
if (connection.connectionPurpose === "ai") {
const metadata = aiConnectionMetadataSchema.safeParse(connection.config?.ai);
if (metadata.success) return aiConnectionCatalogSlug(metadata.data.provider, metadata.data.routing);
}
const router = aiConnectionRouterPluginKey(connection);
if (router) return aiConnectionRouterSlug(router);
const source = connection.config?.sourceTemplateKey ?? connection.transportConfig?.sourceTemplateKey;
ui/storybook/prototypes/agent-settings/AgentSettingsPreview.tsx
+115 -96
View File
@@ -4,6 +4,7 @@ import {
useRef,
useState,
type ComponentProps,
type ReactNode,
} from "react";
import { Route, Routes, useLocation, useNavigate } from "react-router-dom";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
@@ -67,11 +68,14 @@ export function AgentSettingsPreview({
adapterType = "claude_local",
testOutcome = "pass",
saveFails = false,
runtimeContent,
}: {
initialTab?: AgentLocalDetailView;
adapterType?: string;
testOutcome?: TestOutcome;
saveFails?: boolean;
/** Storybook-only extension point for reviewing proposed runtime controls in the existing page. */
runtimeContent?: ReactNode;
}) {
const [fixtures] = useState(() =>
createSettingsFixtures(adapterType, testOutcome, saveFails),
@@ -115,7 +119,7 @@ export function AgentSettingsPreview({
<Routes>
<Route
path="/:companyPrefix/agents/:agentId/:tab?"
element={<SettingsPage />}
element={<SettingsPage runtimeContent={runtimeContent} />}
/>
<Route
path="*"
@@ -136,7 +140,7 @@ export function AgentSettingsPreview({
);
}
function SettingsPage() {
function SettingsPage({ runtimeContent }: { runtimeContent?: ReactNode }) {
const location = useLocation();
const view = parseAgentDetailView(
location.pathname.split("/").pop() ?? "overview",
@@ -246,7 +250,7 @@ function SettingsPage() {
</div>
<div className="flex min-h-screen">
<div
className="w-52 shrink-0"
className="hidden w-52 shrink-0 md:block"
onClickCapture={(event) => {
if (
dirty &&
@@ -267,6 +271,16 @@ function SettingsPage() {
</div>
<main className="min-w-0 flex-1 px-6 py-8 lg:px-10">
<div className="mx-auto max-w-5xl space-y-8">
<details className="md:hidden">
<summary className="cursor-pointer text-sm">
{agent.name} · Agent navigation
</summary>
<AgentContextualSidebar
agentRef={REF}
agentId={ID}
agentName={agent.name}
/>
</details>
<header className="flex flex-wrap items-center justify-between gap-5 border-b border-border pb-6">
<div className="flex min-w-0 items-center gap-4">
<div
@@ -349,72 +363,76 @@ function SettingsPage() {
{view === "skills" && (
<AgentSkillsTab agent={agent} companyId={COMPANY} />
)}
{(view === "runtime" || view === "secrets") && (
<div className="space-y-6">
<div className="agent-settings-form">
<AgentConfigForm
mode="edit"
agent={agent}
onSave={saveAgent}
isSaving={saving}
hideInlineSave
hidePromptTemplate
hideInstructionsFile
content={view === "runtime" ? "configuration" : "secrets"}
environmentVariablesPlacement="secrets"
sectionLayout="cards"
canConfigureProviderTrace
sectionOrder={[
"adapter",
"permissions",
"environment",
"run-policy",
"identity",
]}
sectionTitles={{
adapter: "Harness",
permissions: "Model & execution",
identity: "Agent identity",
}}
onDirtyChange={onDirty}
onSaveActionChange={onSaveAction}
onCancelActionChange={onCancelAction}
onTestActionChange={onTestAction}
onTestActionStateChange={setTestAction}
onTestFeedbackChange={setFeedback}
/>
</div>
{view === "runtime" && (
<>
<RuntimeTestCard
state={
testAction.pending
? "running"
: feedback.errorMessage ||
feedback.result?.status === "fail"
? "fail"
: feedback.result?.status === "warn"
? "warn"
: feedback.result
? "pass"
: "idle"
}
result={feedback.result}
error={feedback.errorMessage}
disabled={testAction.disabled}
onTest={() => test.current?.()}
/>
{feedback.login && (
<AdapterLoginPanel
{...feedback.login}
onStored={() => test.current?.()}
onApplyStored={() => test.current?.()}
{view === "runtime" && runtimeContent
? runtimeContent
: (view === "runtime" || view === "secrets") && (
<div className="space-y-6">
<div className="agent-settings-form">
<AgentConfigForm
mode="edit"
agent={agent}
onSave={saveAgent}
isSaving={saving}
hideInlineSave
hidePromptTemplate
hideInstructionsFile
content={
view === "runtime" ? "configuration" : "secrets"
}
environmentVariablesPlacement="secrets"
sectionLayout="cards"
canConfigureProviderTrace
sectionOrder={[
"adapter",
"permissions",
"environment",
"run-policy",
"identity",
]}
sectionTitles={{
adapter: "Harness",
permissions: "Model & execution",
identity: "Agent identity",
}}
onDirtyChange={onDirty}
onSaveActionChange={onSaveAction}
onCancelActionChange={onCancelAction}
onTestActionChange={onTestAction}
onTestActionStateChange={setTestAction}
onTestFeedbackChange={setFeedback}
/>
</div>
{view === "runtime" && (
<>
<RuntimeTestCard
state={
testAction.pending
? "running"
: feedback.errorMessage ||
feedback.result?.status === "fail"
? "fail"
: feedback.result?.status === "warn"
? "warn"
: feedback.result
? "pass"
: "idle"
}
result={feedback.result}
error={feedback.errorMessage}
disabled={testAction.disabled}
onTest={() => test.current?.()}
/>
{feedback.login && (
<AdapterLoginPanel
{...feedback.login}
onStored={() => test.current?.()}
onApplyStored={() => test.current?.()}
/>
)}
</>
)}
</>
</div>
)}
</div>
)}
{view === "permissions" && (
<ConfigurationTab
agent={agent}
@@ -441,36 +459,37 @@ function SettingsPage() {
)}
{(["runtime", "secrets", "instructions"] as string[]).includes(
view,
) && (
<footer className="sticky bottom-0 flex flex-wrap items-center justify-between gap-3 border-t border-border bg-background py-4">
<p role="status" className="text-xs text-muted-foreground">
{saving
? "Saving changes…"
: dirty
? "You have unsaved changes."
: saved
? "Changes saved."
: ""}
</p>
<div className="flex gap-2">
<Button
variant="ghost"
disabled={!dirty || saving}
onClick={() => cancel.current?.()}
>
Discard
</Button>
<Button
disabled={!dirty || saving}
onClick={() => {
Promise.resolve(save.current?.()).catch(() => {});
}}
>
{saving ? "Saving…" : "Save changes"}
</Button>
</div>
</footer>
)}
) &&
!(view === "runtime" && runtimeContent) && (
<footer className="sticky bottom-0 flex flex-wrap items-center justify-between gap-3 border-t border-border bg-background py-4">
<p role="status" className="text-xs text-muted-foreground">
{saving
? "Saving changes…"
: dirty
? "You have unsaved changes."
: saved
? "Changes saved."
: ""}
</p>
<div className="flex gap-2">
<Button
variant="ghost"
disabled={!dirty || saving}
onClick={() => cancel.current?.()}
>
Discard
</Button>
<Button
disabled={!dirty || saving}
onClick={() => {
Promise.resolve(save.current?.()).catch(() => {});
}}
>
{saving ? "Saving…" : "Save changes"}
</Button>
</div>
</footer>
)}
</div>
</main>
</div>
ui/storybook/prototypes/provider-routing/AgentSetup.tsx
+573
View File
@@ -0,0 +1,573 @@
import { useEffect, useRef, useState } from "react";
import { Button } from "@/components/ui/button";
import { RuntimeTestCard, type TestState } from "@/components/RuntimeTestCard";
import { ModelSourceTiles } from "@/components/onboarding/ModelSourceTiles";
import { CredentialModeLink } from "@/components/onboarding/CredentialModeLink";
import { AdapterMark } from "@/components/AdapterMark";
import {
AdapterTypeDropdown,
ModelDropdown,
} from "@/components/AgentConfigForm";
import { Field } from "@/components/agent-config-primitives";
import { ConnectionChoiceList } from "@/features/connections/ConnectionChoiceList";
import { AiConnectionLegacyNotice } from "@/components/ai-connections/AiConnectionManagement";
import { AgentSettingsPreview } from "../agent-settings/AgentSettingsPreview";
import { ProviderApiKeyCard } from "@/components/AdapterLoginChrome";
import { ConnectionSetup } from "./ConnectionSetup";
import { ConnectionPicker } from "./ConnectionPicker";
import {
AdvancedOptions,
Choice,
Footer,
Notice,
Surface,
TextField,
} from "./shared";
import {
compatibility,
connections,
defaultConnection,
harnesses,
modelsFor,
type Connection,
type Harness,
} from "./model";
export type AgentProps = {
harness?: Harness;
runner?: "new" | "legacy";
initialConnectionId?: string;
addedConnection?: Connection;
initialModel?: string;
initialTest?: TestState;
scenario?:
| "normal"
| "empty"
| "loading"
| "catalog-error"
| "denied"
| "missing-personal"
| "legacy"
| "readonly";
testFailure?: "tools" | "network";
newAgent?: boolean;
initialAdvanced?: boolean;
};
const adapterTypes: Record<Harness, string> = {
codex: "codex_local",
claude: "claude_local",
opencode: "opencode_local",
grok: "grok_local",
pi: "pi_local",
gemini: "gemini_local",
kimi: "kimi_local",
hermes: "hermes_local",
cursor: "cursor",
copilot: "copilot_local",
};
const excludedAdapters = new Set([
"process",
"http",
"openclaw_gateway",
"hermes_gateway",
"claude_managed",
"aws_agentcore",
"acpx_local",
"paperclip_runner",
"cursor_cloud",
]);
export function AgentSetup(props: AgentProps) {
const content = <AgentRoutingFields {...props} />;
return props.newAgent ? (
content
) : (
<AgentSettingsPreview
initialTab="runtime"
adapterType={adapterTypes[props.harness ?? "codex"]}
runtimeContent={content}
/>
);
}
function AgentRoutingFields({
harness: initialHarness = "codex",
runner = "new",
initialConnectionId,
addedConnection,
initialModel,
initialTest = "idle",
scenario = "normal",
testFailure,
newAgent,
initialAdvanced = false,
}: AgentProps) {
const startingConnection =
addedConnection ??
connections.find((entry) => entry.id === initialConnectionId) ??
defaultConnection(initialHarness);
const [harness, setHarness] = useState(initialHarness);
const [rows, setRows] = useState(
addedConnection
? [
...connections.filter((entry) => entry.id !== addedConnection.id),
addedConnection,
]
: scenario === "empty"
? []
: connections,
);
const [connectionId, setConnectionId] = useState(startingConnection.id);
const [model, setModel] = useState(
initialModel ?? modelsFor(startingConnection, harness)[0],
);
const [manual, setManual] = useState(false);
const [modelOpen, setModelOpen] = useState(false);
const [advancedOpen, setAdvancedOpen] = useState(initialAdvanced);
const [effort, setEffort] = useState("default");
const [environment, setEnvironment] = useState("cloud");
const [state, setState] = useState<TestState>(initialTest);
const [catalogError, setCatalogError] = useState(
scenario === "catalog-error",
);
const [creating, setCreating] = useState(false);
const [saved, setSaved] = useState(false);
const [discarded, setDiscarded] = useState(false);
const timer = useRef<ReturnType<typeof setTimeout> | undefined>(undefined);
useEffect(() => () => clearTimeout(timer.current), []);
const selected = rows.find((entry) => entry.id === connectionId);
const availableModels = selected ? modelsFor(selected, harness) : [];
const unknownModel = !manual && !availableModels.includes(model);
const reason = selected ? compatibility(selected, harness) : undefined;
const denied = scenario === "denied" || scenario === "missing-personal";
const readOnly = scenario === "readonly";
const nativePending =
runner === "new" &&
!harnesses.find((entry) => entry.value === harness)!.native;
const invalid =
!selected ||
!!reason ||
unknownModel ||
!model.trim() ||
denied ||
nativePending ||
readOnly;
const change = () => {
clearTimeout(timer.current);
setState("idle");
setSaved(false);
setDiscarded(false);
};
const error =
testFailure === "network"
? "This endpoint is unreachable from the selected environment. Check network access or choose another environment."
: "The model responded, but the tool-call test failed. Choose a model with tool support or check the gateway’s API format.";
const test = () => {
setState("running");
timer.current = setTimeout(
() => setState(testFailure ? "fail" : "pass"),
400,
);
};
if (creating)
return (
<ConnectionSetup
initialAdvanced={advancedOpen}
onCancel={() => setCreating(false)}
onComplete={(connection) => {
setRows((current) => [
...current.filter((entry) => entry.id !== connection.id),
connection,
]);
setConnectionId(connection.id);
setCreating(false);
change();
}}
/>
);
return (
<Surface
embedded={!newAgent}
title={newAgent ? "Configure Nova" : "Harness, connection & model"}
description="Choose a harness and model, then test your agent."
>
<fieldset
disabled={readOnly}
aria-label="Harness selection"
className="min-w-0"
>
<Field label="Harness">
<AdapterTypeDropdown
value={adapterTypes[harness]}
disabledTypes={excludedAdapters}
onChange={(value) => {
const next = (Object.keys(adapterTypes) as Harness[]).find(
(key) => adapterTypes[key] === value,
);
if (next) {
setHarness(next);
change();
}
}}
/>
</Field>
</fieldset>
{nativePending && (
<Notice error>
This harness is not yet available through Paperclip Runner. Select a
supported harness or configure its legacy adapter.
</Notice>
)}
{scenario === "legacy" && (
<AiConnectionLegacyNotice
readOnly={readOnly}
onAdopt={() => setCreating(true)}
/>
)}
{scenario === "loading" ? (
<p role="status" className="text-sm text-muted-foreground">
Loading your connections…
</p>
) : selected ? (
<>
<ConnectionPicker
selected={selected}
connections={rows.filter((entry) => !compatibility(entry, harness))}
disabled={readOnly}
onSelect={(id) => {
if (id === connectionId) return;
setConnectionId(id);
setManual(false);
change();
}}
onConnect={() => setCreating(true)}
/>
{reason && (
<Notice error>{reason} Choose a compatible connection.</Notice>
)}
{denied && (
<Notice error>
{scenario === "missing-personal"
? "Sam has no credential for Engineering gateway. Sam needs to connect to this destination, or choose an authorized company credential."
: "You don’t have access to this company credential. Ask its owner for access or choose another connection."}
</Notice>
)}
</>
) : (
<Notice>
No compatible connections yet. Connect a provider to choose a model
for Nova.
</Notice>
)}
{!selected && (
<Button
variant="link"
className="h-auto justify-start self-start p-0"
disabled={readOnly || scenario === "loading"}
onClick={() => setCreating(true)}
>
Connect an account
</Button>
)}
{selected && scenario !== "loading" && (
<>
{catalogError ? (
<Notice error>
<p>Couldn’t load models from this connection.</p>
<Button
size="sm"
variant="outline"
className="mt-3"
onClick={() => setCatalogError(false)}
>
Retry model discovery
</Button>
</Notice>
) : (
<fieldset
disabled={readOnly || Boolean(reason)}
aria-label="Model selection"
className="min-w-0"
>
<ModelDropdown
models={availableModels.map((id) => ({ id, label: id }))}
value={model}
open={modelOpen}
onOpenChange={setModelOpen}
allowDefault={false}
required
groupByProvider={selected.provider === "openrouter"}
creatable={advancedOpen}
onChange={(value) => {
setManual(!availableModels.includes(value));
setModel(value);
change();
}}
/>
</fieldset>
)}
{catalogError && advancedOpen && (
<TextField
label="Model ID or alias"
value={model}
disabled={readOnly}
onChange={(event) => {
setManual(true);
setModel(event.target.value);
change();
}}
hint="Use the exact ID accepted by this connection. Custom model metadata is managed on the connection."
/>
)}
{unknownModel && !catalogError && (
<Notice error>
Your previous model is unavailable through {selected.name}. Choose
a model to continue. Your harness is still{" "}
{harnesses.find((entry) => entry.value === harness)!.label}.
</Notice>
)}
<Choice
label="Reasoning"
value={effort}
disabled={readOnly}
onChange={(value) => {
setEffort(value);
change();
}}
options={[
{ value: "default", label: "Model default" },
{ value: "low", label: "Low" },
{ value: "high", label: "High" },
]}
/>
<Choice
label="Execution environment"
value={environment}
disabled={readOnly}
onChange={(value) => {
setEnvironment(value);
change();
}}
options={[
{ value: "cloud", label: "Company AWS workspace" },
{ value: "local", label: "Local machine" },
]}
/>
<AdvancedOptions
label="Advanced model settings"
open={advancedOpen}
onOpenChange={setAdvancedOpen}
>
<Button
variant="link"
className="h-auto self-start p-0"
disabled={readOnly}
onClick={() => setCreating(true)}
>
Connect another provider or gateway
</Button>
<p className="text-xs text-muted-foreground">
To use a custom model ID or gateway alias, enter it in the Model
picker above.
</p>
</AdvancedOptions>
<RuntimeTestCard
state={state}
onTest={test}
disabled={invalid || (catalogError && !manual)}
error={error}
result={
state === "pass" || state === "fail"
? {
adapterType: harness,
status: state,
testedAt: "2026-10-02T12:00:00Z",
checks: [
{
code: "destination",
level: "info",
message: `Destination: ${selected.endpoint}`,
},
{
code: "environment",
level: "info",
message: `Tested in ${environment === "cloud" ? "Company AWS workspace" : "Local machine"}`,
},
{
code: "tools",
level: state === "pass" ? "info" : "error",
message:
state === "pass"
? "Authentication, streaming, tool call, and follow-up passed."
: error,
},
],
}
: null
}
/>
</>
)}
{saved && (
<Notice>
{scenario === "legacy"
? "Managed connection adopted."
: "Nova’s configuration saved."}{" "}
{selected?.name} · {model}
</Notice>
)}
{discarded && <Notice>Changes discarded.</Notice>}
<Footer
back="Discard"
onBack={() => {
setHarness(initialHarness);
setConnectionId(startingConnection.id);
setModel(
initialModel ?? modelsFor(startingConnection, initialHarness)[0],
);
setManual(false);
setEffort("default");
setEnvironment("cloud");
change();
setDiscarded(true);
}}
next={
scenario === "legacy"
? "Adopt connection"
: newAgent
? "Finish setup"
: "Save changes"
}
onNext={() => setSaved(true)}
disabled={invalid || state !== "pass" || scenario === "loading"}
/>
</Surface>
);
}
export function Onboarding({
initialAlternate = false,
savedConnection = false,
}: {
initialAlternate?: boolean;
savedConnection?: boolean;
}) {
const [stage, setStage] = useState(initialAlternate ? "other" : "default");
const [mode, setMode] = useState<"subscription" | "api">("subscription");
const [key, setKey] = useState("");
const [connection, setConnection] = useState<Connection>();
if (stage === "other")
return (
<ConnectionSetup
initialAdvanced
onCancel={() => setStage("default")}
onComplete={(value) => {
setConnection(value);
setStage("agent");
}}
/>
);
if (stage === "agent")
return (
<AgentSetup
harness="codex"
newAgent
initialConnectionId="chatgpt"
addedConnection={connection}
/>
);
return (
<Surface
title="Connect OpenAI"
description="You’ve selected Codex for Nova."
>
<ModelSourceTiles
label="Model provider"
sources={[
{
id: "codex",
label: "OpenAI",
icon: <AdapterMark type="codex_local" />,
},
]}
selectedId="codex"
mode={mode}
onSelect={() => {}}
/>
<CredentialModeLink mode={mode} onChange={setMode} />
{savedConnection ? (
<ConnectionChoiceList
selectedId="chatgpt"
choices={[
{
id: "chatgpt",
name: "My ChatGPT subscription",
description: "OpenAI · Your subscription",
},
]}
onSelect={() => {}}
/>
) : mode === "api" ? (
<ProviderApiKeyCard
providerName="OpenAI"
value={key}
onChange={setKey}
onSubmit={() => {
if (key.trim()) {
setConnection({
...connections[0],
id: "openai-key",
name: "My OpenAI API",
endpoint: "https://api.openai.com/v1",
method: "API key",
});
setKey("");
setStage("agent");
}
}}
/>
) : null}
<AdvancedOptions>
<Button
variant="link"
className="h-auto self-start p-0"
onClick={() => setStage("other")}
>
Use another provider or gateway
</Button>
</AdvancedOptions>
<Footer
back="Back"
onBack={() => {
setMode("subscription");
setKey("");
}}
next={
savedConnection
? "Use saved connection"
: mode === "subscription"
? "Sign in to OpenAI"
: "Connect"
}
disabled={!savedConnection && mode === "api" && !key.trim()}
onNext={() => {
setConnection(
mode === "api" && !savedConnection
? {
...connections[0],
id: "openai-key",
name: "My OpenAI API",
endpoint: "https://api.openai.com/v1",
method: "API key",
}
: connections[0],
);
setKey("");
setStage("agent");
}}
/>
</Surface>
);
}
ui/storybook/prototypes/provider-routing/ConnectionCatalog.tsx
+229
View File
@@ -0,0 +1,229 @@
import { useState, type ComponentProps } from "react";
import {
getAppStoreDefinition,
APP_STORE_DEFINITIONS,
aiProviderSetupPreset,
type ToolConnection,
} from "@paperclipai/shared";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { AiProviderSetup } from "@/components/ai-connections/AiProviderSetup";
import { appDefinitionLogoUrl } from "@/pages/apps/app-definition-display";
import { ConnectorCard } from "@/pages/apps/Browse";
import { Input } from "@/components/ui/input";
import { Button } from "@/components/ui/button";
import {
Dialog,
DialogContent,
DialogHeader,
DialogTitle,
DialogDescription,
DialogFooter,
} from "@/components/ui/dialog";
import { providers, type Connection, type Provider } from "./model";
const date = new Date("2026-10-02T12:00:00Z");
export function asToolConnection(connection: Connection): ToolConnection {
return {
id: connection.id,
uid: connection.id,
companyId: "company-storybook",
applicationId: `provider-routing-${connection.provider}`,
name: connection.name,
connectionKind: "managed",
ownership: "customer",
connectionPurpose: "ai",
transport: "runtime_auth",
authKind:
connection.method === "Subscription"
? "oauth"
: connection.method === "No authentication"
? "none"
: "api_key",
credentialSource: "paperclip_vault",
credentialPolicy: connection.ownership === "shared" ? "shared" : "per_user",
status: connection.status === "connected" ? "active" : "disabled",
enabled: connection.status === "connected",
transportConfig: {},
config: { sourceTemplateKey: connection.provider },
credentialSecretRefs: [],
healthStatus: connection.status === "connected" ? "ok" : "error",
healthCheckedAt: date,
healthMessage: null,
lastError: null,
createdByAgentId: null,
createdByUserId: "dotta",
createdAt: date,
updatedAt: date,
};
}
/** Reuse the actual Connectors provider card, account rows, owner identity, status and menus. */
export function ConnectionCatalog({
rows,
onOpen,
onAdd,
onRemove,
readOnly,
}: {
rows: Connection[];
onOpen: (connection: Connection) => void;
onAdd: (provider: Provider) => void;
onRemove: (id: string) => void;
readOnly?: boolean;
}) {
const [query, setQuery] = useState("");
const [removing, setRemoving] = useState<string>();
const visible = providers.filter((provider) =>
rows.some(
(row) =>
row.provider === provider.value &&
`${provider.label} ${row.name} ${row.endpoint}`
.toLowerCase()
.includes(query.toLowerCase()),
),
);
return (
<div className="space-y-5">
<Input
type="search"
aria-label="Search connectors"
placeholder="Search connectors…"
value={query}
onChange={(event) => setQuery(event.target.value)}
/>
<section className="space-y-3" aria-label="Connector list">
{visible.map((provider) => {
const entry = getAppStoreDefinition(provider.value) ?? null;
const row: ComponentProps<typeof ConnectorCard>["row"] = {
key: provider.value,
slug: provider.value,
name: provider.label,
description: provider.description,
brandKey: provider.value,
entry,
logoUrl: appDefinitionLogoUrl(entry),
connections: rows
.filter((row) => row.provider === provider.value)
.map(asToolConnection),
chatEndpoints: [],
applications: [
{
id: `provider-routing-${provider.value}`,
companyId: "company-storybook",
applicationKey: provider.value,
name: provider.label,
description: provider.description,
type: "mcp_http",
status: "active",
pluginId: null,
ownerAgentId: null,
ownerUserId: "dotta",
metadata: { sourceTemplateKey: provider.value },
archivedAt: null,
createdAt: date,
updatedAt: date,
},
],
};
return (
<ConnectorCard
key={provider.value}
row={row}
userProfileById={
new Map([["dotta", { label: "Dotta", image: null }]])
}
chatConnectorsEnabled={false}
renderAccountDetails={(toolConnection) => {
const connection = rows.find(
(row) => row.id === toolConnection.id,
)!;
return (
<p className="text-xs text-muted-foreground">
{connection.ownership === "shared"
? "Company account"
: "Personal account"}{" "}
· {connection.method}
</p>
);
}}
onNavigate={(href) => {
const connection = rows.find(
(row) => href === `/apps/${row.id}/permissions`,
);
if (connection) onOpen(connection);
else if (!readOnly) onAdd(provider.value);
}}
onRequestRemove={(target) => {
if (!readOnly) setRemoving(target.id);
}}
/>
);
})}
</section>
{!visible.length && (
<p className="text-sm text-muted-foreground">
No connectors match “{query}”.
</p>
)}
<Dialog
open={Boolean(removing)}
onOpenChange={(open) => {
if (!open) setRemoving(undefined);
}}
>
<DialogContent>
<DialogHeader>
<DialogTitle>Remove this connection?</DialogTitle>
<DialogDescription>
Agents using this connection will need another connection before
their next run.
</DialogDescription>
</DialogHeader>
<DialogFooter className="sm:justify-between">
<Button variant="ghost" onClick={() => setRemoving(undefined)}>
Cancel
</Button>
<Button
variant="destructive"
onClick={() => {
onRemove(removing!);
setRemoving(undefined);
}}
>
Remove connection
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
</div>
);
}
/** Regular catalog rows and direct setup, using the production components. */
export function ProviderCatalogReview() {
const [source, setSource] = useState<string>();
const [query, setQuery] = useState("");
const [client] = useState(() => {
const value = new QueryClient({ defaultOptions: { queries: { retry: false, staleTime: Infinity } } });
value.setQueryData(["ai-connections", "company-storybook", undefined], { canManageConnections: true, connections: [] });
value.setQueryData(["agents", "company-storybook", "provider-access"], [{ id: "nova", name: "Nova" }]);
return value;
});
const preset = source ? aiProviderSetupPreset(source) : undefined;
if (source) return <QueryClientProvider client={client}><AiProviderSetup
companyId="company-storybook" initialProvider={preset?.provider ?? source as "openai" | "anthropic" | "xai"}
initialProtocol={preset?.protocol} providerLabel={preset?.label}
onCancel={() => setSource(undefined)} onComplete={() => setSource(undefined)}
/></QueryClientProvider>;
return <div className="space-y-5">
<Input aria-label="Search connectors" placeholder="Search connectors…" value={query} onChange={event => setQuery(event.target.value)} />
<div role="list" aria-label="Connector list" className="space-y-3">
{APP_STORE_DEFINITIONS.filter(entry => entry.tags?.includes("model-provider") && `${entry.name} ${entry.description}`.toLowerCase().includes(query.toLowerCase())).sort((a, b) => a.name.localeCompare(b.name)).map(entry => <ConnectorCard
key={entry.slug}
row={{ key: entry.slug, slug: entry.slug, name: entry.name, description: entry.description, brandKey: entry.slug, entry, logoUrl: entry.branding.logoUrl, darkLogoUrl: entry.branding.darkLogoUrl, applications: [], connections: [], chatEndpoints: [] }}
userProfileById={new Map()} chatConnectorsEnabled={false}
onNavigate={() => setSource(entry.slug)} onRequestRemove={() => {}}
/>)}
</div>
</div>;
}
ui/storybook/prototypes/provider-routing/ConnectionPicker.tsx
+67
View File
@@ -0,0 +1,67 @@
import { useId } from "react";
import {
Select,
SelectContent,
SelectItem,
SelectSeparator,
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
import { isAdvancedProvider, type Connection } from "./model";
const connectAccount = "__connect_account__";
export function ConnectionPicker({
selected,
connections,
disabled,
onSelect,
onConnect,
}: {
selected: Connection;
connections: Connection[];
disabled?: boolean;
onSelect: (id: string) => void;
onConnect: () => void;
}) {
const id = useId();
const choices = [...connections].sort(
(a, b) =>
Number(b.id === selected.id) - Number(a.id === selected.id) ||
Number(isAdvancedProvider(a.provider)) -
Number(isAdvancedProvider(b.provider)),
);
return (
<div className="flex min-w-0 flex-col gap-1">
<label htmlFor={id} className="text-xs text-muted-foreground">
Connection
</label>
<Select
value={selected.id}
disabled={disabled}
onValueChange={(value) => {
if (value === connectAccount) onConnect();
else onSelect(value);
}}
>
<SelectTrigger id={id} className="w-full">
<SelectValue>{selected.name}</SelectValue>
</SelectTrigger>
<SelectContent position="popper" align="start">
{choices.map((connection) => (
<SelectItem key={connection.id} value={connection.id}>
{connection.name}
</SelectItem>
))}
{connections.length === 0 && (
<SelectItem value="__none__" disabled>
No compatible connections
</SelectItem>
)}
<SelectSeparator />
<SelectItem value={connectAccount}>Connect an account…</SelectItem>
</SelectContent>
</Select>
</div>
);
}
ui/storybook/prototypes/provider-routing/ConnectionSetup.tsx
+425
View File
@@ -0,0 +1,425 @@
import { useState } from "react";
import { Network } from "lucide-react";
import { ConnectionChoiceList } from "@/features/connections/ConnectionChoiceList";
import { StepHeader, ConnectionAccessDefaults, connectionDefaultSummarySentence } from "@/features/connections/ConnectionSetupFlow";
import { ProviderApiKeyCard } from "@/components/AdapterLoginChrome";
import { AppLogo } from "@/pages/apps/AppLogo";
import { Button } from "@/components/ui/button";
import {
AdvancedOptions,
Choice,
Footer,
Notice,
Surface,
TextField,
} from "./shared";
import {
connections,
formatLabels,
providers,
isAdvancedProvider,
type Connection,
type Protocol,
type Provider,
} from "./model";
export type SetupProps = {
initialProvider?: Provider;
initialStep?: "provider" | "access" | "connect";
initialMethod?: string;
initialProtocol?: Protocol;
initialError?: "credential" | "endpoint";
initialAdvanced?: boolean;
onComplete?: (connection: Connection) => void;
onCancel?: () => void;
};
/** Shared by the onboarding, Apps, agent, and task prototypes. No network requests. */
export function ConnectionSetup({
initialProvider = "openrouter",
initialStep = "provider",
initialMethod = "key",
initialProtocol = "responses",
initialError,
initialAdvanced = false,
onComplete,
onCancel,
}: SetupProps) {
const [step, setStep] = useState<string>(initialStep === "access" ? "connect" : initialStep);
const [provider, setProvider] = useState(initialProvider);
const [method, setMethod] = useState(initialMethod);
const [protocol, setProtocol] = useState<Protocol>(initialProtocol);
const [endpoint, setEndpoint] = useState("");
const [apiKey, setApiKey] = useState("");
const [secret, setSecret] = useState("");
const [sessionToken, setSessionToken] = useState("");
const [region, setRegion] = useState(
initialProvider === "google" ? "us-central1" : "us-east-1",
);
const [project, setProject] = useState("");
const [header, setHeader] = useState("X-API-Key");
const [ownership, setOwnership] = useState<"personal" | "shared">("shared");
const [allAgents, setAllAgents] = useState(true);
const [agentIds, setAgentIds] = useState(new Set(["nova"]));
const [error, setError] = useState(initialError);
const [result, setResult] = useState<Connection>();
const providerLabel = providers.find(
(entry) => entry.value === provider,
)!.label;
const isDirect = ["openai", "anthropic", "xai"].includes(provider);
const isAws = provider === "bedrock";
const isVertex = provider === "google" && method === "identity";
const needsKey = !["identity", "none", "subscription"].includes(method);
const canConnect =
(!needsKey || Boolean(apiKey.trim())) &&
(method !== "header" || Boolean(header.trim())) &&
(method !== "aws-keys" || Boolean(secret.trim())) &&
(provider !== "custom" || Boolean(endpoint.trim())) &&
(!isVertex || Boolean(project.trim() && region.trim()));
function connect() {
if (provider === "custom") {
try {
const url = new URL(endpoint);
if (
!["http:", "https:"].includes(url.protocol) ||
url.username ||
url.password
)
throw new Error();
} catch {
setError("endpoint");
return;
}
}
const template = connections.find((entry) => entry.provider === provider);
const next: Connection = {
id: "created-connection",
name: `${ownership === "personal" ? "My" : "Company"} ${provider === "custom" ? new URL(endpoint).hostname : providerLabel}`,
provider,
endpoint: isAws
? region
: provider === "custom"
? endpoint
: isVertex
? `${project} · ${region}`
: (template?.endpoint ?? providerLabel),
protocols:
provider === "custom"
? [protocol]
: (template?.protocols ?? ["responses", "chat", "messages"]),
method:
method === "identity"
? "Environment identity"
: method === "subscription"
? "Subscription"
: method === "none"
? "No authentication"
: method === "aws-keys"
? "AWS credentials"
: "API key",
ownership,
models:
provider === "custom"
? ["engineering-coder"]
: (template?.models ?? ["grok-4.7"]),
status: "connected",
};
setApiKey("");
setSecret("");
setSessionToken("");
setError(undefined);
setResult(next);
setStep("done");
}
const cancel = () => {
setApiKey("");
setSecret("");
setSessionToken("");
onCancel?.();
if (!onCancel) setStep("provider");
};
const providerChoices = (advanced: boolean) => (
<ConnectionChoiceList
choices={providers
.filter((entry) => isAdvancedProvider(entry.value) === advanced)
.map((entry) => ({
id: entry.value,
name: entry.label,
description: entry.description,
icon:
entry.value === "custom" ? (
<span className="flex size-9 items-center justify-center rounded-lg bg-muted text-muted-foreground">
<Network className="size-5" />
</span>
) : (
<AppLogo name={entry.label} brandKey={entry.value} />
),
}))}
onSelect={(id) => {
setProvider(id as Provider);
setRegion(id === "google" ? "us-central1" : "us-east-1");
setApiKey("");
setSecret("");
setSessionToken("");
setEndpoint("");
setMethod("key");
setError(undefined);
setStep("connect");
}}
/>
);
if (step === "provider")
return (
<Surface
title="Connect a model provider"
description="Choose the account you want to connect."
>
{providerChoices(false)}
<AdvancedOptions
label="Advanced providers"
defaultOpen={initialAdvanced}
>
{providerChoices(true)}
</AdvancedOptions>
<Button variant="ghost" className="self-start" onClick={cancel}>
Cancel
</Button>
</Surface>
);
if (step === "done" && result)
return (
<Surface title="Connection ready" description={result.name}>
<Notice>
Credentials connected. Choose a model and test it in your agent’s
environment before running tasks.
</Notice>
<dl className="grid grid-cols-2 gap-3 text-sm">
<dt className="text-muted-foreground">Provider</dt>
<dd>{providerLabel}</dd>
<dt className="text-muted-foreground">Destination</dt>
<dd className="break-all font-mono text-xs">{result.endpoint}</dd>
<dt className="text-muted-foreground">Credential access</dt>
<dd>
{ownership === "personal"
? "Only me"
: "Any human in the organization"}
</dd>
<dt className="text-muted-foreground">Agent access</dt>
<dd>
{allAgents
? "Any agent"
: [...agentIds]
.map((id) => (id === "nova" ? "Nova" : "Atlas"))
.join(", ")}
</dd>
</dl>
<Footer
back="Add another"
onBack={() => setStep("provider")}
next={onComplete ? "Use connection" : "Done"}
onNext={() => (onComplete ? onComplete(result) : setStep("provider"))}
/>
</Surface>
);
return (
<div className="mx-auto flex max-w-2xl flex-col gap-6 p-4 sm:p-6">
<StepHeader
title={`Connect ${providerLabel}`}
subtitle="Connect now — permissions and access are yours to change afterwards."
step="key"
activeIndex={0}
labels={["Connect"]}
/>
<>
{isAws && (
<Choice
label="AWS region"
value={region}
onChange={setRegion}
options={[
{ value: "us-east-1", label: "US East · N. Virginia" },
{ value: "us-west-2", label: "US West · Oregon" },
{ value: "eu-west-1", label: "Europe · Ireland" },
]}
/>
)}
{provider === "custom" && (
<>
<Choice
label="API format"
value={protocol}
onChange={(value) => setProtocol(value as Protocol)}
options={Object.entries(formatLabels).map(([value, label]) => ({
value,
label,
}))}
/>
<TextField
label="Base URL"
placeholder="https://models.example.com/v1"
value={endpoint}
onChange={(event) => {
setEndpoint(event.target.value);
setError(undefined);
}}
hint="Use the base URL supplied by your provider, including any API prefix. Model IDs are configured separately."
/>
</>
)}
<Choice
label="Authentication"
value={method}
onChange={(value) => {
setMethod(value);
setApiKey("");
setSecret("");
setError(undefined);
}}
options={[
{ value: "key", label: isAws ? "Bedrock API key" : "API key" },
...(isDirect
? [{ value: "subscription", label: "Subscription" }]
: []),
...(isAws || provider === "google"
? [
{
value: "identity",
label: isAws
? "Use environment’s AWS identity"
: "Vertex AI · environment identity",
},
]
: []),
...(provider === "custom"
? [
{ value: "header", label: "Custom authentication header" },
{ value: "none", label: "No authentication" },
]
: []),
]}
/>
{method === "identity" && (
<>
<Choice
label="Credential environment"
value="cloud"
onChange={() => {}}
options={[
{
value: "cloud",
label: isAws
? "Company AWS workspace"
: "Company Google Cloud workspace",
},
]}
/>
<p className="text-xs text-muted-foreground">
Uses the identity available inside the selected execution
environment. Agents running elsewhere need an identity there
too.
</p>
</>
)}
{isVertex && (
<>
<TextField
label="Google Cloud project"
value={project}
onChange={(event) => setProject(event.target.value)}
/>
<TextField
label="Location"
value={region}
onChange={(event) => setRegion(event.target.value)}
/>
</>
)}
{method === "header" && (
<TextField
label="Header name"
value={header}
onChange={(event) => setHeader(event.target.value)}
/>
)}
{method === "key" ? (
<ProviderApiKeyCard
providerName={providerLabel}
value={apiKey}
onChange={(value) => {
setApiKey(value);
setError(undefined);
}}
onSubmit={() => {
if (canConnect) connect();
}}
/>
) : (
needsKey && (
<TextField
label={
method === "aws-keys"
? "Access key ID"
: method === "header"
? "Header value"
: "API key"
}
type="password"
autoComplete="off"
value={apiKey}
onChange={(event) => {
setApiKey(event.target.value);
setError(undefined);
}}
placeholder="Enter credential"
hint="Stored securely with this connection."
/>
)
)}
{method === "aws-keys" && (
<>
<TextField
label="Secret access key"
type="password"
value={secret}
onChange={(event) => setSecret(event.target.value)}
/>
<TextField
label="Session token (optional)"
type="password"
value={sessionToken}
onChange={(event) => setSessionToken(event.target.value)}
/>
</>
)}
{error && (
<Notice error>
{error === "credential"
? "The provider rejected this credential. Enter a replacement and try again."
: "Enter a valid HTTP or HTTPS base URL without embedded credentials."}
</Notice>
)}
<ConnectionAccessDefaults
companyId="company-storybook"
agents={[{ id: "nova", name: "Nova" }, { id: "atlas", name: "Atlas" }]}
sentence={connectionDefaultSummarySentence({ grantKind: ownership === "shared" ? "organization" : "user", authKind: method === "none" ? "none" : "api_key", installChoice: allAgents ? "all" : "specific", installCount: agentIds.size })}
authKind={method === "none" ? "none" : "api_key"}
grantKinds={["user", "organization"]}
grantKind={ownership === "shared" ? "organization" : "user"}
setGrantKind={kind => setOwnership(kind === "organization" ? "shared" : "personal")}
installChoice={allAgents ? "all" : "specific"}
setInstallChoice={choice => setAllAgents(choice === "all")}
installAgentIds={agentIds}
setInstallAgentIds={setAgentIds}
/>
<Footer
onBack={cancel}
next={method === "subscription" ? "Sign in" : "Connect"}
onNext={connect}
disabled={!canConnect}
/>
</>
</div>
);
}
ui/storybook/prototypes/provider-routing/Management.tsx
+470
View File
@@ -0,0 +1,470 @@
import { useRef, useState } from "react";
import { ArrowLeft, Plus } from "lucide-react";
import { Button } from "@/components/ui/button";
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from "@/components/ui/dialog";
import { ConnectionCatalog } from "./ConnectionCatalog";
import { RoutingAccess } from "./RoutingAccess";
import { ConnectionSetup } from "./ConnectionSetup";
import {
AdvancedOptions,
Choice,
Footer,
Notice,
Surface,
TextField,
} from "./shared";
import {
connections,
formatLabels,
isAdvancedProvider,
type Connection,
} from "./model";
export function Management({
initialView = "list",
initialTab = "details",
readOnly = false,
includeAdvancedConnections = false,
}: {
initialView?: "list" | "details";
initialTab?: "details" | "models" | "access";
readOnly?: boolean;
includeAdvancedConnections?: boolean;
}) {
const [rows, setRows] = useState(
initialView === "details" || includeAdvancedConnections
? connections
: connections.filter(
(connection) => !isAdvancedProvider(connection.provider),
),
);
const [selected, setSelected] = useState<Connection | undefined>(
initialView === "details" ? connections[4] : undefined,
);
const [adding, setAdding] = useState(false);
const [addingProvider, setAddingProvider] =
useState<Connection["provider"]>();
const [tab, setTab] = useState(initialTab);
const [notice, setNotice] = useState("");
const [name, setName] = useState("Engineering gateway");
const [alias, setAlias] = useState("");
const [context, setContext] = useState("128000");
const [helper, setHelper] = useState("engineering-fast");
const [models, setModels] = useState([
"engineering-coder",
"engineering-fast",
]);
const [agents, setAgents] = useState("all");
const [rotate, setRotate] = useState(false);
const [credential, setCredential] = useState("");
const [revoke, setRevoke] = useState(false);
const [contextWindows, setContextWindows] = useState<Record<string, string>>({
"engineering-coder": "128000",
"engineering-fast": "128000",
});
const [preferences, setPreferences] = useState<
Record<
string,
{
helper: string;
agents: string;
contextWindows: Record<string, string>;
}
>
>({});
const rotateTrigger = useRef<HTMLButtonElement>(null);
const revokeTrigger = useRef<HTMLButtonElement>(null);
const open = (connection: Connection) => {
const saved = preferences[connection.id];
setSelected(connection);
setName(connection.name);
setModels(connection.models);
setHelper(saved?.helper ?? connection.models.at(-1)!);
setAgents(saved?.agents ?? "all");
setContextWindows(
saved?.contextWindows ??
Object.fromEntries(connection.models.map((model) => [model, "128000"])),
);
setAlias("");
setTab("details");
setNotice("");
};
const save = () => {
if (!selected) return;
const next = { ...selected, name, models };
setPreferences((current) => ({
...current,
[next.id]: { helper, agents, contextWindows },
}));
setSelected(next);
setRows((current) =>
current.map((row) => (row.id === next.id ? next : row)),
);
setNotice("Connection settings saved.");
};
if (adding)
return (
<ConnectionSetup
initialProvider={addingProvider ?? selected?.provider}
initialStep={selected || addingProvider ? "access" : "provider"}
onCancel={() => setAdding(false)}
onComplete={(connection) => {
setRows((current) => [
...current.filter((row) => row.id !== connection.id),
connection,
]);
setAdding(false);
open(connection);
}}
/>
);
if (!selected)
return (
<Surface
title="Connectors"
description="Manage the accounts your agents use."
>
<Button
className="self-start"
disabled={readOnly}
onClick={() => setAdding(true)}
>
<Plus />
Add connection
</Button>
<ConnectionCatalog
rows={rows}
onOpen={open}
readOnly={readOnly}
onAdd={(provider) => {
setAddingProvider(provider);
setAdding(true);
}}
onRemove={(id) =>
setRows((current) => current.filter((row) => row.id !== id))
}
/>
</Surface>
);
return (
<Surface
title={selected.name}
description={`${selected.ownership === "shared" ? "Company connection" : "Personal connection"} · ${selected.status === "connected" ? "Connected" : "Credential revoked"}`}
>
<Button
variant="ghost"
className="self-start"
onClick={() => setSelected(undefined)}
>
<ArrowLeft />
Connectors
</Button>
<nav aria-label="Connection settings" className="flex flex-wrap gap-2">
{(["details", "models", "access"] as const).map((item) => (
<Button
key={item}
size="sm"
variant={tab === item ? "secondary" : "ghost"}
aria-pressed={tab === item}
onClick={() => {
setTab(item);
setNotice("");
}}
>
{item === "details"
? "Connection"
: item === "models"
? "Models"
: "Access"}
</Button>
))}
</nav>
{tab === "details" ? (
<>
<TextField
label="Connection name"
value={name}
disabled={readOnly}
onChange={(event) => setName(event.target.value)}
/>
<p className="text-sm text-muted-foreground">
{selected.method} ·{" "}
{selected.status === "connected" ? "Connected" : "Revoked"}
</p>
<Button
ref={rotateTrigger}
variant="outline"
className="self-start"
disabled={readOnly}
onClick={() => setRotate(true)}
>
Replace credential
</Button>
<AdvancedOptions label="Advanced connection settings">
<dl className="grid grid-cols-1 gap-2 text-sm sm:grid-cols-2">
<dt className="text-muted-foreground">Destination</dt>
<dd className="break-all font-mono text-xs">
{selected.endpoint}
</dd>
<dt className="text-muted-foreground">API format</dt>
<dd>
{selected.protocols
.map((format) => formatLabels[format])
.join(", ")}
</dd>
</dl>
<div className="flex flex-wrap gap-2">
<Button
variant="ghost"
disabled={readOnly}
onClick={() => setAdding(true)}
>
Use a different endpoint
</Button>
</div>
<p className="text-xs text-muted-foreground">
A different endpoint creates a separate connection. Nova and Atlas
keep their current destination until you select the new connection
for them.
</p>
</AdvancedOptions>
{selected.id === "gateway" && (
<details className="text-sm">
<summary className="cursor-pointer">Used by 2 agents</summary>
<ul className="mt-3 space-y-2 text-muted-foreground">
<li>Nova · Codex · engineering-coder</li>
<li>Atlas · OpenCode · engineering-fast</li>
</ul>
</details>
)}
</>
) : tab === "models" ? (
<>
<p className="text-sm text-muted-foreground">
Add model IDs or gateway aliases this connection serves. Each agent
chooses its own model.
</p>
<div className="flex flex-col gap-3">
{models.map((model) => (
<div
key={model}
className="flex flex-wrap items-center justify-between gap-2 border-b border-border pb-3"
>
<code className="break-all text-xs">{model}</code>
<span className="text-xs text-muted-foreground">
{contextWindows[model] ?? "128000"} tokens · Streaming · Tools
</span>
</div>
))}
</div>
<TextField
label="Model ID or alias"
value={alias}
disabled={readOnly}
onChange={(event) => setAlias(event.target.value)}
placeholder="team-reviewer"
/>
<TextField
label="Context window (tokens)"
type="number"
min={1}
value={context}
disabled={readOnly}
onChange={(event) => setContext(event.target.value)}
hint="Use the limit supplied by your provider for this alias."
/>
<Button
variant="outline"
className="self-start"
disabled={
readOnly ||
!alias.trim() ||
!Number.isFinite(Number(context)) ||
Number(context) <= 0 ||
models.includes(alias.trim())
}
onClick={() => {
setModels((current) => [...current, alias.trim()]);
setContextWindows((current) => ({
...current,
[alias.trim()]: context,
}));
setAlias("");
setNotice(
"Model added to this draft. Save changes to make it available.",
);
}}
>
Add model
</Button>
<Choice
label="Default helper model"
value={helper}
disabled={readOnly}
onChange={setHelper}
options={models.map((model) => ({ value: model, label: model }))}
/>
<p className="text-xs text-muted-foreground">
Used for titles and other auxiliary requests when supported by the
harness. Keeps those requests on this connection.
</p>
</>
) : (
<>
<RoutingAccess
ownership={selected.ownership}
onOwnershipChange={() => {}}
fixedOwnership
agentIds={
agents === "all" ? new Set<string>() : new Set(agents.split(","))
}
onAgentIdsChange={(ids) => setAgents([...ids].join(","))}
allAgents={agents === "all"}
onAllAgentsChange={(all) => setAgents(all ? "all" : "nova")}
disabled={readOnly}
hideFooter
/>
<Notice>
Personal credentials are resolved for this destination. A user’s
default account does not replace a credential for {selected.name}.
</Notice>
<Button
ref={revokeTrigger}
variant="outline"
className="self-start"
disabled={readOnly || selected.status === "expired"}
onClick={() => setRevoke(true)}
>
Revoke credential
</Button>
</>
)}
{readOnly && (
<p className="text-sm text-muted-foreground">
Only the connection owner can edit these settings.
</p>
)}
{notice && <Notice>{notice}</Notice>}
<Footer
back="Cancel"
onBack={() => {
setSelected(undefined);
setNotice("");
}}
next="Save changes"
onNext={save}
disabled={readOnly || !name.trim()}
/>
<Dialog
open={rotate}
onOpenChange={(open) => {
setRotate(open);
if (!open) setCredential("");
}}
>
<DialogContent
onCloseAutoFocus={(event) => {
event.preventDefault();
rotateTrigger.current?.focus();
}}
>
<DialogHeader>
<DialogTitle>Replace credential</DialogTitle>
<DialogDescription>
{selected.name} · {selected.endpoint}. The destination and agent
models stay the same.
</DialogDescription>
</DialogHeader>
{selected.method === "Subscription" ? (
<p className="text-sm text-muted-foreground">
Sign in again to replace this subscription credential.
</p>
) : (
<TextField
label="New API key"
type="password"
value={credential}
onChange={(event) => setCredential(event.target.value)}
autoComplete="off"
/>
)}
<DialogFooter className="sm:justify-between">
<Button
variant="ghost"
onClick={() => {
setRotate(false);
setCredential("");
}}
>
Cancel
</Button>
<Button
disabled={
selected.method !== "Subscription" && !credential.trim()
}
onClick={() => {
const next = { ...selected, status: "connected" as const };
setSelected(next);
setRows((current) =>
current.map((row) => (row.id === next.id ? next : row)),
);
setCredential("");
setRotate(false);
setNotice("Credential replaced. New runs will use it.");
}}
>
{selected.method === "Subscription"
? "Sign in"
: "Replace credential"}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
<Dialog open={revoke} onOpenChange={setRevoke}>
<DialogContent
onCloseAutoFocus={(event) => {
event.preventDefault();
revokeTrigger.current?.focus();
}}
>
<DialogHeader>
<DialogTitle>Revoke this credential?</DialogTitle>
<DialogDescription>
Agents using this connection will need a valid credential before
their next run. Their selected connection will stay the same.
</DialogDescription>
</DialogHeader>
<DialogFooter className="sm:justify-between">
<Button variant="ghost" onClick={() => setRevoke(false)}>
Cancel
</Button>
<Button
variant="destructive"
onClick={() => {
const next = { ...selected, status: "expired" as const };
setSelected(next);
setRows((current) =>
current.map((row) => (row.id === next.id ? next : row)),
);
setRevoke(false);
setNotice(
"Credential revoked. Reconnect to use this connection again.",
);
}}
>
Revoke credential
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
</Surface>
);
}
ui/storybook/prototypes/provider-routing/README.md
+118
View File
@@ -0,0 +1,118 @@
# Provider routing UI review
Open **AI Connections → Provider routing → 00 Overview → Start here**.
The seven numbered neighboring groups cover onboarding, provider setup, agent
configuration, connection management, and task recovery. Every preview includes
links to the other groups. The overview links to the existing stories reviewed;
the existing AI Connections review index links back here.
Groups 01–05 remain design fixtures: their credentials never reach a real provider.
**06 Production components** renders `AiProviderSetup` and `AiConnectionSelect`
directly from the app with fixture data. The running app now implements connection
routing, encrypted credentials, connection-aware model selection, reconnect, and
runner projection. See [AI Connections](../../../../doc/connections/AI-CONNECTIONS.md)
for the implemented compatibility matrix and validation evidence. Exploratory
Vertex, environment-identity, custom-header, and model-discovery prototypes are
not shipped options; the production group is the implementation reference.
Shared production harness/model pickers also receive presentation improvements:
the existing `AdapterMark` is extracted for reuse, and their triggers share the
`Select` trigger styling for consistent sizing.
Provider choices reuse `ConnectionChoiceList` with its optional icon slot and
the production `AppLogo` renderer. Google and Bedrock marks come from the
[LobeHub static SVG library](https://github.com/lobehub/lobe-icons/tree/master/packages/static-svg/icons),
under the existing `ui/public/brands/apps/LOBE-LICENSE`; custom gateways use the
shared network icon. Marks resolve locally through the app brand manifest.
## Existing UI reused
| Surface | Existing pieces | Proposed extension |
| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------- |
| Connectors list | `ConnectorCard` from `pages/apps/Browse`, including its `ConnectionAccountRow`, owner, status, and menus | Provider fixtures; compact account summaries |
| Provider setup | `StepHeader`, `AccessStepContent` (human access and agent picker), `ConnectionChoiceList`, `ProviderApiKeyCard` | Provider/endpoint, protocol, AWS/Vertex identity fields |
| Agent settings | `AgentSettingsPreview` with the real `AgentContextualSidebar`, breadcrumbs, and page header; `AdapterTypeDropdown`, `ModelDropdown`, `Select`, `AiConnectionLegacyNotice`, `RuntimeTestCard` | Connection-aware model selection and compatibility states |
| First onboarding | `ModelSourceTiles`, `CredentialModeLink`, `ConnectionChoiceList`, `ProviderApiKeyCard` | Optional provider/gateway entry |
| Connection detail | `AccessStepContent` and shared form/dialog primitives | Destination summary, model aliases, and credential management |
| Task recovery | `ConnectionIntentInteractionBody` and its inline Fix connection flow; `ProviderApiKeyCard` | Gateway-specific credential setup content |
**03 Agent is located at Agents → Nova → Harness / Runtime**
(`/agents/nova/runtime`). It renders inside the existing agent settings page;
the proposed controls fill its runtime section. This is not a separate page.
The Connectors story reuses the production provider cards and account rows,
while its new routing detail fields remain a prototype.
## Simple by default
- **Onboarding:** normal subscription/API-key path. The gateway entry is inside
collapsed **Advanced**. `Happy path walkthrough` completes setup without opening it.
- **Agent:** native Codex/Claude account by default. **Connection** is a dropdown
between Harness and Model, using the same shared `Select` as Reasoning and
Execution environment. It lists all saved connections compatible with the harness,
including configured gateways, and offers **Connect an account…**. Changing
the harness updates the choices; an incompatible current selection stays visible
with an error until the user chooses a replacement.
**Advanced model settings** exposes provider setup and enables manual model IDs.
Opening or closing it never changes the saved connection/model. Blocking errors
remain visible. `Configured gateway` shows a previously selected gateway with
the same compact view; `Connection dropdown` and `Advanced model settings` show
the expanded surfaces for review. `Change account walkthrough` covers switching
ordinary accounts, dismissing without a change, focus return, and invalidating a
previous test. `Harness connection filtering` checks that switching the harness
updates the available accounts.
- **Add connection:** familiar providers first, with OpenRouter, Bedrock, and
custom endpoints under **Advanced providers**.
- **Manage:** the default fixture shows ordinary connected accounts. `With advanced
connections` shows a company that already uses gateways. Saved gateways remain
visible; URLs and protocols live inside **Advanced connection settings** on detail.
Disclosures reuse the production `Collapsible` and `Button` primitives. The story
assertions cover the hidden advanced fields, completing ordinary onboarding,
explicitly opening advanced setup, and preserving a custom selection on collapse.
## Review sequence
1. **01 Onboarding:** current default sign-in shape, saved reuse, and optional
“Use another provider or gateway” entry under Advanced. Walk through setup into an agent.
2. **02 Connect:** provider → Access → Connect; OpenRouter API key, Bedrock region
with a Bedrock API key; exploratory environment identity, Google Vertex identity,
custom Responses/Messages endpoints, custom auth header, and local endpoint.
3. **03 Agent:** harness/connection/model ownership, new and legacy runners,
model discovery and manual alias, incompatibility, loading/empty/denied states,
environment-specific tests, read-only, and adoption of existing settings.
4. **04 Manage:** list/detail, credential replacement, endpoint replacement as a
separate connection, model aliases/metadata, helper default, access, and revoke.
5. **05 Recovery:** expired/missing credential, inline repair, cancellation/focus,
invalid key retry, protocol mismatch, and quota. Successful inline repair uses
the existing connection request completion and task continuation flow.
Walkthrough stories execute interactions automatically. Other stories start at
the named state and remain interactive. Navigate to reset fixtures. The recovery
demo rejects the example key `invalid`; other nonempty example keys succeed.
Changing an agent connection preserves its model until an explicit replacement
is chosen. Changing settings invalidates the previous test.
## Scope exclusions
OpenClaw Gateway, Hermes Gateway, Claude Managed, AWS AgentCore, Process, HTTP,
and legacy `acpx_local` are excluded. Remote/custom integrations defer model
choices to their external agent/runtime; `acpx_local` is retired. Claude/Grok's
ACPX-backed new runner and Hermes **local** remain in scope. Cursor's custom
endpoint capability and native Pi/Copilot qualification are shown as boundaries,
not presented as shipped integrations.
## Validation
```sh
pnpm --filter @paperclipai/ui exec storybook dev --port 6143 --host 127.0.0.1 --no-open --ci -c storybook/.storybook
pnpm --filter @paperclipai/ui typecheck
pnpm check:token-gates
pnpm build-storybook
```
The walkthrough stories include interaction assertions. Review all story renders,
walkthrough results, internal links, and representative light/dark mobile layouts
in the browser.
Production Bedrock setup accepts only a Bedrock API key. General AWS access keys
are excluded until a credential broker can contain them. Environment identity,
Vertex, and arbitrary-header stories remain design explorations, not shipped
provider options.
ui/storybook/prototypes/provider-routing/Recovery.tsx
+185
View File
@@ -0,0 +1,185 @@
import { useEffect, useRef, useState } from "react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import type { ConnectionIntentInteraction } from "@paperclipai/shared";
import { ConnectionIntentInteractionBody } from "@/features/connections/ConnectionIntentInteractionBody";
import type { ConnectionSetupFlowProps } from "@/features/connections/ConnectionSetupFlow";
import { ProviderApiKeyCard } from "@/components/AdapterLoginChrome";
import { pendingConnectionIntentInteraction } from "@/fixtures/issueThreadInteractionFixtures";
import { Footer, Notice, Surface } from "./shared";
type Problem = "expired" | "missing" | "protocol" | "quota";
const initial: ConnectionIntentInteraction = {
...pendingConnectionIntentInteraction,
id: "provider-routing-repair",
companyId: "company-storybook",
addresseeUserId: "dotta",
payload: {
version: 1,
purpose: "ai",
serviceSlug: "connection:gateway",
serviceName: "Engineering gateway",
requestingAgentId: "nova",
requestingAgentName: "Nova",
phase: "requested",
},
};
export function Recovery({ problem = "expired" }: { problem?: Problem }) {
return (
<Surface
title="Review the release checklist"
description="Task PC-142 · Assigned to Nova"
>
<p className="text-sm text-muted-foreground">
Codex · Engineering gateway · engineering-coder
</p>
{problem === "protocol" || problem === "quota" ? (
<Notice error>
<p>
{problem === "protocol"
? "This endpoint serves Chat Completions. Codex needs Responses with streaming and tool support."
: "Engineering gateway is at its usage limit. The credential is connected; wait for capacity or choose another authorized connection."}
</p>
<a
className="underline underline-offset-4"
href="/?path=/story/ai-connections-provider-routing-03-agent--codex-new-runner"
target="_top"
>
Review agent configuration
</a>
</Notice>
) : (
<TaskRepair problem={problem} />
)}
</Surface>
);
}
/** Use the production task card, inline setup, completion and focus lifecycle. Only API replies are fixtures. */
function TaskRepair({ problem }: { problem: "expired" | "missing" }) {
const [client] = useState(
() =>
new QueryClient({
defaultOptions: {
queries: { retry: false, refetchOnWindowFocus: false },
mutations: { retry: false },
},
}),
);
const [interaction, setInteraction] = useState(initial);
const current = useRef(initial);
const [ready, setReady] = useState(false);
useEffect(() => {
const previous = window.fetch;
window.fetch = async (input, init) => {
const path = new URL(
input instanceof Request ? input.url : String(input),
window.location.origin,
).pathname;
if (!path.startsWith("/api/connection-intents/provider-routing-repair/"))
return previous(input, init);
if (path.endsWith("/complete")) {
current.current = {
...current.current,
status: "accepted",
result: { version: 1, outcome: "connected", connectionId: "gateway" },
};
setInteraction(current.current);
return Response.json(current.current);
}
if (path.endsWith("/setup-options"))
return Response.json({
version: 1,
interaction: current.current,
service: {
service: "connection:gateway",
name: "Engineering gateway",
methods: [],
state: "needs_user_action",
connectionId: "gateway",
},
requestedAgentId: "nova",
existingConnections: [],
});
return Response.json(
{ error: "Unsupported Storybook operation" },
{ status: 400 },
);
};
setReady(true);
return () => {
window.fetch = previous;
client.clear();
};
}, [client]);
return ready ? (
<QueryClientProvider client={client}>
<ConnectionIntentInteractionBody
interaction={interaction}
currentUserId="dotta"
addresseeLabel="Dotta"
renderSetup={(props) => (
<GatewayCredential {...props} problem={problem} />
)}
/>
</QueryClientProvider>
) : (
<p role="status">Loading connection request…</p>
);
}
function GatewayCredential({
problem,
onCancel,
onComplete,
}: ConnectionSetupFlowProps & { problem: Problem }) {
const [credential, setCredential] = useState("");
const [failed, setFailed] = useState(false);
const connect = () => {
if (!credential.trim()) return;
setCredential("");
if (credential === "invalid") {
setFailed(true);
return;
}
onComplete?.({ connectionId: "gateway" });
};
return (
<section aria-label="Gateway credential repair" className="space-y-4">
<p className="text-sm text-muted-foreground">
{problem === "missing"
? "Connect your own credential for this destination."
: "Replace the expired credential for this destination."}{" "}
The harness and selected model stay the same.
</p>
<p className="break-all font-mono text-xs">
https://models.example.com/v1 · OpenAI Responses
</p>
<ProviderApiKeyCard
providerName="Engineering gateway"
value={credential}
onChange={(value) => {
setCredential(value);
setFailed(false);
}}
onSubmit={connect}
autoFocus
/>
{failed && (
<Notice error>
The provider rejected this credential. Enter another key and retry.
</Notice>
)}
<Footer
back="Cancel"
onBack={() => {
setCredential("");
onCancel?.();
}}
next="Connect"
onNext={connect}
disabled={!credential.trim()}
/>
</section>
);
}
ui/storybook/prototypes/provider-routing/ReviewIndex.tsx
+168
View File
@@ -0,0 +1,168 @@
import { reviewGroups, reviewPath, Surface } from "./shared";
const existing = [
[
"AI Connections / Review",
"ai-connections-review--review-index",
"Catalog, account ownership, authentication, task repair, and legacy adoption",
],
[
"Onboarding / Agent arc",
"onboarding-agent-arc--connect-a-model",
"Current first onboarding, using the production wizard",
],
[
"Onboarding / Saved connections",
"onboarding-saved-connections--claude-api-keys",
"Existing account reuse",
],
[
"Onboarding / New agent",
"onboarding-new-agent--interactive-flow",
"Hire, connect, configure, and test",
],
[
"Agents / Configuration refresh",
"agents-configuration-refresh--runtime",
"Harness, model, environment, and save controls",
],
[
"AI Connections / Inline task",
"ai-connections-review--inline-task-connection",
"Current connection request and inline Fix connection flow",
],
] as const;
const descriptions = [
"Review scope, ownership, existing surfaces, and decisions.",
"Sign in, choose a model, test, and finish. Other providers stay under Advanced.",
"Regular provider connector rows; each opens setup with shared permissions folded under Advanced.",
"Everyday account and model selection, then optional advanced providers, aliases, and validation.",
"Connection list, destinations, credential replacement, models, helper defaults, and access.",
"Task-level reconnect, missing personal credentials, protocol failure, and quota failure.",
"Implemented provider setup and compatible connection selector, rendered directly from production components.",
];
export function ReviewIndex() {
return (
<Surface
title="Provider routing · Start here"
description="Review the neighboring groups below. Groups 01–05 explore the design; 06 renders the implemented production components with fixture data."
>
<ol className="flex flex-col gap-4">
{reviewGroups.slice(1).map(([group, story, label], index) => (
<li key={group} className="space-y-1">
<a
className="font-medium underline underline-offset-4"
href={reviewPath(group, story)}
target="_top"
>
{index + 1}. {label}
</a>
<p className="text-sm text-muted-foreground">
{descriptions[index + 1]}
</p>
</li>
))}
</ol>
<section className="space-y-3">
<h2 className="font-semibold">Simple by default</h2>
<p className="text-sm text-muted-foreground">
Start with Default path or Happy path walkthrough in Onboarding, then
Codex new runner or Claude default in Agent. Sign in with the normal
account, choose a model, and test. The Connection dropdown lists saved
connections compatible with the harness, including configured
gateways. Provider URLs, custom provider setup, and custom model entry
stay behind an explicit Advanced action.
</p>
<p className="text-sm text-muted-foreground">
Advanced providers stories show the expanded controls. Configured
gateway shows the everyday view after setup: the connection name and
selected model remain visible while its technical settings are folded
away. Closing Advanced preserves every selection and validation error.
</p>
</section>
<section className="space-y-3">
<h2 className="font-semibold">Where these changes live</h2>
<p className="text-sm text-muted-foreground">
Agent settings live at Agents → Nova → Harness / Runtime. The Agent
stories render the existing settings page, sidebar, and header, with
the proposed connection controls in its runtime section.
</p>
<p className="text-sm text-muted-foreground">
Manage uses the existing Connectors provider cards and account rows.
Setup reuses the Access cards, agent picker, and API key form. Agent
selection uses the existing harness dropdown, connection choices,
model picker, and test card. Recovery uses the existing inline Fix
connection flow. Destination and model routing fields are the proposed
additions.
</p>
</section>
<section className="space-y-3">
<h2 className="font-semibold">Decisions to review</h2>
<ul className="list-disc space-y-2 pl-5 text-sm">
<li>
The connection owns the destination and authentication. The agent
owns its harness and selected model.
</li>
<li>
First onboarding keeps its usual sign-in path. The optional provider
link is inside Advanced; the catalog opens each provider directly. Permissions stay under Advanced.
</li>
<li>
Changing a connection preserves the harness and asks for a
compatible model when needed.
</li>
<li>
Tests run in the selected environment. A key being accepted does not
establish model/tool compatibility.
</li>
<li>
A new endpoint creates a new connection; changing a personal default
never silently selects another destination.
</li>
</ul>
</section>
<section className="space-y-3">
<h2 className="font-semibold">Included harnesses</h2>
<p className="text-sm text-muted-foreground">
Codex, Claude Code, OpenCode, Grok Build, Pi, Gemini CLI, Kimi CLI,
and Hermes local. Both runner paths are represented where available.
Cursor is shown as unverified for custom endpoints; native Pi and
Copilot show their qualification boundary. Fixture model IDs are
examples, not a validated provider catalog.
</p>
</section>
<section className="space-y-3">
<h2 className="font-semibold">Excluded from this update</h2>
<p className="text-sm text-muted-foreground">
OpenClaw Gateway, Hermes Gateway, Claude Managed, AWS AgentCore,
Process, HTTP, and legacy acpx_local are out of scope. Remote/custom
integrations retain model configuration in the agent service or
runtime they invoke. Legacy acpx_local is retired. This exclusion does
not remove Claude/Grok’s ACPX-backed new runner or Hermes local from
the review.
</p>
</section>
<section className="space-y-3">
<h2 className="font-semibold">Existing stories reviewed</h2>
{existing.map(([label, id, detail]) => (
<div key={id} className="space-y-1">
<a
href={`/?path=/story/${id}`}
target="_top"
className="text-sm underline underline-offset-4"
>
{label}
</a>
<p className="text-xs text-muted-foreground">{detail}</p>
</div>
))}
</section>
<p className="text-xs text-muted-foreground">
Use example credentials only. In the recovery demo, “invalid” simulates
rejection. Stories reset on navigation. Local state and intercepted
fixture APIs keep credentials, tests, and settings in the preview;
nothing is sent to a provider or Paperclip server.
</p>
</Surface>
);
}
ui/storybook/prototypes/provider-routing/RoutingAccess.tsx
+71
View File
@@ -0,0 +1,71 @@
import { AccessStepContent } from "@/features/connections/ConnectionSetupFlow";
import { Footer } from "./shared";
const agents = [
{ id: "nova", name: "Nova" },
{ id: "atlas", name: "Atlas" },
];
/** Offline state adapter for the same access cards and agent picker used by Connectors. */
export function RoutingAccess({
ownership,
onOwnershipChange,
allAgents,
onAllAgentsChange,
agentIds,
onAgentIdsChange,
fixedOwnership,
disabled,
hideFooter,
onBack = () => {},
onContinue = () => {},
}: {
ownership: "personal" | "shared";
onOwnershipChange: (value: "personal" | "shared") => void;
allAgents: boolean;
onAllAgentsChange: (value: boolean) => void;
agentIds: Set<string>;
onAgentIdsChange: (value: Set<string>) => void;
fixedOwnership?: boolean;
disabled?: boolean;
hideFooter?: boolean;
onBack?: () => void;
onContinue?: () => void;
}) {
return (
<div className="space-y-6">
<fieldset disabled={disabled} className="min-w-0">
<AccessStepContent
agents={agents}
authKind="api_key"
grantKind={ownership === "shared" ? "organization" : "user"}
grantKinds={
fixedOwnership
? [ownership === "shared" ? "organization" : "user"]
: ["user", "organization"]
}
setGrantKind={(kind) =>
onOwnershipChange(kind === "organization" ? "shared" : "personal")
}
installChoice={allAgents ? "all" : "specific"}
setInstallChoice={(choice) => onAllAgentsChange(choice === "all")}
installAgentIds={agentIds}
setInstallAgentIds={onAgentIdsChange}
submitLabel="Continue"
onBack={onBack}
onContinue={onContinue}
hideFooter
bare
/>
</fieldset>
{!hideFooter && (
<Footer
onBack={onBack}
next="Continue"
onNext={onContinue}
disabled={disabled || (!allAgents && !agentIds.size)}
/>
)}
</div>
);
}
ui/storybook/prototypes/provider-routing/model.ts
+250
View File
@@ -0,0 +1,250 @@
/** Proposed UI fixtures only. These are deliberately separate from the production auth contract. */
export type Harness =
| "codex"
| "claude"
| "opencode"
| "pi"
| "gemini"
| "kimi"
| "grok"
| "hermes"
| "cursor"
| "copilot";
export type Provider =
| "openai"
| "anthropic"
| "openrouter"
| "bedrock"
| "google"
| "xai"
| "custom";
export type Protocol = "responses" | "messages" | "chat" | "gemini";
export const isAdvancedProvider = (provider: Provider) =>
["openrouter", "bedrock", "custom"].includes(provider);
export type Connection = {
id: string;
name: string;
provider: Provider;
endpoint: string;
protocols: Protocol[];
method: string;
ownership: "personal" | "shared";
models: string[];
status: "connected" | "expired" | "denied";
};
export const harnesses: {
value: Harness;
label: string;
native: boolean;
formats: Protocol[];
}[] = [
{ value: "codex", label: "Codex", native: true, formats: ["responses"] },
{
value: "claude",
label: "Claude Code",
native: true,
formats: ["messages"],
},
{
value: "opencode",
label: "OpenCode",
native: true,
formats: ["responses", "messages", "chat"],
},
{
value: "grok",
label: "Grok Build",
native: true,
formats: ["responses", "messages", "chat"],
},
{
value: "pi",
label: "Pi",
native: false,
formats: ["responses", "messages", "chat", "gemini"],
},
{ value: "gemini", label: "Gemini CLI", native: false, formats: ["gemini"] },
{
value: "kimi",
label: "Kimi CLI",
native: false,
formats: ["responses", "messages", "chat", "gemini"],
},
{ value: "hermes", label: "Hermes local", native: false, formats: ["chat"] },
{ value: "cursor", label: "Cursor", native: false, formats: [] },
{
value: "copilot",
label: "GitHub Copilot",
native: false,
formats: ["chat", "messages"],
},
];
export const providers: {
value: Provider;
label: string;
description: string;
}[] = [
{
value: "openai",
label: "OpenAI",
description: "ChatGPT subscription or API key",
},
{
value: "anthropic",
label: "Anthropic",
description: "Claude subscription or API key",
},
{
value: "openrouter",
label: "OpenRouter",
description: "One account, multiple model providers",
},
{
value: "bedrock",
label: "Amazon Bedrock",
description: "Your AWS account and region",
},
{ value: "google", label: "Google", description: "Gemini API or Vertex AI" },
{ value: "xai", label: "xAI", description: "Grok subscription or API key" },
{
value: "custom",
label: "Custom provider or gateway",
description: "A company endpoint or local model service",
},
];
export const formatLabels: Record<Protocol, string> = {
responses: "OpenAI Responses",
messages: "Anthropic Messages",
chat: "OpenAI Chat Completions",
gemini: "Google Gemini",
};
export const connections: Connection[] = [
{
id: "chatgpt",
name: "My ChatGPT subscription",
provider: "openai",
endpoint: "OpenAI",
protocols: ["responses"],
method: "Subscription",
ownership: "personal",
models: ["gpt-5.4"],
status: "connected",
},
{
id: "claude",
name: "My Claude subscription",
provider: "anthropic",
endpoint: "Anthropic",
protocols: ["messages"],
method: "Subscription",
ownership: "personal",
models: ["claude-sonnet-4-6"],
status: "connected",
},
{
id: "router",
name: "Company OpenRouter",
provider: "openrouter",
endpoint: "https://openrouter.ai/api",
protocols: ["responses", "messages", "chat"],
method: "API key",
ownership: "shared",
models: ["openai/gpt-5.4", "anthropic/claude-sonnet-4.6"],
status: "connected",
},
{
id: "bedrock",
name: "Company Bedrock",
provider: "bedrock",
endpoint: "us-east-1",
protocols: ["responses", "messages"],
method: "Environment identity",
ownership: "shared",
models: ["us.anthropic.claude-sonnet-4-6"],
status: "connected",
},
{
id: "gateway",
name: "Engineering gateway",
provider: "custom",
endpoint: "https://models.example.com/v1",
protocols: ["responses"],
method: "API key",
ownership: "shared",
models: ["engineering-coder", "engineering-fast"],
status: "connected",
},
{
id: "chat",
name: "Local model service",
provider: "custom",
endpoint: "http://localhost:11434/v1",
protocols: ["chat"],
method: "No authentication",
ownership: "personal",
models: ["qwen-coder"],
status: "connected",
},
{
id: "gemini",
name: "Company Gemini",
provider: "google",
endpoint: "Google Gemini API",
protocols: ["gemini"],
method: "API key",
ownership: "shared",
models: ["gemini-2.5-pro"],
status: "connected",
},
{
id: "openai-api",
name: "Company OpenAI",
provider: "openai",
endpoint: "https://api.openai.com/v1",
protocols: ["responses"],
method: "API key",
ownership: "shared",
models: ["gpt-5.4"],
status: "connected",
},
];
export function compatibility(
connection: Connection,
harness: Harness,
): string | undefined {
if (harness === "cursor")
return "Custom provider configuration has not been verified for Cursor.";
if (harness === "copilot")
return "This runner profile is awaiting qualification.";
if (
connection.method === "Subscription" &&
!(
(connection.provider === "openai" && harness === "codex") ||
(connection.provider === "anthropic" && harness === "claude")
)
)
return "This subscription is available through its own harness.";
if (
connection.provider === "bedrock" &&
!["codex", "claude", "opencode"].includes(harness)
)
return "Bedrock support has not been verified for this harness in Paperclip.";
if (
!harnesses
.find((item) => item.value === harness)!
.formats.some((format) => connection.protocols.includes(format))
)
return `${harnesses.find((item) => item.value === harness)!.label} needs a different API format.`;
}
export function modelsFor(connection: Connection, harness: Harness): string[] {
if (connection.id === "router" && harness === "claude")
return connection.models.filter((id) => id.startsWith("anthropic/"));
if (connection.provider === "bedrock" && harness === "codex")
return ["openai.gpt-5.4"];
return connection.models;
}
export function defaultConnection(harness: Harness): Connection {
return (
connections.find((item) => !compatibility(item, harness)) ?? connections[2]
);
}
ui/storybook/prototypes/provider-routing/shared.tsx
+255
View File
@@ -0,0 +1,255 @@
import { useId, type ReactNode } from "react";
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import { ChevronDown } from "lucide-react";
import {
Collapsible,
CollapsibleContent,
CollapsibleTrigger,
} from "@/components/ui/collapsible";
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
export const reviewPath = (group: string, story: string) =>
`/?path=/story/ai-connections-provider-routing-${group}--${story}`;
export function AdvancedOptions({
children,
label = "Advanced",
open,
onOpenChange,
defaultOpen = false,
}: {
children: ReactNode;
label?: string;
open?: boolean;
onOpenChange?: (open: boolean) => void;
defaultOpen?: boolean;
}) {
return (
<Collapsible
open={open}
onOpenChange={onOpenChange}
defaultOpen={defaultOpen}
>
<CollapsibleTrigger asChild>
<Button
variant="ghost"
size="sm"
className="group px-0 text-muted-foreground"
>
<ChevronDown className="size-4 -rotate-90 group-data-[state=open]:rotate-0" />
{label}
</Button>
</CollapsibleTrigger>
<CollapsibleContent className="space-y-4 pt-4">
{children}
</CollapsibleContent>
</Collapsible>
);
}
export const reviewGroups = [
["00-overview", "start-here", "Start here"],
["01-onboarding", "default-path", "Onboarding"],
["02-connect", "choose-provider", "Connect"],
["03-agent", "codex-new-runner", "Agent"],
["04-manage", "connection-list", "Manage"],
["05-recovery", "expired-credential", "Recovery"],
["06-production-components", "choose-provider", "Production components"],
] as const;
export function ReviewFrame({
children,
location,
}: {
children: ReactNode;
location: string;
}) {
return (
<div className="mx-auto flex max-w-7xl flex-col gap-6 p-4 sm:p-6">
<aside
aria-label="Prototype review context"
className="flex flex-col gap-3 rounded-lg border border-dashed border-border bg-muted p-4"
>
<div className="flex flex-wrap items-center justify-between gap-2">
<strong className="text-sm">Provider routing · Design review</strong>
<span className="text-xs text-muted-foreground">
Storybook prototype · No live authentication
</span>
</div>
<p className="text-sm text-muted-foreground">
<strong>Location:</strong> {location}
</p>
<nav
aria-label="Provider routing review"
className="flex flex-wrap gap-x-4 gap-y-2"
>
{reviewGroups.map(([group, story, label]) => (
<a
key={group}
className="text-sm underline underline-offset-4"
href={reviewPath(group, story)}
target="_top"
>
{label}
</a>
))}
</nav>
<details className="text-xs text-muted-foreground">
<summary className="cursor-pointer">
Production location and prototype scope
</summary>
<p className="mt-3">
Uses existing Paperclip components with local fixture state.
Provider support, accounts, model catalogs, and tests are
illustrative fixtures; these flows are not wired to the product.
</p>
</details>
</aside>
<div
className="min-w-0 rounded-lg border border-border bg-background"
data-testid="provider-routing-preview"
>
{children}
</div>
</div>
);
}
export function Surface({
title,
description,
children,
embedded = false,
}: {
title: string;
description?: string;
children: ReactNode;
embedded?: boolean;
}) {
return (
<section
className={
embedded
? "flex flex-col gap-6"
: "mx-auto flex max-w-2xl flex-col gap-6 p-4 sm:p-6"
}
>
{!embedded && (
<header className="space-y-2">
<h1 className="text-xl font-semibold">{title}</h1>
{description && (
<p className="text-sm text-muted-foreground">{description}</p>
)}
</header>
)}
{children}
</section>
);
}
export function TextField({
label,
hint,
...props
}: { label: string; hint?: string } & React.ComponentProps<typeof Input>) {
const id = useId();
return (
<div className="flex flex-col gap-1">
<label htmlFor={id} className="text-xs text-muted-foreground">
{label}
</label>
<Input
id={id}
aria-describedby={hint ? `${id}-hint` : undefined}
{...props}
/>
{hint && (
<p id={`${id}-hint`} className="text-xs text-muted-foreground">
{hint}
</p>
)}
</div>
);
}
export function Choice({
label,
value,
onChange,
options,
disabled = false,
}: {
label: string;
value: string;
onChange: (value: string) => void;
options: { value: string; label: string; disabled?: boolean }[];
disabled?: boolean;
}) {
const id = useId();
return (
<div className="flex min-w-0 flex-col gap-1">
<label htmlFor={id} className="text-xs text-muted-foreground">
{label}
</label>
<Select value={value} onValueChange={onChange} disabled={disabled}>
<SelectTrigger id={id} className="w-full">
<SelectValue />
</SelectTrigger>
<SelectContent>
{options.map((option) => (
<SelectItem
key={option.value}
value={option.value}
disabled={option.disabled}
>
{option.label}
</SelectItem>
))}
</SelectContent>
</Select>
</div>
);
}
export function Footer({
back,
next,
onBack,
onNext,
disabled,
}: {
back?: string;
next: string;
onBack: () => void;
onNext: () => void;
disabled?: boolean;
}) {
return (
<footer className="flex items-center justify-between gap-3 border-t border-border pt-4">
<Button variant="ghost" onClick={onBack}>
{back ?? "Back"}
</Button>
<Button onClick={onNext} disabled={disabled}>
{next}
</Button>
</footer>
);
}
export function Notice({
children,
error = false,
}: {
children: ReactNode;
error?: boolean;
}) {
return (
<div
role={error ? "alert" : "status"}
className={`rounded-md border p-3 text-sm ${error ? "border-destructive/30 text-destructive" : "border-border bg-muted text-foreground"}`}
>
{children}
</div>
);
}
ui/storybook/prototypes/provider-routing/story-support.ts
+36
View File
@@ -0,0 +1,36 @@
import { addons } from "storybook/preview-api";
import { userEvent, within } from "storybook/test";
export const reviewLifecycle = {
beforeEach: () => {
delete document.body.dataset.providerRoutingReady;
delete document.body.dataset.providerRoutingError;
const channel = addons.getChannel();
const report = (error: unknown) => {
document.body.dataset.providerRoutingError = JSON.stringify(error);
};
channel.on("playFunctionThrewException", report);
channel.on("unhandledErrorsWhilePlaying", report);
return () => {
channel.off("playFunctionThrewException", report);
channel.off("unhandledErrorsWhilePlaying", report);
};
},
afterEach: ({ id }: { id: string }) => {
document.body.dataset.providerRoutingReady = id;
},
};
export async function choose(
canvasElement: HTMLElement,
label: string,
option: string,
) {
await userEvent.click(
within(canvasElement).getByRole("combobox", { name: label }),
);
await userEvent.click(
await within(canvasElement.ownerDocument.body).findByRole("option", {
name: option,
}),
);
}
ui/storybook/stories/ai-connections.stories.tsx
+3
View File
@@ -75,6 +75,9 @@ export const ReviewIndex: Story = {
render: () => (
<main className="mx-auto flex max-w-3xl flex-col gap-6 p-6">
<h1 className="text-xl font-semibold">AI Connections · Review index</h1>
<a className="text-sm underline underline-offset-2" href="/?path=/story/ai-connections-provider-routing-00-overview--start-here" target="_top">
Review the proposed provider routing flows: onboarding, connections, agents, management, and recovery
</a>
<p className="text-sm text-muted-foreground">
Milestone 1: shared UI, simulated accounts, no live authentication.
Start with the existing Connectors page, then account details, provider
ui/storybook/stories/local-ai-subscription.stories.tsx
+61
View File
@@ -0,0 +1,61 @@
import type { Meta, StoryObj } from "@storybook/react-vite";
import { expect, userEvent, waitFor, within } from "storybook/test";
import { LocalProviderLoginInstructions } from "@/components/AdapterLoginChrome";
const meta = {
title: "AI Connections/Provider routing/07 Subscription sign-in",
component: LocalProviderLoginInstructions,
parameters: { layout: "centered" },
} satisfies Meta<typeof LocalProviderLoginInstructions>;
export default meta;
type Story = StoryObj<typeof meta>;
export const ClaudeBrowserCode: Story = {
args: {
adapterType: "claude_local",
login: {
isolated: true, preparing: false, status: "sign_in_required", error: null,
authorizationUrl: "#storybook-provider-simulator",
submitCode: async () => {}, retry: () => {},
},
},
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await waitFor(() => expect(canvas.getByRole("link", { name: "Sign in to Claude" })).toBeVisible());
await waitFor(() => expect(canvas.getByLabelText("Authorization code")).toBeVisible());
await expect(canvas.queryByText(/claude auth login|CLAUDE_CONFIG_DIR/)).not.toBeInTheDocument();
await userEvent.type(canvas.getByLabelText("Authorization code"), "fixture-code");
await userEvent.click(canvas.getByRole("button", { name: "Submit code" }));
await expect(canvas.getByLabelText("Authorization code")).toHaveValue("");
},
};
export const CodexDeviceCode: Story = {
args: {
adapterType: "codex_local",
login: {
isolated: true, preparing: false, status: "sign_in_required", error: null,
authorizationUrl: "#storybook-provider-simulator", code: "ABCD-12345",
retry: () => {},
},
},
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await waitFor(() => expect(canvas.getByRole("link", { name: "Sign in to OpenAI" })).toBeVisible());
await waitFor(() => expect(canvas.getByText("ABCD-12345")).toBeVisible());
await expect(canvas.queryByText(/codex login|CODEX_HOME/)).not.toBeInTheDocument();
},
};
export const Connected: Story = {
args: { adapterType: "codex_local", login: { isolated: true, preparing: false, status: "ready", error: null, retry: () => {} } },
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await expect(canvas.getByRole("status")).toHaveTextContent("is signed in");
await userEvent.click(canvas.getByRole("button", { name: "Use a different account" }));
},
};
export const Expired: Story = {
args: { adapterType: "claude_local", login: { isolated: true, preparing: false, status: "expired", error: "This sign-in attempt expired. Start sign-in again.", retry: () => {} } },
};
ui/storybook/stories/provider-routing-00-overview.stories.tsx
+19
View File
@@ -0,0 +1,19 @@
import type { Meta, StoryObj } from "@storybook/react-vite";
import { ReviewIndex } from "../prototypes/provider-routing/ReviewIndex";
import { ReviewFrame } from "../prototypes/provider-routing/shared";
import { reviewLifecycle } from "../prototypes/provider-routing/story-support";
const meta = {
title: "AI Connections/Provider routing/00 Overview",
component: ReviewIndex,
parameters: { layout: "fullscreen" },
...reviewLifecycle,
decorators: [
(Story) => (
<ReviewFrame location="Review map and scope; this page has no production app location.">
<Story />
</ReviewFrame>
),
],
} satisfies Meta<typeof ReviewIndex>;
export default meta;
export const StartHere: StoryObj<typeof meta> = {};
ui/storybook/stories/provider-routing-01-onboarding.stories.tsx
+93
View File
@@ -0,0 +1,93 @@
import type { Meta, StoryObj } from "@storybook/react-vite";
import { expect, userEvent, within } from "storybook/test";
import { Onboarding } from "../prototypes/provider-routing/AgentSetup";
import { ReviewFrame } from "../prototypes/provider-routing/shared";
import { reviewLifecycle } from "../prototypes/provider-routing/story-support";
const meta = {
title: "AI Connections/Provider routing/01 Onboarding",
component: Onboarding,
parameters: { layout: "fullscreen" },
...reviewLifecycle,
decorators: [
(Story) => (
<ReviewFrame location="First onboarding → connect provider, after choosing the harness.">
<Story />
</ReviewFrame>
),
],
render: (args) => <Onboarding key={JSON.stringify(args)} {...args} />,
} satisfies Meta<typeof Onboarding>;
export default meta;
type Story = StoryObj<typeof meta>;
export const DefaultPath: Story = {
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await expect(
canvas.getByRole("button", { name: "Sign in to OpenAI" }),
).toBeVisible();
await expect(
canvas.queryByRole("button", { name: "Use another provider or gateway" }),
).not.toBeInTheDocument();
await expect(canvas.queryByLabelText("Base URL")).not.toBeInTheDocument();
await expect(
canvas.queryByText("Use your ChatGPT subscription to get started."),
).not.toBeInTheDocument();
},
};
export const HappyPathWalkthrough: Story = {
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.click(
canvas.getByRole("button", { name: "Sign in to OpenAI" }),
);
await expect(
canvas.getByRole("combobox", { name: "Connection" }),
).toHaveTextContent("My ChatGPT subscription");
await expect(
canvas.getByRole("combobox", { name: "Connection" }),
).toBeVisible();
await expect(
canvas.queryByRole("region", { name: "Connection" }),
).not.toBeInTheDocument();
await expect(
canvas.queryByRole("button", {
name: "Connect another provider or gateway",
}),
).not.toBeInTheDocument();
await userEvent.click(canvas.getByRole("button", { name: "Run test" }));
await expect(
await canvas.findByText("Connection successful"),
).toBeVisible();
await userEvent.click(canvas.getByRole("button", { name: "Finish setup" }));
await expect(canvas.getByText(/Nova’s configuration saved/)).toBeVisible();
},
};
export const ReuseSavedConnection: Story = { args: { savedConnection: true } };
export const OtherProvider: Story = { args: { initialAlternate: true } };
export const Mobile: Story = {
globals: { viewport: { value: "mobile", isRotated: false } },
};
export const OptionalProviderWalkthrough: Story = {
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.click(canvas.getByRole("button", { name: "Advanced" }));
await userEvent.click(
canvas.getByRole("button", { name: "Use another provider or gateway" }),
);
await userEvent.click(canvas.getByRole("button", { name: "OpenRouter" }));
await userEvent.type(canvas.getByLabelText("API key"), "storybook-example");
await userEvent.click(canvas.getByRole("button", { name: "Connect" }));
await userEvent.click(
canvas.getByRole("button", { name: "Use connection" }),
);
await expect(
canvas.getByRole("group", { name: "Harness selection" }),
).toHaveTextContent("Codex");
await expect(
canvas.getByRole("combobox", { name: "Connection" }),
).toHaveTextContent("Company OpenRouter");
await expect(
canvas.getByRole("button", { name: "Advanced model settings" }),
).toHaveAttribute("aria-expanded", "false");
},
};
ui/storybook/stories/provider-routing-02-connect.stories.tsx
+116
View File
@@ -0,0 +1,116 @@
import type { Meta, StoryObj } from "@storybook/react-vite";
import { expect, userEvent, within } from "storybook/test";
import { ProviderCatalogReview } from "../prototypes/provider-routing/ConnectionCatalog";
import { ConnectionSetup } from "../prototypes/provider-routing/ConnectionSetup";
import { ReviewFrame } from "../prototypes/provider-routing/shared";
import {
choose,
reviewLifecycle,
} from "../prototypes/provider-routing/story-support";
const meta = {
title: "AI Connections/Provider routing/02 Connect",
component: ConnectionSetup,
parameters: { layout: "fullscreen" },
...reviewLifecycle,
decorators: [
(Story) => (
<ReviewFrame location="Apps → Connectors → Connect on a provider row. Permissions default to everyone and all agents under Advanced.">
<Story />
</ReviewFrame>
),
],
render: (args) => <ConnectionSetup key={JSON.stringify(args)} {...args} />,
} satisfies Meta<typeof ConnectionSetup>;
export default meta;
type Story = StoryObj<typeof meta>;
export const ChooseProvider: Story = {
name: "Connector catalog",
render: () => <ProviderCatalogReview />,
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await expect(canvas.queryByText("Connect a model provider")).not.toBeInTheDocument();
for (const slug of ["openrouter", "bedrock", "responses-api", "messages-api", "chat-completions-api", "local"]) {
const row = canvasElement.querySelector(`[data-app-slug="${slug}"]`)! as HTMLElement;
await expect(within(row).getByRole("button", { name: /^Connect / })).toBeVisible();
}
},
};
export const Access: Story = { name: "Permissions defaults", args: { initialStep: "access" } };
export const SharedAccess: Story = {
...Access,
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.click(canvas.getByRole("button", { name: "Change" }));
await expect(
canvas.getByRole("radio", { name: "Any human in the organization" }),
).toBeChecked();
},
};
export const OpenRouterKey: Story = { args: { initialStep: "connect" } };
export const BedrockApiKey: Story = {
args: { initialStep: "connect", initialProvider: "bedrock" },
};
export const BedrockEnvironmentIdentity: Story = {
args: {
initialStep: "connect",
initialProvider: "bedrock",
initialMethod: "identity",
},
};
export const GoogleVertexIdentity: Story = {
args: {
initialStep: "connect",
initialProvider: "google",
initialMethod: "identity",
},
};
export const CustomResponses: Story = {
args: { initialStep: "connect", initialProvider: "custom" },
};
export const CustomAnthropic: Story = {
args: {
initialStep: "connect",
initialProvider: "custom",
initialProtocol: "messages",
},
};
export const CustomHeader: Story = {
args: {
initialStep: "connect",
initialProvider: "custom",
initialMethod: "header",
},
};
export const LocalEndpoint: Story = {
args: {
initialStep: "connect",
initialProvider: "custom",
initialMethod: "none",
initialProtocol: "chat",
},
};
export const InvalidCredential: Story = {
args: { initialStep: "connect", initialError: "credential" },
};
export const MobileBedrock: Story = {
...BedrockApiKey,
globals: { viewport: { value: "mobile", isRotated: false } },
};
export const CustomEndpointWalkthrough: Story = {
...CustomResponses,
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.type(
canvas.getByLabelText("Base URL"),
"https://models.example.com/v1",
);
await userEvent.type(canvas.getByLabelText("API key"), "storybook-example");
await userEvent.click(canvas.getByRole("button", { name: /^Connect$/ }));
await expect(
canvas.getByRole("heading", { name: "Connection ready" }),
).toBeVisible();
await expect(
canvas.queryByDisplayValue("storybook-example"),
).not.toBeInTheDocument();
},
};
ui/storybook/stories/provider-routing-03-agent.stories.tsx
+325
View File
@@ -0,0 +1,325 @@
import type { Meta, StoryObj } from "@storybook/react-vite";
import { expect, userEvent, waitFor, within } from "storybook/test";
import { AgentSetup } from "../prototypes/provider-routing/AgentSetup";
import { ReviewFrame } from "../prototypes/provider-routing/shared";
import {
choose,
reviewLifecycle,
} from "../prototypes/provider-routing/story-support";
const meta = {
title: "AI Connections/Provider routing/03 Agent",
component: AgentSetup,
parameters: { layout: "fullscreen" },
...reviewLifecycle,
decorators: [
(Story) => (
<ReviewFrame location="Agents → Nova → Harness / Runtime (/agents/nova/runtime). Existing agent page and sidebar; proposed connection routing fields. The same fields are embedded in Create agent → Configure.">
<Story />
</ReviewFrame>
),
],
render: (args) => <AgentSetup key={JSON.stringify(args)} {...args} />,
} satisfies Meta<typeof AgentSetup>;
export default meta;
type Story = StoryObj<typeof meta>;
export const CodexNewRunner: Story = {
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await expect(
canvas.getByRole("combobox", { name: "Connection" }),
).toHaveTextContent("My ChatGPT subscription");
await expect(
canvas.queryByRole("option", { name: "Engineering gateway" }),
).not.toBeInTheDocument();
await expect(
canvas.queryByText("https://models.example.com/v1"),
).not.toBeInTheDocument();
await userEvent.click(
within(canvas.getByRole("group", { name: "Model selection" })).getByRole(
"button",
{ name: "gpt-5.4" },
),
);
const body = within(canvasElement.ownerDocument.body);
await expect(
await body.findByPlaceholderText("Search models..."),
).toBeVisible();
await expect(
body.queryByPlaceholderText("Search models... (type to create)"),
).not.toBeInTheDocument();
await userEvent.keyboard("{Escape}");
},
};
export const ClaudeDefault: Story = { args: { harness: "claude" } };
export const AdvancedProviders: Story = {
name: "Advanced model settings",
args: { initialAdvanced: true },
};
export const ChangeConnection: Story = {
name: "Connection dropdown",
play: async ({ canvasElement }) => {
await userEvent.click(
within(canvasElement).getByRole("combobox", {
name: "Connection",
}),
);
const options = within(
await within(canvasElement.ownerDocument.body).findByRole("listbox"),
);
await expect(
options.getByRole("option", { name: "My ChatGPT subscription" }),
).toHaveAttribute("aria-selected", "true");
await expect(
options.getByRole("option", { name: "Company OpenAI" }),
).toBeVisible();
await expect(
options.getByRole("option", { name: "Engineering gateway" }),
).toBeVisible();
await expect(
options.getByRole("option", { name: "Company OpenRouter" }),
).toBeVisible();
await expect(
options.getByRole("option", { name: "Company Bedrock" }),
).toBeVisible();
await expect(
options.queryByRole("option", { name: "My Claude subscription" }),
).not.toBeInTheDocument();
await expect(
options.queryByRole("option", { name: "Local model service" }),
).not.toBeInTheDocument();
},
};
export const ChangeAccountWalkthrough: Story = {
args: { initialTest: "pass" },
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
const body = within(canvasElement.ownerDocument.body);
const change = canvas.getByRole("combobox", { name: "Connection" });
await userEvent.click(change);
await body.findByRole("listbox");
await userEvent.keyboard("{Escape}");
await waitFor(() => expect(change).toHaveFocus());
await expect(
canvas.getByRole("button", { name: "Save changes" }),
).toBeEnabled();
await userEvent.click(change);
await userEvent.click(
within(await body.findByRole("listbox")).getByRole("option", {
name: "Company OpenAI",
}),
);
await waitFor(() =>
expect(body.queryByRole("listbox")).not.toBeInTheDocument(),
);
await expect(
canvas.getByRole("combobox", { name: "Connection" }),
).toHaveTextContent("Company OpenAI");
await expect(
canvas.getByRole("group", { name: "Model selection" }),
).toHaveTextContent("gpt-5.4");
await expect(
canvas.getByRole("button", { name: "Save changes" }),
).toBeDisabled();
await expect(
canvas.getByRole("button", { name: "Advanced model settings" }),
).toHaveAttribute("aria-expanded", "false");
await expect(
canvas.queryByText("https://api.openai.com/v1"),
).not.toBeInTheDocument();
await waitFor(() => expect(change).toHaveFocus());
},
};
export const HarnessConnectionFiltering: Story = {
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
const body = within(canvasElement.ownerDocument.body);
await userEvent.click(
within(
canvas.getByRole("group", { name: "Harness selection" }),
).getByRole("button", { name: "Codex" }),
);
await userEvent.click(
await body.findByRole("button", { name: "Claude Code" }),
);
await userEvent.click(canvas.getByRole("combobox", { name: "Connection" }));
const options = within(await body.findByRole("listbox"));
await expect(
options.getByRole("option", { name: "My Claude subscription" }),
).toBeVisible();
await expect(
options.getByRole("option", { name: "Company OpenRouter" }),
).toBeVisible();
await expect(
options.getByRole("option", { name: "Company Bedrock" }),
).toBeVisible();
for (const name of [
"My ChatGPT subscription",
"Company OpenAI",
"Engineering gateway",
"Company Gemini",
"Local model service",
]) {
await expect(
options.queryByRole("option", { name }),
).not.toBeInTheDocument();
}
await userEvent.click(
options.getByRole("option", { name: "My Claude subscription" }),
);
await userEvent.click(
within(canvas.getByRole("group", { name: "Model selection" })).getByRole(
"button",
{ name: "gpt-5.4" },
),
);
await userEvent.click(
await body.findByRole("button", { name: "claude-sonnet-4-6" }),
);
await expect(canvas.queryByRole("alert")).not.toBeInTheDocument();
},
};
export const ConfiguredGateway: Story = {
args: { initialConnectionId: "gateway" },
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await expect(
canvas.getByRole("combobox", { name: "Connection" }),
).toHaveTextContent("Engineering gateway");
await expect(
canvas.getByRole("group", { name: "Model selection" }),
).toHaveTextContent("engineering-coder");
await expect(
canvas.queryByText("https://models.example.com/v1"),
).not.toBeInTheDocument();
await expect(
canvas.getByRole("button", { name: "Advanced model settings" }),
).toHaveAttribute("aria-expanded", "false");
},
};
export const CodexLegacy: Story = { args: { runner: "legacy" } };
export const ClaudeBedrock: Story = {
args: { harness: "claude", initialConnectionId: "bedrock" },
};
export const OpenCodeNewRunner: Story = { args: { harness: "opencode" } };
export const GrokNewRunner: Story = { args: { harness: "grok" } };
export const PiLegacy: Story = { args: { harness: "pi", runner: "legacy" } };
export const GeminiLegacy: Story = {
args: { harness: "gemini", runner: "legacy", initialConnectionId: "gemini" },
};
export const KimiLegacy: Story = {
args: { harness: "kimi", runner: "legacy" },
};
export const HermesLocal: Story = {
args: { harness: "hermes", runner: "legacy", initialConnectionId: "chat" },
};
export const CursorUnverified: Story = {
args: { harness: "cursor", runner: "legacy" },
};
export const NativePiPending: Story = { args: { harness: "pi" } };
export const NativeCopilotPending: Story = { args: { harness: "copilot" } };
export const NewAgent: Story = { args: { newAgent: true } };
export const NoConnections: Story = { args: { scenario: "empty" } };
export const Loading: Story = { args: { scenario: "loading" } };
export const ModelDiscoveryFailed: Story = {
args: { scenario: "catalog-error" },
};
export const IncompatibleProtocol: Story = {
args: { initialConnectionId: "chat" },
};
export const MissingPersonalCredential: Story = {
args: { initialConnectionId: "gateway", scenario: "missing-personal" },
};
export const SharedAccessDenied: Story = {
args: { scenario: "denied", initialConnectionId: "router" },
};
export const ReadOnly: Story = { args: { scenario: "readonly" } };
export const AdoptLegacySettings: Story = {
args: { runner: "legacy", scenario: "legacy" },
};
export const TestRunning: Story = { args: { initialTest: "running" } };
export const TestPassed: Story = { args: { initialTest: "pass" } };
export const ToolSupportFailed: Story = {
args: { initialTest: "fail", testFailure: "tools" },
};
export const EnvironmentUnreachable: Story = {
args: { initialTest: "fail", testFailure: "network" },
};
export const Mobile: Story = {
globals: { viewport: { value: "mobile", isRotated: false } },
};
export const ChangeConnectionWalkthrough: Story = {
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await choose(canvasElement, "Connection", "Engineering gateway");
const body = within(canvasElement.ownerDocument.body);
await waitFor(() =>
expect(body.queryByRole("listbox")).not.toBeInTheDocument(),
);
await expect(
canvas.getByRole("group", { name: "Harness selection" }),
).toHaveTextContent("Codex");
await expect(canvas.getByRole("alert")).toHaveTextContent(
"previous model is unavailable",
);
await expect(
canvas.getByRole("button", { name: "Save changes" }),
).toBeDisabled();
await userEvent.click(
within(canvas.getByRole("group", { name: "Model selection" })).getByRole(
"button",
{ name: "gpt-5.4" },
),
);
await userEvent.click(
await within(canvasElement.ownerDocument.body).findByRole("button", {
name: "engineering-coder",
}),
);
await userEvent.click(canvas.getByRole("button", { name: "Run test" }));
await expect(
await canvas.findByText("Connection successful"),
).toBeVisible();
await userEvent.click(canvas.getByRole("button", { name: "Save changes" }));
await expect(canvas.getByText(/Nova’s configuration saved/)).toBeVisible();
await choose(canvasElement, "Execution environment", "Local machine");
await expect(
canvas.getByRole("button", { name: "Save changes" }),
).toBeDisabled();
},
};
export const CustomModelAlias: Story = {
args: { initialConnectionId: "gateway" },
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.click(
canvas.getByRole("button", { name: "Advanced model settings" }),
);
await userEvent.click(
within(canvas.getByRole("group", { name: "Model selection" })).getByRole(
"button",
{ name: "engineering-coder" },
),
);
const body = within(canvasElement.ownerDocument.body);
await userEvent.type(
await body.findByPlaceholderText("Search models... (type to create)"),
"team-reviewer",
);
await userEvent.click(body.getByRole("button", { name: /team-reviewer/ }));
await expect(
canvas.getByRole("group", { name: "Model selection" }),
).toHaveTextContent("team-reviewer");
await userEvent.click(
canvas.getByRole("button", { name: "Advanced model settings" }),
);
await expect(
canvas.getByRole("group", { name: "Model selection" }),
).toHaveTextContent("team-reviewer");
await expect(
canvas.getByRole("combobox", { name: "Connection" }),
).toHaveTextContent("Engineering gateway");
await expect(
canvas.queryByRole("region", { name: "Connection" }),
).not.toBeInTheDocument();
},
};
ui/storybook/stories/provider-routing-04-manage.stories.tsx
+115
View File
@@ -0,0 +1,115 @@
import type { Meta, StoryObj } from "@storybook/react-vite";
import { expect, userEvent, within } from "storybook/test";
import { Management } from "../prototypes/provider-routing/Management";
import { ReviewFrame } from "../prototypes/provider-routing/shared";
import { reviewLifecycle } from "../prototypes/provider-routing/story-support";
const meta = {
title: "AI Connections/Provider routing/04 Manage",
component: Management,
parameters: { layout: "fullscreen" },
...reviewLifecycle,
decorators: [
(Story) => (
<ReviewFrame location="Connectors (/apps) → provider account → connection details, models, and access. Uses the existing provider cards and account rows.">
<Story />
</ReviewFrame>
),
],
render: (args) => <Management key={JSON.stringify(args)} {...args} />,
} satisfies Meta<typeof Management>;
export default meta;
type Story = StoryObj<typeof meta>;
export const ConnectionList: Story = {
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await expect(
canvas.getByRole("button", {
name: "Open My ChatGPT subscription permissions",
}),
).toBeVisible();
await expect(
canvas.queryByRole("heading", { name: "Custom provider or gateway" }),
).not.toBeInTheDocument();
},
};
export const WithAdvancedConnections: Story = {
args: { includeAdvancedConnections: true },
};
export const ConnectionDetails: Story = { args: { initialView: "details" } };
export const ModelsAndHelperDefaults: Story = {
args: { initialView: "details", initialTab: "models" },
};
export const Access: Story = {
args: { initialView: "details", initialTab: "access" },
};
export const ReadOnly: Story = {
args: { initialView: "details", readOnly: true },
};
export const MobileModels: Story = {
...ModelsAndHelperDefaults,
globals: { viewport: { value: "mobile", isRotated: false } },
};
export const ReplaceCredential: Story = {
...ConnectionDetails,
play: async ({ canvasElement }) => {
await userEvent.click(
within(canvasElement).getByRole("button", { name: "Replace credential" }),
);
},
};
export const NewEndpoint: Story = {
...ConnectionDetails,
play: async ({ canvasElement }) => {
await userEvent.click(
within(canvasElement).getByRole("button", {
name: "Advanced connection settings",
}),
);
await userEvent.click(
within(canvasElement).getByRole("button", {
name: "Use a different endpoint",
}),
);
await expect(
within(canvasElement).getByRole("heading", {
name: "Connect Custom provider or gateway",
}),
).toBeVisible();
},
};
export const AddModelWalkthrough: Story = {
...ModelsAndHelperDefaults,
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.type(
canvas.getByLabelText("Model ID or alias"),
"team-reviewer",
);
await userEvent.click(canvas.getByRole("button", { name: "Add model" }));
await expect(
canvas.getByText("team-reviewer", { exact: true }),
).toBeVisible();
await userEvent.click(canvas.getByRole("button", { name: "Save changes" }));
await expect(canvas.getByRole("status")).toHaveTextContent(
"Connection settings saved",
);
},
};
export const RevokeCredential: Story = {
...Access,
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.click(
canvas.getByRole("button", { name: "Revoke credential" }),
);
const dialog = within(
await within(canvasElement.ownerDocument.body).findByRole("dialog"),
);
await userEvent.click(
dialog.getByRole("button", { name: "Revoke credential" }),
);
await expect(
canvas.getByText(/Credential revoked. Reconnect/),
).toBeVisible();
},
};
ui/storybook/stories/provider-routing-05-recovery.stories.tsx
+71
View File
@@ -0,0 +1,71 @@
import type { Meta, StoryObj } from "@storybook/react-vite";
import { expect, userEvent, within } from "storybook/test";
import { Recovery } from "../prototypes/provider-routing/Recovery";
import { ReviewFrame } from "../prototypes/provider-routing/shared";
import { reviewLifecycle } from "../prototypes/provider-routing/story-support";
const meta = {
title: "AI Connections/Provider routing/05 Recovery",
component: Recovery,
parameters: { layout: "fullscreen" },
...reviewLifecycle,
decorators: [
(Story) => (
<ReviewFrame location="Task thread → existing connection request card → inline Fix connection. Reuses ConnectionIntentInteractionBody and its completion/focus behavior; the gateway credential fields are proposed.">
<Story />
</ReviewFrame>
),
],
render: (args) => <Recovery key={JSON.stringify(args)} {...args} />,
} satisfies Meta<typeof Recovery>;
export default meta;
type Story = StoryObj<typeof meta>;
export const ExpiredCredential: Story = {};
export const MissingPersonalCredential: Story = {
args: { problem: "missing" },
};
export const ProtocolMismatch: Story = { args: { problem: "protocol" } };
export const ProviderQuota: Story = { args: { problem: "quota" } };
export const Mobile: Story = {
globals: { viewport: { value: "mobile", isRotated: false } },
};
export const ReconnectWalkthrough: Story = {
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.click(
canvas.getByRole("button", { name: "Fix connection" }),
);
const dialog = within(
await canvas.findByRole("region", { name: "Gateway credential repair" }),
);
await userEvent.type(dialog.getByLabelText("API key"), "invalid");
await userEvent.click(dialog.getByRole("button", { name: "Connect" }));
await expect(dialog.getByRole("alert")).toHaveTextContent("rejected");
await expect(dialog.getByLabelText("API key")).toHaveValue("");
await userEvent.type(dialog.getByLabelText("API key"), "storybook-example");
await userEvent.click(dialog.getByRole("button", { name: "Connect" }));
await expect(
await canvas.findByText("Engineering gateway connected", { exact: true }),
).toBeVisible();
},
};
export const CancelRestoresFocus: Story = {
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
const trigger = canvas.getByRole("button", { name: "Fix connection" });
await userEvent.click(trigger);
const dialog = within(
await canvas.findByRole("region", { name: "Gateway credential repair" }),
);
await userEvent.type(
dialog.getByLabelText("API key"),
"discard-this-example",
);
await userEvent.click(dialog.getByRole("button", { name: "Cancel" }));
await expect(
canvas.getByTestId("connection-intent-focus-target"),
).toHaveFocus();
await expect(
canvas.queryByDisplayValue("discard-this-example"),
).not.toBeInTheDocument();
},
};
ui/storybook/stories/provider-routing-06-production.stories.tsx
+274
View File
@@ -0,0 +1,274 @@
import { useState } from "react";
import type { Meta, StoryObj } from "@storybook/react-vite";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { AiProviderSetup } from "@/components/ai-connections/AiProviderSetup";
import { AiConnectionSelect } from "@/components/ai-connections/AiConnectionSelect";
import { AiConnectionField } from "@/components/ai-connections/AiConnectionField";
import { AgentProviderConnection, type ProviderConnection } from "@/components/new-agent/AgentProviderConnection";
import { useConnectionModels } from "@/components/ai-connections/useConnectionModels";
import { ModelDropdown } from "@/components/AgentConfigForm";
import { Field } from "@/components/agent-config-primitives";
import { NativeSelect } from "@/components/ui/select";
import { queryKeys } from "@/lib/queryKeys";
import type {
AiConnectionBinding,
AiManagedConnectionSummary,
} from "@paperclipai/shared";
import { expect, userEvent, waitFor, within } from "storybook/test";
import { ProviderCatalogReview } from "../prototypes/provider-routing/ConnectionCatalog";
import { ReviewFrame } from "../prototypes/provider-routing/shared";
const companyId = "00000000-0000-4000-8000-000000000001";
const accounts: AiManagedConnectionSummary[] = [
{
id: "00000000-0000-4000-8000-000000000002",
grantId: "00000000-0000-4000-8000-000000000003",
companyId,
provider: "openai",
method: "subscription",
name: "My ChatGPT subscription",
ownership: "personal",
ownerUserId: "you",
isDefault: true,
status: "connected",
},
{
id: "00000000-0000-4000-8000-000000000004",
grantId: "00000000-0000-4000-8000-000000000005",
companyId,
provider: "openrouter",
method: "api_key",
name: "Company OpenRouter",
ownership: "shared",
isDefault: false,
status: "connected",
routing: {
kind: "openrouter",
protocol: "responses",
auth: "bearer",
models: [{ id: "openai/gpt-5.4" }],
},
},
{
id: "00000000-0000-4000-8000-000000000006",
grantId: "00000000-0000-4000-8000-000000000007",
companyId,
provider: "anthropic",
method: "subscription",
name: "My Claude subscription",
ownership: "personal",
ownerUserId: "you",
isDefault: true,
status: "connected",
},
];
function Providers({ canManageConnections = true }: { canManageConnections?: boolean }) {
const [client] = useState(() => {
const query = new QueryClient({
defaultOptions: { queries: { retry: false, staleTime: Infinity } },
});
query.setQueryData(
["agents", companyId, "provider-access"],
[{ id: "nova", name: "Nova" }],
);
query.setQueryData(["ai-connections", companyId, "nova"], { currentUserId: "you", connections: accounts, canManageConnections });
return query;
});
return (
<QueryClientProvider client={client}>
<AiProviderSetup
companyId={companyId}
agentId="nova"
initialProvider="openrouter"
onCancel={() => {}}
onComplete={() => {}}
/>
</QueryClientProvider>
);
}
function Connection() {
const [value, setValue] = useState<AiConnectionBinding>({
provider: "openai",
method: "subscription",
mode: "responsible_user",
});
return (
<AiConnectionSelect
requirement={{ companyId, provider: "openai" }}
adapterType="codex_local"
connections={accounts}
value={value}
onChange={setValue}
onConnect={() => {}}
currentUserId="you"
agentId="nova"
agentName="Nova"
/>
);
}
function OpenRouterModelPicker() {
const [open, setOpen] = useState(false);
const [value, setValue] = useState("");
const [effort, setEffort] = useState("");
const connection = useConnectionModels(companyId, {
provider: "openrouter", method: "api_key", mode: "shared",
connectionId: accounts[1].id, grantId: accounts[1].grantId,
}, "opencode_local");
return <div className="grid gap-5 sm:grid-cols-2">
<ModelDropdown presentation="native" models={connection?.models ?? []} value={value} onChange={setValue}
open={open} onOpenChange={setOpen} allowDefault={false} required groupByProvider={false}
preserveOrder creatable loadingModels={connection?.isLoading} />
<Field label="Thinking effort"><NativeSelect aria-label="Thinking effort" value={effort} onChange={event => setEffort(event.target.value)}>
<option value="">Auto</option><option value="high">high</option>
</NativeSelect></Field>
</div>;
}
function OpenRouterModels() {
const [client] = useState(() => {
const query = new QueryClient({ defaultOptions: { queries: { staleTime: Infinity, retry: false } } });
query.setQueryData(["ai-connections", companyId], {
currentUserId: "you",
connections: [{ ...accounts[1], routing: { ...accounts[1].routing!, models: [] } }],
});
query.setQueryData(queryKeys.agents.adapterModels(companyId, "opencode_local", null, "openrouter"), [
{ id: "openrouter/z-ai/glm-5", label: "Z.AI GLM 5" },
{ id: "openrouter/anthropic/claude-sonnet-4.5", label: "Claude Sonnet 4.5" },
{ id: "openrouter/openai/gpt-5.4", label: "OpenAI GPT-5.4" },
]);
return query;
});
return <QueryClientProvider client={client}><OpenRouterModelPicker /></QueryClientProvider>;
}
function AgentConnectionModes() {
const [value, setValue] = useState<AiConnectionBinding>();
const [connected, setConnected] = useState<ProviderConnection>();
const [client] = useState(() => {
const query = new QueryClient({ defaultOptions: { queries: { staleTime: Infinity, retry: false } } });
const connections = { currentUserId: "you", connections: accounts, canManageConnections: true };
query.setQueryData(["ai-connections", companyId], connections);
query.setQueryData(["ai-connections", companyId, undefined], connections);
query.setQueryData(queryKeys.secrets.myUserSecrets(companyId), []);
query.setQueryData(queryKeys.secrets.list(companyId), []);
query.setQueryData(queryKeys.agents.authSignal(companyId, "codex_local", null), { status: "present" });
return query;
});
return <QueryClientProvider client={client}>
<AgentProviderConnection companyId={companyId} adapterType="codex_local" environmentId={null}
canLogin={false} onBack={() => {}} testConnection={async () => true} onConnected={setConnected}
advancedConnection={{ value, content: <AiConnectionField companyId={companyId} agentName="Nova"
adapterType="codex_local" value={value} onChange={binding => { if (binding.mode !== "router") setValue(binding); }} preferAdvanced /> }} />
{connected && <p role="status">Continue to Configure with {connected.aiConnection?.provider}.</p>}
</QueryClientProvider>;
}
const meta = {
title: "AI Connections/Provider routing/06 Production components",
parameters: { layout: "fullscreen" },
decorators: [
(Story) => (
<ReviewFrame location="Production components: Connectors → provider row → Connect; Agents → Harness / Runtime. Fixtures supply data only.">
<Story />
</ReviewFrame>
),
],
} satisfies Meta;
export default meta;
type Story = StoryObj<typeof meta>;
export const NewAgentConnectionModes: Story = {
name: "New agent connection modes",
parameters: { docs: { description: { story: "Agents → New agent → Connect. Three persistent production tiles switch between saved subscriptions, API keys, and advanced connections. Environment selection belongs to Configure." } } },
render: () => <AgentConnectionModes />,
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await expect(canvas.getAllByRole("radio")).toHaveLength(3);
await expect(canvas.getByRole("radio", { name: "OpenAI Subscription" })).toBeChecked();
await expect(canvas.getByRole("combobox", { name: "Saved subscription" })).toBeVisible();
await userEvent.click(canvas.getByRole("radio", { name: "OpenAI API key" }));
await waitFor(() => expect(canvas.getByPlaceholderText("Enter API key here")).toBeVisible());
await userEvent.click(canvas.getByRole("radio", { name: "Advanced Custom Gateway" }));
await expect(canvas.queryByPlaceholderText("Enter API key here")).not.toBeInTheDocument();
await expect(canvas.getAllByRole("radio")).toHaveLength(3);
await expect(canvas.getByRole("radio", { name: "Advanced Custom Gateway" })).toBeChecked();
await expect(canvas.getByRole("button", { name: "Use connection" })).toBeDisabled();
await userEvent.click(canvas.getByRole("combobox", { name: "Connection" }));
await userEvent.click(within(canvasElement.ownerDocument.body).getByRole("option", { name: "Company OpenRouter" }));
await userEvent.click(canvas.getByRole("button", { name: "Use connection" }));
await expect(canvas.getByRole("status")).toHaveTextContent("Continue to Configure with openrouter.");
await userEvent.click(canvas.getByRole("radio", { name: "OpenAI Subscription" }));
await expect(canvas.getAllByRole("radio")).toHaveLength(3);
await expect(canvas.getByRole("radio", { name: "OpenAI Subscription" })).toBeChecked();
await expect(canvas.getByRole("combobox", { name: "Saved subscription" })).toBeVisible();
await expect(canvas.queryByRole("combobox", { name: "Connection" })).not.toBeInTheDocument();
},
};
export const OpenRouterPopularModels: Story = {
name: "OpenRouter popular models",
parameters: { docs: { description: { story: "Production model picker and connection discovery hook. The connection has no custom model list; the public catalog fixture supplies models in popularity order." } } },
render: () => <OpenRouterModels />,
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
const model = canvas.getByRole("combobox", { name: "Model" });
const labels = within(model).getAllByRole("option").filter(option => (option as HTMLOptionElement).value.startsWith("openrouter/")).map(option => option.textContent);
await expect(labels).toEqual(["Z.AI GLM 5", "Claude Sonnet 4.5", "OpenAI GPT-5.4"]);
await userEvent.selectOptions(model, "openrouter/anthropic/claude-sonnet-4.5");
await expect(model).toHaveValue("openrouter/anthropic/claude-sonnet-4.5");
await userEvent.selectOptions(canvas.getByRole("combobox", { name: "Thinking effort" }), "high");
await userEvent.selectOptions(model, within(model).getByRole("option", { name: "Enter custom model…" }));
await userEvent.clear(canvas.getByRole("textbox", { name: "Model ID" }));
await userEvent.type(canvas.getByRole("textbox", { name: "Model ID" }), "my-gateway/model-alias");
await expect(canvas.getByRole("textbox", { name: "Model ID" })).toHaveValue("my-gateway/model-alias");
await userEvent.selectOptions(model, "openrouter/anthropic/claude-sonnet-4.5");
await expect(canvas.queryByRole("textbox", { name: "Model ID" })).not.toBeInTheDocument();
},
};
export const ChooseProvider: Story = {
name: "Connector catalog",
render: () => <ProviderCatalogReview />,
play: async ({ canvasElement }) => {
const row = canvasElement.querySelector('[data-app-slug="responses-api"]')! as HTMLElement;
await userEvent.click(within(row).getByRole("button", { name: "Connect Responses API" }));
const canvas = within(canvasElement);
await expect(canvas.getByRole("heading", { name: "Connect Responses API" })).toBeVisible();
await expect(canvas.getByText("Connects for everyone in your organization, available to all agents.")).toBeVisible();
await expect(canvas.queryByRole("radio")).not.toBeInTheDocument();
await userEvent.click(canvas.getByRole("button", { name: "Change" }));
await expect(canvas.getByRole("radio", { name: "Any human in the organization" })).toBeChecked();
await expect(canvas.getByRole("radio", { name: "Any agent" })).toBeChecked();
},
};
export const CompatibleConnectionDropdown: Story = {
render: () => <Connection />,
play: async ({ canvasElement }) => {
await expect(
within(canvasElement).getByRole("combobox", { name: "Connection" }),
).toHaveTextContent("Responsible user’s default");
await userEvent.click(
within(canvasElement).getByRole("combobox", { name: "Connection" }),
);
const body = within(canvasElement.ownerDocument.body);
await expect(
body.getByRole("option", { name: "Responsible user’s default" }),
).toHaveAttribute("data-state", "checked");
await expect(
body.getByRole("option", { name: "Company OpenRouter" }),
).toBeVisible();
await expect(
body.queryByRole("option", { name: "My Claude subscription" }),
).not.toBeInTheDocument();
await userEvent.click(
body.getByRole("option", { name: "Company OpenRouter" }),
);
await expect(
within(canvasElement).getByRole("combobox", { name: "Connection" }),
).toHaveTextContent("Company OpenRouter");
},
};
export const MemberProviderAccess: Story = {
render: () => <Providers canManageConnections={false} />,
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.click(canvas.getByRole("button", { name: "Change" }));
await expect(canvas.getByText("Which agents can use this connection?", { exact: true })).toBeVisible();
await expect(canvas.getByRole("radio", { name: "Any human in the organization" })).toBeDisabled();
},
};