From 6ea47e5480ca17e8e634c203964e99d5614db1dc Mon Sep 17 00:00:00 2001 From: Dotta Date: Tue, 13 Oct 2026 01:32:43 -0500 Subject: [PATCH] Qualify managed Claude setup at Connect Co-Authored-By: Paperclip --- doc/plans/2026-10-12-claude-runner-default.md | 46 ++++++ maps/feature-map/onboarding.md | 13 +- maps/storybook-map/INDEX.md | 12 +- maps/storybook-map/REVIEW.md | 25 ++++ maps/storybook-map/catalog.json | 40 +++--- ui/src/components/OnboardingWizard.test.tsx | 135 ++++++++++++++++++ ui/src/components/OnboardingWizard.tsx | 67 +++++++-- ui/src/components/new-agent/NewAgentSetup.tsx | 35 ++++- ui/src/lib/qualify-claude-setup.test.ts | 67 +++++++++ ui/src/lib/qualify-claude-setup.ts | 48 +++++++ ui/src/pages/NewAgent.test-support.tsx | 4 +- ui/src/pages/NewAgent.test.tsx | 81 +++++++++++ 12 files changed, 532 insertions(+), 41 deletions(-) create mode 100644 ui/src/lib/qualify-claude-setup.test.ts create mode 100644 ui/src/lib/qualify-claude-setup.ts diff --git a/doc/plans/2026-10-12-claude-runner-default.md b/doc/plans/2026-10-12-claude-runner-default.md index 3cb9510b293..9b092303e7e 100644 --- a/doc/plans/2026-10-12-claude-runner-default.md +++ b/doc/plans/2026-10-12-claude-runner-default.md @@ -987,3 +987,49 @@ as work product `38b9e957-df5c-416f-b370-d51405a755ed`, attachment `398c24270b7bd1f2e00883ddf9a5132ad642cb38c7647d830c04ad2541845a81`. [Download checkpoint](http://127.0.0.1:3105/api/attachments/194e1903-5bd3-45b3-af94-af6821a6bf13/content?download=1). Upload receipt: `/private/tmp/claude-checkpoint-20261013-upload.json`. + + +## Claude managed Connect qualification — 2026-10-13 + +The root private patch 665a4bda8dd09ef82911e8b384004c29cbe72da9d534781c548a1bc223f159ef +is Codex-only. Claude now has a separate narrow counterpart in the shipped +onboarding wizard and agent setup. New Claude Code with Auto and an explicit +immutable sandbox image uses fresh company/environment discovery; an unqualified +eligible environment is probed once through the existing permission-gated API +at Connect. Discovery is refreshed afterward, and the measured runner controls +both the environment test and saved configuration. Native forms omit the Legacy +turn-limit default; a measured Legacy result retains it. No runner override is +submitted for Auto. + +Existing API-key storage and managed connection bindings remain in use. A new +API key is saved once before qualification; subsequent test/hire payloads carry +the managed binding rather than the raw key. Local execution, saved agents, +explicit runner choices, other harnesses, disabled/overridden adapters, mutable +images and named snapshots do not acquire this additional qualification path. +Named Cloud snapshot identity remains a separate unresolved acceptance boundary. + +The helper validates the existing probe's recorded qualification and lease ID. +Permission, connection, cleanup or missing-identity failures stop setup without +an automatic retry. Generation and selection checks discard held results after +company/environment/configuration changes. The wizard caches measured discovery +after its operation so its own new default does not invalidate the setup; agent +creation recognizes that same measured transition while retaining stale guards. + +Four mounted positive cases fail against the prior components and pass after +the change. Final focused verification passes all 197 tests: 109 wizard, +69 creation and 19 helper controls. The new cases include native and Legacy +measurements, API-key storage, denied probe access and held-result races. +UI noemit, token gates, map contracts, all nine catalog controls and the catalog +checker pass (224 files, 2,223 stable story IDs). The six affected catalog entries +were source-reviewed; no new rendering, play or live managed acceptance is +claimed. Logs: /private/tmp/claude-managed-cache-full-controls.log, +/private/tmp/claude-managed-cache-types.log, +/private/tmp/claude-managed-cache-catalog-controls.log and +/private/tmp/claude-managed-cache-token.log. The before comparison is retained +in /private/tmp/claude-managed-setup-before-controls.log. + +No provider prompts, paid setup retries or local Docker/Rust builds were used. +The live preview remains the unmodified ad5875fa installed package; the runtime +lookup repair and this UI change still need a new cloud-qualified package and +real API task/revision acceptance. Full repository checks and PR review must run +on the reconciled delivery heads before a PR-ready claim. diff --git a/maps/feature-map/onboarding.md b/maps/feature-map/onboarding.md index 5a411e3ebfa..bf07e962605 100644 --- a/maps/feature-map/onboarding.md +++ b/maps/feature-map/onboarding.md @@ -50,7 +50,7 @@ company's onboarding route, and finish; drafts save automatically. Follow the created first task, verify the selected agent and company, and inspect its output or explicit authentication gate. Reload both company and task. -For Codex, choose the harness and leave runner selection automatic to prove the +For Codex or Claude, choose the harness and leave runner selection automatic to prove the first-agent default. On a qualified target it saves Paperclip Runner; Advanced also permits Legacy. Do not switch runners after onboarding when claiming proof of that default. Test the selected runtime and account, then complete a bounded @@ -60,6 +60,17 @@ Use [runner validation](./runs-adapters.md#adapter-setup) and [environment qualification](./execution-environments.md#environment-settings) for the prerequisites; a successful login alone is insufficient. +For a new Claude agent on an explicit immutable sandbox image, use Connect to +qualify the selected environment through the existing permission-gated probe. +Do not preprobe outside onboarding when claiming this first-use path. Fresh +scoped discovery determines the runner for both the account test and hire; +the measured Legacy result remains valid. An instance-admin denial, incomplete +cleanup, missing lease or changed configuration stops setup before the adapter +test or hire. Named snapshots and mutable image tags do not gain qualification +through this flow. [Helper controls](../../ui/src/lib/qualify-claude-setup.test.ts) +and mounted wizard/new-agent tests use mocks; live managed acceptance remains a +separate gate. + Leave and resume during harness/runner setup. Confirm the draft retains the explicit choice and compatible account bindings, old native Codex setup links recover the harness and runner, and a changed harness or runner invalidates stale diff --git a/maps/storybook-map/INDEX.md b/maps/storybook-map/INDEX.md index fe2a093a9f5..220d3b76db1 100644 --- a/maps/storybook-map/INDEX.md +++ b/maps/storybook-map/INDEX.md @@ -31,7 +31,7 @@ Status describes product alignment; implementation describes the rendered subjec | --- | --- | --- | --- | --- | | [Oversight and operation/Dashboards and audit trails/Tools/Activity (audit feed)](../../ui/storybook/stories/activity/activity-feed.stories.tsx) | `activity/activity`, `gateways-profiles/activity` | unreviewed / production / fixtures | 4 / 0 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. | | [Oversight and operation/Dashboards and audit trails/Apps/App detail · Activity (lifecycle)](../../ui/storybook/stories/activity/app-activity-lifecycle.stories.tsx) | `activity/activity`, `gateways-profiles/activity` | unreviewed / production / fixtures | 4 / 0 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. | -| [Oversight and operation/Dashboards and audit trails/Product/Data Visualization & Misc](../../ui/storybook/stories/activity/data-viz-misc.stories.tsx) | `activity/overview` | unreviewed / production / fixtures | 21 / 0 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner display/default/effort and matching-discovery source changes reviewed; existing fixture, proposal and historical scope retained. No new rendering, play, backend or provider qualification is claimed. | +| [Oversight and operation/Dashboards and audit trails/Product/Data Visualization & Misc](../../ui/storybook/stories/activity/data-viz-misc.stories.tsx) | `activity/overview` | unreviewed / production / fixtures | 21 / 0 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner display/default/effort and matching-discovery source changes reviewed; existing fixture, proposal and historical scope retained. No new rendering, play, backend or provider qualification is claimed. Claude explicit-Connect qualification and stale-result guards source-reviewed; existing mock, local and unqualified-image fixture boundaries remain. No new managed live acceptance or rendering is claimed. | | [Oversight and operation/Dashboards and audit trails/Pages/Work Timeline](../../ui/storybook/stories/activity/work-timeline.stories.tsx) | `activity/timeline` | unreviewed / production / fixtures | 4 / 0 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. | ## agent-chat @@ -58,7 +58,7 @@ Status describes product alignment; implementation describes the rendered subjec | [Agents and reusable capabilities/Hiring, configuration, and organization/Agents/Personas](../../ui/storybook/stories/agents/agent-personas.stories.tsx) | `agents/organization`, `navigation-preferences/profile` | unreviewed / production / fixtures | 19 / 1 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner-default recipe text reviewed; rendered subjects and fixture contracts are unchanged by that documentation update. | | [Agents and reusable capabilities/Hiring, configuration, and organization/Agents/Configuration refresh](../../ui/storybook/stories/agents/agent-settings-redesign.stories.tsx) | `agents/lifecycle`, `agents/instructions` | mixed / mixed / fixtures | 20 / 1 | Production configuration panels in a locally composed settings shell; full AgentDetail route parity remains unreviewed. Runner display/default/effort and matching-discovery source changes reviewed; existing fixture, proposal and historical scope retained. No new rendering, play, backend or provider qualification is claimed. | | [Agents and reusable capabilities/Hiring, configuration, and organization/Product/Built-in Agents](../../ui/storybook/stories/agents/built-in-agents.stories.tsx) | `agents/built-in` | unreviewed / production / fixtures | 4 / 0 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner display/default/effort and matching-discovery source changes reviewed; existing fixture, proposal and historical scope retained. No new rendering, play, backend or provider qualification is claimed. | -| [Agents and reusable capabilities/Hiring, configuration, and organization/Codex runner/Creation](../../ui/storybook/stories/agents/codex-runner-creation.stories.tsx) | `agents/hire`, `runs-adapters/availability`, `runs-adapters/validation` | unreviewed / production / fixtures | 8 / 6 | Production NewAgent and AgentBasicsDialog with an isolated fetch boundary. Native/Legacy replies and save mutations are simulated; source inspection does not establish all play cases or provider/default qualification. Existing eight story IDs retained through explicit metadata and feature placement. Native adapter inventory is not duplicated when the shared preview provides it. | +| [Agents and reusable capabilities/Hiring, configuration, and organization/Codex runner/Creation](../../ui/storybook/stories/agents/codex-runner-creation.stories.tsx) | `agents/hire`, `runs-adapters/availability`, `runs-adapters/validation` | unreviewed / production / fixtures | 8 / 6 | Production NewAgent and AgentBasicsDialog with an isolated fetch boundary. Native/Legacy replies and save mutations are simulated; source inspection does not establish all play cases or provider/default qualification. Existing eight story IDs retained through explicit metadata and feature placement. Native adapter inventory is not duplicated when the shared preview provides it. Claude explicit-Connect qualification and stale-result guards source-reviewed; existing mock, local and unqualified-image fixture boundaries remain. No new managed live acceptance or rendering is claimed. | | [Agents and reusable capabilities/Hiring, configuration, and organization/Onboarding/External agent invitation/Components](../../ui/storybook/stories/agents/external-agent-invite/components.stories.tsx) | `agents/hire`, `access/invitations` | unreviewed / production / fixtures | 8 / 0 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner-default recipe text reviewed; rendered subjects and fixture contracts are unchanged by that documentation update. | | [Agents and reusable capabilities/Hiring, configuration, and organization/Onboarding/External agent invitation/Journeys](../../ui/storybook/stories/agents/external-agent-invite/journeys.stories.tsx) | `agents/hire`, `access/invitations` | unreviewed / production / fixtures | 20 / 4 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner display/default/effort and matching-discovery source changes reviewed; existing fixture, proposal and historical scope retained. No new rendering, play, backend or provider qualification is claimed. | | [Agents and reusable capabilities/Hiring, configuration, and organization/Onboarding/New agent](../../ui/storybook/stories/agents/new-agent-onboarding.stories.tsx) | `agents/hire` | historical / prototype / fixtures | 37 / 1 | Retained local wizard predates production ui/src/components/new-agent flow; shared fields do not make the wizard current. Runner display/default/effort and matching-discovery source changes reviewed; existing fixture, proposal and historical scope retained. No new rendering, play, backend or provider qualification is claimed. | @@ -103,7 +103,7 @@ Status describes product alignment; implementation describes the rendered subjec | [Apps, channels, and extensions/External chat and email/Connections/AgentMail tasks](../../ui/storybook/stories/chat-channels/agentmail.stories.tsx) | `chat-channels/conversation` | unreviewed / prototype / fixtures | 9 / 2 | Local EmailTaskExperience combines production composer/bubbles with simulated email lifecycle; compare against IssueDetail before claiming page parity. Task-workspace reconciliation reviewed the added connection/destination-default recipe; these unchanged setup, identity or conversation examples do not establish default selection or new-task inheritance. | | [Apps, channels, and extensions/External chat and email/Connections/Chat connection actions](../../ui/storybook/stories/chat-channels/chat-connection-actions.stories.tsx) | `chat-channels/providers` | unreviewed / production / fixtures | 1 / 1 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Task-workspace reconciliation reviewed the added connection/destination-default recipe; these unchanged setup, identity or conversation examples do not establish default selection or new-task inheritance. | | [Apps, channels, and extensions/External chat and email/Connections/Chat management](../../ui/storybook/stories/chat-channels/chat-management/chat-management.stories.tsx) | `chat-channels/lifecycle` | unreviewed / production / fixtures | 15 / 6 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Task-workspace reconciliation: the mounted endpoint adds connection and destination editors. Local endpoint PATCH/resource PUT preserve defaults; workspace-choice requests now return only active/idle, unclosed fixture workspaces. Browser save/error/reload and new-conversation inheritance remain unverified. | -| [Apps, channels, and extensions/External chat and email/Connectors/GitHub bot management](../../ui/storybook/stories/chat-channels/github-bot-management/management.stories.tsx) | `chat-channels/providers`, `chat-channels/conversation` | unreviewed / production / fixtures | 33 / 0 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner display/default/effort and matching-discovery source changes reviewed; existing fixture, proposal and historical scope retained. No new rendering, play, backend or provider qualification is claimed. Task-workspace reconciliation: mounted repository and endpoint editors use simulated defaults. Endpoint PATCH now persists omitted/null/concrete values across GET and resets on remount; repository PUT already preserves them. Choice caches intentionally contain no projects/workspaces. Browser editor interactions, access denials and new-task inheritance remain unverified. | +| [Apps, channels, and extensions/External chat and email/Connectors/GitHub bot management](../../ui/storybook/stories/chat-channels/github-bot-management/management.stories.tsx) | `chat-channels/providers`, `chat-channels/conversation` | unreviewed / production / fixtures | 33 / 0 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner display/default/effort and matching-discovery source changes reviewed; existing fixture, proposal and historical scope retained. No new rendering, play, backend or provider qualification is claimed. Task-workspace reconciliation: mounted repository and endpoint editors use simulated defaults. Endpoint PATCH now persists omitted/null/concrete values across GET and resets on remount; repository PUT already preserves them. Choice caches intentionally contain no projects/workspaces. Browser editor interactions, access denials and new-task inheritance remain unverified. Claude explicit-Connect qualification and stale-result guards source-reviewed; existing mock, local and unqualified-image fixture boundaries remain. No new managed live acceptance or rendering is claimed. | | [Apps, channels, and extensions/External chat and email/Historical/GitHub chat prototypes](../../ui/storybook/stories/chat-channels/github-chat.stories.tsx) | `chat-channels/providers`, `chat-channels/conversation` | historical / prototype / fixtures | 48 / 9 | Historical wizard retained for original setup/access/review design; current GitHub setup is ui/src/pages/apps/chat/GitHubChatSetup.tsx. Task-workspace reconciliation reviewed the added connection/destination-default recipe; these unchanged setup, identity or conversation examples do not establish default selection or new-task inheritance. | | [Apps, channels, and extensions/External chat and email/Connections/GitHub and Code Review Bot](../../ui/storybook/stories/chat-channels/github-connections.stories.tsx) | `chat-channels/providers`, `connection-setup/channel-setup` | unreviewed / production / fixtures | 7 / 7 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Task-workspace reconciliation reviewed the added connection/destination-default recipe; these unchanged setup, identity or conversation examples do not establish default selection or new-task inheritance. | | [Apps, channels, and extensions/External chat and email/Connections/iMessage Photon](../../ui/storybook/stories/chat-channels/imessage-photon.stories.tsx) | `chat-channels/conversation` | unreviewed / production / fixtures | 13 / 7 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Task-workspace reconciliation reviewed the endpoint editor addition; this suite selects setup/access and locally simulated conversation states. Endpoint PATCH retains fields, but settings workspace-choice requests and default selection are not covered by these scenarios. | @@ -247,9 +247,9 @@ Status describes product alignment; implementation describes the rendered subjec | Story suite | Capabilities | Product / implementation / data | Stories / play functions | Review note | | --- | --- | --- | --- | --- | -| [Getting started and access/Onboarding and first work/Onboarding/Agent arc](../../ui/storybook/stories/onboarding/onboarding-agent-arc.stories.tsx) | `onboarding/adapter`, `onboarding/first-task` | unreviewed / production / fixtures | 27 / 11 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner display/default/effort and matching-discovery source changes reviewed; existing fixture, proposal and historical scope retained. No new rendering, play, backend or provider qualification is claimed. | -| [Getting started and access/Onboarding and first work/Onboarding/Character](../../ui/storybook/stories/onboarding/onboarding-character.stories.tsx) | `onboarding/company` | unreviewed / production / fixtures | 3 / 0 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner-default recipe text reviewed; rendered subjects and fixture contracts are unchanged by that documentation update. | -| [Getting started and access/Onboarding and first work/Onboarding/Connect a model](../../ui/storybook/stories/onboarding/onboarding-connect-model.stories.tsx) | `onboarding/adapter`, `onboarding/first-task` | historical / prototype / fixtures | 5 / 0 | Older tile/mode preview; onboarding-agent-arc mounts the production wizard. Runner-default recipe text reviewed; rendered subjects and fixture contracts are unchanged by that documentation update. | +| [Getting started and access/Onboarding and first work/Onboarding/Agent arc](../../ui/storybook/stories/onboarding/onboarding-agent-arc.stories.tsx) | `onboarding/adapter`, `onboarding/first-task` | unreviewed / production / fixtures | 27 / 11 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner display/default/effort and matching-discovery source changes reviewed; existing fixture, proposal and historical scope retained. No new rendering, play, backend or provider qualification is claimed. Claude explicit-Connect qualification and stale-result guards source-reviewed; existing mock, local and unqualified-image fixture boundaries remain. No new managed live acceptance or rendering is claimed. | +| [Getting started and access/Onboarding and first work/Onboarding/Character](../../ui/storybook/stories/onboarding/onboarding-character.stories.tsx) | `onboarding/company` | unreviewed / production / fixtures | 3 / 0 | Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner-default recipe text reviewed; rendered subjects and fixture contracts are unchanged by that documentation update. Updated onboarding verification recipe source-reviewed; display subjects and fixture classification remain unchanged. | +| [Getting started and access/Onboarding and first work/Onboarding/Connect a model](../../ui/storybook/stories/onboarding/onboarding-connect-model.stories.tsx) | `onboarding/adapter`, `onboarding/first-task` | historical / prototype / fixtures | 5 / 0 | Older tile/mode preview; onboarding-agent-arc mounts the production wizard. Runner-default recipe text reviewed; rendered subjects and fixture contracts are unchanged by that documentation update. Updated onboarding verification recipe source-reviewed; display subjects and fixture classification remain unchanged. | ## projects diff --git a/maps/storybook-map/REVIEW.md b/maps/storybook-map/REVIEW.md index 1da7a0bd100..1265831c2e8 100644 --- a/maps/storybook-map/REVIEW.md +++ b/maps/storybook-map/REVIEW.md @@ -338,3 +338,28 @@ Recomputed only these two combined fingerprints after inspecting those map diffs; existing sources, notes, statuses and stable IDs remain intact. The catalog checker passes with 224 files and 2,223 stories. No new rendering, play, provider execution or backend acceptance is claimed by this conflict resolution. + + +## Claude managed setup source review — 2026-10-13 + +Reviewed the six affected entries: activity/data-viz-misc, +chat-channels/github-bot-management/management, onboarding/onboarding-agent-arc, +onboarding/onboarding-character, onboarding/onboarding-connect-model and +agents/codex-runner-creation. Four declare the shipped wizard or creation setup; +their sources now include the Claude qualification helper. The other two depend +on the updated onboarding recipe. Existing subjects, mocks, exports, stable IDs +and classifications are retained. + +The shipped setup qualifies new Auto Claude on an explicit immutable sandbox +image at Connect, refreshes scoped discovery and uses the measured runner for +testing and saving. Permission failures and stale company/environment/settings +results stop setup. Wizard discovery is cached after the operation to avoid +cancelling its own measured default transition. Existing local and sandbox +fixtures without an immutable image do not exercise this new boundary. Mounted +unit tests simulate it separately; they do not establish provider execution, +permissions in a real instance or sandbox allocation/cleanup. + +Refreshed only these reviewed source fingerprints. This review adds no browser +rendering, play execution, screenshot baselines, Storybook build or live managed +acceptance. Full parity remains unreviewed for five entries; the older model +connection preview remains historical. diff --git a/maps/storybook-map/catalog.json b/maps/storybook-map/catalog.json index 467a6f90308..c76f1c44386 100644 --- a/maps/storybook-map/catalog.json +++ b/maps/storybook-map/catalog.json @@ -1869,8 +1869,8 @@ "status": "unreviewed", "implementation": "production", "boundary": "fixtures", - "notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner display/default/effort and matching-discovery source changes reviewed; existing fixture, proposal and historical scope retained. No new rendering, play, backend or provider qualification is claimed.", - "reviewedAt": "2026-10-12", + "notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner display/default/effort and matching-discovery source changes reviewed; existing fixture, proposal and historical scope retained. No new rendering, play, backend or provider qualification is claimed. Claude explicit-Connect qualification and stale-result guards source-reviewed; existing mock, local and unqualified-image fixture boundaries remain. No new managed live acceptance or rendering is claimed.", + "reviewedAt": "2026-10-13", "sources": [ "maps/feature-map/activity.md", "ui/src/components/ActivityCharts.tsx", @@ -1886,10 +1886,11 @@ "ui/src/components/PageSkeleton.tsx", "ui/src/components/StatusBadge.tsx", "ui/src/components/SwipeToArchive.tsx", + "ui/src/lib/qualify-claude-setup.ts", "ui/storybook/fixtures/paperclipData.ts", "ui/storybook/stories/activity/data-viz-misc.stories.tsx" ], - "fingerprint": "22270ab32ce08f89b6e5769423b05306fb1ee46d0923d699624da8d138203dce", + "fingerprint": "8586a87722ad7421144fddbc9f58fe1aebf1a4bc19739997245297ac49e7a31e", "stableId": "product-data-visualization-misc", "sidebarTitle": "Oversight and operation/Dashboards and audit trails/Product/Data Visualization & Misc" }, @@ -2528,8 +2529,8 @@ "status": "unreviewed", "implementation": "production", "boundary": "fixtures", - "notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner display/default/effort and matching-discovery source changes reviewed; existing fixture, proposal and historical scope retained. No new rendering, play, backend or provider qualification is claimed. Task-workspace reconciliation: mounted repository and endpoint editors use simulated defaults. Endpoint PATCH now persists omitted/null/concrete values across GET and resets on remount; repository PUT already preserves them. Choice caches intentionally contain no projects/workspaces. Browser editor interactions, access denials and new-task inheritance remain unverified.", - "reviewedAt": "2026-10-12", + "notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner display/default/effort and matching-discovery source changes reviewed; existing fixture, proposal and historical scope retained. No new rendering, play, backend or provider qualification is claimed. Task-workspace reconciliation: mounted repository and endpoint editors use simulated defaults. Endpoint PATCH now persists omitted/null/concrete values across GET and resets on remount; repository PUT already preserves them. Choice caches intentionally contain no projects/workspaces. Browser editor interactions, access denials and new-task inheritance remain unverified. Claude explicit-Connect qualification and stale-result guards source-reviewed; existing mock, local and unqualified-image fixture boundaries remain. No new managed live acceptance or rendering is claimed.", + "reviewedAt": "2026-10-13", "sources": [ "maps/feature-map/chat-channels.md", "ui/src/api/chatEndpoints.ts", @@ -2538,6 +2539,7 @@ "ui/src/components/AgentRunnerSelect.tsx", "ui/src/components/Layout.tsx", "ui/src/components/OnboardingWizard.tsx", + "ui/src/lib/qualify-claude-setup.ts", "ui/src/pages/apps/chat/ChatConversationList.tsx", "ui/src/pages/apps/chat/ChatEndpointDetail.tsx", "ui/src/pages/apps/chat/ChatExecutionDefaults.tsx", @@ -2552,7 +2554,7 @@ "ui/storybook/support/github-bot-management/fixture-api.tsx", "ui/storybook/support/github-bot-management/fixtures.ts" ], - "fingerprint": "a5738c8ecdb983379dacf07138f833f5461facc55e1405f1300c85b2968ef86e", + "fingerprint": "c7d45af1276651c71e023f6af2619b395c466fdd526ff904973fe82b06af2b27", "stableId": "connectors-github-bot-management", "sidebarTitle": "Apps, channels, and extensions/External chat and email/Connectors/GitHub bot management" }, @@ -3228,19 +3230,20 @@ "status": "unreviewed", "implementation": "production", "boundary": "fixtures", - "notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner display/default/effort and matching-discovery source changes reviewed; existing fixture, proposal and historical scope retained. No new rendering, play, backend or provider qualification is claimed.", - "reviewedAt": "2026-10-12", + "notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner display/default/effort and matching-discovery source changes reviewed; existing fixture, proposal and historical scope retained. No new rendering, play, backend or provider qualification is claimed. Claude explicit-Connect qualification and stale-result guards source-reviewed; existing mock, local and unqualified-image fixture boundaries remain. No new managed live acceptance or rendering is claimed.", + "reviewedAt": "2026-10-13", "sources": [ "maps/feature-map/onboarding.md", "ui/src/components/AgentRunnerSelect.tsx", "ui/src/components/OnboardingWizard.tsx", "ui/src/components/onboarding/PillGuy.tsx", "ui/src/components/onboarding/Stepper.tsx", + "ui/src/lib/qualify-claude-setup.ts", "ui/storybook/fixtures/onboardingDraft.ts", "ui/storybook/fixtures/onboardingEnvironment.ts", "ui/storybook/stories/onboarding/onboarding-agent-arc.stories.tsx" ], - "fingerprint": "17189193e8ad01810a0606a38862732c14bf43d541449acd4b342434826db096", + "fingerprint": "d9a34a47c7fb8874f840e2794d80d7aa7a6cde98659c0b3eca6c7a31aa0453cc", "stableId": "onboarding-agent-arc", "sidebarTitle": "Getting started and access/Onboarding and first work/Onboarding/Agent arc" }, @@ -3253,15 +3256,15 @@ "status": "unreviewed", "implementation": "production", "boundary": "fixtures", - "notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner-default recipe text reviewed; rendered subjects and fixture contracts are unchanged by that documentation update.", - "reviewedAt": "2026-10-12", + "notes": "Shipped subjects identified through imports and local wrappers; full state, fixture-contract and route parity remain unreviewed. Runner-default recipe text reviewed; rendered subjects and fixture contracts are unchanged by that documentation update. Updated onboarding verification recipe source-reviewed; display subjects and fixture classification remain unchanged.", + "reviewedAt": "2026-10-13", "sources": [ "maps/feature-map/onboarding.md", "ui/src/components/onboarding/OnboardingCharacter.tsx", "ui/src/components/ui/button.tsx", "ui/storybook/stories/onboarding/onboarding-character.stories.tsx" ], - "fingerprint": "4d3998161d4d1a02a72014f3ea33a41ff096969fec90f7abf50f2735b4453a8f", + "fingerprint": "8c7c138665b104c1bed08b9fb01f5994548ce4c91c8cde6f7cda157798cc5c3d", "stableId": "onboarding-character", "sidebarTitle": "Getting started and access/Onboarding and first work/Onboarding/Character" }, @@ -3275,15 +3278,15 @@ "status": "historical", "implementation": "prototype", "boundary": "fixtures", - "notes": "Older tile/mode preview; onboarding-agent-arc mounts the production wizard. Runner-default recipe text reviewed; rendered subjects and fixture contracts are unchanged by that documentation update.", - "reviewedAt": "2026-10-12", + "notes": "Older tile/mode preview; onboarding-agent-arc mounts the production wizard. Runner-default recipe text reviewed; rendered subjects and fixture contracts are unchanged by that documentation update. Updated onboarding verification recipe source-reviewed; display subjects and fixture classification remain unchanged.", + "reviewedAt": "2026-10-13", "sources": [ "maps/feature-map/onboarding.md", "ui/src/components/onboarding/ConnectModelPreview.tsx", "ui/storybook/stories/onboarding/onboarding-connect-model.stories.tsx" ], "replacement": "ui/storybook/stories/onboarding/onboarding-agent-arc.stories.tsx", - "fingerprint": "96a36e9be6ba34c1e4c5eff1fb222474217b47559450916b79dba77462df9638", + "fingerprint": "74c8e6e695922c2d34505a88b9abbd47771156c70b1bf4a2bef909d79de34786", "stableId": "onboarding-connect-a-model", "sidebarTitle": "Getting started and access/Onboarding and first work/Onboarding/Connect a model" }, @@ -5617,8 +5620,8 @@ "status": "unreviewed", "implementation": "production", "boundary": "fixtures", - "notes": "Production NewAgent and AgentBasicsDialog with an isolated fetch boundary. Native/Legacy replies and save mutations are simulated; source inspection does not establish all play cases or provider/default qualification. Existing eight story IDs retained through explicit metadata and feature placement. Native adapter inventory is not duplicated when the shared preview provides it.", - "reviewedAt": "2026-10-12", + "notes": "Production NewAgent and AgentBasicsDialog with an isolated fetch boundary. Native/Legacy replies and save mutations are simulated; source inspection does not establish all play cases or provider/default qualification. Existing eight story IDs retained through explicit metadata and feature placement. Native adapter inventory is not duplicated when the shared preview provides it. Claude explicit-Connect qualification and stale-result guards source-reviewed; existing mock, local and unqualified-image fixture boundaries remain. No new managed live acceptance or rendering is claimed.", + "reviewedAt": "2026-10-13", "sources": [ "maps/feature-map/agents.md", "maps/feature-map/runs-adapters.md", @@ -5629,12 +5632,13 @@ "ui/src/components/new-agent/NewAgentSetup.tsx", "ui/src/lib/new-agent-adapters.ts", "ui/src/lib/new-agent-runtime-config.ts", + "ui/src/lib/qualify-claude-setup.ts", "ui/src/pages/NewAgent.tsx", "ui/storybook/fixtures/onboardingEnvironment.ts", "ui/storybook/fixtures/paperclipData.ts", "ui/storybook/stories/agents/codex-runner-creation.stories.tsx" ], - "fingerprint": "323f32e42c70dd18285f4b8472d000b9bea4d33c3418770d8be0fcdd3db013d7", + "fingerprint": "b629f51121c44e404db0465daca51275e2bd73139e87050bb3ccc1269639b19e", "stableId": "agents-codex-runner-creation", "sidebarTitle": "Agents and reusable capabilities/Hiring, configuration, and organization/Codex runner/Creation" } diff --git a/ui/src/components/OnboardingWizard.test.tsx b/ui/src/components/OnboardingWizard.test.tsx index d7c39aafa2e..3128da45612 100644 --- a/ui/src/components/OnboardingWizard.test.tsx +++ b/ui/src/components/OnboardingWizard.test.tsx @@ -132,6 +132,7 @@ const mockAdapterBuild = vi.hoisted(() => ({ // the probe dies on "Could not load environment settings" and the hire never // runs — which reads as a mysterious 0-call assertion, not an error. const mockEnvironmentsApi = vi.hoisted(() => ({ + probe: vi.fn(), list: vi.fn(async () => [] as Array>), capabilities: vi.fn( async (): Promise => @@ -1847,6 +1848,8 @@ describe("OnboardingWizard restore-gate (stale localStorage across accounts)", ( beforeEach(() => { mockEnvironmentsApi.list.mockReset(); + mockEnvironmentsApi.probe.mockReset().mockResolvedValue({ ok: true, driver: "sandbox", summary: "Measured image", + details: { platformQualificationRecorded: true, leaseId: "owned-platform-lease" } }); mockEnvironmentsApi.list.mockResolvedValue([SANDBOX_ENVIRONMENT]); mockEnvironmentsApi.capabilities.mockReset(); mockEnvironmentsApi.capabilities.mockResolvedValue( @@ -1899,6 +1902,138 @@ describe("OnboardingWizard restore-gate (stale localStorage across accounts)", ( return { root, queryClient }; } + it.each(["paperclip", "legacy"] as const)("qualifies managed Claude during Connect and tests/hire with measured %s configuration", async measured => { + mockAdapterBuild.buildAdapterConfig.mockImplementation(values => buildClaudeLocalConfig(values!)); + mockEnvironmentsApi.list.mockResolvedValue([{ ...SANDBOX_ENVIRONMENT, config: { + provider: "daytona", image: `registry.example.test/runtime@sha256:${"a".repeat(64)}`, + } }]); + const discovery = (runner: "paperclip" | "legacy") => [ + { type: "claude_local", loaded: true, disabled: false, defaultRunner: runner, supportedRunners: [runner] }, + { type: "paperclip_runner", loaded: true, disabled: false }, + ]; + let qualified = false; + mockAdaptersApi.list.mockImplementation(async () => discovery(qualified ? measured : "legacy")); + const handles = await openStep4({ adapterType: "claude_local", runner: "auto" }); + await act(async () => handles.queryClient.setQueryData(["ai-connections", "company-new"], { + currentUserId: "user-1", connections: [{ id: "saved", grantId: "grant", companyId: "company-new", provider: "anthropic", method: "subscription", name: "My subscription", ownership: "personal", ownerUserId: "user-1", status: "connected", isDefault: true }], + })); + const calls: string[] = []; + mockAdaptersApi.list.mockImplementation(async context => { + expect(context).toEqual({ companyId: "company-new", environmentId: "env-sandbox-1" }); + calls.push(qualified ? "discovery-after" : "discovery-before"); + return discovery(qualified ? measured : "legacy"); + }); + mockEnvironmentsApi.probe.mockImplementation(async () => { + calls.push("platform"); qualified = true; + return { ok: true, driver: "sandbox", summary: "Measured image", details: { platformQualificationRecorded: true, leaseId: "owned-platform-lease" } }; + }); + mockAgentsApi.testEnvironment.mockImplementation(async () => { + calls.push("adapter"); return { adapterType: measured === "paperclip" ? "paperclip_runner" : "claude_local", status: "pass", checks: [], testedAt: "2026-10-13T00:00:00Z" }; + }); + mockAgentsApi.hire.mockImplementation(async () => { calls.push("hire"); return { agent: { id: "agent-1" }, approval: null }; }); + try { + expect(mockEnvironmentsApi.probe).not.toHaveBeenCalled(); + await pickSource(/Claude/); + expect(calls).toEqual(["discovery-before", "platform", "discovery-after", "adapter", "hire"]); + expect(mockEnvironmentsApi.probe).toHaveBeenCalledExactlyOnceWith("env-sandbox-1", "company-new"); + const tested = (mockAgentsApi.testEnvironment.mock.calls[0] as unknown[])[2] as { adapterConfig: Record }; + const saved = (mockAgentsApi.hire.mock.calls[0] as unknown[])[1] as { adapterConfig: Record }; + expect(saved).not.toHaveProperty("runner"); + expect(saved.adapterConfig).toMatchObject(tested.adapterConfig); + if (measured === "paperclip") expect(tested.adapterConfig).not.toHaveProperty("maxTurnsPerRun"); + else expect(tested.adapterConfig.maxTurnsPerRun).toBe(1000); + } finally { await act(async () => handles.root.unmount()); } + }); + + it("stores a new API key once before managed qualification and hires with the same native configuration", async () => { + const key = "example-scoped-api-key"; + mockAdapterBuild.buildAdapterConfig.mockImplementation(values => buildClaudeLocalConfig(values!)); + mockEnvironmentsApi.list.mockResolvedValue([{ ...SANDBOX_ENVIRONMENT, config: { + provider: "daytona", image: `registry.example.test/runtime@sha256:${"a".repeat(64)}`, + } }]); + let qualified = false; + mockAdaptersApi.list.mockImplementation(async () => [ + { type: "claude_local", defaultRunner: qualified ? "paperclip" : "legacy", supportedRunners: qualified ? ["paperclip"] : ["legacy"] }, + { type: "paperclip_runner" }, + ]); + mockEnvironmentsApi.probe.mockImplementation(async () => { + qualified = true; + return { ok: true, driver: "sandbox", summary: "Measured image", details: { platformQualificationRecorded: true, leaseId: "owned-platform-lease" } }; + }); + const handles = await openStep4({ adapterType: "claude_local", runner: "auto", credentialMode: "api" }); + try { + await pickSource(/Claude.*API/); + const field = document.body.querySelector('input[type="password"]') as HTMLInputElement; + await act(async () => setControlledValue(field, key)); + await flushReact(); + await pressArcPrimary(); + expect(managedApi.create).toHaveBeenCalledExactlyOnceWith("company-new", expect.objectContaining({ provider: "anthropic", method: "api_key", apiKey: key })); + expect(mockEnvironmentsApi.probe).toHaveBeenCalledExactlyOnceWith("env-sandbox-1", "company-new"); + expect(mockAgentsApi.testEnvironment).toHaveBeenCalledTimes(1); + expect(mockAgentsApi.hire).toHaveBeenCalledTimes(1); + const tested = (mockAgentsApi.testEnvironment.mock.calls[0] as unknown[])[2] as { adapterConfig: object }; + const saved = (mockAgentsApi.hire.mock.calls[0] as unknown[])[1] as { adapterConfig: object; runtimeConfig: object }; + expect(saved.adapterConfig).toMatchObject(tested.adapterConfig); + expect(saved.adapterConfig).not.toHaveProperty("maxTurnsPerRun"); + expect(saved.runtimeConfig).toMatchObject({ aiConnection: { provider: "anthropic", method: "api_key", mode: "responsible_user" } }); + expect(JSON.stringify(saved)).not.toContain(key); + } finally { await act(async () => handles.root.unmount()); } + }); + + it.each(["back", "environment-config"])("abandons held managed Claude qualification after %s without another probe", async change => { + const image = `registry.example.test/runtime@sha256:${"a".repeat(64)}`; + mockEnvironmentsApi.list.mockResolvedValue([{ ...SANDBOX_ENVIRONMENT, config: { provider: "daytona", image } }]); + mockAdaptersApi.list.mockResolvedValue([ + { type: "claude_local", defaultRunner: "legacy", supportedRunners: ["legacy"] }, { type: "paperclip_runner" }, + ]); + const handles = await openStep4({ adapterType: "claude_local", runner: "auto" }); + await act(async () => handles.queryClient.setQueryData(["ai-connections", "company-new"], { + currentUserId: "user-1", connections: [{ id: "saved", grantId: "grant", companyId: "company-new", provider: "anthropic", method: "subscription", name: "My subscription", ownership: "personal", ownerUserId: "user-1", status: "connected", isDefault: true }], + })); + let finish!: (value: import("@paperclipai/shared").EnvironmentProbeResult) => void; + mockEnvironmentsApi.probe.mockReturnValue(new Promise(resolve => { finish = resolve; })); + try { + await pickSource(/Claude/); + expect(mockEnvironmentsApi.probe).toHaveBeenCalledTimes(1); + await act(async () => document.querySelector('[data-testid="onboarding-wizard"]')!.dispatchEvent(new KeyboardEvent("keydown", { key: "Enter", metaKey: true, bubbles: true }))); + await flushReact(); + expect(mockEnvironmentsApi.probe).toHaveBeenCalledTimes(1); + if (change === "back") await act(async () => [...document.body.querySelectorAll("button")].find(button => button.textContent?.trim() === "Back")!.click()); + else { + await act(async () => handles.queryClient.setQueryData(queryKeys.environments.list("company-new"), [{ ...SANDBOX_ENVIRONMENT, config: { + provider: "daytona", image: `registry.example.test/runtime@sha256:${"b".repeat(64)}`, + } }])); + await flushReact(); + } + await act(async () => finish({ ok: true, driver: "sandbox", summary: "Measured image", details: { platformQualificationRecorded: true, leaseId: "owned-lease" } })); + for (let i = 0; i < 8; i++) await flushReact(); + expect(mockEnvironmentsApi.probe).toHaveBeenCalledTimes(1); + expect(mockAgentsApi.testEnvironment).not.toHaveBeenCalled(); + expect(mockAgentsApi.hire).not.toHaveBeenCalled(); + } finally { await act(async () => handles.root.unmount()); } + }); + + it("stops managed Claude onboarding at the existing platform-probe permission boundary", async () => { + mockEnvironmentsApi.list.mockResolvedValue([{ ...SANDBOX_ENVIRONMENT, config: { + provider: "daytona", image: `registry.example.test/runtime@sha256:${"a".repeat(64)}`, + } }]); + mockAdaptersApi.list.mockResolvedValue([ + { type: "claude_local", defaultRunner: "legacy", supportedRunners: ["legacy"] }, { type: "paperclip_runner" }, + ]); + const handles = await openStep4({ adapterType: "claude_local", runner: "auto" }); + await act(async () => handles.queryClient.setQueryData(["ai-connections", "company-new"], { + currentUserId: "user-1", connections: [{ id: "saved", grantId: "grant", companyId: "company-new", provider: "anthropic", method: "subscription", name: "My subscription", ownership: "personal", ownerUserId: "user-1", status: "connected", isDefault: true }], + })); + mockEnvironmentsApi.probe.mockRejectedValue(new ApiError("Instance admin access required", 403, null)); + try { + await pickSource(/Claude/); + expect(document.body.textContent).toContain("Instance admin access required"); + expect(mockEnvironmentsApi.probe).toHaveBeenCalledTimes(1); + expect(mockAgentsApi.testEnvironment).not.toHaveBeenCalled(); + expect(mockAgentsApi.hire).not.toHaveBeenCalled(); + } finally { await act(async () => handles.root.unmount()); } + }); + it.each([ ["claude_local", "anthropic", /Claude/, "claude-session-1", "claude-setup-token-status"], ["codex_local", "openai", /OpenAI/, "codex-session-1", "adapter-login-status"], diff --git a/ui/src/components/OnboardingWizard.tsx b/ui/src/components/OnboardingWizard.tsx index 931787e0423..f69f67b3167 100644 --- a/ui/src/components/OnboardingWizard.tsx +++ b/ui/src/components/OnboardingWizard.tsx @@ -1,6 +1,7 @@ -import type { AgentRunnerChoice } from "@paperclipai/shared"; +import type { AgentRunner, AgentRunnerChoice } from "@paperclipai/shared"; +import { needsClaudeSetupQualification, qualifyClaudeSetup } from "../lib/qualify-claude-setup"; import { agentHarnessType } from "@paperclipai/shared"; -import { adaptersApi } from "../api/adapters"; +import { adaptersApi, type AdapterInfo } from "../api/adapters"; import { AgentRunnerSelect } from "./AgentRunnerSelect"; import { healthApi } from "@/api/health"; import { LocalProviderLoginInstructions } from "./AdapterLoginChrome"; @@ -1024,6 +1025,23 @@ function OnboardingWizardInner({ null, [loginEnvironmentList, resolvedLoginEnvironmentId], ); + const claudeSetupSelectionKey = JSON.stringify([createdCompanyId, adapterType, runner, model, command, + args, url, credentialMode, apiKey, selectedSavedKey, selectedApiKey?.binding, savedSubscription?.binding, + resolvedLoginEnvironmentId, resolvedLoginEnvironment?.config]); + const claudeSetupSelectionRef = useRef(claudeSetupSelectionKey); + claudeSetupSelectionRef.current = claudeSetupSelectionKey; + const activeClaudeQualification = useRef<{ attempt: number; selection: string } | null>(null); + useEffect(() => { + const active = activeClaudeQualification.current; + if (!active || active.selection === claudeSetupSelectionKey) return; + activeClaudeQualification.current = null; + connectAttemptRef.current++; + hiringAgentRef.current = null; + autoConnectStartedRef.current = true; + setLoading(false); + setAdapterEnvLoading(false); + setError("Configuration changed during setup. Connect again to test the current settings."); + }, [claudeSetupSelectionKey]); // Sandbox provider capabilities for the login pseudo-terminal gate, loaded // only when the adapter declares a login capability — the same query the // agent configuration form runs (AgentConfigForm.tsx:652-658). @@ -1851,13 +1869,13 @@ function OnboardingWizardInner({ } } - function buildAdapterConfig(bindApiKey = false): Record { + function buildAdapterConfig(bindApiKey = false, setupRunner?: AgentRunner): Record { const adapter = getUIAdapter(adapterType); const config = adapter.buildAdapterConfig({ ...defaultCreateValues, adapterType, runner, - resolvedRunner, + resolvedRunner: setupRunner ?? resolvedRunner, model: adapterType === "gemini_local" ? model || DEFAULT_GEMINI_LOCAL_MODEL @@ -1923,6 +1941,7 @@ function OnboardingWizardInner({ adapterConfigOverride?: Record, appliedStoredClaudeLoginBinding = false, isCurrent = () => true, + onQualifiedConfig?: (config: Record, adapters: AdapterInfo[], environmentId: string | null) => void, ): Promise { if (!runnerDiscoveryReady) return null; if (!createdCompanyId) { @@ -1985,11 +2004,27 @@ function OnboardingWizardInner({ visibleEnvironmentIds: environmentList.map((environment) => environment.id), }); if (!isCurrent()) return null; + const selection = { adapterType, runner, savedAgentId: createdAgentId, + environment: environmentList.find(environment => environment.id === environmentId) }; + const qualified = await qualifyClaudeSetup({ ...selection, + readAdapters: () => adaptersApi.list({ companyId: createdCompanyId, environmentId }), + probeEnvironment: () => environmentsApi.probe(environmentId!, createdCompanyId), + isCurrent, + }); + if (!isCurrent() || (needsClaudeSetupQualification(selection) && !qualified)) return null; + let testedConfig = adapterConfigOverride ?? buildAdapterConfig(); + if (qualified) { + testedConfig = buildAdapterConfig(true, qualified.runner); + if (appliedStoredClaudeLoginBinding) testedConfig.env = { + ...(isEnvRecord(testedConfig.env) ? testedConfig.env : {}), ...buildFixedClaudeOAuthBinding(), + }; + onQualifiedConfig?.(testedConfig, qualified.adapters, environmentId); + } const result = await agentsApi.testEnvironment( createdCompanyId, adapterType, { - adapterConfig: adapterConfigOverride ?? buildAdapterConfig(), + adapterConfig: testedConfig, ...(["codex_local", "claude_local"].includes(adapterType) && runner !== "auto" ? { runner } : {}), ...(managedBindingForStep() ? { aiConnection: managedBindingForStep() } : {}), environmentId, @@ -2070,12 +2105,17 @@ function OnboardingWizardInner({ } if (!runnerDiscoveryReady || hiringAgentRef.current !== null) return; const attempt = ++connectAttemptRef.current; + const qualifyManagedClaude = needsClaudeSetupQualification({ adapterType, runner, savedAgentId: createdAgentId, environment: resolvedLoginEnvironment }); + const setupSelection = claudeSetupSelectionRef.current; + activeClaudeQualification.current = qualifyManagedClaude ? { attempt, selection: setupSelection } : null; const isCurrent = () => connectAttemptRef.current === attempt && stillTheSameCompany(createdCompanyId) - && runnerSelectionRef.current === runnerSelectionKey; + && runnerSelectionRef.current === runnerSelectionKey + && (!qualifyManagedClaude || claudeSetupSelectionRef.current === setupSelection); hiringAgentRef.current = attempt; autoConnectStartedRef.current = true; setLoading(true); setError(null); + let qualifiedDiscovery: { adapters: AdapterInfo[]; environmentId: string | null } | undefined; try { if (adapterType === "opencode_local") { const selectedModelId = model.trim(); @@ -2156,7 +2196,7 @@ function OnboardingWizardInner({ setClaudeOAuthStatus(storedClaudeLogin); } const shouldApplyStoredClaudeLogin = storedClaudeLogin !== null; - const hireAdapterConfig = shouldApplyStoredClaudeLogin + let hireAdapterConfig = shouldApplyStoredClaudeLogin ? { ...baseAdapterConfig, env: { @@ -2175,14 +2215,15 @@ function OnboardingWizardInner({ // stored login, would lock out a customer who has since fixed the // problem or signed in. const cachedUsable = - adapterEnvResult && + !needsClaudeSetupQualification({ adapterType, runner, savedAgentId: createdAgentId, environment: resolvedLoginEnvironment }) && adapterEnvResult && adapterEnvResultAppliedStoredLoginRef.current === shouldApplyStoredClaudeLogin && !blocksAgentCreate(adapterEnvResult) ? adapterEnvResult : null; const result = cachedUsable ?? - (await runAdapterEnvironmentTest(hireAdapterConfig, shouldApplyStoredClaudeLogin, isCurrent)); + (await runAdapterEnvironmentTest(hireAdapterConfig, shouldApplyStoredClaudeLogin, isCurrent, + (config, adapters, environmentId) => { hireAdapterConfig = config; qualifiedDiscovery = { adapters, environmentId }; })); if (!result || !isCurrent()) return; // Block the hire on a failed environment test. Also block it on a // pass or a warn result that reports missing authentication — the @@ -2276,7 +2317,13 @@ function OnboardingWizardInner({ setError(err instanceof Error ? err.message : "Failed to create agent"); } finally { if (hiringAgentRef.current === attempt) hiringAgentRef.current = null; - if (isCurrent()) setLoading(false); + if (activeClaudeQualification.current?.attempt === attempt) activeClaudeQualification.current = null; + if (isCurrent()) { + if (qualifiedDiscovery) queryClient.setQueryData( + queryKeys.adapters.availability(createdCompanyId, qualifiedDiscovery.environmentId), qualifiedDiscovery.adapters, + ); + setLoading(false); + } // Authentication is already saved. A failed probe or hire must offer a // retry with that account, rather than keep the completed login busy. if (connectCredentialStored && isCurrent()) { diff --git a/ui/src/components/new-agent/NewAgentSetup.tsx b/ui/src/components/new-agent/NewAgentSetup.tsx index d6b3618eca4..96f5a04ade9 100644 --- a/ui/src/components/new-agent/NewAgentSetup.tsx +++ b/ui/src/components/new-agent/NewAgentSetup.tsx @@ -3,7 +3,7 @@ import { AgentCharacter } from "../AgentCharacter"; import { useAgentAppearanceDraft } from "../../hooks/useAgentAppearanceDraft"; import { AiConnectionField, aiProviderForAdapter } from "../ai-connections/AiConnectionField"; import type { AiConnectionBinding } from "@paperclipai/shared"; -import type { AgentRunnerChoice } from "@paperclipai/shared"; +import type { AgentRunner, AgentRunnerChoice } from "@paperclipai/shared"; import { AgentRunnerSelect } from "../AgentRunnerSelect"; import { SelectPopover } from "../ui/select"; import { DEFAULT_CODEX_LOCAL_MODEL } from "@paperclipai/adapter-codex-local"; @@ -14,6 +14,7 @@ import { setupProviderKeys, } from "@/lib/agent-setup-fields"; import { testAgentSetup } from "@/lib/test-agent-setup"; +import { needsClaudeSetupQualification, qualifyClaudeSetup } from "@/lib/qualify-claude-setup"; import { useCloudInstance } from "@/hooks/useCloudInstance"; import { isNewAgentAdapterAllowed } from "@/lib/new-agent-adapters"; import { useEffect, useRef, useState } from "react"; @@ -189,6 +190,7 @@ function Setup({ ); const [error, setError] = useState(null); const generation = useRef(0); + const qualificationTransition = useRef<{ run: number; companyId: string; environmentId: string | null; runner: AgentRunner } | null>(null); const savingRef = useRef(false); useEffect( () => () => { @@ -197,6 +199,7 @@ function Setup({ [], ); const resetTest = () => { + qualificationTransition.current = null; generation.current++; setResult(null); setTestState("idle"); @@ -283,12 +286,19 @@ function Setup({ : "Could not resolve the environment."; } const environment = envs.data?.find((env) => env.id === environmentId); + const setupEnvironmentKey = JSON.stringify([companyId, environmentId, environment?.config]); + const setupEnvironmentRef = useRef(""); + setupEnvironmentRef.current = setupEnvironmentKey; + useEffect(() => { resetTest(); }, [setupEnvironmentKey]); const adapters = useQuery({ queryKey: queryKeys.adapters.availability(companyId, environmentId), queryFn: () => adaptersApi.list({ companyId, environmentId }), }); const defaultRunner = adapters.data?.find(adapter => adapter.type === adapterType)?.defaultRunner; useEffect(() => { + const transition = qualificationTransition.current; + if (transition?.run === generation.current && transition.companyId === companyId + && transition.environmentId === environmentId && transition.runner === defaultRunner) return; resetTest(); }, [companyId, environmentId, runner === "auto" ? defaultRunner : undefined]); const sandboxProvider = @@ -371,12 +381,13 @@ function Setup({ function buildConfig( nextConnection = connection, binding = selectedBinding, + setupRunner?: AgentRunner, ): Record { const values = { ...defaultCreateValues, adapterType, runner, - resolvedRunner: runner === "auto" ? defaultRunner : runner, + resolvedRunner: runner === "auto" ? setupRunner ?? defaultRunner : runner, model: model || (brandType === "codex_local" ? DEFAULT_CODEX_LOCAL_MODEL : ""), thinkingEffort: effort, @@ -429,7 +440,7 @@ function Setup({ } return config; } - function preparedConfig(nextConnection = connection) { + function preparedConfig(nextConnection = connection, setupRunner?: AgentRunner) { if (isDot && !allowUnmeteredProvider) throw new Error("Acknowledge external provider billing before creating your Dot agent."); if (multiProvider && (!model.trim() || !model.includes("/"))) throw new Error("Choose or enter a model in provider/model format."); @@ -458,7 +469,7 @@ function Setup({ } if (usingKimiApi && !kimiModel.trim()) throw new Error("Enter the Kimi API model name."); - return buildConfig(nextConnection); + return buildConfig(nextConnection, selectedBinding, setupRunner); } function pendingCredentials(nextConnection = connection) { if (aiBinding || nextConnection?.aiConnection) return {}; @@ -473,11 +484,25 @@ function Setup({ async function runTest(nextConnection = connection): Promise { if (!ready) return false; const run = ++generation.current; + const setupEnvironment = setupEnvironmentRef.current; setTestState("running"); setResult(null); setError(null); try { - const config = await preparedConfig(nextConnection); + preparedConfig(nextConnection); + const selection = { adapterType, runner, savedAgentId: createdAgentId, environment }; + const qualified = await qualifyClaudeSetup({ ...selection, + readAdapters: () => adaptersApi.list({ companyId, environmentId }), + probeEnvironment: () => environmentsApi.probe(environmentId!, companyId), + isCurrent: () => run === generation.current && setupEnvironmentRef.current === setupEnvironment, + }); + if (run !== generation.current || setupEnvironmentRef.current !== setupEnvironment + || (needsClaudeSetupQualification(selection) && !qualified)) return false; + if (qualified) { + qualificationTransition.current = { run, companyId, environmentId, runner: qualified.runner }; + cache.setQueryData(queryKeys.adapters.availability(companyId, environmentId), qualified.adapters); + } + const config = preparedConfig(nextConnection, qualified?.runner); const tested = await testAgentSetup({ companyId, adapterType: requestAdapterType, diff --git a/ui/src/lib/qualify-claude-setup.test.ts b/ui/src/lib/qualify-claude-setup.test.ts new file mode 100644 index 00000000000..7d4f467acc8 --- /dev/null +++ b/ui/src/lib/qualify-claude-setup.test.ts @@ -0,0 +1,67 @@ +import { describe, expect, it, vi } from "vitest"; +import type { AdapterInfo } from "../api/adapters"; +import { qualifyClaudeSetup } from "./qualify-claude-setup"; + +const image = `registry.example.test/runtime@sha256:${"a".repeat(64)}`; +const discovery = (runner: "paperclip" | "legacy", overrides = {}) => [ + { type: "claude_local", defaultRunner: runner, supportedRunners: [runner], ...overrides }, + { type: "paperclip_runner" }, +] as AdapterInfo[]; +function setup() { + return { + adapterType: "claude_local", runner: "auto" as const, + environment: { driver: "sandbox" as const, config: { provider: "daytona", image } }, + readAdapters: vi.fn().mockResolvedValueOnce(discovery("legacy")).mockResolvedValue(discovery("paperclip")), + probeEnvironment: vi.fn().mockResolvedValue({ ok: true, driver: "sandbox", summary: "Measured image", + details: { platformQualificationRecorded: true, leaseId: "owned-probe-lease" } }), + isCurrent: vi.fn(() => true), + }; +} + +describe("Claude explicit setup qualification", () => { + it.each(["paperclip", "legacy"] as const)("uses fresh discovery and honors measured %s selection", async runner => { + const s = setup(); + s.readAdapters.mockResolvedValueOnce(discovery(runner)); + const result = await qualifyClaudeSetup(s); + expect(result?.runner).toBe(runner); + expect(s.readAdapters).toHaveBeenCalledTimes(2); + expect(s.probeEnvironment).toHaveBeenCalledTimes(1); + expect(s.readAdapters.mock.invocationCallOrder[0]).toBeLessThan(s.probeEnvironment.mock.invocationCallOrder[0]); + expect(s.probeEnvironment.mock.invocationCallOrder[0]).toBeLessThan(s.readAdapters.mock.invocationCallOrder[1]); + }); + + it.each(["permission", "cleanup", "missing-lease", "connection", "discovery"])("stops before adapter setup after %s failure", async failure => { + const s = setup(); + if (failure === "permission") s.probeEnvironment.mockRejectedValue(new Error("Instance admin access required")); + else if (failure === "discovery") s.readAdapters.mockRejectedValue(new Error("Discovery unavailable")); + else s.probeEnvironment.mockResolvedValue({ ok: failure !== "connection", summary: "Qualification rejected", + details: { platformQualificationRecorded: failure !== "cleanup", ...(failure === "missing-lease" ? {} : { leaseId: "lease" }) } }); + await expect(qualifyClaudeSetup(s)).rejects.toThrow(failure === "permission" ? "Instance admin" : failure === "discovery" ? "Discovery" : "Qualification rejected"); + }); + + it.each(["legacy", "native", "local", "snapshot", "tag", "saved", "codex", "qualified", "disabled", "override", "missing-native"])("does not allocate an extra probe for %s", async mode => { + const s = setup(); + const input: Parameters[0] = s; + if (mode === "legacy" || mode === "native") input.runner = mode === "native" ? "paperclip" : "legacy"; + if (mode === "local") input.environment = { driver: "local", config: {} }; + if (mode === "snapshot") input.environment = { driver: "sandbox", config: { image, snapshot: "named-snapshot" } }; + if (mode === "tag") input.environment = { driver: "sandbox", config: { image: "registry.example.test/runtime:latest" } }; + if (mode === "saved") input.savedAgentId = "saved-agent"; + if (mode === "codex") input.adapterType = "codex_local"; + if (mode === "qualified") s.readAdapters.mockReset().mockResolvedValue(discovery("paperclip")); + if (mode === "disabled" || mode === "override") s.readAdapters.mockReset().mockResolvedValue(discovery("legacy", mode === "disabled" ? { disabled: true } : { overriddenBuiltin: true })); + if (mode === "missing-native") s.readAdapters.mockReset().mockResolvedValue(discovery("legacy").slice(0, 1)); + await qualifyClaudeSetup(input); + expect(s.probeEnvironment).not.toHaveBeenCalled(); + }); + + it("abandons a stale selection before probing and after probing", async () => { + const before = setup(); before.isCurrent.mockReturnValue(false); + expect(await qualifyClaudeSetup(before)).toBeUndefined(); + expect(before.probeEnvironment).not.toHaveBeenCalled(); + const after = setup(); after.isCurrent.mockReturnValueOnce(true).mockReturnValue(false); + expect(await qualifyClaudeSetup(after)).toBeUndefined(); + expect(after.probeEnvironment).toHaveBeenCalledTimes(1); + expect(after.readAdapters).toHaveBeenCalledTimes(1); + }); +}); diff --git a/ui/src/lib/qualify-claude-setup.ts b/ui/src/lib/qualify-claude-setup.ts new file mode 100644 index 00000000000..cfa6a2c8731 --- /dev/null +++ b/ui/src/lib/qualify-claude-setup.ts @@ -0,0 +1,48 @@ +import type { AgentRunner, AgentRunnerChoice, Environment, EnvironmentProbeResult } from "@paperclipai/shared"; +import type { AdapterInfo } from "../api/adapters"; + +export function needsClaudeSetupQualification(input: { + adapterType: string; + runner: AgentRunnerChoice; + savedAgentId?: string | null; + environment?: Pick | null; +}): boolean { + const config = input.environment?.config; + return !input.savedAgentId && input.adapterType === "claude_local" && input.runner === "auto" + && input.environment?.driver === "sandbox" && typeof config?.image === "string" + && /^[^\s@]+@sha256:[a-f0-9]{64}$/.test(config.image) + && !config.snapshot && !config.template && !config.templateRef; +} + +export async function qualifyClaudeSetup(input: Parameters[0] & { + readAdapters: () => Promise; + probeEnvironment: () => Promise; + isCurrent: () => boolean; +}): Promise<{ runner: AgentRunner; adapters: AdapterInfo[] } | undefined> { + if (!needsClaudeSetupQualification(input)) return undefined; + const validate = (adapters: AdapterInfo[]) => { + const claude = adapters.find(adapter => adapter.type === "claude_local"); + const runner = claude?.defaultRunner; + if (!claude || (runner !== "paperclip" && runner !== "legacy") || !claude.supportedRunners?.includes(runner)) { + throw new Error("Could not confirm Claude runner availability for this environment. Retry setup."); + } + return { runner, claude }; + }; + let adapters = await input.readAdapters(); + if (!input.isCurrent()) return undefined; + const selected = validate(adapters); + const native = adapters.find(adapter => adapter.type === "paperclip_runner"); + if (selected.runner === "paperclip" || selected.claude.disabled || selected.claude.overriddenBuiltin + || !native || native.disabled || native.overriddenBuiltin) return { runner: selected.runner, adapters }; + const probe = await input.probeEnvironment(); + if (!input.isCurrent()) return undefined; + if (probe.ok !== true || probe.details?.platformQualificationRecorded !== true + || typeof probe.details.leaseId !== "string" || !probe.details.leaseId) { + throw new Error(typeof probe.details?.platformQualificationDiagnostic === "string" + ? probe.details.platformQualificationDiagnostic + : probe.summary || "The environment's platform qualification could not be confirmed. Retry the connection probe or select Legacy runner in Advanced."); + } + adapters = await input.readAdapters(); + if (!input.isCurrent()) return undefined; + return { runner: validate(adapters).runner, adapters }; +} diff --git a/ui/src/pages/NewAgent.test-support.tsx b/ui/src/pages/NewAgent.test-support.tsx index bbd53fd9493..beab1997d3e 100644 --- a/ui/src/pages/NewAgent.test-support.tsx +++ b/ui/src/pages/NewAgent.test-support.tsx @@ -16,7 +16,7 @@ const api = vi.hoisted(() => ({ getAdapterAuthSignal: vi.fn(), getClaudeOAuthTokenStatus: vi.fn(), })); -const envApi = vi.hoisted(() => ({ list: vi.fn(), capabilities: vi.fn() })); +const envApi = vi.hoisted(() => ({ list: vi.fn(), capabilities: vi.fn(), probe: vi.fn() })); const settings = vi.hoisted(() => ({ get: vi.fn(), getExperimental: vi.fn(), @@ -183,6 +183,8 @@ beforeEach(() => { { id: "local-1", name: "Local", driver: "local", config: {} }, ]); envApi.capabilities.mockResolvedValue({ sandboxProviders: {} }); + envApi.probe.mockReset().mockResolvedValue({ ok: true, driver: "sandbox", summary: "Measured image", + details: { platformQualificationRecorded: true, leaseId: "owned-platform-lease" } }); settings.get.mockResolvedValue({ defaultEnvironmentId: "local-1" }); settings.getExperimental.mockResolvedValue({ enableNativeRunner: true }); settings.getGeneral.mockResolvedValue({ executionMode: "any" }); diff --git a/ui/src/pages/NewAgent.test.tsx b/ui/src/pages/NewAgent.test.tsx index 983924ef71c..b1ae00b2177 100644 --- a/ui/src/pages/NewAgent.test.tsx +++ b/ui/src/pages/NewAgent.test.tsx @@ -9,6 +9,87 @@ import { } from "./NewAgent.test-support"; describe("New agent setup", () => { + it.each(["paperclip", "legacy"] as const)("qualifies new managed Claude on explicit setup and saves measured %s defaults", async measured => { + const image = `registry.example.test/runtime@sha256:${"a".repeat(64)}`; + envApi.list.mockResolvedValue([{ id: "sandbox-1", driver: "sandbox", config: { provider: "daytona", image } }]); + settings.get.mockResolvedValue({ defaultEnvironmentId: "sandbox-1" }); + settings.getExperimental.mockResolvedValue({ enableManagedSandboxOnly: true, enableNativeRunner: false }); + api.getClaudeOAuthTokenStatus.mockResolvedValue({ secretId: "saved-oauth", latestVersion: 1 }); + state.adapters = state.adapters.map((adapter: any) => adapter.type === "claude_local" + ? { ...adapter, defaultRunner: "legacy", supportedRunners: ["legacy"] } : adapter); + let qualified = false; + const calls: string[] = []; + adapterApi.list.mockImplementation(async context => { + if (context?.companyId) calls.push(qualified ? "discovery-after" : "discovery-before"); + return state.adapters; + }); + envApi.probe.mockImplementation(async () => { + calls.push("platform"); qualified = true; + state.adapters = state.adapters.map((adapter: any) => adapter.type === "claude_local" + ? { ...adapter, defaultRunner: measured, supportedRunners: [measured] } : adapter); + return { ok: true, driver: "sandbox", summary: "Measured image", details: { platformQualificationRecorded: true, leaseId: "owned-platform-lease" } }; + }); + api.testEnvironment.mockImplementation(async () => { calls.push("adapter"); return pass; }); + await render("claude_local"); + expect(envApi.probe).not.toHaveBeenCalled(); + await click("ClaudeSubscription"); + calls.length = 0; + await click("Use saved subscription"); + expect(calls).toEqual(["discovery-before", "platform", "discovery-after", "adapter"]); + expect(envApi.probe).toHaveBeenCalledExactlyOnceWith("sandbox-1", "company-1"); + await click("Finish setup"); + expect(api.testEnvironment).toHaveBeenCalledTimes(1); + expect(api.hire).toHaveBeenCalledTimes(1); + const tested = api.testEnvironment.mock.calls[0][2].adapterConfig; + const saved = api.hire.mock.calls[0][1]; + expect(saved).not.toHaveProperty("runner"); + expect(saved.adapterConfig).toEqual(tested); + if (measured === "paperclip") expect(tested).not.toHaveProperty("maxTurnsPerRun"); + else expect(tested.maxTurnsPerRun).toBe(1000); + }); + + it("drops a held managed Claude probe when the same environment changes configuration", async () => { + const config = { provider: "daytona", image: `registry.example.test/runtime@sha256:${"a".repeat(64)}` }; + envApi.list.mockResolvedValue([{ id: "sandbox-1", driver: "sandbox", config }]); + settings.get.mockResolvedValue({ defaultEnvironmentId: "sandbox-1" }); + state.adapters = state.adapters.map((adapter: any) => adapter.type === "claude_local" + ? { ...adapter, defaultRunner: "legacy", supportedRunners: ["legacy"] } : adapter); + api.getClaudeOAuthTokenStatus.mockResolvedValue({ secretId: "saved-oauth", latestVersion: 1 }); + let finish!: (value: object) => void; + envApi.probe.mockReturnValue(new Promise(resolve => { finish = resolve; })); + await render("claude_local"); + await click("ClaudeSubscription"); + await click("Use saved subscription"); + expect(envApi.probe).toHaveBeenCalledTimes(1); + await act(async () => cache.setQueryData(queryKeys.environments.list("company-1"), [{ id: "sandbox-1", driver: "sandbox", config: { + ...config, image: `registry.example.test/runtime@sha256:${"b".repeat(64)}`, + } }])); + await settle(); + await act(async () => finish({ ok: true, driver: "sandbox", summary: "Measured image", details: { platformQualificationRecorded: true, leaseId: "owned-lease" } })); + await settle(); + expect(envApi.probe).toHaveBeenCalledTimes(1); + expect(api.testEnvironment).not.toHaveBeenCalled(); + expect(api.hire).not.toHaveBeenCalled(); + }); + + it("retains the existing managed Claude platform-probe permission gate", async () => { + envApi.list.mockResolvedValue([{ id: "sandbox-1", driver: "sandbox", config: { + provider: "daytona", image: `registry.example.test/runtime@sha256:${"a".repeat(64)}`, + } }]); + settings.get.mockResolvedValue({ defaultEnvironmentId: "sandbox-1" }); + state.adapters = state.adapters.map((adapter: any) => adapter.type === "claude_local" + ? { ...adapter, defaultRunner: "legacy", supportedRunners: ["legacy"] } : adapter); + api.getClaudeOAuthTokenStatus.mockResolvedValue({ secretId: "saved-oauth", latestVersion: 1 }); + envApi.probe.mockRejectedValue(new Error("Instance admin access required")); + await render("claude_local"); + await click("ClaudeSubscription"); + await click("Use saved subscription"); + expect(container.textContent).toContain("Instance admin access required"); + expect(envApi.probe).toHaveBeenCalledTimes(1); + expect(api.testEnvironment).not.toHaveBeenCalled(); + expect(api.hire).not.toHaveBeenCalled(); + }); + it("waits for the current instance default instead of testing cached SSH discovery", async () => { let finish!: (value: object) => void; settings.get.mockReturnValue(new Promise(resolve => { finish = resolve; }));