Files
PaperClipAI/ui/src/components/task-chat/TaskChatThreadView.tsx
T
DottaandPaperclip d72389bee2 feat: add Browser Use Cloud connector and live task browsers (#14627)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Apps gateway gives agents governed access to external tools.
> - Browser Use Cloud can run browser work, but a tool result alone does
not let a person watch or take over.
> - A task needs a durable browser session, a visible viewer, and
recorded costs.
> - This pull request adds a Browser Use Cloud v4 connection and
interactive browser tabs on tasks.
> - People can follow the work, interact with the page, and retain the
browser after the agent finishes.

## Linked Issues or Issue Description

**Problem or motivation**

Agents need governed access to Browser Use Cloud. People need to see and
interact with the same browser from the task. A browser must remain
available after a run finishes and appear at the correct point in the
task feed.

**Proposed solution**

Add a native REST connection for the v4 API. Bind each session to its
company, task, agent, and credential grant. Open its interactive viewer
in the task side panel. Record provider costs as financial events. Use
`browser-use-cloud` as the app and connector key. Keep its skill with
the connector and deliver it only with authorized connection tools.

**Alternatives considered**

A v3 MCP connection would expose tools without the v4 lifecycle
integration. An external viewer link would leave the task. A fixed
viewer size would prevent pages from responding to changes in the task
pane.

**Roadmap alignment**

This extends the governed Apps gateway and Connected Apps roadmap. It
uses the existing task, grant, secret, approval, and financial records.
The work was requested by the maintainer. A search found no duplicate
Browser Use connector PR or issue.

## What Changed

- Add the Browser Use Cloud app, brand asset, API-key connection, and
profile settings under the `browser-use-cloud` key.
- Bundle the `browser-use-cloud` skill with the connector. Keep it out
of global `skills/` discovery. Deliver it only with authorized task/run
connection tools. Remove retired connector skill keys from runtime
overlays and preserve unrelated browser skills.
- Expose seven v4 tools through the governed gateway and deliver them to
native and CLI agents.
- Persist sessions, browsers, runs, event and recovery cursors, shutdown
leases, and cumulative cost accounting. Recover uncertain paid starts
without replaying them.
- Enforce task ownership, credential grants, approvals, revoked access,
and budget limits.
- Add interactive task browser tabs and compact chronological feed
entries. Retain the viewer across tab switches and keep visible idle
browsers open.
- Add debounced automatic viewport fitting, standard size presets, and a
viewer ownership lease.
- Add lifecycle, authorization, accounting, viewport, UI, and Storybook
coverage.
- Add an idempotent database migration after the current master
migration. Preserve deployed migration hashes. Migrate pre-release Cloud
connection and financial keys without replacing grants, credentials, or
browser history.
- Document provider behavior, live acceptance results, and the lack of
documented passkey forwarding.

## Verification

- Full workspace typecheck and production build pass on the updated
branch.
- Token gates, brand asset validation, module boundaries, and migration
ordering pass.
- Cloud tests verify global skill exclusion, authorized task/run
delivery, unassigned agents, disabled connections, revocation, adapter
isolation, and secret exclusion. The existing AgentMail connector
assignment test also passes.
- Migration replay runs twice against existing browser work and
financial records. It preserves the records and avoids duplicate costs.
- The focused provider, app catalog, OpenAPI, connection gateway, and
migration regression suites pass. Recovery coverage includes lost
replies, process crashes, provider rejection, and browser arrival
acknowledgement.
- All 54 checks pass on `2974b5f03641ad0cea3c941d8c02579316fa8c92`,
including the full test matrix, browser E2E shards, build, typecheck,
security, and release canary. Two optional Storybook jobs are skipped.
- Greptile is 5/5 on the same commit, with zero unresolved review
threads. The corrected review uses the actual master-to-head diff.
- The local `pnpm test:run` started and was stopped after the full CI
matrix passed. It did not complete locally; the full-suite result above
comes from CI.
- Earlier live acceptance used an isolated company with a capped
provider credential. The agent opened paperclip.ing, the embedded viewer
accepted navigation, and the same browser stayed available after
completion and tab switches.
- The local Storybook build passes. Stories cover the panel, footer,
feed entries, settings, lifecycle failures, and viewport modes with an
offline viewer fixture.

## Risks

- Browser Use charges for hosted work. Provider caps and local budget
checks reduce exposure; reported costs can arrive after work completes.
- Viewer and CDP URLs grant access to the browser. The server validates
and restricts them. They are excluded from agent results and durable
event data.
- Runtime resizing of v4 agent browsers uses a provider option confirmed
by live testing but absent from its published agent schema. Resizing
during a click may invalidate coordinates. Fixed presets remain
available.
- Viewport ownership is process-local and resets on restart. The
lifecycle and accounting records remain in the database.
- The original intermittent embedded-viewer stall has not been fully
diagnosed. A bounded reconnect and active-session recovery cover the
observed failure paths.
- Live tests did not cover every revocation, approval, rate-limit, or
restart case. Deterministic integration tests cover those paths. Passkey
forwarding is not claimed.
- Unknown create outcomes keep the credential available for cleanup.
Run-list absence cannot prove a paid POST was rejected, so recovery
stays pending until it can identify provider work.

## Model Used

OpenAI Codex, GPT-6. Used reasoning, repository search, code execution,
browser interaction, and test tools. The exact serving model ID and
context-window size were not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-30 07:13:16 -05:00

494 lines
18 KiB
TypeScript

import { TaskBrowserActivity } from "../task-side-panel/TaskBrowserActivity";
import { TaskChatProjectCreatedCard } from "./TaskChatProjectCreatedCard";
import { TaskChatSkillCreatedCard } from "./TaskChatSkillCreatedCard";
import { useMemo, type ReactNode } from "react";
import type { IssueAttachment } from "@paperclipai/shared";
import { cn } from "@/lib/utils";
import { useStreamlinedTaskChatPresentation } from "./presentation-mode";
import type {
TaskChatInteractionItem,
TaskChatItem,
TaskChatMessageItem,
TaskChatRuntimeRequestDecision,
TaskChatRuntimeRequestItem,
} from "./task-chat-model";
import { TaskChatTurn } from "./TaskChatTurn";
import { TaskChatBubble } from "./TaskChatBubble";
import { TaskChatMarker } from "./TaskChatMarker";
import { TaskChatStatusPill } from "./TaskChatStatusPill";
import { TaskChatToolCard } from "./TaskChatToolCard";
import { TaskChatUsageReadout } from "./TaskChatUsageReadout";
import { TaskChatRunnerActivityGroup } from "./TaskChatRunnerActivityGroup";
import { TaskChatActivityPhase } from "./TaskChatActivityPhase";
import { TaskChatThinking } from "./TaskChatThinking";
import { TaskMessageScroller } from "./TaskMessageScroller";
import { TaskChatProtocolCard } from "./TaskChatProtocolCard";
import { TaskChatProtocolActivityRow } from "./TaskChatProtocolActivityRow";
import { TaskChatPlanPreviewCard } from "./TaskChatPlanPreviewCard";
const EMPTY_ATTACHMENTS: IssueAttachment[] = [];
interface TaskChatThreadViewProps {
items: TaskChatItem[];
/**
* Content rendered above the first message INSIDE the scroll viewport (the
* issue header when hosted by the live thread) — it scrolls away with the
* messages instead of staying pinned above the thread.
*/
header?: ReactNode;
onApprovalDecision?: (statusItemId: string, optionId: string) => void;
onRuntimeRequestDecision?: (
item: TaskChatRuntimeRequestItem,
decision: TaskChatRuntimeRequestDecision,
) => void | Promise<void>;
/**
* Renders an interleaved issue-thread interaction (the live thread supplies
* TaskChatInteractionCard bound to its accept/reject handlers). Interaction
* items render nothing without it — the harness has no control plane.
*/
renderInteraction?: (item: TaskChatInteractionItem) => ReactNode;
/**
* Renders the description-as-first-bubble placeholder (PAP-375). The live
* thread binds TaskChatDescriptionBubble to the issue; brief items render
* nothing without it.
*/
renderBrief?: () => ReactNode;
/**
* Renders the copy/👍/👎 action cluster prepended to an agent bubble's footer
* line (PAP-413). The live thread binds it to the feedback-vote API; harness
* fixtures omit it and the bubbles render actionless.
*/
renderMessageActions?: (item: TaskChatMessageItem) => ReactNode;
/** Renders an interrupt action beside a queued human message. */
renderQueuedAction?: (item: TaskChatMessageItem) => ReactNode;
/** Requeues a blocked task from its no-live-execution-path failure surfaces. */
onTryAgainNoLiveExecutionPath?: () => Promise<void> | void;
tryAgainNoLiveExecutionPathPending?: boolean;
onRetryFailedRun?: (runId: string) => Promise<void> | void;
retryFailedRunId?: string | null;
/** Content appended inside the transcript scroller after the settled thread. */
tail?: ReactNode;
/** Optional streaming-aware key when `tail` changes without changing `items`. */
contentKey?: unknown;
className?: string;
/** When false, render the list without the scroll container (e.g. previews). */
scroll?: boolean;
attachments?: IssueAttachment[];
onOpenSkill?: (skillId: string, name: string) => void;
onOpenBrowser?: (browserId: string) => void;
}
function renderItem(
item: TaskChatItem,
onApprovalDecision?: (statusItemId: string, optionId: string) => void,
renderInteraction?: (item: TaskChatInteractionItem) => ReactNode,
renderBrief?: () => ReactNode,
renderMessageActions?: (item: TaskChatMessageItem) => ReactNode,
renderQueuedAction?: (item: TaskChatMessageItem) => ReactNode,
onRuntimeRequestDecision?: (
item: TaskChatRuntimeRequestItem,
decision: TaskChatRuntimeRequestDecision,
) => void | Promise<void>,
activityAppearance: "classic" | "runner" = "classic",
onTryAgainNoLiveExecutionPath?: () => Promise<void> | void,
tryAgainNoLiveExecutionPathPending = false,
retryableMarkerId?: string,
onRetryFailedRun?: (runId: string) => Promise<void> | void,
retryFailedRunId?: string | null,
attachments: IssueAttachment[] = [],
onOpenSkill?: (skillId: string, name: string) => void,
onOpenBrowser?: (browserId: string) => void,
) {
switch (item.kind) {
case "browser": return <TaskBrowserActivity browser={item.browser} label={item.label} onOpen={onOpenBrowser} />;
case "project_created": return <TaskChatProjectCreatedCard item={item} />;
case "skill_created": return <TaskChatSkillCreatedCard item={item} onOpen={onOpenSkill} />;
case "message": {
// Compute the actions once: the bubble renders them for a runless reply
// (footer = actions + timestamp), while an attached turn hands them to
// TaskChatTurn's `leading` slot so they ride the summary line and stay
// put when the tool history expands (PAP-413). The two paths are mutually
// exclusive at runtime, so only one host ever mounts the node.
const actions = renderMessageActions?.(item);
const attachedTurnItem = item.attachedTurn?.standaloneHeader
? {
...item.attachedTurn,
agentName: item.attachedTurn.agentName ?? item.authorName,
agentIcon: item.attachedTurn.agentIcon ?? item.agentIcon,
agent: item.attachedTurn.agent ?? item.agent,
}
: item.attachedTurn;
const turn = attachedTurnItem ? (
<TaskChatTurn
item={attachedTurnItem}
timestampPrefix={
attachedTurnItem.standaloneHeader ? undefined : item.timestamp
}
leading={attachedTurnItem.standaloneHeader ? undefined : actions}
renderChild={(child) =>
renderItem(
child,
onApprovalDecision,
undefined,
undefined,
undefined,
undefined,
onRuntimeRequestDecision,
item.attachedTurn?.standaloneHeader ? "runner" : "classic",
undefined,
false,
undefined,
undefined,
undefined,
attachments,
onOpenSkill,
onOpenBrowser,
)
}
/>
) : undefined;
return (
<TaskChatBubble
item={item}
// Hydration and live-to-durable reconciliation may move a logical
// message between parents. Mounting must never replay a fade.
animateEntry={false}
actions={
item.attachedTurn?.standaloneHeader
? actions
: item.attachedTurn
? undefined
: actions
}
queuedAction={renderQueuedAction?.(item)}
beforeTurn={item.attachedTurn?.standaloneHeader ? turn : undefined}
attachedTurn={item.attachedTurn?.standaloneHeader ? undefined : turn}
hideAgentIdentity={Boolean(item.attachedTurn?.standaloneHeader)}
onTryAgainNoLiveExecutionPath={onTryAgainNoLiveExecutionPath}
tryAgainNoLiveExecutionPathPending={
tryAgainNoLiveExecutionPathPending
}
attachments={attachments}
/>
);
}
case "marker":
return (
<TaskChatMarker
item={item}
onTryAgain={
item.id === retryableMarkerId && item.retryable !== false
? item.runId && onRetryFailedRun
? () => onRetryFailedRun(item.runId!)
: onTryAgainNoLiveExecutionPath
: undefined
}
tryAgainPending={
item.runId
? retryFailedRunId === item.runId
: tryAgainNoLiveExecutionPathPending
}
/>
);
case "thinking":
return <TaskChatThinking item={item} />;
case "tool":
return <TaskChatToolCard item={item} />;
case "status":
return (
<TaskChatStatusPill
item={item}
onApprovalDecision={(optionId) =>
onApprovalDecision?.(item.id, optionId)
}
/>
);
case "usage":
return <TaskChatUsageReadout item={item} />;
case "activity_phase":
// Legacy adapter transcripts and native runner transcripts now share the
// same compact activity treatment. Keeping this decision at the common
// renderer boundary also gives old persisted runs the current taxonomy,
// alignment, one-line targets, and collapsed-by-default behavior.
return <TaskChatRunnerActivityGroup item={item} />;
case "interaction":
return renderInteraction ? renderInteraction(item) : null;
case "plan_document":
return (
<TaskChatPlanPreviewCard
source={{ kind: "saved", document: item.document }}
testId={
item.placement === "fallback"
? "task-chat-plan-preview-fallback"
: "task-chat-plan-preview"
}
/>
);
case "brief":
return renderBrief ? renderBrief() : null;
case "turn":
return (
<TaskChatTurn
item={item}
renderChild={(child) =>
renderItem(
child,
onApprovalDecision,
undefined,
undefined,
undefined,
undefined,
onRuntimeRequestDecision,
item.standaloneHeader ? "runner" : "classic",
undefined,
false,
undefined,
undefined,
undefined,
attachments,
onOpenSkill,
onOpenBrowser,
)
}
/>
);
case "protocol":
return (
<TaskChatProtocolCard
item={item}
onRuntimeRequestDecision={onRuntimeRequestDecision}
/>
);
default: {
// Exhaustiveness guard: a new item kind must add a branch above.
const _never: never = item;
return _never;
}
}
}
function isSystemLikeItem(item: TaskChatItem): boolean {
return (
item.kind === "marker" ||
(item.kind === "message" && item.author === "system")
);
}
export function taskChatItemSpacingClass(
item: TaskChatItem,
previousItem: TaskChatItem | null,
): string | undefined {
if (!previousItem) return undefined;
const currentIsSystemLike = isSystemLikeItem(item);
const previousIsSystemLike = isSystemLikeItem(previousItem);
if (currentIsSystemLike && previousIsSystemLike) return "mt-2";
if (currentIsSystemLike || previousIsSystemLike) return "mt-3";
if (item.kind === "turn" || previousItem.kind === "turn") return "mt-3";
if (item.kind === "interaction" || previousItem.kind === "interaction")
return "mt-4";
return "mt-6";
}
/**
* Presentational render layer for the redesigned task thread. Consumed by both
* the live thread (adapter over comment/run props) and the dev harness
* (synthetic fixtures). Owns no data fetching — it maps a normalized
* TaskChatItem[] onto the primitives inside the auto-follow scroller.
*/
export function TaskChatThreadView({
items,
header,
onApprovalDecision,
onRuntimeRequestDecision,
renderInteraction,
renderBrief,
renderMessageActions,
renderQueuedAction,
onTryAgainNoLiveExecutionPath,
tryAgainNoLiveExecutionPathPending = false,
onRetryFailedRun,
retryFailedRunId = null,
tail,
contentKey,
className,
scroll = true,
attachments = EMPTY_ATTACHMENTS,
onOpenSkill,
onOpenBrowser,
}: TaskChatThreadViewProps) {
const streamlined = useStreamlinedTaskChatPresentation();
const retryableMarkerId =
onRetryFailedRun || onTryAgainNoLiveExecutionPath
? [...items]
.reverse()
.find(
(item) =>
item.kind === "marker" &&
item.variant === "interrupted" &&
(item.label === "Run failed" ||
item.label === "Usage limit reached"),
)?.id
: undefined;
// Streaming tail and header updates must not rebuild settled markdown/tool trees.
const history = useMemo(() => {
const renderedItems = streamlined
? items
.map((item) => ({
item,
content: renderItem(
item,
onApprovalDecision,
renderInteraction,
renderBrief,
renderMessageActions,
renderQueuedAction,
onRuntimeRequestDecision,
"classic",
onTryAgainNoLiveExecutionPath,
tryAgainNoLiveExecutionPathPending,
retryableMarkerId,
onRetryFailedRun,
retryFailedRunId,
attachments,
onOpenSkill,
onOpenBrowser,
),
}))
.filter((entry) => entry.content !== null)
: [];
return (
<>
{streamlined
? renderedItems.map(({ item, content }, index) => (
<div
key={
item.kind === "message" ? (item.renderKey ?? item.id) : item.id
}
data-thread-anchor={
item.kind === "message" ? (item.renderKey ?? item.id) : item.id
}
id={item.kind === "message" ? `comment-${item.id}` : undefined}
className={taskChatItemSpacingClass(
item,
renderedItems[index - 1]?.item ?? null,
)}
data-thread-item-kind={
item.kind === "message" ? item.author : item.kind
}
>
{content}
</div>
))
: items.map((item, index) => (
<div
key={
item.kind === "message" ? (item.renderKey ?? item.id) : item.id
}
data-thread-anchor={
item.kind === "message" ? (item.renderKey ?? item.id) : item.id
}
id={item.kind === "message" ? `comment-${item.id}` : undefined}
className={cn(
index > 0 &&
item.kind === "interaction" &&
item.interaction.status !== "pending" &&
"-mt-3",
)}
>
{renderItem(
item,
onApprovalDecision,
renderInteraction,
renderBrief,
renderMessageActions,
renderQueuedAction,
onRuntimeRequestDecision,
"classic",
onTryAgainNoLiveExecutionPath,
tryAgainNoLiveExecutionPathPending,
retryableMarkerId,
onRetryFailedRun,
retryFailedRunId,
attachments,
onOpenSkill,
onOpenBrowser,
)}
</div>
))}
</>
);
}, [
items, streamlined, onApprovalDecision, onRuntimeRequestDecision,
renderInteraction, renderBrief, renderMessageActions, renderQueuedAction,
onTryAgainNoLiveExecutionPath, tryAgainNoLiveExecutionPathPending,
retryableMarkerId, onRetryFailedRun, retryFailedRunId, attachments, onOpenSkill, onOpenBrowser,
]);
const body = (
<div
className={cn(
"paperclip-mobile-thread mx-auto flex w-full max-w-(--tc-shell-max-w) flex-col px-1 py-3 md:px-4 md:py-4",
streamlined ? "md:px-0" : "gap-5",
className,
)}
>
{header ? (
<div
className={cn("flex flex-col gap-6", streamlined ? "pb-4" : "pb-2")}
data-testid="task-chat-thread-header"
>
{header}
</div>
) : null}
{history}
{tail ? streamlined ? <div className="mt-4">{tail}</div> : tail : null}
</div>
);
if (!scroll) return body;
return (
<TaskMessageScroller contentKey={contentKey ?? taskChatContentKey(items)}>
{body}
</TaskMessageScroller>
);
}
// Cheap content signature so streaming growth (text lengthening without the
// item count changing) still advances the auto-follow key. Shared by the
// desktop scroller above and the mobile window-scroll follow (TaskChatThread).
function signatureOf(it: TaskChatItem): number {
if (it.kind === "message") return it.text.length + (it.attachedTurn ? 1 : 0);
if (it.kind === "thinking") return it.lines.reduce((n, l) => n + l.length, 0);
if (it.kind === "tool")
return (it.diff?.lines?.length ?? 0) + (it.status === "completed" ? 1 : 0);
if (it.kind === "plan_document") {
return it.document.body.length + it.document.latestRevisionNumber;
}
if (it.kind === "turn") {
if (it.settled) return 1 + (it.finalResponse?.text.length ?? 0);
// The live parent row's header changes (gerund ↔ tool-state flashes,
// streaming interstitial text growing) count too, so the collapsed
// single-line turn still advances the key.
const headerSig = it.liveStatus
? it.liveStatus.label.length +
(it.liveStatus.detail?.length ?? 0) +
(it.liveStatus.selfTalk?.length ?? 0)
: 0;
return it.items.reduce(
(n, child) => n + signatureOf(child),
it.items.length + headerSig + (it.finalResponse?.text.length ?? 0),
);
}
if (it.kind === "activity_phase") {
return it.items.reduce(
(n, child) => n + signatureOf(child),
it.summary.length + (it.interstitial?.text.length ?? 0),
);
}
return 1;
}
export function taskChatContentKey(items: TaskChatItem[]): number {
return items.reduce((acc, it) => acc + signatureOf(it), items.length);
}