mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 21:34:48 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents call tools from connected apps through the Paperclip MCP gateway > - Some tool calls take a long time: exports, builds, large searches > - The MCP progress utility lets an app report how far a call is, but Paperclip ignored it > - So the task chat showed only a spinner until the call finished, with no sign of life > - This pull request reads the app's progress and shows it on the tool's row while the call runs > - The benefit is that people can see that a long call is working, how far it is, and how long it has run ## Linked Issues or Issue Description **Subsystem affected** Connected apps (MCP gateway) and the task chat. **Problem or motivation** A long tool call from a connected MCP app shows only a spinner in the task chat. Paperclip does not send a `progressToken`, and it drops `notifications/progress` from the app. **Proposed solution** Send a `progressToken` on each `tools/call` that an agent run makes. Read the app's progress notifications from the call's response stream. Show a progress bar (or an indeterminate sweep), the latest message, and elapsed time on the running tool row. Keep the latest state so that a reload mid-call shows it. **Alternatives considered** A new transport for progress. The existing company live-events socket already carries per-run events with run read checks, so this change uses it. **Additional context** Spec: https://modelcontextprotocol.io/specification/2026-07-28/basic/utilities/progress ## What Changed - `mcp-http.ts`: `mcpHttpRequestInit` can add `_meta.progressToken`. `readMcpHttpResponse` gives each notification on the response stream to an observer before the result. The observer cannot fail or delay the call. New `mcpProgressNotificationOf` matches a notification to its token. - `tool-gateway.ts`: remote HTTP and local stdio `tools/call` from an agent run send a per-call token and forward progress to a reporter. Calls without a run send no token. - New `server/src/services/tool-call-progress.ts`: checks each update (finite, increasing, plain single-line text of at most 200 characters). It publishes a `heartbeat.run.tool_progress` live event at most every 250 ms (latest wins) and saves the latest update on the invocation at most every 1 s. Nothing is awaited by the call. A final `done` event goes out when the call ends. - DB: nullable `tool_invocations.progress` jsonb column (migration `0479`, `ADD COLUMN` only). A save never writes onto a finished call. - Live delivery: the new event follows its run like transcript output. A client gets it only for runs it follows, after the run read check, in the run's company only. - New route `GET /api/issues/:issueId/tool-call-progress` (task read check) returns the latest progress of the task's running calls, for a reload mid-call. Documented in `docs/api/issues.md`. - UI: `ToolCallProgressProvider` (one per chat) keeps the latest update per call. A running tool row subscribes and shows a bar or sweep, the message, the percent and elapsed time (design tokens, reduced motion honored). Finished rows drop it and never fetch or re-render for progress. `LiveUpdatesProvider` ignores the event for query invalidation. - `packages/kv-demo-mcp-server`: new `slow_with_progress` demo tool (`steps`, `stepMs`, `withTotal`) in its own file. - Storybook: `Tasks/Tool progress` stories (running with total, running without total, done). - Docs: new `doc/connections/MCP-PROGRESS.md`; `MCP-PROTOCOL-VERSIONS.md` no longer lists progress as unsupported. ## Verification - Red/green: on the original source, 5 of the new server tests fail (token in `_meta`, SSE with interleaved progress and result, notification matching, run-interest delivery, gateway end-to-end). With this change they pass. - `server/src/__tests__/mcp-http.test.ts` (13), `tool-call-progress.test.ts` (9: throttling, latest wins, bounded text, out-of-order drop, finish, failures never throw, company/run/tool scoping), `tool-call-progress-store.test.ts` (2, embedded Postgres: company and task scoping, no write onto a finished call), `realtime/live-event-delivery.test.ts` (run interest), `tool-gateway.test.ts` end-to-end against the real SDK kv demo server over Streamable HTTP: 6 provider updates over about 1.2 s give 3 to 7 throttled events, all for the right company, run, task and tool, the last one `done`; no event reaches another company; the invocation keeps the progress; the result is unchanged. - UI: `tool-call-progress.test.tsx` (10: name matching, row states, plain-text rendering, drop on complete, restore after reload, no fetch for finished rows). All 67 task-chat test files pass. - `packages/kv-demo-mcp-server`: 13 tests, including progress over HTTP with and without a total. - Storybook: checked the three stories in a browser. The bar fills to 40%, the sweep animates, the elapsed time ticks, and the done row shows no progress. - `pnpm verify`: see the CI checks for the final revision. ## Risks - Low. The migration adds one nullable column. - An SSE response is now bounded per message, not per stream, so handled notifications (such as progress) do not use up the result's size limit. Each message is still bounded. - Progress code runs only while a run's tool call is in flight. It adds a few bytes to the request and throttled, fire-and-forget work. Agent boot and time to first token do not change. - A row finds its call by run and tool name. If one run calls the same tool twice at the same time, both rows show the newer call's progress. - Progress from a call that became an MCP Task is not shown. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Claude Code agent with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
222 lines
11 KiB
TypeScript
222 lines
11 KiB
TypeScript
import { createContext, useContext, useEffect, useMemo, useState, useSyncExternalStore, type ReactNode } from "react";
|
|
import { useQuery } from "@tanstack/react-query";
|
|
import {
|
|
boundToolCallProgressMessage,
|
|
normalizeToolCallProgress,
|
|
toolCallProgressFraction,
|
|
type ToolCallProgressSnapshot,
|
|
} from "@paperclipai/shared";
|
|
import { mcpAppsApi } from "@/api/mcpApps";
|
|
import { useSecondTick } from "@/hooks/useSecondTick";
|
|
import { useCompanyLiveEvent } from "@/context/LiveUpdatesProvider";
|
|
import { queryKeys } from "@/lib/queryKeys";
|
|
import { cn, formatDurationMs } from "@/lib/utils";
|
|
import type { TaskChatToolItem } from "./task-chat-model";
|
|
|
|
/**
|
|
* Live progress for a running connected-app tool call. The provider streams
|
|
* MCP progress to Paperclip, which publishes it on the run's live events and
|
|
* saves the latest on the call (read once, for a reload mid-call).
|
|
*/
|
|
|
|
/** The run whose rows are showing, when known; progress from other runs is ignored. */
|
|
const ToolCallProgressRunContext = createContext<string | null>(null);
|
|
|
|
export function ToolCallProgressRunProvider({ runId, children }: { runId: string | null | undefined; children: ReactNode }) {
|
|
return <ToolCallProgressRunContext.Provider value={runId ?? null}>{children}</ToolCallProgressRunContext.Provider>;
|
|
}
|
|
|
|
/** Lowercase words joined by "_", so "mcp.kv-demo-1a:slow" and "mcp_kv_demo_1a_slow" compare equal. */
|
|
function comparableToolName(name: string) {
|
|
return name.toLowerCase().replace(/[^a-z0-9]+/g, "_").replace(/^_+|_+$/g, "");
|
|
}
|
|
|
|
/**
|
|
* Whether a transcript row's tool name ("mcp__paperclip__<gateway tool>") is
|
|
* the gateway tool a call reported. Agents may rewrite characters their tool
|
|
* names cannot hold, so names are compared word by word, as a whole suffix.
|
|
*/
|
|
export function toolCallNameMatches(rawName: string | undefined, toolName: string): boolean {
|
|
if (!rawName || !toolName) return false;
|
|
const row = comparableToolName(rawName);
|
|
const tool = comparableToolName(toolName);
|
|
return Boolean(tool) && (row === tool || row.endsWith(`_${tool}`));
|
|
}
|
|
|
|
/** A live event payload as a snapshot, or null when it is not one. Values are checked again here. */
|
|
export function readToolCallProgressEvent(payload: Record<string, unknown>): ToolCallProgressSnapshot | null {
|
|
const invocationId = typeof payload.invocationId === "string" ? payload.invocationId : null;
|
|
const runId = typeof payload.runId === "string" ? payload.runId : null;
|
|
const toolName = typeof payload.toolName === "string" ? payload.toolName : null;
|
|
const nativeToolName = typeof payload.nativeToolName === "string" ? payload.nativeToolName : null;
|
|
const startedAt = typeof payload.startedAt === "string" && Number.isFinite(Date.parse(payload.startedAt)) ? payload.startedAt : null;
|
|
if (!invocationId || !runId || !toolName || !startedAt) return null;
|
|
const updatedAt = typeof payload.updatedAt === "string" && Number.isFinite(Date.parse(payload.updatedAt)) ? new Date(payload.updatedAt) : new Date();
|
|
const progress = normalizeToolCallProgress(payload, updatedAt);
|
|
if (!progress) return null;
|
|
return { ...progress, invocationId, runId, toolName, nativeToolName, startedAt, done: payload.done === true };
|
|
}
|
|
|
|
/** Whether a row is a call of this tool, under its gateway name or the native runner's name for it. */
|
|
export function rowShowsTool(item: Pick<TaskChatToolItem, "rawName" | "name" | "appToolName">, entry: Pick<ToolCallProgressSnapshot, "toolName" | "nativeToolName">): boolean {
|
|
// A wrapper call (OpenCode's paperclip_call_assigned_tool) is matched by the tool it runs.
|
|
return [item.appToolName, item.rawName ?? item.name].some((rowName) => Boolean(rowName) && (
|
|
toolCallNameMatches(rowName, entry.toolName)
|
|
|| (Boolean(entry.nativeToolName) && toolCallNameMatches(rowName, entry.nativeToolName!))));
|
|
}
|
|
|
|
/** The progress for a row: the latest unfinished call of the same tool, in the same run when known. */
|
|
export function pickToolCallProgress(
|
|
entries: Iterable<ToolCallProgressSnapshot>,
|
|
item: Pick<TaskChatToolItem, "rawName" | "name" | "appToolName">,
|
|
runId: string | null,
|
|
): ToolCallProgressSnapshot | null {
|
|
let best: ToolCallProgressSnapshot | null = null;
|
|
for (const entry of entries) {
|
|
if (entry.done || (runId && entry.runId !== runId)) continue;
|
|
if (!rowShowsTool(item, entry)) continue;
|
|
if (!best || entry.startedAt > best.startedAt) best = entry;
|
|
}
|
|
return best;
|
|
}
|
|
|
|
/** Running calls kept per chat; the oldest are forgotten first. */
|
|
const MAX_TRACKED_CALLS = 50;
|
|
/** Finished call ids remembered per chat, so a delayed read cannot revive one. */
|
|
const MAX_REMEMBERED_FINISHED = 500;
|
|
|
|
/**
|
|
* The chat's known tool call progress. Rows subscribe only while running, so a
|
|
* burst of updates re-renders the running row, never the whole chat.
|
|
*/
|
|
export class ToolCallProgressStore {
|
|
#entries = new Map<string, ToolCallProgressSnapshot>();
|
|
#finished = new Set<string>();
|
|
#listeners = new Set<() => void>();
|
|
#onWanted: () => void;
|
|
constructor(onWanted: () => void = () => undefined) { this.#onWanted = onWanted; }
|
|
readonly subscribe = (listener: () => void) => {
|
|
this.#listeners.add(listener);
|
|
return () => { this.#listeners.delete(listener); };
|
|
};
|
|
/** A running row asks once for the saved progress (a reload mid-call). */
|
|
want() { this.#onWanted(); }
|
|
entries() { return this.#entries.values(); }
|
|
/**
|
|
* Keeps the newer update of each running call. A finished call is removed
|
|
* and remembered (bounded, apart from shown progress), so a later "running"
|
|
* snapshot of it, from a delayed read, never brings its progress back.
|
|
*/
|
|
put(snapshot: ToolCallProgressSnapshot) {
|
|
const id = snapshot.invocationId;
|
|
if (this.#finished.has(id)) return;
|
|
if (snapshot.done) {
|
|
this.#finished.add(id);
|
|
while (this.#finished.size > MAX_REMEMBERED_FINISHED) this.#finished.delete(this.#finished.values().next().value!);
|
|
if (!this.#entries.delete(id)) return;
|
|
} else {
|
|
const previous = this.#entries.get(id);
|
|
if (previous && previous.updatedAt > snapshot.updatedAt) return;
|
|
this.#entries.delete(id);
|
|
this.#entries.set(id, snapshot);
|
|
while (this.#entries.size > MAX_TRACKED_CALLS) this.#entries.delete(this.#entries.keys().next().value!);
|
|
}
|
|
for (const listener of this.#listeners) listener();
|
|
}
|
|
}
|
|
|
|
const ToolCallProgressStoreContext = createContext<ToolCallProgressStore | null>(null);
|
|
|
|
/**
|
|
* One per task chat: listens for the task's tool call progress on the shared
|
|
* live socket, and reads the saved progress once, only when a call is running.
|
|
*/
|
|
export function ToolCallProgressProvider({ issueId, children }: { issueId: string | null | undefined; children: ReactNode }) {
|
|
const [wantedFor, setWantedFor] = useState<string | null>(null);
|
|
// A new store (and a fresh read) per task and per mount: updates sent while the
|
|
// chat was closed are only in the saved snapshot.
|
|
const store = useMemo(() => new ToolCallProgressStore(() => setWantedFor(issueId ?? null)), [issueId]);
|
|
const saved = useQuery({
|
|
queryKey: queryKeys.toolCallProgress(issueId ?? ""),
|
|
queryFn: () => mcpAppsApi.toolCallProgress(issueId!),
|
|
enabled: Boolean(issueId) && wantedFor === issueId,
|
|
// Read again whenever a chat with a running call mounts; live events cover the rest.
|
|
staleTime: 0,
|
|
refetchOnMount: "always",
|
|
// Live events are not handled while the tab is hidden; read again on return.
|
|
refetchOnWindowFocus: true,
|
|
});
|
|
useEffect(() => {
|
|
// Finished calls arrive as done; a call missing from the read is never dropped,
|
|
// since a running call may not have saved its first update yet.
|
|
for (const entry of saved.data ?? []) store.put(entry);
|
|
}, [saved.data, store]);
|
|
useCompanyLiveEvent((event) => {
|
|
if (event.type !== "heartbeat.run.tool_progress" || !issueId || event.payload?.issueId !== issueId) return;
|
|
const snapshot = readToolCallProgressEvent(event.payload);
|
|
if (snapshot) store.put(snapshot);
|
|
});
|
|
return <ToolCallProgressStoreContext.Provider value={store}>{children}</ToolCallProgressStoreContext.Provider>;
|
|
}
|
|
|
|
const noSubscription = () => () => undefined;
|
|
|
|
/** Progress for a running tool row; null for any other row, which never re-renders for progress. */
|
|
export function useToolCallProgress(item: TaskChatToolItem): ToolCallProgressSnapshot | null {
|
|
const store = useContext(ToolCallProgressStoreContext);
|
|
const runId = useContext(ToolCallProgressRunContext);
|
|
const running = item.status === "in_progress" && store !== null;
|
|
useEffect(() => { if (running) store!.want(); }, [running, store]);
|
|
return useSyncExternalStore(
|
|
running ? store!.subscribe : noSubscription,
|
|
() => (running ? pickToolCallProgress(store!.entries(), item, runId) : null),
|
|
);
|
|
}
|
|
|
|
/**
|
|
* The running row's progress: a bar that fills when the total is known (a
|
|
* quiet sweep otherwise), the provider's latest message as plain text, and
|
|
* elapsed time.
|
|
*/
|
|
export function TaskChatToolProgress({ progress, nowMs }: { progress: ToolCallProgressSnapshot; nowMs?: number }) {
|
|
// The shared one-second ticker, only while no fixed time is given.
|
|
useSecondTick(nowMs === undefined);
|
|
const now = nowMs ?? Date.now();
|
|
const fraction = toolCallProgressFraction(progress);
|
|
const percent = fraction === null ? null : Math.round(fraction * 100);
|
|
// Rendered as text only: React escapes it, and it is bounded again here.
|
|
const message = boundToolCallProgressMessage(progress.message);
|
|
const elapsed = formatDurationMs(Math.max(0, now - Date.parse(progress.startedAt)));
|
|
return (
|
|
<div
|
|
className="ml-1.5 mt-0.5 flex min-w-0 max-w-full flex-col gap-1 border-l-2 border-border py-1 pl-2.5"
|
|
data-testid="task-chat-tool-progress"
|
|
>
|
|
<div
|
|
role="progressbar"
|
|
aria-label={message ?? "Working"}
|
|
aria-valuemin={0}
|
|
aria-valuemax={100}
|
|
aria-valuenow={percent ?? undefined}
|
|
className="relative h-1 w-full overflow-hidden rounded-sm bg-muted"
|
|
>
|
|
<div
|
|
data-testid={percent === null ? "task-chat-tool-progress-indeterminate" : "task-chat-tool-progress-fill"}
|
|
className={cn(
|
|
"absolute inset-y-0 left-0 rounded-sm bg-(--status-agent-running)",
|
|
percent === null ? "tool-call-progress-sweep" : "tool-call-progress-fill",
|
|
)}
|
|
style={percent === null ? undefined : { width: `${percent}%` }}
|
|
/>
|
|
</div>
|
|
<div className="flex min-w-0 items-baseline gap-2 text-(length:--text-micro) leading-4 text-muted-foreground">
|
|
{message ? (
|
|
<span className="min-w-0 flex-1 truncate" data-testid="task-chat-tool-progress-message">{message}</span>
|
|
) : <span className="flex-1" />}
|
|
{percent !== null ? <span className="shrink-0 tabular-nums">{percent}%</span> : null}
|
|
<span className="shrink-0 tabular-nums" data-testid="task-chat-tool-progress-elapsed">{elapsed}</span>
|
|
</div>
|
|
</div>
|
|
);
|
|
}
|