mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-18 06:01:49 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents call tools from connected apps through the Paperclip MCP gateway > - Some tool calls take a long time: exports, builds, large searches > - The MCP progress utility lets an app report how far a call is, but Paperclip ignored it > - So the task chat showed only a spinner until the call finished, with no sign of life > - This pull request reads the app's progress and shows it on the tool's row while the call runs > - The benefit is that people can see that a long call is working, how far it is, and how long it has run ## Linked Issues or Issue Description **Subsystem affected** Connected apps (MCP gateway) and the task chat. **Problem or motivation** A long tool call from a connected MCP app shows only a spinner in the task chat. Paperclip does not send a `progressToken`, and it drops `notifications/progress` from the app. **Proposed solution** Send a `progressToken` on each `tools/call` that an agent run makes. Read the app's progress notifications from the call's response stream. Show a progress bar (or an indeterminate sweep), the latest message, and elapsed time on the running tool row. Keep the latest state so that a reload mid-call shows it. **Alternatives considered** A new transport for progress. The existing company live-events socket already carries per-run events with run read checks, so this change uses it. **Additional context** Spec: https://modelcontextprotocol.io/specification/2026-07-28/basic/utilities/progress ## What Changed - `mcp-http.ts`: `mcpHttpRequestInit` can add `_meta.progressToken`. `readMcpHttpResponse` gives each notification on the response stream to an observer before the result. The observer cannot fail or delay the call. New `mcpProgressNotificationOf` matches a notification to its token. - `tool-gateway.ts`: remote HTTP and local stdio `tools/call` from an agent run send a per-call token and forward progress to a reporter. Calls without a run send no token. - New `server/src/services/tool-call-progress.ts`: checks each update (finite, increasing, plain single-line text of at most 200 characters). It publishes a `heartbeat.run.tool_progress` live event at most every 250 ms (latest wins) and saves the latest update on the invocation at most every 1 s. Nothing is awaited by the call. A final `done` event goes out when the call ends. - DB: nullable `tool_invocations.progress` jsonb column (migration `0479`, `ADD COLUMN` only). A save never writes onto a finished call. - Live delivery: the new event follows its run like transcript output. A client gets it only for runs it follows, after the run read check, in the run's company only. - New route `GET /api/issues/:issueId/tool-call-progress` (task read check) returns the latest progress of the task's running calls, for a reload mid-call. Documented in `docs/api/issues.md`. - UI: `ToolCallProgressProvider` (one per chat) keeps the latest update per call. A running tool row subscribes and shows a bar or sweep, the message, the percent and elapsed time (design tokens, reduced motion honored). Finished rows drop it and never fetch or re-render for progress. `LiveUpdatesProvider` ignores the event for query invalidation. - `packages/kv-demo-mcp-server`: new `slow_with_progress` demo tool (`steps`, `stepMs`, `withTotal`) in its own file. - Storybook: `Tasks/Tool progress` stories (running with total, running without total, done). - Docs: new `doc/connections/MCP-PROGRESS.md`; `MCP-PROTOCOL-VERSIONS.md` no longer lists progress as unsupported. ## Verification - Red/green: on the original source, 5 of the new server tests fail (token in `_meta`, SSE with interleaved progress and result, notification matching, run-interest delivery, gateway end-to-end). With this change they pass. - `server/src/__tests__/mcp-http.test.ts` (13), `tool-call-progress.test.ts` (9: throttling, latest wins, bounded text, out-of-order drop, finish, failures never throw, company/run/tool scoping), `tool-call-progress-store.test.ts` (2, embedded Postgres: company and task scoping, no write onto a finished call), `realtime/live-event-delivery.test.ts` (run interest), `tool-gateway.test.ts` end-to-end against the real SDK kv demo server over Streamable HTTP: 6 provider updates over about 1.2 s give 3 to 7 throttled events, all for the right company, run, task and tool, the last one `done`; no event reaches another company; the invocation keeps the progress; the result is unchanged. - UI: `tool-call-progress.test.tsx` (10: name matching, row states, plain-text rendering, drop on complete, restore after reload, no fetch for finished rows). All 67 task-chat test files pass. - `packages/kv-demo-mcp-server`: 13 tests, including progress over HTTP with and without a total. - Storybook: checked the three stories in a browser. The bar fills to 40%, the sweep animates, the elapsed time ticks, and the done row shows no progress. - `pnpm verify`: see the CI checks for the final revision. ## Risks - Low. The migration adds one nullable column. - An SSE response is now bounded per message, not per stream, so handled notifications (such as progress) do not use up the result's size limit. Each message is still bounded. - Progress code runs only while a run's tool call is in flight. It adds a few bytes to the request and throttled, fire-and-forget work. Agent boot and time to first token do not change. - A row finds its call by run and tool name. If one run calls the same tool twice at the same time, both rows show the newer call's progress. - Progress from a call that became an MCP Task is not shown. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Claude Code agent with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
212 lines
11 KiB
TypeScript
212 lines
11 KiB
TypeScript
import { describe, expect, it } from "vitest";
|
|
import {
|
|
initializeMcpHttpSession,
|
|
MCP_HTTP_ACCEPT,
|
|
MCP_PROTOCOL_VERSION,
|
|
mcpHttpRequestHeaders,
|
|
parseMcpHttpResponseBody,
|
|
readMcpHttpResponse,
|
|
} from "../services/mcp-http.js";
|
|
|
|
describe("MCP response progress", () => {
|
|
it("retains consumed SSE progress when a partial event fails and releases the reader", async () => {
|
|
const partial = 'data: {"jsonrpc":"2.0","method":"notifications/progress"}\n\ndata: {"jsonrpc":';
|
|
const failure = new Error("peer reset");
|
|
let reads = 0;
|
|
const body = new ReadableStream<Uint8Array>({
|
|
pull(controller) {
|
|
if (reads++ === 0) controller.enqueue(new TextEncoder().encode(partial));
|
|
else controller.error(failure);
|
|
},
|
|
}, { highWaterMark: 0 });
|
|
let progress: unknown;
|
|
await expect(readMcpHttpResponse(new Response(body, { headers: { "content-type": "text/event-stream" } }), "request", {
|
|
onProgress: (value) => { progress = value; },
|
|
})).rejects.toBe(failure);
|
|
expect(progress).toEqual({ bytesReceived: Buffer.byteLength(partial), chunksReceived: 1, eventsReceived: 1, matchedResponse: false, ended: false });
|
|
expect(body.locked).toBe(false);
|
|
});
|
|
});
|
|
|
|
describe("mcpHttpRequestHeaders", () => {
|
|
it("advertises both JSON and SSE on every request", () => {
|
|
expect(mcpHttpRequestHeaders()).toMatchObject({
|
|
"content-type": "application/json",
|
|
accept: "application/json, text/event-stream",
|
|
});
|
|
expect(MCP_HTTP_ACCEPT).toBe("application/json, text/event-stream");
|
|
});
|
|
|
|
it("preserves caller-supplied headers while keeping the required Accept value", () => {
|
|
expect(mcpHttpRequestHeaders({ Authorization: "Bearer x", accept: "application/json" })).toMatchObject({
|
|
accept: "application/json, text/event-stream",
|
|
Authorization: "Bearer x",
|
|
});
|
|
});
|
|
});
|
|
|
|
describe("initializeMcpHttpSession", () => {
|
|
it("returns the negotiated protocol and ephemeral session headers", async () => {
|
|
const requests: Array<{ headers: Headers; payload: Record<string, unknown> }> = [];
|
|
const sessionHeaders = await initializeMcpHttpSession({
|
|
requestId: "test-request",
|
|
headers: { Authorization: "Bearer token" },
|
|
send: async (init) => {
|
|
const payload = JSON.parse(String(init.body)) as Record<string, unknown>;
|
|
requests.push({ headers: new Headers(init.headers), payload });
|
|
if (payload.method === "initialize") {
|
|
return new Response(JSON.stringify({
|
|
jsonrpc: "2.0",
|
|
id: payload.id,
|
|
result: {
|
|
protocolVersion: MCP_PROTOCOL_VERSION,
|
|
capabilities: { tools: {} },
|
|
serverInfo: { name: "stateful-test", version: "1" },
|
|
},
|
|
}), {
|
|
status: 200,
|
|
headers: { "content-type": "application/json", "mcp-session-id": "session-123" },
|
|
});
|
|
}
|
|
return new Response(null, { status: 202 });
|
|
},
|
|
});
|
|
|
|
// A legacy server does not answer `server/discover`, so it gets the handshake.
|
|
expect(requests.map(({ payload }) => payload.method)).toEqual([
|
|
"server/discover",
|
|
"initialize",
|
|
"notifications/initialized",
|
|
]);
|
|
expect(requests[2]!.headers.get("authorization")).toBe("Bearer token");
|
|
expect(requests[2]!.headers.get("mcp-session-id")).toBe("session-123");
|
|
expect(requests[2]!.headers.get("mcp-protocol-version")).toBe(MCP_PROTOCOL_VERSION);
|
|
expect(sessionHeaders).toMatchObject({
|
|
Authorization: "Bearer token",
|
|
"Mcp-Session-Id": "session-123",
|
|
"MCP-Protocol-Version": MCP_PROTOCOL_VERSION,
|
|
});
|
|
});
|
|
});
|
|
|
|
describe("parseMcpHttpResponseBody", () => {
|
|
it("parses a plain application/json body", () => {
|
|
const payload = { jsonrpc: "2.0", id: "1", result: { tools: [] } };
|
|
expect(parseMcpHttpResponseBody(JSON.stringify(payload), "application/json")).toEqual(payload);
|
|
});
|
|
|
|
it("parses an SSE-framed body, extracting the JSON-RPC message", () => {
|
|
const payload = { jsonrpc: "2.0", id: "1", result: { tools: [{ name: "kv_get" }] } };
|
|
const body = `event: message\ndata: ${JSON.stringify(payload)}\n\n`;
|
|
expect(parseMcpHttpResponseBody(body, "text/event-stream; charset=utf-8")).toEqual(payload);
|
|
});
|
|
|
|
it("skips non-JSON-RPC SSE events and returns the response message", () => {
|
|
const ping = "event: ping\ndata: {\"type\":\"ping\"}";
|
|
const message = { jsonrpc: "2.0", id: "1", result: { ok: true } };
|
|
const body = `${ping}\n\nevent: message\ndata: ${JSON.stringify(message)}\n\n`;
|
|
expect(parseMcpHttpResponseBody(body, "text/event-stream")).toEqual(message);
|
|
});
|
|
|
|
it("handles multi-line SSE data fields", () => {
|
|
const payload = { jsonrpc: "2.0", id: "1", result: { note: "line" } };
|
|
const json = JSON.stringify(payload, null, 2);
|
|
const body = `data: ${json.split("\n").join("\ndata: ")}\n\n`;
|
|
expect(parseMcpHttpResponseBody(body, "text/event-stream")).toEqual(payload);
|
|
});
|
|
|
|
it("throws when an SSE stream carries no data events", () => {
|
|
expect(() => parseMcpHttpResponseBody("event: ping\n\n", "text/event-stream")).toThrow();
|
|
});
|
|
});
|
|
|
|
describe("MCP progress notifications", () => {
|
|
it("sends the progress token in the request's _meta and keeps other metadata", async () => {
|
|
const { mcpHttpRequestInit } = await import("../services/mcp-http.js");
|
|
const init = mcpHttpRequestInit({ "MCP-Protocol-Version": "2026-07-28" }, {
|
|
id: "call-1", method: "tools/call", params: { name: "export", arguments: {}, _meta: { other: true } },
|
|
}, { progressToken: "token-1" });
|
|
const meta = JSON.parse(init.body).params._meta;
|
|
expect(meta.progressToken).toBe("token-1");
|
|
expect(meta.other).toBe(true);
|
|
expect(meta["io.modelcontextprotocol/protocolVersion"]).toBe("2026-07-28");
|
|
const without = mcpHttpRequestInit(undefined, { id: "call-2", method: "tools/call", params: { name: "export" } });
|
|
expect(JSON.parse(without.body).params._meta?.progressToken).toBeUndefined();
|
|
});
|
|
|
|
it("reads progress interleaved with the result on one SSE stream, before the result", async () => {
|
|
const { mcpProgressNotificationOf } = await import("../services/mcp-http.js");
|
|
const frames = [
|
|
{ jsonrpc: "2.0", method: "notifications/progress", params: { progressToken: "token-1", progress: 1, total: 3, message: "Exporting rows" } },
|
|
{ jsonrpc: "2.0", method: "notifications/message", params: { level: "info", data: "unrelated" } },
|
|
{ jsonrpc: "2.0", method: "notifications/progress", params: { progressToken: "someone-else", progress: 9 } },
|
|
{ jsonrpc: "2.0", method: "notifications/progress", params: { progressToken: "token-1", progress: 2, total: 3 } },
|
|
{ jsonrpc: "2.0", id: "call-1", result: { content: [{ type: "text", text: "done" }] } },
|
|
{ jsonrpc: "2.0", method: "notifications/progress", params: { progressToken: "token-1", progress: 3, total: 3 } },
|
|
];
|
|
// Split mid-event so frames cross chunk boundaries.
|
|
const text = frames.map((frame) => `event: message\r\ndata: ${JSON.stringify(frame)}\r\n\r\n`).join("");
|
|
const encoded = new TextEncoder().encode(text);
|
|
const body = new ReadableStream<Uint8Array>({
|
|
start(controller) {
|
|
for (let i = 0; i < encoded.length; i += 37) controller.enqueue(encoded.slice(i, i + 37));
|
|
controller.close();
|
|
},
|
|
});
|
|
const seen: unknown[] = [];
|
|
let resultSeenBeforeProgress = false;
|
|
const result = await readMcpHttpResponse(new Response(body, { headers: { "content-type": "text/event-stream" } }), "call-1", {
|
|
onNotification: (message) => {
|
|
const update = mcpProgressNotificationOf(message, "token-1");
|
|
if (update) seen.push(update);
|
|
},
|
|
onProgress: (progress) => { if (progress.matchedResponse && seen.length === 0) resultSeenBeforeProgress = true; },
|
|
});
|
|
expect(result).toMatchObject({ id: "call-1", result: { content: [{ text: "done" }] } });
|
|
expect(resultSeenBeforeProgress).toBe(false);
|
|
expect(seen).toEqual([
|
|
{ progress: 1, total: 3, message: "Exporting rows" },
|
|
{ progress: 2, total: 3, message: undefined },
|
|
]);
|
|
});
|
|
|
|
it("never lets a failing progress observer fail the call", async () => {
|
|
const text = [
|
|
{ jsonrpc: "2.0", method: "notifications/progress", params: { progressToken: "t", progress: 1 } },
|
|
{ jsonrpc: "2.0", id: 7, result: { ok: true } },
|
|
].map((frame) => `data: ${JSON.stringify(frame)}\n\n`).join("");
|
|
const result = await readMcpHttpResponse(new Response(text, { headers: { "content-type": "text/event-stream" } }), 7, {
|
|
onNotification: () => { throw new Error("observer broke"); },
|
|
});
|
|
expect(result).toMatchObject({ result: { ok: true } });
|
|
});
|
|
|
|
it("ignores requests, responses and other tokens", async () => {
|
|
const { mcpProgressNotificationOf } = await import("../services/mcp-http.js");
|
|
expect(mcpProgressNotificationOf({ method: "notifications/progress", params: { progressToken: "t", progress: 1 } }, undefined)).toBeNull();
|
|
expect(mcpProgressNotificationOf({ id: 1, method: "notifications/progress", params: { progressToken: "t", progress: 1 } }, "t")).toBeNull();
|
|
expect(mcpProgressNotificationOf({ method: "notifications/progress", params: { progressToken: "x", progress: 1 } }, "t")).toBeNull();
|
|
expect(mcpProgressNotificationOf({ method: "notifications/progress", params: { progressToken: "t", progress: 1 } }, "t")).toEqual({ progress: 1, total: undefined, message: undefined });
|
|
});
|
|
|
|
it("does not count handled progress against the result's size limit", async () => {
|
|
const note = `data: ${JSON.stringify({ jsonrpc: "2.0", method: "notifications/progress", params: { progressToken: "t", progress: 1, message: "x".repeat(200) } })}\n\n`;
|
|
const text = note.repeat(50) + `data: ${JSON.stringify({ jsonrpc: "2.0", id: 1, result: { ok: true } })}\n\n`;
|
|
expect(Buffer.byteLength(text)).toBeGreaterThan(5_000);
|
|
const result = await readMcpHttpResponse(new Response(text, { headers: { "content-type": "text/event-stream" } }), 1, { maxBytes: 1_000, limitPerMessage: true });
|
|
expect(result).toMatchObject({ result: { ok: true } });
|
|
// Callers without an overall deadline (catalog discovery) keep the total limit,
|
|
// so a stream of small notifications cannot keep them waiting forever.
|
|
await expect(readMcpHttpResponse(new Response(text, { headers: { "content-type": "text/event-stream" } }), 1, { maxBytes: 1_000 }))
|
|
.rejects.toMatchObject({ reason: "too_large" });
|
|
});
|
|
|
|
it("still rejects one SSE message over the limit, complete or not", async () => {
|
|
const big = `data: ${JSON.stringify({ jsonrpc: "2.0", id: 1, result: { text: "x".repeat(2_000) } })}`;
|
|
for (const text of [`${big}\n\n`, big]) {
|
|
await expect(readMcpHttpResponse(new Response(text, { headers: { "content-type": "text/event-stream" } }), 1, { maxBytes: 1_000, limitPerMessage: true }))
|
|
.rejects.toMatchObject({ reason: "too_large" });
|
|
}
|
|
});
|
|
});
|