mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 03:31:44 +01:00
feat(mcp): show live progress for long connected-app tool calls in the task chat (#21449)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents call tools from connected apps through the Paperclip MCP gateway > - Some tool calls take a long time: exports, builds, large searches > - The MCP progress utility lets an app report how far a call is, but Paperclip ignored it > - So the task chat showed only a spinner until the call finished, with no sign of life > - This pull request reads the app's progress and shows it on the tool's row while the call runs > - The benefit is that people can see that a long call is working, how far it is, and how long it has run ## Linked Issues or Issue Description **Subsystem affected** Connected apps (MCP gateway) and the task chat. **Problem or motivation** A long tool call from a connected MCP app shows only a spinner in the task chat. Paperclip does not send a `progressToken`, and it drops `notifications/progress` from the app. **Proposed solution** Send a `progressToken` on each `tools/call` that an agent run makes. Read the app's progress notifications from the call's response stream. Show a progress bar (or an indeterminate sweep), the latest message, and elapsed time on the running tool row. Keep the latest state so that a reload mid-call shows it. **Alternatives considered** A new transport for progress. The existing company live-events socket already carries per-run events with run read checks, so this change uses it. **Additional context** Spec: https://modelcontextprotocol.io/specification/2026-07-28/basic/utilities/progress ## What Changed - `mcp-http.ts`: `mcpHttpRequestInit` can add `_meta.progressToken`. `readMcpHttpResponse` gives each notification on the response stream to an observer before the result. The observer cannot fail or delay the call. New `mcpProgressNotificationOf` matches a notification to its token. - `tool-gateway.ts`: remote HTTP and local stdio `tools/call` from an agent run send a per-call token and forward progress to a reporter. Calls without a run send no token. - New `server/src/services/tool-call-progress.ts`: checks each update (finite, increasing, plain single-line text of at most 200 characters). It publishes a `heartbeat.run.tool_progress` live event at most every 250 ms (latest wins) and saves the latest update on the invocation at most every 1 s. Nothing is awaited by the call. A final `done` event goes out when the call ends. - DB: nullable `tool_invocations.progress` jsonb column (migration `0479`, `ADD COLUMN` only). A save never writes onto a finished call. - Live delivery: the new event follows its run like transcript output. A client gets it only for runs it follows, after the run read check, in the run's company only. - New route `GET /api/issues/:issueId/tool-call-progress` (task read check) returns the latest progress of the task's running calls, for a reload mid-call. Documented in `docs/api/issues.md`. - UI: `ToolCallProgressProvider` (one per chat) keeps the latest update per call. A running tool row subscribes and shows a bar or sweep, the message, the percent and elapsed time (design tokens, reduced motion honored). Finished rows drop it and never fetch or re-render for progress. `LiveUpdatesProvider` ignores the event for query invalidation. - `packages/kv-demo-mcp-server`: new `slow_with_progress` demo tool (`steps`, `stepMs`, `withTotal`) in its own file. - Storybook: `Tasks/Tool progress` stories (running with total, running without total, done). - Docs: new `doc/connections/MCP-PROGRESS.md`; `MCP-PROTOCOL-VERSIONS.md` no longer lists progress as unsupported. ## Verification - Red/green: on the original source, 5 of the new server tests fail (token in `_meta`, SSE with interleaved progress and result, notification matching, run-interest delivery, gateway end-to-end). With this change they pass. - `server/src/__tests__/mcp-http.test.ts` (13), `tool-call-progress.test.ts` (9: throttling, latest wins, bounded text, out-of-order drop, finish, failures never throw, company/run/tool scoping), `tool-call-progress-store.test.ts` (2, embedded Postgres: company and task scoping, no write onto a finished call), `realtime/live-event-delivery.test.ts` (run interest), `tool-gateway.test.ts` end-to-end against the real SDK kv demo server over Streamable HTTP: 6 provider updates over about 1.2 s give 3 to 7 throttled events, all for the right company, run, task and tool, the last one `done`; no event reaches another company; the invocation keeps the progress; the result is unchanged. - UI: `tool-call-progress.test.tsx` (10: name matching, row states, plain-text rendering, drop on complete, restore after reload, no fetch for finished rows). All 67 task-chat test files pass. - `packages/kv-demo-mcp-server`: 13 tests, including progress over HTTP with and without a total. - Storybook: checked the three stories in a browser. The bar fills to 40%, the sweep animates, the elapsed time ticks, and the done row shows no progress. - `pnpm verify`: see the CI checks for the final revision. ## Risks - Low. The migration adds one nullable column. - An SSE response is now bounded per message, not per stream, so handled notifications (such as progress) do not use up the result's size limit. Each message is still bounded. - Progress code runs only while a run's tool call is in flight. It adds a few bytes to the request and throttled, fire-and-forget work. Agent boot and time to first token do not change. - A row finds its call by run and tool name. If one run calls the same tool twice at the same time, both rows show the newer call's progress. - Progress from a call that became an MCP Task is not shown. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Claude Code agent with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
5d0b27acca
commit
a726dce4f0
41 files changed
+1946
-22
No files matched your search
@@ -0,0 +1,66 @@
|
||||
# Progress for long MCP tool calls
|
||||
|
||||
When an agent run calls a connected app's tool and the app reports progress,
|
||||
the task chat shows it on the tool's row while the call runs: a bar that fills
|
||||
when the app knows the total (a quiet sweep when it does not), the app's latest
|
||||
status line, and elapsed time. The row drops it when the call finishes.
|
||||
|
||||
Spec: [MCP 2026-07-28, Progress](https://modelcontextprotocol.io/specification/2026-07-28/basic/utilities/progress).
|
||||
|
||||
## How it works
|
||||
|
||||
1. **Request.** For a `tools/call` made by an agent run, the gateway adds a
|
||||
per-call `_meta.progressToken` (`paperclip-progress-<uuid>`). This applies to
|
||||
remote (Streamable HTTP) and local stdio connections, under every protocol
|
||||
version. Calls not made by a run (board test calls, MCP Apps views) send no
|
||||
token.
|
||||
2. **Response stream.** The app sends `notifications/progress` (`progress`,
|
||||
optional `total`, optional `message`) on the call's own response stream: SSE
|
||||
events on the POST response for HTTP, stdout lines for stdio. Under
|
||||
2026-07-28, request-scoped notifications always stay on that stream.
|
||||
`readMcpHttpResponse` hands each notification to an observer before it
|
||||
reaches the result, and the observer can never fail or delay the call.
|
||||
3. **Reporter** (`server/src/services/tool-call-progress.ts`). Each update is
|
||||
checked (finite, non-negative, increasing; `total` kept only when positive and
|
||||
at least `progress`) and its message reduced to plain single-line text of at
|
||||
most 200 characters. The reporter then:
|
||||
- publishes a `heartbeat.run.tool_progress` live event at most every 250 ms
|
||||
(the latest update wins);
|
||||
- saves the latest update on the `tool_invocations.progress` column at most
|
||||
every second, and never onto a finished call;
|
||||
- publishes a final event with `done: true` when the call ends.
|
||||
Nothing the reporter does is awaited by the call.
|
||||
4. **Delivery.** The event goes to the run's company only. The live socket
|
||||
treats it like transcript output: a client receives it only for runs it
|
||||
asked to follow, after the same run read check.
|
||||
5. **Task chat.** `ToolCallProgressProvider` (one per chat) keeps the latest
|
||||
update per call. A running tool row subscribes to it and finds its call by
|
||||
run and tool name. Each update carries both the gateway name and the name
|
||||
native runners give the tool (`assignedToolName`: shortened, with a hash),
|
||||
since native transcripts show the latter. When a running row first mounts, the chat reads
|
||||
`GET /api/issues/:issueId/tool-call-progress` (again on mount and on tab
|
||||
focus), so a reload mid-call shows the latest saved state. The read also
|
||||
marks recently finished calls `done`; a call is never dropped just because
|
||||
the read does not list it, since its first save may still be in flight. Finished rows never fetch or re-render for
|
||||
progress.
|
||||
|
||||
## Speed
|
||||
|
||||
The token adds a few bytes to the request. Parsing happens on the stream that
|
||||
was already being read. Publishing and saving are throttled, fire-and-forget,
|
||||
and stop when the call ends. Agent boot and time to first token are untouched:
|
||||
the code runs only while a tool call is in flight.
|
||||
|
||||
## Trying it
|
||||
|
||||
`packages/kv-demo-mcp-server` has a `slow_with_progress` tool
|
||||
(`steps`, `stepMs`, `withTotal`). Connect the demo server, then ask an agent to
|
||||
call it with, say, `{ "steps": 20, "stepMs": 1000 }`; the tool row fills over
|
||||
20 seconds. `withTotal: false` shows the indeterminate state.
|
||||
|
||||
## Not covered
|
||||
|
||||
- Matching uses the run and the tool name. If one run calls the same tool
|
||||
twice at once, both rows show the newer call's progress.
|
||||
- Progress from a call that became an MCP Task (see `MCP-TASKS.md`) is not shown.
|
||||
- Progress is not forwarded to the agent's own MCP client.
|
||||
@@ -169,5 +169,5 @@ Tracked as follow-ups:
|
||||
agent to retry.
|
||||
- `subscriptions/listen` change notifications (cache invalidation relies on
|
||||
`ttlMs`).
|
||||
- The Tasks extension, Skills over MCP, progress and cancellation, icons, and
|
||||
enterprise-managed authorization.
|
||||
- Cancellation, icons, and enterprise-managed authorization. (Progress is
|
||||
supported: see [MCP-PROGRESS.md](MCP-PROGRESS.md).)
|
||||
@@ -374,6 +374,33 @@ GET /api/attachments/{attachmentId}/content
|
||||
DELETE /api/attachments/{attachmentId}
|
||||
```
|
||||
|
||||
## Tool Call Progress
|
||||
|
||||
```
|
||||
GET /api/issues/{issueId}/tool-call-progress
|
||||
```
|
||||
|
||||
Returns the latest progress of the task's connected-app tool calls that are still running, for a page that loads mid-call, and the calls that finished in the last hour, marked `done`. A client should drop a call's progress only on `done`: a running call may not have saved its first update yet. Live updates arrive as `heartbeat.run.tool_progress` events on the company live-events socket.
|
||||
|
||||
- Board callers only. The caller needs company access and read access to the task; otherwise the response is `404`.
|
||||
- Lists unfinished calls made by a run in the last 24 hours that reported progress (at most 50), then calls finished in the last hour (at most 50).
|
||||
|
||||
Each item:
|
||||
|
||||
| Field | Type | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `invocationId` | string | The tool call. |
|
||||
| `runId` | string | The run that made the call. |
|
||||
| `toolName` | string | The tool name the agent called. |
|
||||
| `startedAt` | string (ISO) | When the call started. |
|
||||
| `updatedAt` | string (ISO) | When the app last reported progress. |
|
||||
| `progress` | number | Progress so far. |
|
||||
| `total` | number or null | Known end value; `null` when the app does not know it. |
|
||||
| `message` | string or null | The app's latest status line: plain text, at most 200 characters. |
|
||||
| `done` | boolean | `true` once the call has finished. |
|
||||
|
||||
See `doc/connections/MCP-PROGRESS.md`.
|
||||
|
||||
## Issue Lifecycle
|
||||
|
||||
```
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
ALTER TABLE "tool_invocations" ADD COLUMN "progress" jsonb;
|
||||
+432
-2
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"id": "f5f852bd-0568-4b2d-bc35-0dac5d9c8c5b",
|
||||
"prevId": "349cdd89-db04-4340-a5d3-3b877bf7227d",
|
||||
"id": "9389124c-379a-4b64-9c20-c7b37538e7c8",
|
||||
"prevId": "ceaa38c4-681b-430e-9578-0854650d85b3",
|
||||
"version": "7",
|
||||
"dialect": "postgresql",
|
||||
"tables": {
|
||||
@@ -18092,6 +18092,12 @@
|
||||
"notNull": true,
|
||||
"default": "'company'"
|
||||
},
|
||||
"system_purpose": {
|
||||
"name": "system_purpose",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"owner_user_id": {
|
||||
"name": "owner_user_id",
|
||||
"type": "text",
|
||||
@@ -55953,6 +55959,153 @@
|
||||
"checkConstraints": {},
|
||||
"isRLSEnabled": false
|
||||
},
|
||||
"public.tailscale_connections": {
|
||||
"name": "tailscale_connections",
|
||||
"schema": "",
|
||||
"columns": {
|
||||
"id": {
|
||||
"name": "id",
|
||||
"type": "uuid",
|
||||
"primaryKey": true,
|
||||
"notNull": true,
|
||||
"default": "gen_random_uuid()"
|
||||
},
|
||||
"company_id": {
|
||||
"name": "company_id",
|
||||
"type": "uuid",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"name": {
|
||||
"name": "name",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"status": {
|
||||
"name": "status",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "'pending'"
|
||||
},
|
||||
"secret_id": {
|
||||
"name": "secret_id",
|
||||
"type": "uuid",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"credential_version": {
|
||||
"name": "credential_version",
|
||||
"type": "integer",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"credential_hash": {
|
||||
"name": "credential_hash",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"created_by_user_id": {
|
||||
"name": "created_by_user_id",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"oauth_state_hash": {
|
||||
"name": "oauth_state_hash",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"oauth_expires_at": {
|
||||
"name": "oauth_expires_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"created_at": {
|
||||
"name": "created_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "now()"
|
||||
},
|
||||
"updated_at": {
|
||||
"name": "updated_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "now()"
|
||||
}
|
||||
},
|
||||
"indexes": {
|
||||
"tailscale_connections_company_idx": {
|
||||
"name": "tailscale_connections_company_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "company_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": false,
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"tailscale_connections_oauth_state_idx": {
|
||||
"name": "tailscale_connections_oauth_state_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "oauth_state_hash",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": true,
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
}
|
||||
},
|
||||
"foreignKeys": {
|
||||
"tailscale_connections_company_id_companies_id_fk": {
|
||||
"name": "tailscale_connections_company_id_companies_id_fk",
|
||||
"tableFrom": "tailscale_connections",
|
||||
"tableTo": "companies",
|
||||
"columnsFrom": [
|
||||
"company_id"
|
||||
],
|
||||
"columnsTo": [
|
||||
"id"
|
||||
],
|
||||
"onDelete": "cascade",
|
||||
"onUpdate": "no action"
|
||||
},
|
||||
"tailscale_connections_secret_id_company_secrets_id_fk": {
|
||||
"name": "tailscale_connections_secret_id_company_secrets_id_fk",
|
||||
"tableFrom": "tailscale_connections",
|
||||
"tableTo": "company_secrets",
|
||||
"columnsFrom": [
|
||||
"secret_id"
|
||||
],
|
||||
"columnsTo": [
|
||||
"id"
|
||||
],
|
||||
"onDelete": "no action",
|
||||
"onUpdate": "no action"
|
||||
}
|
||||
},
|
||||
"compositePrimaryKeys": {},
|
||||
"uniqueConstraints": {},
|
||||
"policies": {},
|
||||
"checkConstraints": {},
|
||||
"isRLSEnabled": false
|
||||
},
|
||||
"public.tool_access_audit_events": {
|
||||
"name": "tool_access_audit_events",
|
||||
"schema": "",
|
||||
@@ -59454,6 +59607,12 @@
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"progress": {
|
||||
"name": "progress",
|
||||
"type": "jsonb",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"created_at": {
|
||||
"name": "created_at",
|
||||
"type": "timestamp with time zone",
|
||||
@@ -59960,6 +60119,277 @@
|
||||
"checkConstraints": {},
|
||||
"isRLSEnabled": false
|
||||
},
|
||||
"public.tool_mcp_elicitations": {
|
||||
"name": "tool_mcp_elicitations",
|
||||
"schema": "",
|
||||
"columns": {
|
||||
"id": {
|
||||
"name": "id",
|
||||
"type": "uuid",
|
||||
"primaryKey": true,
|
||||
"notNull": true,
|
||||
"default": "gen_random_uuid()"
|
||||
},
|
||||
"company_id": {
|
||||
"name": "company_id",
|
||||
"type": "uuid",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"issue_id": {
|
||||
"name": "issue_id",
|
||||
"type": "uuid",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"agent_id": {
|
||||
"name": "agent_id",
|
||||
"type": "uuid",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"run_id": {
|
||||
"name": "run_id",
|
||||
"type": "uuid",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"invocation_id": {
|
||||
"name": "invocation_id",
|
||||
"type": "uuid",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"connection_id": {
|
||||
"name": "connection_id",
|
||||
"type": "uuid",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"grant_id": {
|
||||
"name": "grant_id",
|
||||
"type": "uuid",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"catalog_tool_name": {
|
||||
"name": "catalog_tool_name",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"call_identity": {
|
||||
"name": "call_identity",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"era": {
|
||||
"name": "era",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"sealed_request_state": {
|
||||
"name": "sealed_request_state",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"questions": {
|
||||
"name": "questions",
|
||||
"type": "jsonb",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"arguments_identity": {
|
||||
"name": "arguments_identity",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"dedupe_key": {
|
||||
"name": "dedupe_key",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"status": {
|
||||
"name": "status",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"claimed_until": {
|
||||
"name": "claimed_until",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"expires_at": {
|
||||
"name": "expires_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"sent_at": {
|
||||
"name": "sent_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"created_at": {
|
||||
"name": "created_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "now()"
|
||||
},
|
||||
"updated_at": {
|
||||
"name": "updated_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "now()"
|
||||
}
|
||||
},
|
||||
"indexes": {
|
||||
"tool_mcp_elicitations_call_idx": {
|
||||
"name": "tool_mcp_elicitations_call_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "company_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "issue_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "connection_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "call_identity",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "status",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": false,
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"tool_mcp_elicitations_invocation_idx": {
|
||||
"name": "tool_mcp_elicitations_invocation_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "company_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "invocation_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": false,
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"tool_mcp_elicitations_waiting_uq": {
|
||||
"name": "tool_mcp_elicitations_waiting_uq",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "company_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "issue_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "dedupe_key",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": true,
|
||||
"where": "\"tool_mcp_elicitations\".\"status\" = 'waiting'",
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
}
|
||||
},
|
||||
"foreignKeys": {
|
||||
"tool_mcp_elicitations_company_id_companies_id_fk": {
|
||||
"name": "tool_mcp_elicitations_company_id_companies_id_fk",
|
||||
"tableFrom": "tool_mcp_elicitations",
|
||||
"tableTo": "companies",
|
||||
"columnsFrom": [
|
||||
"company_id"
|
||||
],
|
||||
"columnsTo": [
|
||||
"id"
|
||||
],
|
||||
"onDelete": "cascade",
|
||||
"onUpdate": "no action"
|
||||
},
|
||||
"tool_mcp_elicitations_invocation_id_tool_invocations_id_fk": {
|
||||
"name": "tool_mcp_elicitations_invocation_id_tool_invocations_id_fk",
|
||||
"tableFrom": "tool_mcp_elicitations",
|
||||
"tableTo": "tool_invocations",
|
||||
"columnsFrom": [
|
||||
"invocation_id"
|
||||
],
|
||||
"columnsTo": [
|
||||
"id"
|
||||
],
|
||||
"onDelete": "cascade",
|
||||
"onUpdate": "no action"
|
||||
},
|
||||
"tool_mcp_elicitations_connection_id_tool_connections_id_fk": {
|
||||
"name": "tool_mcp_elicitations_connection_id_tool_connections_id_fk",
|
||||
"tableFrom": "tool_mcp_elicitations",
|
||||
"tableTo": "tool_connections",
|
||||
"columnsFrom": [
|
||||
"connection_id"
|
||||
],
|
||||
"columnsTo": [
|
||||
"id"
|
||||
],
|
||||
"onDelete": "cascade",
|
||||
"onUpdate": "no action"
|
||||
}
|
||||
},
|
||||
"compositePrimaryKeys": {},
|
||||
"uniqueConstraints": {},
|
||||
"policies": {},
|
||||
"checkConstraints": {},
|
||||
"isRLSEnabled": false
|
||||
},
|
||||
"public.tool_mcp_gateway_tokens": {
|
||||
"name": "tool_mcp_gateway_tokens",
|
||||
"schema": "",
|
||||
+7
@@ -3375,6 +3375,13 @@
|
||||
"when": 1794611818460,
|
||||
"tag": "0483_busy_tombstone",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 484,
|
||||
"version": "7",
|
||||
"when": 1794621009918,
|
||||
"tag": "0484_certain_zemo",
|
||||
"breakpoints": true
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -16,6 +16,7 @@ import {
|
||||
uuid,
|
||||
} from "drizzle-orm/pg-core";
|
||||
import type {
|
||||
ToolCallProgress,
|
||||
ConnectionAgentInstructions,
|
||||
ConnectionTokenIssuanceOutcome,
|
||||
ConnectionTokenIssuancePath,
|
||||
@@ -804,6 +805,8 @@ export const toolInvocations = pgTable(
|
||||
errorMessage: text("error_message"),
|
||||
startedAt: timestamp("started_at", { withTimezone: true }),
|
||||
completedAt: timestamp("completed_at", { withTimezone: true }),
|
||||
/** The provider's latest progress while the call runs (MCP `notifications/progress`), bounded. */
|
||||
progress: jsonb("progress").$type<ToolCallProgress>(),
|
||||
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
|
||||
updatedAt: timestamp("updated_at", { withTimezone: true }).notNull().defaultNow(),
|
||||
},
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# KV Demo MCP Server
|
||||
|
||||
A standalone, self-contained MCP server for demos. One process exposes four
|
||||
key/value MCP tools **and** a tiny web UI that renders the values those tools
|
||||
key/value MCP tools (plus a progress demo tool) **and** a tiny web UI that renders the values those tools
|
||||
mutate — so you can call a tool from a Paperclip agent and watch the value
|
||||
appear in a browser tab.
|
||||
|
||||
@@ -24,6 +24,10 @@ recorded walkthrough in [doc/MCP-DEMO-SCRIPT.md](../../doc/MCP-DEMO-SCRIPT.md).
|
||||
- `kv_set` (write) — set one key to a string value.
|
||||
- `kv_delete` (destructive) — delete a key. Carries `destructiveHint: true`
|
||||
so Paperclip's catalog quarantines it on first sight.
|
||||
- `slow_with_progress` (read) — waits `steps` × `stepMs` and sends MCP
|
||||
`notifications/progress` after each step (`withTotal: false` leaves the total
|
||||
out). For the task chat's tool progress; see
|
||||
[doc/connections/MCP-PROGRESS.md](../../doc/connections/MCP-PROGRESS.md).
|
||||
- A Values UI at `/` — an auto-refreshing HTML table over the same store.
|
||||
- A JSON state route at `/api/state` — what the UI polls.
|
||||
|
||||
|
||||
@@ -83,4 +83,26 @@ describe("kv demo HTTP server", () => {
|
||||
expect(state.count).toBe(1);
|
||||
await client.close();
|
||||
});
|
||||
|
||||
it("streams progress for slow_with_progress on the call's response, before the result", async () => {
|
||||
const { base } = await startServer();
|
||||
const client = await mcpClient(base);
|
||||
const updates: Array<{ progress: number; total?: number; message?: string }> = [];
|
||||
const result = await client.callTool(
|
||||
{ name: "slow_with_progress", arguments: { steps: 3, stepMs: 5 } },
|
||||
undefined,
|
||||
{ onprogress: (update) => { updates.push(update); } },
|
||||
);
|
||||
expect(updates).toEqual([
|
||||
{ progress: 1, total: 3, message: "Finished step 1 of 3" },
|
||||
{ progress: 2, total: 3, message: "Finished step 2 of 3" },
|
||||
{ progress: 3, total: 3, message: "Finished step 3 of 3" },
|
||||
]);
|
||||
expect(JSON.stringify(result.content)).toContain("completed");
|
||||
const indeterminate: Array<{ total?: number }> = [];
|
||||
await client.callTool({ name: "slow_with_progress", arguments: { steps: 1, stepMs: 0, withTotal: false } }, undefined,
|
||||
{ onprogress: (update) => { indeterminate.push(update); } });
|
||||
expect(indeterminate[0]?.total).toBeUndefined();
|
||||
await client.close();
|
||||
});
|
||||
});
|
||||
@@ -1,6 +1,7 @@
|
||||
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
|
||||
import { KvStore } from "./store.js";
|
||||
import { createToolDefinitions } from "./tools.js";
|
||||
import { registerSlowWithProgressTool } from "./slow-progress.js";
|
||||
|
||||
export interface CreateKvDemoMcpServerResult {
|
||||
server: McpServer;
|
||||
@@ -29,6 +30,7 @@ export function createKvDemoMcpServer(store: KvStore = new KvStore()): CreateKvD
|
||||
tool.execute,
|
||||
);
|
||||
}
|
||||
registerSlowWithProgressTool(server);
|
||||
|
||||
return { server, store };
|
||||
}
|
||||
@@ -36,6 +38,7 @@ export function createKvDemoMcpServer(store: KvStore = new KvStore()): CreateKvD
|
||||
export { KvStore } from "./store.js";
|
||||
export type { KvEntry, KvStateSnapshot } from "./store.js";
|
||||
export { createToolDefinitions } from "./tools.js";
|
||||
export { SLOW_WITH_PROGRESS_TOOL, registerSlowWithProgressTool } from "./slow-progress.js";
|
||||
export { createKvDemoHttpServer } from "./http.js";
|
||||
export type { KvDemoHttpServer, KvDemoHttpOptions } from "./http.js";
|
||||
export { readConfigFromEnv } from "./config.js";
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
import type { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
|
||||
import { z } from "zod";
|
||||
|
||||
/**
|
||||
* A demo tool that takes a while and reports MCP progress as it goes, so a
|
||||
* client can show it. Each step waits `stepMs`, then sends
|
||||
* `notifications/progress` for the caller's `progressToken` (when it sent
|
||||
* one). `withTotal: false` leaves `total` out, for an indeterminate display.
|
||||
*/
|
||||
export const SLOW_WITH_PROGRESS_TOOL = "slow_with_progress";
|
||||
|
||||
const schema = z.object({
|
||||
steps: z.number().int().min(1).max(100).default(5),
|
||||
stepMs: z.number().int().min(0).max(10_000).default(1_000),
|
||||
withTotal: z.boolean().default(true),
|
||||
});
|
||||
|
||||
export function registerSlowWithProgressTool(server: McpServer): void {
|
||||
server.registerTool(
|
||||
SLOW_WITH_PROGRESS_TOOL,
|
||||
{
|
||||
description: "Wait for a few steps, reporting progress after each one. For progress demos.",
|
||||
inputSchema: schema.shape,
|
||||
annotations: { title: "Slow task with progress", readOnlyHint: true, openWorldHint: false },
|
||||
},
|
||||
async (input, extra) => {
|
||||
const { steps, stepMs, withTotal } = schema.parse(input);
|
||||
const progressToken = extra._meta?.progressToken;
|
||||
for (let step = 1; step <= steps; step++) {
|
||||
await new Promise((resolve) => setTimeout(resolve, stepMs));
|
||||
if (extra.signal.aborted) break;
|
||||
if (progressToken !== undefined) {
|
||||
await extra.sendNotification({
|
||||
method: "notifications/progress",
|
||||
params: {
|
||||
progressToken,
|
||||
progress: step,
|
||||
...(withTotal ? { total: steps } : {}),
|
||||
message: `Finished step ${step} of ${steps}`,
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
return { content: [{ type: "text", text: JSON.stringify({ steps, completed: true }) }] };
|
||||
},
|
||||
);
|
||||
}
|
||||
@@ -19,10 +19,10 @@ async function connectedClient() {
|
||||
}
|
||||
|
||||
describe("kv demo MCP tools", () => {
|
||||
it("exposes exactly kv_set, kv_get, kv_list, kv_delete", async () => {
|
||||
it("exposes exactly kv_set, kv_get, kv_list, kv_delete and the progress demo tool", async () => {
|
||||
const { client } = await connectedClient();
|
||||
const { tools } = await client.listTools();
|
||||
expect(tools.map((t) => t.name).sort()).toEqual(["kv_delete", "kv_get", "kv_list", "kv_set"]);
|
||||
expect(tools.map((t) => t.name).sort()).toEqual(["kv_delete", "kv_get", "kv_list", "kv_set", "slow_with_progress"]);
|
||||
});
|
||||
|
||||
it("reflects kv_set writes in the shared store", async () => {
|
||||
|
||||
@@ -973,6 +973,7 @@ export const LIVE_EVENT_TYPES = [
|
||||
"heartbeat.run.event",
|
||||
"heartbeat.run.log",
|
||||
"heartbeat.run.activity",
|
||||
"heartbeat.run.tool_progress",
|
||||
"agent.session.goal.changed",
|
||||
"agent.status",
|
||||
"agent.runtime_state.updated",
|
||||
|
||||
@@ -2966,5 +2966,6 @@ export {
|
||||
} from "./issue-activity-identity.js";
|
||||
export { recognizeLinkPreview, type LinkPreviewTarget, type LinkPreviewState, type LinkPreview, type LinkPreviewResult } from "./link-preview.js";
|
||||
export * from "./mcp-apps.js";
|
||||
export * from "./tool-call-progress.js";
|
||||
export * from "./mcp-skills.js";
|
||||
export type { TailscaleConnection, TailscaleDeviceLogin } from "./types/tailscale.js";
|
||||
@@ -0,0 +1,68 @@
|
||||
/**
|
||||
* Live progress of one connected-app tool call, from the provider's MCP
|
||||
* `notifications/progress`. Everything here is untrusted provider content:
|
||||
* numbers are checked and the message is plain text, bounded in length.
|
||||
*/
|
||||
export const TOOL_CALL_PROGRESS_MESSAGE_MAX_LENGTH = 200;
|
||||
|
||||
export interface ToolCallProgress {
|
||||
/** Increases with each update. Never shown as a raw number without `total`. */
|
||||
progress: number;
|
||||
/** Known end value; absent when the provider does not know the total. */
|
||||
total: number | null;
|
||||
/** The provider's latest status line, plain text. */
|
||||
message: string | null;
|
||||
/** When the provider last reported progress (ISO). */
|
||||
updatedAt: string;
|
||||
}
|
||||
|
||||
/** A running tool call's progress, as the task chat reads it. */
|
||||
export interface ToolCallProgressSnapshot extends ToolCallProgress {
|
||||
invocationId: string;
|
||||
runId: string;
|
||||
/** The tool name the agent called through Paperclip. */
|
||||
toolName: string;
|
||||
/** The name native runners give that tool (shortened, with a hash); their transcripts show it. */
|
||||
nativeToolName?: string | null;
|
||||
/** When the call started (ISO), for elapsed time. */
|
||||
startedAt: string;
|
||||
/** True once the call has finished; the row then drops its progress. */
|
||||
done: boolean;
|
||||
}
|
||||
|
||||
function finite(value: unknown): number | null {
|
||||
return typeof value === "number" && Number.isFinite(value) ? value : null;
|
||||
}
|
||||
|
||||
/** Plain, single-line text: control characters removed, whitespace collapsed, bounded. */
|
||||
export function boundToolCallProgressMessage(value: unknown): string | null {
|
||||
if (typeof value !== "string") return null;
|
||||
// eslint-disable-next-line no-control-regex
|
||||
const plain = value.replace(/[\u0000-\u001f\u007f-\u009f---]+/g, " ").replace(/\s+/g, " ").trim();
|
||||
if (!plain) return null;
|
||||
return plain.length > TOOL_CALL_PROGRESS_MESSAGE_MAX_LENGTH
|
||||
? `${plain.slice(0, TOOL_CALL_PROGRESS_MESSAGE_MAX_LENGTH - 1)}…`
|
||||
: plain;
|
||||
}
|
||||
|
||||
/**
|
||||
* The progress a provider reported, or null when it is not usable. `total` is
|
||||
* kept only when it is positive and at least the progress so far.
|
||||
*/
|
||||
export function normalizeToolCallProgress(input: { progress?: unknown; total?: unknown; message?: unknown }, now = new Date()): ToolCallProgress | null {
|
||||
const progress = finite(input.progress);
|
||||
if (progress === null || progress < 0) return null;
|
||||
const total = finite(input.total);
|
||||
return {
|
||||
progress,
|
||||
total: total !== null && total > 0 && total >= progress ? total : null,
|
||||
message: boundToolCallProgressMessage(input.message),
|
||||
updatedAt: now.toISOString(),
|
||||
};
|
||||
}
|
||||
|
||||
/** A fraction in [0, 1] when the total is known, otherwise null (indeterminate). */
|
||||
export function toolCallProgressFraction(progress: Pick<ToolCallProgress, "progress" | "total">): number | null {
|
||||
if (progress.total === null || progress.total <= 0) return null;
|
||||
return Math.min(1, Math.max(0, progress.progress / progress.total));
|
||||
}
|
||||
@@ -119,3 +119,93 @@ describe("parseMcpHttpResponseBody", () => {
|
||||
expect(() => parseMcpHttpResponseBody("event: ping\n\n", "text/event-stream")).toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe("MCP progress notifications", () => {
|
||||
it("sends the progress token in the request's _meta and keeps other metadata", async () => {
|
||||
const { mcpHttpRequestInit } = await import("../services/mcp-http.js");
|
||||
const init = mcpHttpRequestInit({ "MCP-Protocol-Version": "2026-07-28" }, {
|
||||
id: "call-1", method: "tools/call", params: { name: "export", arguments: {}, _meta: { other: true } },
|
||||
}, { progressToken: "token-1" });
|
||||
const meta = JSON.parse(init.body).params._meta;
|
||||
expect(meta.progressToken).toBe("token-1");
|
||||
expect(meta.other).toBe(true);
|
||||
expect(meta["io.modelcontextprotocol/protocolVersion"]).toBe("2026-07-28");
|
||||
const without = mcpHttpRequestInit(undefined, { id: "call-2", method: "tools/call", params: { name: "export" } });
|
||||
expect(JSON.parse(without.body).params._meta?.progressToken).toBeUndefined();
|
||||
});
|
||||
|
||||
it("reads progress interleaved with the result on one SSE stream, before the result", async () => {
|
||||
const { mcpProgressNotificationOf } = await import("../services/mcp-http.js");
|
||||
const frames = [
|
||||
{ jsonrpc: "2.0", method: "notifications/progress", params: { progressToken: "token-1", progress: 1, total: 3, message: "Exporting rows" } },
|
||||
{ jsonrpc: "2.0", method: "notifications/message", params: { level: "info", data: "unrelated" } },
|
||||
{ jsonrpc: "2.0", method: "notifications/progress", params: { progressToken: "someone-else", progress: 9 } },
|
||||
{ jsonrpc: "2.0", method: "notifications/progress", params: { progressToken: "token-1", progress: 2, total: 3 } },
|
||||
{ jsonrpc: "2.0", id: "call-1", result: { content: [{ type: "text", text: "done" }] } },
|
||||
{ jsonrpc: "2.0", method: "notifications/progress", params: { progressToken: "token-1", progress: 3, total: 3 } },
|
||||
];
|
||||
// Split mid-event so frames cross chunk boundaries.
|
||||
const text = frames.map((frame) => `event: message\r\ndata: ${JSON.stringify(frame)}\r\n\r\n`).join("");
|
||||
const encoded = new TextEncoder().encode(text);
|
||||
const body = new ReadableStream<Uint8Array>({
|
||||
start(controller) {
|
||||
for (let i = 0; i < encoded.length; i += 37) controller.enqueue(encoded.slice(i, i + 37));
|
||||
controller.close();
|
||||
},
|
||||
});
|
||||
const seen: unknown[] = [];
|
||||
let resultSeenBeforeProgress = false;
|
||||
const result = await readMcpHttpResponse(new Response(body, { headers: { "content-type": "text/event-stream" } }), "call-1", {
|
||||
onNotification: (message) => {
|
||||
const update = mcpProgressNotificationOf(message, "token-1");
|
||||
if (update) seen.push(update);
|
||||
},
|
||||
onProgress: (progress) => { if (progress.matchedResponse && seen.length === 0) resultSeenBeforeProgress = true; },
|
||||
});
|
||||
expect(result).toMatchObject({ id: "call-1", result: { content: [{ text: "done" }] } });
|
||||
expect(resultSeenBeforeProgress).toBe(false);
|
||||
expect(seen).toEqual([
|
||||
{ progress: 1, total: 3, message: "Exporting rows" },
|
||||
{ progress: 2, total: 3, message: undefined },
|
||||
]);
|
||||
});
|
||||
|
||||
it("never lets a failing progress observer fail the call", async () => {
|
||||
const text = [
|
||||
{ jsonrpc: "2.0", method: "notifications/progress", params: { progressToken: "t", progress: 1 } },
|
||||
{ jsonrpc: "2.0", id: 7, result: { ok: true } },
|
||||
].map((frame) => `data: ${JSON.stringify(frame)}\n\n`).join("");
|
||||
const result = await readMcpHttpResponse(new Response(text, { headers: { "content-type": "text/event-stream" } }), 7, {
|
||||
onNotification: () => { throw new Error("observer broke"); },
|
||||
});
|
||||
expect(result).toMatchObject({ result: { ok: true } });
|
||||
});
|
||||
|
||||
it("ignores requests, responses and other tokens", async () => {
|
||||
const { mcpProgressNotificationOf } = await import("../services/mcp-http.js");
|
||||
expect(mcpProgressNotificationOf({ method: "notifications/progress", params: { progressToken: "t", progress: 1 } }, undefined)).toBeNull();
|
||||
expect(mcpProgressNotificationOf({ id: 1, method: "notifications/progress", params: { progressToken: "t", progress: 1 } }, "t")).toBeNull();
|
||||
expect(mcpProgressNotificationOf({ method: "notifications/progress", params: { progressToken: "x", progress: 1 } }, "t")).toBeNull();
|
||||
expect(mcpProgressNotificationOf({ method: "notifications/progress", params: { progressToken: "t", progress: 1 } }, "t")).toEqual({ progress: 1, total: undefined, message: undefined });
|
||||
});
|
||||
|
||||
it("does not count handled progress against the result's size limit", async () => {
|
||||
const note = `data: ${JSON.stringify({ jsonrpc: "2.0", method: "notifications/progress", params: { progressToken: "t", progress: 1, message: "x".repeat(200) } })}\n\n`;
|
||||
const text = note.repeat(50) + `data: ${JSON.stringify({ jsonrpc: "2.0", id: 1, result: { ok: true } })}\n\n`;
|
||||
expect(Buffer.byteLength(text)).toBeGreaterThan(5_000);
|
||||
const result = await readMcpHttpResponse(new Response(text, { headers: { "content-type": "text/event-stream" } }), 1, { maxBytes: 1_000, limitPerMessage: true });
|
||||
expect(result).toMatchObject({ result: { ok: true } });
|
||||
// Callers without an overall deadline (catalog discovery) keep the total limit,
|
||||
// so a stream of small notifications cannot keep them waiting forever.
|
||||
await expect(readMcpHttpResponse(new Response(text, { headers: { "content-type": "text/event-stream" } }), 1, { maxBytes: 1_000 }))
|
||||
.rejects.toMatchObject({ reason: "too_large" });
|
||||
});
|
||||
|
||||
it("still rejects one SSE message over the limit, complete or not", async () => {
|
||||
const big = `data: ${JSON.stringify({ jsonrpc: "2.0", id: 1, result: { text: "x".repeat(2_000) } })}`;
|
||||
for (const text of [`${big}\n\n`, big]) {
|
||||
await expect(readMcpHttpResponse(new Response(text, { headers: { "content-type": "text/event-stream" } }), 1, { maxBytes: 1_000, limitPerMessage: true }))
|
||||
.rejects.toMatchObject({ reason: "too_large" });
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,74 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||
import { agents, companies, createDb, heartbeatRuns, issues, toolInvocations, type Db } from "@paperclipai/db";
|
||||
import { listToolCallProgress, saveToolCallProgress } from "../services/tool-call-progress.js";
|
||||
import { assignedToolName } from "../services/native-runtime/assigned-mcp-tools.js";
|
||||
import { startRecoveryPostgres } from "./helpers/recovery-postgres.js";
|
||||
|
||||
describe("saved tool call progress", () => {
|
||||
let temporary: Awaited<ReturnType<typeof startRecoveryPostgres>>;
|
||||
let db: Db;
|
||||
|
||||
beforeAll(async () => {
|
||||
temporary = await startRecoveryPostgres("tool-call-progress-");
|
||||
db = createDb(temporary.connectionString);
|
||||
}, 120_000);
|
||||
afterAll(async () => { await temporary?.cleanup(); });
|
||||
|
||||
async function task(prefix: string) {
|
||||
const companyId = randomUUID();
|
||||
const issueId = randomUUID();
|
||||
await db.insert(companies).values({ id: companyId, name: prefix, issuePrefix: prefix });
|
||||
const [agent] = await db.insert(agents).values({ companyId, name: "Agent", adapterType: "process" }).returning();
|
||||
const [run] = await db.insert(heartbeatRuns).values({ companyId, agentId: agent!.id, status: "running" }).returning();
|
||||
await db.insert(issues).values({ id: issueId, companyId, title: "Export" });
|
||||
const call = async (values: Partial<typeof toolInvocations.$inferInsert> = {}) => (await db.insert(toolInvocations).values({
|
||||
companyId, issueId, runId: run!.id, agentId: agent!.id, toolName: "kv_demo__slow_with_progress",
|
||||
status: "executing", startedAt: new Date("2026-11-13T10:00:00Z"), ...values,
|
||||
}).returning())[0]!;
|
||||
return { companyId, issueId, runId: run!.id, call };
|
||||
}
|
||||
const progress = (value: number) => ({ progress: value, total: 10, message: "Exporting", updatedAt: "2026-11-13T10:00:05.000Z" });
|
||||
|
||||
it("returns a task's running calls with their latest progress, and nothing from another company or task", async () => {
|
||||
const mine = await task("PRGA");
|
||||
const theirs = await task("PRGB");
|
||||
const running = await mine.call();
|
||||
await saveToolCallProgress(db, mine.companyId, running.id)(progress(3));
|
||||
await saveToolCallProgress(db, mine.companyId, running.id)(progress(4));
|
||||
const other = await theirs.call();
|
||||
await saveToolCallProgress(db, theirs.companyId, other.id)(progress(9));
|
||||
// A write naming the wrong company never lands.
|
||||
await saveToolCallProgress(db, mine.companyId, other.id)(progress(1));
|
||||
const now = new Date("2026-11-13T10:01:00Z");
|
||||
|
||||
expect(await listToolCallProgress(db, mine.companyId, mine.issueId, now)).toEqual([{
|
||||
invocationId: running.id, runId: mine.runId, toolName: "kv_demo__slow_with_progress",
|
||||
nativeToolName: assignedToolName("kv_demo__slow_with_progress"),
|
||||
startedAt: "2026-11-13T10:00:00.000Z", progress: 4, total: 10, message: "Exporting",
|
||||
updatedAt: "2026-11-13T10:00:05.000Z", done: false,
|
||||
}]);
|
||||
expect(await listToolCallProgress(db, mine.companyId, theirs.issueId, now)).toEqual([]);
|
||||
expect(await listToolCallProgress(db, theirs.companyId, mine.issueId, now)).toEqual([]);
|
||||
expect((await db.select().from(toolInvocations).where(eq(toolInvocations.id, other.id)))[0]!.progress).toMatchObject({ progress: 9 });
|
||||
});
|
||||
|
||||
it("never writes progress onto a finished call, and reports it as done", async () => {
|
||||
const mine = await task("PRGC");
|
||||
const running = await mine.call({ createdAt: new Date() });
|
||||
await saveToolCallProgress(db, mine.companyId, running.id)(progress(2));
|
||||
expect(await listToolCallProgress(db, mine.companyId, mine.issueId)).toHaveLength(1);
|
||||
await db.update(toolInvocations).set({ status: "succeeded", completedAt: new Date() }).where(eq(toolInvocations.id, running.id));
|
||||
// A late, throttled save after the result must not overwrite it.
|
||||
await saveToolCallProgress(db, mine.companyId, running.id)(progress(8));
|
||||
expect((await db.select().from(toolInvocations).where(eq(toolInvocations.id, running.id)))[0]!.progress).toMatchObject({ progress: 2 });
|
||||
// A chat that missed the end learns it; a call that never saved progress is reported too.
|
||||
const notSaved = await mine.call({ createdAt: new Date() });
|
||||
await db.update(toolInvocations).set({ status: "succeeded", completedAt: new Date() }).where(eq(toolInvocations.id, notSaved.id));
|
||||
const listed = await listToolCallProgress(db, mine.companyId, mine.issueId);
|
||||
expect(listed.map((entry) => [entry.invocationId, entry.done, entry.progress]).sort()).toEqual(
|
||||
[[running.id, true, 2], [notSaved.id, true, 0]].sort(),
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,174 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { TOOL_CALL_PROGRESS_MESSAGE_MAX_LENGTH } from "@paperclipai/shared";
|
||||
import {
|
||||
createToolCallProgressReporter,
|
||||
TOOL_CALL_PROGRESS_PUBLISH_INTERVAL_MS,
|
||||
TOOL_CALL_PROGRESS_SAVE_INTERVAL_MS,
|
||||
} from "../services/tool-call-progress.js";
|
||||
import { assignedToolName } from "../services/native-runtime/assigned-mcp-tools.js";
|
||||
|
||||
function reporter() {
|
||||
const published: Array<{ companyId: string; type: string; payload: Record<string, unknown> }> = [];
|
||||
const saved: unknown[] = [];
|
||||
const instance = createToolCallProgressReporter({
|
||||
companyId: "company-a",
|
||||
runId: "run-a",
|
||||
issueId: "issue-a",
|
||||
agentId: "agent-a",
|
||||
invocationId: "invocation-a",
|
||||
toolName: "kv_demo__slow_with_progress",
|
||||
startedAt: new Date("2026-11-13T10:00:00Z"),
|
||||
publish: (event) => { published.push(event); },
|
||||
save: async (progress) => { saved.push(progress); },
|
||||
});
|
||||
return { instance, published, saved };
|
||||
}
|
||||
|
||||
describe("tool call progress reporter", () => {
|
||||
beforeEach(() => { vi.useFakeTimers(); vi.setSystemTime(new Date("2026-11-13T10:00:01Z")); });
|
||||
afterEach(() => { vi.useRealTimers(); });
|
||||
|
||||
it("publishes at most one update per interval and always the latest", () => {
|
||||
const { instance, published } = reporter();
|
||||
for (let step = 1; step <= 50; step++) instance.report({ progress: step, total: 50, message: `Step ${step}` });
|
||||
// The first update goes out at once; the rest wait for the interval.
|
||||
expect(published.map((event) => event.payload.progress)).toEqual([1]);
|
||||
vi.advanceTimersByTime(TOOL_CALL_PROGRESS_PUBLISH_INTERVAL_MS);
|
||||
expect(published.map((event) => event.payload.progress)).toEqual([1, 50]);
|
||||
vi.advanceTimersByTime(TOOL_CALL_PROGRESS_PUBLISH_INTERVAL_MS * 4);
|
||||
expect(published).toHaveLength(2);
|
||||
// A steady stream for one second publishes about four times, never per update.
|
||||
for (let tick = 0; tick < 100; tick++) {
|
||||
instance.report({ progress: 51 + tick, total: 200 });
|
||||
vi.advanceTimersByTime(10);
|
||||
}
|
||||
expect(published.length - 2).toBeLessThanOrEqual(1_000 / TOOL_CALL_PROGRESS_PUBLISH_INTERVAL_MS + 1);
|
||||
});
|
||||
|
||||
it("saves on the invocation less often than it publishes", async () => {
|
||||
const { instance, saved } = reporter();
|
||||
for (let tick = 0; tick < 200; tick++) {
|
||||
instance.report({ progress: tick, total: 200 });
|
||||
await vi.advanceTimersByTimeAsync(10);
|
||||
}
|
||||
expect(saved.length).toBeLessThanOrEqual(2_000 / TOOL_CALL_PROGRESS_SAVE_INTERVAL_MS + 1);
|
||||
expect(saved.length).toBeGreaterThanOrEqual(2);
|
||||
});
|
||||
|
||||
it("ties every event to its company, run and tool call", () => {
|
||||
const { instance, published } = reporter();
|
||||
instance.report({ progress: 1, total: 4, message: "Reading" });
|
||||
expect(published[0]).toEqual({
|
||||
companyId: "company-a",
|
||||
type: "heartbeat.run.tool_progress",
|
||||
payload: {
|
||||
invocationId: "invocation-a",
|
||||
runId: "run-a",
|
||||
issueId: "issue-a",
|
||||
agentId: "agent-a",
|
||||
toolName: "kv_demo__slow_with_progress",
|
||||
nativeToolName: assignedToolName("kv_demo__slow_with_progress"),
|
||||
startedAt: "2026-11-13T10:00:00.000Z",
|
||||
updatedAt: "2026-11-13T10:00:01.000Z",
|
||||
progress: 1,
|
||||
total: 4,
|
||||
message: "Reading",
|
||||
done: false,
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("treats provider text as untrusted: plain, single-line and bounded", () => {
|
||||
const { instance, published } = reporter();
|
||||
instance.report({ progress: 1, message: `<b>bold</b>\n\u0007line${"x".repeat(1_000)}` });
|
||||
const message = published[0]!.payload.message as string;
|
||||
expect(message.startsWith("<b>bold</b> line ")).toBe(true);
|
||||
expect(message).not.toMatch(/[\n\u0007]/);
|
||||
expect(message.length).toBe(TOOL_CALL_PROGRESS_MESSAGE_MAX_LENGTH);
|
||||
});
|
||||
|
||||
it("drops unusable and out-of-order updates", () => {
|
||||
const { instance, published } = reporter();
|
||||
instance.report({ progress: "1", total: 2 });
|
||||
instance.report({ progress: Number.NaN });
|
||||
instance.report({ progress: -1 });
|
||||
expect(published).toEqual([]);
|
||||
instance.report({ progress: 5, total: 2 });
|
||||
expect(published[0]!.payload).toMatchObject({ progress: 5, total: null });
|
||||
vi.advanceTimersByTime(TOOL_CALL_PROGRESS_PUBLISH_INTERVAL_MS);
|
||||
instance.report({ progress: 3, total: 10 });
|
||||
vi.advanceTimersByTime(TOOL_CALL_PROGRESS_PUBLISH_INTERVAL_MS);
|
||||
expect(published).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("on finish stops pending work and tells viewers to drop the progress", () => {
|
||||
const { instance, published, saved } = reporter();
|
||||
instance.report({ progress: 1, total: 3 });
|
||||
instance.report({ progress: 2, total: 3 });
|
||||
instance.finish();
|
||||
vi.advanceTimersByTime(TOOL_CALL_PROGRESS_SAVE_INTERVAL_MS * 3);
|
||||
expect(published.map((event) => [event.payload.progress, event.payload.done])).toEqual([[1, false], [2, true]]);
|
||||
expect(saved).toHaveLength(1);
|
||||
instance.report({ progress: 3, total: 3 });
|
||||
expect(published).toHaveLength(2);
|
||||
});
|
||||
|
||||
it("a call that never reported progress finishes silently", () => {
|
||||
const { instance, published } = reporter();
|
||||
instance.finish();
|
||||
expect(published).toEqual([]);
|
||||
});
|
||||
|
||||
it("never throws into the call when publishing or saving fails", async () => {
|
||||
const instance = createToolCallProgressReporter({
|
||||
companyId: "c", runId: "r", issueId: null, agentId: null, invocationId: "i", toolName: "t",
|
||||
publish: () => { throw new Error("socket gone"); },
|
||||
save: async () => { throw new Error("db gone"); },
|
||||
});
|
||||
expect(() => instance.report({ progress: 1 })).not.toThrow();
|
||||
expect(() => instance.finish()).not.toThrow();
|
||||
await vi.runAllTimersAsync();
|
||||
});
|
||||
|
||||
it("keeps one save in flight, only the newest update waits, and finish drops it", async () => {
|
||||
let release!: () => void;
|
||||
const saves: number[] = [];
|
||||
const instance = createToolCallProgressReporter({
|
||||
companyId: "c", runId: "r", issueId: null, agentId: null, invocationId: "i", toolName: "t",
|
||||
publish: () => undefined,
|
||||
save: (progress) => { saves.push(progress.progress); return new Promise<void>((resolve) => { release = resolve; }); },
|
||||
});
|
||||
instance.report({ progress: 1 });
|
||||
// The database is slow: three more save intervals pass while the first write runs.
|
||||
for (let step = 2; step <= 4; step++) {
|
||||
vi.advanceTimersByTime(TOOL_CALL_PROGRESS_SAVE_INTERVAL_MS);
|
||||
instance.report({ progress: step });
|
||||
}
|
||||
vi.advanceTimersByTime(TOOL_CALL_PROGRESS_SAVE_INTERVAL_MS);
|
||||
expect(saves).toEqual([1]);
|
||||
release();
|
||||
await vi.advanceTimersByTimeAsync(0);
|
||||
expect(saves).toEqual([1, 4]);
|
||||
instance.report({ progress: 5 });
|
||||
vi.advanceTimersByTime(TOOL_CALL_PROGRESS_SAVE_INTERVAL_MS);
|
||||
instance.finish();
|
||||
release();
|
||||
await vi.advanceTimersByTimeAsync(TOOL_CALL_PROGRESS_SAVE_INTERVAL_MS * 3);
|
||||
expect(saves).toEqual([1, 4]);
|
||||
});
|
||||
|
||||
it("names the tool the way native runners name it, so their rows can find the call", () => {
|
||||
const instance = createToolCallProgressReporter({
|
||||
companyId: "c", runId: "r", issueId: null, agentId: null, invocationId: "i",
|
||||
toolName: "mcp.kv-demo-1a2b3c4d:slow-with-progress",
|
||||
publish: (event) => { names.push(event.payload.nativeToolName); },
|
||||
});
|
||||
const names: unknown[] = [];
|
||||
instance.report({ progress: 1 });
|
||||
expect(names).toEqual(["app_mcp_kv_demo_1a2b3c4d_slow_with_progress_889fb9a8d98e"]);
|
||||
});
|
||||
|
||||
it("gives each call its own progress token", () => {
|
||||
expect(reporter().instance.token).not.toBe(reporter().instance.token);
|
||||
});
|
||||
});
|
||||
@@ -3297,7 +3297,7 @@ describeEmbeddedPostgres("tool gateway service", () => {
|
||||
const calls = provider.mock.calls.map(([, init]) => JSON.parse(String(init.body)))
|
||||
.filter((payload) => payload.method === "tools/call");
|
||||
expect(calls).toHaveLength(1);
|
||||
expect(calls[0].params).toEqual({ name: "search_messages", arguments: parameters });
|
||||
expect(calls[0].params).toEqual({ name: "search_messages", arguments: parameters, _meta: { progressToken: expect.stringMatching(/^paperclip-progress-/) } });
|
||||
});
|
||||
|
||||
it("explains Google Workspace preview enrollment when a tool call is denied", async () => {
|
||||
|
||||
@@ -6,6 +6,9 @@ import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import express from "express";
|
||||
import { and, eq, sql } from "drizzle-orm";
|
||||
import type { LiveEvent } from "@paperclipai/shared";
|
||||
import { subscribeCompanyLiveEvents } from "../services/live-events.js";
|
||||
import { assignedToolName as nativeAssignedToolName } from "../services/native-runtime/assigned-mcp-tools.js";
|
||||
import request from "supertest";
|
||||
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
@@ -3584,6 +3587,7 @@ rl.on("line", (line) => {
|
||||
"kv_get",
|
||||
"kv_list",
|
||||
"kv_set",
|
||||
"slow_with_progress",
|
||||
]);
|
||||
|
||||
await allowAllToolsForAgent(db, company.id, agent.id);
|
||||
@@ -3618,6 +3622,61 @@ rl.on("line", (line) => {
|
||||
}
|
||||
});
|
||||
|
||||
it("shows a run's viewers the provider's progress for a long call, saves it on the call, and returns the same result", async () => {
|
||||
const company = await createCompany(db);
|
||||
const other = await createCompany(db);
|
||||
const agent = await createAgent(db, company.id);
|
||||
const { issue, run } = await createIssueAndRun(db, company.id, agent.id);
|
||||
const kvDemo: KvDemoHttpServer = createKvDemoHttpServer();
|
||||
const port = await kvDemo.listen(0, "127.0.0.1");
|
||||
const events: LiveEvent[] = [];
|
||||
const otherEvents: LiveEvent[] = [];
|
||||
const stop = subscribeCompanyLiveEvents(company.id, (event) => { events.push(event); });
|
||||
const stopOther = subscribeCompanyLiveEvents(other.id, (event) => { otherEvents.push(event); });
|
||||
try {
|
||||
const access = toolAccessService(db);
|
||||
const connection = await access.createConnection(company.id, {
|
||||
name: "KV demo progress fixture", connectionPurpose: "tool", transport: "mcp_remote", authKind: "none",
|
||||
ownership: "customer", connectionKind: "managed",
|
||||
config: { url: `http://127.0.0.1:${port}/mcp` }, transportConfig: { url: `http://127.0.0.1:${port}/mcp` },
|
||||
credentialSecretRefs: [], enabled: true, status: "active",
|
||||
});
|
||||
await access.refreshCatalog(connection.id, { actorType: "user", actorId: "board" });
|
||||
await allowAllToolsForAgent(db, company.id, agent.id);
|
||||
const gateway = createTestToolGatewayService(db);
|
||||
const session = await gateway.createSession({ companyId: company.id, agentId: agent.id, runId: run.id, issueId: issue.id });
|
||||
const slowTool = (await gateway.listToolsForSession(session.token))
|
||||
.find((tool) => tool.upstreamToolName === "slow_with_progress");
|
||||
expect(slowTool).toBeTruthy();
|
||||
|
||||
const result = await gateway.executeTool({
|
||||
sessionToken: session.token,
|
||||
tool: slowTool!.name,
|
||||
parameters: { steps: 6, stepMs: 200 },
|
||||
});
|
||||
expect(result).toMatchObject({ status: "completed", result: { data: { isError: false, transport: "mcp_http" } } });
|
||||
|
||||
const progress = events.filter((event) => event.type === "heartbeat.run.tool_progress");
|
||||
// Throttled: six provider updates over ~1.2s, a handful of events, the last one marks the end.
|
||||
expect(progress.length).toBeGreaterThanOrEqual(3);
|
||||
expect(progress.length).toBeLessThanOrEqual(7);
|
||||
expect(progress.every((event) => event.companyId === company.id && event.payload.runId === run.id
|
||||
&& event.payload.issueId === issue.id && event.payload.toolName === slowTool!.name
|
||||
&& event.payload.nativeToolName === nativeAssignedToolName(slowTool!.name))).toBe(true);
|
||||
expect(progress[0]!.payload).toMatchObject({ progress: 1, total: 6, message: "Finished step 1 of 6", done: false });
|
||||
expect(progress.at(-1)!.payload).toMatchObject({ done: true });
|
||||
expect(otherEvents.filter((event) => event.type === "heartbeat.run.tool_progress")).toEqual([]);
|
||||
const [invocation] = await db.select().from(toolInvocations)
|
||||
.where(eq(toolInvocations.id, String(progress[0]!.payload.invocationId)));
|
||||
expect(invocation).toMatchObject({ companyId: company.id, runId: run.id, status: "succeeded" });
|
||||
expect(invocation!.progress).toMatchObject({ total: 6 });
|
||||
} finally {
|
||||
stop();
|
||||
stopOther();
|
||||
await kvDemo.close();
|
||||
}
|
||||
});
|
||||
|
||||
it("expires legacy managed-connector approvals before provider dispatch", async () => {
|
||||
const company = await createCompany(db);
|
||||
const agent = await createAgent(db, company.id);
|
||||
|
||||
@@ -303,3 +303,15 @@ it("keeps a read set at 249 ms and replaces it at 250 ms", async () => {
|
||||
f.delivery.dispose();
|
||||
} finally { clock.mockRestore(); }
|
||||
});
|
||||
it("sends tool call progress only to clients watching that run", async () => {
|
||||
const f = fixture(); f.subscribe(["run-1"]);
|
||||
const progress = (id: number, runId: string): LiveEvent => ({ id, companyId: "company", type: "heartbeat.run.tool_progress",
|
||||
createdAt: "2026-11-13T00:00:00Z", payload: { runId, agentId: "agent", invocationId: `call-${id}`, progress: id, total: 10, message: "Exporting" } });
|
||||
f.delivery.enqueue(progress(1, "run-1")); f.delivery.enqueue(progress(2, "run-2")); await settle();
|
||||
const delivered = f.frames.filter(frame => frame.type === "heartbeat.run.tool_progress");
|
||||
expect(delivered.map(frame => frame.payload.runId)).toEqual(["run-1"]);
|
||||
// Another run's progress carries no content to this client.
|
||||
expect(JSON.stringify(f.frames)).not.toContain("call-2");
|
||||
expect(f.decorate).not.toHaveBeenCalled();
|
||||
f.delivery.dispose();
|
||||
});
|
||||
@@ -16,7 +16,10 @@ export function sharedDeliveryRead<T>(reads: LiveDeliveryReads | undefined, key:
|
||||
pending.catch(() => { if (reads.get(key) === pending) reads.delete(key); });
|
||||
return pending;
|
||||
}
|
||||
const transcript = (event: LiveEvent) => event.type === "heartbeat.run.event" || event.type === "heartbeat.run.log";
|
||||
// Per-run streams: sent only to clients that asked for the run. Tool progress
|
||||
// follows its run's transcript, since only an open transcript shows it.
|
||||
const transcript = (event: LiveEvent) => event.type === "heartbeat.run.event" || event.type === "heartbeat.run.log"
|
||||
|| event.type === "heartbeat.run.tool_progress";
|
||||
const runId = (event: LiveEvent) => typeof event.payload.runId === "string" ? event.payload.runId : null;
|
||||
interface Socket {
|
||||
readyState: number;
|
||||
|
||||
@@ -315,7 +315,7 @@ export function setupLiveEventsWebSocketServer(
|
||||
: payload.entityType === "issue" && typeof payload.entityId === "string" ? payload.entityId : null;
|
||||
const runId = typeof payload.runId === "string" ? payload.runId
|
||||
: payload.entityType === "heartbeat_run" && typeof payload.entityId === "string" ? payload.entityId : null;
|
||||
if (event.type === "heartbeat.run.event" || event.type === "heartbeat.run.log") {
|
||||
if (event.type === "heartbeat.run.event" || event.type === "heartbeat.run.log" || event.type === "heartbeat.run.tool_progress") {
|
||||
const allowed = await sharedDeliveryRead(reads, "transcript-scope", async () => (await access.decide({
|
||||
actor: context!.actor, action: "company_scope:read",
|
||||
resource: { type: "company", companyId: event.companyId } })).allowed);
|
||||
|
||||
@@ -15,6 +15,7 @@ import { assertBoard, getAccessibleResource } from "./authz.js";
|
||||
import { connectionRequestUserId } from "./connection-management-authz.js";
|
||||
import { authorizationService } from "../services/authorization.js";
|
||||
import { mcpAppViewsService, toMcpAppViewSummary } from "../services/mcp-app-views.js";
|
||||
import { listToolCallProgress } from "../services/tool-call-progress.js";
|
||||
import { ToolGatewayHttpError, type ToolGatewayService } from "../services/tool-gateway.js";
|
||||
|
||||
const uuidSchema = z.string().uuid();
|
||||
@@ -34,7 +35,7 @@ export function mcpAppsRoutes(db: Db, toolGateway: McpAppsGateway) {
|
||||
const router = Router();
|
||||
const views = mcpAppViewsService(db);
|
||||
|
||||
router.use("/issues/:issueId/mcp-app-views", async (req, res, next) => {
|
||||
const readableTask = async (req: Request, res: Response, next: () => void) => {
|
||||
assertBoard(req);
|
||||
if (!uuidSchema.safeParse(req.params.issueId).success) {
|
||||
res.status(404).json({ error: "Task not found" });
|
||||
@@ -57,7 +58,9 @@ export function mcpAppsRoutes(db: Db, toolGateway: McpAppsGateway) {
|
||||
res.locals.mcpAppIssue = issue;
|
||||
res.setHeader("Cache-Control", "no-store");
|
||||
next();
|
||||
});
|
||||
};
|
||||
router.use("/issues/:issueId/mcp-app-views", readableTask);
|
||||
router.use("/issues/:issueId/tool-call-progress", readableTask);
|
||||
|
||||
async function viewFor(req: Request, res: Response) {
|
||||
const issue = res.locals.mcpAppIssue as { id: string; companyId: string };
|
||||
@@ -162,6 +165,12 @@ export function mcpAppsRoutes(db: Db, toolGateway: McpAppsGateway) {
|
||||
res.status(201).json({ commentId: comment.id });
|
||||
});
|
||||
|
||||
// The latest progress of the task's running app tool calls (and recently finished ones, as done), for a reload mid-call.
|
||||
router.get("/issues/:issueId/tool-call-progress", async (_req, res) => {
|
||||
const issue = res.locals.mcpAppIssue as { id: string; companyId: string };
|
||||
res.json(await listToolCallProgress(db, issue.companyId, issue.id));
|
||||
});
|
||||
|
||||
router.get("/issues/:issueId/mcp-app-views", async (req, res) => {
|
||||
const issue = res.locals.mcpAppIssue as { id: string; companyId: string };
|
||||
res.json(await views.listForIssue(issue.companyId, issue.id));
|
||||
|
||||
@@ -1447,6 +1447,7 @@ const browserUseOperations = [
|
||||
// MCP Apps views in the task chat. Board only; reads follow task read access.
|
||||
const mcpAppsOperations = [
|
||||
["get", "/api/issues/{issueId}/mcp-app-views", "List a task's MCP Apps views", undefined],
|
||||
["get", "/api/issues/{issueId}/tool-call-progress", "Read the latest progress of a task's running app tool calls", undefined],
|
||||
["get", "/api/issues/{issueId}/mcp-app-views/{viewId}", "Read an MCP Apps view with its document", undefined],
|
||||
["post", "/api/issues/{issueId}/mcp-app-views/{viewId}/tool-calls", "Call a tool from an MCP Apps view", mcpAppToolCallSchema],
|
||||
["put", "/api/issues/{issueId}/mcp-app-views/{viewId}/model-context", "Store an MCP Apps view's context for the task's next run", mcpAppModelContextUpdateSchema],
|
||||
|
||||
@@ -209,11 +209,13 @@ export type McpJsonRpcRequest = { id?: string | number; method: string; params?:
|
||||
export function mcpHttpRequestInit(
|
||||
headers: Record<string, string> | undefined,
|
||||
message: McpJsonRpcRequest,
|
||||
options: { toolInputSchema?: unknown; declareTasks?: boolean; elicitationForm?: boolean } = {},
|
||||
options: { toolInputSchema?: unknown; declareTasks?: boolean; elicitationForm?: boolean; progressToken?: string } = {},
|
||||
): { method: "POST"; headers: Record<string, string>; body: string } {
|
||||
const requestHeaders = mcpHttpRequestHeaders(headers);
|
||||
const protocolVersion = mcpProtocolVersionOf(headers);
|
||||
let params = message.params ?? {};
|
||||
// Asks the server to stream `notifications/progress` for this request.
|
||||
if (options.progressToken) params = { ...params, _meta: { ...record(params._meta), progressToken: options.progressToken } };
|
||||
if (isModernMcpProtocolVersion(protocolVersion)) {
|
||||
params = {
|
||||
...params,
|
||||
@@ -666,6 +668,15 @@ export function parseMcpHttpResponseBody(bodyText: string, contentType: string |
|
||||
throw new SyntaxError("MCP SSE response contained no data events");
|
||||
}
|
||||
|
||||
/** A `notifications/progress` for this request's token, or null for any other message. */
|
||||
export function mcpProgressNotificationOf(message: unknown, progressToken: string | undefined): { progress: unknown; total: unknown; message: unknown } | null {
|
||||
const payload = record(message);
|
||||
if (!progressToken || !payload || payload.method !== "notifications/progress" || "id" in payload) return null;
|
||||
const params = record(payload.params);
|
||||
if (!params || params.progressToken !== progressToken) return null;
|
||||
return { progress: params.progress, total: params.total, message: params.message };
|
||||
}
|
||||
|
||||
export type McpHttpResponseProgress = {
|
||||
bytesReceived: number;
|
||||
chunksReceived: number;
|
||||
@@ -683,6 +694,15 @@ export async function readMcpHttpResponse(
|
||||
maxBytes?: number;
|
||||
onRequest?: (message: Record<string, unknown>) => Promise<void>;
|
||||
onProgress?: (progress: McpHttpResponseProgress) => void;
|
||||
/**
|
||||
* Bound an SSE stream per message instead of in total, so handled
|
||||
* notifications (such as progress) do not use up the result's limit. Only
|
||||
* for callers with an overall deadline: a stream of small notifications
|
||||
* could otherwise never end.
|
||||
*/
|
||||
limitPerMessage?: boolean;
|
||||
/** Notifications sent on this request's stream before its result. Must not block. */
|
||||
onNotification?: (message: Record<string, unknown>) => void;
|
||||
} = {},
|
||||
): Promise<unknown> {
|
||||
const progress: McpHttpResponseProgress = { bytesReceived: 0, chunksReceived: 0, eventsReceived: 0, matchedResponse: false, ended: false };
|
||||
@@ -714,6 +734,10 @@ export async function readMcpHttpResponse(
|
||||
return record;
|
||||
}
|
||||
if ("method" in record && "id" in record) await options.onRequest?.(record);
|
||||
else if ("method" in record) {
|
||||
// A notification observer never fails or delays the call.
|
||||
try { options.onNotification?.(record); } catch { /* ignored */ }
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
const event = async (value: string) => {
|
||||
@@ -729,17 +753,22 @@ export async function readMcpHttpResponse(
|
||||
progress.bytesReceived = bytes;
|
||||
if (value) progress.chunksReceived += 1;
|
||||
progress.ended = done;
|
||||
if (bytes > maxBytes) throw new McpHttpResponseError("too_large", "MCP response exceeded the size limit");
|
||||
// A JSON body is one message. An SSE stream is bounded in total unless the
|
||||
// caller asks for a per-message bound (see limitPerMessage).
|
||||
const perMessage = isStream && options.limitPerMessage === true;
|
||||
if (!perMessage && bytes > maxBytes) throw new McpHttpResponseError("too_large", "MCP response exceeded the size limit");
|
||||
buffer += decoder.decode(value, { stream: !done });
|
||||
if (isStream) {
|
||||
// Normalize CRLF after concatenating chunks, including split CR/LF pairs.
|
||||
buffer = buffer.replace(/\r\n/g, "\n");
|
||||
let boundary: number;
|
||||
while ((boundary = buffer.indexOf("\n\n")) >= 0) {
|
||||
if (perMessage && Buffer.byteLength(buffer.slice(0, boundary), "utf8") > maxBytes) throw new McpHttpResponseError("too_large", "MCP response exceeded the size limit");
|
||||
const result = await event(buffer.slice(0, boundary));
|
||||
buffer = buffer.slice(boundary + 2);
|
||||
if (result !== undefined) return result;
|
||||
}
|
||||
if (perMessage && Buffer.byteLength(buffer, "utf8") > maxBytes) throw new McpHttpResponseError("too_large", "MCP response exceeded the size limit");
|
||||
}
|
||||
if (done) break;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,195 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { and, desc, eq, gte, isNotNull, isNull } from "drizzle-orm";
|
||||
import { toolInvocations, type Db } from "@paperclipai/db";
|
||||
import {
|
||||
normalizeToolCallProgress,
|
||||
type LiveEventType,
|
||||
type ToolCallProgress,
|
||||
type ToolCallProgressSnapshot,
|
||||
} from "@paperclipai/shared";
|
||||
import { logger } from "../middleware/logger.js";
|
||||
import { publishLiveEvent } from "./live-events.js";
|
||||
import { assignedToolName } from "./native-runtime/assigned-mcp-tools.js";
|
||||
|
||||
/**
|
||||
* Live progress for a connected-app tool call that an agent run made.
|
||||
*
|
||||
* The provider streams MCP `notifications/progress` on the call's response.
|
||||
* Each update is published to the run's viewers at most every
|
||||
* PUBLISH_INTERVAL_MS (the latest one wins) and saved on the invocation at
|
||||
* most every SAVE_INTERVAL_MS, so a reload mid-call shows the latest state.
|
||||
* Nothing here is awaited by the call: the result is never delayed.
|
||||
*/
|
||||
export const TOOL_CALL_PROGRESS_PUBLISH_INTERVAL_MS = 250;
|
||||
export const TOOL_CALL_PROGRESS_SAVE_INTERVAL_MS = 1_000;
|
||||
export const TOOL_CALL_PROGRESS_EVENT: LiveEventType = "heartbeat.run.tool_progress";
|
||||
/** Progress older than this is not shown after a reload (the call is long gone). */
|
||||
const SNAPSHOT_MAX_AGE_MS = 24 * 60 * 60_000;
|
||||
|
||||
export interface ToolCallProgressReporter {
|
||||
/** Sent as the request's `_meta.progressToken`. */
|
||||
readonly token: string;
|
||||
/** Records a provider update. Never throws, never blocks. */
|
||||
report(update: { progress: unknown; total?: unknown; message?: unknown }): void;
|
||||
/** The call ended: stop timers and tell viewers to drop the progress. */
|
||||
finish(): void;
|
||||
}
|
||||
|
||||
type Publish = (input: { companyId: string; type: LiveEventType; payload: Record<string, unknown> }) => void;
|
||||
type Save = (progress: ToolCallProgress) => Promise<void>;
|
||||
|
||||
/** Leading-edge throttle that keeps only the latest value for its trailing run. */
|
||||
function latestThrottle<T>(intervalMs: number, run: (value: T) => void) {
|
||||
let lastAt = -Infinity;
|
||||
let pending: { value: T } | null = null;
|
||||
let timer: ReturnType<typeof setTimeout> | null = null;
|
||||
const flush = () => {
|
||||
timer = null;
|
||||
if (!pending) return;
|
||||
const { value } = pending;
|
||||
pending = null;
|
||||
lastAt = Date.now();
|
||||
run(value);
|
||||
};
|
||||
return {
|
||||
push(value: T) {
|
||||
pending = { value };
|
||||
if (timer) return;
|
||||
const wait = lastAt + intervalMs - Date.now();
|
||||
if (wait <= 0) flush();
|
||||
else { timer = setTimeout(flush, wait); timer.unref?.(); }
|
||||
},
|
||||
cancel() {
|
||||
if (timer) clearTimeout(timer);
|
||||
timer = null;
|
||||
pending = null;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export function createToolCallProgressReporter(input: {
|
||||
companyId: string;
|
||||
runId: string;
|
||||
issueId: string | null;
|
||||
agentId: string | null;
|
||||
invocationId: string;
|
||||
toolName: string;
|
||||
startedAt?: Date;
|
||||
publish?: Publish;
|
||||
save?: Save;
|
||||
}): ToolCallProgressReporter {
|
||||
const publish: Publish = input.publish ?? ((event) => { publishLiveEvent(event); });
|
||||
const startedAt = (input.startedAt ?? new Date()).toISOString();
|
||||
const token = `paperclip-progress-${randomUUID()}`;
|
||||
let finished = false;
|
||||
let last: ToolCallProgress | null = null;
|
||||
const snapshot = (progress: ToolCallProgress, done: boolean): ToolCallProgressSnapshot => ({
|
||||
...progress, invocationId: input.invocationId, runId: input.runId, toolName: input.toolName,
|
||||
nativeToolName: assignedToolName(input.toolName), startedAt, done,
|
||||
});
|
||||
const send = (value: ToolCallProgressSnapshot) => {
|
||||
try {
|
||||
publish({
|
||||
companyId: input.companyId,
|
||||
type: TOOL_CALL_PROGRESS_EVENT,
|
||||
payload: { ...value, issueId: input.issueId, agentId: input.agentId },
|
||||
});
|
||||
} catch (err) {
|
||||
logger.debug({ err, invocationId: input.invocationId }, "tool call progress publish failed");
|
||||
}
|
||||
};
|
||||
const live = latestThrottle<ToolCallProgress>(TOOL_CALL_PROGRESS_PUBLISH_INTERVAL_MS, (progress) => send(snapshot(progress, false)));
|
||||
// At most one save in flight; while it runs only the newest update waits.
|
||||
let saving = false;
|
||||
let queuedSave: ToolCallProgress | null = null;
|
||||
const save = (progress: ToolCallProgress) => {
|
||||
if (!input.save || finished) return;
|
||||
if (saving) { queuedSave = progress; return; }
|
||||
saving = true;
|
||||
input.save(progress)
|
||||
.catch((err) => logger.debug({ err, invocationId: input.invocationId }, "tool call progress save failed"))
|
||||
.finally(() => {
|
||||
saving = false;
|
||||
const next = queuedSave;
|
||||
queuedSave = null;
|
||||
if (next) save(next);
|
||||
});
|
||||
};
|
||||
const saved = latestThrottle<ToolCallProgress>(TOOL_CALL_PROGRESS_SAVE_INTERVAL_MS, save);
|
||||
return {
|
||||
token,
|
||||
report(update) {
|
||||
if (finished) return;
|
||||
const progress = normalizeToolCallProgress(update);
|
||||
// Progress must increase; a lower value is out of order and is dropped.
|
||||
if (!progress || (last && progress.progress < last.progress)) return;
|
||||
last = progress;
|
||||
live.push(progress);
|
||||
if (input.save) saved.push(progress);
|
||||
},
|
||||
finish() {
|
||||
if (finished) return;
|
||||
finished = true;
|
||||
live.cancel();
|
||||
saved.cancel();
|
||||
queuedSave = null;
|
||||
if (last) send(snapshot(last, true));
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** Saves a running call's latest progress. A finished call is never touched. */
|
||||
export function saveToolCallProgress(db: Db, companyId: string, invocationId: string): Save {
|
||||
return async (progress) => {
|
||||
await db.update(toolInvocations)
|
||||
.set({ progress })
|
||||
.where(and(eq(toolInvocations.id, invocationId), eq(toolInvocations.companyId, companyId), isNull(toolInvocations.completedAt)));
|
||||
};
|
||||
}
|
||||
|
||||
/** Finished calls are reported for this long, so a chat that missed their end learns it. */
|
||||
const FINISHED_REPORT_MS = 60 * 60_000;
|
||||
|
||||
/**
|
||||
* The latest progress of a task's running tool calls, for a reload mid-call,
|
||||
* plus its recently finished calls marked `done`. A chat drops a call only on
|
||||
* `done`: a running call may not have saved its first update yet.
|
||||
*/
|
||||
export async function listToolCallProgress(db: Db, companyId: string, issueId: string, now = new Date()): Promise<ToolCallProgressSnapshot[]> {
|
||||
const columns = {
|
||||
id: toolInvocations.id,
|
||||
runId: toolInvocations.runId,
|
||||
toolName: toolInvocations.toolName,
|
||||
progress: toolInvocations.progress,
|
||||
startedAt: toolInvocations.startedAt,
|
||||
createdAt: toolInvocations.createdAt,
|
||||
completedAt: toolInvocations.completedAt,
|
||||
};
|
||||
const task = and(eq(toolInvocations.companyId, companyId), eq(toolInvocations.issueId, issueId), isNotNull(toolInvocations.runId));
|
||||
const [running, finished] = await Promise.all([
|
||||
db.select(columns).from(toolInvocations).where(and(task,
|
||||
isNull(toolInvocations.completedAt),
|
||||
isNotNull(toolInvocations.progress),
|
||||
gte(toolInvocations.createdAt, new Date(now.getTime() - SNAPSHOT_MAX_AGE_MS)),
|
||||
)).orderBy(desc(toolInvocations.createdAt)).limit(50),
|
||||
db.select(columns).from(toolInvocations).where(and(task,
|
||||
gte(toolInvocations.completedAt, new Date(now.getTime() - FINISHED_REPORT_MS)),
|
||||
)).orderBy(desc(toolInvocations.completedAt)).limit(50),
|
||||
]);
|
||||
return [...running, ...finished].flatMap((row) => {
|
||||
const done = row.completedAt !== null;
|
||||
const savedAt = row.progress ? Date.parse(String(row.progress.updatedAt)) : NaN;
|
||||
const at = done ? row.completedAt! : Number.isFinite(savedAt) ? new Date(savedAt) : now;
|
||||
const progress = normalizeToolCallProgress(row.progress ?? (done ? { progress: 0 } : {}), at);
|
||||
if (!progress || !row.runId) return [];
|
||||
return [{
|
||||
...progress,
|
||||
invocationId: row.id,
|
||||
runId: row.runId,
|
||||
toolName: row.toolName,
|
||||
nativeToolName: assignedToolName(row.toolName),
|
||||
startedAt: (row.startedAt ?? row.createdAt).toISOString(),
|
||||
done,
|
||||
}];
|
||||
});
|
||||
}
|
||||
@@ -203,7 +203,9 @@ import {
|
||||
type McpInputRequired,
|
||||
mcpHttpRequestHeaders,
|
||||
parseMcpHttpResponseBody,
|
||||
mcpProgressNotificationOf,
|
||||
} from "./mcp-http.js";
|
||||
import { createToolCallProgressReporter, saveToolCallProgress, type ToolCallProgressReporter } from "./tool-call-progress.js";
|
||||
import {
|
||||
buildElicitationQuestionSet,
|
||||
elicitationQuestionHash,
|
||||
@@ -5505,6 +5507,8 @@ export function createToolGatewayService(
|
||||
protocolMethod?: string;
|
||||
protocolParams?: Record<string, unknown>;
|
||||
timeoutMs: number;
|
||||
/** Receives the call's `notifications/progress`; its token is sent with tools/call. */
|
||||
progressReporter?: ToolCallProgressReporter | null;
|
||||
},
|
||||
/** One restart and one deadline are shared by every protocol recovery in a call. */
|
||||
recovery: { restarted: boolean; deadline: number } = { restarted: false, deadline: Date.now() + input.timeoutMs },
|
||||
@@ -5585,6 +5589,8 @@ export function createToolGatewayService(
|
||||
try {
|
||||
const message = JSON.parse(line) as Record<string, unknown>;
|
||||
const id = typeof message.id === "number" ? message.id : null;
|
||||
const progressUpdate = mcpProgressNotificationOf(message, input.progressReporter?.token);
|
||||
if (progressUpdate) input.progressReporter!.report(progressUpdate);
|
||||
if (id !== null && pending.has(id)) {
|
||||
const waiter = pending.get(id)!;
|
||||
pending.delete(id);
|
||||
@@ -5760,6 +5766,7 @@ export function createToolGatewayService(
|
||||
withMeta(protocolVersion, input.protocolParams ?? {
|
||||
name: input.entry?.toolName,
|
||||
arguments: input.parameters ?? {},
|
||||
...(input.progressReporter ? { _meta: { progressToken: input.progressReporter.token } } : {}),
|
||||
}),
|
||||
);
|
||||
} catch (error) {
|
||||
@@ -7816,6 +7823,7 @@ export function createToolGatewayService(
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), ms);
|
||||
timer.unref?.();
|
||||
const progressReporter = toolCallProgressReporterFor(session, tool, invocationId);
|
||||
// The cached MCP session this call used, so a failure resets only this
|
||||
// identity's session and leaves other agents on the connection alone.
|
||||
let mcpSession: { scope: string; headers: Record<string, string>; sessionId: string } | undefined;
|
||||
@@ -8026,6 +8034,7 @@ export function createToolGatewayService(
|
||||
toolInputSchema: entry.inputSchema ?? undefined,
|
||||
// Paperclip can wait on a task here, durably (see waitForMcpTask).
|
||||
declareTasks: tasksOn,
|
||||
progressToken: progressReporter?.token,
|
||||
// A person can answer the app's form questions in the task chat.
|
||||
elicitationForm: Boolean(elicitationCall),
|
||||
}),
|
||||
@@ -8245,7 +8254,13 @@ export function createToolGatewayService(
|
||||
const body = response.ok
|
||||
? JSON.stringify(await readMcpHttpResponse(response, callRequestId, {
|
||||
maxBytes: MAX_REMOTE_MCP_RESPONSE_BYTES,
|
||||
// The call has a deadline (controller), so progress may outlast the total limit.
|
||||
limitPerMessage: true,
|
||||
onProgress: (progress) => { execution.stream = progress; },
|
||||
onNotification: (message) => {
|
||||
const update = mcpProgressNotificationOf(message, progressReporter?.token);
|
||||
if (update) progressReporter!.report(update);
|
||||
},
|
||||
onRequest: async (message) => {
|
||||
const pending = extractRemoteMcpPending(message);
|
||||
if (pending) {
|
||||
@@ -8508,15 +8523,31 @@ export function createToolGatewayService(
|
||||
);
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
progressReporter?.finish();
|
||||
}
|
||||
}
|
||||
|
||||
/** Progress is shown to the run's viewers, so only a call made by a run reports it. */
|
||||
function toolCallProgressReporterFor(session: ToolGatewaySession, tool: ToolGatewayDescriptor, invocationId: string | undefined) {
|
||||
if (!session.runId || !invocationId) return null;
|
||||
return createToolCallProgressReporter({
|
||||
companyId: session.companyId,
|
||||
runId: session.runId,
|
||||
issueId: session.issueId,
|
||||
agentId: session.agentId,
|
||||
invocationId,
|
||||
toolName: tool.name,
|
||||
save: saveToolCallProgress(db, session.companyId, invocationId),
|
||||
});
|
||||
}
|
||||
|
||||
async function executeLocalStdioTool(
|
||||
session: ToolGatewaySession,
|
||||
tool: ToolGatewayDescriptor,
|
||||
parameters: unknown,
|
||||
ms: number,
|
||||
useProviderDefaultTimeout = false,
|
||||
invocationId?: string,
|
||||
): Promise<RemoteHttpExecutionResult> {
|
||||
const { entry, connection } = await resolveConnectedLocalStdioTool(
|
||||
session,
|
||||
@@ -8530,8 +8561,10 @@ export function createToolGatewayService(
|
||||
template,
|
||||
grant,
|
||||
);
|
||||
const invoke = () => callLocalStdioMcp({ connection, entry, template, env, parameters,
|
||||
timeoutMs: useProviderDefaultTimeout && template.templateId === "paperclip.cognee-cloud" ? 60_000 : ms });
|
||||
const progressReporter = template.templateId === "paperclip.cognee-cloud" ? null : toolCallProgressReporterFor(session, tool, invocationId);
|
||||
const invoke = () => callLocalStdioMcp({ connection, entry, template, env, parameters, progressReporter,
|
||||
timeoutMs: useProviderDefaultTimeout && template.templateId === "paperclip.cognee-cloud" ? 60_000 : ms })
|
||||
.finally(() => progressReporter?.finish());
|
||||
// Cognee is a bundled HTTP client. Provider failures are tool failures, not
|
||||
// crashed local processes, and must never consume slots or restart budgets.
|
||||
const result = template.templateId === "paperclip.cognee-cloud"
|
||||
@@ -9579,6 +9612,7 @@ export function createToolGatewayService(
|
||||
args.parameters,
|
||||
executionTimeoutMs,
|
||||
args.timeoutMs === undefined,
|
||||
args.invocationId,
|
||||
)
|
||||
: null;
|
||||
if (!connectedMcpExecution) {
|
||||
@@ -10456,6 +10490,8 @@ export function createToolGatewayService(
|
||||
tool,
|
||||
parameters,
|
||||
executionTimeoutMs,
|
||||
false,
|
||||
invocation.id,
|
||||
)
|
||||
: null;
|
||||
const result = connectedMcpExecution
|
||||
@@ -13432,6 +13468,7 @@ export function createToolGatewayService(
|
||||
effectiveParameters,
|
||||
executionTimeoutMs,
|
||||
input.timeoutMs === undefined,
|
||||
invocationId,
|
||||
)
|
||||
: null;
|
||||
const result = recoveredGitHubPublication ?? (connectedMcpExecution
|
||||
|
||||
@@ -229,7 +229,7 @@
|
||||
"packages/shared/src/connector-availability.ts": "82aec0dfb2e9b8cb04e21c4bcd9aab74b087c76a",
|
||||
"packages/shared/src/connector-identity.test.ts": "d58357707faefa7c98bb906cdaeb4db39d31081a",
|
||||
"packages/shared/src/connector-identity.ts": "5ede83c9c5225dbe8af9cf3a49c74e477ef5b955",
|
||||
"packages/shared/src/constants.ts": "3149ee1cc5ffa34b50ecd493276b3a70fc46adc2",
|
||||
"packages/shared/src/constants.ts": "47973de810c0dfac21625e2687af80b37e8da00c",
|
||||
"packages/shared/src/crash-point-registry.ts": "3cbf2d290bc111061376b43020848623e493737c",
|
||||
"packages/shared/src/crash-points.test.ts": "3564a1f62d22f54a3c2fb0a72b3a6cde17c957d2",
|
||||
"packages/shared/src/crash-points.ts": "c15259f39610e352eac5423a491548ec1bae94fe",
|
||||
@@ -268,7 +268,7 @@
|
||||
"packages/shared/src/home-paths.ts": "fa4f1c440653b76b3dbb7a51026d557287deb45d",
|
||||
"packages/shared/src/humanize-connection.test.ts": "ba9e90a44c63452b0bf9f733adb90241921e8eca",
|
||||
"packages/shared/src/humanize-connection.ts": "fbd414476a0ef7106a7c80d8572208781b091699",
|
||||
"packages/shared/src/index.ts": "2d2b5ba044c9c7a22ff99022c0fac1e0da5e4a94",
|
||||
"packages/shared/src/index.ts": "7128882e69b34ebeeefbab5b1ef5f0252c13d4b4",
|
||||
"packages/shared/src/issue-activity-identity.test.ts": "3dc294640f54c2be67943e3e0149cf4eff2c95ca",
|
||||
"packages/shared/src/issue-activity-identity.ts": "1b0f07ec3fe0506aaaf80a4fa9893f3baf8f0075",
|
||||
"packages/shared/src/issue-attribution.test.ts": "84563685e7f3c48029243cf1cc24a3a09ee3cde9",
|
||||
@@ -366,6 +366,7 @@
|
||||
"packages/shared/src/telemetry/server-started-event.test.ts": "3d0e0e2289badb41c92ba0d69e3668d00b3b5fdb",
|
||||
"packages/shared/src/telemetry/state.ts": "a060af26a8e9a82a8afb0e2fccbfe6a59e9d0a8a",
|
||||
"packages/shared/src/telemetry/types.ts": "083d4341a72ac47bc1a3bbff163965de8b06f90a",
|
||||
"packages/shared/src/tool-call-progress.ts": "89655767bfd0180e99ac517754d72e25f753eb33",
|
||||
"packages/shared/src/trust-policy.ts": "88001a8d629a2af66dcd8d75b7b4d36298455ef6",
|
||||
"packages/shared/src/types/access.ts": "015d51792da07acaebe59e0e395698d207811d0d",
|
||||
"packages/shared/src/types/activity.ts": "6f5f27417a0bb26ec4daba6ec16a04ce31e484f2",
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import type { McpAppCallToolResult, McpAppModelContext, McpAppViewDetail, McpAppViewSummary } from "@paperclipai/shared";
|
||||
import type { McpAppCallToolResult, McpAppModelContext, McpAppViewDetail, McpAppViewSummary, ToolCallProgressSnapshot } from "@paperclipai/shared";
|
||||
import { api } from "./client";
|
||||
|
||||
/** MCP Apps views recorded for a task's tool calls. */
|
||||
@@ -17,4 +17,7 @@ export const mcpAppsApi = {
|
||||
/** Records that the viewer sent a view's proposed message (sent with this client request id). */
|
||||
messageSent: (issueId: string, viewId: string, clientRequestId: string) =>
|
||||
api.post<{ commentId: string }>(`/issues/${issueId}/mcp-app-views/${viewId}/messages`, { clientRequestId }),
|
||||
/** The latest progress of the task's running app tool calls (for a reload mid-call). */
|
||||
toolCallProgress: (issueId: string) =>
|
||||
api.get<ToolCallProgressSnapshot[]>(`/issues/${issueId}/tool-call-progress`, { cache: "no-store" }),
|
||||
};
|
||||
@@ -47,6 +47,7 @@ import { TaskChatLiveTail } from "@/components/task-chat/TaskChatLiveTail";
|
||||
import { McpAppIssueProvider } from "@/components/mcp-app/McpAppIssueContext";
|
||||
import { ProviderToolIconsProvider } from "@/components/ProviderToolIcon";
|
||||
import { McpAppTurnViews, mcpAppRunRefreshKey } from "@/components/mcp-app/McpAppTurnViews";
|
||||
import { ToolCallProgressProvider } from "@/components/task-chat/tool-call-progress";
|
||||
import { TaskChatRunnerHeader, TaskChatRunnerTurn } from "@/components/task-chat/TaskChatRunnerTurn";
|
||||
import { partitionRunHealthItems } from "@/components/task-chat/TaskChatRunHealthBanner";
|
||||
import {
|
||||
@@ -3372,6 +3373,7 @@ export function TaskChatThread(props: TaskChatThreadProps) {
|
||||
<TaskChatRequestHistoryContext.Provider value={issueId ?? "unknown"}>
|
||||
<McpAppIssueProvider issueId={issueId} sendMessage={composerDisabledReason ? null : sendMcpAppMessage}>
|
||||
<ProviderToolIconsProvider companyId={companyId}>
|
||||
<ToolCallProgressProvider issueId={issueId}>
|
||||
<TaskChatExpansionState.Provider value={expansionState.current}>
|
||||
<TaskChatScrollReady.Provider value={(historyRevealed || !historyPending || historyWaitExpired) && !scrollTargetPending}>
|
||||
<TaskChatPresentationProvider
|
||||
@@ -3719,6 +3721,7 @@ export function TaskChatThread(props: TaskChatThreadProps) {
|
||||
</TaskChatPresentationProvider>
|
||||
</TaskChatScrollReady.Provider>
|
||||
</TaskChatExpansionState.Provider>
|
||||
</ToolCallProgressProvider>
|
||||
</ProviderToolIconsProvider>
|
||||
</McpAppIssueProvider>
|
||||
</TaskChatRequestHistoryContext.Provider>
|
||||
|
||||
@@ -2,6 +2,7 @@ import { groupImageResources } from "./image-resources";
|
||||
import { AgentQueueWait } from "@/components/AgentQueueWait";
|
||||
import { agentQueueWaitOf } from "@/lib/agent-queue-wait";
|
||||
import { McpAppTurnViews, mcpAppRunRefreshKey } from "@/components/mcp-app/McpAppTurnViews";
|
||||
import { ToolCallProgressRunProvider } from "./tool-call-progress";
|
||||
import { useRef, type ReactNode } from "react";
|
||||
import type { ExecutionProjection } from "@paperclipai/shared";
|
||||
import { useSecondTick } from "@/hooks/useSecondTick";
|
||||
@@ -258,7 +259,7 @@ export function TaskChatRunnerTurn({
|
||||
));
|
||||
|
||||
return (
|
||||
<TaskChatShimmerScope items={items}><div
|
||||
<TaskChatShimmerScope items={items}><ToolCallProgressRunProvider runId={runId}><div
|
||||
className="flex min-w-0 flex-col"
|
||||
data-testid="task-chat-runner-turn"
|
||||
data-phase={status === "queued" ? "startup" : undefined}
|
||||
@@ -329,6 +330,6 @@ export function TaskChatRunnerTurn({
|
||||
</div>
|
||||
) : null}
|
||||
{!final && currentActivityItems.length === 0 ? <RunnerCurrentActivityTail status={status} waitingLabel={waitingLabel} /> : null}
|
||||
</div></TaskChatShimmerScope>
|
||||
</div></ToolCallProgressRunProvider></TaskChatShimmerScope>
|
||||
);
|
||||
}
|
||||
@@ -14,6 +14,8 @@ import { useTaskChatShimmer } from "./TaskChatShimmer";
|
||||
import { TaskChatDiffCounts, TaskChatDiffView } from "./TaskChatDiffView";
|
||||
import { useTaskReferenceLabel, withTaskReference } from "./task-reference";
|
||||
import { ProviderToolIcon, useProviderToolIcons } from "@/components/ProviderToolIcon";
|
||||
import type { ToolCallProgressSnapshot } from "@paperclipai/shared";
|
||||
import { TaskChatToolProgress, useToolCallProgress } from "./tool-call-progress";
|
||||
|
||||
const STATUS_ICON = {
|
||||
pending: { Icon: Loader2, spin: false, tone: "text-muted-foreground" },
|
||||
@@ -48,8 +50,15 @@ function TaskReferenceToolTarget({ taskRefId, target }: { taskRefId: string; tar
|
||||
* inset). An edit row carries its `+N −M` counts and opens to a compact,
|
||||
* height-capped diff of that one edit.
|
||||
*/
|
||||
export function TaskChatToolCard({ item }: { item: TaskChatToolItem }) {
|
||||
export function TaskChatToolCard({ item, progress: givenProgress }: {
|
||||
item: TaskChatToolItem;
|
||||
/** A fixed progress state (stories); otherwise a running call's live progress is shown. */
|
||||
progress?: ToolCallProgressSnapshot | null;
|
||||
}) {
|
||||
const { Icon, spin, tone } = STATUS_ICON[item.status];
|
||||
const liveProgress = useToolCallProgress(item);
|
||||
// Progress belongs to the running call only; a finished row drops it.
|
||||
const progress = item.status === "in_progress" ? (givenProgress !== undefined ? givenProgress : liveProgress) : null;
|
||||
const shimmer = useTaskChatShimmer(item.id);
|
||||
const RowIcon = toolTaxonomy(item.rawName ?? item.name).icon;
|
||||
const providerIcons = useProviderToolIcons(item.rawName ?? item.name);
|
||||
@@ -130,6 +139,7 @@ export function TaskChatToolCard({ item }: { item: TaskChatToolItem }) {
|
||||
) : null}
|
||||
</span>
|
||||
</button>
|
||||
{progress ? <TaskChatToolProgress progress={progress} /> : null}
|
||||
{item.target && showDetail && !item.diff?.lines?.length ? (
|
||||
<div
|
||||
className="task-chat-expanded-line-wrap ml-1.5 mt-0.5 min-w-0 max-w-full overflow-hidden border-l-2 border-border py-1 pl-2.5 font-mono text-(length:--text-micro) leading-relaxed text-muted-foreground"
|
||||
|
||||
@@ -226,6 +226,8 @@ export interface TaskChatToolItem {
|
||||
name: string;
|
||||
/** Raw ACP tool name pre-collapse ("mcp__server__tool") for taxonomy lookup. */
|
||||
rawName?: string;
|
||||
/** The app tool a wrapper call runs (`paperclip_call_assigned_tool`'s `name`), for its progress. */
|
||||
appToolName?: string;
|
||||
/** ACP ToolKind. */
|
||||
toolKind?: string;
|
||||
status: TaskChatToolStatus;
|
||||
|
||||
@@ -0,0 +1,224 @@
|
||||
// @vitest-environment jsdom
|
||||
import { act } from "react";
|
||||
import { createRoot, type Root } from "react-dom/client";
|
||||
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import type { ToolCallProgressSnapshot } from "@paperclipai/shared";
|
||||
import { TaskChatToolCard } from "./TaskChatToolCard";
|
||||
import {
|
||||
pickToolCallProgress,
|
||||
readToolCallProgressEvent,
|
||||
TaskChatToolProgress,
|
||||
toolCallNameMatches,
|
||||
ToolCallProgressProvider,
|
||||
ToolCallProgressRunProvider,
|
||||
ToolCallProgressStore,
|
||||
} from "./tool-call-progress";
|
||||
import type { TaskChatToolItem } from "./task-chat-model";
|
||||
import { transcriptToTaskChatItems, wrappedAppToolName } from "./transcript-adapter";
|
||||
|
||||
const savedProgress = vi.hoisted(() => ({ list: vi.fn() }));
|
||||
vi.mock("@/api/mcpApps", () => ({ mcpAppsApi: { toolCallProgress: savedProgress.list } }));
|
||||
|
||||
(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true;
|
||||
|
||||
const snapshot = (fields: Partial<ToolCallProgressSnapshot> = {}): ToolCallProgressSnapshot => ({
|
||||
invocationId: "call-1", runId: "run-1", toolName: "kv_demo__slow_with_progress",
|
||||
startedAt: "2026-11-13T10:00:00.000Z", updatedAt: "2026-11-13T10:00:05.000Z",
|
||||
progress: 2, total: 5, message: "Exporting rows", done: false, ...fields,
|
||||
});
|
||||
const tool = (fields: Partial<TaskChatToolItem> = {}): TaskChatToolItem => ({
|
||||
id: "toolu_1", kind: "tool", name: "slow_with_progress", rawName: "mcp__paperclip__kv_demo__slow_with_progress",
|
||||
status: "in_progress", ...fields,
|
||||
});
|
||||
|
||||
describe("tool call progress matching", () => {
|
||||
it("matches the gateway tool name inside the agent's tool name, and nothing else", () => {
|
||||
expect(toolCallNameMatches("mcp__paperclip__kv_demo__slow_with_progress", "kv_demo__slow_with_progress")).toBe(true);
|
||||
expect(toolCallNameMatches("kv_demo__slow_with_progress", "kv_demo__slow_with_progress")).toBe(true);
|
||||
expect(toolCallNameMatches("mcp__paperclip__kv_demo__slow_with_progressive", "kv_demo__slow_with_progress")).toBe(false);
|
||||
expect(toolCallNameMatches("mcp__paperclip__other_slow_with_progress", "kv_demo__slow_with_progress")).toBe(false);
|
||||
expect(toolCallNameMatches(undefined, "x")).toBe(false);
|
||||
// Gateway names carry "." and ":", which agents may rewrite.
|
||||
expect(toolCallNameMatches("mcp__paperclip__mcp_kv-demo-1a2b3c4d_slow-with-progress", "mcp.kv-demo-1a2b3c4d:slow-with-progress")).toBe(true);
|
||||
expect(toolCallNameMatches("mcp__paperclip__mcp.kv-demo-1a2b3c4d:slow-with-progress", "mcp.kv-demo-1a2b3c4d:slow-with-progress")).toBe(true);
|
||||
expect(toolCallNameMatches("mcp__paperclip__mcp.kv-demo-99999999:slow-with-progress", "mcp.kv-demo-1a2b3c4d:slow-with-progress")).toBe(false);
|
||||
});
|
||||
|
||||
it("finds a native runner's row by the shortened, hashed name it gives the tool", () => {
|
||||
const gatewayName = "mcp.kv-demo-1a2b3c4d:slow-with-progress";
|
||||
const nativeName = "app_mcp_kv_demo_1a2b3c4d_slow_with_progress_889fb9a8d98e";
|
||||
const entry = snapshot({ toolName: gatewayName, nativeToolName: nativeName });
|
||||
expect(pickToolCallProgress([entry], tool({ rawName: nativeName, name: nativeName }), "run-1")).toBe(entry);
|
||||
expect(pickToolCallProgress([entry], tool({ rawName: `mcp__paperclip__${nativeName}` }), "run-1")).toBe(entry);
|
||||
// Another tool's hash never matches.
|
||||
expect(pickToolCallProgress([entry], tool({ rawName: "app_mcp_kv_demo_1a2b3c4d_slow_with_progress_000000000000" }), "run-1")).toBeNull();
|
||||
expect(readToolCallProgressEvent({ ...entry })?.nativeToolName).toBe(nativeName);
|
||||
});
|
||||
|
||||
it("finds a wrapper call's row (OpenCode's paperclip_call_assigned_tool) by the tool it runs", () => {
|
||||
const gatewayName = "mcp.kv-demo-1a2b3c4d:slow-with-progress";
|
||||
const nativeName = "app_mcp_kv_demo_1a2b3c4d_slow_with_progress_889fb9a8d98e";
|
||||
const items = transcriptToTaskChatItems([{
|
||||
kind: "tool_call", ts: "2026-11-13T10:00:00.000Z", toolUseId: "call-1",
|
||||
name: "paperclip_call_assigned_tool", input: { name: nativeName, arguments: { steps: 10 } },
|
||||
}], { runId: "run-1", running: true });
|
||||
const row = items.find((item): item is TaskChatToolItem => item.kind === "tool")!;
|
||||
expect(row.appToolName).toBe(nativeName);
|
||||
const entry = snapshot({ toolName: gatewayName, nativeToolName: nativeName });
|
||||
expect(pickToolCallProgress([entry], row, "run-1")).toBe(entry);
|
||||
// Any other call, or a wrapper without a name, carries none.
|
||||
expect(wrappedAppToolName("Bash", { name: nativeName })).toBeUndefined();
|
||||
expect(wrappedAppToolName("mcp__paperclip__paperclip_call_assigned_tool", { name: nativeName })).toBe(nativeName);
|
||||
expect(wrappedAppToolName("paperclip_call_assigned_tool", { arguments: {} })).toBeUndefined();
|
||||
});
|
||||
|
||||
it("picks the newest unfinished call of the row's tool in the row's run", () => {
|
||||
const entries = [
|
||||
snapshot({ invocationId: "old", startedAt: "2026-11-13T09:00:00.000Z" }),
|
||||
snapshot({ invocationId: "new", startedAt: "2026-11-13T10:00:00.000Z" }),
|
||||
snapshot({ invocationId: "finished", startedAt: "2026-11-13T11:00:00.000Z", done: true }),
|
||||
snapshot({ invocationId: "other-run", runId: "run-2", startedAt: "2026-11-13T12:00:00.000Z" }),
|
||||
snapshot({ invocationId: "other-tool", toolName: "kv_demo__get", startedAt: "2026-11-13T13:00:00.000Z" }),
|
||||
];
|
||||
expect(pickToolCallProgress(entries, tool(), "run-1")?.invocationId).toBe("new");
|
||||
expect(pickToolCallProgress(entries, tool(), null)?.invocationId).toBe("other-run");
|
||||
expect(pickToolCallProgress(entries, tool({ rawName: "Bash", name: "Bash" }), "run-1")).toBeNull();
|
||||
});
|
||||
|
||||
it("keeps the newest state of each call, a finished call stays finished, and memory is bounded", () => {
|
||||
const store = new ToolCallProgressStore();
|
||||
store.put(snapshot({ progress: 3, updatedAt: "2026-11-13T10:00:06.000Z" }));
|
||||
store.put(snapshot({ progress: 2, updatedAt: "2026-11-13T10:00:05.000Z" }));
|
||||
expect([...store.entries()].map((entry) => entry.progress)).toEqual([3]);
|
||||
store.put(snapshot({ progress: 3, done: true, updatedAt: "2026-11-13T10:00:06.000Z" }));
|
||||
store.put(snapshot({ progress: 4, updatedAt: "2026-11-13T10:00:09.000Z" }));
|
||||
expect([...store.entries()]).toEqual([]);
|
||||
for (let i = 0; i < 80; i++) store.put(snapshot({ invocationId: `call-${i + 2}` }));
|
||||
expect([...store.entries()]).toHaveLength(50);
|
||||
});
|
||||
|
||||
it("a fresh read ends calls that finished unseen, and never drops a call it does not list", () => {
|
||||
const store = new ToolCallProgressStore();
|
||||
store.put(snapshot({ invocationId: "finished-while-hidden", updatedAt: "2026-11-13T10:00:05.000Z" }));
|
||||
store.put(snapshot({ invocationId: "not-saved-yet", updatedAt: "2026-11-13T10:00:06.000Z" }));
|
||||
// The read reports the finished call as done and does not list the other one yet.
|
||||
for (const entry of [snapshot({ invocationId: "finished-while-hidden", done: true, updatedAt: "2026-11-13T10:00:09.000Z" })]) store.put(entry);
|
||||
expect(pickToolCallProgress(store.entries(), tool(), "run-1")?.invocationId).toBe("not-saved-yet");
|
||||
expect([...store.entries()].map((entry) => entry.invocationId)).toEqual(["not-saved-yet"]);
|
||||
});
|
||||
|
||||
it("keeps running calls when a read reports many finished ones", () => {
|
||||
const store = new ToolCallProgressStore();
|
||||
for (let i = 0; i < 50; i++) store.put(snapshot({ invocationId: `running-${i}` }));
|
||||
for (let i = 0; i < 50; i++) store.put(snapshot({ invocationId: `history-${i}`, done: true }));
|
||||
expect([...store.entries()]).toHaveLength(50);
|
||||
expect([...store.entries()].every((entry) => entry.invocationId.startsWith("running-") && !entry.done)).toBe(true);
|
||||
});
|
||||
|
||||
it("a call's live end before a delayed read listing it as running keeps it ended", () => {
|
||||
const store = new ToolCallProgressStore();
|
||||
// Opened near the end: the live end arrives first, then the read made just before it.
|
||||
store.put(snapshot({ invocationId: "ending", done: true, updatedAt: "2026-11-13T10:00:10.000Z" }));
|
||||
store.put(snapshot({ invocationId: "ending", progress: 9, total: 10, updatedAt: "2026-11-13T10:00:09.000Z" }));
|
||||
expect([...store.entries()]).toEqual([]);
|
||||
expect(pickToolCallProgress(store.entries(), tool(), "run-1")).toBeNull();
|
||||
});
|
||||
|
||||
it("reads live events defensively", () => {
|
||||
expect(readToolCallProgressEvent({ ...snapshot(), message: "x".repeat(500) })?.message).toHaveLength(200);
|
||||
expect(readToolCallProgressEvent({ ...snapshot(), runId: undefined })).toBeNull();
|
||||
expect(readToolCallProgressEvent({ ...snapshot(), progress: "2" })).toBeNull();
|
||||
expect(readToolCallProgressEvent({ ...snapshot(), total: -1 })?.total).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("tool row progress states", () => {
|
||||
let container: HTMLDivElement;
|
||||
let root: Root;
|
||||
let queryClient: QueryClient;
|
||||
beforeEach(() => {
|
||||
container = document.createElement("div");
|
||||
document.body.appendChild(container);
|
||||
root = createRoot(container);
|
||||
queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } });
|
||||
savedProgress.list.mockReset();
|
||||
savedProgress.list.mockResolvedValue([]);
|
||||
});
|
||||
afterEach(() => {
|
||||
act(() => root.unmount());
|
||||
container.remove();
|
||||
});
|
||||
const render = (node: React.ReactNode) => act(() => root.render(
|
||||
<QueryClientProvider client={queryClient}>
|
||||
<ToolCallProgressProvider issueId="issue-1"><ToolCallProgressRunProvider runId="run-1">{node}</ToolCallProgressRunProvider></ToolCallProgressProvider>
|
||||
</QueryClientProvider>,
|
||||
));
|
||||
const settle = () => act(async () => { for (let i = 0; i < 5; i++) await Promise.resolve(); });
|
||||
|
||||
it("running with a total: a filling bar, the message, percent and elapsed time", () => {
|
||||
act(() => root.render(<TaskChatToolProgress progress={snapshot()} nowMs={Date.parse("2026-11-13T10:01:05.000Z")} />));
|
||||
const bar = container.querySelector('[role="progressbar"]')!;
|
||||
expect(bar.getAttribute("aria-valuenow")).toBe("40");
|
||||
expect(container.querySelector<HTMLElement>('[data-testid="task-chat-tool-progress-fill"]')!.style.width).toBe("40%");
|
||||
expect(container.textContent).toContain("Exporting rows");
|
||||
expect(container.textContent).toContain("40%");
|
||||
expect(container.querySelector('[data-testid="task-chat-tool-progress-elapsed"]')!.textContent).toBe("1m 5s");
|
||||
});
|
||||
|
||||
it("running without a total: an indeterminate sweep and no percent", () => {
|
||||
act(() => root.render(<TaskChatToolProgress progress={snapshot({ total: null, message: null })} nowMs={Date.parse("2026-11-13T10:00:12.000Z")} />));
|
||||
expect(container.querySelector('[data-testid="task-chat-tool-progress-indeterminate"]')).not.toBeNull();
|
||||
expect(container.querySelector('[role="progressbar"]')!.hasAttribute("aria-valuenow")).toBe(false);
|
||||
expect(container.textContent).not.toContain("%");
|
||||
expect(container.querySelector('[data-testid="task-chat-tool-progress-message"]')).toBeNull();
|
||||
expect(container.textContent).toContain("12s");
|
||||
});
|
||||
|
||||
it("renders provider text as plain text, never markup", () => {
|
||||
act(() => root.render(<TaskChatToolProgress progress={snapshot({ message: '<img src=x onerror="alert(1)"> done' })} nowMs={Date.parse("2026-11-13T10:00:01.000Z")} />));
|
||||
expect(container.querySelector("img")).toBeNull();
|
||||
expect(container.textContent).toContain('<img src=x onerror="alert(1)"> done');
|
||||
});
|
||||
|
||||
it("drops the progress when the call completes", () => {
|
||||
render(<TaskChatToolCard item={tool()} progress={snapshot()} />);
|
||||
expect(container.querySelector('[data-testid="task-chat-tool-progress"]')).not.toBeNull();
|
||||
render(<TaskChatToolCard item={tool({ status: "completed" })} progress={snapshot()} />);
|
||||
expect(container.querySelector('[data-testid="task-chat-tool-progress"]')).toBeNull();
|
||||
});
|
||||
|
||||
it("restores the saved progress of a running call after a reload", async () => {
|
||||
savedProgress.list.mockResolvedValue([snapshot({ progress: 3, message: "Writing file" })]);
|
||||
render(<TaskChatToolCard item={tool()} />);
|
||||
await vi.waitFor(async () => {
|
||||
await settle();
|
||||
expect(container.querySelector('[data-testid="task-chat-tool-progress-message"]')?.textContent).toBe("Writing file");
|
||||
});
|
||||
expect(savedProgress.list).toHaveBeenCalledWith("issue-1");
|
||||
});
|
||||
|
||||
it("reads fresh saved progress when someone leaves the task and comes back mid-call", async () => {
|
||||
savedProgress.list.mockResolvedValue([snapshot({ progress: 1, message: "Step 1" })]);
|
||||
render(<TaskChatToolCard item={tool()} />);
|
||||
await vi.waitFor(async () => {
|
||||
await settle();
|
||||
expect(container.querySelector('[data-testid="task-chat-tool-progress-message"]')?.textContent).toBe("Step 1");
|
||||
});
|
||||
act(() => root.render(<QueryClientProvider client={queryClient}><div /></QueryClientProvider>));
|
||||
savedProgress.list.mockResolvedValue([snapshot({ progress: 4, message: "Step 4", updatedAt: "2026-11-13T10:00:09.000Z" })]);
|
||||
render(<TaskChatToolCard item={tool()} />);
|
||||
await vi.waitFor(async () => {
|
||||
await settle();
|
||||
expect(container.querySelector('[data-testid="task-chat-tool-progress-message"]')?.textContent).toBe("Step 4");
|
||||
});
|
||||
expect(savedProgress.list).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it("never asks for progress for a finished row", async () => {
|
||||
render(<TaskChatToolCard item={tool({ status: "completed" })} />);
|
||||
await settle();
|
||||
expect(savedProgress.list).not.toHaveBeenCalled();
|
||||
expect(container.querySelector('[data-testid="task-chat-tool-progress"]')).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,221 @@
|
||||
import { createContext, useContext, useEffect, useMemo, useState, useSyncExternalStore, type ReactNode } from "react";
|
||||
import { useQuery } from "@tanstack/react-query";
|
||||
import {
|
||||
boundToolCallProgressMessage,
|
||||
normalizeToolCallProgress,
|
||||
toolCallProgressFraction,
|
||||
type ToolCallProgressSnapshot,
|
||||
} from "@paperclipai/shared";
|
||||
import { mcpAppsApi } from "@/api/mcpApps";
|
||||
import { useSecondTick } from "@/hooks/useSecondTick";
|
||||
import { useCompanyLiveEvent } from "@/context/LiveUpdatesProvider";
|
||||
import { queryKeys } from "@/lib/queryKeys";
|
||||
import { cn, formatDurationMs } from "@/lib/utils";
|
||||
import type { TaskChatToolItem } from "./task-chat-model";
|
||||
|
||||
/**
|
||||
* Live progress for a running connected-app tool call. The provider streams
|
||||
* MCP progress to Paperclip, which publishes it on the run's live events and
|
||||
* saves the latest on the call (read once, for a reload mid-call).
|
||||
*/
|
||||
|
||||
/** The run whose rows are showing, when known; progress from other runs is ignored. */
|
||||
const ToolCallProgressRunContext = createContext<string | null>(null);
|
||||
|
||||
export function ToolCallProgressRunProvider({ runId, children }: { runId: string | null | undefined; children: ReactNode }) {
|
||||
return <ToolCallProgressRunContext.Provider value={runId ?? null}>{children}</ToolCallProgressRunContext.Provider>;
|
||||
}
|
||||
|
||||
/** Lowercase words joined by "_", so "mcp.kv-demo-1a:slow" and "mcp_kv_demo_1a_slow" compare equal. */
|
||||
function comparableToolName(name: string) {
|
||||
return name.toLowerCase().replace(/[^a-z0-9]+/g, "_").replace(/^_+|_+$/g, "");
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a transcript row's tool name ("mcp__paperclip__<gateway tool>") is
|
||||
* the gateway tool a call reported. Agents may rewrite characters their tool
|
||||
* names cannot hold, so names are compared word by word, as a whole suffix.
|
||||
*/
|
||||
export function toolCallNameMatches(rawName: string | undefined, toolName: string): boolean {
|
||||
if (!rawName || !toolName) return false;
|
||||
const row = comparableToolName(rawName);
|
||||
const tool = comparableToolName(toolName);
|
||||
return Boolean(tool) && (row === tool || row.endsWith(`_${tool}`));
|
||||
}
|
||||
|
||||
/** A live event payload as a snapshot, or null when it is not one. Values are checked again here. */
|
||||
export function readToolCallProgressEvent(payload: Record<string, unknown>): ToolCallProgressSnapshot | null {
|
||||
const invocationId = typeof payload.invocationId === "string" ? payload.invocationId : null;
|
||||
const runId = typeof payload.runId === "string" ? payload.runId : null;
|
||||
const toolName = typeof payload.toolName === "string" ? payload.toolName : null;
|
||||
const nativeToolName = typeof payload.nativeToolName === "string" ? payload.nativeToolName : null;
|
||||
const startedAt = typeof payload.startedAt === "string" && Number.isFinite(Date.parse(payload.startedAt)) ? payload.startedAt : null;
|
||||
if (!invocationId || !runId || !toolName || !startedAt) return null;
|
||||
const updatedAt = typeof payload.updatedAt === "string" && Number.isFinite(Date.parse(payload.updatedAt)) ? new Date(payload.updatedAt) : new Date();
|
||||
const progress = normalizeToolCallProgress(payload, updatedAt);
|
||||
if (!progress) return null;
|
||||
return { ...progress, invocationId, runId, toolName, nativeToolName, startedAt, done: payload.done === true };
|
||||
}
|
||||
|
||||
/** Whether a row is a call of this tool, under its gateway name or the native runner's name for it. */
|
||||
export function rowShowsTool(item: Pick<TaskChatToolItem, "rawName" | "name" | "appToolName">, entry: Pick<ToolCallProgressSnapshot, "toolName" | "nativeToolName">): boolean {
|
||||
// A wrapper call (OpenCode's paperclip_call_assigned_tool) is matched by the tool it runs.
|
||||
return [item.appToolName, item.rawName ?? item.name].some((rowName) => Boolean(rowName) && (
|
||||
toolCallNameMatches(rowName, entry.toolName)
|
||||
|| (Boolean(entry.nativeToolName) && toolCallNameMatches(rowName, entry.nativeToolName!))));
|
||||
}
|
||||
|
||||
/** The progress for a row: the latest unfinished call of the same tool, in the same run when known. */
|
||||
export function pickToolCallProgress(
|
||||
entries: Iterable<ToolCallProgressSnapshot>,
|
||||
item: Pick<TaskChatToolItem, "rawName" | "name" | "appToolName">,
|
||||
runId: string | null,
|
||||
): ToolCallProgressSnapshot | null {
|
||||
let best: ToolCallProgressSnapshot | null = null;
|
||||
for (const entry of entries) {
|
||||
if (entry.done || (runId && entry.runId !== runId)) continue;
|
||||
if (!rowShowsTool(item, entry)) continue;
|
||||
if (!best || entry.startedAt > best.startedAt) best = entry;
|
||||
}
|
||||
return best;
|
||||
}
|
||||
|
||||
/** Running calls kept per chat; the oldest are forgotten first. */
|
||||
const MAX_TRACKED_CALLS = 50;
|
||||
/** Finished call ids remembered per chat, so a delayed read cannot revive one. */
|
||||
const MAX_REMEMBERED_FINISHED = 500;
|
||||
|
||||
/**
|
||||
* The chat's known tool call progress. Rows subscribe only while running, so a
|
||||
* burst of updates re-renders the running row, never the whole chat.
|
||||
*/
|
||||
export class ToolCallProgressStore {
|
||||
#entries = new Map<string, ToolCallProgressSnapshot>();
|
||||
#finished = new Set<string>();
|
||||
#listeners = new Set<() => void>();
|
||||
#onWanted: () => void;
|
||||
constructor(onWanted: () => void = () => undefined) { this.#onWanted = onWanted; }
|
||||
readonly subscribe = (listener: () => void) => {
|
||||
this.#listeners.add(listener);
|
||||
return () => { this.#listeners.delete(listener); };
|
||||
};
|
||||
/** A running row asks once for the saved progress (a reload mid-call). */
|
||||
want() { this.#onWanted(); }
|
||||
entries() { return this.#entries.values(); }
|
||||
/**
|
||||
* Keeps the newer update of each running call. A finished call is removed
|
||||
* and remembered (bounded, apart from shown progress), so a later "running"
|
||||
* snapshot of it, from a delayed read, never brings its progress back.
|
||||
*/
|
||||
put(snapshot: ToolCallProgressSnapshot) {
|
||||
const id = snapshot.invocationId;
|
||||
if (this.#finished.has(id)) return;
|
||||
if (snapshot.done) {
|
||||
this.#finished.add(id);
|
||||
while (this.#finished.size > MAX_REMEMBERED_FINISHED) this.#finished.delete(this.#finished.values().next().value!);
|
||||
if (!this.#entries.delete(id)) return;
|
||||
} else {
|
||||
const previous = this.#entries.get(id);
|
||||
if (previous && previous.updatedAt > snapshot.updatedAt) return;
|
||||
this.#entries.delete(id);
|
||||
this.#entries.set(id, snapshot);
|
||||
while (this.#entries.size > MAX_TRACKED_CALLS) this.#entries.delete(this.#entries.keys().next().value!);
|
||||
}
|
||||
for (const listener of this.#listeners) listener();
|
||||
}
|
||||
}
|
||||
|
||||
const ToolCallProgressStoreContext = createContext<ToolCallProgressStore | null>(null);
|
||||
|
||||
/**
|
||||
* One per task chat: listens for the task's tool call progress on the shared
|
||||
* live socket, and reads the saved progress once, only when a call is running.
|
||||
*/
|
||||
export function ToolCallProgressProvider({ issueId, children }: { issueId: string | null | undefined; children: ReactNode }) {
|
||||
const [wantedFor, setWantedFor] = useState<string | null>(null);
|
||||
// A new store (and a fresh read) per task and per mount: updates sent while the
|
||||
// chat was closed are only in the saved snapshot.
|
||||
const store = useMemo(() => new ToolCallProgressStore(() => setWantedFor(issueId ?? null)), [issueId]);
|
||||
const saved = useQuery({
|
||||
queryKey: queryKeys.toolCallProgress(issueId ?? ""),
|
||||
queryFn: () => mcpAppsApi.toolCallProgress(issueId!),
|
||||
enabled: Boolean(issueId) && wantedFor === issueId,
|
||||
// Read again whenever a chat with a running call mounts; live events cover the rest.
|
||||
staleTime: 0,
|
||||
refetchOnMount: "always",
|
||||
// Live events are not handled while the tab is hidden; read again on return.
|
||||
refetchOnWindowFocus: true,
|
||||
});
|
||||
useEffect(() => {
|
||||
// Finished calls arrive as done; a call missing from the read is never dropped,
|
||||
// since a running call may not have saved its first update yet.
|
||||
for (const entry of saved.data ?? []) store.put(entry);
|
||||
}, [saved.data, store]);
|
||||
useCompanyLiveEvent((event) => {
|
||||
if (event.type !== "heartbeat.run.tool_progress" || !issueId || event.payload?.issueId !== issueId) return;
|
||||
const snapshot = readToolCallProgressEvent(event.payload);
|
||||
if (snapshot) store.put(snapshot);
|
||||
});
|
||||
return <ToolCallProgressStoreContext.Provider value={store}>{children}</ToolCallProgressStoreContext.Provider>;
|
||||
}
|
||||
|
||||
const noSubscription = () => () => undefined;
|
||||
|
||||
/** Progress for a running tool row; null for any other row, which never re-renders for progress. */
|
||||
export function useToolCallProgress(item: TaskChatToolItem): ToolCallProgressSnapshot | null {
|
||||
const store = useContext(ToolCallProgressStoreContext);
|
||||
const runId = useContext(ToolCallProgressRunContext);
|
||||
const running = item.status === "in_progress" && store !== null;
|
||||
useEffect(() => { if (running) store!.want(); }, [running, store]);
|
||||
return useSyncExternalStore(
|
||||
running ? store!.subscribe : noSubscription,
|
||||
() => (running ? pickToolCallProgress(store!.entries(), item, runId) : null),
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* The running row's progress: a bar that fills when the total is known (a
|
||||
* quiet sweep otherwise), the provider's latest message as plain text, and
|
||||
* elapsed time.
|
||||
*/
|
||||
export function TaskChatToolProgress({ progress, nowMs }: { progress: ToolCallProgressSnapshot; nowMs?: number }) {
|
||||
// The shared one-second ticker, only while no fixed time is given.
|
||||
useSecondTick(nowMs === undefined);
|
||||
const now = nowMs ?? Date.now();
|
||||
const fraction = toolCallProgressFraction(progress);
|
||||
const percent = fraction === null ? null : Math.round(fraction * 100);
|
||||
// Rendered as text only: React escapes it, and it is bounded again here.
|
||||
const message = boundToolCallProgressMessage(progress.message);
|
||||
const elapsed = formatDurationMs(Math.max(0, now - Date.parse(progress.startedAt)));
|
||||
return (
|
||||
<div
|
||||
className="ml-1.5 mt-0.5 flex min-w-0 max-w-full flex-col gap-1 border-l-2 border-border py-1 pl-2.5"
|
||||
data-testid="task-chat-tool-progress"
|
||||
>
|
||||
<div
|
||||
role="progressbar"
|
||||
aria-label={message ?? "Working"}
|
||||
aria-valuemin={0}
|
||||
aria-valuemax={100}
|
||||
aria-valuenow={percent ?? undefined}
|
||||
className="relative h-1 w-full overflow-hidden rounded-sm bg-muted"
|
||||
>
|
||||
<div
|
||||
data-testid={percent === null ? "task-chat-tool-progress-indeterminate" : "task-chat-tool-progress-fill"}
|
||||
className={cn(
|
||||
"absolute inset-y-0 left-0 rounded-sm bg-(--status-agent-running)",
|
||||
percent === null ? "tool-call-progress-sweep" : "tool-call-progress-fill",
|
||||
)}
|
||||
style={percent === null ? undefined : { width: `${percent}%` }}
|
||||
/>
|
||||
</div>
|
||||
<div className="flex min-w-0 items-baseline gap-2 text-(length:--text-micro) leading-4 text-muted-foreground">
|
||||
{message ? (
|
||||
<span className="min-w-0 flex-1 truncate" data-testid="task-chat-tool-progress-message">{message}</span>
|
||||
) : <span className="flex-1" />}
|
||||
{percent !== null ? <span className="shrink-0 tabular-nums">{percent}%</span> : null}
|
||||
<span className="shrink-0 tabular-nums" data-testid="task-chat-tool-progress-elapsed">{elapsed}</span>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -197,6 +197,17 @@ function isOpaqueId(value: unknown): boolean {
|
||||
* The task a tool call is about, when its input names one by UUID. The row
|
||||
* shows that task's identifier (resolved at render time) instead of the id.
|
||||
*/
|
||||
/** Runners without direct app tools call them through this wrapper, naming the tool in its input. */
|
||||
const WRAPPED_APP_TOOL = "paperclip_call_assigned_tool";
|
||||
|
||||
/** The app tool a wrapper call runs (its input's `name`), or undefined for any other call. */
|
||||
export function wrappedAppToolName(rawName: string | null | undefined, input: unknown): string | undefined {
|
||||
if (!rawName || !(rawName === WRAPPED_APP_TOOL || rawName.endsWith(`_${WRAPPED_APP_TOOL}`) || rawName.endsWith(`.${WRAPPED_APP_TOOL}`))) return undefined;
|
||||
if (input == null || typeof input !== "object" || Array.isArray(input)) return undefined;
|
||||
const name = (input as Record<string, unknown>).name;
|
||||
return typeof name === "string" && name.trim() ? name.trim() : undefined;
|
||||
}
|
||||
|
||||
export function toolInputTaskRef(input: unknown): string | undefined {
|
||||
if (input == null || typeof input !== "object" || Array.isArray(input)) return undefined;
|
||||
const record = input as Record<string, unknown>;
|
||||
@@ -908,6 +919,7 @@ export function transcriptToTaskChatItems(
|
||||
}
|
||||
}
|
||||
if (entry.detail) applyToolDetail(existing, entry.detail);
|
||||
existing.appToolName ??= wrappedAppToolName(existing.rawName, entry.input);
|
||||
lastToolIndex = existingIndex!;
|
||||
} else {
|
||||
const tool: TaskChatToolItem = {
|
||||
@@ -917,6 +929,7 @@ export function transcriptToTaskChatItems(
|
||||
rawName: entry.name ?? undefined,
|
||||
target: summarizeToolInput(entry.input),
|
||||
...(toolInputTaskRef(entry.input) ? { taskRefId: toolInputTaskRef(entry.input) } : {}),
|
||||
...(wrappedAppToolName(entry.name, entry.input) ? { appToolName: wrappedAppToolName(entry.name, entry.input) } : {}),
|
||||
status: "in_progress",
|
||||
...(entry.parentToolUseId ? { parentToolUseId: entry.parentToolUseId } : {}),
|
||||
};
|
||||
|
||||
@@ -2155,6 +2155,9 @@ function handleLiveEvent(
|
||||
|
||||
const nameOf = (id: string) =>
|
||||
resolveAgentName(queryClient, expectedCompanyId, id);
|
||||
// Tool call progress is read by the tool row itself (useToolCallProgress); it
|
||||
// changes no cached run state and must not invalidate queries several times a second.
|
||||
if (event.type === "heartbeat.run.tool_progress") return;
|
||||
const payload = event.payload ?? {};
|
||||
// Resolve membership before terminal lifecycle patches remove live-run rows.
|
||||
const suppressRunToast =
|
||||
|
||||
@@ -3773,6 +3773,19 @@ span.paperclip-mention-chip[data-mention-kind="external-object"] {
|
||||
.skill-import-progress::-webkit-progress-value { transition: none; }
|
||||
}
|
||||
|
||||
/* Connected-app tool call progress: a filling bar when the total is known, a quiet sweep otherwise. */
|
||||
.tool-call-progress-fill {
|
||||
transition: width var(--motion-line-scroll) var(--motion-ease-standard);
|
||||
}
|
||||
.tool-call-progress-sweep {
|
||||
width: 40%;
|
||||
animation: skill-import-sweep var(--motion-approval-pulse) var(--motion-ease-standard) infinite;
|
||||
}
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
.tool-call-progress-fill { transition: none; }
|
||||
.tool-call-progress-sweep { animation: none; }
|
||||
}
|
||||
|
||||
/* Agent setup handoff: the same logo stack travels from trigger to prompt. */
|
||||
:root {
|
||||
--agent-setup-flight-duration: var(--motion-duration-deliberate);
|
||||
|
||||
@@ -318,6 +318,7 @@ export const queryKeys = {
|
||||
views: (issueId: string) => ["mcp-app-views", issueId] as const,
|
||||
detail: (issueId: string, viewId: string) => ["mcp-app-views", issueId, "detail", viewId] as const,
|
||||
},
|
||||
toolCallProgress: (issueId: string) => ["tool-call-progress", issueId] as const,
|
||||
issues: {
|
||||
recoveryActions: (id: string, actionId: string | null, runId: string | null) =>
|
||||
["issues", id, "recovery-actions", actionId, runId] as const,
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
import type { Meta, StoryObj } from "@storybook/react-vite";
|
||||
import type { ToolCallProgressSnapshot } from "@paperclipai/shared";
|
||||
import { TaskChatToolCard } from "@/components/task-chat/TaskChatToolCard";
|
||||
import type { TaskChatToolItem } from "@/components/task-chat/task-chat-model";
|
||||
|
||||
type Props = { state: "total" | "no-total" | "done" };
|
||||
|
||||
const TOOL: TaskChatToolItem = {
|
||||
id: "toolu_progress",
|
||||
kind: "tool",
|
||||
name: "slow_with_progress",
|
||||
rawName: "mcp__paperclip__kv_demo__slow_with_progress",
|
||||
target: "steps: 10",
|
||||
status: "in_progress",
|
||||
};
|
||||
|
||||
function progressFor(state: Props["state"]): ToolCallProgressSnapshot {
|
||||
const now = Date.now();
|
||||
return {
|
||||
invocationId: "call-1",
|
||||
runId: "run-1",
|
||||
toolName: "kv_demo__slow_with_progress",
|
||||
startedAt: new Date(now - 65_000).toISOString(),
|
||||
updatedAt: new Date(now - 400).toISOString(),
|
||||
progress: 4,
|
||||
total: state === "total" ? 10 : null,
|
||||
message: state === "total" ? "Exporting rows 4,000 of 10,000" : "Indexing documents",
|
||||
done: false,
|
||||
};
|
||||
}
|
||||
|
||||
function ToolProgressPreview({ state }: Props) {
|
||||
return (
|
||||
<div className="mx-auto flex w-full max-w-xl flex-col gap-1 p-6">
|
||||
<TaskChatToolCard item={{ ...TOOL, status: state === "done" ? "completed" : "in_progress" }} progress={progressFor(state)} />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
const meta = {
|
||||
id: "tasks-task-chat-tool-progress",
|
||||
title: "Tasks and conversations/Task creation and lifecycle/Tasks/Tool progress",
|
||||
component: ToolProgressPreview,
|
||||
parameters: {
|
||||
layout: "fullscreen",
|
||||
docs: {
|
||||
description: {
|
||||
component:
|
||||
"A long connected-app tool call shows the provider's progress while it runs: a filling bar when the total is known, a quiet sweep otherwise, the latest status line as plain text, and elapsed time. The row drops it when the call completes.",
|
||||
},
|
||||
},
|
||||
paperclip: {
|
||||
capabilities: ["tasks/discussion", "runs-adapters/runs"],
|
||||
status: "current",
|
||||
implementation: "production",
|
||||
boundary: "fixtures",
|
||||
notes: "Production tool row with fixed progress. Live progress arrives from MCP notifications/progress through the run's live events.",
|
||||
},
|
||||
},
|
||||
} satisfies Meta<typeof ToolProgressPreview>;
|
||||
export default meta;
|
||||
type Story = StoryObj<typeof meta>;
|
||||
|
||||
export const RunningWithTotal: Story = { name: "01 · Running, total known", args: { state: "total" } };
|
||||
export const RunningWithoutTotal: Story = { name: "02 · Running, total unknown", args: { state: "no-total" } };
|
||||
export const Done: Story = { name: "03 · Done", args: { state: "done" } };
|
||||
Reference in new issue
Block a user