fix(server): give remote MCP tool calls a 30 second default deadline and record stream failure evidence (#19768)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The tool gateway calls remote MCP servers over Streamable HTTP, and
a server can answer `tools/call` with a `text/event-stream` body
> - Such a server can send progress notifications before the response,
and a slow tool can take more than 10 seconds
> - The gateway aborts a remote MCP `tools/call` that uses the default
deadline after 10 seconds, the general tool default
> - When the call fails, the call evidence does not show how far the
stream got, or whether the cause was the deadline or a broken transport
> - This pull request gives remote MCP calls a default deadline of 30
seconds, the default response timeout of the remote HTTP client, and
records the failure cause and the stream progress
> - The benefit is that slow remote tools complete, and an operator can
tell a deadline from a peer reset in the call evidence

## Linked Issues or Issue Description

Refs #10147. That issue reports that a named MCP gateway aborts remote
tools after 10 seconds and ignores the connection timeoutMs. This pull
request raises the default deadline of a remote MCP `tools/call` to 30
seconds. It does not make the named gateway read the connection
timeoutMs.

Related: #11910 (open) keeps a connection callable after one timed-out
tools/call. On current master a timeout and a JSON-RPC response error
already keep the connection healthy, and this pull request keeps that
policy.

I searched pull requests and issues for "event stream tools/call", "SSE
MCP tool call hang", "remote MCP timeout stream", "mcp
text/event-stream" and "tool_timeout remote MCP". The only related open
pull request is #11910.

## What Changed

- `server/src/services/remote-http-fetch.ts`: the default response
timeout is exported as `DEFAULT_REMOTE_HTTP_RESPONSE_TIMEOUT_MS` (30,000
ms, unchanged value).
- `server/src/services/tool-gateway.ts`:
- A remote MCP `tools/call` that uses the default deadline now gets
`DEFAULT_REMOTE_HTTP_RESPONSE_TIMEOUT_MS` instead of the general tool
default of 10 seconds. The Railway `run-command` budget and an explicit
`timeoutMs` keep their values.
- The call evidence records the response status, content type and
request id before the body is read. A body that fails partway records
the bytes received so far as its size.
- A failed call records `failureKind` (`invocation_timeout`,
`protocol_failure` or `transport_failure`), the deadline and whether it
was aborted, the stream progress, and the error with its cause chain.
Each string in the chain is redacted with `redactSensitiveText`, and a
cyclic cause chain is cut with a "[Circular cause]" marker.
- A deadline abort is a 504 `tool_timeout` and starts a fresh MCP
session on the next call, as before. The connection health policy of
master does not change.
- A deliberate stop of a call (`elicitation_required`,
`provider_interaction_required`, `oauth_insufficient_scope`,
`oauth_refreshed_retry_required`) keeps its own reason when the deadline
passes while it is recorded, as on master. An error that is already a
gateway error also gets a failure kind: `tool_timeout` is
`invocation_timeout`; a malformed, invalid or oversized response is
`protocol_failure`; and `remote_http_*` (for example a DNS failure) and
`mcp_remote_fetch_failed` are `transport_failure`.
- A non-success response body is read as a stream, so a body that fails
partway records the bytes that arrived. The reader stops at the size
limit and cancels the body, as the `tools/call` reader does. #19785
makes the same reader change for the context path.
- `server/src/services/mcp-http.ts`: `readMcpHttpResponse` takes an
`onProgress` callback. It reports bytes, chunks and events received,
whether the matching response arrived, and whether the stream ended. As
on master, the read returns as soon as the response with the matching id
arrives.
- Tests:
- `server/src/__tests__/mcp-http.test.ts`: a partial event followed by a
peer error keeps the progress that was consumed, and releases the
reader.
- `server/src/__tests__/tool-gateway.test.ts`: six scenarios run against
a remote server that streams one progress event first: slow success
after 12 seconds, the default deadline, an explicit 1,500 ms deadline, a
peer reset, a cyclic error cause, and a primitive error value.
- `server/src/__tests__/tool-gateway.test.ts`: three regression tests: a
tool question sent after the deadline passed keeps
`elicitation_required`; a DNS failure records `transport_failure`; and a
503 body that fails partway records the bytes that arrived. All three
fail on 717e6c77c and pass on this head.
- `server/src/__tests__/tool-gateway-service.test.ts`: when the call
deadline passes during a forced Vercel token refresh, the call keeps the
409 `oauth_refreshed_retry_required` stop. `tool-gateway.ts` adds this
reason to the deliberate stops, so the deadline does not change it to a
504 `tool_timeout`.

## Verification

Head SHA: `25c6e703d1ded895989e9d2fcd6d6352a4f851f5`. It is the previous
head `7836447f2` plus commit `25c6e703d`, which keeps the
`oauth_refreshed_retry_required` reason when the call deadline passes
during a token refresh. At this head, run from `server` with pnpm
9.15.4: `pnpm typecheck`: no errors; `mcp-http.test.ts`,
`tool-gateway.test.ts` and `tool-gateway-service.test.ts`: 307 passed.
Without the one-line change in `tool-gateway.ts`, the new test "keeps
the explicit retry reason when the call deadline passes during the
Vercel token refresh" fails: it receives 504 `tool_timeout`, not 409
`oauth_refreshed_retry_required`.

Previous head: `7836447f24e2e6e9e0c402d4644d1f7f69b75cb8`. It is an
ordinary merge of master `76994c50275dbc005cc248e11189a38baa21719f`,
which includes #19877, into the previous head `00150e59c`. At this head,
run from `server` with pnpm 9.15.4: `pnpm typecheck` and the test
commands below. Results: typecheck: no errors; the 2 test files above:
116 passed.

Previous head: 00150e59c1. It merges
master 18b6b71d8. Run from `server`:

```sh
pnpm exec vitest run src/__tests__/mcp-http.test.ts src/__tests__/tool-gateway.test.ts
```

- With this branch: 116 passed. With tool-gateway-service.test.ts and
remote-mcp-socks-proxy.test.ts added: 315 passed in 4 files.
- With `services/mcp-http.ts`, `services/remote-http-fetch.ts` and
`services/tool-gateway.ts` from master and the test files from this
branch, measured at 717e6c77c: 7 failed, 106 passed. The failing tests
are the six "records remote SSE ... without confusing a call deadline
with a broken transport" scenarios and "retains consumed SSE progress
when a partial event fails and releases the reader".
- The existing master tests "keeps a remote MCP connection healthy and
starts a fresh MCP session after a tool call timeout" and "... after a
response body timeout" pass with this branch.
- Typecheck: `pnpm --filter @paperclipai/server typecheck` reports 29
errors, all "Cannot find module @paperclipai/plugin-sdk" and the
implicit any that follows, in the sandbox provider plugin packages of my
local install. None is in a file that this pull request touches.
- I ran the tests with an embedded PostgreSQL from the repository
dependencies. The full upstream suite runs in CI. I ran only the test
files named above.

## Risks

- Medium risk for slow tools: a remote MCP call that uses the default
deadline can now run for up to 30 seconds instead of 10 before it fails
with `tool_timeout`. An explicit `timeoutMs` still applies.
- The call evidence is larger: it can hold a redacted error chain and
its stack traces. All strings go through `redactSensitiveText`.
- No automatic replay: a timed-out `tools/call` is not sent again,
because it can have changed data in the remote app.

## Model Used

- Original change: the model that wrote it was not recorded. Port to
current master, the adaptation to the connection health policy of
master, and this description: Claude Opus 5.5 (claude-opus-5-5), with
tool use and terminal code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (no
user-facing documentation change needed)
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

server/src/__tests__/mcp-http.test.ts

@@ -5,8 +5,29 @@ import {
   MCP_PROTOCOL_VERSION,
   mcpHttpRequestHeaders,
   parseMcpHttpResponseBody,
+  readMcpHttpResponse,
 } from "../services/mcp-http.js";
 
+describe("MCP response progress", () => {
+  it("retains consumed SSE progress when a partial event fails and releases the reader", async () => {
+    const partial = 'data: {"jsonrpc":"2.0","method":"notifications/progress"}\n\ndata: {"jsonrpc":';
+    const failure = new Error("peer reset");
+    let reads = 0;
+    const body = new ReadableStream<Uint8Array>({
+      pull(controller) {
+        if (reads++ === 0) controller.enqueue(new TextEncoder().encode(partial));
+        else controller.error(failure);
+      },
+    }, { highWaterMark: 0 });
+    let progress: unknown;
+    await expect(readMcpHttpResponse(new Response(body, { headers: { "content-type": "text/event-stream" } }), "request", {
+      onProgress: (value) => { progress = value; },
+    })).rejects.toBe(failure);
+    expect(progress).toEqual({ bytesReceived: Buffer.byteLength(partial), chunksReceived: 1, eventsReceived: 1, matchedResponse: false, ended: false });
+    expect(body.locked).toBe(false);
+  });
+});
+
 describe("mcpHttpRequestHeaders", () => {
   it("advertises both JSON and SSE on every request", () => {
     expect(mcpHttpRequestHeaders()).toMatchObject({

server/src/__tests__/tool-gateway-service.test.ts

@@ -3330,7 +3330,9 @@ describeEmbeddedPostgres("tool gateway service", () => {
     });
   });
 
-  it("refreshes Vercel tokens after an upstream 401 without replaying the tool call", async () => {
+  for (const deadlinePassesDuringRefresh of [false, true]) it(deadlinePassesDuringRefresh
+    ? "keeps the explicit retry reason when the call deadline passes during the Vercel token refresh"
+    : "refreshes Vercel tokens after an upstream 401 without replaying the tool call", async () => {
     const { company, agent, run } = await createRunFixture(db);
     const { connection } = await createRemoteMcpToolFixture(db, company.id);
     await db.update(toolConnections).set({
@@ -3355,12 +3357,20 @@ describeEmbeddedPostgres("tool gateway service", () => {
     }).where(eq(connectionGrants.connectionId, connection.id));
     await db.insert(toolPolicies).values(readPolicyRow(company, "Allow Vercel-backed reads"));
 
-    const getToken = vi.fn<VercelConnectClient["getToken"]>(async (_request, options) => ({
-      token: options?.forceRefresh ? "fresh-provider-bearer" : "stale-provider-bearer",
-      tokenId: options?.forceRefresh ? "stk_fresh" : "stk_stale",
-      expiresAt: Date.now() + 60_000,
-      connector: { id: "scl_posthog", uid: "posthog-paperclip", type: "api-key" },
-    }));
+    const refreshing = Promise.withResolvers<void>();
+    const refreshGate = Promise.withResolvers<void>();
+    const getToken = vi.fn<VercelConnectClient["getToken"]>(async (_request, options) => {
+      if (options?.forceRefresh && deadlinePassesDuringRefresh) {
+        refreshing.resolve();
+        await refreshGate.promise;
+      }
+      return {
+        token: options?.forceRefresh ? "fresh-provider-bearer" : "stale-provider-bearer",
+        tokenId: options?.forceRefresh ? "stk_fresh" : "stk_stale",
+        expiresAt: Date.now() + 60_000,
+        connector: { id: "scl_posthog", uid: "posthog-paperclip", type: "api-key" },
+      };
+    });
     const evict = vi.fn<VercelConnectClient["evict"]>();
     const vercelConnectClient: VercelConnectClient = {
       getConnectorMetadata: vi.fn(),
@@ -3387,11 +3397,23 @@ describeEmbeddedPostgres("tool gateway service", () => {
     const tool = (await gateway.listToolsForSession(session.token))
       .find((candidate) => candidate.providerType === "mcp_remote_http");
 
-    await expect(gateway.executeTool({
+    if (deadlinePassesDuringRefresh) vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] });
+    const call = gateway.executeTool({
       sessionToken: session.token,
       tool: tool!.name,
       parameters: {},
-    })).rejects.toMatchObject({
+      ...(deadlinePassesDuringRefresh ? { timeoutMs: 1500 } : {}),
+    });
+    if (deadlinePassesDuringRefresh) {
+      try {
+        await refreshing.promise;
+        await vi.advanceTimersByTimeAsync(1500);
+      } finally {
+        vi.useRealTimers();
+        refreshGate.resolve();
+      }
+    }
+    await expect(call).rejects.toMatchObject({
       status: 409,
       reasonCode: "oauth_refreshed_retry_required",
     });

server/src/__tests__/tool-gateway.test.ts

@@ -4627,6 +4627,154 @@ rl.on("line", (line) => {
     }
   });
 
+  for (const scenario of ["slow success", "default deadline", "explicit deadline", "peer reset", "cyclic peer reset", "primitive peer reset"] as const) {
+    it(`records remote SSE ${scenario} without confusing a call deadline with a broken transport`, async () => {
+      const company = await createCompany(db);
+      const agent = await createAgent(db, company.id);
+      const { run } = await createIssueAndRun(db, company.id, agent.id);
+      const { connection } = await createRemoteMcpTool(db, company.id);
+      await allowAllToolsForAgent(db, company.id, agent.id);
+      const reading = Promise.withResolvers<void>();
+      let bodyController!: ReadableStreamDefaultController<Uint8Array>;
+      let requestId!: string;
+      let signal!: AbortSignal;
+      const progress = 'data: {"jsonrpc":"2.0","method":"notifications/progress","params":{"progress":1}}\n\n';
+      const cause = Object.assign(new Error("peer failure Authorization: Bearer fixture-private-value"), { code: "ECONNRESET" });
+      const failure = Object.assign(new Error("aborted", { cause }), { code: "ECONNRESET" });
+      if (scenario === "cyclic peer reset") Object.assign(cause, { cause: failure });
+      const gateway = createTestToolGatewayService(db, {
+        remoteHttpRequest: async (_url, init) => {
+          requestId = JSON.parse(String(init.body)).id;
+          signal = init.signal!;
+          const body = new ReadableStream<Uint8Array>({
+            start(controller) {
+              bodyController = controller;
+              controller.enqueue(new TextEncoder().encode(progress));
+              init.signal!.addEventListener("abort", () => controller.error(failure), { once: true });
+            },
+            pull() { reading.resolve(); },
+          }, { highWaterMark: 0 });
+          return new Response(body, { headers: { "content-type": "text/event-stream", "x-request-id": "sse-request" } });
+        },
+      });
+      const session = await gateway.createSession({ companyId: company.id, agentId: agent.id, runId: run.id });
+      const tool = (await gateway.listToolsForSession(session.token)).find((entry) => entry.providerType === "mcp_remote_http")!;
+      vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] });
+      const call = gateway.executeTool({
+        sessionToken: session.token, tool: tool.name, parameters: { key: "a", value: "b" },
+        ...(scenario === "explicit deadline" ? { timeoutMs: 1500 } : {}),
+      }).then((value) => ({ value, error: null }), (error: unknown) => ({ value: null, error }));
+      try {
+        await reading.promise;
+        if (scenario.includes("peer reset")) bodyController.error(scenario === "primitive peer reset" ? "peer reset" : failure);
+        else await vi.advanceTimersByTimeAsync(scenario === "slow success" ? 12_000 : scenario === "explicit deadline" ? 1500 : 30_000);
+        if (scenario === "slow success") {
+          expect(signal.aborted).toBe(false);
+          bodyController.enqueue(new TextEncoder().encode(`data: ${JSON.stringify({ jsonrpc: "2.0", id: requestId, result: { content: [{ type: "text", text: "completed after twelve seconds" }] } })}\n\n`));
+        }
+        const result = await call;
+        const [event] = await db.select().from(toolCallEvents).where(eq(toolCallEvents.eventType, scenario === "slow success" ? "call_completed" : "call_failed"));
+        const [stored] = await db.select().from(toolConnections).where(eq(toolConnections.id, connection.id));
+        if (scenario === "slow success") {
+          expect(result.error).toBeNull();
+          expect(JSON.stringify(result.value)).toContain("completed after twelve seconds");
+          expect(stored.healthStatus).toBe("ok");
+          expect(event.metadata).toMatchObject({ execution: { stream: { eventsReceived: 2, matchedResponse: true, ended: false } } });
+        } else {
+          const timeout = scenario === "default deadline" || scenario === "explicit deadline";
+          expectGatewayError(result.error, timeout ? 504 : 502, timeout ? "tool_timeout" : "mcp_remote_fetch_failed");
+          expect(stored.healthStatus).toBe("ok");
+          expect(event.metadata).toMatchObject({ execution: {
+            failureKind: timeout ? "invocation_timeout" : "transport_failure",
+            deadline: { timeoutMs: scenario === "explicit deadline" ? 1500 : 30_000, aborted: timeout },
+            error: scenario === "primitive peer reset"
+              ? { name: null, message: "peer reset", code: null, cause: null }
+              : { name: "Error", message: "aborted", code: "ECONNRESET", cause: { name: "Error", code: "ECONNRESET" } },
+            response: { httpStatus: 200, contentType: "text/event-stream", upstreamRequestId: "sse-request" },
+            stream: { bytesReceived: Buffer.byteLength(progress), chunksReceived: 1, eventsReceived: 1, matchedResponse: false, ended: false },
+          } });
+          expect(JSON.stringify(event.metadata)).not.toContain("fixture-private-value");
+          if (scenario !== "primitive peer reset") expect(JSON.stringify(event.metadata)).toContain("peer failure");
+          if (scenario === "cyclic peer reset") expect(JSON.stringify(event.metadata)).toContain("[Circular cause]");
+        }
+      } finally {
+        vi.useRealTimers();
+        bodyController.error(new Error("fixture cleanup"));
+        await call;
+      }
+    });
+  }
+
+  async function startRemoteCallWithControlledBody(respond: (body: ReadableStream<Uint8Array>) => Response, waitForRead = true) {
+    const company = await createCompany(db);
+    const agent = await createAgent(db, company.id);
+    const { run } = await createIssueAndRun(db, company.id, agent.id);
+    await createRemoteMcpTool(db, company.id);
+    await allowAllToolsForAgent(db, company.id, agent.id);
+    const reading = Promise.withResolvers<void>();
+    let bodyController!: ReadableStreamDefaultController<Uint8Array>;
+    let requestId!: string;
+    const gateway = createTestToolGatewayService(db, {
+      remoteHttpRequest: async (_url, init) => {
+        requestId = JSON.parse(String(init.body)).id;
+        return respond(new ReadableStream<Uint8Array>({
+          start(controller) { bodyController = controller; },
+          pull() { reading.resolve(); },
+        }, { highWaterMark: 0 }));
+      },
+    });
+    const session = await gateway.createSession({ companyId: company.id, agentId: agent.id, runId: run.id });
+    const tool = (await gateway.listToolsForSession(session.token)).find((entry) => entry.providerType === "mcp_remote_http")!;
+    vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] });
+    const call = gateway.executeTool({ sessionToken: session.token, tool: tool.name, parameters: { key: "a", value: "b" }, timeoutMs: 1500 })
+      .then((value) => ({ value, error: null }), (error: unknown) => ({ value: null, error }));
+    if (waitForRead) await reading.promise;
+    const failedEvent = async () => (await db.select().from(toolCallEvents).where(eq(toolCallEvents.eventType, "call_failed")))[0];
+    return { call, failedEvent, body: () => bodyController, requestId: () => requestId };
+  }
+
+  it("keeps a tool question when the call deadline passes while the question is recorded", async () => {
+    const remote = await startRemoteCallWithControlledBody((body) => new Response(body, { headers: { "content-type": "text/event-stream" } }));
+    try {
+      await vi.advanceTimersByTimeAsync(1500);
+      const question = { jsonrpc: "2.0", id: "question-1", method: "elicitation/create", params: { message: "Which folder?", requestedSchema: { type: "object" } } };
+      remote.body().enqueue(new TextEncoder().encode(`data: ${JSON.stringify(question)}\n\n`));
+      expectGatewayError((await remote.call).error, 409, "elicitation_required");
+    } finally {
+      vi.useRealTimers();
+      remote.body().error(new Error("fixture cleanup"));
+      await remote.call;
+    }
+  });
+
+  it("records a transport failure kind when the remote hostname does not resolve", async () => {
+    const remote = await startRemoteCallWithControlledBody(() => {
+      throw new ToolGatewayHttpError(502, "Remote MCP connection hostname did not resolve", "remote_http_dns_failed");
+    }, false);
+    try {
+      expectGatewayError((await remote.call).error, 502, "remote_http_dns_failed");
+      expect((await remote.failedEvent()).metadata).toMatchObject({ execution: { failureKind: "transport_failure" } });
+    } finally {
+      vi.useRealTimers();
+    }
+  });
+
+  it("records the bytes of an error response body that fails partway", async () => {
+    const remote = await startRemoteCallWithControlledBody((body) => new Response(body, { status: 503, headers: { "content-type": "text/plain" } }));
+    try {
+      const partial = "upstream overloaded, partial";
+      remote.body().enqueue(new TextEncoder().encode(partial));
+      remote.body().error(Object.assign(new Error("aborted"), { code: "ECONNRESET" }));
+      expectGatewayError((await remote.call).error, 502, "mcp_remote_fetch_failed");
+      expect((await remote.failedEvent()).metadata).toMatchObject({ execution: {
+        failureKind: "transport_failure",
+        response: { httpStatus: 503, bodySizeBytes: Buffer.byteLength(partial) },
+      } });
+    } finally {
+      vi.useRealTimers();
+    }
+  });
+
   const remoteFailureCases = [
     {
       name: "HTTP status",

server/src/services/mcp-http.ts

@@ -175,13 +175,27 @@ export function parseMcpHttpResponseBody(bodyText: string, contentType: string |
   throw new SyntaxError("MCP SSE response contained no data events");
 }
 
+export type McpHttpResponseProgress = {
+  bytesReceived: number;
+  chunksReceived: number;
+  eventsReceived: number;
+  matchedResponse: boolean;
+  ended: boolean;
+};
+
 /** Read until the response for this request arrives, without waiting for an SSE
  * connection to close. Notifications and responses for other IDs are ignored. */
 export async function readMcpHttpResponse(
   response: Response,
   requestId: string | number,
-  options: { maxBytes?: number; onRequest?: (message: Record<string, unknown>) => Promise<void> } = {},
+  options: {
+    maxBytes?: number;
+    onRequest?: (message: Record<string, unknown>) => Promise<void>;
+    onProgress?: (progress: McpHttpResponseProgress) => void;
+  } = {},
 ): Promise<unknown> {
+  const progress: McpHttpResponseProgress = { bytesReceived: 0, chunksReceived: 0, eventsReceived: 0, matchedResponse: false, ended: false };
+  options.onProgress?.(progress);
   const maxBytes = options.maxBytes ?? 8 * 1024 * 1024;
   const isStream = response.headers.get("content-type")?.toLowerCase().includes("text/event-stream");
   const reader = response.body?.getReader();
@@ -204,19 +218,26 @@ export async function readMcpHttpResponse(
   const inspect = async (message: unknown): Promise<unknown | undefined> => {
     if (!message || typeof message !== "object") return undefined;
     const record = message as Record<string, unknown>;
-    if (record.id === requestId && ("result" in record || "error" in record)) return record;
+    if (record.id === requestId && ("result" in record || "error" in record)) {
+      progress.matchedResponse = true;
+      return record;
+    }
     if ("method" in record && "id" in record) await options.onRequest?.(record);
     return undefined;
   };
   const event = async (value: string) => {
     const data = value.split("\n").filter((line) => line.startsWith("data:")).map((line) => line.slice(5).replace(/^ /, "")).join("\n");
     if (!data) return undefined;
+    progress.eventsReceived += 1;
     return inspect(parse(data));
   };
   try {
     while (true) {
       const { value, done } = await reader.read();
       bytes += value?.byteLength ?? 0;
+      progress.bytesReceived = bytes;
+      if (value) progress.chunksReceived += 1;
+      progress.ended = done;
       if (bytes > maxBytes) throw new McpHttpResponseError("too_large", "MCP response exceeded the size limit");
       buffer += decoder.decode(value, { stream: !done });
       if (isStream) {

server/src/services/remote-http-fetch.ts

@@ -33,7 +33,7 @@ const DEFAULT_CONNECT_TIMEOUT_MS = 10_000;
  * that own a longer budget — `tools/call`, which an operator can raise to 60 s —
  * pass `responseTimeoutMs` so this default never truncates it.
  */
-const DEFAULT_RESPONSE_TIMEOUT_MS = 30_000;
+export const DEFAULT_REMOTE_HTTP_RESPONSE_TIMEOUT_MS = 30_000;
 const DNS_RESOLUTION_ERROR_CODES = new Set(["ENODATA", "ENOTFOUND", "EAI_AGAIN"]);
 
 /** How a verified socket is opened. Overridable so tests can simulate a rebind. */
@@ -156,7 +156,7 @@ async function pinnedRequest(
       socket,
       init,
       signal,
-      responseTimeoutMs: options.responseTimeoutMs ?? DEFAULT_RESPONSE_TIMEOUT_MS,
+      responseTimeoutMs: options.responseTimeoutMs ?? DEFAULT_REMOTE_HTTP_RESPONSE_TIMEOUT_MS,
       error: options.error,
     });
   } catch (error) {

server/src/services/tool-gateway.ts

@@ -47,6 +47,7 @@ import { extractRemoteMcpPending } from "./remote-mcp-pending.js";
 import { logger } from "../middleware/logger.js";
 import { getAppDefinitionForApplicationKey, isConnectorFeatureEnabled } from "@paperclipai/shared";
 import { instanceSettingsService } from "./instance-settings.js";
+import { redactSensitiveText } from "../redaction.js";
 import { spawn } from "node:child_process";
 import { createHash, randomBytes, randomUUID } from "node:crypto";
 import { isDeepStrictEqual } from "node:util";
@@ -143,6 +144,7 @@ import {
   readMcpHttpResponse,
   McpHttpResponseError,
   McpHttpInitializationError,
+  type McpHttpResponseProgress,
   mcpHttpRequestHeaders,
   parseMcpHttpResponseBody,
 } from "./mcp-http.js";
@@ -156,6 +158,7 @@ import { assertGmailArchiveToolArgumentsSupported, gmailArchiveToolDescription }
 import { parseRemoteHttpEndpoint } from "./remote-http-endpoint-guard.js";
 import {
   guardedRemoteHttpFetch,
+  DEFAULT_REMOTE_HTTP_RESPONSE_TIMEOUT_MS,
   type GuardedRemoteHttpFetchOptions,
 } from "./remote-http-fetch.js";
 import {
@@ -491,8 +494,55 @@ type RemoteHttpExecutionAudit = {
     bodySizeBytes: number;
     upstreamRequestId: string | null;
   };
+  failureKind?: "invocation_timeout" | "transport_failure" | "protocol_failure";
+  error?: RemoteHttpErrorEvidence;
+  deadline?: { timeoutMs: number; aborted: boolean };
+  stream?: McpHttpResponseProgress;
 };
 
+type RemoteHttpErrorEvidence = {
+  name: string | null;
+  message: string;
+  code: string | null;
+  stack: string | null;
+  cause: RemoteHttpErrorEvidence | null;
+};
+
+// Deliberate stops of a remote tool call. They are not failures, and a deadline
+// that passes while they are recorded must not turn them into a timeout.
+const REMOTE_TOOL_CALL_WORKFLOW_REASON_CODES = new Set([
+  "elicitation_required",
+  "provider_interaction_required",
+  "oauth_insufficient_scope",
+  "oauth_refreshed_retry_required",
+]);
+
+const REMOTE_TOOL_CALL_PROTOCOL_REASON_CODES = new Set([
+  "mcp_remote_invalid_json",
+  "remote_mcp_malformed_response",
+  "mcp_remote_response_too_large",
+]);
+
+function remoteToolCallFailureKind(reasonCode: string): RemoteHttpExecutionAudit["failureKind"] {
+  if (reasonCode === "tool_timeout") return "invocation_timeout";
+  if (REMOTE_TOOL_CALL_PROTOCOL_REASON_CODES.has(reasonCode)) return "protocol_failure";
+  if (reasonCode === "mcp_remote_fetch_failed" || reasonCode.startsWith("remote_http_")) return "transport_failure";
+  return undefined;
+}
+
+function remoteHttpErrorEvidence(error: unknown, seen = new Set<unknown>()): RemoteHttpErrorEvidence {
+  if (seen.has(error)) return { name: null, message: "[Circular cause]", code: null, stack: null, cause: null };
+  const record = asRecord(error);
+  seen.add(error);
+  return {
+    name: typeof record?.name === "string" ? redactSensitiveText(record.name) : null,
+    message: redactSensitiveText(typeof record?.message === "string" ? record.message : String(error)),
+    code: typeof record?.code === "string" ? redactSensitiveText(record.code) : null,
+    stack: typeof record?.stack === "string" ? redactSensitiveText(record.stack) : null,
+    cause: record?.cause == null ? null : remoteHttpErrorEvidence(record.cause, seen),
+  };
+}
+
 type LocalStdioRuntimeTemplate = {
   templateId: string;
   command: string | null;
@@ -5920,6 +5970,7 @@ export function createToolGatewayService(
 
   async function readBoundedRemoteResponse(
     response: Response,
+    onBytes?: (bytes: number) => void,
   ): Promise<string> {
     const contentLength = response.headers.get("content-length");
     if (
@@ -5928,11 +5979,36 @@ export function createToolGatewayService(
     ) {
       throw responseTooLargeError();
     }
-    const body = await response.text();
-    if (Buffer.byteLength(body, "utf8") > MAX_REMOTE_MCP_RESPONSE_BYTES) {
-      throw responseTooLargeError();
+    const reader = response.body?.getReader();
+    // Injected HTTP transports can expose a buffered text response rather than a
+    // Web ReadableStream. Keep the same size check for that form.
+    if (!reader) {
+      const body = await response.text();
+      onBytes?.(Buffer.byteLength(body, "utf8"));
+      if (Buffer.byteLength(body, "utf8") > MAX_REMOTE_MCP_RESPONSE_BYTES) {
+        throw responseTooLargeError();
+      }
+      return body;
+    }
+    // Count each chunk as it arrives, so a body that fails partway still
+    // reports the bytes that were read.
+    const decoder = new TextDecoder();
+    let body = "";
+    let bytes = 0;
+    try {
+      while (true) {
+        const { value, done } = await reader.read();
+        if (done) break;
+        bytes += value.byteLength;
+        onBytes?.(bytes);
+        if (bytes > MAX_REMOTE_MCP_RESPONSE_BYTES) throw responseTooLargeError();
+        body += decoder.decode(value, { stream: true });
+      }
+      return body + decoder.decode();
+    } finally {
+      await reader.cancel().catch(() => undefined);
+      reader.releaseLock();
     }
-    return body;
   }
 
   function malformedRemoteMcpResponse(): ToolGatewayHttpError {
@@ -6214,6 +6290,7 @@ export function createToolGatewayService(
     // and connections whose stored grant/catalog predates scope reduction.
     assertGoogleChatToolArgumentsSupported(connection, entry.toolName, parameters);
     assertGmailArchiveToolArgumentsSupported(connection, entry.toolName, parameters);
+    if (useDefaultTimeout) ms = DEFAULT_REMOTE_HTTP_RESPONSE_TIMEOUT_MS;
     if (useDefaultTimeout && isRailwayConnection(connection) && entry.toolName === `${RAILWAY_TOOL_PREFIX}run-command`) {
       ms = railwayCommandBudgetMs(parameters);
     }
@@ -6498,9 +6575,21 @@ export function createToolGatewayService(
         // automatically: the failed call may have changed app data.
         if (mcpSession) forgetMcpHttpSession(mcpSession);
       }
+      // Record the response headers before the body is read, so a body that
+      // fails partway still has its status and content type in the evidence.
+      execution.response = {
+        httpStatus: response.status,
+        contentType: response.headers.get("content-type"),
+        bodySizeBytes: 0,
+        upstreamRequestId:
+          response.headers.get("x-request-id") ??
+          response.headers.get("x-zapier-request-id") ??
+          response.headers.get("traceparent"),
+      };
       const body = response.ok
         ? JSON.stringify(await readMcpHttpResponse(response, requestId, {
             maxBytes: MAX_REMOTE_MCP_RESPONSE_BYTES,
+            onProgress: (progress) => { execution.stream = progress; },
             onRequest: async (message) => {
               const pending = extractRemoteMcpPending(message);
               if (pending) {
@@ -6517,16 +6606,10 @@ export function createToolGatewayService(
               if (request) await requestElicitationForRecordedToolCall({ session, tool, invocationId, request });
             },
           }))
-        : await readBoundedRemoteResponse(response);
-      execution.response = {
-        httpStatus: response.status,
-        contentType: response.headers.get("content-type"),
-        bodySizeBytes: Buffer.byteLength(body, "utf8"),
-        upstreamRequestId:
-          response.headers.get("x-request-id") ??
-          response.headers.get("x-zapier-request-id") ??
-          response.headers.get("traceparent"),
-      };
+        : await readBoundedRemoteResponse(response, (bytes) => {
+            execution.response!.bodySizeBytes = bytes;
+          });
+      execution.response.bodySizeBytes = Buffer.byteLength(body, "utf8");
       if (isInsufficientConnectionScope(response, body)) {
         await markCallHealth("degraded", INSUFFICIENT_CONNECTION_SCOPE_MESSAGE);
         throw new ToolGatewayHttpError(403, INSUFFICIENT_CONNECTION_SCOPE_MESSAGE, "oauth_insufficient_scope", {
@@ -6627,7 +6710,26 @@ export function createToolGatewayService(
       );
       return { result, failureInstructions, headerSummary, execution };
     } catch (error) {
+      execution.error = remoteHttpErrorEvidence(error);
+      execution.deadline = { timeoutMs: ms, aborted: controller.signal.aborted };
+      if (execution.response && execution.stream) execution.response.bodySizeBytes = execution.stream.bytesReceived;
+      // A deliberate workflow stop, such as a tool question for the user, keeps
+      // its own reason even when the deadline passes while it is recorded.
+      if (error instanceof ToolGatewayHttpError && REMOTE_TOOL_CALL_WORKFLOW_REASON_CODES.has(error.reasonCode)) {
+        throw new ToolGatewayHttpError(error.status, error.message, error.reasonCode, {
+          ...error.details,
+          execution: error.details.execution ?? execution,
+        });
+      }
+      if (controller.signal.aborted || (error instanceof Error && error.name === "AbortError")) {
+        if (mcpSession) forgetMcpHttpSession(mcpSession);
+        execution.failureKind = "invocation_timeout";
+        throw new ToolGatewayHttpError(504, "Remote MCP tool call timed out", "tool_timeout", {
+          connectionId: connection.id, catalogEntryId: entry.id, execution, failureKind: "invocation_timeout",
+        });
+      }
       if (error instanceof McpHttpResponseError) {
+        execution.failureKind = "protocol_failure";
         const failure = error.reason === "too_large" ? responseTooLargeError()
           : error.reason === "malformed_response" ? malformedRemoteMcpResponse()
           : new ToolGatewayHttpError(502, "Remote MCP server returned invalid JSON", "mcp_remote_invalid_json");
@@ -6645,6 +6747,8 @@ export function createToolGatewayService(
         throw new ToolGatewayHttpError(error.status, error.message, error.code, { connectionId: connection.id, catalogEntryId: entry.id, execution });
       }
       if (error instanceof ToolGatewayHttpError) {
+        const failureKind = remoteToolCallFailureKind(error.reasonCode);
+        if (failureKind) execution.failureKind = failureKind;
         throw new ToolGatewayHttpError(
           error.status,
           error.message,
@@ -6655,19 +6759,7 @@ export function createToolGatewayService(
           },
         );
       }
-      if (error instanceof Error && error.name === "AbortError") {
-        if (mcpSession) forgetMcpHttpSession(mcpSession);
-        throw new ToolGatewayHttpError(
-          504,
-          "Remote MCP tool call timed out",
-          "tool_timeout",
-          {
-            connectionId: connection.id,
-            catalogEntryId: entry.id,
-            execution,
-          },
-        );
-      }
+      execution.failureKind = "transport_failure";
       if (error instanceof McpHttpInitializationError && (error.status === 401 || error.status === 403)) {
         await markCallHealth("error", "Remote MCP session authorization failed.");
       }