feat(google-workspace): gateway sets the REST tool result size limit for all six services; update connector docs (#22050)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents read Gmail, Calendar, Drive, Docs, Sheets and Slides through
the built-in Google Workspace connection
> - Paperclip now has its own REST-backed Google tools in
`server/src/google-workspace-rest-tools/`. No service uses them yet
> - That folder guessed a result size limit of 250,000 bytes. It applied
the limit only to Drive, Docs, Sheets and Slides. Gmail and Calendar had
no limit
> - The guess was wrong in both directions. It refused documents that
Paperclip can deliver, and the folder should not know Paperclip's limits
> - This pull request makes the limit a number that the gateway gives.
The folder applies it to every tool in all six services through one
check. It also brings three Google planning docs up to date
> - The benefit is that a REST result is refused only when the gateway
would also refuse the same result from Google's own MCP server

## Linked Issues or Issue Description

Refs #22027, #22028, #22029, #22026

**Is your feature request related to a problem?**
The REST tools refused results over a size the folder chose itself
(250,000 bytes). Paperclip can deliver larger results: the runner saves
a large result as an artifact and gives the agent a pointer. Gmail and
Calendar results had no limit at all.

**Describe the solution you'd like**
The gateway gives the limit. The folder applies the given number to the
final serialized result of every tool, with one helper. A result over
the limit is an error that says how to ask for less. It is never cut
short.

**Describe alternatives you've considered**
A larger number chosen from the runner's 1 GiB capture limit. Google's
own transfer limit (20 MB per response) and memory use make that a poor
choice. Also, results would then differ between the REST path and
Google's MCP path.

### Limits a tool result passes on the way to an agent

| Point | Limit | What happens over it |
| --- | --- | --- |
| Google response read by `google-http.ts` | 20 MB
(`MAX_RESPONSE_BYTES`) | Tool error. Nothing returned |
| REST folder result (new) | the number the gateway passes: 999,885
bytes | Tool error with "read less" advice. The gateway returns
`tool_error` (502) |
| Gateway read of a remote MCP response (`MAX_REMOTE_MCP_RESPONSE_BYTES`
in `tool-gateway.ts`) | 1,000,000 bytes per JSON body or event-stream
message | `mcp_remote_response_too_large` (502). This is the "1,000,000
bytes are rejected" statement in `doc/prompt-injection-prevention.md`.
It applies to the Google MCP path only; the REST path has no HTTP read |
| Injection screening model check | 256 KiB model request | Not a
delivery limit. A larger result is checked by the built-in patterns only
|
| Gateway HTTP route (`POST /tool-gateway/.../tools/call`) | none in
Paperclip | `res.json(result)` |
| Runner (`captureRunnerToolResult`) | 500,000 bytes
(`RUNNER_TOOL_RESULT_MAX_BYTES`) | Saved as an artifact. The agent gets
a pointer and reads it with `read_tool_result` |
| Runner capture | 1 GiB (`RUNNER_API_RESPONSE_MAX_BYTES`) |
`SemanticToolOutcomeUnknownError` |

The gateway wraps the result as `{ content, data: { content,
structuredContent, ... } }`, so the payload occurs again. Measured: a
Gmail result of 676,459 bytes from the folder became 1,017,386 bytes
delivered (ratio 1.50). The ratio cannot be more than 2.

### The budget

`GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES` in `tool-gateway.ts` =
`MAX_REMOTE_MCP_RESPONSE_BYTES` less the JSON-RPC envelope and
event-stream framing (`event: message\ndata:
{"jsonrpc":"2.0","id":"paperclip-tool-<uuid>-version","result":...}`) =
**999,885 bytes**. A result that Google's MCP server could deliver
through the gateway is also delivered over REST, and the reverse. With
the wrapper, the largest delivered result is less than 2 MB. The runner
saves it as an artifact (far below 1 GiB). So the folder never refuses a
result that the runner can give as a pointer.

Practical limits at that budget (plain ASCII; escaped and non-ASCII
characters count more):

| Tool | Largest content |
| --- | --- |
| Gmail `get_message` | about 500 KB plain body, or about 250 KB each
for plain plus HTML |
| Calendar `get_event` | about 500 KB description |
| Drive `read_file_content` | about 500 KB text |
| Drive `download_file_content` | about 375 KB file bytes |
| Docs `read_doc` | about 240 KB document text |
| Slides `read_presentation` | about 250 KB text |
| Sheets `get_values` | about 500 KB cell text, at most 50,000 cells |

## What Changed

- `tool.ts`: `defineTool` takes `whenTooLarge` advice per read tool
(required for reads, refused for writes) and a `ToolLimits` argument. It
measures the final `CallToolResult` once (`withinResultBudget`) and
replaces an oversized result with a `too_large` tool error.
`FALLBACK_MAX_RESULT_BYTES` (250,000) applies only when a direct caller
gives no limit.
- `callGoogleWorkspaceRestTool` and `createGoogleWorkspaceRestServer`
take `maxResultBytes`. A value that is not a positive integer is refused
before any request.
- All 23 read tools state their advice: a smaller `pageSize` for list
tools, `messageFormat` or `view` for Gmail, a narrower range for Sheets,
`fields` for Slides. Write tools are not checked: a completed write is
never reported as an error, so an agent is not led to repeat it.
- Drive, Docs, Slides and Sheets no longer have their own byte checks
(`boundedStructuredResult`, `boundedResult`, `sizedResult`,
`MAX_CONTENT_BYTES` removed). Drive downloads stop reading at the given
limit. The Sheets 50,000-cell limit stays.
- Two Drive tool descriptions no longer state a fixed number.
- Adapter `google-workspace-rest-dispatch.ts`: `maxResultBytes` is a
required input and is passed through.
- `tool-gateway.ts`: new `GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES`,
passed on the REST branch.
- Folder README: new "Result size" section with the limits and costs;
old 250,000-byte text replaced.
- Docs: `doc/memory/google-connector-memory.md`, the production
checklist and the code-fix plan now list #22026 to #22029, the REST
status (36 of 36, none switched on), the November 17 decisions, the
known differences from Google's tools, open risks and next steps.

## Verification

- `cd server && npx vitest run src/google-workspace-rest-tools` passes
(53 files, 1,361 tests with the adapter test).
- New `result-budget.test.ts`: for Gmail (read, list), Calendar (list,
read), Drive, Docs, Slides and Sheets, a result just under a given
budget is returned and one just over is refused with the tool's advice.
Without a budget, the fallback applies; with a larger budget, the same
result is returned. Drive downloads use the given cap. A Gmail list
tells the caller to lower `pageSize`. A Gmail draft result over the
budget is still returned, not an error. `defineTool` refuses a read tool
without advice and a write tool with it.
- `npx vitest run src/services/google-workspace-rest-dispatch.test.ts`:
the adapter passes the gateway's number, and the folder applies exactly
that number.
- `npx vitest run src/__tests__/google-workspace-rest-transport.test.ts`
(24 tests): with Gmail on REST, a 330 KB email passes the gateway with
screening, is more than 500,000 bytes delivered, and
`captureRunnerToolResult` turns it into a `read_tool_result` pointer. A
520 KB email is refused with `tool_error` and the advice, after one
Google request.
- `npx tsc --noEmit -p server/tsconfig.json`: no errors.
- `pnpm verify --explain`: broader verification required (a doc under
`doc/` is a global CI input). CI supplies that coverage.
- Time per call: the check is one `JSON.stringify` of the result. About
5 µs for a 4 KB result, 18 µs for 42 KB, 0.24 ms for 700 KB. Drive,
Docs, Sheets and Slides had the same pass before, so they add nothing.
Gmail and Calendar add this one pass.

## Risks

- Gmail and Calendar results now have a limit. Direct callers that pass
no limit get the 250,000-byte fallback. The gateway always passes its
number.
- Drive, Docs, Slides and Sheets accept results up to about 4 times
larger than before through the gateway. Large results reach agents as
artifact pointers.
- Error text for oversized results changed to one shared form.
- No service is switched to REST, so users see no change today.

## Model Used

Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code with
tool use (shell, file edits, GitHub CLI) and extended reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed 2026-11-17 13:05:13 -06:00
1 parent 98ff5e002a
commit 56023fe80f
47 files changed
+598 -216

No files matched your search

@@ -10,8 +10,9 @@ import {
type GoogleWorkspaceService, type GoogleWorkspaceToolTransport,
} from "@paperclipai/shared";
import * as restDispatch from "../services/google-workspace-rest-dispatch.js";
import { captureRunnerToolResult, RUNNER_TOOL_RESULT_MAX_BYTES } from "../services/native-runtime/runner-tool-results.js";
import { toolAccessService } from "../services/tool-access.js";
import { createToolGatewayService } from "../services/tool-gateway.js";
import { createToolGatewayService, GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES } from "../services/tool-gateway.js";
import { instanceSettingsService } from "../services/instance-settings.js";
import { initializeRunIdentity } from "../services/run-identity.js";
import type { PaperclipCloudConnector } from "../services/paperclip-cloud-connector.js";
@@ -302,6 +303,42 @@ const support = await getEmbeddedPostgresTestSupport();
expect(JSON.stringify(invocation)).not.toContain("ignore all previous instructions");
});
it("delivers a large email that Google's MCP path could also deliver, as a saved result the runner points to", async () => {
useTransport("gmail", "paperclip_rest");
const f = await connectWorkspace();
const body = `${BENIGN_BODY}\n`.repeat(Math.ceil(330_000 / (BENIGN_BODY.length + 1)));
f.setMessageBody(body);
const result = await f.call("gmail__get_message", { messageId: "msg1" });
expect(result.status).toBe("completed");
const outcome = await vi.mocked(restDispatch.callGoogleWorkspaceRest).mock.results[0]!.value;
const folderBytes = Buffer.byteLength(JSON.stringify(outcome.result), "utf8");
const deliveredBytes = Buffer.byteLength(JSON.stringify(result.result), "utf8");
expect(folderBytes).toBeLessThanOrEqual(GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES);
expect(deliveredBytes).toBeGreaterThan(RUNNER_TOOL_RESULT_MAX_BYTES);
expect(deliveredBytes).toBeLessThan(2 * folderBytes);
const saved: Buffer[] = [];
const pointer = await captureRunnerToolResult(result.result, {
saveResponse: vi.fn(async (bytes: Buffer) => { saved.push(bytes); return { artifactId: randomUUID() }; }),
} as unknown as Parameters<typeof captureRunnerToolResult>[1]);
expect(pointer).toMatchObject({ readWith: { tool: "read_tool_result" } });
expect(saved).toHaveLength(1);
expect(JSON.parse(saved[0]!.toString("utf8"))).toEqual(result.result);
expect((result.result as { data: { structuredContent: { plaintextBody: string } } }).data.structuredContent.plaintextBody).toBe(body);
});
it("refuses an email larger than the gateway's budget with advice to read less, without contacting Google again", async () => {
useTransport("gmail", "paperclip_rest");
const f = await connectWorkspace();
f.setMessageBody(`${BENIGN_BODY}\n`.repeat(Math.ceil(520_000 / (BENIGN_BODY.length + 1))));
f.gmailRequests.length = 0;
await expect(f.call("gmail__get_message", { messageId: "msg1" })).rejects.toMatchObject({
status: 502, reasonCode: "tool_error",
message: expect.stringMatching(new RegExp(`more than the ${GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES.toLocaleString("en-US")} bytes this tool can return, so it was not returned\\. Read it with messageFormat METADATA_ONLY or MINIMAL`)),
});
expect(f.gmailRequests).toHaveLength(1);
expect(vi.mocked(restDispatch.callGoogleWorkspaceRest).mock.calls[0]![0].maxResultBytes).toBe(GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES);
});
it("cancels the Google request when the call's deadline passes", async () => {
useTransport("gmail", "paperclip_rest");
const f = await connectWorkspace();
@@ -121,7 +121,8 @@ plain constant: no environment variable, instance setting or database row.
- **Adapter.** `server/src/services/google-workspace-rest-dispatch.ts` is the
only server file that imports this folder. It takes the gateway's resolved
credential headers (it keeps only the bearer token), the call's cancellation
signal and the gateway's egress-guarded fetch, and returns either the MCP
signal, the gateway's egress-guarded fetch and the gateway's result limit
(see "Result size"), and returns either the MCP
result or a token problem (`unauthenticated` or `insufficient_scope`).
- **Calls.** `executeRemoteHttpTool()` in `server/src/services/tool-gateway.ts`.
The REST branch runs after everything the MCP path runs first: the
@@ -176,6 +177,56 @@ Measured with a stubbed fetch, the adapter adds about 15 µs per call. A full
`gmail__get_message` through `executeTool` (policy, records, screening) took a
median of 18.2 ms on either transport.
### Result size
The caller sets the size limit. `callGoogleWorkspaceRestTool` takes
`maxResultBytes`, and every tool in all six services checks it the same way:
`defineTool` in `tool.ts` measures the final serialized `CallToolResult` once
(`withinResultBudget`) after the tool has built it. A result over the limit is
replaced by a tool error that gives its size and the limit, says it was not
returned, and says how to ask for less. Each tool states that advice in
`whenTooLarge`: a smaller `pageSize` for list tools, a lighter
`messageFormat` or view for Gmail, a narrower range for Sheets, `fields` for
Slides. Nothing is ever cut short. Write tools are not checked: their result
comes after the change was made, and an error then could lead the agent to
make the change twice. A write result echoes its request or Google's reply
to it, so its size follows the request. `defineTool` refuses a read tool without advice
and a write tool with it. Drive downloads also stop reading at the
limit, since larger content could never fit.
`FALLBACK_MAX_RESULT_BYTES` (250,000) in `tool.ts` applies only when a direct
caller passes no limit (tests, or `createGoogleWorkspaceRestServer` without the
option). This folder does not know Paperclip's limits.
The gateway passes `GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES` from
`server/src/services/tool-gateway.ts`: the gateway's limit on a response from a
remote MCP server (`MAX_REMOTE_MCP_RESPONSE_BYTES`, 1,000,000 bytes) less the
JSON-RPC and event-stream framing around the result, so 999,885 bytes. A
result Google's own MCP server could deliver through the gateway is also
delivered over REST, and the reverse. The gateway wraps the result for the
agent and repeats the payload, so the delivered result is up to twice as large
(measured 1.50 times for a plain-text email). The agent runner saves a
delivered result over 500,000 bytes as an artifact and gives the agent a
pointer it reads with `read_tool_result`, so a result near the limit is still
delivered in full.
Practical limits at that budget, measured with plain ASCII text (characters
that JSON escapes, and non-ASCII characters, count more):
| Service | Largest content one call returns |
| --- | --- |
| Gmail `get_message` | about 500 KB of plain-text body, or about 250 KB each when the message has both plain and HTML bodies |
| Gmail lists | the whole page must fit; lower `pageSize` |
| Calendar `get_event` | about 500 KB of description |
| Drive `read_file_content` | about 500 KB of text |
| Drive `download_file_content` | about 375 KB of file bytes (base64 adds a third) |
| Docs `read_doc` | about 240 KB of document text (the text and the edit structure both carry it) |
| Slides `read_presentation` | about 250 KB of slide text; use `fields` for more |
| Sheets `get_values` | about 500 KB of cell text, and at most 50,000 cells |
The check costs one `JSON.stringify` of the result: about 5 µs for a 4 KB
result and 0.24 ms for a 700 KB one.
### Flipping a service
1. Implement every tool in that service's allowlist (`toolScopes` in the
@@ -564,15 +615,12 @@ type of the returned bytes (the export format for Google files).
true`, the field Google's result has for this case. Text that is not valid
UTF-8 is also not supported.
**Size cap.** Every Drive read bounds its final serialized result (the whole
`CallToolResult`, which carries the payload twice: as `structuredContent` and
as its JSON text) to 250,000 bytes (the folder's shared `MAX_RESULT_BYTES`,
checked by `boundedStructuredResult`, both in `tool.ts`). That is about 90 KB of file
bytes for `download_file_content` (base64 adds a third) and about 120 KB of
plain text for `read_file_content`. Content is also never read past 250,000
bytes (`MAX_CONTENT_BYTES`), since more could never fit: a stored file whose
metadata size is larger is refused before any download, and exports and
revisions are read through `GoogleHttp.download`, which stops at the cap. Both
**Size cap.** Every Drive tool is bound by the caller's result limit (see
"Result size"). The result carries the payload twice, as `structuredContent`
and as its JSON text. Content is also never read past the limit, since more
could never fit: a stored file whose metadata size is larger is refused before
any download, and exports and revisions are read through
`GoogleHttp.download`, which stops at the limit. Both
limits are far under Google's 10 MB export limit
(<https://developers.google.com/workspace/drive/api/reference/rest/v3/files/export>)
and the 20 MB `MAX_RESPONSE_BYTES`. A refused result is an error that says
@@ -752,15 +800,11 @@ Formatting other than headings, lists and links (bold, colors, fonts, sizes)
is not in `text`. Checklist items render as plain list items: the Docs API
does not say whether a box is ticked.
A result over 250,000 bytes is refused with an error that gives the size and
returns nothing; it is never cut short. It is the folder's one limit,
`MAX_RESULT_BYTES` in `tool.ts`, checked by `boundedStructuredResult` (which
`sizedResult` in `documents-shared.ts` calls) on the final serialized MCP
result, which carries the content twice (as
JSON text and as structured content), so the content itself is limited to
about 125,000 bytes. Paperclip delivers tool results over 500,000 bytes as a
pointer instead of content; the lower limit keeps every read inline with
room to spare. To read a
A result over the caller's limit (see "Result size") is refused with an error
that gives the size and returns nothing; it is never cut short. The final
serialized MCP result carries the content twice (as JSON text and as
structured content), and `read_doc` also repeats each paragraph's text in its
edit structure. To read a
large presentation in parts, pass `fields`, for example
`["slides(objectId,pageElements(objectId,shape.text))"]`. `read_doc` has no
way to read part of a document, since Google's tool has no such input and
@@ -943,15 +987,9 @@ made" warning.
Reads count the cells Google returns (`values` rows, or `rowData` values in
grid data); writes count the cells sent (`values`/`formulas`, or the rows of
`updateCells` and `appendCells`).
- A read's whole tool result is at most **250,000 bytes** of UTF-8 JSON
(`MAX_RESULT_BYTES`, checked by `boundedStructuredResult` in `tool.ts`). The
measure is this folder's own result, which carries the value twice (as
`structuredContent` and as JSON text, since MCP clients read either). This
is a conservative default, not a measured fit: how a caller wraps and
delivers the result, and its byte budget, are the caller's business and are
not modeled here. The caller, which knows its own delivery budget, is
expected to supply the exact limit in a follow-up that applies one rule to
every service in this folder.
- A tool's whole result is bound by the caller's limit (see "Result size"),
measured on this folder's own result, which carries the value twice (as
`structuredContent` and as JSON text, since MCP clients read either).
A write request body is at most 1,000,000 characters.
- Over a limit, a read returns an error that starts with the size and asks for
a narrower range, fewer or narrower `ranges`, or fewer `fields`, to read in
@@ -1045,7 +1083,8 @@ says `false`.
2. Delete this folder, `server/src/google-workspace-rest-tools/`.
3. Delete `server/src/services/google-workspace-rest-dispatch.ts` and its
test, and the two call sites: the `paperclip_rest` branch in
`executeRemoteHttpTool` (`server/src/services/tool-gateway.ts`) and
`executeRemoteHttpTool` (`server/src/services/tool-gateway.ts`), with
`GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES` there, and
`googleWorkspaceRestTools` in
`server/src/services/tool-access/catalog/discovery.ts`. Delete
`server/src/__tests__/google-workspace-rest-transport.test.ts`.
@@ -27,6 +27,7 @@ export const getEvent = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "This tool cannot return part of an event.",
async run(input, http) {
return getEventResult(await http.send<RestEvent>(getEventRequest(input.calendarId, input.eventId)));
},
@@ -72,6 +72,7 @@ export const listCalendars = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Ask for fewer calendars with a smaller pageSize.",
async run(input, http) {
return listCalendarsResult(await http.send<RestCalendarList>(listCalendarsRequest(input)));
},
@@ -201,6 +201,7 @@ export const listEvents = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Ask for fewer events with a smaller pageSize or a narrower time range.",
async run(input, http) {
assertTimeZone(input.timeZone);
const page = decodePageToken(input.pageToken, input.orderBy);
@@ -58,6 +58,7 @@ export const searchEvents = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Ask for fewer events with a smaller pageSize.",
async run(input, http) {
return searchEventsResult(await http.send<RestEventList>(searchEventsRequest(input)));
},
@@ -301,6 +301,7 @@ export const suggestTime = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Ask for fewer suggestions with a smaller preferences.pageSize.",
async run(input, http) {
assertTimeZone(input.timeZone);
parseTimeInput(input.startTime, "startTime");
@@ -1,5 +1,5 @@
import { describe, expect, it } from "vitest";
import { MAX_RESULT_BYTES } from "../tool.js";
import { FALLBACK_MAX_RESULT_BYTES } from "../tool.js";
import { fakeGoogle, json } from "../fixtures/fake-google.js";
import { createGoogleHttp } from "../google-http.js";
import { MULTI_TAB_DOCUMENT, MULTI_TAB_TEXT, oneTab, para, run, text } from "./fixtures/docs-rest.js";
@@ -104,22 +104,22 @@ describe("read_doc tool", () => {
it("measures the limit on the whole result, which carries the content twice", async () => {
const under = (count: number) => oneTab(Array.from({ length: count }, () => text("z".repeat(1_000))));
const perParagraph = 2 * 2 * 1_000;
const fits = Math.floor(MAX_RESULT_BYTES / perParagraph) - 10;
const fits = Math.floor(FALLBACK_MAX_RESULT_BYTES / perParagraph) - 10;
const http = (doc: unknown) => createGoogleHttp({ accessToken: "t", fetch: fakeGoogle(() => json(doc)).fetch });
const ok = await readDoc.call({ documentId: "d" }, http(under(fits)));
expect(ok.isError).toBeUndefined();
expect(Buffer.byteLength(JSON.stringify(ok), "utf8")).toBeLessThanOrEqual(MAX_RESULT_BYTES);
expect(Buffer.byteLength(JSON.stringify(ok.structuredContent), "utf8")).toBeLessThan(MAX_RESULT_BYTES / 2 + 1);
expect(Buffer.byteLength(JSON.stringify(ok), "utf8")).toBeLessThanOrEqual(FALLBACK_MAX_RESULT_BYTES);
expect(Buffer.byteLength(JSON.stringify(ok.structuredContent), "utf8")).toBeLessThan(FALLBACK_MAX_RESULT_BYTES / 2 + 1);
expect((await readDoc.call({ documentId: "d" }, http(under(fits + 40)))).isError).toBe(true);
});
it("refuses a result over the size limit with a clear error instead of cutting it short", async () => {
const paragraph = "x".repeat(10_000);
const big = oneTab(Array.from({ length: Math.ceil(MAX_RESULT_BYTES / 20_000) + 5 }, () => text(paragraph)));
const big = oneTab(Array.from({ length: Math.ceil(FALLBACK_MAX_RESULT_BYTES / 20_000) + 5 }, () => text(paragraph)));
const { fetch } = fakeGoogle(() => json(big));
const result = await readDoc.call({ documentId: "big" }, createGoogleHttp({ accessToken: "t", fetch }));
expect(result.isError).toBe(true);
expect((result.content[0] as { text: string }).text).toMatch(/^This document is too large to return in one result \(\d+ KB; the limit is 250 KB\)\. Nothing was returned\./);
expect((result.content[0] as { text: string }).text).toMatch(/^The result is [\d,]+ bytes, more than the 250,000 bytes this tool can return, so it was not returned\. This tool cannot read part of a document\./);
expect(result.structuredContent).toBeUndefined();
});
});
@@ -1,9 +1,9 @@
import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { fileIdArg, sizedResult } from "../documents-shared.js";
import { fileIdArg } from "../documents-shared.js";
import { GoogleToolError } from "../errors.js";
import type { GoogleHttp, GoogleRequest } from "../google-http.js";
import { defineTool } from "../tool.js";
import { defineTool, structuredResult } from "../tool.js";
import { DOCS_HOST, documentPath, type RestDocument } from "./document.js";
import { renderDocument, type SuggestionsView } from "./render.js";
@@ -38,7 +38,7 @@ export async function readDocument(http: GoogleHttp, input: ReadDocArgs): Promis
export function readDocResult(document: RestDocument, input: ReadDocArgs, view: SuggestionsView = "SUGGESTIONS_INLINE"): CallToolResult {
const content = renderDocument(document, { commentsRequested: input.commentsIncluded === true, suggestionsView: view });
return sizedResult({ content }, "document", "Google Docs cannot return part of a document through this tool.");
return structuredResult({ content });
}
export const readDoc = defineTool({
@@ -68,6 +68,7 @@ export const readDoc = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: true },
},
args,
whenTooLarge: "This tool cannot read part of a document. Read it without commentsIncluded if that was set, or read only its text with read_file_content.",
async run(input, http) {
const { document, view } = await readDocument(http, input);
return readDocResult(document, input, view);
@@ -1,8 +1,6 @@
import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { GoogleToolError } from "./errors.js";
import type { GoogleHttp, GoogleRequest } from "./google-http.js";
import { boundedStructuredResult, MAX_RESULT_BYTES } from "./tool.js";
export const MAX_BATCH_REQUESTS = 500;
export const OBJECT_PLACEHOLDER = "";
@@ -15,11 +13,6 @@ export const batchRequestArg = z.record(z.string(), z.unknown()).refine(
"Each request must name exactly one kind of update, such as insertText",
);
export function sizedResult(value: Record<string, unknown>, what: string, hint: string): CallToolResult {
return boundedStructuredResult(value, MAX_RESULT_BYTES, (bytes) =>
`This ${what} is too large to return in one result (${Math.ceil(bytes / 1000)} KB; the limit is ${MAX_RESULT_BYTES / 1000} KB). Nothing was returned. ${hint}`);
}
const CONFLICT_STATUSES = new Set(["FAILED_PRECONDITION", "ABORTED"]);
const CONFLICT_REASON = /REVISION|CONDITION_NOT_MET|conditionNotMet/i;
const CONFLICT_MESSAGE = /\brequired revision\b|\brevision id\b|\blatest revision\b/i;
@@ -1,4 +1,5 @@
import { GoogleToolError } from "../errors.js";
import { byteCount } from "../tool.js";
import type { GoogleHttp, GoogleRequest } from "../google-http.js";
import {
DRIVE_HOST,
@@ -10,7 +11,6 @@ import {
GOOGLE_SHEET,
GOOGLE_SLIDES,
isGoogleType,
MAX_CONTENT_BYTES,
type RestFile,
} from "./file.js";
@@ -52,14 +52,14 @@ export function mediaRequest(id: string, revision?: string): GoogleRequest {
return { method: "GET", host: DRIVE_HOST, path, query: { alt: "media", supportsAllDrives: revision ? undefined : true } };
}
export function tooLargeContent(advice: string, size?: number): GoogleToolError {
const what = size === undefined ? "This file's content is" : `This file is ${size} bytes,`;
return new GoogleToolError("too_large", `${what} more than the ${MAX_CONTENT_BYTES} bytes this tool returns, so nothing was returned. ${advice}`);
export function tooLargeContent(advice: string, maxBytes: number, size?: number): GoogleToolError {
const what = size === undefined ? "This file's content is" : `This file is ${byteCount(size)} bytes,`;
return new GoogleToolError("too_large", `${what} more than the ${byteCount(maxBytes)} bytes this tool can return, so nothing was returned. ${advice}`);
}
export function assertWithinCap(file: RestFile, advice: string): void {
export function assertWithinCap(file: RestFile, advice: string, maxBytes: number): void {
const size = Number(file.size ?? "0");
if (size > MAX_CONTENT_BYTES) throw tooLargeContent(advice, size);
if (size > maxBytes) throw tooLargeContent(advice, maxBytes, size);
}
export function exportFormat(mimeType: string, requested: string | undefined): string {
@@ -72,14 +72,14 @@ export function exportFormat(mimeType: string, requested: string | undefined): s
return requested;
}
export function downloadPlan(id: string, file: RestFile, exportMimeType: string | undefined, revision: string | undefined, advice: string): ContentPlan {
export function downloadPlan(id: string, file: RestFile, exportMimeType: string | undefined, revision: string | undefined, advice: string, maxBytes: number): ContentPlan {
const mimeType = file.mimeType ?? "application/octet-stream";
if (isGoogleType(mimeType)) {
const target = exportFormat(mimeType, exportMimeType);
if (revision) throw new GoogleToolError("unsupported", "Earlier revisions of Google Docs, Sheets, Slides and other Google files cannot be downloaded with this tool. Omit revisionId to export the current version.");
return { request: exportRequest(id, target), contentType: target };
}
if (!revision) assertWithinCap(file, advice);
if (!revision) assertWithinCap(file, advice, maxBytes);
return { request: mediaRequest(id, revision), contentType: mimeType };
}
@@ -87,11 +87,11 @@ export async function readContentMeta(id: string, http: GoogleHttp): Promise<Res
return http.send<RestFile>(contentMetaRequest(id));
}
export async function fetchContent(plan: ContentPlan, http: GoogleHttp, advice: string): Promise<Buffer> {
export async function fetchContent(plan: ContentPlan, http: GoogleHttp, advice: string, maxBytes: number): Promise<Buffer> {
try {
return (await http.download(plan.request, MAX_CONTENT_BYTES)).data;
return (await http.download(plan.request, maxBytes)).data;
} catch (error) {
if (error instanceof GoogleToolError && error.kind === "too_large" && error.httpStatus !== 403) throw tooLargeContent(advice);
if (error instanceof GoogleToolError && error.kind === "too_large" && error.httpStatus !== 403) throw tooLargeContent(advice, maxBytes);
throw error;
}
}
@@ -2,13 +2,13 @@ import { describe, expect, it } from "vitest";
import { GoogleToolError } from "../errors.js";
import { fakeGoogle, json } from "../fixtures/fake-google.js";
import { createGoogleHttp } from "../google-http.js";
import { MAX_RESULT_BYTES, serializedResultBytes } from "../tool.js";
import { FALLBACK_MAX_RESULT_BYTES, serializedResultBytes } from "../tool.js";
import { downloadPlan } from "./content.js";
import { downloadFileContent } from "./download-file-content.js";
import { MAX_CONTENT_BYTES, READ_LESS, type RestFile } from "./file.js";
import { READ_LESS, type RestFile } from "./file.js";
import { DOC, DRAWING, FOLDER, FORM, HUGE_PDF, PDF, SHEET, SLIDES, TEXT_FILE } from "./fixtures/drive-rest.js";
const plan = (file: RestFile, exportMimeType?: string, revision?: string) => downloadPlan(file.id!, file, exportMimeType, revision, READ_LESS.download);
const plan = (file: RestFile, exportMimeType?: string, revision?: string) => downloadPlan(file.id!, file, exportMimeType, revision, READ_LESS.download, FALLBACK_MAX_RESULT_BYTES);
const target = (file: RestFile, exportMimeType?: string, revision?: string) => {
const planned = plan(file, exportMimeType, revision);
const query = Object.entries(planned.request.query ?? {}).filter(([, value]) => value !== undefined).map(([key, value]) => [key, String(value)]);
@@ -36,7 +36,7 @@ describe("download plan", () => {
[FOLDER, undefined, undefined, /have no content to download/],
[FORM, undefined, undefined, /have no content to download/],
[DOC, undefined, "rev0001", /Earlier revisions/],
[HUGE_PDF, undefined, undefined, /This file is 10485761 bytes, more than the 250000 bytes/],
[HUGE_PDF, undefined, undefined, /This file is 10,485,761 bytes, more than the 250,000 bytes/],
[DOC, "image/png", undefined, /cannot be exported as image\/png. Use one of: text\/plain/],
[SHEET, "text/plain", undefined, /cannot be exported as text\/plain/],
[SLIDES, "application/x-anything", undefined, /cannot be exported/],
@@ -46,8 +46,8 @@ describe("download plan", () => {
});
it("always addresses the file the caller named, not an ID from Google's answer", () => {
expect(downloadPlan("callerFile1", { ...PDF, id: "otherFile9" }, undefined, undefined, "").request.path).toBe("/drive/v3/files/callerFile1");
expect(downloadPlan("callerFile1", { ...DOC, id: "otherFile9" }, undefined, undefined, "").request.path).toBe("/drive/v3/files/callerFile1/export");
expect(downloadPlan("callerFile1", { ...PDF, id: "otherFile9" }, undefined, undefined, "", FALLBACK_MAX_RESULT_BYTES).request.path).toBe("/drive/v3/files/callerFile1");
expect(downloadPlan("callerFile1", { ...DOC, id: "otherFile9" }, undefined, undefined, "", FALLBACK_MAX_RESULT_BYTES).request.path).toBe("/drive/v3/files/callerFile1/export");
});
});
@@ -82,21 +82,21 @@ describe("download_file_content tool", () => {
const { fetch, calls } = fakeGoogle(() => json(HUGE_PDF));
const result = await downloadFileContent.call({ fileId: HUGE_PDF.id }, createGoogleHttp({ accessToken: "t", fetch }));
expect(calls).toHaveLength(1);
expect(result).toEqual({ isError: true, content: [{ type: "text", text: `This file is 10485761 bytes, more than the 250000 bytes this tool returns, so nothing was returned. ${READ_LESS.download}` }] });
expect(result).toEqual({ isError: true, content: [{ type: "text", text: `This file is 10,485,761 bytes, more than the 250,000 bytes this tool can return, so nothing was returned. ${READ_LESS.download}` }] });
});
it("refuses an export that streams past the cap, and never returns a cut-short result", async () => {
const { fetch } = fakeGoogle(({ url }) => (url.pathname.endsWith("/export")
? new Response(new ReadableStream({
start(controller) {
controller.enqueue(new Uint8Array(MAX_CONTENT_BYTES));
controller.enqueue(new Uint8Array(FALLBACK_MAX_RESULT_BYTES));
controller.enqueue(new Uint8Array(1));
controller.close();
},
}))
: json(SHEET)));
const result = await downloadFileContent.call({ fileId: SHEET.id }, createGoogleHttp({ accessToken: "t", fetch }));
expect(result).toEqual({ isError: true, content: [{ type: "text", text: `This file's content is more than the 250000 bytes this tool returns, so nothing was returned. ${READ_LESS.download}` }] });
expect(result).toEqual({ isError: true, content: [{ type: "text", text: `This file's content is more than the 250,000 bytes this tool can return, so nothing was returned. ${READ_LESS.download}` }] });
});
it("bounds the final serialized result: the largest file that fits is returned, one byte more is refused", async () => {
@@ -111,10 +111,10 @@ describe("download_file_content tool", () => {
if (!(await run(largest + step)).isError) largest += step;
}
const fits = await run(largest);
expect(serializedResultBytes(fits)).toBeLessThanOrEqual(MAX_RESULT_BYTES);
expect(serializedResultBytes(fits)).toBeGreaterThan(MAX_RESULT_BYTES - 16);
expect(serializedResultBytes(fits)).toBeLessThanOrEqual(FALLBACK_MAX_RESULT_BYTES);
expect(serializedResultBytes(fits)).toBeGreaterThan(FALLBACK_MAX_RESULT_BYTES - 16);
const over = await run(largest + 3);
expect((over.content[0] as { text: string }).text).toMatch(/^The result is \d+ bytes, more than the 250000 bytes this tool returns, so nothing was returned\. This tool cannot return part of a file/);
expect((over.content[0] as { text: string }).text).toMatch(/^The result is [\d,]+ bytes, more than the 250,000 bytes this tool can return, so it was not returned\. This tool cannot return part of a file/);
});
it("reports Google's export size limit as too large", async () => {
@@ -1,20 +1,20 @@
import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { defineTool } from "../tool.js";
import { defineTool, structuredResult } from "../tool.js";
import { downloadPlan, fetchContent, readContentMeta } from "./content.js";
import { boundedResult, fileId, fileIdInput, mimeTypeArg, READ_LESS, revisionId, type RestFile } from "./file.js";
import { fileId, fileIdInput, mimeTypeArg, READ_LESS, revisionId, type RestFile } from "./file.js";
const args = z.object({ fileId, exportMimeType: mimeTypeArg.optional(), revisionId: revisionId.optional() });
export type DownloadFileContentArgs = z.infer<typeof args>;
export function downloadFileContentResult(id: string, file: RestFile, contentType: string, data: Buffer): CallToolResult {
return boundedResult({ id, title: file.name ?? "", mimeType: contentType, content: data.toString("base64") }, READ_LESS.download);
return structuredResult({ id, title: file.name ?? "", mimeType: contentType, content: data.toString("base64") });
}
export const downloadFileContent = defineTool({
definition: {
name: "download_file_content",
description: "Downloads a Drive file's content as base64. Google Docs, Sheets, Slides, Drawings and Apps Script files are exported: by default Docs and Slides as text/plain, Sheets as text/csv (first sheet only; pass exportMimeType application/vnd.openxmlformats-officedocument.spreadsheetml.sheet for every sheet), Drawings as image/svg+xml, Apps Script as JSON. Other files are returned as stored. The whole result is at most 250,000 bytes (about 90 KB of file content, since base64 is larger and the result carries it twice); a larger file is refused with an error, never cut short. For a readable text version of a document, use read_file_content.",
description: "Downloads a Drive file's content as base64. Google Docs, Sheets, Slides, Drawings and Apps Script files are exported: by default Docs and Slides as text/plain, Sheets as text/csv (first sheet only; pass exportMimeType application/vnd.openxmlformats-officedocument.spreadsheetml.sheet for every sheet), Drawings as image/svg+xml, Apps Script as JSON. Other files are returned as stored. The whole result has a size limit; a file too large for it is refused with an error that gives the limit, never cut short. For a readable text version of a document, use read_file_content.",
inputSchema: {
type: "object",
properties: {
@@ -37,9 +37,10 @@ export const downloadFileContent = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
async run(input, http) {
whenTooLarge: READ_LESS.download,
async run(input, http, limits) {
const file = await readContentMeta(input.fileId, http);
const plan = downloadPlan(input.fileId, file, input.exportMimeType, input.revisionId, READ_LESS.download);
return downloadFileContentResult(input.fileId, file, plan.contentType, await fetchContent(plan, http, READ_LESS.download));
const plan = downloadPlan(input.fileId, file, input.exportMimeType, input.revisionId, READ_LESS.download, limits.maxResultBytes);
return downloadFileContentResult(input.fileId, file, plan.contentType, await fetchContent(plan, http, READ_LESS.download, limits.maxResultBytes));
},
});
@@ -1,14 +1,12 @@
import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { GoogleToolError } from "../errors.js";
import type { GoogleRequest } from "../google-http.js";
import { boundedStructuredResult, compact, MAX_RESULT_BYTES } from "../tool.js";
import { compact } from "../tool.js";
export const DRIVE_HOST = "www.googleapis.com" as const;
export const DRIVE_BASE = "/drive/v3";
export const DRIVE_UPLOAD_BASE = "/upload/drive/v3";
export const MAX_CONTENT_BYTES = MAX_RESULT_BYTES;
export const MAX_UPLOAD_BYTES = 5 * 1024 * 1024;
export const DEFAULT_PAGE_SIZE = 10;
export const MAX_PAGE_SIZE = 100;
@@ -179,14 +177,6 @@ export const READ_LESS = {
permissions: "This file has too many permissions to list with this tool.",
} as const;
export function tooLargeMessage(bytes: number, advice: string): string {
return `The result is ${bytes} bytes, more than the ${MAX_RESULT_BYTES} bytes this tool returns, so nothing was returned. ${advice}`;
}
export function boundedResult(value: Record<string, unknown>, advice: string): CallToolResult {
return boundedStructuredResult(value, MAX_RESULT_BYTES, (bytes) => tooLargeMessage(bytes, advice));
}
export function isGoogleType(mimeType: string | undefined): boolean {
return Boolean(mimeType?.startsWith(GOOGLE_APPS_PREFIX));
}
@@ -1,9 +1,8 @@
import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { defineTool } from "../tool.js";
import { defineTool, structuredResult } from "../tool.js";
import {
assertSnippetArgs,
boundedResult,
READ_LESS,
fileFields,
fileId,
@@ -21,7 +20,7 @@ const args = z.object({ fileId, includeRichMetadata: z.boolean().optional(), ...
export type GetFileMetadataArgs = z.infer<typeof args>;
export function getFileMetadataResult(file: RestFile, rich: boolean): CallToolResult {
return boundedResult(toMcpFile(file, rich), READ_LESS.metadata);
return structuredResult(toMcpFile(file, rich));
}
export const getFileMetadata = defineTool({
@@ -41,6 +40,7 @@ export const getFileMetadata = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: READ_LESS.metadata,
async run(input, http) {
assertSnippetArgs(input);
const rich = input.includeRichMetadata === true;
@@ -2,8 +2,8 @@ import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { GoogleToolError } from "../errors.js";
import type { GoogleHttp, GoogleRequest } from "../google-http.js";
import { compact, defineTool } from "../tool.js";
import { boundedResult, DRIVE_HOST, fileId, fileIdInput, filePath, READ_LESS, type RestPermission } from "./file.js";
import { compact, defineTool, structuredResult } from "../tool.js";
import { DRIVE_HOST, fileId, fileIdInput, filePath, READ_LESS, type RestPermission } from "./file.js";
const args = z.object({ fileId });
export type GetFilePermissionsArgs = z.infer<typeof args>;
@@ -41,7 +41,7 @@ export function toMcpPermission(permission: RestPermission): Record<string, unkn
}
export function getFilePermissionsResult(permissions: RestPermission[]): CallToolResult {
return boundedResult({ permissions: permissions.filter((permission) => !permission.deleted).map(toMcpPermission) }, READ_LESS.permissions);
return structuredResult({ permissions: permissions.filter((permission) => !permission.deleted).map(toMcpPermission) });
}
export async function readPermissions(id: string, http: GoogleHttp): Promise<RestPermission[]> {
@@ -88,6 +88,7 @@ export const getFilePermissions = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: READ_LESS.permissions,
async run(input, http) {
return getFilePermissionsResult(await readPermissions(input.fileId, http));
},
@@ -1,10 +1,9 @@
import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import type { GoogleRequest } from "../google-http.js";
import { defineTool } from "../tool.js";
import { defineTool, structuredResult } from "../tool.js";
import {
assertSnippetArgs,
boundedResult,
READ_LESS,
DRIVE_BASE,
DRIVE_HOST,
@@ -50,7 +49,7 @@ export function listRecentFilesRequest(input: ListRecentFilesArgs): GoogleReques
}
export function listRecentFilesResult(list: RestFileList): CallToolResult {
return boundedResult(listResult(list), READ_LESS.list);
return structuredResult(listResult(list));
}
export const listRecentFiles = defineTool({
@@ -70,6 +69,7 @@ export const listRecentFiles = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: READ_LESS.list,
async run(input, http) {
assertSnippetArgs(input);
return listRecentFilesResult(await http.send<RestFileList>(listRecentFilesRequest(input)));
@@ -1,8 +1,8 @@
import { describe, expect, it } from "vitest";
import { fakeGoogle, json, type RecordedCall } from "../fixtures/fake-google.js";
import { createGoogleHttp } from "../google-http.js";
import { MAX_RESULT_BYTES, serializedResultBytes } from "../tool.js";
import { MAX_CONTENT_BYTES, READ_LESS, type RestFile } from "./file.js";
import { FALLBACK_MAX_RESULT_BYTES, serializedResultBytes } from "../tool.js";
import { READ_LESS, type RestFile } from "./file.js";
import { DOC, DRAWING, FOLDER, HUGE_PDF, JSON_FILE, PDF, SHEET, SLIDES, TEXT_FILE } from "./fixtures/drive-rest.js";
import { decodeText, readFileContent, readPlan, SHEET_NOTE, toCommentThread } from "./read-file-content.js";
@@ -15,7 +15,7 @@ describe("read plan", () => {
[JSON_FILE, "/drive/v3/files/json0001synthetic", "application/json"],
[{ id: "x1", mimeType: "application/ld+json", size: "3" }, "/drive/v3/files/x1", "application/ld+json"],
] as const)("%#: reads %s from %s as %s", (file, path, contentType) => {
const plan = readPlan(file.id!, file);
const plan = readPlan(file.id!, file, FALLBACK_MAX_RESULT_BYTES);
expect(plan?.request.path).toBe(path);
expect(plan?.contentType).toBe(contentType);
});
@@ -23,12 +23,12 @@ describe("read plan", () => {
it.each([PDF, FOLDER, DRAWING, { id: "i1", mimeType: "image/png" }, { id: "w1", mimeType: "application/vnd.openxmlformats-officedocument.wordprocessingml.document" }])(
"%#: has no text version of %s",
(file: RestFile) => {
expect(readPlan(file.id!, file)).toBeNull();
expect(readPlan(file.id!, file, FALLBACK_MAX_RESULT_BYTES)).toBeNull();
},
);
it("refuses a text file over the cap before downloading", () => {
expect(() => readPlan("big1", { ...HUGE_PDF, mimeType: "text/plain" })).toThrow(/more than the 250000 bytes/);
expect(() => readPlan("big1", { ...HUGE_PDF, mimeType: "text/plain" }, FALLBACK_MAX_RESULT_BYTES)).toThrow(/more than the 250,000 bytes/);
});
it("decodes UTF-8, drops a byte order mark, and rejects bytes that are not UTF-8", () => {
@@ -106,9 +106,9 @@ describe("read_file_content tool", () => {
});
it("refuses content that grows past the cap while streaming", async () => {
const { fetch } = route(SLIDES, new Uint8Array(MAX_CONTENT_BYTES + 1));
const { fetch } = route(SLIDES, new Uint8Array(FALLBACK_MAX_RESULT_BYTES + 1));
const result = await readFileContent.call({ fileId: SLIDES.id }, createGoogleHttp({ accessToken: "t", fetch }));
expect(result).toEqual({ isError: true, content: [{ type: "text", text: `This file's content is more than the 250000 bytes this tool returns, so nothing was returned. ${READ_LESS.read}` }] });
expect(result).toEqual({ isError: true, content: [{ type: "text", text: `This file's content is more than the 250,000 bytes this tool can return, so nothing was returned. ${READ_LESS.read}` }] });
});
it("bounds the final serialized result, including JSON escaping, with advice on reading less", async () => {
@@ -116,17 +116,17 @@ describe("read_file_content tool", () => {
const result = await readFileContent.call({ fileId: SLIDES.id }, createGoogleHttp({ accessToken: "t", fetch }));
expect(result.isError).toBe(true);
const text = (result.content[0] as { text: string }).text;
expect(text).toMatch(/^The result is \d+ bytes, more than the 250000 bytes this tool returns, so nothing was returned\. /);
expect(text).toMatch(/^The result is [\d,]+ bytes, more than the 250,000 bytes this tool can return, so it was not returned\. /);
expect(text.endsWith(READ_LESS.read)).toBe(true);
expect(Number(/\d+/.exec(text)![0])).toBeGreaterThan(2 * 6 * 30_000);
expect(Number(/[\d,]+/.exec(text)![0].replaceAll(",", ""))).toBeGreaterThan(2 * 6 * 30_000);
});
it("returns the largest text that fits and refuses one byte more", async () => {
const run = async (size: number) => readFileContent.call({ fileId: SLIDES.id }, createGoogleHttp({ accessToken: "t", fetch: route(SLIDES, "a".repeat(size)).fetch }));
let largest = 0;
for (let step = 1 << 17; step >= 1; step >>= 1) if (!(await run(largest + step)).isError) largest += step;
expect(serializedResultBytes(await run(largest))).toBeLessThanOrEqual(MAX_RESULT_BYTES);
expect(serializedResultBytes(await run(largest))).toBeGreaterThan(MAX_RESULT_BYTES - 4);
expect(serializedResultBytes(await run(largest))).toBeLessThanOrEqual(FALLBACK_MAX_RESULT_BYTES);
expect(serializedResultBytes(await run(largest))).toBeGreaterThan(FALLBACK_MAX_RESULT_BYTES - 4);
expect((await run(largest + 1)).isError).toBe(true);
});
});
@@ -2,9 +2,9 @@ import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { GoogleToolError } from "../errors.js";
import type { GoogleHttp, GoogleRequest } from "../google-http.js";
import { compact, defineTool } from "../tool.js";
import { compact, defineTool, structuredResult } from "../tool.js";
import { assertWithinCap, exportRequest, fetchContent, mediaRequest, readContentMeta, type ContentPlan } from "./content.js";
import { boundedResult, DRIVE_HOST, fileId, fileIdInput, filePath, GOOGLE_DOC, GOOGLE_SHEET, GOOGLE_SLIDES, READ_LESS, type RestFile } from "./file.js";
import { DRIVE_HOST, fileId, fileIdInput, filePath, GOOGLE_DOC, GOOGLE_SHEET, GOOGLE_SLIDES, READ_LESS, type RestFile } from "./file.js";
const args = z.object({ fileId, includeComments: z.boolean().optional() });
export type ReadFileContentArgs = z.infer<typeof args>;
@@ -22,13 +22,13 @@ export interface ReadPlan extends ContentPlan {
prefix?: string;
}
export function readPlan(id: string, file: RestFile): ReadPlan | null {
export function readPlan(id: string, file: RestFile, maxBytes: number): ReadPlan | null {
const mimeType = file.mimeType ?? "";
if (mimeType === GOOGLE_DOC) return { request: exportRequest(id, "text/markdown"), contentType: "text/markdown" };
if (mimeType === GOOGLE_SLIDES) return { request: exportRequest(id, "text/plain"), contentType: "text/plain" };
if (mimeType === GOOGLE_SHEET) return { request: exportRequest(id, "text/csv"), contentType: "text/csv", prefix: SHEET_NOTE };
if (mimeType.startsWith("text/") || TEXT_APPLICATION_TYPES.has(mimeType) || /\+(json|xml)$/.test(mimeType)) {
assertWithinCap(file, READ_LESS.read);
assertWithinCap(file, READ_LESS.read, maxBytes);
return { request: mediaRequest(id), contentType: mimeType };
}
return null;
@@ -90,13 +90,13 @@ export async function readComments(id: string, http: GoogleHttp): Promise<RestCo
export function readFileContentResult(file: RestFile, text: string | null, comments: RestComment[] | null, includeComments: boolean): CallToolResult {
const commentable = COMMENTABLE.has(file.mimeType ?? "");
return boundedResult(compact({
return structuredResult(compact({
fileContent: text ?? "",
contentAnchoredComments: includeComments ? [] : undefined,
unanchoredComments: includeComments ? (comments ?? []).map(toCommentThread) : undefined,
textFormattingNotSupported: text === null ? true : undefined,
commentsNotSupported: includeComments && !commentable ? true : undefined,
}), READ_LESS.read);
}));
}
const postSchema = {
@@ -121,7 +121,7 @@ const threadSchema = {
export const readFileContent = defineTool({
definition: {
name: "read_file_content",
description: "Returns a text version of a Drive file. Google Docs come back as Markdown, Slides as plain text, and Sheets as CSV of the first sheet only (the text says so). Text files (text/*, JSON, XML and similar) are returned as stored. Other files, such as PDFs, Office files and images, are not converted: the result has textFormattingNotSupported true and empty fileContent; use download_file_content for their bytes. The whole result is at most 250,000 bytes (about 120 KB of plain text, since the result carries it twice); a larger file is refused with an error, never cut short. With includeComments, the comment threads of a Google Doc, Sheet or Slides file are listed in unanchoredComments; they are not placed in the text. Get fileId from search_files or list_recent_files first.",
description: "Returns a text version of a Drive file. Google Docs come back as Markdown, Slides as plain text, and Sheets as CSV of the first sheet only (the text says so). Text files (text/*, JSON, XML and similar) are returned as stored. Other files, such as PDFs, Office files and images, are not converted: the result has textFormattingNotSupported true and empty fileContent; use download_file_content for their bytes. The whole result has a size limit; a file too large for it is refused with an error that gives the limit, never cut short. With includeComments, the comment threads of a Google Doc, Sheet or Slides file are listed in unanchoredComments; they are not placed in the text. Get fileId from search_files or list_recent_files first.",
inputSchema: {
type: "object",
properties: {
@@ -144,12 +144,13 @@ export const readFileContent = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
async run(input, http) {
whenTooLarge: READ_LESS.read,
async run(input, http, limits) {
const file = await readContentMeta(input.fileId, http);
const plan = readPlan(input.fileId, file);
const plan = readPlan(input.fileId, file, limits.maxResultBytes);
let text: string | null = null;
if (plan) {
const decoded = decodeText(await fetchContent(plan, http, READ_LESS.read));
const decoded = decodeText(await fetchContent(plan, http, READ_LESS.read, limits.maxResultBytes));
text = decoded === null ? null : plan.prefix ? `${plan.prefix}\n${decoded}` : decoded;
}
const comments = input.includeComments && COMMENTABLE.has(file.mimeType ?? "") ? await readComments(input.fileId, http) : null;
@@ -108,7 +108,7 @@ describe("search_files tool", () => {
const { fetch } = fakeGoogle(() => json({ files }));
const result = await searchFiles.call({ pageSize: 100 }, createGoogleHttp({ accessToken: "t", fetch }));
expect(result.isError).toBe(true);
expect((result.content[0] as { text: string }).text).toMatch(/more than the 250000 bytes this tool returns, so nothing was returned\. Ask for fewer files with a smaller pageSize/);
expect((result.content[0] as { text: string }).text).toMatch(/more than the 250,000 bytes this tool can return, so it was not returned\. Ask for fewer files with a smaller pageSize/);
});
it("returns an empty list when nothing matches", async () => {
@@ -2,10 +2,9 @@ import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { GoogleToolError } from "../errors.js";
import type { GoogleRequest } from "../google-http.js";
import { defineTool } from "../tool.js";
import { defineTool, structuredResult } from "../tool.js";
import {
assertSnippetArgs,
boundedResult,
READ_LESS,
DRIVE_BASE,
DRIVE_HOST,
@@ -58,7 +57,7 @@ export function searchFilesRequest(input: SearchFilesArgs): GoogleRequest {
}
export function searchFilesResult(list: RestFileList, rich: boolean): CallToolResult {
return boundedResult(listResult(list, rich), READ_LESS.list);
return structuredResult(listResult(list, rich));
}
const QUERY_HELP = [
@@ -88,6 +87,7 @@ export const searchFiles = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: READ_LESS.list,
async run(input, http) {
assertSnippetArgs(input);
const rich = input.includeRichMetadata === true;
@@ -78,6 +78,7 @@ export const getDraft = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Read it with view DRAFT_VIEW_METADATA_ONLY, which leaves out the body.",
async run(input, http) {
const draft = await http.send<GmailRestDraftFull>(readDraftRequest(input.draftId, input.view));
return getDraftResult(draft, input, await readDraftBodies([draft], input.view, http));
@@ -70,6 +70,7 @@ export const getMessage = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Read it with messageFormat METADATA_ONLY or MINIMAL, which leave out the bodies.",
async run(input, http) {
const message = await http.send<GmailRestMessage>(getMessageRequest(input));
return getMessageResult(message, input, await readExternalBodies(message, input, http));
@@ -72,7 +72,7 @@ describe("get_thread with bodies Gmail stores separately", () => {
if (url.pathname.endsWith("/attachments/ANGjdJ_m2_html")) return json({ data: Buffer.from(`<pre>${large}</pre>`).toString("base64url") });
return json(thread);
});
const result = await getThread.call({ threadId: "th1" }, createGoogleHttp({ accessToken: "t", fetch }));
const result = await getThread.call({ threadId: "th1" }, createGoogleHttp({ accessToken: "t", fetch }), { maxResultBytes: 1_000_000 });
const messages = (result.structuredContent as { messages: Array<{ plaintextBody?: string; htmlBody?: string }> }).messages;
expect(messages[0]!.plaintextBody).toBe("short");
expect(messages[1]!.plaintextBody).toBe(large);
@@ -58,6 +58,7 @@ export const getThread = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Read it with messageFormat METADATA_ONLY or MINIMAL, which leave out the bodies, then read single messages with get_message.",
async run(input, http) {
const thread = await http.send<GmailRestFullThread>(readThreadRequest(input));
const bodies = messageDetail(input.messageFormat) === "full" ? await readMessageBodies(thread.messages ?? [], http) : undefined;
@@ -76,6 +76,7 @@ export const listDrafts = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: false, openWorldHint: false },
},
args,
whenTooLarge: "Ask for fewer drafts with a smaller pageSize, or use view DRAFT_VIEW_METADATA_ONLY, then read single drafts with get_draft.",
async run(input, http) {
const list = await http.send<GmailDraftList>(listDraftsRequest(input));
const drafts = await readDrafts(draftIds(list), input.view, http);
@@ -89,6 +89,7 @@ export const listLabels = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Ask for fewer labels with a smaller pageSize.",
async run(input, http) {
return listLabelsResult(await http.send<GmailLabelList>(listLabelsRequest()), input);
},
@@ -89,6 +89,7 @@ export const listThreads = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Ask for fewer threads with a smaller pageSize, then read single threads with get_thread.",
async run(input, http) {
const list = await http.send<GmailThreadList>(listThreadsToolRequest(input));
return searchThreadsResult(list, await readThreads(threadIds(list), http), { view: input.view });
@@ -98,6 +98,7 @@ export const searchMessages = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Ask for fewer messages with a smaller pageSize, or use messageFormat MINIMAL, then read single messages with get_message.",
async run(input, http) {
const list = await http.send<GmailMessageList>(listMessagesRequest(input));
const messages = await readMessages(messageIds(list), input, http);
@@ -148,6 +148,7 @@ export const searchThreads = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Ask for fewer threads with a smaller pageSize, then read single threads with get_thread.",
async run(input, http) {
const list = await http.send<GmailThreadList>(listThreadsRequest(input));
return searchThreadsResult(list, await readThreads(threadIds(list), http), input);
@@ -0,0 +1,182 @@
import { describe, expect, it } from "vitest";
import { oneTab, text } from "./docs/fixtures/docs-rest.js";
import { fakeGoogle, json } from "./fixtures/fake-google.js";
import { shape, slide } from "./slides/fixtures/slides-rest.js";
import { z } from "zod";
import { defineTool, FALLBACK_MAX_RESULT_BYTES, serializedResultBytes } from "./tool.js";
import { callGoogleWorkspaceRestTool, type GoogleWorkspaceRestService } from "./tools.js";
import { READ_LESS } from "./drive/file.js";
interface BudgetCase {
service: GoogleWorkspaceRestService;
name: string;
arguments: (size: number) => Record<string, unknown>;
answer: (size: number, url: URL) => Response;
advice: string;
}
const filler = (size: number) => "a".repeat(size);
const b64 = (value: string) => Buffer.from(value, "utf8").toString("base64url");
const CASES: BudgetCase[] = [
{
service: "gmail",
name: "get_message",
arguments: () => ({ messageId: "msg1" }),
answer: (size) => json({
id: "msg1", threadId: "thr1", internalDate: "1763337600000",
payload: { mimeType: "text/plain", headers: [{ name: "Subject", value: "Report" }], body: { size, data: b64(filler(size)) } },
}),
advice: "Read it with messageFormat METADATA_ONLY or MINIMAL, which leave out the bodies.",
},
{
service: "gmail",
name: "list_labels",
arguments: () => ({}),
answer: (size) => json({ labels: [{ id: "Label_1", name: filler(size), type: "user" }] }),
advice: "Ask for fewer labels with a smaller pageSize.",
},
{
service: "calendar",
name: "list_calendars",
arguments: () => ({}),
answer: (size) => json({ items: [{ id: "primary", summary: filler(size), timeZone: "UTC" }] }),
advice: "Ask for fewer calendars with a smaller pageSize.",
},
{
service: "calendar",
name: "get_event",
arguments: () => ({ eventId: "evt1" }),
answer: (size) => json({ id: "evt1", status: "confirmed", summary: "Review", description: filler(size),
start: { dateTime: "2026-11-18T10:00:00Z" }, end: { dateTime: "2026-11-18T11:00:00Z" } }),
advice: "This tool cannot return part of an event.",
},
{
service: "drive",
name: "read_file_content",
arguments: () => ({ fileId: "text0001synthetic" }),
answer: (size, url) => url.searchParams.get("alt") === "media"
? new Response(filler(size))
: json({ id: "text0001synthetic", name: "notes.txt", mimeType: "text/plain", size: String(size) }),
advice: READ_LESS.read,
},
{
service: "docs",
name: "read_doc",
arguments: () => ({ documentId: "doc1" }),
answer: (size) => json(oneTab([text(filler(size))])),
advice: "This tool cannot read part of a document. Read it without commentsIncluded if that was set, or read only its text with read_file_content.",
},
{
service: "slides",
name: "read_presentation",
arguments: () => ({ presentationId: "deck1" }),
answer: (size) => json({ presentationId: "deck1", revisionId: "r1", slides: [slide("s1", [shape("e1", [{ text: filler(size) }])])] }),
advice: "Pass fields to read only part of it, for example [\"slides.objectId\", \"slides.pageElements(objectId,shape.text)\"].",
},
{
service: "sheets",
name: "get_values",
arguments: () => ({ spreadsheetId: "sheet1", range: "Sheet1!A1" }),
answer: (size) => json({ range: "Sheet1!A1", majorDimension: "ROWS", values: [[filler(size)]] }),
advice: "Ask for a narrower range, with fewer rows or columns (for example Sheet1!A1:Z1000), and read the rest in further calls.",
},
];
function call(testCase: BudgetCase, size: number, maxResultBytes?: number) {
const { fetch } = fakeGoogle(({ url }) => testCase.answer(size, url));
return callGoogleWorkspaceRestTool({
service: testCase.service, name: testCase.name, arguments: testCase.arguments(size), accessToken: "t", fetch,
...(maxResultBytes === undefined ? {} : { maxResultBytes }),
});
}
async function largestThatFits(testCase: BudgetCase, maxResultBytes?: number): Promise<number> {
let largest = 0;
for (let step = 1 << 21; step >= 1; step >>= 1) {
if (!(await call(testCase, largest + step, maxResultBytes)).isError) largest += step;
}
return largest;
}
const errorText = (result: { content: unknown[] }) => (result.content[0] as { text: string }).text;
describe("result budget", () => {
it.each(CASES.map((testCase) => [`${testCase.service} ${testCase.name}`, testCase] as const))(
"%s: returns a result just under the budget it is given and refuses one just over it, saying how to ask for less",
async (_label, testCase) => {
const budget = 40_000;
const largest = await largestThatFits(testCase, budget);
const fits = await call(testCase, largest, budget);
expect(fits.isError).toBeUndefined();
expect(serializedResultBytes(fits)).toBeLessThanOrEqual(budget);
expect(serializedResultBytes(fits)).toBeGreaterThan(budget - 64);
const over = await call(testCase, largest + 1, budget);
expect(over.isError).toBe(true);
expect(over.structuredContent).toBeUndefined();
expect(errorText(over)).toMatch(/^The result is [\d,]+ bytes, more than the 40,000 bytes this tool can return, so it was not returned\. /);
expect(errorText(over).endsWith(testCase.advice)).toBe(true);
},
);
it.each(CASES.map((testCase) => [`${testCase.service} ${testCase.name}`, testCase] as const))(
"%s: uses the fallback budget only when the caller passes none",
async (_label, testCase) => {
const size = 150_000;
const fallback = await call(testCase, size);
expect(fallback.isError).toBe(true);
expect(errorText(fallback)).toMatch(/more than the 250,000 bytes this tool can return/);
const supplied = await call(testCase, size, 1_000_000);
expect(supplied.isError).toBeUndefined();
expect(serializedResultBytes(supplied)).toBeGreaterThan(FALLBACK_MAX_RESULT_BYTES);
},
);
it("never turns a completed write into an error, whatever the size of its result", async () => {
const { fetch, calls } = fakeGoogle(() => json({ id: "draft1", message: { id: "msg2", threadId: "thr2" } }));
const body = filler(30_000);
const result = await callGoogleWorkspaceRestTool({ service: "gmail", name: "create_draft", arguments: { to: ["sam@example.test"], subject: "Report", body },
accessToken: "t", fetch, maxResultBytes: 1_000 });
expect(result.isError).toBeUndefined();
expect(serializedResultBytes(result)).toBeGreaterThan(1_000);
expect(result.structuredContent).toMatchObject({ id: "draft1", plaintextBody: body });
expect(calls).toHaveLength(1);
});
it("requires advice from every read tool and none from a write tool", () => {
const definition = (readOnlyHint: boolean) => ({ name: "t", inputSchema: { type: "object" as const }, annotations: { readOnlyHint } });
const run = async () => ({ content: [] });
expect(() => defineTool({ definition: definition(true), args: z.object({}), run })).toThrow(/read tool must say how to ask for less/);
expect(() => defineTool({ definition: definition(false), args: z.object({}), whenTooLarge: "x", run })).toThrow(/write tool must not/);
});
it("refuses a budget that is not a positive whole number before calling Google", async () => {
const { fetch, calls } = fakeGoogle(() => json({ labels: [] }));
for (const maxResultBytes of [0, -1, 1.5, Number.NaN, Number.POSITIVE_INFINITY]) {
await expect(callGoogleWorkspaceRestTool({ service: "gmail", name: "list_labels", accessToken: "t", fetch, maxResultBytes }))
.rejects.toThrow(/maxResultBytes must be a positive integer/);
}
expect(calls).toHaveLength(0);
});
it("caps a Drive download at the budget it is given, not at the fallback", async () => {
const big = { id: "pdf1", name: "big.pdf", mimeType: "application/pdf", size: String(FALLBACK_MAX_RESULT_BYTES + 1) };
const { fetch, calls } = fakeGoogle(({ url }) => url.searchParams.get("alt") === "media" ? new Response(new Uint8Array(FALLBACK_MAX_RESULT_BYTES + 1)) : json(big));
const fallback = await callGoogleWorkspaceRestTool({ service: "drive", name: "download_file_content", arguments: { fileId: "pdf1" }, accessToken: "t", fetch });
expect(errorText(fallback)).toMatch(/^This file is 250,001 bytes, more than the 250,000 bytes this tool can return/);
expect(calls).toHaveLength(1);
const supplied = await callGoogleWorkspaceRestTool({ service: "drive", name: "download_file_content", arguments: { fileId: "pdf1" }, accessToken: "t", fetch, maxResultBytes: 1_000_000 });
expect(supplied.isError).toBeUndefined();
expect(Buffer.from((supplied.structuredContent as { content: string }).content, "base64")).toHaveLength(FALLBACK_MAX_RESULT_BYTES + 1);
});
it("tells a Gmail list caller to lower the page size", async () => {
const thread = (id: string) => ({ id, messages: [{ id: `${id}m`, threadId: id, internalDate: "1763337600000", snippet: filler(2_000),
payload: { headers: [{ name: "Subject", value: filler(2_000) }] } }] });
const { fetch } = fakeGoogle(({ url }) => url.pathname.endsWith("/threads")
? json({ threads: Array.from({ length: 20 }, (_, index) => ({ id: `t${index}` })) })
: json(thread(url.pathname.split("/").at(-1)!)));
const result = await callGoogleWorkspaceRestTool({ service: "gmail", name: "search_threads", arguments: { query: "report" }, accessToken: "t", fetch, maxResultBytes: 20_000 });
expect(errorText(result)).toMatch(/so it was not returned\. Ask for fewer threads with a smaller pageSize, then read single threads with get_thread\.$/);
});
});
@@ -7,6 +7,7 @@ import { callGoogleWorkspaceRestTool, listGoogleWorkspaceRestTools, type GoogleW
export interface GoogleWorkspaceRestServerOptions {
fetch?: GoogleFetch;
timeoutMs?: number;
maxResultBytes?: number;
}
export function createGoogleWorkspaceRestServer(service: GoogleWorkspaceRestService, options: GoogleWorkspaceRestServerOptions = {}): Server {
@@ -26,6 +27,7 @@ export function createGoogleWorkspaceRestServer(service: GoogleWorkspaceRestServ
fetch: options.fetch,
signal: extra.signal,
timeoutMs: options.timeoutMs,
maxResultBytes: options.maxResultBytes,
});
});
return server;
@@ -3,7 +3,6 @@ import { fakeGoogle, json } from "../fixtures/fake-google.js";
import { createGoogleHttp } from "../google-http.js";
import { SPREADSHEET_ID, SPREADSHEET_METADATA, SPREADSHEET_WITH_GRID } from "./fixtures/sheets-rest.js";
import { serializedResultBytes } from "../tool.js";
import { MAX_RESULT_BYTES } from "./spreadsheet.js";
import { getSpreadsheet, getSpreadsheetRequest, getSpreadsheetResult, gridCellCount } from "./get-spreadsheet.js";
const text = (result: { content: unknown[] }) => (result.content[0] as { text: string }).text;
@@ -75,11 +74,13 @@ describe("get_spreadsheet result", () => {
expect(gridCellCount(SPREADSHEET_WITH_GRID)).toBe(3);
});
it("bounds the whole serialized result in bytes and asks for fewer or narrower ranges", () => {
it("bounds the whole serialized result in bytes and asks for fewer or narrower ranges", async () => {
const sheet = (note: string) => ({ properties: { title: note } });
const near = getSpreadsheetResult({ sheets: [sheet("x".repeat(124_900))] });
expect(serializedResultBytes(near)).toBeLessThanOrEqual(MAX_RESULT_BYTES);
expect(() => getSpreadsheetResult({ sheets: [sheet("x".repeat(125_000))] })).toThrow(/bytes; the limit is 250,000 per call\. Pass fewer or narrower ranges/);
const call = (title: string) => getSpreadsheet.call({ spreadsheetId: SPREADSHEET_ID }, createGoogleHttp({ accessToken: "t", fetch: fakeGoogle(() => json({ sheets: [sheet(title)] })).fetch }));
const near = await call("x".repeat(124_900));
expect(near.isError).toBeUndefined();
expect(serializedResultBytes(near)).toBeLessThanOrEqual(250_000);
expect(text(await call("x".repeat(125_000)))).toMatch(/bytes, more than the 250,000 bytes this tool can return, so it was not returned\. Pass fewer or narrower ranges/);
});
it("refuses a result over the cell limit instead of cutting it", () => {
@@ -110,6 +110,7 @@ export const getSpreadsheet = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: true },
},
args,
whenTooLarge: NARROW,
async run(input, http) {
return getSpreadsheetResult(await sendRead<RestSpreadsheet>(http, getSpreadsheetRequest(input), NARROW));
},
@@ -3,8 +3,8 @@ import { fakeGoogle, json } from "../fixtures/fake-google.js";
import { createGoogleHttp } from "../google-http.js";
import { EMPTY_VALUES, FORMATTED_VALUES, SPREADSHEET_ID } from "./fixtures/sheets-rest.js";
import { getValues, getValuesRequest, getValuesResult } from "./get-values.js";
import { serializedResultBytes } from "../tool.js";
import { boundedReadResult, MAX_CELLS_PER_CALL, MAX_RESULT_BYTES } from "./spreadsheet.js";
import { FALLBACK_MAX_RESULT_BYTES, serializedResultBytes } from "../tool.js";
import { boundedReadResult, MAX_CELLS_PER_CALL } from "./spreadsheet.js";
const text = (result: { content: unknown[] }) => (result.content[0] as { text: string }).text;
@@ -34,24 +34,29 @@ describe("get_values result", () => {
expect(getValuesResult(EMPTY_VALUES).structuredContent).toEqual({ range: "Sheet1!A1:Z100" });
});
const callWith = (values: unknown[][]) => getValues.call({ spreadsheetId: SPREADSHEET_ID, range: "Sheet1!A1:Z" },
createGoogleHttp({ accessToken: "t", fetch: fakeGoogle(() => json({ range: "Sheet1!A1:Z", values })).fetch }));
it.each([
["too many cells", [Array.from({ length: MAX_CELLS_PER_CALL + 1 }, () => "")], /50,001 cells; the limit is 50,000/],
["too many bytes", [["x".repeat(130_000)]], /bytes; the limit is 250,000 per call\. Ask for a narrower range/],
])("refuses %s instead of cutting the result", (_name, values, message) => {
expect(() => getValuesResult({ range: "Sheet1!A1:Z", values })).toThrow(message);
["too many bytes", [["x".repeat(130_000)]], /bytes, more than the 250,000 bytes this tool can return, so it was not returned\. Ask for a narrower range/],
])("refuses %s instead of cutting the result", async (_name, values, message) => {
const result = await callWith(values);
expect(result.isError).toBe(true);
expect(text(result)).toMatch(message);
});
it.each([["x"], ["é"], ["\""]])("bounds the whole serialized result, text and structured content together, for cells of %j", (unit) => {
const sized = (count: number) => ({ range: "Sheet1!A1:B2", values: [[unit.repeat(count)], ["tail"]] });
it.each([["x"], ["é"], ["\""]])("bounds the whole serialized result, text and structured content together, for cells of %j", async (unit) => {
const sized = (count: number) => ({ range: "Sheet1!A1:Z", values: [[unit.repeat(count)], ["tail"]] });
const bytes = (count: number) => serializedResultBytes({ content: [{ type: "text", text: JSON.stringify(sized(count)) }], structuredContent: sized(count) });
let largest = 0;
for (let step = 1 << 20; step >= 1; step >>= 1) if (bytes(largest + step) <= MAX_RESULT_BYTES) largest += step;
const near = getValuesResult(sized(largest));
for (let step = 1 << 20; step >= 1; step >>= 1) if (bytes(largest + step) <= FALLBACK_MAX_RESULT_BYTES) largest += step;
const near = await callWith(sized(largest).values);
expect(near.isError).toBeUndefined();
expect(serializedResultBytes(near)).toBe(bytes(largest));
expect(serializedResultBytes(near)).toBeLessThanOrEqual(MAX_RESULT_BYTES);
expect(serializedResultBytes(near)).toBeGreaterThan(MAX_RESULT_BYTES - 16);
expect(bytes(largest + 1)).toBeGreaterThan(MAX_RESULT_BYTES);
expect(() => getValuesResult(sized(largest + 1))).toThrow(/bytes; the limit is 250,000 per call\. Ask for a narrower range/);
expect(serializedResultBytes(near)).toBeGreaterThan(FALLBACK_MAX_RESULT_BYTES - 16);
expect(bytes(largest + 1)).toBeGreaterThan(FALLBACK_MAX_RESULT_BYTES);
expect(text(await callWith(sized(largest + 1).values))).toMatch(/bytes, more than the 250,000 bytes this tool can return, so it was not returned\. Ask for a narrower range/);
});
it("accepts exactly the cell limit and refuses one more", () => {
@@ -65,6 +65,7 @@ export const getValues = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: true },
},
args,
whenTooLarge: NARROW,
async run(input, http) {
return getValuesResult(await sendRead<RestValueRange>(http, getValuesRequest(input), NARROW));
},
@@ -2,13 +2,12 @@ import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { GoogleToolError } from "../errors.js";
import type { GoogleHttp, GoogleRequest } from "../google-http.js";
import { boundedStructuredResult, MAX_RESULT_BYTES } from "../tool.js";
import { structuredResult } from "../tool.js";
import { checkSheetRange } from "./range.js";
export const SHEETS_HOST = "sheets.googleapis.com" as const;
export const SHEETS_BASE = "/v4/spreadsheets";
export const MAX_CELLS_PER_CALL = 50_000;
export { MAX_RESULT_BYTES };
export const MAX_REQUEST_CHARS = 1_000_000;
const formatCount = (value: number) => value.toLocaleString("en-US");
@@ -53,7 +52,7 @@ export function boundedReadResult(value: Record<string, unknown>, cells: number,
if (cells > MAX_CELLS_PER_CALL) {
throw new GoogleToolError("incomplete", `The result has ${formatCount(cells)} cells; the limit is ${formatCount(MAX_CELLS_PER_CALL)} per call. ${narrow}`);
}
return boundedStructuredResult(value, MAX_RESULT_BYTES, (bytes) => `The result is ${formatCount(bytes)} bytes; the limit is ${formatCount(MAX_RESULT_BYTES)} per call. ${narrow}`);
return structuredResult(value);
}
export async function sendRead<T>(http: GoogleHttp, request: GoogleRequest, narrow: string): Promise<T> {
@@ -1,5 +1,5 @@
import { describe, expect, it } from "vitest";
import { MAX_RESULT_BYTES } from "../tool.js";
import { FALLBACK_MAX_RESULT_BYTES } from "../tool.js";
import { fakeGoogle, json } from "../fixtures/fake-google.js";
import { createGoogleHttp } from "../google-http.js";
import { DECK, DECK_TEXT, shape, slide } from "./fixtures/slides-rest.js";
@@ -61,11 +61,12 @@ describe("read_presentation tool", () => {
it("refuses a result over the size limit and suggests fields", async () => {
const line = "y".repeat(10_000);
const big = { presentationId: "big", slides: Array.from({ length: Math.ceil(MAX_RESULT_BYTES / 20_000) + 5 }, (_, index) => slide(`s${index}`, [shape(`e${index}`, [{ text: line }])])) };
const big = { presentationId: "big", slides: Array.from({ length: Math.ceil(FALLBACK_MAX_RESULT_BYTES / 20_000) + 5 }, (_, index) => slide(`s${index}`, [shape(`e${index}`, [{ text: line }])])) };
const { fetch } = fakeGoogle(() => json(big));
const result = await readPresentation.call({ presentationId: "big" }, createGoogleHttp({ accessToken: "t", fetch }));
expect(result.isError).toBe(true);
expect((result.content[0] as { text: string }).text).toMatch(/^This presentation is too large to return in one result .*Pass fields to read only part of it/);
expect((result.content[0] as { text: string }).text).toMatch(/\(\d+ KB; the limit is 250 KB\)/);
expect((result.content[0] as { text: string }).text).toMatch(/^The result is [\d,]+ bytes, more than the 250,000 bytes this tool can return, so it was not returned\. Pass fields to read only part of it/);
const fields = await readPresentation.call({ presentationId: "big", fields: ["slides"] }, createGoogleHttp({ accessToken: "t", fetch }));
expect((fields.content[0] as { text: string }).text).toMatch(/so it was not returned\. Ask for fewer fields\.$/);
});
});
@@ -1,8 +1,8 @@
import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { fileIdArg, readWithComments, sizedResult } from "../documents-shared.js";
import { fileIdArg, readWithComments } from "../documents-shared.js";
import type { GoogleRequest } from "../google-http.js";
import { defineTool } from "../tool.js";
import { defineTool, structuredResult } from "../tool.js";
import { presentationPath, SLIDES_HOST, type RestPresentation } from "./presentation.js";
import { renderPresentation } from "./render.js";
@@ -35,10 +35,10 @@ export const COMMENTS_UNAVAILABLE = "Comments are not available for this present
export function readPresentationResult(presentation: RestPresentation, input: ReadPresentationArgs): CallToolResult {
if (input.fields?.length) {
return sizedResult({ content: presentation as Record<string, unknown> }, "presentation", "Ask for fewer fields.");
return structuredResult({ content: presentation as Record<string, unknown> });
}
const content = renderPresentation(presentation, { commentsRequested: input.commentsIncluded === true });
return sizedResult({ content }, "presentation", "Pass fields to read only part of it, for example [\"slides.objectId\", \"slides.pageElements(objectId,shape.text)\"].");
return structuredResult({ content });
}
export const readPresentation = defineTool({
@@ -73,6 +73,7 @@ export const readPresentation = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: true },
},
args,
whenTooLarge: (input) => input.fields?.length ? "Ask for fewer fields." : "Pass fields to read only part of it, for example [\"slides.objectId\", \"slides.pageElements(objectId,shape.text)\"].",
async run(input, http) {
const plain = input.commentsIncluded ? readPresentationAccessCheckRequest(input) : undefined;
return readPresentationResult(await readWithComments<RestPresentation>(http, readPresentationRequest(input), plain, COMMENTS_UNAVAILABLE), input);
+29 -10
View File
@@ -3,28 +3,44 @@ import type { z } from "zod";
import { GoogleToolError } from "./errors.js";
import type { GoogleHttp } from "./google-http.js";
export const FALLBACK_MAX_RESULT_BYTES = 250_000;
export interface ToolLimits {
maxResultBytes: number;
}
export const FALLBACK_TOOL_LIMITS: ToolLimits = { maxResultBytes: FALLBACK_MAX_RESULT_BYTES };
export interface RestToolSpec<Args> {
definition: Tool;
args: z.ZodType<Args>;
run(args: Args, http: GoogleHttp): Promise<CallToolResult>;
whenTooLarge?: string | ((args: Args) => string);
run(args: Args, http: GoogleHttp, limits: ToolLimits): Promise<CallToolResult>;
}
export interface RestTool {
definition: Tool;
call(rawArgs: unknown, http: GoogleHttp): Promise<CallToolResult>;
call(rawArgs: unknown, http: GoogleHttp, limits?: ToolLimits): Promise<CallToolResult>;
}
export function defineTool<Args>(spec: RestToolSpec<Args>): RestTool {
const readOnly = spec.definition.annotations?.readOnlyHint === true;
if (readOnly !== (spec.whenTooLarge !== undefined)) {
throw new Error(`${spec.definition.name}: a read tool must say how to ask for less, and a write tool must not`);
}
return {
definition: spec.definition,
async call(rawArgs, http) {
async call(rawArgs, http, limits = FALLBACK_TOOL_LIMITS) {
const parsed = spec.args.safeParse(rawArgs ?? {});
if (!parsed.success) {
const detail = parsed.error.issues.map((issue) => `${issue.path.join(".") || "arguments"}: ${issue.message}`).join("; ");
return { isError: true, content: [{ type: "text", text: `Invalid arguments for ${spec.definition.name}: ${detail}` }] };
}
try {
return await spec.run(parsed.data, http);
const result = await spec.run(parsed.data, http, limits);
if (!spec.whenTooLarge) return result;
const advice = typeof spec.whenTooLarge === "string" ? spec.whenTooLarge : spec.whenTooLarge(parsed.data);
return withinResultBudget(result, limits.maxResultBytes, advice);
} catch (error) {
if (error instanceof GoogleToolError) return toolErrorResult(error);
throw error;
@@ -37,17 +53,20 @@ export function structuredResult(value: Record<string, unknown>): CallToolResult
return { content: [{ type: "text", text: JSON.stringify(value) }], structuredContent: value };
}
export const MAX_RESULT_BYTES = 250_000;
export function serializedResultBytes(result: CallToolResult): number {
return Buffer.byteLength(JSON.stringify(result), "utf8");
}
export function boundedStructuredResult(value: Record<string, unknown>, maxBytes: number, tooLarge: (bytes: number) => string): CallToolResult {
const result = structuredResult(value);
export const byteCount = (bytes: number) => bytes.toLocaleString("en-US");
export function resultTooLargeMessage(bytes: number, maxBytes: number, advice: string): string {
return `The result is ${byteCount(bytes)} bytes, more than the ${byteCount(maxBytes)} bytes this tool can return, so it was not returned. ${advice}`;
}
export function withinResultBudget(result: CallToolResult, maxBytes: number, advice: string): CallToolResult {
const bytes = serializedResultBytes(result);
if (bytes > maxBytes) throw new GoogleToolError("incomplete", tooLarge(bytes));
return result;
if (bytes <= maxBytes) return result;
return toolErrorResult(new GoogleToolError("too_large", resultTooLargeMessage(bytes, maxBytes, advice)));
}
export function toolErrorResult(error: GoogleToolError): CallToolResult {
@@ -8,7 +8,7 @@ import { GMAIL_TOOLS } from "./gmail/index.js";
import { SLIDES_TOOLS } from "./slides/index.js";
import { createGoogleHttp, type GoogleFetch } from "./google-http.js";
import { SHEETS_TOOLS } from "./sheets/index.js";
import type { RestTool } from "./tool.js";
import { FALLBACK_MAX_RESULT_BYTES, type RestTool } from "./tool.js";
export const GOOGLE_WORKSPACE_REST_SERVICES = ["gmail", "calendar", "drive", "docs", "slides", "sheets"] as const;
export type GoogleWorkspaceRestService = (typeof GOOGLE_WORKSPACE_REST_SERVICES)[number];
@@ -38,12 +38,15 @@ export interface CallGoogleWorkspaceRestToolInput {
fetch?: GoogleFetch;
signal?: AbortSignal;
timeoutMs?: number;
maxResultBytes?: number;
}
export async function callGoogleWorkspaceRestTool(input: CallGoogleWorkspaceRestToolInput): Promise<CallToolResult> {
const tool = TOOLS[input.service].find((candidate) => candidate.definition.name === input.name);
if (!tool) throw new McpError(ErrorCode.InvalidParams, `Unknown tool: ${input.name}`);
const maxResultBytes = input.maxResultBytes ?? FALLBACK_MAX_RESULT_BYTES;
if (!Number.isSafeInteger(maxResultBytes) || maxResultBytes <= 0) throw new RangeError("maxResultBytes must be a positive integer");
if (!input.accessToken) throw new GoogleWorkspaceAuthError("unauthenticated");
const http = createGoogleHttp({ accessToken: input.accessToken, fetch: input.fetch, signal: input.signal, timeoutMs: input.timeoutMs });
return tool.call(input.arguments, http);
return tool.call(input.arguments, http, { maxResultBytes });
}
@@ -27,7 +27,7 @@ describe("callGoogleWorkspaceRest", () => {
const controller = new AbortController();
const fetch = vi.fn(async (_url: string, _init: RequestInit) => Response.json({ id: "draft1", message: { id: "m", threadId: "t" } }));
const outcome = await callGoogleWorkspaceRest({
service: "gmail", toolName: "create_draft", arguments: { subject: "Hi", body: "Hello" },
service: "gmail", toolName: "create_draft", arguments: { subject: "Hi", body: "Hello" }, maxResultBytes: 1_000_000,
credentialHeaders: { authorization: "Bearer token-1", "x-paperclip-run": "run-1" }, fetch, signal: controller.signal,
});
expect(outcome).toMatchObject({ ok: true, result: { structuredContent: { id: "draft1", subject: "Hi" } } });
@@ -41,7 +41,7 @@ describe("callGoogleWorkspaceRest", () => {
it("returns token problems as outcomes and leaves other failures as tool errors", async () => {
const answer = (status: number, body: unknown) => vi.fn(async () => Response.json(body, { status }));
const call = (fetch: ReturnType<typeof answer>, credentialHeaders: Record<string, string> = { Authorization: "Bearer t" }) =>
callGoogleWorkspaceRest({ service: "gmail", toolName: "get_message", arguments: { messageId: "m1" }, credentialHeaders, fetch });
callGoogleWorkspaceRest({ service: "gmail", toolName: "get_message", arguments: { messageId: "m1" }, maxResultBytes: 1_000_000, credentialHeaders, fetch });
await expect(call(answer(401, {}))).resolves.toEqual({ ok: false, authFailure: "unauthenticated" });
await expect(call(answer(200, {}), {})).resolves.toEqual({ ok: false, authFailure: "unauthenticated" });
await expect(call(answer(403, { error: { errors: [{ reason: "insufficientPermissions" }] } })))
@@ -49,8 +49,25 @@ describe("callGoogleWorkspaceRest", () => {
await expect(call(answer(404, { error: { code: 404 } }))).resolves.toMatchObject({ ok: true, result: { isError: true } });
});
it("passes the gateway's result budget to the REST tools, which enforce exactly that number", async () => {
const body = "a".repeat(30_000);
const message = { id: "m1", threadId: "t1", internalDate: "1763337600000",
payload: { mimeType: "text/plain", headers: [], body: { size: body.length, data: Buffer.from(body).toString("base64url") } } };
const call = (maxResultBytes: number) => callGoogleWorkspaceRest({
service: "gmail", toolName: "get_message", arguments: { messageId: "m1" }, maxResultBytes,
credentialHeaders: { Authorization: "Bearer t" }, fetch: vi.fn(async () => Response.json(message)),
});
const fits = await call(1_000_000);
expect(fits).toMatchObject({ ok: true, result: { structuredContent: { plaintextBody: body } } });
const bytes = Buffer.byteLength(JSON.stringify(fits.ok && fits.result), "utf8");
await expect(call(bytes)).resolves.toMatchObject({ ok: true, result: { structuredContent: { plaintextBody: body } } });
const over = await call(bytes - 1);
expect(over).toMatchObject({ ok: true, result: { isError: true } });
expect(JSON.stringify(over)).toContain(`more than the ${(bytes - 1).toLocaleString("en-US")} bytes this tool can return`);
});
it("refuses a service the REST tools do not implement", async () => {
await expect(callGoogleWorkspaceRest({ service: "chat", toolName: "send_message", arguments: {}, credentialHeaders: {}, fetch: vi.fn() }))
await expect(callGoogleWorkspaceRest({ service: "chat", toolName: "send_message", arguments: {}, maxResultBytes: 1_000_000, credentialHeaders: {}, fetch: vi.fn() }))
.rejects.toThrow("no Paperclip REST tools");
});
});
@@ -42,7 +42,7 @@ export function listGoogleWorkspaceRest(service: GoogleWorkspaceService): Tool[]
}
export async function callGoogleWorkspaceRest(
input: GoogleWorkspaceRestContext & { toolName: string; arguments: unknown },
input: GoogleWorkspaceRestContext & { toolName: string; arguments: unknown; maxResultBytes: number },
): Promise<GoogleWorkspaceRestCallOutcome> {
try {
const result = await callGoogleWorkspaceRestTool({
@@ -52,6 +52,7 @@ export async function callGoogleWorkspaceRest(
accessToken: bearerToken(input.credentialHeaders),
fetch: input.fetch,
signal: input.signal,
maxResultBytes: input.maxResultBytes,
});
return { ok: true, result };
} catch (error) {
+4
View File
@@ -344,6 +344,9 @@ const ACTION_REQUEST_EXECUTION_WAIT_MS = APPROVED_EXECUTION_TIMEOUT_MS + 5_000;
// grace must exceed the normal sign path (approval-snapshot fetch + interaction
// create) so a live create keeps its own row.
const MAX_REMOTE_MCP_RESPONSE_BYTES = 1_000_000;
const MCP_RESULT_FRAMING_BYTES = Buffer.byteLength(
`event: message\ndata: ${JSON.stringify({ jsonrpc: "2.0", id: `paperclip-tool-${randomUUID()}-version`, result: {} })}`, "utf8") - "{}".length;
export const GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES = MAX_REMOTE_MCP_RESPONSE_BYTES - MCP_RESULT_FRAMING_BYTES;
const ACTIVE_GATEWAY_RUN_STATUSES = new Set(["running"]);
// A tool listing decides access for every tool in the company. The decisions
// of one listing share a read cache, so this cap only limits how many of them
@@ -8127,6 +8130,7 @@ export function createToolGatewayService(
service: workspaceTarget.service,
toolName: workspaceTarget.leaf,
arguments: parameters,
maxResultBytes: GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES,
credentialHeaders,
signal,
fetch: (url, init) => options.remoteHttpRequest