feat(google-workspace): gateway sets the REST tool result size limit for all six services; update connector docs (#22050)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents read Gmail, Calendar, Drive, Docs, Sheets and Slides through
the built-in Google Workspace connection
> - Paperclip now has its own REST-backed Google tools in
`server/src/google-workspace-rest-tools/`. No service uses them yet
> - That folder guessed a result size limit of 250,000 bytes. It applied
the limit only to Drive, Docs, Sheets and Slides. Gmail and Calendar had
no limit
> - The guess was wrong in both directions. It refused documents that
Paperclip can deliver, and the folder should not know Paperclip's limits
> - This pull request makes the limit a number that the gateway gives.
The folder applies it to every tool in all six services through one
check. It also brings three Google planning docs up to date
> - The benefit is that a REST result is refused only when the gateway
would also refuse the same result from Google's own MCP server

## Linked Issues or Issue Description

Refs #22027, #22028, #22029, #22026

**Is your feature request related to a problem?**
The REST tools refused results over a size the folder chose itself
(250,000 bytes). Paperclip can deliver larger results: the runner saves
a large result as an artifact and gives the agent a pointer. Gmail and
Calendar results had no limit at all.

**Describe the solution you'd like**
The gateway gives the limit. The folder applies the given number to the
final serialized result of every tool, with one helper. A result over
the limit is an error that says how to ask for less. It is never cut
short.

**Describe alternatives you've considered**
A larger number chosen from the runner's 1 GiB capture limit. Google's
own transfer limit (20 MB per response) and memory use make that a poor
choice. Also, results would then differ between the REST path and
Google's MCP path.

### Limits a tool result passes on the way to an agent

| Point | Limit | What happens over it |
| --- | --- | --- |
| Google response read by `google-http.ts` | 20 MB
(`MAX_RESPONSE_BYTES`) | Tool error. Nothing returned |
| REST folder result (new) | the number the gateway passes: 999,885
bytes | Tool error with "read less" advice. The gateway returns
`tool_error` (502) |
| Gateway read of a remote MCP response (`MAX_REMOTE_MCP_RESPONSE_BYTES`
in `tool-gateway.ts`) | 1,000,000 bytes per JSON body or event-stream
message | `mcp_remote_response_too_large` (502). This is the "1,000,000
bytes are rejected" statement in `doc/prompt-injection-prevention.md`.
It applies to the Google MCP path only; the REST path has no HTTP read |
| Injection screening model check | 256 KiB model request | Not a
delivery limit. A larger result is checked by the built-in patterns only
|
| Gateway HTTP route (`POST /tool-gateway/.../tools/call`) | none in
Paperclip | `res.json(result)` |
| Runner (`captureRunnerToolResult`) | 500,000 bytes
(`RUNNER_TOOL_RESULT_MAX_BYTES`) | Saved as an artifact. The agent gets
a pointer and reads it with `read_tool_result` |
| Runner capture | 1 GiB (`RUNNER_API_RESPONSE_MAX_BYTES`) |
`SemanticToolOutcomeUnknownError` |

The gateway wraps the result as `{ content, data: { content,
structuredContent, ... } }`, so the payload occurs again. Measured: a
Gmail result of 676,459 bytes from the folder became 1,017,386 bytes
delivered (ratio 1.50). The ratio cannot be more than 2.

### The budget

`GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES` in `tool-gateway.ts` =
`MAX_REMOTE_MCP_RESPONSE_BYTES` less the JSON-RPC envelope and
event-stream framing (`event: message\ndata:
{"jsonrpc":"2.0","id":"paperclip-tool-<uuid>-version","result":...}`) =
**999,885 bytes**. A result that Google's MCP server could deliver
through the gateway is also delivered over REST, and the reverse. With
the wrapper, the largest delivered result is less than 2 MB. The runner
saves it as an artifact (far below 1 GiB). So the folder never refuses a
result that the runner can give as a pointer.

Practical limits at that budget (plain ASCII; escaped and non-ASCII
characters count more):

| Tool | Largest content |
| --- | --- |
| Gmail `get_message` | about 500 KB plain body, or about 250 KB each
for plain plus HTML |
| Calendar `get_event` | about 500 KB description |
| Drive `read_file_content` | about 500 KB text |
| Drive `download_file_content` | about 375 KB file bytes |
| Docs `read_doc` | about 240 KB document text |
| Slides `read_presentation` | about 250 KB text |
| Sheets `get_values` | about 500 KB cell text, at most 50,000 cells |

## What Changed

- `tool.ts`: `defineTool` takes `whenTooLarge` advice per read tool
(required for reads, refused for writes) and a `ToolLimits` argument. It
measures the final `CallToolResult` once (`withinResultBudget`) and
replaces an oversized result with a `too_large` tool error.
`FALLBACK_MAX_RESULT_BYTES` (250,000) applies only when a direct caller
gives no limit.
- `callGoogleWorkspaceRestTool` and `createGoogleWorkspaceRestServer`
take `maxResultBytes`. A value that is not a positive integer is refused
before any request.
- All 23 read tools state their advice: a smaller `pageSize` for list
tools, `messageFormat` or `view` for Gmail, a narrower range for Sheets,
`fields` for Slides. Write tools are not checked: a completed write is
never reported as an error, so an agent is not led to repeat it.
- Drive, Docs, Slides and Sheets no longer have their own byte checks
(`boundedStructuredResult`, `boundedResult`, `sizedResult`,
`MAX_CONTENT_BYTES` removed). Drive downloads stop reading at the given
limit. The Sheets 50,000-cell limit stays.
- Two Drive tool descriptions no longer state a fixed number.
- Adapter `google-workspace-rest-dispatch.ts`: `maxResultBytes` is a
required input and is passed through.
- `tool-gateway.ts`: new `GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES`,
passed on the REST branch.
- Folder README: new "Result size" section with the limits and costs;
old 250,000-byte text replaced.
- Docs: `doc/memory/google-connector-memory.md`, the production
checklist and the code-fix plan now list #22026 to #22029, the REST
status (36 of 36, none switched on), the November 17 decisions, the
known differences from Google's tools, open risks and next steps.

## Verification

- `cd server && npx vitest run src/google-workspace-rest-tools` passes
(53 files, 1,361 tests with the adapter test).
- New `result-budget.test.ts`: for Gmail (read, list), Calendar (list,
read), Drive, Docs, Slides and Sheets, a result just under a given
budget is returned and one just over is refused with the tool's advice.
Without a budget, the fallback applies; with a larger budget, the same
result is returned. Drive downloads use the given cap. A Gmail list
tells the caller to lower `pageSize`. A Gmail draft result over the
budget is still returned, not an error. `defineTool` refuses a read tool
without advice and a write tool with it.
- `npx vitest run src/services/google-workspace-rest-dispatch.test.ts`:
the adapter passes the gateway's number, and the folder applies exactly
that number.
- `npx vitest run src/__tests__/google-workspace-rest-transport.test.ts`
(24 tests): with Gmail on REST, a 330 KB email passes the gateway with
screening, is more than 500,000 bytes delivered, and
`captureRunnerToolResult` turns it into a `read_tool_result` pointer. A
520 KB email is refused with `tool_error` and the advice, after one
Google request.
- `npx tsc --noEmit -p server/tsconfig.json`: no errors.
- `pnpm verify --explain`: broader verification required (a doc under
`doc/` is a global CI input). CI supplies that coverage.
- Time per call: the check is one `JSON.stringify` of the result. About
5 µs for a 4 KB result, 18 µs for 42 KB, 0.24 ms for 700 KB. Drive,
Docs, Sheets and Slides had the same pass before, so they add nothing.
Gmail and Calendar add this one pass.

## Risks

- Gmail and Calendar results now have a limit. Direct callers that pass
no limit get the 250,000-byte fallback. The gateway always passes its
number.
- Drive, Docs, Slides and Sheets accept results up to about 4 times
larger than before through the gateway. Large results reach agents as
artifact pointers.
- Error text for oversized results changed to one shared form.
- No service is switched to REST, so users see no change today.

## Model Used

Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code with
tool use (shell, file edits, GitHub CLI) and extended reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed 2026-11-17 13:05:13 -06:00
1 parent 98ff5e002a
commit 56023fe80f
47 files changed
+598 -216

No files matched your search

+72 -24
View File
@@ -12,7 +12,7 @@ Related: [production checklist](../plans/2026-11-17-google-workspace-production-
[code-fix plan](../plans/2026-11-17-google-workspace-code-fixes.md),
[connector doc](../connections/GOOGLE-WORKSPACE.md).
Last updated: November 17, 2026 (late evening).
Last updated: November 17, 2026 (night).
## Decisions
@@ -32,6 +32,14 @@ Last updated: November 17, 2026 (late evening).
| Nov 17 | **`delete_event` never deletes a whole recurring series in this release** | A series ID is refused; the agent must pass one occurrence's ID. |
| Nov 17 | **A health probe may only use a request the granted scopes permit** | Each service lists cheap reads with the scopes that allow each; the probe uses the first one the grant allows. |
| Nov 17 | **Read & write Calendar also requests free/busy** | See "Scopes". |
| Nov 17 | **Tool labels must be truthful, even where Google's differ** | Every update tool is marked destructive: Calendar `update_event`, Docs `update_doc`, Slides `update_presentation`, Sheets `update_spreadsheet`. Google marks them non-destructive. Approval and confirmation follow our label. |
| Nov 17 | **Docs and Slides edits go through a closed list of request types** | Each request type is allowed or refused by name. A type not on the list, including one Google adds later, is refused and nothing is sent. |
| Nov 17 | **No tool may reach a file other than the one named** | Request types that read another file (smart-chip links, linked Sheets charts, imported slides, Drive audio and video) are refused. Image links on `drive.google.com` or `docs.google.com` are refused. Drive tools use only the file ID the caller passed. |
| Nov 17 | **Docs and Slides edits require the revision that was read** | `requiredRevisionId` is mandatory. If the file changed since the read, nothing is changed and the agent is told to read it again. |
| Nov 17 | **Sheet tabs and ranges may be deleted inside a spreadsheet** | Through `update_spreadsheet`, because the approver sees each request type (`deleteSheet`, `deleteDimension`, and so on). No tool deletes, creates or shares a whole spreadsheet. |
| Nov 17 | **Deleting a whole recurring series is refused** | Same rule as the `delete_event` decision above. |
| Nov 17 | **A service is switched to REST only after a live test on the staging tester** | One service at a time. |
| Nov 17 | **The gateway sets the REST result size limit** | The REST tools enforce the number the gateway passes: the gateway's MCP response limit less framing, 999,885 bytes. A result Google's MCP server could deliver is also delivered over REST. See the folder README's "Result size". |
## What has shipped
@@ -46,8 +54,12 @@ Last updated: November 17, 2026 (late evening).
| [#22017](https://github.com/paperclipai/paperclip/pull/22017) | Nine Calendar tools in the REST tool set (Calendar 9 of 9). | `c41ca4a992` |
| [#22016](https://github.com/paperclipai/paperclip/pull/22016) | Per-service transport switch (`GOOGLE_WORKSPACE_TOOL_TRANSPORT`) and in-process dispatch to the REST tools. Every service is still on Google's MCP. Discovery and health run a scope-aware probe for a service on REST. | `c652c813bc` |
| [#22014](https://github.com/paperclipai/paperclip/pull/22014) | The remaining six Gmail read tools in the REST tool set (Gmail 9 of 9). | `1d3b4201e3` |
| [#22026](https://github.com/paperclipai/paperclip/pull/22026) | Read & write Calendar also requests `calendar.events.freebusy`, and the planning docs were brought up to date. | `7f1f0eef67` |
| [#22027](https://github.com/paperclipai/paperclip/pull/22027) | Six Sheets tools in the REST tool set (Sheets 6 of 6). | `c366a606b1` |
| [#22029](https://github.com/paperclipai/paperclip/pull/22029) | Docs and Slides tools in the REST tool set (Docs 2 of 2, Slides 2 of 2). Also marks Calendar `update_event` destructive. | `f61b919701` |
| [#22028](https://github.com/paperclipai/paperclip/pull/22028) | Eight Drive tools in the REST tool set (Drive 8 of 8). Also limits the Gmail host to its API path prefix (before, any path was accepted there) and requires a path prefix for every allowed host. | `ceaf574421` |
In progress: the Drive, Docs and Slides, and Sheets REST tools.
The REST tool set now covers all 36 actions. No service is switched to it.
### REST tools status
@@ -55,16 +67,35 @@ In progress: the Drive, Docs and Slides, and Sheets REST tools.
| --- | --- | --- |
| Gmail | 9 of 9 | No |
| Calendar | 9 of 9 | No |
| Drive | 0 of 8 | No |
| Docs | 0 of 2 | No |
| Sheets | 0 of 6 | No |
| Slides | 0 of 2 | No |
| Drive | 8 of 8 | No |
| Docs | 2 of 2 | No |
| Sheets | 6 of 6 | No |
| Slides | 2 of 2 | No |
"Switched on" means the service is `paperclip_rest` in
`GOOGLE_WORKSPACE_TOOL_TRANSPORT`. Today every service is `google_mcp`.
### Known differences from Google's tools
From the folder README (`server/src/google-workspace-rest-tools/README.md`).
| Service | Difference |
| --- | --- |
| Gmail | A reply draft is threaded but does not quote the original message. |
| Gmail | `get_draft`, `list_threads` and `search_messages` are not in Google's reference; their inputs and results are inferred from sibling tools. |
| Docs | The format of `read_doc`'s `content` has not been compared with an answer from Google's server. |
| Sheets | Whether Google's `get_values` returns formatted values and its `update_values` writes raw values is inferred, not verified. |
| Sheets | `get_spreadsheet` refuses `commentsIncluded: true`, and `update_spreadsheet` refuses comment requests: comments are not in the Sheets API. |
| Drive | `read_file_content` cannot extract text from PDF, Office or image files; they return `textFormattingNotSupported`. |
| Drive | Whether `copy_file` under `drive.file` can copy a file the app never created or opened is not verified. |
## Still open
- **Images by public web address in Docs and Slides.** An edit may insert an
image from any public link, and Google then fetches that address. The
address is chosen by the agent, so it can carry data out (a leak path).
Allowed today. Recommendation: refuse it before Docs or Slides is switched
to REST.
- **What we send to the reviewers.** Not decided: the new staging tester or
something else. Whatever it is must use the separate Google app (below).
- **How the staging tester reaches the separate Google app.** The broker
@@ -80,13 +111,33 @@ In progress: the Drive, Docs and Slides, and Sheets REST tools.
- **Three Gmail tools are inferred.** Google's MCP reference does not list
`get_draft`, `list_threads` and `search_messages`; their schemas copy the
sibling tools (#22014).
- **Privacy policy.** paperclipai/paperclip-website PR 197 is a draft waiting
on Dotta's sign-off. It does not yet mention the OpenRouter model used by
the optional injection-screening check; it must (see "Privacy policy").
- **Privacy policy.** paperclipai/paperclip-website PR 197 merged on
November 17 and the page shows "Last updated: November 17, 2026". Its
no-training promise is narrower than before: it now covers only the model
providers Paperclip selects and configures. That needs legal review; it is
not required for Google approval. Whether it covers the OpenRouter model
used by the optional injection-screening check is not confirmed (see
"Privacy policy").
- **Lockfile on master.** `pnpm-lock.yaml` is behind `server/package.json`
(`posthog-node` is missing), so a frozen install fails. Checked November 17
at `ceaf574421`.
- **The "no review correspondence" rule.** Narrow it or keep it (see "Google
review history").
- Which security lab to use, and the budget.
## Next steps
1. Cloud broker changes: limit the unified profile to the 14; cap extra
reported scopes instead of failing; stop offering Chat, People and Search
on the verified client; turn on revoke support; add a per-workspace Google
client override.
2. Create the staging tester (`googletester.staging.paperclip.app`).
3. Live acceptance on the staging tester.
4. Switch services to REST one at a time, each after its live test.
5. Revoke at Google when the last connection is removed.
6. Security scans.
7. Run the same data cleanup when a task or a company is deleted.
## Environments
| Name | Address | Notes |
@@ -261,21 +312,18 @@ review correspondence in a public repository. We kept the summary.
published at <https://paperclip.ing/privacy>. Last updated July 23, 2026.
- It already has what Google looks for first: a Google user data section
(4.1) with the Limited Use statement and links to Google's policy.
- The draft edits are in paperclipai/paperclip-website PR 197 (draft, awaiting
Dotta's sign-off on the wording). It must also:
- cover the OpenRouter model used by the optional injection-screening check;
- match the picker's statement that results stay in task and run history
until deleted.
- Gaps found November 17, to be fixed by that PR:
- It does not say that Google content is sent to AI model providers to carry
out tasks. It only names "cloud hosting and infrastructure providers".
- It says connected-account data is kept only while the connection is active.
In the product, email and file content that an agent read stays in task and
run history after disconnect.
- It says "read or send access to Gmail". We create drafts and do not send.
- It names `connect.paperclip.ing` as the sign-in path; check that this
matches the current flow.
- It gives no steps or timeline for deleting data.
- The edits are in paperclipai/paperclip-website PR 197, merged November 17.
The published page now says that Google content is sent to the AI model
provider configured for an agent, that agents create Gmail drafts and do not
send, that content read by agents stays in tasks and runs until deleted, and
how to delete data (full deletion within 30 days of a request).
- Still to check:
- The no-training promise now covers only providers Paperclip selects and
configures. Legal review needed.
- Whether it covers the OpenRouter model used by the optional
injection-screening check.
- It names `my.paperclip.app` as the Google connection service on Paperclip
Cloud; check that this matches the current flow.
## Other things to remember
@@ -6,8 +6,9 @@ land. It covers section 2 of the
Based on a read of master at `56654adecd`. Items marked **confirmed** were seen
in the code. Items marked **to check** are suspected gaps that need a look
before any change. Fixes 1 to 4 and 6 landed on November 17. Work on fixes 9
and 10 has started. Fixes 5, 7 and 8 are not started.
before any change. Fixes 1 to 4 and 6 landed on November 17. Fix 9 has every
tool built but no service switched on. Fix 10 is done in the app. Fixes 5, 7
and 8 are not started.
Each fix is its own pull request with a failing test first. For the scope
change in fix 2, the deploy order is app first, then broker (see fix 10).
@@ -24,7 +25,7 @@ change in fix 2, the deploy order is app first, then broker (see fix 10).
| 6 | Retention and deletion of Google content | Audit, then unknown | Yes | Done with limits, [#22010](https://github.com/paperclipai/paperclip/pull/22010) |
| 7 | Security scan findings | Unknown | Yes (assessment) | Not started |
| 8 | Remove the experimental gate | Small | No, release step | Not started |
| 9 | REST-backed tool set | Large | Yes | Gmail and Calendar done, switch merged, none switched on; Drive, Docs, Sheets, Slides in progress |
| 9 | REST-backed tool set | Large | Yes | All 36 tools and the switch merged; none switched on |
| 10 | Scope cleanup across code, Cloud and Console | Medium | Yes | App done; broker and Console open |
## 1. Data-use disclosure in the picker
@@ -214,21 +215,34 @@ deleted cleanly. Part 1 is
folder and Gmail `get_message`, `search_threads` and `create_draft`. Its README
records the dispatch trace and the recommended in-process seam.
Status (Nov 17, late evening):
Status (Nov 17, night):
| Service | Implemented | Switched on | PRs |
| --- | --- | --- | --- |
| Gmail | 9 of 9 | No | [#22000](https://github.com/paperclipai/paperclip/pull/22000), [#22014](https://github.com/paperclipai/paperclip/pull/22014) |
| Calendar | 9 of 9 | No | [#22017](https://github.com/paperclipai/paperclip/pull/22017) |
| Drive | 0 of 8 | No | In progress |
| Docs | 0 of 2 | No | In progress |
| Sheets | 0 of 6 | No | In progress |
| Slides | 0 of 2 | No | In progress |
| Drive | 8 of 8 | No | [#22028](https://github.com/paperclipai/paperclip/pull/22028) |
| Docs | 2 of 2 | No | [#22029](https://github.com/paperclipai/paperclip/pull/22029) |
| Sheets | 6 of 6 | No | [#22027](https://github.com/paperclipai/paperclip/pull/22027) |
| Slides | 2 of 2 | No | [#22029](https://github.com/paperclipai/paperclip/pull/22029) |
The per-service switch and in-process dispatch landed in
[#22016](https://github.com/paperclipai/paperclip/pull/22016). Every service is
still on Google's MCP. The gateway calls the REST tools in process, as the
trace in #22000 recommended.
trace in #22000 recommended. The gateway also sets the size limit for REST
results (999,885 bytes, matching what it accepts from Google's MCP servers);
see "Result size" in the folder README.
Remaining work:
- Refuse inserting an image by public web address in Docs and Slides before
they are switched on (Google fetches an address the agent chose).
- Create the staging tester and run live acceptance there.
- Switch services to REST one at a time, each after its live test.
- Record a `tools/list` and sample answers from Google's servers for the
contract test, and compare `read_doc` and `read_presentation` content.
- Check `copy_file` under `drive.file` on a file the app never opened.
- Removal test (step 4 below).
### The seam
@@ -302,7 +316,7 @@ server/src/google-workspace-rest-tools/
2. Folder skeleton, `google-http.ts`, and Gmail `get_message`,
`search_threads` and `create_draft` (#22000, merged). Then the switch, end to
end on the staging tester. The demo depends on Gmail.
3. Rest of Gmail, then Calendar, Drive, Docs, Sheets, Slides.
3. Rest of Gmail, then Calendar, Drive, Docs, Sheets, Slides (done Nov 17).
4. Contract and removal tests.
Tools to build (36): Gmail 9, Drive 8, Docs 2, Sheets 6, Slides 2, Calendar 9.
@@ -311,7 +325,8 @@ Tools to build (36): Gmail 9, Drive 8, Docs 2, Sheets 6, Slides 2, Calendar 9.
- Whether a Google client marked for AI agents may call the REST APIs.
- Each Cloud project needs the six REST APIs enabled.
- Speed: this sits in the tool-call path, so measure added time per call.
- Speed: this sits in the tool-call path. Measured: the adapter adds about
15 µs per call; the result size check adds about 5 µs for a 4 KB result.
### Lessons
@@ -79,16 +79,18 @@ Decisions and the details behind them are kept in the
is kept"). Users can now delete it per connection. Content agents wrote
into tasks stays, and task or company deletion does not yet run the same
cleanup.
- [ ] REST-backed tool set for all 36 actions, in its own folder inside the
server, with the same tool names and shapes as Google's MCP tools.
18 of 36 so far: Gmail 9 of 9
- [x] REST-backed tool set for all 36 actions exists but is not switched on
(Nov 17): Gmail 9
([#22000](https://github.com/paperclipai/paperclip/pull/22000),
[#22014](https://github.com/paperclipai/paperclip/pull/22014)) and
Calendar 9 of 9
([#22017](https://github.com/paperclipai/paperclip/pull/22017)). The
[#22014](https://github.com/paperclipai/paperclip/pull/22014)),
Calendar 9 ([#22017](https://github.com/paperclipai/paperclip/pull/22017)),
Sheets 6 ([#22027](https://github.com/paperclipai/paperclip/pull/22027)),
Docs 2 and Slides 2
([#22029](https://github.com/paperclipai/paperclip/pull/22029)), Drive 8
([#22028](https://github.com/paperclipai/paperclip/pull/22028)). The
per-service switch exists
([#22016](https://github.com/paperclipai/paperclip/pull/22016)); no
service is switched on yet.
([#22016](https://github.com/paperclipai/paperclip/pull/22016)).
- [ ] Each service switched to REST after a live test on the staging tester.
- [ ] Scopes identical in the picker, the server, the Cloud broker and the
Google Console.
- [ ] `googletester.staging.paperclip.app` created and wired to the new Google
@@ -101,12 +103,16 @@ The policy is in the `paperclip-website` repository
- [x] Policy has a Google user data section with the Limited Use statement.
- [x] Policy states that Google data is not used to train general models.
- [ ] Policy says Google content is sent to AI model providers to carry out
tasks, and names them or the category.
- [ ] Policy's retention wording matches the product (content read by agents
stays in task and run history after disconnect), or the product changes.
- [ ] Policy says Gmail drafts, not "send".
- [ ] Policy gives the steps and timeline for deleting data.
- [x] Policy says Google content is sent to AI model providers to carry out
tasks, and names them or the category (Nov 17, [paperclip-website PR 197](https://github.com/paperclipai/paperclip-website/pull/197)).
- [x] Policy's retention wording matches the product (content read by agents
stays in task and run history after disconnect), or the product changes
(Nov 17, [paperclip-website PR 197](https://github.com/paperclipai/paperclip-website/pull/197)).
- [x] Policy says Gmail drafts, not "send" (Nov 17, [paperclip-website PR 197](https://github.com/paperclipai/paperclip-website/pull/197)).
- [x] Policy gives the steps and timeline for deleting data (Nov 17, [paperclip-website PR 197](https://github.com/paperclipai/paperclip-website/pull/197)).
- [ ] Legal review of the narrowed no-training wording in [paperclip-website PR 197](https://github.com/paperclipai/paperclip-website/pull/197) (it now
covers only providers Paperclip selects and configures). Not required
for Google approval.
- [ ] Policy's description of the sign-in path matches the current flow.
- [ ] Model-provider terms and settings checked against that statement.
- [ ] Data-flow inventory written: Google, broker, server, runner, model
@@ -10,8 +10,9 @@ import {
type GoogleWorkspaceService, type GoogleWorkspaceToolTransport,
} from "@paperclipai/shared";
import * as restDispatch from "../services/google-workspace-rest-dispatch.js";
import { captureRunnerToolResult, RUNNER_TOOL_RESULT_MAX_BYTES } from "../services/native-runtime/runner-tool-results.js";
import { toolAccessService } from "../services/tool-access.js";
import { createToolGatewayService } from "../services/tool-gateway.js";
import { createToolGatewayService, GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES } from "../services/tool-gateway.js";
import { instanceSettingsService } from "../services/instance-settings.js";
import { initializeRunIdentity } from "../services/run-identity.js";
import type { PaperclipCloudConnector } from "../services/paperclip-cloud-connector.js";
@@ -302,6 +303,42 @@ const support = await getEmbeddedPostgresTestSupport();
expect(JSON.stringify(invocation)).not.toContain("ignore all previous instructions");
});
it("delivers a large email that Google's MCP path could also deliver, as a saved result the runner points to", async () => {
useTransport("gmail", "paperclip_rest");
const f = await connectWorkspace();
const body = `${BENIGN_BODY}\n`.repeat(Math.ceil(330_000 / (BENIGN_BODY.length + 1)));
f.setMessageBody(body);
const result = await f.call("gmail__get_message", { messageId: "msg1" });
expect(result.status).toBe("completed");
const outcome = await vi.mocked(restDispatch.callGoogleWorkspaceRest).mock.results[0]!.value;
const folderBytes = Buffer.byteLength(JSON.stringify(outcome.result), "utf8");
const deliveredBytes = Buffer.byteLength(JSON.stringify(result.result), "utf8");
expect(folderBytes).toBeLessThanOrEqual(GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES);
expect(deliveredBytes).toBeGreaterThan(RUNNER_TOOL_RESULT_MAX_BYTES);
expect(deliveredBytes).toBeLessThan(2 * folderBytes);
const saved: Buffer[] = [];
const pointer = await captureRunnerToolResult(result.result, {
saveResponse: vi.fn(async (bytes: Buffer) => { saved.push(bytes); return { artifactId: randomUUID() }; }),
} as unknown as Parameters<typeof captureRunnerToolResult>[1]);
expect(pointer).toMatchObject({ readWith: { tool: "read_tool_result" } });
expect(saved).toHaveLength(1);
expect(JSON.parse(saved[0]!.toString("utf8"))).toEqual(result.result);
expect((result.result as { data: { structuredContent: { plaintextBody: string } } }).data.structuredContent.plaintextBody).toBe(body);
});
it("refuses an email larger than the gateway's budget with advice to read less, without contacting Google again", async () => {
useTransport("gmail", "paperclip_rest");
const f = await connectWorkspace();
f.setMessageBody(`${BENIGN_BODY}\n`.repeat(Math.ceil(520_000 / (BENIGN_BODY.length + 1))));
f.gmailRequests.length = 0;
await expect(f.call("gmail__get_message", { messageId: "msg1" })).rejects.toMatchObject({
status: 502, reasonCode: "tool_error",
message: expect.stringMatching(new RegExp(`more than the ${GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES.toLocaleString("en-US")} bytes this tool can return, so it was not returned\\. Read it with messageFormat METADATA_ONLY or MINIMAL`)),
});
expect(f.gmailRequests).toHaveLength(1);
expect(vi.mocked(restDispatch.callGoogleWorkspaceRest).mock.calls[0]![0].maxResultBytes).toBe(GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES);
});
it("cancels the Google request when the call's deadline passes", async () => {
useTransport("gmail", "paperclip_rest");
const f = await connectWorkspace();
@@ -121,7 +121,8 @@ plain constant: no environment variable, instance setting or database row.
- **Adapter.** `server/src/services/google-workspace-rest-dispatch.ts` is the
only server file that imports this folder. It takes the gateway's resolved
credential headers (it keeps only the bearer token), the call's cancellation
signal and the gateway's egress-guarded fetch, and returns either the MCP
signal, the gateway's egress-guarded fetch and the gateway's result limit
(see "Result size"), and returns either the MCP
result or a token problem (`unauthenticated` or `insufficient_scope`).
- **Calls.** `executeRemoteHttpTool()` in `server/src/services/tool-gateway.ts`.
The REST branch runs after everything the MCP path runs first: the
@@ -176,6 +177,56 @@ Measured with a stubbed fetch, the adapter adds about 15 µs per call. A full
`gmail__get_message` through `executeTool` (policy, records, screening) took a
median of 18.2 ms on either transport.
### Result size
The caller sets the size limit. `callGoogleWorkspaceRestTool` takes
`maxResultBytes`, and every tool in all six services checks it the same way:
`defineTool` in `tool.ts` measures the final serialized `CallToolResult` once
(`withinResultBudget`) after the tool has built it. A result over the limit is
replaced by a tool error that gives its size and the limit, says it was not
returned, and says how to ask for less. Each tool states that advice in
`whenTooLarge`: a smaller `pageSize` for list tools, a lighter
`messageFormat` or view for Gmail, a narrower range for Sheets, `fields` for
Slides. Nothing is ever cut short. Write tools are not checked: their result
comes after the change was made, and an error then could lead the agent to
make the change twice. A write result echoes its request or Google's reply
to it, so its size follows the request. `defineTool` refuses a read tool without advice
and a write tool with it. Drive downloads also stop reading at the
limit, since larger content could never fit.
`FALLBACK_MAX_RESULT_BYTES` (250,000) in `tool.ts` applies only when a direct
caller passes no limit (tests, or `createGoogleWorkspaceRestServer` without the
option). This folder does not know Paperclip's limits.
The gateway passes `GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES` from
`server/src/services/tool-gateway.ts`: the gateway's limit on a response from a
remote MCP server (`MAX_REMOTE_MCP_RESPONSE_BYTES`, 1,000,000 bytes) less the
JSON-RPC and event-stream framing around the result, so 999,885 bytes. A
result Google's own MCP server could deliver through the gateway is also
delivered over REST, and the reverse. The gateway wraps the result for the
agent and repeats the payload, so the delivered result is up to twice as large
(measured 1.50 times for a plain-text email). The agent runner saves a
delivered result over 500,000 bytes as an artifact and gives the agent a
pointer it reads with `read_tool_result`, so a result near the limit is still
delivered in full.
Practical limits at that budget, measured with plain ASCII text (characters
that JSON escapes, and non-ASCII characters, count more):
| Service | Largest content one call returns |
| --- | --- |
| Gmail `get_message` | about 500 KB of plain-text body, or about 250 KB each when the message has both plain and HTML bodies |
| Gmail lists | the whole page must fit; lower `pageSize` |
| Calendar `get_event` | about 500 KB of description |
| Drive `read_file_content` | about 500 KB of text |
| Drive `download_file_content` | about 375 KB of file bytes (base64 adds a third) |
| Docs `read_doc` | about 240 KB of document text (the text and the edit structure both carry it) |
| Slides `read_presentation` | about 250 KB of slide text; use `fields` for more |
| Sheets `get_values` | about 500 KB of cell text, and at most 50,000 cells |
The check costs one `JSON.stringify` of the result: about 5 µs for a 4 KB
result and 0.24 ms for a 700 KB one.
### Flipping a service
1. Implement every tool in that service's allowlist (`toolScopes` in the
@@ -564,15 +615,12 @@ type of the returned bytes (the export format for Google files).
true`, the field Google's result has for this case. Text that is not valid
UTF-8 is also not supported.
**Size cap.** Every Drive read bounds its final serialized result (the whole
`CallToolResult`, which carries the payload twice: as `structuredContent` and
as its JSON text) to 250,000 bytes (the folder's shared `MAX_RESULT_BYTES`,
checked by `boundedStructuredResult`, both in `tool.ts`). That is about 90 KB of file
bytes for `download_file_content` (base64 adds a third) and about 120 KB of
plain text for `read_file_content`. Content is also never read past 250,000
bytes (`MAX_CONTENT_BYTES`), since more could never fit: a stored file whose
metadata size is larger is refused before any download, and exports and
revisions are read through `GoogleHttp.download`, which stops at the cap. Both
**Size cap.** Every Drive tool is bound by the caller's result limit (see
"Result size"). The result carries the payload twice, as `structuredContent`
and as its JSON text. Content is also never read past the limit, since more
could never fit: a stored file whose metadata size is larger is refused before
any download, and exports and revisions are read through
`GoogleHttp.download`, which stops at the limit. Both
limits are far under Google's 10 MB export limit
(<https://developers.google.com/workspace/drive/api/reference/rest/v3/files/export>)
and the 20 MB `MAX_RESPONSE_BYTES`. A refused result is an error that says
@@ -752,15 +800,11 @@ Formatting other than headings, lists and links (bold, colors, fonts, sizes)
is not in `text`. Checklist items render as plain list items: the Docs API
does not say whether a box is ticked.
A result over 250,000 bytes is refused with an error that gives the size and
returns nothing; it is never cut short. It is the folder's one limit,
`MAX_RESULT_BYTES` in `tool.ts`, checked by `boundedStructuredResult` (which
`sizedResult` in `documents-shared.ts` calls) on the final serialized MCP
result, which carries the content twice (as
JSON text and as structured content), so the content itself is limited to
about 125,000 bytes. Paperclip delivers tool results over 500,000 bytes as a
pointer instead of content; the lower limit keeps every read inline with
room to spare. To read a
A result over the caller's limit (see "Result size") is refused with an error
that gives the size and returns nothing; it is never cut short. The final
serialized MCP result carries the content twice (as JSON text and as
structured content), and `read_doc` also repeats each paragraph's text in its
edit structure. To read a
large presentation in parts, pass `fields`, for example
`["slides(objectId,pageElements(objectId,shape.text))"]`. `read_doc` has no
way to read part of a document, since Google's tool has no such input and
@@ -943,15 +987,9 @@ made" warning.
Reads count the cells Google returns (`values` rows, or `rowData` values in
grid data); writes count the cells sent (`values`/`formulas`, or the rows of
`updateCells` and `appendCells`).
- A read's whole tool result is at most **250,000 bytes** of UTF-8 JSON
(`MAX_RESULT_BYTES`, checked by `boundedStructuredResult` in `tool.ts`). The
measure is this folder's own result, which carries the value twice (as
`structuredContent` and as JSON text, since MCP clients read either). This
is a conservative default, not a measured fit: how a caller wraps and
delivers the result, and its byte budget, are the caller's business and are
not modeled here. The caller, which knows its own delivery budget, is
expected to supply the exact limit in a follow-up that applies one rule to
every service in this folder.
- A tool's whole result is bound by the caller's limit (see "Result size"),
measured on this folder's own result, which carries the value twice (as
`structuredContent` and as JSON text, since MCP clients read either).
A write request body is at most 1,000,000 characters.
- Over a limit, a read returns an error that starts with the size and asks for
a narrower range, fewer or narrower `ranges`, or fewer `fields`, to read in
@@ -1045,7 +1083,8 @@ says `false`.
2. Delete this folder, `server/src/google-workspace-rest-tools/`.
3. Delete `server/src/services/google-workspace-rest-dispatch.ts` and its
test, and the two call sites: the `paperclip_rest` branch in
`executeRemoteHttpTool` (`server/src/services/tool-gateway.ts`) and
`executeRemoteHttpTool` (`server/src/services/tool-gateway.ts`), with
`GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES` there, and
`googleWorkspaceRestTools` in
`server/src/services/tool-access/catalog/discovery.ts`. Delete
`server/src/__tests__/google-workspace-rest-transport.test.ts`.
@@ -27,6 +27,7 @@ export const getEvent = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "This tool cannot return part of an event.",
async run(input, http) {
return getEventResult(await http.send<RestEvent>(getEventRequest(input.calendarId, input.eventId)));
},
@@ -72,6 +72,7 @@ export const listCalendars = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Ask for fewer calendars with a smaller pageSize.",
async run(input, http) {
return listCalendarsResult(await http.send<RestCalendarList>(listCalendarsRequest(input)));
},
@@ -201,6 +201,7 @@ export const listEvents = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Ask for fewer events with a smaller pageSize or a narrower time range.",
async run(input, http) {
assertTimeZone(input.timeZone);
const page = decodePageToken(input.pageToken, input.orderBy);
@@ -58,6 +58,7 @@ export const searchEvents = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Ask for fewer events with a smaller pageSize.",
async run(input, http) {
return searchEventsResult(await http.send<RestEventList>(searchEventsRequest(input)));
},
@@ -301,6 +301,7 @@ export const suggestTime = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Ask for fewer suggestions with a smaller preferences.pageSize.",
async run(input, http) {
assertTimeZone(input.timeZone);
parseTimeInput(input.startTime, "startTime");
@@ -1,5 +1,5 @@
import { describe, expect, it } from "vitest";
import { MAX_RESULT_BYTES } from "../tool.js";
import { FALLBACK_MAX_RESULT_BYTES } from "../tool.js";
import { fakeGoogle, json } from "../fixtures/fake-google.js";
import { createGoogleHttp } from "../google-http.js";
import { MULTI_TAB_DOCUMENT, MULTI_TAB_TEXT, oneTab, para, run, text } from "./fixtures/docs-rest.js";
@@ -104,22 +104,22 @@ describe("read_doc tool", () => {
it("measures the limit on the whole result, which carries the content twice", async () => {
const under = (count: number) => oneTab(Array.from({ length: count }, () => text("z".repeat(1_000))));
const perParagraph = 2 * 2 * 1_000;
const fits = Math.floor(MAX_RESULT_BYTES / perParagraph) - 10;
const fits = Math.floor(FALLBACK_MAX_RESULT_BYTES / perParagraph) - 10;
const http = (doc: unknown) => createGoogleHttp({ accessToken: "t", fetch: fakeGoogle(() => json(doc)).fetch });
const ok = await readDoc.call({ documentId: "d" }, http(under(fits)));
expect(ok.isError).toBeUndefined();
expect(Buffer.byteLength(JSON.stringify(ok), "utf8")).toBeLessThanOrEqual(MAX_RESULT_BYTES);
expect(Buffer.byteLength(JSON.stringify(ok.structuredContent), "utf8")).toBeLessThan(MAX_RESULT_BYTES / 2 + 1);
expect(Buffer.byteLength(JSON.stringify(ok), "utf8")).toBeLessThanOrEqual(FALLBACK_MAX_RESULT_BYTES);
expect(Buffer.byteLength(JSON.stringify(ok.structuredContent), "utf8")).toBeLessThan(FALLBACK_MAX_RESULT_BYTES / 2 + 1);
expect((await readDoc.call({ documentId: "d" }, http(under(fits + 40)))).isError).toBe(true);
});
it("refuses a result over the size limit with a clear error instead of cutting it short", async () => {
const paragraph = "x".repeat(10_000);
const big = oneTab(Array.from({ length: Math.ceil(MAX_RESULT_BYTES / 20_000) + 5 }, () => text(paragraph)));
const big = oneTab(Array.from({ length: Math.ceil(FALLBACK_MAX_RESULT_BYTES / 20_000) + 5 }, () => text(paragraph)));
const { fetch } = fakeGoogle(() => json(big));
const result = await readDoc.call({ documentId: "big" }, createGoogleHttp({ accessToken: "t", fetch }));
expect(result.isError).toBe(true);
expect((result.content[0] as { text: string }).text).toMatch(/^This document is too large to return in one result \(\d+ KB; the limit is 250 KB\)\. Nothing was returned\./);
expect((result.content[0] as { text: string }).text).toMatch(/^The result is [\d,]+ bytes, more than the 250,000 bytes this tool can return, so it was not returned\. This tool cannot read part of a document\./);
expect(result.structuredContent).toBeUndefined();
});
});
@@ -1,9 +1,9 @@
import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { fileIdArg, sizedResult } from "../documents-shared.js";
import { fileIdArg } from "../documents-shared.js";
import { GoogleToolError } from "../errors.js";
import type { GoogleHttp, GoogleRequest } from "../google-http.js";
import { defineTool } from "../tool.js";
import { defineTool, structuredResult } from "../tool.js";
import { DOCS_HOST, documentPath, type RestDocument } from "./document.js";
import { renderDocument, type SuggestionsView } from "./render.js";
@@ -38,7 +38,7 @@ export async function readDocument(http: GoogleHttp, input: ReadDocArgs): Promis
export function readDocResult(document: RestDocument, input: ReadDocArgs, view: SuggestionsView = "SUGGESTIONS_INLINE"): CallToolResult {
const content = renderDocument(document, { commentsRequested: input.commentsIncluded === true, suggestionsView: view });
return sizedResult({ content }, "document", "Google Docs cannot return part of a document through this tool.");
return structuredResult({ content });
}
export const readDoc = defineTool({
@@ -68,6 +68,7 @@ export const readDoc = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: true },
},
args,
whenTooLarge: "This tool cannot read part of a document. Read it without commentsIncluded if that was set, or read only its text with read_file_content.",
async run(input, http) {
const { document, view } = await readDocument(http, input);
return readDocResult(document, input, view);
@@ -1,8 +1,6 @@
import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { GoogleToolError } from "./errors.js";
import type { GoogleHttp, GoogleRequest } from "./google-http.js";
import { boundedStructuredResult, MAX_RESULT_BYTES } from "./tool.js";
export const MAX_BATCH_REQUESTS = 500;
export const OBJECT_PLACEHOLDER = "";
@@ -15,11 +13,6 @@ export const batchRequestArg = z.record(z.string(), z.unknown()).refine(
"Each request must name exactly one kind of update, such as insertText",
);
export function sizedResult(value: Record<string, unknown>, what: string, hint: string): CallToolResult {
return boundedStructuredResult(value, MAX_RESULT_BYTES, (bytes) =>
`This ${what} is too large to return in one result (${Math.ceil(bytes / 1000)} KB; the limit is ${MAX_RESULT_BYTES / 1000} KB). Nothing was returned. ${hint}`);
}
const CONFLICT_STATUSES = new Set(["FAILED_PRECONDITION", "ABORTED"]);
const CONFLICT_REASON = /REVISION|CONDITION_NOT_MET|conditionNotMet/i;
const CONFLICT_MESSAGE = /\brequired revision\b|\brevision id\b|\blatest revision\b/i;
@@ -1,4 +1,5 @@
import { GoogleToolError } from "../errors.js";
import { byteCount } from "../tool.js";
import type { GoogleHttp, GoogleRequest } from "../google-http.js";
import {
DRIVE_HOST,
@@ -10,7 +11,6 @@ import {
GOOGLE_SHEET,
GOOGLE_SLIDES,
isGoogleType,
MAX_CONTENT_BYTES,
type RestFile,
} from "./file.js";
@@ -52,14 +52,14 @@ export function mediaRequest(id: string, revision?: string): GoogleRequest {
return { method: "GET", host: DRIVE_HOST, path, query: { alt: "media", supportsAllDrives: revision ? undefined : true } };
}
export function tooLargeContent(advice: string, size?: number): GoogleToolError {
const what = size === undefined ? "This file's content is" : `This file is ${size} bytes,`;
return new GoogleToolError("too_large", `${what} more than the ${MAX_CONTENT_BYTES} bytes this tool returns, so nothing was returned. ${advice}`);
export function tooLargeContent(advice: string, maxBytes: number, size?: number): GoogleToolError {
const what = size === undefined ? "This file's content is" : `This file is ${byteCount(size)} bytes,`;
return new GoogleToolError("too_large", `${what} more than the ${byteCount(maxBytes)} bytes this tool can return, so nothing was returned. ${advice}`);
}
export function assertWithinCap(file: RestFile, advice: string): void {
export function assertWithinCap(file: RestFile, advice: string, maxBytes: number): void {
const size = Number(file.size ?? "0");
if (size > MAX_CONTENT_BYTES) throw tooLargeContent(advice, size);
if (size > maxBytes) throw tooLargeContent(advice, maxBytes, size);
}
export function exportFormat(mimeType: string, requested: string | undefined): string {
@@ -72,14 +72,14 @@ export function exportFormat(mimeType: string, requested: string | undefined): s
return requested;
}
export function downloadPlan(id: string, file: RestFile, exportMimeType: string | undefined, revision: string | undefined, advice: string): ContentPlan {
export function downloadPlan(id: string, file: RestFile, exportMimeType: string | undefined, revision: string | undefined, advice: string, maxBytes: number): ContentPlan {
const mimeType = file.mimeType ?? "application/octet-stream";
if (isGoogleType(mimeType)) {
const target = exportFormat(mimeType, exportMimeType);
if (revision) throw new GoogleToolError("unsupported", "Earlier revisions of Google Docs, Sheets, Slides and other Google files cannot be downloaded with this tool. Omit revisionId to export the current version.");
return { request: exportRequest(id, target), contentType: target };
}
if (!revision) assertWithinCap(file, advice);
if (!revision) assertWithinCap(file, advice, maxBytes);
return { request: mediaRequest(id, revision), contentType: mimeType };
}
@@ -87,11 +87,11 @@ export async function readContentMeta(id: string, http: GoogleHttp): Promise<Res
return http.send<RestFile>(contentMetaRequest(id));
}
export async function fetchContent(plan: ContentPlan, http: GoogleHttp, advice: string): Promise<Buffer> {
export async function fetchContent(plan: ContentPlan, http: GoogleHttp, advice: string, maxBytes: number): Promise<Buffer> {
try {
return (await http.download(plan.request, MAX_CONTENT_BYTES)).data;
return (await http.download(plan.request, maxBytes)).data;
} catch (error) {
if (error instanceof GoogleToolError && error.kind === "too_large" && error.httpStatus !== 403) throw tooLargeContent(advice);
if (error instanceof GoogleToolError && error.kind === "too_large" && error.httpStatus !== 403) throw tooLargeContent(advice, maxBytes);
throw error;
}
}
@@ -2,13 +2,13 @@ import { describe, expect, it } from "vitest";
import { GoogleToolError } from "../errors.js";
import { fakeGoogle, json } from "../fixtures/fake-google.js";
import { createGoogleHttp } from "../google-http.js";
import { MAX_RESULT_BYTES, serializedResultBytes } from "../tool.js";
import { FALLBACK_MAX_RESULT_BYTES, serializedResultBytes } from "../tool.js";
import { downloadPlan } from "./content.js";
import { downloadFileContent } from "./download-file-content.js";
import { MAX_CONTENT_BYTES, READ_LESS, type RestFile } from "./file.js";
import { READ_LESS, type RestFile } from "./file.js";
import { DOC, DRAWING, FOLDER, FORM, HUGE_PDF, PDF, SHEET, SLIDES, TEXT_FILE } from "./fixtures/drive-rest.js";
const plan = (file: RestFile, exportMimeType?: string, revision?: string) => downloadPlan(file.id!, file, exportMimeType, revision, READ_LESS.download);
const plan = (file: RestFile, exportMimeType?: string, revision?: string) => downloadPlan(file.id!, file, exportMimeType, revision, READ_LESS.download, FALLBACK_MAX_RESULT_BYTES);
const target = (file: RestFile, exportMimeType?: string, revision?: string) => {
const planned = plan(file, exportMimeType, revision);
const query = Object.entries(planned.request.query ?? {}).filter(([, value]) => value !== undefined).map(([key, value]) => [key, String(value)]);
@@ -36,7 +36,7 @@ describe("download plan", () => {
[FOLDER, undefined, undefined, /have no content to download/],
[FORM, undefined, undefined, /have no content to download/],
[DOC, undefined, "rev0001", /Earlier revisions/],
[HUGE_PDF, undefined, undefined, /This file is 10485761 bytes, more than the 250000 bytes/],
[HUGE_PDF, undefined, undefined, /This file is 10,485,761 bytes, more than the 250,000 bytes/],
[DOC, "image/png", undefined, /cannot be exported as image\/png. Use one of: text\/plain/],
[SHEET, "text/plain", undefined, /cannot be exported as text\/plain/],
[SLIDES, "application/x-anything", undefined, /cannot be exported/],
@@ -46,8 +46,8 @@ describe("download plan", () => {
});
it("always addresses the file the caller named, not an ID from Google's answer", () => {
expect(downloadPlan("callerFile1", { ...PDF, id: "otherFile9" }, undefined, undefined, "").request.path).toBe("/drive/v3/files/callerFile1");
expect(downloadPlan("callerFile1", { ...DOC, id: "otherFile9" }, undefined, undefined, "").request.path).toBe("/drive/v3/files/callerFile1/export");
expect(downloadPlan("callerFile1", { ...PDF, id: "otherFile9" }, undefined, undefined, "", FALLBACK_MAX_RESULT_BYTES).request.path).toBe("/drive/v3/files/callerFile1");
expect(downloadPlan("callerFile1", { ...DOC, id: "otherFile9" }, undefined, undefined, "", FALLBACK_MAX_RESULT_BYTES).request.path).toBe("/drive/v3/files/callerFile1/export");
});
});
@@ -82,21 +82,21 @@ describe("download_file_content tool", () => {
const { fetch, calls } = fakeGoogle(() => json(HUGE_PDF));
const result = await downloadFileContent.call({ fileId: HUGE_PDF.id }, createGoogleHttp({ accessToken: "t", fetch }));
expect(calls).toHaveLength(1);
expect(result).toEqual({ isError: true, content: [{ type: "text", text: `This file is 10485761 bytes, more than the 250000 bytes this tool returns, so nothing was returned. ${READ_LESS.download}` }] });
expect(result).toEqual({ isError: true, content: [{ type: "text", text: `This file is 10,485,761 bytes, more than the 250,000 bytes this tool can return, so nothing was returned. ${READ_LESS.download}` }] });
});
it("refuses an export that streams past the cap, and never returns a cut-short result", async () => {
const { fetch } = fakeGoogle(({ url }) => (url.pathname.endsWith("/export")
? new Response(new ReadableStream({
start(controller) {
controller.enqueue(new Uint8Array(MAX_CONTENT_BYTES));
controller.enqueue(new Uint8Array(FALLBACK_MAX_RESULT_BYTES));
controller.enqueue(new Uint8Array(1));
controller.close();
},
}))
: json(SHEET)));
const result = await downloadFileContent.call({ fileId: SHEET.id }, createGoogleHttp({ accessToken: "t", fetch }));
expect(result).toEqual({ isError: true, content: [{ type: "text", text: `This file's content is more than the 250000 bytes this tool returns, so nothing was returned. ${READ_LESS.download}` }] });
expect(result).toEqual({ isError: true, content: [{ type: "text", text: `This file's content is more than the 250,000 bytes this tool can return, so nothing was returned. ${READ_LESS.download}` }] });
});
it("bounds the final serialized result: the largest file that fits is returned, one byte more is refused", async () => {
@@ -111,10 +111,10 @@ describe("download_file_content tool", () => {
if (!(await run(largest + step)).isError) largest += step;
}
const fits = await run(largest);
expect(serializedResultBytes(fits)).toBeLessThanOrEqual(MAX_RESULT_BYTES);
expect(serializedResultBytes(fits)).toBeGreaterThan(MAX_RESULT_BYTES - 16);
expect(serializedResultBytes(fits)).toBeLessThanOrEqual(FALLBACK_MAX_RESULT_BYTES);
expect(serializedResultBytes(fits)).toBeGreaterThan(FALLBACK_MAX_RESULT_BYTES - 16);
const over = await run(largest + 3);
expect((over.content[0] as { text: string }).text).toMatch(/^The result is \d+ bytes, more than the 250000 bytes this tool returns, so nothing was returned\. This tool cannot return part of a file/);
expect((over.content[0] as { text: string }).text).toMatch(/^The result is [\d,]+ bytes, more than the 250,000 bytes this tool can return, so it was not returned\. This tool cannot return part of a file/);
});
it("reports Google's export size limit as too large", async () => {
@@ -1,20 +1,20 @@
import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { defineTool } from "../tool.js";
import { defineTool, structuredResult } from "../tool.js";
import { downloadPlan, fetchContent, readContentMeta } from "./content.js";
import { boundedResult, fileId, fileIdInput, mimeTypeArg, READ_LESS, revisionId, type RestFile } from "./file.js";
import { fileId, fileIdInput, mimeTypeArg, READ_LESS, revisionId, type RestFile } from "./file.js";
const args = z.object({ fileId, exportMimeType: mimeTypeArg.optional(), revisionId: revisionId.optional() });
export type DownloadFileContentArgs = z.infer<typeof args>;
export function downloadFileContentResult(id: string, file: RestFile, contentType: string, data: Buffer): CallToolResult {
return boundedResult({ id, title: file.name ?? "", mimeType: contentType, content: data.toString("base64") }, READ_LESS.download);
return structuredResult({ id, title: file.name ?? "", mimeType: contentType, content: data.toString("base64") });
}
export const downloadFileContent = defineTool({
definition: {
name: "download_file_content",
description: "Downloads a Drive file's content as base64. Google Docs, Sheets, Slides, Drawings and Apps Script files are exported: by default Docs and Slides as text/plain, Sheets as text/csv (first sheet only; pass exportMimeType application/vnd.openxmlformats-officedocument.spreadsheetml.sheet for every sheet), Drawings as image/svg+xml, Apps Script as JSON. Other files are returned as stored. The whole result is at most 250,000 bytes (about 90 KB of file content, since base64 is larger and the result carries it twice); a larger file is refused with an error, never cut short. For a readable text version of a document, use read_file_content.",
description: "Downloads a Drive file's content as base64. Google Docs, Sheets, Slides, Drawings and Apps Script files are exported: by default Docs and Slides as text/plain, Sheets as text/csv (first sheet only; pass exportMimeType application/vnd.openxmlformats-officedocument.spreadsheetml.sheet for every sheet), Drawings as image/svg+xml, Apps Script as JSON. Other files are returned as stored. The whole result has a size limit; a file too large for it is refused with an error that gives the limit, never cut short. For a readable text version of a document, use read_file_content.",
inputSchema: {
type: "object",
properties: {
@@ -37,9 +37,10 @@ export const downloadFileContent = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
async run(input, http) {
whenTooLarge: READ_LESS.download,
async run(input, http, limits) {
const file = await readContentMeta(input.fileId, http);
const plan = downloadPlan(input.fileId, file, input.exportMimeType, input.revisionId, READ_LESS.download);
return downloadFileContentResult(input.fileId, file, plan.contentType, await fetchContent(plan, http, READ_LESS.download));
const plan = downloadPlan(input.fileId, file, input.exportMimeType, input.revisionId, READ_LESS.download, limits.maxResultBytes);
return downloadFileContentResult(input.fileId, file, plan.contentType, await fetchContent(plan, http, READ_LESS.download, limits.maxResultBytes));
},
});
@@ -1,14 +1,12 @@
import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { GoogleToolError } from "../errors.js";
import type { GoogleRequest } from "../google-http.js";
import { boundedStructuredResult, compact, MAX_RESULT_BYTES } from "../tool.js";
import { compact } from "../tool.js";
export const DRIVE_HOST = "www.googleapis.com" as const;
export const DRIVE_BASE = "/drive/v3";
export const DRIVE_UPLOAD_BASE = "/upload/drive/v3";
export const MAX_CONTENT_BYTES = MAX_RESULT_BYTES;
export const MAX_UPLOAD_BYTES = 5 * 1024 * 1024;
export const DEFAULT_PAGE_SIZE = 10;
export const MAX_PAGE_SIZE = 100;
@@ -179,14 +177,6 @@ export const READ_LESS = {
permissions: "This file has too many permissions to list with this tool.",
} as const;
export function tooLargeMessage(bytes: number, advice: string): string {
return `The result is ${bytes} bytes, more than the ${MAX_RESULT_BYTES} bytes this tool returns, so nothing was returned. ${advice}`;
}
export function boundedResult(value: Record<string, unknown>, advice: string): CallToolResult {
return boundedStructuredResult(value, MAX_RESULT_BYTES, (bytes) => tooLargeMessage(bytes, advice));
}
export function isGoogleType(mimeType: string | undefined): boolean {
return Boolean(mimeType?.startsWith(GOOGLE_APPS_PREFIX));
}
@@ -1,9 +1,8 @@
import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { defineTool } from "../tool.js";
import { defineTool, structuredResult } from "../tool.js";
import {
assertSnippetArgs,
boundedResult,
READ_LESS,
fileFields,
fileId,
@@ -21,7 +20,7 @@ const args = z.object({ fileId, includeRichMetadata: z.boolean().optional(), ...
export type GetFileMetadataArgs = z.infer<typeof args>;
export function getFileMetadataResult(file: RestFile, rich: boolean): CallToolResult {
return boundedResult(toMcpFile(file, rich), READ_LESS.metadata);
return structuredResult(toMcpFile(file, rich));
}
export const getFileMetadata = defineTool({
@@ -41,6 +40,7 @@ export const getFileMetadata = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: READ_LESS.metadata,
async run(input, http) {
assertSnippetArgs(input);
const rich = input.includeRichMetadata === true;
@@ -2,8 +2,8 @@ import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { GoogleToolError } from "../errors.js";
import type { GoogleHttp, GoogleRequest } from "../google-http.js";
import { compact, defineTool } from "../tool.js";
import { boundedResult, DRIVE_HOST, fileId, fileIdInput, filePath, READ_LESS, type RestPermission } from "./file.js";
import { compact, defineTool, structuredResult } from "../tool.js";
import { DRIVE_HOST, fileId, fileIdInput, filePath, READ_LESS, type RestPermission } from "./file.js";
const args = z.object({ fileId });
export type GetFilePermissionsArgs = z.infer<typeof args>;
@@ -41,7 +41,7 @@ export function toMcpPermission(permission: RestPermission): Record<string, unkn
}
export function getFilePermissionsResult(permissions: RestPermission[]): CallToolResult {
return boundedResult({ permissions: permissions.filter((permission) => !permission.deleted).map(toMcpPermission) }, READ_LESS.permissions);
return structuredResult({ permissions: permissions.filter((permission) => !permission.deleted).map(toMcpPermission) });
}
export async function readPermissions(id: string, http: GoogleHttp): Promise<RestPermission[]> {
@@ -88,6 +88,7 @@ export const getFilePermissions = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: READ_LESS.permissions,
async run(input, http) {
return getFilePermissionsResult(await readPermissions(input.fileId, http));
},
@@ -1,10 +1,9 @@
import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import type { GoogleRequest } from "../google-http.js";
import { defineTool } from "../tool.js";
import { defineTool, structuredResult } from "../tool.js";
import {
assertSnippetArgs,
boundedResult,
READ_LESS,
DRIVE_BASE,
DRIVE_HOST,
@@ -50,7 +49,7 @@ export function listRecentFilesRequest(input: ListRecentFilesArgs): GoogleReques
}
export function listRecentFilesResult(list: RestFileList): CallToolResult {
return boundedResult(listResult(list), READ_LESS.list);
return structuredResult(listResult(list));
}
export const listRecentFiles = defineTool({
@@ -70,6 +69,7 @@ export const listRecentFiles = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: READ_LESS.list,
async run(input, http) {
assertSnippetArgs(input);
return listRecentFilesResult(await http.send<RestFileList>(listRecentFilesRequest(input)));
@@ -1,8 +1,8 @@
import { describe, expect, it } from "vitest";
import { fakeGoogle, json, type RecordedCall } from "../fixtures/fake-google.js";
import { createGoogleHttp } from "../google-http.js";
import { MAX_RESULT_BYTES, serializedResultBytes } from "../tool.js";
import { MAX_CONTENT_BYTES, READ_LESS, type RestFile } from "./file.js";
import { FALLBACK_MAX_RESULT_BYTES, serializedResultBytes } from "../tool.js";
import { READ_LESS, type RestFile } from "./file.js";
import { DOC, DRAWING, FOLDER, HUGE_PDF, JSON_FILE, PDF, SHEET, SLIDES, TEXT_FILE } from "./fixtures/drive-rest.js";
import { decodeText, readFileContent, readPlan, SHEET_NOTE, toCommentThread } from "./read-file-content.js";
@@ -15,7 +15,7 @@ describe("read plan", () => {
[JSON_FILE, "/drive/v3/files/json0001synthetic", "application/json"],
[{ id: "x1", mimeType: "application/ld+json", size: "3" }, "/drive/v3/files/x1", "application/ld+json"],
] as const)("%#: reads %s from %s as %s", (file, path, contentType) => {
const plan = readPlan(file.id!, file);
const plan = readPlan(file.id!, file, FALLBACK_MAX_RESULT_BYTES);
expect(plan?.request.path).toBe(path);
expect(plan?.contentType).toBe(contentType);
});
@@ -23,12 +23,12 @@ describe("read plan", () => {
it.each([PDF, FOLDER, DRAWING, { id: "i1", mimeType: "image/png" }, { id: "w1", mimeType: "application/vnd.openxmlformats-officedocument.wordprocessingml.document" }])(
"%#: has no text version of %s",
(file: RestFile) => {
expect(readPlan(file.id!, file)).toBeNull();
expect(readPlan(file.id!, file, FALLBACK_MAX_RESULT_BYTES)).toBeNull();
},
);
it("refuses a text file over the cap before downloading", () => {
expect(() => readPlan("big1", { ...HUGE_PDF, mimeType: "text/plain" })).toThrow(/more than the 250000 bytes/);
expect(() => readPlan("big1", { ...HUGE_PDF, mimeType: "text/plain" }, FALLBACK_MAX_RESULT_BYTES)).toThrow(/more than the 250,000 bytes/);
});
it("decodes UTF-8, drops a byte order mark, and rejects bytes that are not UTF-8", () => {
@@ -106,9 +106,9 @@ describe("read_file_content tool", () => {
});
it("refuses content that grows past the cap while streaming", async () => {
const { fetch } = route(SLIDES, new Uint8Array(MAX_CONTENT_BYTES + 1));
const { fetch } = route(SLIDES, new Uint8Array(FALLBACK_MAX_RESULT_BYTES + 1));
const result = await readFileContent.call({ fileId: SLIDES.id }, createGoogleHttp({ accessToken: "t", fetch }));
expect(result).toEqual({ isError: true, content: [{ type: "text", text: `This file's content is more than the 250000 bytes this tool returns, so nothing was returned. ${READ_LESS.read}` }] });
expect(result).toEqual({ isError: true, content: [{ type: "text", text: `This file's content is more than the 250,000 bytes this tool can return, so nothing was returned. ${READ_LESS.read}` }] });
});
it("bounds the final serialized result, including JSON escaping, with advice on reading less", async () => {
@@ -116,17 +116,17 @@ describe("read_file_content tool", () => {
const result = await readFileContent.call({ fileId: SLIDES.id }, createGoogleHttp({ accessToken: "t", fetch }));
expect(result.isError).toBe(true);
const text = (result.content[0] as { text: string }).text;
expect(text).toMatch(/^The result is \d+ bytes, more than the 250000 bytes this tool returns, so nothing was returned\. /);
expect(text).toMatch(/^The result is [\d,]+ bytes, more than the 250,000 bytes this tool can return, so it was not returned\. /);
expect(text.endsWith(READ_LESS.read)).toBe(true);
expect(Number(/\d+/.exec(text)![0])).toBeGreaterThan(2 * 6 * 30_000);
expect(Number(/[\d,]+/.exec(text)![0].replaceAll(",", ""))).toBeGreaterThan(2 * 6 * 30_000);
});
it("returns the largest text that fits and refuses one byte more", async () => {
const run = async (size: number) => readFileContent.call({ fileId: SLIDES.id }, createGoogleHttp({ accessToken: "t", fetch: route(SLIDES, "a".repeat(size)).fetch }));
let largest = 0;
for (let step = 1 << 17; step >= 1; step >>= 1) if (!(await run(largest + step)).isError) largest += step;
expect(serializedResultBytes(await run(largest))).toBeLessThanOrEqual(MAX_RESULT_BYTES);
expect(serializedResultBytes(await run(largest))).toBeGreaterThan(MAX_RESULT_BYTES - 4);
expect(serializedResultBytes(await run(largest))).toBeLessThanOrEqual(FALLBACK_MAX_RESULT_BYTES);
expect(serializedResultBytes(await run(largest))).toBeGreaterThan(FALLBACK_MAX_RESULT_BYTES - 4);
expect((await run(largest + 1)).isError).toBe(true);
});
});
@@ -2,9 +2,9 @@ import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { GoogleToolError } from "../errors.js";
import type { GoogleHttp, GoogleRequest } from "../google-http.js";
import { compact, defineTool } from "../tool.js";
import { compact, defineTool, structuredResult } from "../tool.js";
import { assertWithinCap, exportRequest, fetchContent, mediaRequest, readContentMeta, type ContentPlan } from "./content.js";
import { boundedResult, DRIVE_HOST, fileId, fileIdInput, filePath, GOOGLE_DOC, GOOGLE_SHEET, GOOGLE_SLIDES, READ_LESS, type RestFile } from "./file.js";
import { DRIVE_HOST, fileId, fileIdInput, filePath, GOOGLE_DOC, GOOGLE_SHEET, GOOGLE_SLIDES, READ_LESS, type RestFile } from "./file.js";
const args = z.object({ fileId, includeComments: z.boolean().optional() });
export type ReadFileContentArgs = z.infer<typeof args>;
@@ -22,13 +22,13 @@ export interface ReadPlan extends ContentPlan {
prefix?: string;
}
export function readPlan(id: string, file: RestFile): ReadPlan | null {
export function readPlan(id: string, file: RestFile, maxBytes: number): ReadPlan | null {
const mimeType = file.mimeType ?? "";
if (mimeType === GOOGLE_DOC) return { request: exportRequest(id, "text/markdown"), contentType: "text/markdown" };
if (mimeType === GOOGLE_SLIDES) return { request: exportRequest(id, "text/plain"), contentType: "text/plain" };
if (mimeType === GOOGLE_SHEET) return { request: exportRequest(id, "text/csv"), contentType: "text/csv", prefix: SHEET_NOTE };
if (mimeType.startsWith("text/") || TEXT_APPLICATION_TYPES.has(mimeType) || /\+(json|xml)$/.test(mimeType)) {
assertWithinCap(file, READ_LESS.read);
assertWithinCap(file, READ_LESS.read, maxBytes);
return { request: mediaRequest(id), contentType: mimeType };
}
return null;
@@ -90,13 +90,13 @@ export async function readComments(id: string, http: GoogleHttp): Promise<RestCo
export function readFileContentResult(file: RestFile, text: string | null, comments: RestComment[] | null, includeComments: boolean): CallToolResult {
const commentable = COMMENTABLE.has(file.mimeType ?? "");
return boundedResult(compact({
return structuredResult(compact({
fileContent: text ?? "",
contentAnchoredComments: includeComments ? [] : undefined,
unanchoredComments: includeComments ? (comments ?? []).map(toCommentThread) : undefined,
textFormattingNotSupported: text === null ? true : undefined,
commentsNotSupported: includeComments && !commentable ? true : undefined,
}), READ_LESS.read);
}));
}
const postSchema = {
@@ -121,7 +121,7 @@ const threadSchema = {
export const readFileContent = defineTool({
definition: {
name: "read_file_content",
description: "Returns a text version of a Drive file. Google Docs come back as Markdown, Slides as plain text, and Sheets as CSV of the first sheet only (the text says so). Text files (text/*, JSON, XML and similar) are returned as stored. Other files, such as PDFs, Office files and images, are not converted: the result has textFormattingNotSupported true and empty fileContent; use download_file_content for their bytes. The whole result is at most 250,000 bytes (about 120 KB of plain text, since the result carries it twice); a larger file is refused with an error, never cut short. With includeComments, the comment threads of a Google Doc, Sheet or Slides file are listed in unanchoredComments; they are not placed in the text. Get fileId from search_files or list_recent_files first.",
description: "Returns a text version of a Drive file. Google Docs come back as Markdown, Slides as plain text, and Sheets as CSV of the first sheet only (the text says so). Text files (text/*, JSON, XML and similar) are returned as stored. Other files, such as PDFs, Office files and images, are not converted: the result has textFormattingNotSupported true and empty fileContent; use download_file_content for their bytes. The whole result has a size limit; a file too large for it is refused with an error that gives the limit, never cut short. With includeComments, the comment threads of a Google Doc, Sheet or Slides file are listed in unanchoredComments; they are not placed in the text. Get fileId from search_files or list_recent_files first.",
inputSchema: {
type: "object",
properties: {
@@ -144,12 +144,13 @@ export const readFileContent = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
async run(input, http) {
whenTooLarge: READ_LESS.read,
async run(input, http, limits) {
const file = await readContentMeta(input.fileId, http);
const plan = readPlan(input.fileId, file);
const plan = readPlan(input.fileId, file, limits.maxResultBytes);
let text: string | null = null;
if (plan) {
const decoded = decodeText(await fetchContent(plan, http, READ_LESS.read));
const decoded = decodeText(await fetchContent(plan, http, READ_LESS.read, limits.maxResultBytes));
text = decoded === null ? null : plan.prefix ? `${plan.prefix}\n${decoded}` : decoded;
}
const comments = input.includeComments && COMMENTABLE.has(file.mimeType ?? "") ? await readComments(input.fileId, http) : null;
@@ -108,7 +108,7 @@ describe("search_files tool", () => {
const { fetch } = fakeGoogle(() => json({ files }));
const result = await searchFiles.call({ pageSize: 100 }, createGoogleHttp({ accessToken: "t", fetch }));
expect(result.isError).toBe(true);
expect((result.content[0] as { text: string }).text).toMatch(/more than the 250000 bytes this tool returns, so nothing was returned\. Ask for fewer files with a smaller pageSize/);
expect((result.content[0] as { text: string }).text).toMatch(/more than the 250,000 bytes this tool can return, so it was not returned\. Ask for fewer files with a smaller pageSize/);
});
it("returns an empty list when nothing matches", async () => {
@@ -2,10 +2,9 @@ import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { GoogleToolError } from "../errors.js";
import type { GoogleRequest } from "../google-http.js";
import { defineTool } from "../tool.js";
import { defineTool, structuredResult } from "../tool.js";
import {
assertSnippetArgs,
boundedResult,
READ_LESS,
DRIVE_BASE,
DRIVE_HOST,
@@ -58,7 +57,7 @@ export function searchFilesRequest(input: SearchFilesArgs): GoogleRequest {
}
export function searchFilesResult(list: RestFileList, rich: boolean): CallToolResult {
return boundedResult(listResult(list, rich), READ_LESS.list);
return structuredResult(listResult(list, rich));
}
const QUERY_HELP = [
@@ -88,6 +87,7 @@ export const searchFiles = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: READ_LESS.list,
async run(input, http) {
assertSnippetArgs(input);
const rich = input.includeRichMetadata === true;
@@ -78,6 +78,7 @@ export const getDraft = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Read it with view DRAFT_VIEW_METADATA_ONLY, which leaves out the body.",
async run(input, http) {
const draft = await http.send<GmailRestDraftFull>(readDraftRequest(input.draftId, input.view));
return getDraftResult(draft, input, await readDraftBodies([draft], input.view, http));
@@ -70,6 +70,7 @@ export const getMessage = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Read it with messageFormat METADATA_ONLY or MINIMAL, which leave out the bodies.",
async run(input, http) {
const message = await http.send<GmailRestMessage>(getMessageRequest(input));
return getMessageResult(message, input, await readExternalBodies(message, input, http));
@@ -72,7 +72,7 @@ describe("get_thread with bodies Gmail stores separately", () => {
if (url.pathname.endsWith("/attachments/ANGjdJ_m2_html")) return json({ data: Buffer.from(`<pre>${large}</pre>`).toString("base64url") });
return json(thread);
});
const result = await getThread.call({ threadId: "th1" }, createGoogleHttp({ accessToken: "t", fetch }));
const result = await getThread.call({ threadId: "th1" }, createGoogleHttp({ accessToken: "t", fetch }), { maxResultBytes: 1_000_000 });
const messages = (result.structuredContent as { messages: Array<{ plaintextBody?: string; htmlBody?: string }> }).messages;
expect(messages[0]!.plaintextBody).toBe("short");
expect(messages[1]!.plaintextBody).toBe(large);
@@ -58,6 +58,7 @@ export const getThread = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Read it with messageFormat METADATA_ONLY or MINIMAL, which leave out the bodies, then read single messages with get_message.",
async run(input, http) {
const thread = await http.send<GmailRestFullThread>(readThreadRequest(input));
const bodies = messageDetail(input.messageFormat) === "full" ? await readMessageBodies(thread.messages ?? [], http) : undefined;
@@ -76,6 +76,7 @@ export const listDrafts = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: false, openWorldHint: false },
},
args,
whenTooLarge: "Ask for fewer drafts with a smaller pageSize, or use view DRAFT_VIEW_METADATA_ONLY, then read single drafts with get_draft.",
async run(input, http) {
const list = await http.send<GmailDraftList>(listDraftsRequest(input));
const drafts = await readDrafts(draftIds(list), input.view, http);
@@ -89,6 +89,7 @@ export const listLabels = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Ask for fewer labels with a smaller pageSize.",
async run(input, http) {
return listLabelsResult(await http.send<GmailLabelList>(listLabelsRequest()), input);
},
@@ -89,6 +89,7 @@ export const listThreads = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Ask for fewer threads with a smaller pageSize, then read single threads with get_thread.",
async run(input, http) {
const list = await http.send<GmailThreadList>(listThreadsToolRequest(input));
return searchThreadsResult(list, await readThreads(threadIds(list), http), { view: input.view });
@@ -98,6 +98,7 @@ export const searchMessages = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Ask for fewer messages with a smaller pageSize, or use messageFormat MINIMAL, then read single messages with get_message.",
async run(input, http) {
const list = await http.send<GmailMessageList>(listMessagesRequest(input));
const messages = await readMessages(messageIds(list), input, http);
@@ -148,6 +148,7 @@ export const searchThreads = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },
},
args,
whenTooLarge: "Ask for fewer threads with a smaller pageSize, then read single threads with get_thread.",
async run(input, http) {
const list = await http.send<GmailThreadList>(listThreadsRequest(input));
return searchThreadsResult(list, await readThreads(threadIds(list), http), input);
@@ -0,0 +1,182 @@
import { describe, expect, it } from "vitest";
import { oneTab, text } from "./docs/fixtures/docs-rest.js";
import { fakeGoogle, json } from "./fixtures/fake-google.js";
import { shape, slide } from "./slides/fixtures/slides-rest.js";
import { z } from "zod";
import { defineTool, FALLBACK_MAX_RESULT_BYTES, serializedResultBytes } from "./tool.js";
import { callGoogleWorkspaceRestTool, type GoogleWorkspaceRestService } from "./tools.js";
import { READ_LESS } from "./drive/file.js";
interface BudgetCase {
service: GoogleWorkspaceRestService;
name: string;
arguments: (size: number) => Record<string, unknown>;
answer: (size: number, url: URL) => Response;
advice: string;
}
const filler = (size: number) => "a".repeat(size);
const b64 = (value: string) => Buffer.from(value, "utf8").toString("base64url");
const CASES: BudgetCase[] = [
{
service: "gmail",
name: "get_message",
arguments: () => ({ messageId: "msg1" }),
answer: (size) => json({
id: "msg1", threadId: "thr1", internalDate: "1763337600000",
payload: { mimeType: "text/plain", headers: [{ name: "Subject", value: "Report" }], body: { size, data: b64(filler(size)) } },
}),
advice: "Read it with messageFormat METADATA_ONLY or MINIMAL, which leave out the bodies.",
},
{
service: "gmail",
name: "list_labels",
arguments: () => ({}),
answer: (size) => json({ labels: [{ id: "Label_1", name: filler(size), type: "user" }] }),
advice: "Ask for fewer labels with a smaller pageSize.",
},
{
service: "calendar",
name: "list_calendars",
arguments: () => ({}),
answer: (size) => json({ items: [{ id: "primary", summary: filler(size), timeZone: "UTC" }] }),
advice: "Ask for fewer calendars with a smaller pageSize.",
},
{
service: "calendar",
name: "get_event",
arguments: () => ({ eventId: "evt1" }),
answer: (size) => json({ id: "evt1", status: "confirmed", summary: "Review", description: filler(size),
start: { dateTime: "2026-11-18T10:00:00Z" }, end: { dateTime: "2026-11-18T11:00:00Z" } }),
advice: "This tool cannot return part of an event.",
},
{
service: "drive",
name: "read_file_content",
arguments: () => ({ fileId: "text0001synthetic" }),
answer: (size, url) => url.searchParams.get("alt") === "media"
? new Response(filler(size))
: json({ id: "text0001synthetic", name: "notes.txt", mimeType: "text/plain", size: String(size) }),
advice: READ_LESS.read,
},
{
service: "docs",
name: "read_doc",
arguments: () => ({ documentId: "doc1" }),
answer: (size) => json(oneTab([text(filler(size))])),
advice: "This tool cannot read part of a document. Read it without commentsIncluded if that was set, or read only its text with read_file_content.",
},
{
service: "slides",
name: "read_presentation",
arguments: () => ({ presentationId: "deck1" }),
answer: (size) => json({ presentationId: "deck1", revisionId: "r1", slides: [slide("s1", [shape("e1", [{ text: filler(size) }])])] }),
advice: "Pass fields to read only part of it, for example [\"slides.objectId\", \"slides.pageElements(objectId,shape.text)\"].",
},
{
service: "sheets",
name: "get_values",
arguments: () => ({ spreadsheetId: "sheet1", range: "Sheet1!A1" }),
answer: (size) => json({ range: "Sheet1!A1", majorDimension: "ROWS", values: [[filler(size)]] }),
advice: "Ask for a narrower range, with fewer rows or columns (for example Sheet1!A1:Z1000), and read the rest in further calls.",
},
];
function call(testCase: BudgetCase, size: number, maxResultBytes?: number) {
const { fetch } = fakeGoogle(({ url }) => testCase.answer(size, url));
return callGoogleWorkspaceRestTool({
service: testCase.service, name: testCase.name, arguments: testCase.arguments(size), accessToken: "t", fetch,
...(maxResultBytes === undefined ? {} : { maxResultBytes }),
});
}
async function largestThatFits(testCase: BudgetCase, maxResultBytes?: number): Promise<number> {
let largest = 0;
for (let step = 1 << 21; step >= 1; step >>= 1) {
if (!(await call(testCase, largest + step, maxResultBytes)).isError) largest += step;
}
return largest;
}
const errorText = (result: { content: unknown[] }) => (result.content[0] as { text: string }).text;
describe("result budget", () => {
it.each(CASES.map((testCase) => [`${testCase.service} ${testCase.name}`, testCase] as const))(
"%s: returns a result just under the budget it is given and refuses one just over it, saying how to ask for less",
async (_label, testCase) => {
const budget = 40_000;
const largest = await largestThatFits(testCase, budget);
const fits = await call(testCase, largest, budget);
expect(fits.isError).toBeUndefined();
expect(serializedResultBytes(fits)).toBeLessThanOrEqual(budget);
expect(serializedResultBytes(fits)).toBeGreaterThan(budget - 64);
const over = await call(testCase, largest + 1, budget);
expect(over.isError).toBe(true);
expect(over.structuredContent).toBeUndefined();
expect(errorText(over)).toMatch(/^The result is [\d,]+ bytes, more than the 40,000 bytes this tool can return, so it was not returned\. /);
expect(errorText(over).endsWith(testCase.advice)).toBe(true);
},
);
it.each(CASES.map((testCase) => [`${testCase.service} ${testCase.name}`, testCase] as const))(
"%s: uses the fallback budget only when the caller passes none",
async (_label, testCase) => {
const size = 150_000;
const fallback = await call(testCase, size);
expect(fallback.isError).toBe(true);
expect(errorText(fallback)).toMatch(/more than the 250,000 bytes this tool can return/);
const supplied = await call(testCase, size, 1_000_000);
expect(supplied.isError).toBeUndefined();
expect(serializedResultBytes(supplied)).toBeGreaterThan(FALLBACK_MAX_RESULT_BYTES);
},
);
it("never turns a completed write into an error, whatever the size of its result", async () => {
const { fetch, calls } = fakeGoogle(() => json({ id: "draft1", message: { id: "msg2", threadId: "thr2" } }));
const body = filler(30_000);
const result = await callGoogleWorkspaceRestTool({ service: "gmail", name: "create_draft", arguments: { to: ["sam@example.test"], subject: "Report", body },
accessToken: "t", fetch, maxResultBytes: 1_000 });
expect(result.isError).toBeUndefined();
expect(serializedResultBytes(result)).toBeGreaterThan(1_000);
expect(result.structuredContent).toMatchObject({ id: "draft1", plaintextBody: body });
expect(calls).toHaveLength(1);
});
it("requires advice from every read tool and none from a write tool", () => {
const definition = (readOnlyHint: boolean) => ({ name: "t", inputSchema: { type: "object" as const }, annotations: { readOnlyHint } });
const run = async () => ({ content: [] });
expect(() => defineTool({ definition: definition(true), args: z.object({}), run })).toThrow(/read tool must say how to ask for less/);
expect(() => defineTool({ definition: definition(false), args: z.object({}), whenTooLarge: "x", run })).toThrow(/write tool must not/);
});
it("refuses a budget that is not a positive whole number before calling Google", async () => {
const { fetch, calls } = fakeGoogle(() => json({ labels: [] }));
for (const maxResultBytes of [0, -1, 1.5, Number.NaN, Number.POSITIVE_INFINITY]) {
await expect(callGoogleWorkspaceRestTool({ service: "gmail", name: "list_labels", accessToken: "t", fetch, maxResultBytes }))
.rejects.toThrow(/maxResultBytes must be a positive integer/);
}
expect(calls).toHaveLength(0);
});
it("caps a Drive download at the budget it is given, not at the fallback", async () => {
const big = { id: "pdf1", name: "big.pdf", mimeType: "application/pdf", size: String(FALLBACK_MAX_RESULT_BYTES + 1) };
const { fetch, calls } = fakeGoogle(({ url }) => url.searchParams.get("alt") === "media" ? new Response(new Uint8Array(FALLBACK_MAX_RESULT_BYTES + 1)) : json(big));
const fallback = await callGoogleWorkspaceRestTool({ service: "drive", name: "download_file_content", arguments: { fileId: "pdf1" }, accessToken: "t", fetch });
expect(errorText(fallback)).toMatch(/^This file is 250,001 bytes, more than the 250,000 bytes this tool can return/);
expect(calls).toHaveLength(1);
const supplied = await callGoogleWorkspaceRestTool({ service: "drive", name: "download_file_content", arguments: { fileId: "pdf1" }, accessToken: "t", fetch, maxResultBytes: 1_000_000 });
expect(supplied.isError).toBeUndefined();
expect(Buffer.from((supplied.structuredContent as { content: string }).content, "base64")).toHaveLength(FALLBACK_MAX_RESULT_BYTES + 1);
});
it("tells a Gmail list caller to lower the page size", async () => {
const thread = (id: string) => ({ id, messages: [{ id: `${id}m`, threadId: id, internalDate: "1763337600000", snippet: filler(2_000),
payload: { headers: [{ name: "Subject", value: filler(2_000) }] } }] });
const { fetch } = fakeGoogle(({ url }) => url.pathname.endsWith("/threads")
? json({ threads: Array.from({ length: 20 }, (_, index) => ({ id: `t${index}` })) })
: json(thread(url.pathname.split("/").at(-1)!)));
const result = await callGoogleWorkspaceRestTool({ service: "gmail", name: "search_threads", arguments: { query: "report" }, accessToken: "t", fetch, maxResultBytes: 20_000 });
expect(errorText(result)).toMatch(/so it was not returned\. Ask for fewer threads with a smaller pageSize, then read single threads with get_thread\.$/);
});
});
@@ -7,6 +7,7 @@ import { callGoogleWorkspaceRestTool, listGoogleWorkspaceRestTools, type GoogleW
export interface GoogleWorkspaceRestServerOptions {
fetch?: GoogleFetch;
timeoutMs?: number;
maxResultBytes?: number;
}
export function createGoogleWorkspaceRestServer(service: GoogleWorkspaceRestService, options: GoogleWorkspaceRestServerOptions = {}): Server {
@@ -26,6 +27,7 @@ export function createGoogleWorkspaceRestServer(service: GoogleWorkspaceRestServ
fetch: options.fetch,
signal: extra.signal,
timeoutMs: options.timeoutMs,
maxResultBytes: options.maxResultBytes,
});
});
return server;
@@ -3,7 +3,6 @@ import { fakeGoogle, json } from "../fixtures/fake-google.js";
import { createGoogleHttp } from "../google-http.js";
import { SPREADSHEET_ID, SPREADSHEET_METADATA, SPREADSHEET_WITH_GRID } from "./fixtures/sheets-rest.js";
import { serializedResultBytes } from "../tool.js";
import { MAX_RESULT_BYTES } from "./spreadsheet.js";
import { getSpreadsheet, getSpreadsheetRequest, getSpreadsheetResult, gridCellCount } from "./get-spreadsheet.js";
const text = (result: { content: unknown[] }) => (result.content[0] as { text: string }).text;
@@ -75,11 +74,13 @@ describe("get_spreadsheet result", () => {
expect(gridCellCount(SPREADSHEET_WITH_GRID)).toBe(3);
});
it("bounds the whole serialized result in bytes and asks for fewer or narrower ranges", () => {
it("bounds the whole serialized result in bytes and asks for fewer or narrower ranges", async () => {
const sheet = (note: string) => ({ properties: { title: note } });
const near = getSpreadsheetResult({ sheets: [sheet("x".repeat(124_900))] });
expect(serializedResultBytes(near)).toBeLessThanOrEqual(MAX_RESULT_BYTES);
expect(() => getSpreadsheetResult({ sheets: [sheet("x".repeat(125_000))] })).toThrow(/bytes; the limit is 250,000 per call\. Pass fewer or narrower ranges/);
const call = (title: string) => getSpreadsheet.call({ spreadsheetId: SPREADSHEET_ID }, createGoogleHttp({ accessToken: "t", fetch: fakeGoogle(() => json({ sheets: [sheet(title)] })).fetch }));
const near = await call("x".repeat(124_900));
expect(near.isError).toBeUndefined();
expect(serializedResultBytes(near)).toBeLessThanOrEqual(250_000);
expect(text(await call("x".repeat(125_000)))).toMatch(/bytes, more than the 250,000 bytes this tool can return, so it was not returned\. Pass fewer or narrower ranges/);
});
it("refuses a result over the cell limit instead of cutting it", () => {
@@ -110,6 +110,7 @@ export const getSpreadsheet = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: true },
},
args,
whenTooLarge: NARROW,
async run(input, http) {
return getSpreadsheetResult(await sendRead<RestSpreadsheet>(http, getSpreadsheetRequest(input), NARROW));
},
@@ -3,8 +3,8 @@ import { fakeGoogle, json } from "../fixtures/fake-google.js";
import { createGoogleHttp } from "../google-http.js";
import { EMPTY_VALUES, FORMATTED_VALUES, SPREADSHEET_ID } from "./fixtures/sheets-rest.js";
import { getValues, getValuesRequest, getValuesResult } from "./get-values.js";
import { serializedResultBytes } from "../tool.js";
import { boundedReadResult, MAX_CELLS_PER_CALL, MAX_RESULT_BYTES } from "./spreadsheet.js";
import { FALLBACK_MAX_RESULT_BYTES, serializedResultBytes } from "../tool.js";
import { boundedReadResult, MAX_CELLS_PER_CALL } from "./spreadsheet.js";
const text = (result: { content: unknown[] }) => (result.content[0] as { text: string }).text;
@@ -34,24 +34,29 @@ describe("get_values result", () => {
expect(getValuesResult(EMPTY_VALUES).structuredContent).toEqual({ range: "Sheet1!A1:Z100" });
});
const callWith = (values: unknown[][]) => getValues.call({ spreadsheetId: SPREADSHEET_ID, range: "Sheet1!A1:Z" },
createGoogleHttp({ accessToken: "t", fetch: fakeGoogle(() => json({ range: "Sheet1!A1:Z", values })).fetch }));
it.each([
["too many cells", [Array.from({ length: MAX_CELLS_PER_CALL + 1 }, () => "")], /50,001 cells; the limit is 50,000/],
["too many bytes", [["x".repeat(130_000)]], /bytes; the limit is 250,000 per call\. Ask for a narrower range/],
])("refuses %s instead of cutting the result", (_name, values, message) => {
expect(() => getValuesResult({ range: "Sheet1!A1:Z", values })).toThrow(message);
["too many bytes", [["x".repeat(130_000)]], /bytes, more than the 250,000 bytes this tool can return, so it was not returned\. Ask for a narrower range/],
])("refuses %s instead of cutting the result", async (_name, values, message) => {
const result = await callWith(values);
expect(result.isError).toBe(true);
expect(text(result)).toMatch(message);
});
it.each([["x"], ["é"], ["\""]])("bounds the whole serialized result, text and structured content together, for cells of %j", (unit) => {
const sized = (count: number) => ({ range: "Sheet1!A1:B2", values: [[unit.repeat(count)], ["tail"]] });
it.each([["x"], ["é"], ["\""]])("bounds the whole serialized result, text and structured content together, for cells of %j", async (unit) => {
const sized = (count: number) => ({ range: "Sheet1!A1:Z", values: [[unit.repeat(count)], ["tail"]] });
const bytes = (count: number) => serializedResultBytes({ content: [{ type: "text", text: JSON.stringify(sized(count)) }], structuredContent: sized(count) });
let largest = 0;
for (let step = 1 << 20; step >= 1; step >>= 1) if (bytes(largest + step) <= MAX_RESULT_BYTES) largest += step;
const near = getValuesResult(sized(largest));
for (let step = 1 << 20; step >= 1; step >>= 1) if (bytes(largest + step) <= FALLBACK_MAX_RESULT_BYTES) largest += step;
const near = await callWith(sized(largest).values);
expect(near.isError).toBeUndefined();
expect(serializedResultBytes(near)).toBe(bytes(largest));
expect(serializedResultBytes(near)).toBeLessThanOrEqual(MAX_RESULT_BYTES);
expect(serializedResultBytes(near)).toBeGreaterThan(MAX_RESULT_BYTES - 16);
expect(bytes(largest + 1)).toBeGreaterThan(MAX_RESULT_BYTES);
expect(() => getValuesResult(sized(largest + 1))).toThrow(/bytes; the limit is 250,000 per call\. Ask for a narrower range/);
expect(serializedResultBytes(near)).toBeGreaterThan(FALLBACK_MAX_RESULT_BYTES - 16);
expect(bytes(largest + 1)).toBeGreaterThan(FALLBACK_MAX_RESULT_BYTES);
expect(text(await callWith(sized(largest + 1).values))).toMatch(/bytes, more than the 250,000 bytes this tool can return, so it was not returned\. Ask for a narrower range/);
});
it("accepts exactly the cell limit and refuses one more", () => {
@@ -65,6 +65,7 @@ export const getValues = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: true },
},
args,
whenTooLarge: NARROW,
async run(input, http) {
return getValuesResult(await sendRead<RestValueRange>(http, getValuesRequest(input), NARROW));
},
@@ -2,13 +2,12 @@ import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { GoogleToolError } from "../errors.js";
import type { GoogleHttp, GoogleRequest } from "../google-http.js";
import { boundedStructuredResult, MAX_RESULT_BYTES } from "../tool.js";
import { structuredResult } from "../tool.js";
import { checkSheetRange } from "./range.js";
export const SHEETS_HOST = "sheets.googleapis.com" as const;
export const SHEETS_BASE = "/v4/spreadsheets";
export const MAX_CELLS_PER_CALL = 50_000;
export { MAX_RESULT_BYTES };
export const MAX_REQUEST_CHARS = 1_000_000;
const formatCount = (value: number) => value.toLocaleString("en-US");
@@ -53,7 +52,7 @@ export function boundedReadResult(value: Record<string, unknown>, cells: number,
if (cells > MAX_CELLS_PER_CALL) {
throw new GoogleToolError("incomplete", `The result has ${formatCount(cells)} cells; the limit is ${formatCount(MAX_CELLS_PER_CALL)} per call. ${narrow}`);
}
return boundedStructuredResult(value, MAX_RESULT_BYTES, (bytes) => `The result is ${formatCount(bytes)} bytes; the limit is ${formatCount(MAX_RESULT_BYTES)} per call. ${narrow}`);
return structuredResult(value);
}
export async function sendRead<T>(http: GoogleHttp, request: GoogleRequest, narrow: string): Promise<T> {
@@ -1,5 +1,5 @@
import { describe, expect, it } from "vitest";
import { MAX_RESULT_BYTES } from "../tool.js";
import { FALLBACK_MAX_RESULT_BYTES } from "../tool.js";
import { fakeGoogle, json } from "../fixtures/fake-google.js";
import { createGoogleHttp } from "../google-http.js";
import { DECK, DECK_TEXT, shape, slide } from "./fixtures/slides-rest.js";
@@ -61,11 +61,12 @@ describe("read_presentation tool", () => {
it("refuses a result over the size limit and suggests fields", async () => {
const line = "y".repeat(10_000);
const big = { presentationId: "big", slides: Array.from({ length: Math.ceil(MAX_RESULT_BYTES / 20_000) + 5 }, (_, index) => slide(`s${index}`, [shape(`e${index}`, [{ text: line }])])) };
const big = { presentationId: "big", slides: Array.from({ length: Math.ceil(FALLBACK_MAX_RESULT_BYTES / 20_000) + 5 }, (_, index) => slide(`s${index}`, [shape(`e${index}`, [{ text: line }])])) };
const { fetch } = fakeGoogle(() => json(big));
const result = await readPresentation.call({ presentationId: "big" }, createGoogleHttp({ accessToken: "t", fetch }));
expect(result.isError).toBe(true);
expect((result.content[0] as { text: string }).text).toMatch(/^This presentation is too large to return in one result .*Pass fields to read only part of it/);
expect((result.content[0] as { text: string }).text).toMatch(/\(\d+ KB; the limit is 250 KB\)/);
expect((result.content[0] as { text: string }).text).toMatch(/^The result is [\d,]+ bytes, more than the 250,000 bytes this tool can return, so it was not returned\. Pass fields to read only part of it/);
const fields = await readPresentation.call({ presentationId: "big", fields: ["slides"] }, createGoogleHttp({ accessToken: "t", fetch }));
expect((fields.content[0] as { text: string }).text).toMatch(/so it was not returned\. Ask for fewer fields\.$/);
});
});
@@ -1,8 +1,8 @@
import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
import { z } from "zod";
import { fileIdArg, readWithComments, sizedResult } from "../documents-shared.js";
import { fileIdArg, readWithComments } from "../documents-shared.js";
import type { GoogleRequest } from "../google-http.js";
import { defineTool } from "../tool.js";
import { defineTool, structuredResult } from "../tool.js";
import { presentationPath, SLIDES_HOST, type RestPresentation } from "./presentation.js";
import { renderPresentation } from "./render.js";
@@ -35,10 +35,10 @@ export const COMMENTS_UNAVAILABLE = "Comments are not available for this present
export function readPresentationResult(presentation: RestPresentation, input: ReadPresentationArgs): CallToolResult {
if (input.fields?.length) {
return sizedResult({ content: presentation as Record<string, unknown> }, "presentation", "Ask for fewer fields.");
return structuredResult({ content: presentation as Record<string, unknown> });
}
const content = renderPresentation(presentation, { commentsRequested: input.commentsIncluded === true });
return sizedResult({ content }, "presentation", "Pass fields to read only part of it, for example [\"slides.objectId\", \"slides.pageElements(objectId,shape.text)\"].");
return structuredResult({ content });
}
export const readPresentation = defineTool({
@@ -73,6 +73,7 @@ export const readPresentation = defineTool({
annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: true },
},
args,
whenTooLarge: (input) => input.fields?.length ? "Ask for fewer fields." : "Pass fields to read only part of it, for example [\"slides.objectId\", \"slides.pageElements(objectId,shape.text)\"].",
async run(input, http) {
const plain = input.commentsIncluded ? readPresentationAccessCheckRequest(input) : undefined;
return readPresentationResult(await readWithComments<RestPresentation>(http, readPresentationRequest(input), plain, COMMENTS_UNAVAILABLE), input);
+29 -10
View File
@@ -3,28 +3,44 @@ import type { z } from "zod";
import { GoogleToolError } from "./errors.js";
import type { GoogleHttp } from "./google-http.js";
export const FALLBACK_MAX_RESULT_BYTES = 250_000;
export interface ToolLimits {
maxResultBytes: number;
}
export const FALLBACK_TOOL_LIMITS: ToolLimits = { maxResultBytes: FALLBACK_MAX_RESULT_BYTES };
export interface RestToolSpec<Args> {
definition: Tool;
args: z.ZodType<Args>;
run(args: Args, http: GoogleHttp): Promise<CallToolResult>;
whenTooLarge?: string | ((args: Args) => string);
run(args: Args, http: GoogleHttp, limits: ToolLimits): Promise<CallToolResult>;
}
export interface RestTool {
definition: Tool;
call(rawArgs: unknown, http: GoogleHttp): Promise<CallToolResult>;
call(rawArgs: unknown, http: GoogleHttp, limits?: ToolLimits): Promise<CallToolResult>;
}
export function defineTool<Args>(spec: RestToolSpec<Args>): RestTool {
const readOnly = spec.definition.annotations?.readOnlyHint === true;
if (readOnly !== (spec.whenTooLarge !== undefined)) {
throw new Error(`${spec.definition.name}: a read tool must say how to ask for less, and a write tool must not`);
}
return {
definition: spec.definition,
async call(rawArgs, http) {
async call(rawArgs, http, limits = FALLBACK_TOOL_LIMITS) {
const parsed = spec.args.safeParse(rawArgs ?? {});
if (!parsed.success) {
const detail = parsed.error.issues.map((issue) => `${issue.path.join(".") || "arguments"}: ${issue.message}`).join("; ");
return { isError: true, content: [{ type: "text", text: `Invalid arguments for ${spec.definition.name}: ${detail}` }] };
}
try {
return await spec.run(parsed.data, http);
const result = await spec.run(parsed.data, http, limits);
if (!spec.whenTooLarge) return result;
const advice = typeof spec.whenTooLarge === "string" ? spec.whenTooLarge : spec.whenTooLarge(parsed.data);
return withinResultBudget(result, limits.maxResultBytes, advice);
} catch (error) {
if (error instanceof GoogleToolError) return toolErrorResult(error);
throw error;
@@ -37,17 +53,20 @@ export function structuredResult(value: Record<string, unknown>): CallToolResult
return { content: [{ type: "text", text: JSON.stringify(value) }], structuredContent: value };
}
export const MAX_RESULT_BYTES = 250_000;
export function serializedResultBytes(result: CallToolResult): number {
return Buffer.byteLength(JSON.stringify(result), "utf8");
}
export function boundedStructuredResult(value: Record<string, unknown>, maxBytes: number, tooLarge: (bytes: number) => string): CallToolResult {
const result = structuredResult(value);
export const byteCount = (bytes: number) => bytes.toLocaleString("en-US");
export function resultTooLargeMessage(bytes: number, maxBytes: number, advice: string): string {
return `The result is ${byteCount(bytes)} bytes, more than the ${byteCount(maxBytes)} bytes this tool can return, so it was not returned. ${advice}`;
}
export function withinResultBudget(result: CallToolResult, maxBytes: number, advice: string): CallToolResult {
const bytes = serializedResultBytes(result);
if (bytes > maxBytes) throw new GoogleToolError("incomplete", tooLarge(bytes));
return result;
if (bytes <= maxBytes) return result;
return toolErrorResult(new GoogleToolError("too_large", resultTooLargeMessage(bytes, maxBytes, advice)));
}
export function toolErrorResult(error: GoogleToolError): CallToolResult {
@@ -8,7 +8,7 @@ import { GMAIL_TOOLS } from "./gmail/index.js";
import { SLIDES_TOOLS } from "./slides/index.js";
import { createGoogleHttp, type GoogleFetch } from "./google-http.js";
import { SHEETS_TOOLS } from "./sheets/index.js";
import type { RestTool } from "./tool.js";
import { FALLBACK_MAX_RESULT_BYTES, type RestTool } from "./tool.js";
export const GOOGLE_WORKSPACE_REST_SERVICES = ["gmail", "calendar", "drive", "docs", "slides", "sheets"] as const;
export type GoogleWorkspaceRestService = (typeof GOOGLE_WORKSPACE_REST_SERVICES)[number];
@@ -38,12 +38,15 @@ export interface CallGoogleWorkspaceRestToolInput {
fetch?: GoogleFetch;
signal?: AbortSignal;
timeoutMs?: number;
maxResultBytes?: number;
}
export async function callGoogleWorkspaceRestTool(input: CallGoogleWorkspaceRestToolInput): Promise<CallToolResult> {
const tool = TOOLS[input.service].find((candidate) => candidate.definition.name === input.name);
if (!tool) throw new McpError(ErrorCode.InvalidParams, `Unknown tool: ${input.name}`);
const maxResultBytes = input.maxResultBytes ?? FALLBACK_MAX_RESULT_BYTES;
if (!Number.isSafeInteger(maxResultBytes) || maxResultBytes <= 0) throw new RangeError("maxResultBytes must be a positive integer");
if (!input.accessToken) throw new GoogleWorkspaceAuthError("unauthenticated");
const http = createGoogleHttp({ accessToken: input.accessToken, fetch: input.fetch, signal: input.signal, timeoutMs: input.timeoutMs });
return tool.call(input.arguments, http);
return tool.call(input.arguments, http, { maxResultBytes });
}
@@ -27,7 +27,7 @@ describe("callGoogleWorkspaceRest", () => {
const controller = new AbortController();
const fetch = vi.fn(async (_url: string, _init: RequestInit) => Response.json({ id: "draft1", message: { id: "m", threadId: "t" } }));
const outcome = await callGoogleWorkspaceRest({
service: "gmail", toolName: "create_draft", arguments: { subject: "Hi", body: "Hello" },
service: "gmail", toolName: "create_draft", arguments: { subject: "Hi", body: "Hello" }, maxResultBytes: 1_000_000,
credentialHeaders: { authorization: "Bearer token-1", "x-paperclip-run": "run-1" }, fetch, signal: controller.signal,
});
expect(outcome).toMatchObject({ ok: true, result: { structuredContent: { id: "draft1", subject: "Hi" } } });
@@ -41,7 +41,7 @@ describe("callGoogleWorkspaceRest", () => {
it("returns token problems as outcomes and leaves other failures as tool errors", async () => {
const answer = (status: number, body: unknown) => vi.fn(async () => Response.json(body, { status }));
const call = (fetch: ReturnType<typeof answer>, credentialHeaders: Record<string, string> = { Authorization: "Bearer t" }) =>
callGoogleWorkspaceRest({ service: "gmail", toolName: "get_message", arguments: { messageId: "m1" }, credentialHeaders, fetch });
callGoogleWorkspaceRest({ service: "gmail", toolName: "get_message", arguments: { messageId: "m1" }, maxResultBytes: 1_000_000, credentialHeaders, fetch });
await expect(call(answer(401, {}))).resolves.toEqual({ ok: false, authFailure: "unauthenticated" });
await expect(call(answer(200, {}), {})).resolves.toEqual({ ok: false, authFailure: "unauthenticated" });
await expect(call(answer(403, { error: { errors: [{ reason: "insufficientPermissions" }] } })))
@@ -49,8 +49,25 @@ describe("callGoogleWorkspaceRest", () => {
await expect(call(answer(404, { error: { code: 404 } }))).resolves.toMatchObject({ ok: true, result: { isError: true } });
});
it("passes the gateway's result budget to the REST tools, which enforce exactly that number", async () => {
const body = "a".repeat(30_000);
const message = { id: "m1", threadId: "t1", internalDate: "1763337600000",
payload: { mimeType: "text/plain", headers: [], body: { size: body.length, data: Buffer.from(body).toString("base64url") } } };
const call = (maxResultBytes: number) => callGoogleWorkspaceRest({
service: "gmail", toolName: "get_message", arguments: { messageId: "m1" }, maxResultBytes,
credentialHeaders: { Authorization: "Bearer t" }, fetch: vi.fn(async () => Response.json(message)),
});
const fits = await call(1_000_000);
expect(fits).toMatchObject({ ok: true, result: { structuredContent: { plaintextBody: body } } });
const bytes = Buffer.byteLength(JSON.stringify(fits.ok && fits.result), "utf8");
await expect(call(bytes)).resolves.toMatchObject({ ok: true, result: { structuredContent: { plaintextBody: body } } });
const over = await call(bytes - 1);
expect(over).toMatchObject({ ok: true, result: { isError: true } });
expect(JSON.stringify(over)).toContain(`more than the ${(bytes - 1).toLocaleString("en-US")} bytes this tool can return`);
});
it("refuses a service the REST tools do not implement", async () => {
await expect(callGoogleWorkspaceRest({ service: "chat", toolName: "send_message", arguments: {}, credentialHeaders: {}, fetch: vi.fn() }))
await expect(callGoogleWorkspaceRest({ service: "chat", toolName: "send_message", arguments: {}, maxResultBytes: 1_000_000, credentialHeaders: {}, fetch: vi.fn() }))
.rejects.toThrow("no Paperclip REST tools");
});
});
@@ -42,7 +42,7 @@ export function listGoogleWorkspaceRest(service: GoogleWorkspaceService): Tool[]
}
export async function callGoogleWorkspaceRest(
input: GoogleWorkspaceRestContext & { toolName: string; arguments: unknown },
input: GoogleWorkspaceRestContext & { toolName: string; arguments: unknown; maxResultBytes: number },
): Promise<GoogleWorkspaceRestCallOutcome> {
try {
const result = await callGoogleWorkspaceRestTool({
@@ -52,6 +52,7 @@ export async function callGoogleWorkspaceRest(
accessToken: bearerToken(input.credentialHeaders),
fetch: input.fetch,
signal: input.signal,
maxResultBytes: input.maxResultBytes,
});
return { ok: true, result };
} catch (error) {
+4
View File
@@ -344,6 +344,9 @@ const ACTION_REQUEST_EXECUTION_WAIT_MS = APPROVED_EXECUTION_TIMEOUT_MS + 5_000;
// grace must exceed the normal sign path (approval-snapshot fetch + interaction
// create) so a live create keeps its own row.
const MAX_REMOTE_MCP_RESPONSE_BYTES = 1_000_000;
const MCP_RESULT_FRAMING_BYTES = Buffer.byteLength(
`event: message\ndata: ${JSON.stringify({ jsonrpc: "2.0", id: `paperclip-tool-${randomUUID()}-version`, result: {} })}`, "utf8") - "{}".length;
export const GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES = MAX_REMOTE_MCP_RESPONSE_BYTES - MCP_RESULT_FRAMING_BYTES;
const ACTIVE_GATEWAY_RUN_STATUSES = new Set(["running"]);
// A tool listing decides access for every tool in the company. The decisions
// of one listing share a read cache, so this cap only limits how many of them
@@ -8127,6 +8130,7 @@ export function createToolGatewayService(
service: workspaceTarget.service,
toolName: workspaceTarget.leaf,
arguments: parameters,
maxResultBytes: GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES,
credentialHeaders,
signal,
fetch: (url, init) => options.remoteHttpRequest