Files
PaperClipAI/server/src/google-workspace-rest-tools/docs/read-doc.test.ts
T
DottaandClaude Opus 5.5 56023fe80f feat(google-workspace): gateway sets the REST tool result size limit for all six services; update connector docs (#22050)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents read Gmail, Calendar, Drive, Docs, Sheets and Slides through
the built-in Google Workspace connection
> - Paperclip now has its own REST-backed Google tools in
`server/src/google-workspace-rest-tools/`. No service uses them yet
> - That folder guessed a result size limit of 250,000 bytes. It applied
the limit only to Drive, Docs, Sheets and Slides. Gmail and Calendar had
no limit
> - The guess was wrong in both directions. It refused documents that
Paperclip can deliver, and the folder should not know Paperclip's limits
> - This pull request makes the limit a number that the gateway gives.
The folder applies it to every tool in all six services through one
check. It also brings three Google planning docs up to date
> - The benefit is that a REST result is refused only when the gateway
would also refuse the same result from Google's own MCP server

## Linked Issues or Issue Description

Refs #22027, #22028, #22029, #22026

**Is your feature request related to a problem?**
The REST tools refused results over a size the folder chose itself
(250,000 bytes). Paperclip can deliver larger results: the runner saves
a large result as an artifact and gives the agent a pointer. Gmail and
Calendar results had no limit at all.

**Describe the solution you'd like**
The gateway gives the limit. The folder applies the given number to the
final serialized result of every tool, with one helper. A result over
the limit is an error that says how to ask for less. It is never cut
short.

**Describe alternatives you've considered**
A larger number chosen from the runner's 1 GiB capture limit. Google's
own transfer limit (20 MB per response) and memory use make that a poor
choice. Also, results would then differ between the REST path and
Google's MCP path.

### Limits a tool result passes on the way to an agent

| Point | Limit | What happens over it |
| --- | --- | --- |
| Google response read by `google-http.ts` | 20 MB
(`MAX_RESPONSE_BYTES`) | Tool error. Nothing returned |
| REST folder result (new) | the number the gateway passes: 999,885
bytes | Tool error with "read less" advice. The gateway returns
`tool_error` (502) |
| Gateway read of a remote MCP response (`MAX_REMOTE_MCP_RESPONSE_BYTES`
in `tool-gateway.ts`) | 1,000,000 bytes per JSON body or event-stream
message | `mcp_remote_response_too_large` (502). This is the "1,000,000
bytes are rejected" statement in `doc/prompt-injection-prevention.md`.
It applies to the Google MCP path only; the REST path has no HTTP read |
| Injection screening model check | 256 KiB model request | Not a
delivery limit. A larger result is checked by the built-in patterns only
|
| Gateway HTTP route (`POST /tool-gateway/.../tools/call`) | none in
Paperclip | `res.json(result)` |
| Runner (`captureRunnerToolResult`) | 500,000 bytes
(`RUNNER_TOOL_RESULT_MAX_BYTES`) | Saved as an artifact. The agent gets
a pointer and reads it with `read_tool_result` |
| Runner capture | 1 GiB (`RUNNER_API_RESPONSE_MAX_BYTES`) |
`SemanticToolOutcomeUnknownError` |

The gateway wraps the result as `{ content, data: { content,
structuredContent, ... } }`, so the payload occurs again. Measured: a
Gmail result of 676,459 bytes from the folder became 1,017,386 bytes
delivered (ratio 1.50). The ratio cannot be more than 2.

### The budget

`GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES` in `tool-gateway.ts` =
`MAX_REMOTE_MCP_RESPONSE_BYTES` less the JSON-RPC envelope and
event-stream framing (`event: message\ndata:
{"jsonrpc":"2.0","id":"paperclip-tool-<uuid>-version","result":...}`) =
**999,885 bytes**. A result that Google's MCP server could deliver
through the gateway is also delivered over REST, and the reverse. With
the wrapper, the largest delivered result is less than 2 MB. The runner
saves it as an artifact (far below 1 GiB). So the folder never refuses a
result that the runner can give as a pointer.

Practical limits at that budget (plain ASCII; escaped and non-ASCII
characters count more):

| Tool | Largest content |
| --- | --- |
| Gmail `get_message` | about 500 KB plain body, or about 250 KB each
for plain plus HTML |
| Calendar `get_event` | about 500 KB description |
| Drive `read_file_content` | about 500 KB text |
| Drive `download_file_content` | about 375 KB file bytes |
| Docs `read_doc` | about 240 KB document text |
| Slides `read_presentation` | about 250 KB text |
| Sheets `get_values` | about 500 KB cell text, at most 50,000 cells |

## What Changed

- `tool.ts`: `defineTool` takes `whenTooLarge` advice per read tool
(required for reads, refused for writes) and a `ToolLimits` argument. It
measures the final `CallToolResult` once (`withinResultBudget`) and
replaces an oversized result with a `too_large` tool error.
`FALLBACK_MAX_RESULT_BYTES` (250,000) applies only when a direct caller
gives no limit.
- `callGoogleWorkspaceRestTool` and `createGoogleWorkspaceRestServer`
take `maxResultBytes`. A value that is not a positive integer is refused
before any request.
- All 23 read tools state their advice: a smaller `pageSize` for list
tools, `messageFormat` or `view` for Gmail, a narrower range for Sheets,
`fields` for Slides. Write tools are not checked: a completed write is
never reported as an error, so an agent is not led to repeat it.
- Drive, Docs, Slides and Sheets no longer have their own byte checks
(`boundedStructuredResult`, `boundedResult`, `sizedResult`,
`MAX_CONTENT_BYTES` removed). Drive downloads stop reading at the given
limit. The Sheets 50,000-cell limit stays.
- Two Drive tool descriptions no longer state a fixed number.
- Adapter `google-workspace-rest-dispatch.ts`: `maxResultBytes` is a
required input and is passed through.
- `tool-gateway.ts`: new `GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES`,
passed on the REST branch.
- Folder README: new "Result size" section with the limits and costs;
old 250,000-byte text replaced.
- Docs: `doc/memory/google-connector-memory.md`, the production
checklist and the code-fix plan now list #22026 to #22029, the REST
status (36 of 36, none switched on), the November 17 decisions, the
known differences from Google's tools, open risks and next steps.

## Verification

- `cd server && npx vitest run src/google-workspace-rest-tools` passes
(53 files, 1,361 tests with the adapter test).
- New `result-budget.test.ts`: for Gmail (read, list), Calendar (list,
read), Drive, Docs, Slides and Sheets, a result just under a given
budget is returned and one just over is refused with the tool's advice.
Without a budget, the fallback applies; with a larger budget, the same
result is returned. Drive downloads use the given cap. A Gmail list
tells the caller to lower `pageSize`. A Gmail draft result over the
budget is still returned, not an error. `defineTool` refuses a read tool
without advice and a write tool with it.
- `npx vitest run src/services/google-workspace-rest-dispatch.test.ts`:
the adapter passes the gateway's number, and the folder applies exactly
that number.
- `npx vitest run src/__tests__/google-workspace-rest-transport.test.ts`
(24 tests): with Gmail on REST, a 330 KB email passes the gateway with
screening, is more than 500,000 bytes delivered, and
`captureRunnerToolResult` turns it into a `read_tool_result` pointer. A
520 KB email is refused with `tool_error` and the advice, after one
Google request.
- `npx tsc --noEmit -p server/tsconfig.json`: no errors.
- `pnpm verify --explain`: broader verification required (a doc under
`doc/` is a global CI input). CI supplies that coverage.
- Time per call: the check is one `JSON.stringify` of the result. About
5 µs for a 4 KB result, 18 µs for 42 KB, 0.24 ms for 700 KB. Drive,
Docs, Sheets and Slides had the same pass before, so they add nothing.
Gmail and Calendar add this one pass.

## Risks

- Gmail and Calendar results now have a limit. Direct callers that pass
no limit get the 250,000-byte fallback. The gateway always passes its
number.
- Drive, Docs, Slides and Sheets accept results up to about 4 times
larger than before through the gateway. Large results reach agents as
artifact pointers.
- Error text for oversized results changed to one shared form.
- No service is switched to REST, so users see no change today.

## Model Used

Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code with
tool use (shell, file edits, GitHub CLI) and extended reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 13:05:13 -06:00

126 lines
8.6 KiB
TypeScript

import { describe, expect, it } from "vitest";
import { FALLBACK_MAX_RESULT_BYTES } from "../tool.js";
import { fakeGoogle, json } from "../fixtures/fake-google.js";
import { createGoogleHttp } from "../google-http.js";
import { MULTI_TAB_DOCUMENT, MULTI_TAB_TEXT, oneTab, para, run, text } from "./fixtures/docs-rest.js";
import { readDoc, readDocRequest } from "./read-doc.js";
import { COMMENTS_UNAVAILABLE_NOTE, INLINE_NOTE, PREVIEW_NOTE } from "./render.js";
const INLINE = "?includeTabsContent=true&suggestionsViewMode=SUGGESTIONS_INLINE";
const INLINE_WITH_COMMENTS = `${INLINE}&commentsViewMode=COMMENTS_VIEW_MODE_INCLUDED`;
const PREVIEW = "?includeTabsContent=true&suggestionsViewMode=PREVIEW_WITHOUT_SUGGESTIONS";
const denied = () => json({ error: { code: 403, status: "PERMISSION_DENIED", message: "The caller does not have permission" } }, 403);
const SUGGESTED = oneTab([para([run("Keep "), run("new", { suggestedInsertionIds: ["s1"] }), run("\n")])]);
const PREVIEWED = oneTab([text("Keep")]);
function content(result: { structuredContent?: unknown }): Record<string, unknown> {
return (result.structuredContent as { content: Record<string, unknown> }).content;
}
describe("read_doc request", () => {
it.each([
[{ documentId: "1AbC-d_9" }, undefined, { includeTabsContent: true, suggestionsViewMode: "SUGGESTIONS_INLINE" }],
[{ documentId: "1AbC-d_9", commentsIncluded: false }, undefined, { includeTabsContent: true, suggestionsViewMode: "SUGGESTIONS_INLINE" }],
[{ documentId: "1AbC-d_9", commentsIncluded: true }, undefined, { includeTabsContent: true, suggestionsViewMode: "SUGGESTIONS_INLINE", commentsViewMode: "COMMENTS_VIEW_MODE_INCLUDED" }],
[{ documentId: "1AbC-d_9" }, "PREVIEW_WITHOUT_SUGGESTIONS", { includeTabsContent: true, suggestionsViewMode: "PREVIEW_WITHOUT_SUGGESTIONS" }],
[{ documentId: "1AbC-d_9", commentsIncluded: true }, "PREVIEW_WITHOUT_SUGGESTIONS", { includeTabsContent: true, suggestionsViewMode: "PREVIEW_WITHOUT_SUGGESTIONS" }],
] as const)("reads every tab of %j in view %s with an explicit suggestions view", (input, view, query) => {
expect(readDocRequest(input, view)).toEqual({ method: "GET", host: "docs.googleapis.com", path: "/v1/documents/1AbC-d_9", query });
});
});
describe("read_doc tool", () => {
it("reads the document once with suggestions inline and returns its text, structure, view and revision", async () => {
const { fetch, calls } = fakeGoogle(() => json(MULTI_TAB_DOCUMENT));
const result = await readDoc.call({ documentId: "doc-synthetic-multi" }, createGoogleHttp({ accessToken: "t", fetch }));
expect(calls.map((call) => `${call.method} ${call.url.host}${call.url.pathname}${call.url.search}`)).toEqual([
`GET docs.googleapis.com/v1/documents/doc-synthetic-multi${INLINE}`,
]);
expect(content(result)).toMatchObject({ documentId: "doc-synthetic-multi", revisionId: "ALm37BV-synthetic-rev", suggestionsViewMode: "SUGGESTIONS_INLINE", text: MULTI_TAB_TEXT });
expect(JSON.parse((result.content[0] as { text: string }).text)).toEqual(result.structuredContent);
});
it("shows an editor's pending suggestions inline", async () => {
const { fetch, calls } = fakeGoogle(() => json(SUGGESTED));
const result = await readDoc.call({ documentId: "doc-1" }, createGoogleHttp({ accessToken: "t", fetch }));
expect(calls).toHaveLength(1);
expect(content(result)).toMatchObject({ suggestionsViewMode: "SUGGESTIONS_INLINE", text: "Keep {+new+}" });
expect(content(result).notes).toEqual([INLINE_NOTE]);
});
it("reads a viewer's document without suggestions when Google refuses the inline view, and says so", async () => {
const { fetch, calls } = fakeGoogle(({ url }) => (url.searchParams.get("suggestionsViewMode") === "SUGGESTIONS_INLINE" ? denied() : json(PREVIEWED)));
const result = await readDoc.call({ documentId: "doc-1" }, createGoogleHttp({ accessToken: "t", fetch }));
expect(calls.map((call) => call.url.search)).toEqual([INLINE, PREVIEW]);
expect(result.isError).toBeUndefined();
expect(content(result)).toMatchObject({ suggestionsViewMode: "PREVIEW_WITHOUT_SUGGESTIONS", text: "Keep", notes: [PREVIEW_NOTE] });
expect(PREVIEW_NOTE).toMatch(/may not be valid for update_doc/);
});
it("returns a viewer's document without comments when comments were asked for, and says so", async () => {
const { fetch, calls } = fakeGoogle(({ url }) => (url.searchParams.get("suggestionsViewMode") === "SUGGESTIONS_INLINE" ? denied() : json(PREVIEWED)));
const result = await readDoc.call({ documentId: "doc-1", commentsIncluded: true }, createGoogleHttp({ accessToken: "t", fetch }));
expect(calls.map((call) => call.url.search)).toEqual([INLINE_WITH_COMMENTS, PREVIEW]);
expect(content(result)).toMatchObject({
suggestionsViewMode: "PREVIEW_WITHOUT_SUGGESTIONS",
text: "Keep\n\n[Comments]\nNot available with your access to this document.",
notes: [PREVIEW_NOTE, COMMENTS_UNAVAILABLE_NOTE],
});
});
it("returns the real access error when the document cannot be read in either view", async () => {
const { fetch, calls } = fakeGoogle(denied);
const result = await readDoc.call({ documentId: "doc-1" }, createGoogleHttp({ accessToken: "t", fetch }));
expect(result).toEqual({ isError: true, content: [{ type: "text", text: "The caller does not have permission" }] });
expect(calls.map((call) => call.url.search)).toEqual([INLINE, PREVIEW]);
});
it.each([
["a missing document", () => json({ error: { code: 404 } }, 404), "Google could not find that item."],
["a bad request", () => json({ error: { code: 400, message: "Invalid document." } }, 400), "Invalid document."],
["a disabled API", () => json({ error: { code: 403, errors: [{ reason: "accessNotConfigured" }] } }, 403), "This Google API is not enabled for the Google Cloud project behind this connection."],
])("does not fall back for %s", async (_name, answer, message) => {
const { fetch, calls } = fakeGoogle(answer);
const result = await readDoc.call({ documentId: "doc-1" }, createGoogleHttp({ accessToken: "t", fetch }));
expect(result).toEqual({ isError: true, content: [{ type: "text", text: message }] });
expect(calls).toHaveLength(1);
});
it("does not fall back when the grant lacks the Docs permission", async () => {
const { fetch, calls } = fakeGoogle(() => json({ error: { code: 403, details: [{ reason: "ACCESS_TOKEN_SCOPE_INSUFFICIENT" }] } }, 403));
await expect(readDoc.call({ documentId: "doc-1" }, createGoogleHttp({ accessToken: "t", fetch }))).rejects.toMatchObject({ reason: "insufficient_scope" });
expect(calls).toHaveLength(1);
});
it("refuses a malformed document ID without calling Google", async () => {
const { fetch, calls } = fakeGoogle(() => json({}));
const http = createGoogleHttp({ accessToken: "t", fetch });
for (const documentId of ["", "../x", "a/b", "a b", "a:batchUpdate", "x".repeat(257)]) {
expect((await readDoc.call({ documentId }, http)).isError, documentId).toBe(true);
}
expect(calls).toHaveLength(0);
});
it("measures the limit on the whole result, which carries the content twice", async () => {
const under = (count: number) => oneTab(Array.from({ length: count }, () => text("z".repeat(1_000))));
const perParagraph = 2 * 2 * 1_000;
const fits = Math.floor(FALLBACK_MAX_RESULT_BYTES / perParagraph) - 10;
const http = (doc: unknown) => createGoogleHttp({ accessToken: "t", fetch: fakeGoogle(() => json(doc)).fetch });
const ok = await readDoc.call({ documentId: "d" }, http(under(fits)));
expect(ok.isError).toBeUndefined();
expect(Buffer.byteLength(JSON.stringify(ok), "utf8")).toBeLessThanOrEqual(FALLBACK_MAX_RESULT_BYTES);
expect(Buffer.byteLength(JSON.stringify(ok.structuredContent), "utf8")).toBeLessThan(FALLBACK_MAX_RESULT_BYTES / 2 + 1);
expect((await readDoc.call({ documentId: "d" }, http(under(fits + 40)))).isError).toBe(true);
});
it("refuses a result over the size limit with a clear error instead of cutting it short", async () => {
const paragraph = "x".repeat(10_000);
const big = oneTab(Array.from({ length: Math.ceil(FALLBACK_MAX_RESULT_BYTES / 20_000) + 5 }, () => text(paragraph)));
const { fetch } = fakeGoogle(() => json(big));
const result = await readDoc.call({ documentId: "big" }, createGoogleHttp({ accessToken: "t", fetch }));
expect(result.isError).toBe(true);
expect((result.content[0] as { text: string }).text).toMatch(/^The result is [\d,]+ bytes, more than the 250,000 bytes this tool can return, so it was not returned\. This tool cannot read part of a document\./);
expect(result.structuredContent).toBeUndefined();
});
});