Files
PaperClipAI/server/src/google-workspace-rest-tools/documents-shared.ts
T
DottaandClaude Opus 5.5 56023fe80f feat(google-workspace): gateway sets the REST tool result size limit for all six services; update connector docs (#22050)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents read Gmail, Calendar, Drive, Docs, Sheets and Slides through
the built-in Google Workspace connection
> - Paperclip now has its own REST-backed Google tools in
`server/src/google-workspace-rest-tools/`. No service uses them yet
> - That folder guessed a result size limit of 250,000 bytes. It applied
the limit only to Drive, Docs, Sheets and Slides. Gmail and Calendar had
no limit
> - The guess was wrong in both directions. It refused documents that
Paperclip can deliver, and the folder should not know Paperclip's limits
> - This pull request makes the limit a number that the gateway gives.
The folder applies it to every tool in all six services through one
check. It also brings three Google planning docs up to date
> - The benefit is that a REST result is refused only when the gateway
would also refuse the same result from Google's own MCP server

## Linked Issues or Issue Description

Refs #22027, #22028, #22029, #22026

**Is your feature request related to a problem?**
The REST tools refused results over a size the folder chose itself
(250,000 bytes). Paperclip can deliver larger results: the runner saves
a large result as an artifact and gives the agent a pointer. Gmail and
Calendar results had no limit at all.

**Describe the solution you'd like**
The gateway gives the limit. The folder applies the given number to the
final serialized result of every tool, with one helper. A result over
the limit is an error that says how to ask for less. It is never cut
short.

**Describe alternatives you've considered**
A larger number chosen from the runner's 1 GiB capture limit. Google's
own transfer limit (20 MB per response) and memory use make that a poor
choice. Also, results would then differ between the REST path and
Google's MCP path.

### Limits a tool result passes on the way to an agent

| Point | Limit | What happens over it |
| --- | --- | --- |
| Google response read by `google-http.ts` | 20 MB
(`MAX_RESPONSE_BYTES`) | Tool error. Nothing returned |
| REST folder result (new) | the number the gateway passes: 999,885
bytes | Tool error with "read less" advice. The gateway returns
`tool_error` (502) |
| Gateway read of a remote MCP response (`MAX_REMOTE_MCP_RESPONSE_BYTES`
in `tool-gateway.ts`) | 1,000,000 bytes per JSON body or event-stream
message | `mcp_remote_response_too_large` (502). This is the "1,000,000
bytes are rejected" statement in `doc/prompt-injection-prevention.md`.
It applies to the Google MCP path only; the REST path has no HTTP read |
| Injection screening model check | 256 KiB model request | Not a
delivery limit. A larger result is checked by the built-in patterns only
|
| Gateway HTTP route (`POST /tool-gateway/.../tools/call`) | none in
Paperclip | `res.json(result)` |
| Runner (`captureRunnerToolResult`) | 500,000 bytes
(`RUNNER_TOOL_RESULT_MAX_BYTES`) | Saved as an artifact. The agent gets
a pointer and reads it with `read_tool_result` |
| Runner capture | 1 GiB (`RUNNER_API_RESPONSE_MAX_BYTES`) |
`SemanticToolOutcomeUnknownError` |

The gateway wraps the result as `{ content, data: { content,
structuredContent, ... } }`, so the payload occurs again. Measured: a
Gmail result of 676,459 bytes from the folder became 1,017,386 bytes
delivered (ratio 1.50). The ratio cannot be more than 2.

### The budget

`GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES` in `tool-gateway.ts` =
`MAX_REMOTE_MCP_RESPONSE_BYTES` less the JSON-RPC envelope and
event-stream framing (`event: message\ndata:
{"jsonrpc":"2.0","id":"paperclip-tool-<uuid>-version","result":...}`) =
**999,885 bytes**. A result that Google's MCP server could deliver
through the gateway is also delivered over REST, and the reverse. With
the wrapper, the largest delivered result is less than 2 MB. The runner
saves it as an artifact (far below 1 GiB). So the folder never refuses a
result that the runner can give as a pointer.

Practical limits at that budget (plain ASCII; escaped and non-ASCII
characters count more):

| Tool | Largest content |
| --- | --- |
| Gmail `get_message` | about 500 KB plain body, or about 250 KB each
for plain plus HTML |
| Calendar `get_event` | about 500 KB description |
| Drive `read_file_content` | about 500 KB text |
| Drive `download_file_content` | about 375 KB file bytes |
| Docs `read_doc` | about 240 KB document text |
| Slides `read_presentation` | about 250 KB text |
| Sheets `get_values` | about 500 KB cell text, at most 50,000 cells |

## What Changed

- `tool.ts`: `defineTool` takes `whenTooLarge` advice per read tool
(required for reads, refused for writes) and a `ToolLimits` argument. It
measures the final `CallToolResult` once (`withinResultBudget`) and
replaces an oversized result with a `too_large` tool error.
`FALLBACK_MAX_RESULT_BYTES` (250,000) applies only when a direct caller
gives no limit.
- `callGoogleWorkspaceRestTool` and `createGoogleWorkspaceRestServer`
take `maxResultBytes`. A value that is not a positive integer is refused
before any request.
- All 23 read tools state their advice: a smaller `pageSize` for list
tools, `messageFormat` or `view` for Gmail, a narrower range for Sheets,
`fields` for Slides. Write tools are not checked: a completed write is
never reported as an error, so an agent is not led to repeat it.
- Drive, Docs, Slides and Sheets no longer have their own byte checks
(`boundedStructuredResult`, `boundedResult`, `sizedResult`,
`MAX_CONTENT_BYTES` removed). Drive downloads stop reading at the given
limit. The Sheets 50,000-cell limit stays.
- Two Drive tool descriptions no longer state a fixed number.
- Adapter `google-workspace-rest-dispatch.ts`: `maxResultBytes` is a
required input and is passed through.
- `tool-gateway.ts`: new `GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES`,
passed on the REST branch.
- Folder README: new "Result size" section with the limits and costs;
old 250,000-byte text replaced.
- Docs: `doc/memory/google-connector-memory.md`, the production
checklist and the code-fix plan now list #22026 to #22029, the REST
status (36 of 36, none switched on), the November 17 decisions, the
known differences from Google's tools, open risks and next steps.

## Verification

- `cd server && npx vitest run src/google-workspace-rest-tools` passes
(53 files, 1,361 tests with the adapter test).
- New `result-budget.test.ts`: for Gmail (read, list), Calendar (list,
read), Drive, Docs, Slides and Sheets, a result just under a given
budget is returned and one just over is refused with the tool's advice.
Without a budget, the fallback applies; with a larger budget, the same
result is returned. Drive downloads use the given cap. A Gmail list
tells the caller to lower `pageSize`. A Gmail draft result over the
budget is still returned, not an error. `defineTool` refuses a read tool
without advice and a write tool with it.
- `npx vitest run src/services/google-workspace-rest-dispatch.test.ts`:
the adapter passes the gateway's number, and the folder applies exactly
that number.
- `npx vitest run src/__tests__/google-workspace-rest-transport.test.ts`
(24 tests): with Gmail on REST, a 330 KB email passes the gateway with
screening, is more than 500,000 bytes delivered, and
`captureRunnerToolResult` turns it into a `read_tool_result` pointer. A
520 KB email is refused with `tool_error` and the advice, after one
Google request.
- `npx tsc --noEmit -p server/tsconfig.json`: no errors.
- `pnpm verify --explain`: broader verification required (a doc under
`doc/` is a global CI input). CI supplies that coverage.
- Time per call: the check is one `JSON.stringify` of the result. About
5 µs for a 4 KB result, 18 µs for 42 KB, 0.24 ms for 700 KB. Drive,
Docs, Sheets and Slides had the same pass before, so they add nothing.
Gmail and Calendar add this one pass.

## Risks

- Gmail and Calendar results now have a limit. Direct callers that pass
no limit get the 250,000-byte fallback. The gateway always passes its
number.
- Drive, Docs, Slides and Sheets accept results up to about 4 times
larger than before through the gateway. Large results reach agents as
artifact pointers.
- Error text for oversized results changed to one shared form.
- No service is switched to REST, so users see no change today.

## Model Used

Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code with
tool use (shell, file edits, GitHub CLI) and extended reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 13:05:13 -06:00

185 lines
7.9 KiB
TypeScript
Raw Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import { z } from "zod";
import { GoogleToolError } from "./errors.js";
import type { GoogleHttp, GoogleRequest } from "./google-http.js";
export const MAX_BATCH_REQUESTS = 500;
export const OBJECT_PLACEHOLDER = "";
export const fileIdArg = (kind: string) => z.string().regex(/^[A-Za-z0-9_-]{1,256}$/, `Must be a Google ${kind} ID`);
export const revisionIdArg = z.string().min(1).max(1024);
export const batchRequestArg = z.record(z.string(), z.unknown()).refine(
(value) => Object.keys(value).length === 1,
"Each request must name exactly one kind of update, such as insertText",
);
const CONFLICT_STATUSES = new Set(["FAILED_PRECONDITION", "ABORTED"]);
const CONFLICT_REASON = /REVISION|CONDITION_NOT_MET|conditionNotMet/i;
const CONFLICT_MESSAGE = /\brequired revision\b|\brevision id\b|\blatest revision\b/i;
export function isRevisionConflict(error: GoogleToolError): boolean {
if (error.kind !== "invalid_argument" || (error.httpStatus !== 400 && error.httpStatus !== 409)) return false;
if (error.google.status && CONFLICT_STATUSES.has(error.google.status)) return true;
if ((error.google.reasons ?? []).some((reason) => CONFLICT_REASON.test(reason))) return true;
return CONFLICT_MESSAGE.test(error.message);
}
export type RequestKindRule =
| { allowed: true; refuse?: (body: Record<string, unknown>) => string | undefined; example?: Record<string, unknown> }
| { allowed: false; reason: string };
export type RequestKindRules = Readonly<Record<string, RequestKindRule>>;
const isRecord = (value: unknown): value is Record<string, unknown> => typeof value === "object" && value !== null && !Array.isArray(value);
const GOOGLE_FILE_HOSTS = new Set(["drive.google.com", "docs.google.com"]);
export function googleFileUrl(value: unknown): boolean {
if (typeof value !== "string") return false;
try {
return GOOGLE_FILE_HOSTS.has(new URL(value).hostname.toLowerCase());
} catch {
return false;
}
}
export const refuseGoogleFileUrl = (...path: string[]) => (body: Record<string, unknown>): string | undefined => {
let value: unknown = body;
for (const key of path) value = isRecord(value) ? value[key] : undefined;
return googleFileUrl(value) ? `takes its picture from a Google Drive or Docs link (${path.join(".")}), which is another file.` : undefined;
};
export function allowedKinds(rules: RequestKindRules): string[] {
return Object.entries(rules).filter(([, rule]) => rule.allowed).map(([kind]) => kind);
}
export function refusedKinds(rules: RequestKindRules): string[] {
return Object.entries(rules).filter(([, rule]) => !rule.allowed).map(([kind]) => kind);
}
export function assertRequestKinds(requests: ReadonlyArray<Record<string, unknown>>, rules: RequestKindRules): void {
const problems: string[] = [];
for (const [index, request] of requests.entries()) {
const kind = Object.keys(request)[0]!;
const rule = Object.hasOwn(rules, kind) ? rules[kind] : undefined;
if (!rule) problems.push(`requests[${index}].${kind} is not a request kind this tool accepts.`);
else if (!rule.allowed) problems.push(`requests[${index}].${kind} ${rule.reason}`);
else {
const body = request[kind];
const reason = rule.refuse?.(isRecord(body) ? body : {});
if (reason) problems.push(`requests[${index}].${kind} ${reason}`);
}
}
if (problems.length) {
throw new GoogleToolError("invalid_argument", `${problems.join(" ")} This tool only changes the one file it is given, so nothing was changed.`);
}
}
export interface TableCellText {
text: string;
rowSpan?: number;
columnSpan?: number;
row?: number;
column?: number;
}
export const MERGED_LEFT = "(merged left)";
export const MERGED_UP = "(merged up)";
const span = (value: number | undefined) => (value !== undefined && value > 1 ? Math.floor(value) : 1);
export function tableGrid(rows: readonly (readonly TableCellText[])[], columns: number): string[][] {
const grid: Array<Array<string | undefined>> = rows.map(() => []);
for (const [listedRow, cells] of rows.entries()) {
const positional = cells.length === columns && cells.every((cell) => cell.column === undefined);
let column = 0;
for (const [cellIndex, cell] of cells.entries()) {
const rowIndex = cell.row !== undefined && cell.row >= 0 && cell.row < rows.length ? cell.row : listedRow;
if (positional) column = cellIndex;
else if (cell.column !== undefined) column = cell.column;
else while (grid[rowIndex]![column] !== undefined) column += 1;
if (positional && grid[rowIndex]![column] !== undefined && cell.text === "") continue;
grid[rowIndex]![column] = cell.text;
for (let down = 0; down < span(cell.rowSpan) && rowIndex + down < rows.length; down += 1) {
for (let across = 0; across < span(cell.columnSpan); across += 1) {
if (down === 0 && across === 0) continue;
grid[rowIndex + down]![column + across] = down === 0 ? MERGED_LEFT : MERGED_UP;
}
}
column += span(cell.columnSpan);
}
}
const width = Math.max(columns, ...grid.map((row) => row.length));
return grid.map((row) => Array.from({ length: width }, (_, index) => row[index] ?? ""));
}
export function escapeTableCell(text: string): string {
return text.replace(/\\/g, "\\\\").replace(/\|/g, "\\|").replace(/\r?\n/g, "<br>");
}
export function markdownTable(rows: readonly (readonly TableCellText[])[], columns: number): string[] {
if (rows.length === 0) return ["[empty table]"];
const grid = tableGrid(rows, columns).map((cells) => cells.map(escapeTableCell));
const lines = grid.map((cells) => `| ${cells.join(" | ")} |`);
return [lines[0]!, `| ${grid[0]!.map(() => "---").join(" | ")} |`, ...lines.slice(1)];
}
export async function readWithComments<T>(http: GoogleHttp, request: GoogleRequest, plain: GoogleRequest | undefined, unavailable: string): Promise<T> {
try {
return await http.send<T>(request);
} catch (error) {
if (!plain || !(error instanceof GoogleToolError) || error.kind !== "permission_denied") throw error;
await http.send(plain);
throw new GoogleToolError("permission_denied", unavailable, error.httpStatus);
}
}
export async function sendBatchUpdate<T>(http: GoogleHttp, request: GoogleRequest, what: string, readTool: string): Promise<T> {
try {
return await http.send<T>(request);
} catch (error) {
if (error instanceof GoogleToolError && isRevisionConflict(error)) {
throw new GoogleToolError(
"invalid_argument",
`The ${what} is no longer at the revision you read (or that revision ID is not valid), so nothing was changed. Read it again with ${readTool} and retry with its new revisionId.`,
error.httpStatus,
);
}
throw error;
}
}
export interface RestCommentPost {
content?: string;
author?: { displayName?: string; anonymous?: boolean };
deleted?: boolean;
}
export interface RestCommentThread {
commentId?: string;
anchorId?: string;
status?: string;
plainTextQuote?: string;
headPost?: RestCommentPost;
replies?: RestCommentPost[];
}
function postLine(post: RestCommentPost): string {
if (post.deleted) return "(deleted)";
const author = post.author?.displayName || (post.author?.anonymous ? "Anonymous" : "Unknown author");
return `${author}: ${(post.content ?? "").replace(/\s*\n\s*/g, " ").trim()}`;
}
export function renderComments(threads: readonly RestCommentThread[], where: (thread: RestCommentThread) => string | undefined): string[] {
return threads.map((thread) => {
const place = where(thread);
const quote = thread.plainTextQuote ? ` on "${thread.plainTextQuote.replace(/\s+/g, " ").trim()}"` : "";
const state = thread.status === "RESOLVED" ? " (resolved)" : "";
const head = `- ${thread.headPost ? postLine(thread.headPost) : "(no text)"}${quote}${place ? ` [${place}]` : ""}${state}`;
return [head, ...(thread.replies ?? []).map((reply) => ` - ${postLine(reply)}`)].join("\n");
});
}
export function plural(count: number, one: string, many: string): string {
return count === 1 ? `1 ${one}` : `${count} ${many}`;
}