mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-21 10:52:06 +01:00
46e38e13875123d8be0aff4edcfec255a59029b5
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
56023fe80f |
feat(google-workspace): gateway sets the REST tool result size limit for all six services; update connector docs (#22050)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents read Gmail, Calendar, Drive, Docs, Sheets and Slides through the built-in Google Workspace connection > - Paperclip now has its own REST-backed Google tools in `server/src/google-workspace-rest-tools/`. No service uses them yet > - That folder guessed a result size limit of 250,000 bytes. It applied the limit only to Drive, Docs, Sheets and Slides. Gmail and Calendar had no limit > - The guess was wrong in both directions. It refused documents that Paperclip can deliver, and the folder should not know Paperclip's limits > - This pull request makes the limit a number that the gateway gives. The folder applies it to every tool in all six services through one check. It also brings three Google planning docs up to date > - The benefit is that a REST result is refused only when the gateway would also refuse the same result from Google's own MCP server ## Linked Issues or Issue Description Refs #22027, #22028, #22029, #22026 **Is your feature request related to a problem?** The REST tools refused results over a size the folder chose itself (250,000 bytes). Paperclip can deliver larger results: the runner saves a large result as an artifact and gives the agent a pointer. Gmail and Calendar results had no limit at all. **Describe the solution you'd like** The gateway gives the limit. The folder applies the given number to the final serialized result of every tool, with one helper. A result over the limit is an error that says how to ask for less. It is never cut short. **Describe alternatives you've considered** A larger number chosen from the runner's 1 GiB capture limit. Google's own transfer limit (20 MB per response) and memory use make that a poor choice. Also, results would then differ between the REST path and Google's MCP path. ### Limits a tool result passes on the way to an agent | Point | Limit | What happens over it | | --- | --- | --- | | Google response read by `google-http.ts` | 20 MB (`MAX_RESPONSE_BYTES`) | Tool error. Nothing returned | | REST folder result (new) | the number the gateway passes: 999,885 bytes | Tool error with "read less" advice. The gateway returns `tool_error` (502) | | Gateway read of a remote MCP response (`MAX_REMOTE_MCP_RESPONSE_BYTES` in `tool-gateway.ts`) | 1,000,000 bytes per JSON body or event-stream message | `mcp_remote_response_too_large` (502). This is the "1,000,000 bytes are rejected" statement in `doc/prompt-injection-prevention.md`. It applies to the Google MCP path only; the REST path has no HTTP read | | Injection screening model check | 256 KiB model request | Not a delivery limit. A larger result is checked by the built-in patterns only | | Gateway HTTP route (`POST /tool-gateway/.../tools/call`) | none in Paperclip | `res.json(result)` | | Runner (`captureRunnerToolResult`) | 500,000 bytes (`RUNNER_TOOL_RESULT_MAX_BYTES`) | Saved as an artifact. The agent gets a pointer and reads it with `read_tool_result` | | Runner capture | 1 GiB (`RUNNER_API_RESPONSE_MAX_BYTES`) | `SemanticToolOutcomeUnknownError` | The gateway wraps the result as `{ content, data: { content, structuredContent, ... } }`, so the payload occurs again. Measured: a Gmail result of 676,459 bytes from the folder became 1,017,386 bytes delivered (ratio 1.50). The ratio cannot be more than 2. ### The budget `GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES` in `tool-gateway.ts` = `MAX_REMOTE_MCP_RESPONSE_BYTES` less the JSON-RPC envelope and event-stream framing (`event: message\ndata: {"jsonrpc":"2.0","id":"paperclip-tool-<uuid>-version","result":...}`) = **999,885 bytes**. A result that Google's MCP server could deliver through the gateway is also delivered over REST, and the reverse. With the wrapper, the largest delivered result is less than 2 MB. The runner saves it as an artifact (far below 1 GiB). So the folder never refuses a result that the runner can give as a pointer. Practical limits at that budget (plain ASCII; escaped and non-ASCII characters count more): | Tool | Largest content | | --- | --- | | Gmail `get_message` | about 500 KB plain body, or about 250 KB each for plain plus HTML | | Calendar `get_event` | about 500 KB description | | Drive `read_file_content` | about 500 KB text | | Drive `download_file_content` | about 375 KB file bytes | | Docs `read_doc` | about 240 KB document text | | Slides `read_presentation` | about 250 KB text | | Sheets `get_values` | about 500 KB cell text, at most 50,000 cells | ## What Changed - `tool.ts`: `defineTool` takes `whenTooLarge` advice per read tool (required for reads, refused for writes) and a `ToolLimits` argument. It measures the final `CallToolResult` once (`withinResultBudget`) and replaces an oversized result with a `too_large` tool error. `FALLBACK_MAX_RESULT_BYTES` (250,000) applies only when a direct caller gives no limit. - `callGoogleWorkspaceRestTool` and `createGoogleWorkspaceRestServer` take `maxResultBytes`. A value that is not a positive integer is refused before any request. - All 23 read tools state their advice: a smaller `pageSize` for list tools, `messageFormat` or `view` for Gmail, a narrower range for Sheets, `fields` for Slides. Write tools are not checked: a completed write is never reported as an error, so an agent is not led to repeat it. - Drive, Docs, Slides and Sheets no longer have their own byte checks (`boundedStructuredResult`, `boundedResult`, `sizedResult`, `MAX_CONTENT_BYTES` removed). Drive downloads stop reading at the given limit. The Sheets 50,000-cell limit stays. - Two Drive tool descriptions no longer state a fixed number. - Adapter `google-workspace-rest-dispatch.ts`: `maxResultBytes` is a required input and is passed through. - `tool-gateway.ts`: new `GOOGLE_WORKSPACE_REST_MAX_RESULT_BYTES`, passed on the REST branch. - Folder README: new "Result size" section with the limits and costs; old 250,000-byte text replaced. - Docs: `doc/memory/google-connector-memory.md`, the production checklist and the code-fix plan now list #22026 to #22029, the REST status (36 of 36, none switched on), the November 17 decisions, the known differences from Google's tools, open risks and next steps. ## Verification - `cd server && npx vitest run src/google-workspace-rest-tools` passes (53 files, 1,361 tests with the adapter test). - New `result-budget.test.ts`: for Gmail (read, list), Calendar (list, read), Drive, Docs, Slides and Sheets, a result just under a given budget is returned and one just over is refused with the tool's advice. Without a budget, the fallback applies; with a larger budget, the same result is returned. Drive downloads use the given cap. A Gmail list tells the caller to lower `pageSize`. A Gmail draft result over the budget is still returned, not an error. `defineTool` refuses a read tool without advice and a write tool with it. - `npx vitest run src/services/google-workspace-rest-dispatch.test.ts`: the adapter passes the gateway's number, and the folder applies exactly that number. - `npx vitest run src/__tests__/google-workspace-rest-transport.test.ts` (24 tests): with Gmail on REST, a 330 KB email passes the gateway with screening, is more than 500,000 bytes delivered, and `captureRunnerToolResult` turns it into a `read_tool_result` pointer. A 520 KB email is refused with `tool_error` and the advice, after one Google request. - `npx tsc --noEmit -p server/tsconfig.json`: no errors. - `pnpm verify --explain`: broader verification required (a doc under `doc/` is a global CI input). CI supplies that coverage. - Time per call: the check is one `JSON.stringify` of the result. About 5 µs for a 4 KB result, 18 µs for 42 KB, 0.24 ms for 700 KB. Drive, Docs, Sheets and Slides had the same pass before, so they add nothing. Gmail and Calendar add this one pass. ## Risks - Gmail and Calendar results now have a limit. Direct callers that pass no limit get the 250,000-byte fallback. The gateway always passes its number. - Drive, Docs, Slides and Sheets accept results up to about 4 times larger than before through the gateway. Large results reach agents as artifact pointers. - Error text for oversized results changed to one shared form. - No service is switched to REST, so users see no change today. ## Model Used Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code with tool use (shell, file edits, GitHub CLI) and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
f61b919701 |
feat(google-workspace): REST-backed Docs and Slides tools; mark Calendar update_event destructive (#22029)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Google Workspace connector gives agents Google tools through the tool gateway > - Google's hosted Workspace MCP servers are a Developer Preview, so Paperclip launches on its own tools over Google's REST APIs, in `server/src/google-workspace-rest-tools/` > - Gmail and Calendar are done there; Docs and Slides have no tools yet > - This pull request adds `read_doc`, `update_doc`, `read_presentation` and `update_presentation` with the same names and inputs as Google's MCP tools > - The benefit is that the Docs and Slides services can switch to `paperclip_rest` once their API is enabled, and reviewers see a full document in Paperclip ## Linked Issues or Issue Description **Subsystem affected** Google Workspace REST tool set (`server/src/google-workspace-rest-tools/`), Docs and Slides services. **Problem or motivation** The REST tool set has no Docs or Slides tools. The allowlist (`toolScopes.docs`, `toolScopes.slides`) names two tools for each service. A service cannot switch to `paperclip_rest` until all of its allowlisted tools exist. **Proposed solution** Add the four tools in new `docs/` and `slides/` folders. Each tool makes exactly one REST call. The read tools return the whole file as Markdown-style text plus an index structure for edits. The update tools post `batchUpdate` requests in order and require the revision ID that the caller read. **Alternatives considered** Return the raw `documents.get` JSON. It is 10 to 20 times larger than the text, so ordinary documents go over the result size limit, and it is hard to compare with Google Docs. Reorder edit requests by index. That changes what dependent requests do, and Google's tool does not do it. Refs #22000, #22014, #22017 ## What Changed - `docs/`: `read_doc` (`documents.get` with `includeTabsContent=true`) and `update_doc` (`documents.batchUpdate`), a renderer for every tab, child tab, heading, list (real numbering), table, link, header, footer, footnote, chip, suggestion and comment, and an index structure where each paragraph's `text.length` equals `endIndex - startIndex`. - `slides/`: `read_presentation` (`presentations.get`; with `fields` it returns Google's resource as is) and `update_presentation` (`presentations.batchUpdate`), a renderer for every slide's text, tables, groups, media and speaker notes. - `documents-shared.ts`: the 250,000-byte limit on the final serialized result (an error, never a cut-short result), the revision-conflict error, comment rendering, and argument checks. - Update tools require `writeControl.requiredRevisionId` and refuse `targetRevisionId`. A revision conflict returns "nothing was changed; read it again". Writes are never retried. - `google-http.ts`: allows `docs.googleapis.com` only under `/v1/documents/` and `slides.googleapis.com` only under `/v1/presentations/`. - `tools.ts`, `probe.ts`: register the services. Docs and Slides have no network probe, because every read needs a file ID. - Update tools accept only a closed list of batch request kinds (`docs/request-kinds.ts`, `slides/request-kinds.ts`). Kinds and forms that read another file are refused by name: `insertRichLink`, the Sheets chart kinds, `importSlides`, `createAudio`, Drive videos, and images from Drive or Docs links. Unknown kinds are refused too. - `read_doc` always requests `SUGGESTIONS_INLINE` and falls back to an explicit `PREVIEW_WITHOUT_SUGGESTIONS` read for view-only users; the result says which view it used. - Tables with merged cells are placed on the full grid; covered cells read `(merged left)` or `(merged up)`. - Revision conflicts are recognized from Google's `error.status` and `error.details` first, with the message as a fallback. - Annotations: `update_doc` and `update_presentation` are destructive and not idempotent. **Calendar `update_event` is now also marked destructive** (it overwrites fields, removes attendees and replaces reminders). Google's references mark both non-destructive; the README records this. - `README.md`: endpoint and scope table with Google's documentation links, the result format, what cannot be shown, update order and revision rules, and every difference from Google's tools. ## Verification - `cd server && npx vitest run src/google-workspace-rest-tools`: 33 files, 573 tests pass (169 new in `docs/` and `slides/`, plus host, probe and allowlist tests). - `cd server && pnpm typecheck`: passes. - `pnpm verify`: exit 2 (README is a broad input), so required CI supplies broad coverage. - `src/services/google-workspace-rest-dispatch.test.ts` fails the same way on `master` (it still expects Gmail to be incomplete after #22014). This PR does not touch it. ## Risks - Low risk. Nothing calls these tools until the Docs or Slides transport is set to `paperclip_rest`; both stay `google_mcp`. - The inner format of `content` differs from Google's tool, which Google does not publish. The README lists this. - Revision-conflict detection matches Google's 400 message on "revision". Any 400 rejects the whole batch, so nothing changes in either case. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |