mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-19 16:29:42 +01:00
46e38e13875123d8be0aff4edcfec255a59029b5
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
ceaf574421 |
feat(google-workspace): REST-backed Drive tools in server/src/google-workspace-rest-tools (#22028)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents use Google Workspace through the tool gateway, and Paperclip is replacing Google's preview MCP servers with its own tools over Google's REST APIs (`server/src/google-workspace-rest-tools/`) > - Gmail and Calendar are done there. Drive has no REST tools yet, so Drive cannot switch to the REST transport > - Drive is the hard service: file content must be exported or downloaded, kept under a size cap, and never sent to another host; search needs a safe query; writes run under the narrow `drive.file` scope > - This pull request adds the eight allowlisted Drive tools, with the same names, inputs and result shapes as Google's Drive MCP tools > - The benefit is that Drive can switch to `paperclip_rest` once the switch is approved, with no broader scope and no new dependency ## Linked Issues or Issue Description Refs #22000, #22014, #22016, #22017 (earlier parts of the same REST tool set). Related open PRs in the same folder: #22027 (Sheets). **What problem does this solve?** The REST tool set has no Drive tools. `googleWorkspaceRestTransportGaps()` blocks a Drive switch while any allowlisted Drive tool is missing. **Proposed solution** Add `drive/` with the eight tools on the allowlist (`toolScopes.drive`), a Drive probe, and the Drive paths in `google-http.ts`. **Alternatives considered** Converting PDFs and Office files to Google Docs to extract text. That is a write, so the read tool does not do it. ## What Changed - `drive/`: `search_files`, `list_recent_files`, `get_file_metadata`, `get_file_permissions`, `read_file_content`, `download_file_content` (reads, `drive.readonly`), `create_file`, `copy_file` (writes, `drive.file`). One file per tool, pure request and result functions, table-driven tests with synthetic fixtures. - `drive/query.ts`: parses Google's MCP search syntax into a tree, checks terms and operators, and writes a new Drive `q` string with every value re-escaped. Adds `trashed = false` unless the query uses `trashed`. Unsupported terms are errors, not ignored. - Content: Google Docs are exported as Markdown (read) or plain text (download), Sheets as CSV of the first sheet (the read text says so), Slides as plain text. Binary files are downloaded and base64 encoded. Every Drive read result is bounded after serialization to 250,000 bytes (the payload is carried twice, in text and structured content); a larger result is an error that says how to read less, never cut-short content. `exportMimeType` must be a format Google lists for the file type, and every request uses the caller's `fileId`. `create_file` and `copy_file` are annotated as non-idempotent writes; an annotations test checks all eight tools. - `google-http.ts`: per-host path prefix lists, required for every host (`www.googleapis.com` → `/calendar/v3/`, `/drive/v3/`, `/upload/drive/v3/`; `sheets.googleapis.com` → `/v4/spreadsheets/`; `gmail.googleapis.com` → `/gmail/v1/users/me/`, which had no prefix before, so any path was accepted on that host); a bounded `download()` for raw bytes; raw upload bodies for multipart uploads; `redirect: "manual"` with any 3xx turned into a tool error; Drive error reasons `exportSizeLimitExceeded` and `appNotAuthorizedToFile` mapped to clear messages. - `google-http.test.ts`: the host and path allowlist test is now one table with Sheets rows (allows `/v4/spreadsheets/{id}` and `/v4/spreadsheets/{id}:batchUpdate`; rejects `/v4/spreadsheets`, `/v4/other`, `/v4/spreadsheets/../other`, and the Sheets path on `www.googleapis.com`) and a structural check that every host in `GOOGLE_API_HOSTS` has at least one allowed and one rejected row. - `errors.ts`: new `too_large` kind. `probe.ts`: Drive probe (`about.get`, `fields=kind`), allowed by `drive.readonly` or `drive.file`. - `allowlist.test.ts`: Drive 8 of 8; only `create_file` and `copy_file` write; no update, delete, trash, move or share tool. - README: Drive section with an endpoint-and-scope table (one Google reference URL per row), content formats, size caps, requests per call, and differences from Google's tools. - Outside the folder, one line: `server/src/services/google-workspace-rest-dispatch.test.ts` used `drive` as its example of an unimplemented service. It now uses `chat`. After merging master (#22026), this file and `google-workspace-rest-transport.test.ts` pass with Drive added; no other change was needed. ## Verification - `cd server && npx vitest run src/google-workspace-rest-tools`: 46 files with the dispatch and transport tests (after merging master with Sheets), 992 tests pass. - `cd server && npx tsc --noEmit -p .`: no errors in the folder. - `pnpm verify`: exit 2 (README is a broad input); required CI supplies the broad checks. - Not run: live calls to Google. All tests use synthetic responses. ## Risks - `copy_file` under `drive.file`: Google does not document whether a token with `drive.readonly` + `drive.file` can copy a file the app never created or opened. If not, the tool returns a clear error and nothing is copied. A live check on the test account is needed before the switch. - `google-http.ts` now uses `redirect: "manual"` for every request. A redirect from Gmail or Calendar was a retried network error before and is now a clear tool error. Google's APIs do not redirect these calls. - Drive stays on `google_mcp`; nothing calls these tools until the switch is flipped. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits, web fetch of Google's public docs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
f61b919701 |
feat(google-workspace): REST-backed Docs and Slides tools; mark Calendar update_event destructive (#22029)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Google Workspace connector gives agents Google tools through the tool gateway > - Google's hosted Workspace MCP servers are a Developer Preview, so Paperclip launches on its own tools over Google's REST APIs, in `server/src/google-workspace-rest-tools/` > - Gmail and Calendar are done there; Docs and Slides have no tools yet > - This pull request adds `read_doc`, `update_doc`, `read_presentation` and `update_presentation` with the same names and inputs as Google's MCP tools > - The benefit is that the Docs and Slides services can switch to `paperclip_rest` once their API is enabled, and reviewers see a full document in Paperclip ## Linked Issues or Issue Description **Subsystem affected** Google Workspace REST tool set (`server/src/google-workspace-rest-tools/`), Docs and Slides services. **Problem or motivation** The REST tool set has no Docs or Slides tools. The allowlist (`toolScopes.docs`, `toolScopes.slides`) names two tools for each service. A service cannot switch to `paperclip_rest` until all of its allowlisted tools exist. **Proposed solution** Add the four tools in new `docs/` and `slides/` folders. Each tool makes exactly one REST call. The read tools return the whole file as Markdown-style text plus an index structure for edits. The update tools post `batchUpdate` requests in order and require the revision ID that the caller read. **Alternatives considered** Return the raw `documents.get` JSON. It is 10 to 20 times larger than the text, so ordinary documents go over the result size limit, and it is hard to compare with Google Docs. Reorder edit requests by index. That changes what dependent requests do, and Google's tool does not do it. Refs #22000, #22014, #22017 ## What Changed - `docs/`: `read_doc` (`documents.get` with `includeTabsContent=true`) and `update_doc` (`documents.batchUpdate`), a renderer for every tab, child tab, heading, list (real numbering), table, link, header, footer, footnote, chip, suggestion and comment, and an index structure where each paragraph's `text.length` equals `endIndex - startIndex`. - `slides/`: `read_presentation` (`presentations.get`; with `fields` it returns Google's resource as is) and `update_presentation` (`presentations.batchUpdate`), a renderer for every slide's text, tables, groups, media and speaker notes. - `documents-shared.ts`: the 250,000-byte limit on the final serialized result (an error, never a cut-short result), the revision-conflict error, comment rendering, and argument checks. - Update tools require `writeControl.requiredRevisionId` and refuse `targetRevisionId`. A revision conflict returns "nothing was changed; read it again". Writes are never retried. - `google-http.ts`: allows `docs.googleapis.com` only under `/v1/documents/` and `slides.googleapis.com` only under `/v1/presentations/`. - `tools.ts`, `probe.ts`: register the services. Docs and Slides have no network probe, because every read needs a file ID. - Update tools accept only a closed list of batch request kinds (`docs/request-kinds.ts`, `slides/request-kinds.ts`). Kinds and forms that read another file are refused by name: `insertRichLink`, the Sheets chart kinds, `importSlides`, `createAudio`, Drive videos, and images from Drive or Docs links. Unknown kinds are refused too. - `read_doc` always requests `SUGGESTIONS_INLINE` and falls back to an explicit `PREVIEW_WITHOUT_SUGGESTIONS` read for view-only users; the result says which view it used. - Tables with merged cells are placed on the full grid; covered cells read `(merged left)` or `(merged up)`. - Revision conflicts are recognized from Google's `error.status` and `error.details` first, with the message as a fallback. - Annotations: `update_doc` and `update_presentation` are destructive and not idempotent. **Calendar `update_event` is now also marked destructive** (it overwrites fields, removes attendees and replaces reminders). Google's references mark both non-destructive; the README records this. - `README.md`: endpoint and scope table with Google's documentation links, the result format, what cannot be shown, update order and revision rules, and every difference from Google's tools. ## Verification - `cd server && npx vitest run src/google-workspace-rest-tools`: 33 files, 573 tests pass (169 new in `docs/` and `slides/`, plus host, probe and allowlist tests). - `cd server && pnpm typecheck`: passes. - `pnpm verify`: exit 2 (README is a broad input), so required CI supplies broad coverage. - `src/services/google-workspace-rest-dispatch.test.ts` fails the same way on `master` (it still expects Gmail to be incomplete after #22014). This PR does not touch it. ## Risks - Low risk. Nothing calls these tools until the Docs or Slides transport is set to `paperclip_rest`; both stay `google_mcp`. - The inner format of `content` differs from Google's tool, which Google does not publish. The README lists this. - Revision-conflict detection matches Google's 400 message on "revision". Any 400 rejects the whole batch, so nothing changes in either case. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
a29658b5b7 |
feat(google-workspace): REST-backed Gmail tools in server/src/google-workspace-rest-tools (#22000)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents use Google Workspace through the built-in Google Workspace connector > - Today the connector sends each tool call to Google's hosted Workspace MCP servers > - Those servers are a Developer Preview, so we cannot offer them to the public > - The owner decided to launch on our own tools over Google's GA REST APIs, with the same tool names, inputs and results, in one folder that we can delete later > - This pull request adds that folder inside the server, with its first three Gmail tools, and a written trace of where it will plug in > - The benefit is a public-ready Gmail path that we can switch back to Google's MCP servers with one setting, with no new package to publish ## Linked Issues or Issue Description Refs #21996 (plan and working memory for the Google Workspace connector, section 9) Refs #15412 (an earlier community Gmail REST MCP server; this code follows Google's MCP tool contract instead) **Subsystem affected** Connections: the Google Workspace connector and the tool gateway. **Problem or motivation** Google's Workspace MCP servers are a Developer Preview. We cannot launch the Google Workspace connector to the public on them. **Proposed solution** Add `server/src/google-workspace-rest-tools/`. It exposes the same tools as Google's MCP servers over the GA REST APIs. It knows nothing about Paperclip. A later pull request selects it per service with one switch. **Alternatives considered** A separate npm package (it would have to be published, because the server is published) and a loopback or hosted HTTP MCP endpoint. The README explains why an in-process call is better: speed, token handling, and the egress guard, which refuses loopback addresses on public deployments. ## What Changed - New folder `server/src/google-workspace-rest-tools/` inside the existing server package. No new npm package, no new dependency, no workspace, Docker, lockfile or release change. Every changed file is in this folder. - `index.ts` is the only entry point. `boundary.test.ts` enforces that the folder imports only its own files, Node built-ins, `@modelcontextprotocol/sdk` and `zod` (both existing server dependencies; tests may also import `vitest` and `@paperclipai/shared`), and that no other server code imports the folder yet. - `google-http.ts` is the only code that calls `googleapis.com`. It sets the bearer header, allows only listed hosts, has a 20 s deadline and a 20 MB response limit, and retries a read once. It never retries a write. Every write that fails after it is sent says that the change may or may not have been made. - Token errors are thrown as `GoogleWorkspaceAuthError` (an `McpError`). 401 gives `unauthenticated`. 403 insufficient scope gives `insufficient_scope`, the code the gateway already recognizes. Other Google errors become tool error results. - Gmail tools `get_message`, `search_threads` and `create_draft`. Names, fields, types and enums follow Google's Gmail MCP reference. `create_draft` calls only `drafts.create`. There is no send tool. - `get_message` reads bodies that Gmail stores as attachments, and returns an explicit error, never a body-less `FULL_CONTENT` result. Address lists are parsed by a single-pass RFC 5322 tokenizer (`gmail/address.ts`, table-driven tests including IPv6 domain literals and malformed input); quoted search text and long draft headers are handled. - `server.ts`: a thin MCP server (`tools/list`, `tools/call`). `callGoogleWorkspaceRestTool()` is the in-process equivalent. - `allowlist.test.ts` fails if a tool is not in `toolScopes` in `packages/shared/src/google-workspace-connectors.ts`, if its read-only hint disagrees with the allowlist, or if a description names a tool the folder does not list. - `README.md`: purpose, boundary rules, the dispatch trace, the recommended in-process seam and the files the switch PR touches, known differences from Google's tools, and the removal procedure (delete the folder, set the switch to `google_mcp`). - No code comments. Explanations are in the README. ## Verification - `cd server && npx vitest run src/google-workspace-rest-tools`: 8 files, 98 tests pass. Tests use synthetic REST fixtures and a fake fetch. No network. - Each review fix has a test that fails on the revision before it (10 tests red on `d65b51d245`, 3 more red on `5d7d52ec86`). The boundary test fails when a probe file in the folder imports `../errors.js`. - `tsc --noEmit` with the server's `tsconfig.json` settings over the folder passes. - `pnpm check:module-boundaries` passes. - `pnpm verify` exits 2 because a new file under `server/src` is a broad input. Required CI supplies the broad coverage. - Local timing probe with a stubbed fetch: about 13 µs overhead per call. ## Risks - Low risk. Nothing imports the folder yet, so runtime behavior does not change. - The tool descriptions are our own wording. A contract test against a recorded `tools/list` from Google's servers comes in a later pull request. - The allowlist names three Gmail tools (`get_draft`, `list_threads`, `search_messages`) that Google's published Gmail MCP reference does not list. The README flags this. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, web fetch of Google's public reference pages). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |