From 37de2b72597b25ac3baebe252595172ea4e77565 Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Wed, 18 Nov 2026 15:24:18 -0600 Subject: [PATCH] fix(tools): keep audit summaries valid jsonb when the cut lands inside an emoji (#22207) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents call connector tools through the tool gateway. The gateway writes an audit row in `tool_call_events` for each call. > - The audit row keeps short request and result summaries in `jsonb` columns. `summarizeToolValue` cuts long summaries to 3997 UTF-16 code units. > - When the cut falls between the two halves of an emoji, the summary ends in a lone high surrogate. PostgreSQL `jsonb` rejects this value. > - The audit insert then fails. The agent gets the database error and not the connector result. > - This pull request makes the cut keep whole characters. > - The benefit is that connector calls with long, emoji-rich results return their result and keep their audit record. ## Linked Issues or Issue Description **What happened?** Some connector tool calls failed with a database error. The error text starts with `Failed query: insert into "tool_call_events"`. The connector call itself worked, but the agent did not get the result. This occurred in a small part of calls (less than 1%). **Expected behavior** The connector call returns its result. Paperclip writes the audit row for the call. **Steps to reproduce** 1. Call a connector tool that returns a long result (more than 4000 characters) with emoji in it. 2. Make sure an emoji is at the summary cut point (UTF-16 code unit 3997 of the serialized value). 3. The `tool_call_events` insert fails with `22P02 invalid input syntax for type json` ("Unicode low surrogate must follow a high surrogate"). The tool call returns that error. **Relevant logs or output** `Failed query: insert into "tool_call_events" (...)` from the gateway `call_completed` audit write. The row has request and result summaries. ## What Changed - `server/src/services/tool-content-guards.ts`: `summarizeToolValue` now cuts long summaries with a small `cutWholeCharacters` helper. If the last kept code unit is a high surrogate, the helper drops it. The summary is still at most 4000 code units. The `sha256` and `sizeBytes` values do not change, because they use the full serialized value. - `server/src/__tests__/tool-content-guards.test.ts`: add a regression test. It puts an emoji across the cut point and checks that the summary has no lone surrogate. ## Verification - Failing before the fix: `pnpm --dir server exec vitest run src/__tests__/tool-content-guards.test.ts -t "without splitting an emoji"` failed. The received summary ended with a lone high surrogate before `...`, and the expected summary did not. - Passing after the fix: the same command passes. The full file passes (47 tests). - Database check: on embedded PostgreSQL, the old summary shape as `jsonb` fails with `22P02` ("Unicode low surrogate must follow a high surrogate"). The new summary shape is accepted. I used a temporary probe for this check. I did not commit it. - `pnpm --dir server typecheck`: no errors in `server`. The local run showed errors only in `cli` and `kv-demo-mcp-server`, because I installed only the server dependencies (low disk space). - Not run: the full `pnpm verify`. `prompt-injection-prevention.test.ts` does not load in my environment (`baseURL must be a non-empty string`). The same error occurs on unchanged `master`. ## Risks - Low risk. The change affects only the readable summary text. A summary can be one code unit shorter when the cut point is inside an emoji. Hashes, sizes, redaction, screening, and authorization do not change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution, in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 5.5 --- server/src/__tests__/tool-content-guards.test.ts | 8 ++++++++ server/src/services/tool-content-guards.ts | 12 +++++++++++- 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/server/src/__tests__/tool-content-guards.test.ts b/server/src/__tests__/tool-content-guards.test.ts index b5e26475ca1..65d054c7412 100644 --- a/server/src/__tests__/tool-content-guards.test.ts +++ b/server/src/__tests__/tool-content-guards.test.ts @@ -48,6 +48,14 @@ describe("tool content guards", () => { expect(identity).not.toBe(createHmac("sha256", signingSecret).update(canonicalToolArguments(first)).digest("hex")); }); + it("truncates audit summaries without splitting an emoji, so jsonb accepts them", () => { + // `{"text":"` is 9 code units, so the cut at 3997 lands between the emoji's surrogate halves. + const value = { text: `${"x".repeat(3987)}${"\u{1F600}".repeat(20)}` }; + const { summary } = summarizeToolValue(value); + expect(summary).toBe(`{"text":"${"x".repeat(3987)}...`); + expect(summary).not.toMatch(/[\uD800-\uDBFF](?![\uDC00-\uDFFF])/); + }); + it("includes values beyond audit preview truncation in request identities", () => { const prefix = "x".repeat(5000); expect(toolRequestIdentityHash({ body: `${prefix}first` })).not.toBe(toolRequestIdentityHash({ body: `${prefix}second` })); diff --git a/server/src/services/tool-content-guards.ts b/server/src/services/tool-content-guards.ts index 8ed7ee1ca31..27c9aeecf16 100644 --- a/server/src/services/tool-content-guards.ts +++ b/server/src/services/tool-content-guards.ts @@ -295,12 +295,22 @@ export function readSignedToolArguments(input: { return readSignedToolArgumentsPayload(input)?.arguments ?? null; } +/** + * Cuts to at most `max` UTF-16 code units without splitting a surrogate pair. + * Audit summaries are stored in `jsonb`, which rejects a lone surrogate, so a + * split emoji would fail the audit insert and with it the tool call. + */ +function cutWholeCharacters(text: string, max: number): string { + const last = text.charCodeAt(max - 1); + return text.slice(0, last >= 0xd800 && last <= 0xdbff ? max - 1 : max); +} + export function summarizeToolValue(value: unknown) { const redacted = isPlainObject(value) ? redactEventPayload(value) : value; const serialized = stableSerialize(redacted); const redactedText = redactSensitiveText(serialized); return { - summary: redactedText.length > 4000 ? `${redactedText.slice(0, 3997)}...` : redactedText, + summary: redactedText.length > 4000 ? `${cutWholeCharacters(redactedText, 3997)}...` : redactedText, sizeBytes: Buffer.byteLength(serialized, "utf8"), sha256: createHash("sha256").update(serialized).digest("hex"), redactedFields: redactedText.includes(REDACTED_EVENT_VALUE) ? ["sensitive_value"] : [],