From 8424a510181984c753bdf205d8f01056288711dc Mon Sep 17 00:00:00 2001 From: AlexFrontegg <140060214+AlexFrontegg@users.noreply.github.com> Date: Mon, 2 Nov 2026 18:49:08 +0200 Subject: [PATCH] fix(redaction): scan long letter/digit runs in linear time (#16999) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip removes secrets from run logs, tool results and diagnostics with `redactSensitiveText` before it stores or shows them. > - The JSON field-name patterns in that function start with an unanchored `[A-Za-z0-9_-]*`. Each start position inside a long run of letters and digits scans to the end of the run, so the cost grows with the square of the run length. > - Base64 of text is one long run of letters and digits. A 40 KB base64 blob of source code in a shell command takes about 5 s to redact, and a large Gmail RAW tool result does not finish. > - The redaction runs on the main event loop, so one such chunk blocks every request and every run on the instance. > - This pull request makes these scans start only at the start of a run. The scans become linear, and the redacted output does not change. > - The benefit is that large base64 payloads no longer freeze the server, and every secret that was redacted before is still redacted. ## Linked Issues or Issue Description Fixes #15393. The tool content guard runs `redactSensitiveText` on tool results. A Gmail RAW body is base64url of a MIME message, which is one long letter/digit run. `validateToolContent` on a Gmail-shaped result with a 68 KB `raw` field takes 59 s on master and 19 ms with this change; a 5.4 MB `raw` field takes 0.65 s. Thanks @ts-beat for the clear report in #15393. Refs #12309 (false positives in the same patterns; this pull request does not change which text matches). I searched issues and pull requests for "redaction", "redactSensitiveText", "quadratic", "backtracking", "hangs the server" and "100% CPU". #15393 reports the hang. No open pull request fixes it. **Subsystem affected:** secret redaction (`server/src/redaction.ts`, `packages/adapter-utils/src/command-redaction.ts`), used by run logs, tool content guards and run diagnostics. **Problem:** `compactRunLogChunk` redacts each run log chunk before it truncates the chunk. A shell command such as `printf '%s' '' | base64 -d > file.ts` with base64 of source text makes one long letter/digit run. Base64 of ASCII text contains `+` or `/` only where every third byte is `>`, `?`, `~` or DEL, so random base64 does not show the problem. One `redactSensitiveText` call on such a tool-call line takes (local measurement): | Blob | Before | After | | --- | --- | --- | | 4 KB | 66 ms | 1.1 ms | | 8 KB | 272 ms | 0.5 ms | | 16 KB | 1.1 s | 1.0 ms | | 40 KB | 5.4 s | 2.5 ms | | 1 MB | not measured | 36 ms | The `\b`-anchored command patterns (`KEY=value`, JWT, `Authorization`) and the CLI option pattern have the same quadratic cost on dash-separated runs, for example base64url data. **Proposed solution:** start the unanchored field-name scans only where no name character precedes them (`(?` are still redacted, and `my-token is here` is not changed. - Without the change, the timing tests fail: 9.3 s, 9.8 s, 32 s and 60 s against the 1 s budget. The correctness tests pass with and without the change. - Mutation check: I reverted each of the six pattern changes one at a time. Each revert makes at least one test fail. - Differential fuzzing: 1.1 million random inputs built from secret names, quotes, escaped quotes, dashes, dots, colons and JWT parts gave the same output from the old and new `redactSensitiveText`, `redactCommandText` and `redactDiagnosticText`. - `cd server && npx vitest run `: all 33 server test files that use the redaction helpers, with embedded Postgres, 1730 passed. - `npx vitest run --project @paperclipai/adapter-utils packages/adapter-utils/src/command-redaction.test.ts`: 25 passed. - `npx vitest run --project @paperclipai/adapter-utils`, complete suite: 80 of 82 files passed in the parallel run. The 2 other files (`github-launcher-environment`, `sandbox-managed-runtime`) had 5 s timeouts in git-based tests and passed on rerun. ## Risks - Lookbehind assertions need ES2018. `redactCommandText` also runs in the UI bundle. Node and current browsers support lookbehind, and the server (`server/src/log-redaction.ts`) and the UI (`ui/src/components/task-chat/task-chat-attachments.ts`) already use it. - A change to a redaction pattern could leak a secret. The reasoning above, the unchanged existing tests and the differential fuzzing show the same output as before. - Not changed here, found while checking: `redactDiagnosticText` backtracks exponentially on an escaped JSON secret value made of many escaped backslashes with no closing quote, and `redactSensitiveText` redacts a value only up to its first quote of either kind (`{"apiKey":"abc\"tail"}` keeps `tail"`). Both behave the same before and after this pull request. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) by Anthropic, in Claude Code, with tool use (shell, code search, and test runs) and extended thinking. A human directed the design choices. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --- .../src/command-redaction.test.ts | 39 +++++++++++ .../adapter-utils/src/command-redaction.ts | 11 +-- .../src/__tests__/heartbeat-run-log.test.ts | 18 +++++ server/src/__tests__/redaction.test.ts | 70 +++++++++++++++++++ server/src/redaction.ts | 7 +- 5 files changed, 138 insertions(+), 7 deletions(-) diff --git a/packages/adapter-utils/src/command-redaction.test.ts b/packages/adapter-utils/src/command-redaction.test.ts index 321866bf7c7..2afd383a6d9 100644 --- a/packages/adapter-utils/src/command-redaction.test.ts +++ b/packages/adapter-utils/src/command-redaction.test.ts @@ -145,3 +145,42 @@ second-line\" status=401`; expect(output).toContain(REDACTED_COMMAND_TEXT_VALUE); }); }); + +describe("redactCommandText on long runs", () => { + const jwt = + "eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c"; + + function redactWithinTimeBudget(input: string) { + const startedAt = performance.now(); + const output = redactCommandText(input); + expect(performance.now() - startedAt).toBeLessThan(1_000); + return output; + } + + it("keeps redacting options, assignments, and tokens inside dash-separated runs", () => { + const cases = [ + ["x--token flag-secret", `x--token ${REDACTED_COMMAND_TEXT_VALUE}`], + ["--x--token flag-secret", `--x--token ${REDACTED_COMMAND_TEXT_VALUE}`], + [ + "a-b--api-key=assigned-secret --password 'quoted-secret'", + `a-b--api-key=${REDACTED_COMMAND_TEXT_VALUE} --password '${REDACTED_COMMAND_TEXT_VALUE}'`, + ], + ["prefix-MY_API_KEY=env-secret", `prefix-MY_API_KEY=${REDACTED_COMMAND_TEXT_VALUE}`], + [`--${jwt}`, `--${REDACTED_COMMAND_TEXT_VALUE}`], + [`session=${jwt}`, `session=${REDACTED_COMMAND_TEXT_VALUE}`], + ["my-token is here", "my-token is here"], + ]; + + for (const [input, expected] of cases) { + expect(redactCommandText(input)).toBe(expected); + } + }); + + it("scans long letter/digit and dash-separated runs in linear time", () => { + const base64Url = Buffer.from(Array.from({ length: 90_000 }, (_, index) => (index * 7919) % 256)).toString("base64url"); + for (const run of ["a".repeat(120_000), "a-".repeat(60_000), "a--".repeat(40_000), base64Url]) { + expect(redactWithinTimeBudget(`${run} --token run-secret`)).toBe(`${run} --token ${REDACTED_COMMAND_TEXT_VALUE}`); + expect(redactWithinTimeBudget(`${run}--token run-secret`)).toBe(`${run}--token ${REDACTED_COMMAND_TEXT_VALUE}`); + } + }); +}); diff --git a/packages/adapter-utils/src/command-redaction.ts b/packages/adapter-utils/src/command-redaction.ts index 46eeb6f5040..451f0ccf58e 100644 --- a/packages/adapter-utils/src/command-redaction.ts +++ b/packages/adapter-utils/src/command-redaction.ts @@ -11,16 +11,19 @@ export function isPublicExecutorToolSelector(value: string): boolean { return PUBLIC_EXECUTOR_TOOL_SELECTORS.has(value); } -const SECRET_NAME_PATTERN = String.raw`[A-Za-z0-9_-]*(?:api[-_]?key|(?:access[-_]?|auth[-_]?)?token|token|authorization(?:[-_]?code)?|bearer|secrets?|passwd|passwords?|credentials?|jwt|private[-_]?key|cookie|connectionstring)(?:[-_]?(?:value|header|prod(?:uction)?|dev(?:elopment)?|test|staging|primary|secondary))*`; +const SECRET_NAME_WORDS = String.raw`(?:api[-_]?key|(?:access[-_]?|auth[-_]?)?token|token|authorization(?:[-_]?code)?|bearer|secrets?|passwd|passwords?|credentials?|jwt|private[-_]?key|cookie|connectionstring)(?:[-_]?(?:value|header|prod(?:uction)?|dev(?:elopment)?|test|staging|primary|secondary))*`; +const SECRET_NAME_PATTERN = String.raw`[A-Za-z0-9_-]*${SECRET_NAME_WORDS}`; +const SECRET_NAME_START = String.raw`(? { expect(compacted).not.toContain("paperclip-flag-secret"); }); }); + +describe("compactRunLogChunk with long base64 commands", () => { + it("redacts a tool call that writes base64 of source text without stalling", () => { + const typescriptSource = "export const limit = 65536;\nexport function keep(value: string) {\n return value.slice(0, limit);\n}\n"; + const blob = Buffer.from(typescriptSource.repeat(1_000)).toString("base64").slice(0, 48 * 1024); + expect(blob).toMatch(/^[A-Za-z0-9]+$/); + const command = `printf '%s' '${blob}' | base64 -d > src/keep.ts && echo '{"GITHUB_TOKEN":"tool-call-secret"}'`; + const chunk = `${JSON.stringify({ type: "assistant", message: { content: [{ type: "tool_use", input: { command } }] } })}\n`; + + const startedAt = performance.now(); + const compacted = compactRunLogChunk(chunk); + + expect(performance.now() - startedAt).toBeLessThan(1_000); + expect(compacted).toContain(blob); + expect(compacted).not.toContain("tool-call-secret"); + expect(compacted).toContain("***REDACTED***"); + }); +}); diff --git a/server/src/__tests__/redaction.test.ts b/server/src/__tests__/redaction.test.ts index 4ab09bbd80e..5d060d67d1a 100644 --- a/server/src/__tests__/redaction.test.ts +++ b/server/src/__tests__/redaction.test.ts @@ -899,3 +899,73 @@ second-line\" status=401`, }); }); }); + +describe("redaction of long letter/digit runs", () => { + const typescriptSource = [ + "export function compactRunLogChunk(chunk: string, maxChars = 65536) {", + " const normalized = redactSensitiveText(chunk);", + " if (normalized.length <= maxChars) return normalized;", + " return normalized.slice(0, maxChars);", + "}", + "", + ].join("\n"); + + function base64OfTypeScriptSource(length: number) { + const repeatedSource = typescriptSource.repeat(Math.ceil(length / typescriptSource.length)); + return Buffer.from(repeatedSource).toString("base64").slice(0, length); + } + + function redactWithinTimeBudget(input: string) { + const startedAt = performance.now(); + const output = redactSensitiveText(input); + expect(performance.now() - startedAt).toBeLessThan(1_000); + return output; + } + + it("redacts secret field names that carry a prefix in plain and escaped JSON", () => { + const cases = [ + [`{"myApiKey": "plain-prefixed-secret"}`, `{"myApiKey": "${REDACTED_EVENT_VALUE}"}`], + [`{"x_auth_token":"snake-prefixed-secret","status":"ok"}`, `{"x_auth_token":"${REDACTED_EVENT_VALUE}","status":"ok"}`], + [`GITHUB_TOKEN: "upper-prefixed-secret"`, `GITHUB_TOKEN: "${REDACTED_EVENT_VALUE}"`], + [`{'db_password': 'single-quoted-secret'}`, `{'db_password': '${REDACTED_EVENT_VALUE}'}`], + [`text"apiKey":"glued-quote-secret"`, `text"apiKey":"${REDACTED_EVENT_VALUE}"`], + [`aGVsbG8apiKey:"glued-run-secret"`, `aGVsbG8apiKey:"${REDACTED_EVENT_VALUE}"`], + [String.raw`{\"myApiKey\":\"escaped-prefixed-secret\"}`, String.raw`{\"myApiKey\":\"${REDACTED_EVENT_VALUE}\"}`], + [ + String.raw`{\"x_auth_token\":\"escaped-snake-secret\",\"status\":\"ok\"}`, + String.raw`{\"x_auth_token\":\"${REDACTED_EVENT_VALUE}\",\"status\":\"ok\"}`, + ], + [String.raw`aGVsbG8\"GITHUB_TOKEN\":\"escaped-glued-secret\"`, String.raw`aGVsbG8\"GITHUB_TOKEN\":\"${REDACTED_EVENT_VALUE}\"`], + ]; + + for (const [input, expected] of cases) { + expect(redactSensitiveText(input)).toBe(expected); + expect(redactSensitiveText(expected)).toBe(expected); + } + }); + + it("keeps base64 of source text intact and fast while redacting a secret glued to its end", () => { + const blob = base64OfTypeScriptSource(48 * 1024); + expect(blob).toMatch(/^[A-Za-z0-9]+$/); + + const shellCommand = `printf '%s' '${blob}' | base64 -d > src/compact.ts`; + expect(redactWithinTimeBudget(shellCommand)).toBe(shellCommand); + + const toolCallLine = JSON.stringify({ type: "tool_use", input: { command: shellCommand } }); + expect(redactWithinTimeBudget(toolCallLine)).toBe(toolCallLine); + const serializedToolCallLine = JSON.stringify(toolCallLine); + expect(redactWithinTimeBudget(serializedToolCallLine)).toBe(serializedToolCallLine); + + const gluedSecret = redactWithinTimeBudget(`${blob}apiKey:"run-glued-secret"`); + expect(gluedSecret).toBe(`${blob}apiKey:"${REDACTED_EVENT_VALUE}"`); + }); + + it("scans long dash-separated runs in linear time", () => { + for (const run of ["a-".repeat(60_000), "a--".repeat(40_000)]) { + expect(redactWithinTimeBudget(`${run} Authorization: Bearer dash-run-secret`)) + .toBe(`${run} Authorization: ${REDACTED_EVENT_VALUE}`); + expect(redactWithinTimeBudget(`${run}Authorization: Bearer glued-run-secret`)) + .toBe(`${run}Authorization: ${REDACTED_EVENT_VALUE}`); + } + }); +}); diff --git a/server/src/redaction.ts b/server/src/redaction.ts index 28693d13fa0..dd3edf95680 100644 --- a/server/src/redaction.ts +++ b/server/src/redaction.ts @@ -2,6 +2,7 @@ import { redactCommandText } from "@paperclipai/adapter-utils"; import { isPublicExecutorToolSelector, looksLikeCredentialJwt } from "@paperclipai/adapter-utils/command-redaction"; const SECRET_FIELD_NAME_PATTERN = String.raw`[A-Za-z0-9_-]*(?:api[-_]?key|access[-_]?token|auth(?:_?token)?|token|authorization(?:[-_]?code)?|bearer|secrets?|passwd|passwords?|credentials?|jwt|private[-_]?key|cookie|connectionstring|browser[-_]?code|login[-_]?url)(?:[-_]?(?:value|header|prod(?:uction)?|dev(?:elopment)?|test|staging|primary|secondary))*`; +const SECRET_FIELD_NAME_START = String.raw`(?