Files
PaperClipAI/packages/adapter-utils/src/command-redaction.test.ts
T
AlexFrontegg 8424a51018 fix(redaction): scan long letter/digit runs in linear time (#16999)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip removes secrets from run logs, tool results and
diagnostics with `redactSensitiveText` before it stores or shows them.
> - The JSON field-name patterns in that function start with an
unanchored `[A-Za-z0-9_-]*`. Each start position inside a long run of
letters and digits scans to the end of the run, so the cost grows with
the square of the run length.
> - Base64 of text is one long run of letters and digits. A 40 KB base64
blob of source code in a shell command takes about 5 s to redact, and a
large Gmail RAW tool result does not finish.
> - The redaction runs on the main event loop, so one such chunk blocks
every request and every run on the instance.
> - This pull request makes these scans start only at the start of a
run. The scans become linear, and the redacted output does not change.
> - The benefit is that large base64 payloads no longer freeze the
server, and every secret that was redacted before is still redacted.

## Linked Issues or Issue Description

Fixes #15393. The tool content guard runs `redactSensitiveText` on tool
results. A Gmail RAW body is base64url of a MIME message, which is one
long letter/digit run. `validateToolContent` on a Gmail-shaped result
with a 68 KB `raw` field takes 59 s on master and 19 ms with this
change; a 5.4 MB `raw` field takes 0.65 s.

Thanks @ts-beat for the clear report in #15393.

Refs #12309 (false positives in the same patterns; this pull request
does not change which text matches).

I searched issues and pull requests for "redaction",
"redactSensitiveText", "quadratic", "backtracking", "hangs the server"
and "100% CPU". #15393 reports the hang. No open pull request fixes it.

**Subsystem affected:** secret redaction (`server/src/redaction.ts`,
`packages/adapter-utils/src/command-redaction.ts`), used by run logs,
tool content guards and run diagnostics.

**Problem:** `compactRunLogChunk` redacts each run log chunk before it
truncates the chunk. A shell command such as `printf '%s' '<base64>' |
base64 -d > file.ts` with base64 of source text makes one long
letter/digit run. Base64 of ASCII text contains `+` or `/` only where
every third byte is `>`, `?`, `~` or DEL, so random base64 does not show
the problem. One `redactSensitiveText` call on such a tool-call line
takes (local measurement):

| Blob | Before | After |
| --- | --- | --- |
| 4 KB | 66 ms | 1.1 ms |
| 8 KB | 272 ms | 0.5 ms |
| 16 KB | 1.1 s | 1.0 ms |
| 40 KB | 5.4 s | 2.5 ms |
| 1 MB | not measured | 36 ms |

The `\b`-anchored command patterns (`KEY=value`, JWT, `Authorization`)
and the CLI option pattern have the same quadratic cost on
dash-separated runs, for example base64url data.

**Proposed solution:** start the unanchored field-name scans only where
no name character precedes them (`(?<![A-Za-z0-9_-])`). A prefixed name
such as `myApiKey`, `x_auth_token` or `GITHUB_TOKEN` still matches,
because the scan from the start of the run absorbs the prefix.

**Alternatives considered:**

- Truncate or cap the input before redaction. Rejected: a cut can split
a secret and leave part of it unredacted, and the scan stays quadratic
below the cap.
- Run the scan in a worker thread with a deadline, as #15393 suggests.
That limits the damage, but the CPU cost stays, and a timeout must then
drop or block the content. It is still useful as an additional defense.

## What Changed

- `server/src/redaction.ts`: `JSON_SECRET_FIELD_TEXT_RE` and
`ESCAPED_JSON_SECRET_FIELD_TEXT_RE` start a field name only where no
name character precedes it. The lookbehind comes after the optional
opening quote, so quoted keys behave as before. The `Authorization` word
scan uses the same start instead of `\b`.
- `packages/adapter-utils/src/command-redaction.ts`: the `KEY=value`
pattern uses the same start. The JWT pattern starts only at the first
word character of a run. A CLI option name can no longer span `--`, so
each `--` starts its own short scan.
- Redaction still runs before truncation. `compactRunLogChunk` does not
change.

Why the output does not change: when an old pattern could match from the
middle of a run, it could also match from the start of that run, because
the `[A-Za-z0-9_-]*` prefix absorbs the extra characters. The engine
tries the start of the run first. So the old patterns never matched from
the middle of a run, and the new start condition only skips positions
that cannot produce a match. The old JWT pattern also always matched
from the first word character of a run. A CLI option name always ends at
the end of its run, so the match from the last `--` in the run ends at
the same place.

## Screenshots

No UI change.

## Verification

- New tests:
- prefixed secret field names in plain, single-quoted and escaped JSON
are still redacted, also when the name is glued to the end of a long
run;
- a 48 KB base64-of-source tool call through `compactRunLogChunk` stays
intact, the secret in the same command is redacted, and the call
finishes within 1 s;
- `redactSensitiveText` and `redactCommandText` handle 48-120 KB
letter/digit, dash-separated, `--`-separated and base64url runs within 1
s, and redact the secret that follows each run;
- `x--token`, `--x--token`, `a-b--api-key=`, `prefix-MY_API_KEY=` and
`--<jwt>` are still redacted, and `my-token is here` is not changed.
- Without the change, the timing tests fail: 9.3 s, 9.8 s, 32 s and 60 s
against the 1 s budget. The correctness tests pass with and without the
change.
- Mutation check: I reverted each of the six pattern changes one at a
time. Each revert makes at least one test fail.
- Differential fuzzing: 1.1 million random inputs built from secret
names, quotes, escaped quotes, dashes, dots, colons and JWT parts gave
the same output from the old and new `redactSensitiveText`,
`redactCommandText` and `redactDiagnosticText`.
- `cd server && npx vitest run <files>`: all 33 server test files that
use the redaction helpers, with embedded Postgres, 1730 passed.
- `npx vitest run --project @paperclipai/adapter-utils
packages/adapter-utils/src/command-redaction.test.ts`: 25 passed.
- `npx vitest run --project @paperclipai/adapter-utils`, complete suite:
80 of 82 files passed in the parallel run. The 2 other files
(`github-launcher-environment`, `sandbox-managed-runtime`) had 5 s
timeouts in git-based tests and passed on rerun.

## Risks

- Lookbehind assertions need ES2018. `redactCommandText` also runs in
the UI bundle. Node and current browsers support lookbehind, and the
server (`server/src/log-redaction.ts`) and the UI
(`ui/src/components/task-chat/task-chat-attachments.ts`) already use it.
- A change to a redaction pattern could leak a secret. The reasoning
above, the unchanged existing tests and the differential fuzzing show
the same output as before.
- Not changed here, found while checking: `redactDiagnosticText`
backtracks exponentially on an escaped JSON secret value made of many
escaped backslashes with no closing quote, and `redactSensitiveText`
redacts a value only up to its first quote of either kind
(`{"apiKey":"abc\"tail"}` keeps `tail"`). Both behave the same before
and after this pull request.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) by Anthropic, in Claude Code, with
tool use (shell, code search, and test runs) and extended thinking. A
human directed the design choices.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-11-02 10:49:08 -06:00

187 lines
7.9 KiB
TypeScript

import { describe, expect, it } from "vitest";
describe("explicit diagnostic credential forms", () => {
it("masks suffixed CLI options and encoded JSON headers with whitespace", () => {
const jwt = `${Buffer.from(' {"alg":"HS256","typ":"JWT"}').toString("base64url")}.abcdefghijk.abcdefghijkl`;
expect(redactCommandText("tool --api-key-prod sensitivevalue --token-policy readable"))
.not.toContain("sensitivevalue");
expect(redactCommandText(`provider ${jwt}`)).not.toContain(jwt);
expect(redactCommandText("plan.security.credentials.md")).toBe("plan.security.credentials.md");
});
});
import {
REDACTED_COMMAND_TEXT_VALUE,
looksLikeCredentialJwt,
redactDiagnosticText,
redactCommandText,
} from "./command-redaction.js";
describe("looksLikeCredentialJwt", () => {
it("recognizes encoded headers even with short payloads and signatures", () => {
const header = Buffer.from('{"alg":"HS256"}').toString("base64url");
expect(looksLikeCredentialJwt(`${header}.e30.c2ln`)).toBe(true);
});
it("recognizes direct-encryption JWTs with an empty encrypted-key segment", () => {
const header = Buffer.from('{"alg":"dir","enc":"A256GCM"}').toString("base64url");
expect(looksLikeCredentialJwt(`${header}..abcdefghijk.abcdefghijkl.abcdefghijk`)).toBe(true);
});
it.each([
"eyJ..signature",
"eyJ.key..ciphertext.tag",
"eyJ.key.iv..tag",
"eyJ.key.iv.ciphertext.",
])("rejects empty segments outside the JWE encrypted key: %s", (input) => {
expect(looksLikeCredentialJwt(input)).toBe(false);
});
it.each([
"api.openai.com",
"agents.staging.example.com",
"deployment.credentials.example.md",
"paperclip.native-model-envelope.v3",
])("rejects public dotted identifiers: %s", (input) => {
expect(looksLikeCredentialJwt(input)).toBe(false);
});
});
describe("redactDiagnosticText", () => {
it("preserves credential metadata and dotted identifiers", () => {
for (const text of [
'"tokenBudget":4000 --token-budget 4000 SECRET_STORAGE=vault',
'{"credentialHandling":"harness","authorizationRequired":true}',
"executor.customTools.integrations.list deployment.credentials.example.md api.openai.com",
"Use a private key and secret manager with credential handling.",
"Use bearer tokens and bearer authentication.",
]) expect(redactDiagnosticText(text)).toBe(text);
});
it("redacts a JSON secret field value", () => {
const input = '{"token":"opaque-value","status":"error"}';
const output = redactDiagnosticText(input);
expect(output).not.toContain("opaque-value");
expect(output).toContain(`"token":"${REDACTED_COMMAND_TEXT_VALUE}"`);
// The non-secret field keeps its value.
expect(output).toContain('"status":"error"');
});
it("redacts an api_key JSON field with whitespace around the colon", () => {
const input = '{ "api_key" : "sk-secret-123" }';
const output = redactDiagnosticText(input);
expect(output).not.toContain("sk-secret-123");
expect(output).toContain(REDACTED_COMMAND_TEXT_VALUE);
});
it("redacts an escaped-JSON secret field value", () => {
// A diagnostic can carry a JSON string, so the double quotes appear as `\"`.
const input = '{\\"token\\":\\"opaque-value\\"}';
const output = redactDiagnosticText(input);
expect(output).not.toContain("opaque-value");
expect(output).toContain(
`\\"token\\":\\"${REDACTED_COMMAND_TEXT_VALUE}\\"`,
);
});
it("still redacts a shell KEY=value secret", () => {
const input = "ANTHROPIC_API_KEY=super-secret-value claude --print";
const output = redactDiagnosticText(input);
expect(output).not.toContain("super-secret-value");
expect(output).toContain(REDACTED_COMMAND_TEXT_VALUE);
});
it("redacts an escaped quoted assignment across a literal newline", () => {
const input = String.raw`authorization=\"Bearer first-line
second-line\" status=401`;
const expected = String.raw`authorization=\"***REDACTED***\" status=401`;
const output = redactDiagnosticText(input);
expect(output).toBe(expected);
expect(redactDiagnosticText(output)).toBe(expected);
});
it("keeps non-secret text and non-secret JSON fields intact", () => {
const input = '{"status":"ok","message":"probe finished"}';
expect(redactDiagnosticText(input)).toBe(input);
});
it("redacts the secret but keeps a non-secret marker in the same string", () => {
const input = 'DIAGMARKER1234 said {"authorization":"Bearer opaque"}';
const output = redactDiagnosticText(input);
expect(output).toContain("DIAGMARKER1234");
expect(output).not.toContain("opaque");
});
it("redacts a JSON secret value that contains an escaped quote", () => {
// The value holds an escaped quote, so a naive matcher stops at the `\"` and
// leaves the rest of the credential. The marker sits after the escaped quote.
const input = '{"token":"pre\\"MARKERQUOTE_A"}';
const output = redactDiagnosticText(input);
expect(output).not.toContain("MARKERQUOTE_A");
expect(output).toContain(`"token":"${REDACTED_COMMAND_TEXT_VALUE}"`);
});
it("redacts a JSON secret value that contains an escaped backslash", () => {
const input = '{"secret":"pre\\\\MARKERBACKSLASH_A"}';
const output = redactDiagnosticText(input);
expect(output).not.toContain("MARKERBACKSLASH_A");
expect(output).toContain(`"secret":"${REDACTED_COMMAND_TEXT_VALUE}"`);
});
it("redacts an escaped-JSON secret value that contains an escaped quote", () => {
// A diagnostic can carry a serialized JSON string, so the whole JSON is
// escaped a second time. The inner value still holds an escaped quote.
const innerJson = '{"token":"pre\\"MARKERQUOTE_B"}';
const input = JSON.stringify(innerJson);
const output = redactDiagnosticText(input);
expect(output).not.toContain("MARKERQUOTE_B");
expect(output).toContain(REDACTED_COMMAND_TEXT_VALUE);
});
it("redacts an escaped-JSON secret value that contains an escaped backslash", () => {
const innerJson = '{"password":"pre\\\\MARKERBACKSLASH_B"}';
const input = JSON.stringify(innerJson);
const output = redactDiagnosticText(input);
expect(output).not.toContain("MARKERBACKSLASH_B");
expect(output).toContain(REDACTED_COMMAND_TEXT_VALUE);
});
});
describe("redactCommandText on long runs", () => {
const jwt =
"eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c";
function redactWithinTimeBudget(input: string) {
const startedAt = performance.now();
const output = redactCommandText(input);
expect(performance.now() - startedAt).toBeLessThan(1_000);
return output;
}
it("keeps redacting options, assignments, and tokens inside dash-separated runs", () => {
const cases = [
["x--token flag-secret", `x--token ${REDACTED_COMMAND_TEXT_VALUE}`],
["--x--token flag-secret", `--x--token ${REDACTED_COMMAND_TEXT_VALUE}`],
[
"a-b--api-key=assigned-secret --password 'quoted-secret'",
`a-b--api-key=${REDACTED_COMMAND_TEXT_VALUE} --password '${REDACTED_COMMAND_TEXT_VALUE}'`,
],
["prefix-MY_API_KEY=env-secret", `prefix-MY_API_KEY=${REDACTED_COMMAND_TEXT_VALUE}`],
[`--${jwt}`, `--${REDACTED_COMMAND_TEXT_VALUE}`],
[`session=${jwt}`, `session=${REDACTED_COMMAND_TEXT_VALUE}`],
["my-token is here", "my-token is here"],
];
for (const [input, expected] of cases) {
expect(redactCommandText(input)).toBe(expected);
}
});
it("scans long letter/digit and dash-separated runs in linear time", () => {
const base64Url = Buffer.from(Array.from({ length: 90_000 }, (_, index) => (index * 7919) % 256)).toString("base64url");
for (const run of ["a".repeat(120_000), "a-".repeat(60_000), "a--".repeat(40_000), base64Url]) {
expect(redactWithinTimeBudget(`${run} --token run-secret`)).toBe(`${run} --token ${REDACTED_COMMAND_TEXT_VALUE}`);
expect(redactWithinTimeBudget(`${run}--token run-secret`)).toBe(`${run}--token ${REDACTED_COMMAND_TEXT_VALUE}`);
}
});
});