mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-05 21:02:34 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip removes secrets from run logs, tool results and diagnostics with `redactSensitiveText` before it stores or shows them. > - The JSON field-name patterns in that function start with an unanchored `[A-Za-z0-9_-]*`. Each start position inside a long run of letters and digits scans to the end of the run, so the cost grows with the square of the run length. > - Base64 of text is one long run of letters and digits. A 40 KB base64 blob of source code in a shell command takes about 5 s to redact, and a large Gmail RAW tool result does not finish. > - The redaction runs on the main event loop, so one such chunk blocks every request and every run on the instance. > - This pull request makes these scans start only at the start of a run. The scans become linear, and the redacted output does not change. > - The benefit is that large base64 payloads no longer freeze the server, and every secret that was redacted before is still redacted. ## Linked Issues or Issue Description Fixes #15393. The tool content guard runs `redactSensitiveText` on tool results. A Gmail RAW body is base64url of a MIME message, which is one long letter/digit run. `validateToolContent` on a Gmail-shaped result with a 68 KB `raw` field takes 59 s on master and 19 ms with this change; a 5.4 MB `raw` field takes 0.65 s. Thanks @ts-beat for the clear report in #15393. Refs #12309 (false positives in the same patterns; this pull request does not change which text matches). I searched issues and pull requests for "redaction", "redactSensitiveText", "quadratic", "backtracking", "hangs the server" and "100% CPU". #15393 reports the hang. No open pull request fixes it. **Subsystem affected:** secret redaction (`server/src/redaction.ts`, `packages/adapter-utils/src/command-redaction.ts`), used by run logs, tool content guards and run diagnostics. **Problem:** `compactRunLogChunk` redacts each run log chunk before it truncates the chunk. A shell command such as `printf '%s' '<base64>' | base64 -d > file.ts` with base64 of source text makes one long letter/digit run. Base64 of ASCII text contains `+` or `/` only where every third byte is `>`, `?`, `~` or DEL, so random base64 does not show the problem. One `redactSensitiveText` call on such a tool-call line takes (local measurement): | Blob | Before | After | | --- | --- | --- | | 4 KB | 66 ms | 1.1 ms | | 8 KB | 272 ms | 0.5 ms | | 16 KB | 1.1 s | 1.0 ms | | 40 KB | 5.4 s | 2.5 ms | | 1 MB | not measured | 36 ms | The `\b`-anchored command patterns (`KEY=value`, JWT, `Authorization`) and the CLI option pattern have the same quadratic cost on dash-separated runs, for example base64url data. **Proposed solution:** start the unanchored field-name scans only where no name character precedes them (`(?<![A-Za-z0-9_-])`). A prefixed name such as `myApiKey`, `x_auth_token` or `GITHUB_TOKEN` still matches, because the scan from the start of the run absorbs the prefix. **Alternatives considered:** - Truncate or cap the input before redaction. Rejected: a cut can split a secret and leave part of it unredacted, and the scan stays quadratic below the cap. - Run the scan in a worker thread with a deadline, as #15393 suggests. That limits the damage, but the CPU cost stays, and a timeout must then drop or block the content. It is still useful as an additional defense. ## What Changed - `server/src/redaction.ts`: `JSON_SECRET_FIELD_TEXT_RE` and `ESCAPED_JSON_SECRET_FIELD_TEXT_RE` start a field name only where no name character precedes it. The lookbehind comes after the optional opening quote, so quoted keys behave as before. The `Authorization` word scan uses the same start instead of `\b`. - `packages/adapter-utils/src/command-redaction.ts`: the `KEY=value` pattern uses the same start. The JWT pattern starts only at the first word character of a run. A CLI option name can no longer span `--`, so each `--` starts its own short scan. - Redaction still runs before truncation. `compactRunLogChunk` does not change. Why the output does not change: when an old pattern could match from the middle of a run, it could also match from the start of that run, because the `[A-Za-z0-9_-]*` prefix absorbs the extra characters. The engine tries the start of the run first. So the old patterns never matched from the middle of a run, and the new start condition only skips positions that cannot produce a match. The old JWT pattern also always matched from the first word character of a run. A CLI option name always ends at the end of its run, so the match from the last `--` in the run ends at the same place. ## Screenshots No UI change. ## Verification - New tests: - prefixed secret field names in plain, single-quoted and escaped JSON are still redacted, also when the name is glued to the end of a long run; - a 48 KB base64-of-source tool call through `compactRunLogChunk` stays intact, the secret in the same command is redacted, and the call finishes within 1 s; - `redactSensitiveText` and `redactCommandText` handle 48-120 KB letter/digit, dash-separated, `--`-separated and base64url runs within 1 s, and redact the secret that follows each run; - `x--token`, `--x--token`, `a-b--api-key=`, `prefix-MY_API_KEY=` and `--<jwt>` are still redacted, and `my-token is here` is not changed. - Without the change, the timing tests fail: 9.3 s, 9.8 s, 32 s and 60 s against the 1 s budget. The correctness tests pass with and without the change. - Mutation check: I reverted each of the six pattern changes one at a time. Each revert makes at least one test fail. - Differential fuzzing: 1.1 million random inputs built from secret names, quotes, escaped quotes, dashes, dots, colons and JWT parts gave the same output from the old and new `redactSensitiveText`, `redactCommandText` and `redactDiagnosticText`. - `cd server && npx vitest run <files>`: all 33 server test files that use the redaction helpers, with embedded Postgres, 1730 passed. - `npx vitest run --project @paperclipai/adapter-utils packages/adapter-utils/src/command-redaction.test.ts`: 25 passed. - `npx vitest run --project @paperclipai/adapter-utils`, complete suite: 80 of 82 files passed in the parallel run. The 2 other files (`github-launcher-environment`, `sandbox-managed-runtime`) had 5 s timeouts in git-based tests and passed on rerun. ## Risks - Lookbehind assertions need ES2018. `redactCommandText` also runs in the UI bundle. Node and current browsers support lookbehind, and the server (`server/src/log-redaction.ts`) and the UI (`ui/src/components/task-chat/task-chat-attachments.ts`) already use it. - A change to a redaction pattern could leak a secret. The reasoning above, the unchanged existing tests and the differential fuzzing show the same output as before. - Not changed here, found while checking: `redactDiagnosticText` backtracks exponentially on an escaped JSON secret value made of many escaped backslashes with no closing quote, and `redactSensitiveText` redacts a value only up to its first quote of either kind (`{"apiKey":"abc\"tail"}` keeps `tail"`). Both behave the same before and after this pull request. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) by Anthropic, in Claude Code, with tool use (shell, code search, and test runs) and extended thinking. A human directed the design choices. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge
187 lines
7.9 KiB
TypeScript
187 lines
7.9 KiB
TypeScript
import { describe, expect, it } from "vitest";
|
|
|
|
describe("explicit diagnostic credential forms", () => {
|
|
it("masks suffixed CLI options and encoded JSON headers with whitespace", () => {
|
|
const jwt = `${Buffer.from(' {"alg":"HS256","typ":"JWT"}').toString("base64url")}.abcdefghijk.abcdefghijkl`;
|
|
expect(redactCommandText("tool --api-key-prod sensitivevalue --token-policy readable"))
|
|
.not.toContain("sensitivevalue");
|
|
expect(redactCommandText(`provider ${jwt}`)).not.toContain(jwt);
|
|
expect(redactCommandText("plan.security.credentials.md")).toBe("plan.security.credentials.md");
|
|
});
|
|
});
|
|
import {
|
|
REDACTED_COMMAND_TEXT_VALUE,
|
|
looksLikeCredentialJwt,
|
|
redactDiagnosticText,
|
|
redactCommandText,
|
|
} from "./command-redaction.js";
|
|
|
|
describe("looksLikeCredentialJwt", () => {
|
|
it("recognizes encoded headers even with short payloads and signatures", () => {
|
|
const header = Buffer.from('{"alg":"HS256"}').toString("base64url");
|
|
expect(looksLikeCredentialJwt(`${header}.e30.c2ln`)).toBe(true);
|
|
});
|
|
|
|
it("recognizes direct-encryption JWTs with an empty encrypted-key segment", () => {
|
|
const header = Buffer.from('{"alg":"dir","enc":"A256GCM"}').toString("base64url");
|
|
expect(looksLikeCredentialJwt(`${header}..abcdefghijk.abcdefghijkl.abcdefghijk`)).toBe(true);
|
|
});
|
|
|
|
it.each([
|
|
"eyJ..signature",
|
|
"eyJ.key..ciphertext.tag",
|
|
"eyJ.key.iv..tag",
|
|
"eyJ.key.iv.ciphertext.",
|
|
])("rejects empty segments outside the JWE encrypted key: %s", (input) => {
|
|
expect(looksLikeCredentialJwt(input)).toBe(false);
|
|
});
|
|
|
|
it.each([
|
|
"api.openai.com",
|
|
"agents.staging.example.com",
|
|
"deployment.credentials.example.md",
|
|
"paperclip.native-model-envelope.v3",
|
|
])("rejects public dotted identifiers: %s", (input) => {
|
|
expect(looksLikeCredentialJwt(input)).toBe(false);
|
|
});
|
|
});
|
|
|
|
describe("redactDiagnosticText", () => {
|
|
it("preserves credential metadata and dotted identifiers", () => {
|
|
for (const text of [
|
|
'"tokenBudget":4000 --token-budget 4000 SECRET_STORAGE=vault',
|
|
'{"credentialHandling":"harness","authorizationRequired":true}',
|
|
"executor.customTools.integrations.list deployment.credentials.example.md api.openai.com",
|
|
"Use a private key and secret manager with credential handling.",
|
|
"Use bearer tokens and bearer authentication.",
|
|
]) expect(redactDiagnosticText(text)).toBe(text);
|
|
});
|
|
|
|
it("redacts a JSON secret field value", () => {
|
|
const input = '{"token":"opaque-value","status":"error"}';
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).not.toContain("opaque-value");
|
|
expect(output).toContain(`"token":"${REDACTED_COMMAND_TEXT_VALUE}"`);
|
|
// The non-secret field keeps its value.
|
|
expect(output).toContain('"status":"error"');
|
|
});
|
|
|
|
it("redacts an api_key JSON field with whitespace around the colon", () => {
|
|
const input = '{ "api_key" : "sk-secret-123" }';
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).not.toContain("sk-secret-123");
|
|
expect(output).toContain(REDACTED_COMMAND_TEXT_VALUE);
|
|
});
|
|
|
|
it("redacts an escaped-JSON secret field value", () => {
|
|
// A diagnostic can carry a JSON string, so the double quotes appear as `\"`.
|
|
const input = '{\\"token\\":\\"opaque-value\\"}';
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).not.toContain("opaque-value");
|
|
expect(output).toContain(
|
|
`\\"token\\":\\"${REDACTED_COMMAND_TEXT_VALUE}\\"`,
|
|
);
|
|
});
|
|
|
|
it("still redacts a shell KEY=value secret", () => {
|
|
const input = "ANTHROPIC_API_KEY=super-secret-value claude --print";
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).not.toContain("super-secret-value");
|
|
expect(output).toContain(REDACTED_COMMAND_TEXT_VALUE);
|
|
});
|
|
|
|
it("redacts an escaped quoted assignment across a literal newline", () => {
|
|
const input = String.raw`authorization=\"Bearer first-line
|
|
second-line\" status=401`;
|
|
const expected = String.raw`authorization=\"***REDACTED***\" status=401`;
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).toBe(expected);
|
|
expect(redactDiagnosticText(output)).toBe(expected);
|
|
});
|
|
|
|
it("keeps non-secret text and non-secret JSON fields intact", () => {
|
|
const input = '{"status":"ok","message":"probe finished"}';
|
|
expect(redactDiagnosticText(input)).toBe(input);
|
|
});
|
|
|
|
it("redacts the secret but keeps a non-secret marker in the same string", () => {
|
|
const input = 'DIAGMARKER1234 said {"authorization":"Bearer opaque"}';
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).toContain("DIAGMARKER1234");
|
|
expect(output).not.toContain("opaque");
|
|
});
|
|
|
|
it("redacts a JSON secret value that contains an escaped quote", () => {
|
|
// The value holds an escaped quote, so a naive matcher stops at the `\"` and
|
|
// leaves the rest of the credential. The marker sits after the escaped quote.
|
|
const input = '{"token":"pre\\"MARKERQUOTE_A"}';
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).not.toContain("MARKERQUOTE_A");
|
|
expect(output).toContain(`"token":"${REDACTED_COMMAND_TEXT_VALUE}"`);
|
|
});
|
|
|
|
it("redacts a JSON secret value that contains an escaped backslash", () => {
|
|
const input = '{"secret":"pre\\\\MARKERBACKSLASH_A"}';
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).not.toContain("MARKERBACKSLASH_A");
|
|
expect(output).toContain(`"secret":"${REDACTED_COMMAND_TEXT_VALUE}"`);
|
|
});
|
|
|
|
it("redacts an escaped-JSON secret value that contains an escaped quote", () => {
|
|
// A diagnostic can carry a serialized JSON string, so the whole JSON is
|
|
// escaped a second time. The inner value still holds an escaped quote.
|
|
const innerJson = '{"token":"pre\\"MARKERQUOTE_B"}';
|
|
const input = JSON.stringify(innerJson);
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).not.toContain("MARKERQUOTE_B");
|
|
expect(output).toContain(REDACTED_COMMAND_TEXT_VALUE);
|
|
});
|
|
|
|
it("redacts an escaped-JSON secret value that contains an escaped backslash", () => {
|
|
const innerJson = '{"password":"pre\\\\MARKERBACKSLASH_B"}';
|
|
const input = JSON.stringify(innerJson);
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).not.toContain("MARKERBACKSLASH_B");
|
|
expect(output).toContain(REDACTED_COMMAND_TEXT_VALUE);
|
|
});
|
|
});
|
|
|
|
describe("redactCommandText on long runs", () => {
|
|
const jwt =
|
|
"eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c";
|
|
|
|
function redactWithinTimeBudget(input: string) {
|
|
const startedAt = performance.now();
|
|
const output = redactCommandText(input);
|
|
expect(performance.now() - startedAt).toBeLessThan(1_000);
|
|
return output;
|
|
}
|
|
|
|
it("keeps redacting options, assignments, and tokens inside dash-separated runs", () => {
|
|
const cases = [
|
|
["x--token flag-secret", `x--token ${REDACTED_COMMAND_TEXT_VALUE}`],
|
|
["--x--token flag-secret", `--x--token ${REDACTED_COMMAND_TEXT_VALUE}`],
|
|
[
|
|
"a-b--api-key=assigned-secret --password 'quoted-secret'",
|
|
`a-b--api-key=${REDACTED_COMMAND_TEXT_VALUE} --password '${REDACTED_COMMAND_TEXT_VALUE}'`,
|
|
],
|
|
["prefix-MY_API_KEY=env-secret", `prefix-MY_API_KEY=${REDACTED_COMMAND_TEXT_VALUE}`],
|
|
[`--${jwt}`, `--${REDACTED_COMMAND_TEXT_VALUE}`],
|
|
[`session=${jwt}`, `session=${REDACTED_COMMAND_TEXT_VALUE}`],
|
|
["my-token is here", "my-token is here"],
|
|
];
|
|
|
|
for (const [input, expected] of cases) {
|
|
expect(redactCommandText(input)).toBe(expected);
|
|
}
|
|
});
|
|
|
|
it("scans long letter/digit and dash-separated runs in linear time", () => {
|
|
const base64Url = Buffer.from(Array.from({ length: 90_000 }, (_, index) => (index * 7919) % 256)).toString("base64url");
|
|
for (const run of ["a".repeat(120_000), "a-".repeat(60_000), "a--".repeat(40_000), base64Url]) {
|
|
expect(redactWithinTimeBudget(`${run} --token run-secret`)).toBe(`${run} --token ${REDACTED_COMMAND_TEXT_VALUE}`);
|
|
expect(redactWithinTimeBudget(`${run}--token run-secret`)).toBe(`${run}--token ${REDACTED_COMMAND_TEXT_VALUE}`);
|
|
}
|
|
});
|
|
});
|