mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 00:05:49 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People rate agent comments and documents to capture feedback. > - Feedback bundles remove credentials and personal information before export. > - The feedback scanner treats ordinary dotted text as JWTs and authentication terms as bearer tokens. > - This pull request checks encoded JWT headers and preserves known authentication terms. > - The benefit is readable feedback context with credential masking kept in place. ## Linked Issues or Issue Description Refs #12309 for the related dotted-identifier false-positive report. This PR fixes the separate feedback scanner. It does not close every case in that report. Related PR: #14318 adds fine-grained GitHub token coverage. It addresses a separate credential form and is not duplicated here. **What happened?** Feedback exports replace `api.openai.com`, dotted filenames, and public schema IDs with a JWT placeholder. They also replace the words after `bearer` in phrases such as “bearer tokens.” Five-part encrypted JWTs are only partly masked. **Expected behavior** Keep public identifiers and authentication terms readable. Mask complete credential tokens. Count only text replacements in the redaction summary. **Steps to reproduce** 1. Pass `api.openai.com` or `Use bearer tokens.` to `sanitizeFeedbackText`. 2. Check that the original text is returned and the redaction counts stay empty. 3. Pass a signed JWT or an encrypted JWT to the same function. 4. Check that the full token is replaced and one JWT replacement is counted. **Paperclip version or commit** Reproduced in the feedback scanner on master at `5e8e992df2b3d71f82091b7bbf29fd482854d9f5`. **Deployment mode** Built from source. The defect is in the shared feedback export service and does not depend on deployment mode. ## What Changed - Use the existing JWT header predicate in the feedback scanner. - Match complete three-part and five-part tokens, including direct-encryption tokens with an empty key segment. Accept short token segments in the predicate. - Preserve the closed set of authentication terms already used by the general scanner, including terms inside closing prose delimiters. - Count only replacements that change the text. - Add tests for readable context, credential masking, nested values, and redaction counts. - Document feedback text redaction. ## Verification - Focused redaction and feedback service tests: 89 tests pass across five suites. - `pnpm --filter @paperclipai/adapter-utils typecheck` passes. Full local typecheck, tests, and build were attempted. The kernel killed compiler and test processes at the sandbox memory limit of 4 GB. Serial retries and a capped Node heap still could not complete the server compiler. CI runs the full repository checks. - `git diff --check` passes. - Latest commit `0eb8d3fb4257bb79e67fab615a8299ef69230ad8`: **55 checks passed**, including full typecheck, tests, build, and E2E; two optional Storybook checks were skipped. [CI evidence](https://github.com/paperclipai/paperclip/actions/runs/38407932875). Both Greptile findings have regression tests and fixes. [Greptile gives this commit 5/5](https://github.com/paperclipai/paperclip/pull/16168#issuecomment-6146917423), with both review threads resolved. ## Risks - This changes the feedback export matching rules. Public dotted text no longer triggers credential masking by shape alone. - Explicit credential assignments, provider keys, structured secret fields, email addresses, and phone numbers remain masked. Tests cover these cases. - Existing feedback snapshots are not rewritten. No schema change or migration is needed. - The encoded-header check remains a heuristic. It is not a full JWT validator. ## Model Used OpenAI Codex, GPT-6, with reasoning, tool use, and code execution. The runtime identifies the model as GPT-6. It does not expose the exact provider deployment ID or context window size. The earlier patch and this PR preparation were AI-assisted. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
148 lines
6.2 KiB
TypeScript
148 lines
6.2 KiB
TypeScript
import { describe, expect, it } from "vitest";
|
|
|
|
describe("explicit diagnostic credential forms", () => {
|
|
it("masks suffixed CLI options and encoded JSON headers with whitespace", () => {
|
|
const jwt = `${Buffer.from(' {"alg":"HS256","typ":"JWT"}').toString("base64url")}.abcdefghijk.abcdefghijkl`;
|
|
expect(redactCommandText("tool --api-key-prod sensitivevalue --token-policy readable"))
|
|
.not.toContain("sensitivevalue");
|
|
expect(redactCommandText(`provider ${jwt}`)).not.toContain(jwt);
|
|
expect(redactCommandText("plan.security.credentials.md")).toBe("plan.security.credentials.md");
|
|
});
|
|
});
|
|
import {
|
|
REDACTED_COMMAND_TEXT_VALUE,
|
|
looksLikeCredentialJwt,
|
|
redactDiagnosticText,
|
|
redactCommandText,
|
|
} from "./command-redaction.js";
|
|
|
|
describe("looksLikeCredentialJwt", () => {
|
|
it("recognizes encoded headers even with short payloads and signatures", () => {
|
|
const header = Buffer.from('{"alg":"HS256"}').toString("base64url");
|
|
expect(looksLikeCredentialJwt(`${header}.e30.c2ln`)).toBe(true);
|
|
});
|
|
|
|
it("recognizes direct-encryption JWTs with an empty encrypted-key segment", () => {
|
|
const header = Buffer.from('{"alg":"dir","enc":"A256GCM"}').toString("base64url");
|
|
expect(looksLikeCredentialJwt(`${header}..abcdefghijk.abcdefghijkl.abcdefghijk`)).toBe(true);
|
|
});
|
|
|
|
it.each([
|
|
"eyJ..signature",
|
|
"eyJ.key..ciphertext.tag",
|
|
"eyJ.key.iv..tag",
|
|
"eyJ.key.iv.ciphertext.",
|
|
])("rejects empty segments outside the JWE encrypted key: %s", (input) => {
|
|
expect(looksLikeCredentialJwt(input)).toBe(false);
|
|
});
|
|
|
|
it.each([
|
|
"api.openai.com",
|
|
"agents.staging.example.com",
|
|
"deployment.credentials.example.md",
|
|
"paperclip.native-model-envelope.v3",
|
|
])("rejects public dotted identifiers: %s", (input) => {
|
|
expect(looksLikeCredentialJwt(input)).toBe(false);
|
|
});
|
|
});
|
|
|
|
describe("redactDiagnosticText", () => {
|
|
it("preserves credential metadata and dotted identifiers", () => {
|
|
for (const text of [
|
|
'"tokenBudget":4000 --token-budget 4000 SECRET_STORAGE=vault',
|
|
'{"credentialHandling":"harness","authorizationRequired":true}',
|
|
"executor.customTools.integrations.list deployment.credentials.example.md api.openai.com",
|
|
"Use a private key and secret manager with credential handling.",
|
|
"Use bearer tokens and bearer authentication.",
|
|
]) expect(redactDiagnosticText(text)).toBe(text);
|
|
});
|
|
|
|
it("redacts a JSON secret field value", () => {
|
|
const input = '{"token":"opaque-value","status":"error"}';
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).not.toContain("opaque-value");
|
|
expect(output).toContain(`"token":"${REDACTED_COMMAND_TEXT_VALUE}"`);
|
|
// The non-secret field keeps its value.
|
|
expect(output).toContain('"status":"error"');
|
|
});
|
|
|
|
it("redacts an api_key JSON field with whitespace around the colon", () => {
|
|
const input = '{ "api_key" : "sk-secret-123" }';
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).not.toContain("sk-secret-123");
|
|
expect(output).toContain(REDACTED_COMMAND_TEXT_VALUE);
|
|
});
|
|
|
|
it("redacts an escaped-JSON secret field value", () => {
|
|
// A diagnostic can carry a JSON string, so the double quotes appear as `\"`.
|
|
const input = '{\\"token\\":\\"opaque-value\\"}';
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).not.toContain("opaque-value");
|
|
expect(output).toContain(
|
|
`\\"token\\":\\"${REDACTED_COMMAND_TEXT_VALUE}\\"`,
|
|
);
|
|
});
|
|
|
|
it("still redacts a shell KEY=value secret", () => {
|
|
const input = "ANTHROPIC_API_KEY=super-secret-value claude --print";
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).not.toContain("super-secret-value");
|
|
expect(output).toContain(REDACTED_COMMAND_TEXT_VALUE);
|
|
});
|
|
|
|
it("redacts an escaped quoted assignment across a literal newline", () => {
|
|
const input = String.raw`authorization=\"Bearer first-line
|
|
second-line\" status=401`;
|
|
const expected = String.raw`authorization=\"***REDACTED***\" status=401`;
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).toBe(expected);
|
|
expect(redactDiagnosticText(output)).toBe(expected);
|
|
});
|
|
|
|
it("keeps non-secret text and non-secret JSON fields intact", () => {
|
|
const input = '{"status":"ok","message":"probe finished"}';
|
|
expect(redactDiagnosticText(input)).toBe(input);
|
|
});
|
|
|
|
it("redacts the secret but keeps a non-secret marker in the same string", () => {
|
|
const input = 'DIAGMARKER1234 said {"authorization":"Bearer opaque"}';
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).toContain("DIAGMARKER1234");
|
|
expect(output).not.toContain("opaque");
|
|
});
|
|
|
|
it("redacts a JSON secret value that contains an escaped quote", () => {
|
|
// The value holds an escaped quote, so a naive matcher stops at the `\"` and
|
|
// leaves the rest of the credential. The marker sits after the escaped quote.
|
|
const input = '{"token":"pre\\"MARKERQUOTE_A"}';
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).not.toContain("MARKERQUOTE_A");
|
|
expect(output).toContain(`"token":"${REDACTED_COMMAND_TEXT_VALUE}"`);
|
|
});
|
|
|
|
it("redacts a JSON secret value that contains an escaped backslash", () => {
|
|
const input = '{"secret":"pre\\\\MARKERBACKSLASH_A"}';
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).not.toContain("MARKERBACKSLASH_A");
|
|
expect(output).toContain(`"secret":"${REDACTED_COMMAND_TEXT_VALUE}"`);
|
|
});
|
|
|
|
it("redacts an escaped-JSON secret value that contains an escaped quote", () => {
|
|
// A diagnostic can carry a serialized JSON string, so the whole JSON is
|
|
// escaped a second time. The inner value still holds an escaped quote.
|
|
const innerJson = '{"token":"pre\\"MARKERQUOTE_B"}';
|
|
const input = JSON.stringify(innerJson);
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).not.toContain("MARKERQUOTE_B");
|
|
expect(output).toContain(REDACTED_COMMAND_TEXT_VALUE);
|
|
});
|
|
|
|
it("redacts an escaped-JSON secret value that contains an escaped backslash", () => {
|
|
const innerJson = '{"password":"pre\\\\MARKERBACKSLASH_B"}';
|
|
const input = JSON.stringify(innerJson);
|
|
const output = redactDiagnosticText(input);
|
|
expect(output).not.toContain("MARKERBACKSLASH_B");
|
|
expect(output).toContain(REDACTED_COMMAND_TEXT_VALUE);
|
|
});
|
|
});
|