Files
PaperClipAI/server/src/__tests__/feedback-redaction.test.ts
T
DottaandClaude Opus 5.5 9025597ce4 fix(feedback): keep run ids and dates intact when redacting phone numbers (#21251)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - People can export a feedback bundle that holds run records for
review
> - The bundle is redacted before export, and one rule removes phone
numbers
> - The phone rule also matched two all-digit groups inside a run UUID
> - A run id such as `9224d1c8-1dbf-4314-9924-e16fcf3ad1fa` became
`9224d1c8-1dbf-[REDACTED_PHONE]-e16fcf3ad1fa`, so the bundle no longer
named its run
> - This pull request keeps UUIDs and ISO dates out of the phone rule
> - The benefit is that exported ids and dates stay correct and real
phone numbers stay redacted

## Linked Issues or Issue Description

**What happened?**
In the paid chat eval `cli-operations.feedback-bundle`, the exported
bundle's `envelope.targetId` was
`9224d1c8-1dbf-[REDACTED_PHONE]-e16fcf3ad1fa`. The phone rule
`(?<!\w)(?:\+?\d[\d ()-]{7,}\d)(?!\w)` matches two all-digit UUID groups
joined by `-`. About 2 to 3 percent of random UUIDs have that shape. The
same rule also matched a bare date such as `2026-11-12`.

**Steps to reproduce**
1. Call `sanitizeFeedbackText` with text that contains
`9224d1c8-1dbf-4314-9924-e16fcf3ad1fa`.
2. Read the output.

**Expected behavior**
The UUID stays unchanged. A real phone number such as `+1 (415)
555-1234` is still redacted.

## What Changed

- `server/src/services/feedback-redaction.ts`: the phone rule now runs
only on the text between public identifiers. UUIDs and ISO-8601 dates
and timestamps are copied through unchanged. Other rules are unchanged.
- `server/src/__tests__/feedback-redaction.test.ts`: a regression test
with the `-4314-9924-` UUID, an all-digit UUID, a date and a timestamp,
next to a real phone number that must stay redacted.

## Verification

- `cd server && npx vitest run src/__tests__/feedback-redaction.test.ts`
passes (13 tests). The new test fails on master: the UUID and the date
are replaced with `[REDACTED_PHONE]`.

## Risks

- Low risk. A phone number written in the exact shape of a UUID or an
ISO date is no longer redacted. Those shapes are not phone numbers.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 20:06:39 -06:00

110 lines
4.9 KiB
TypeScript

import { describe, expect, it } from "vitest";
import {
createFeedbackRedactionState,
finalizeFeedbackRedactionSummary,
sanitizeFeedbackText,
sanitizeFeedbackValue,
} from "../services/feedback-redaction.js";
describe("feedback redaction", () => {
it.each([
"https://agents.staging.example.com/team/chats/assistant",
"api.openai.com and deployment.credentials.example.md",
"paperclip.native-model-envelope.v3 and executor.coreTools.integrations.list",
"Use bearer tokens for authentication and bearer authentication for requests.",
"Document bearer authorization, bearer credentials, bearer schemes, and bearer flows.",
"Use (bearer tokens). Preserve [bearer authentication] and {bearer credentials}.",
])("preserves non-secret chat context: %s", (input) => {
const state = createFeedbackRedactionState();
expect(sanitizeFeedbackText(input, state, "body", 10_000)).toBe(input);
expect(finalizeFeedbackRedactionSummary(state).redactedFields).toEqual([]);
expect(finalizeFeedbackRedactionSummary(state).counts).toEqual({});
});
it("redacts JWTs without counting neighboring hostnames as credentials", () => {
const header = Buffer.from('{"alg":"HS256","typ":"JWT"}').toString("base64url");
const jwt = `${header}.abcdefghijk.abcdefghijkl`;
const state = createFeedbackRedactionState();
expect(sanitizeFeedbackText(`Visit api.openai.com; token ${jwt}`, state, "body", 10_000))
.toBe("Visit api.openai.com; token [REDACTED_JWT]");
expect(finalizeFeedbackRedactionSummary(state).counts).toEqual({ jwt: 1 });
expect(finalizeFeedbackRedactionSummary(state).redactedFields).toEqual(["body"]);
});
it("still redacts a JWT with a short payload and signature", () => {
const header = Buffer.from('{"alg":"HS256"}').toString("base64url");
const state = createFeedbackRedactionState();
const jwt = `${header}.e30.c2ln`;
expect(sanitizeFeedbackText(jwt, state, "body", 10_000)).toBe("[REDACTED_JWT]");
expect(finalizeFeedbackRedactionSummary(state).counts).toEqual({ jwt: 1 });
});
it("redacts encrypted JWTs", () => {
const header = Buffer.from('{"alg":"RSA-OAEP","enc":"A256GCM"}').toString("base64url");
const jwt = `${header}.abcdefghijk.abcdefghijkl.abcdefghijk.abcdefghijkl`;
const state = createFeedbackRedactionState();
expect(sanitizeFeedbackText(jwt, state, "body", 10_000)).toBe("[REDACTED_JWT]");
expect(finalizeFeedbackRedactionSummary(state).counts).toEqual({ jwt: 1 });
});
it("redacts direct-encryption JWTs with an empty encrypted-key segment", () => {
const header = Buffer.from('{"alg":"dir","enc":"A256GCM"}').toString("base64url");
const jwt = `${header}..abcdefghijk.abcdefghijkl.abcdefghijk`;
const state = createFeedbackRedactionState();
expect(sanitizeFeedbackText(`Token ${jwt}; visit api.openai.com.`, state, "body", 10_000))
.toBe("Token [REDACTED_JWT]; visit api.openai.com.");
expect(finalizeFeedbackRedactionSummary(state).counts).toEqual({ jwt: 1 });
expect(finalizeFeedbackRedactionSummary(state).redactedFields).toEqual(["body"]);
});
it("still redacts explicit credentials and personal information", () => {
const state = createFeedbackRedactionState();
const input = [
"api_key=opaque-value",
"Bearer abcdefghijklmnop",
"ghp_abcdefghijklmnopqrstuv",
"sk-abcdefghijklmnopqrstuv",
"postgresql://admin:password@example.com/db",
"person@example.com",
"+1 (415) 555-1234",
].join("\n");
const output = sanitizeFeedbackText(input, state, "body", 10_000);
for (const value of input.split("\n")) expect(output).not.toContain(value);
expect(output).toContain("Bearer [REDACTED_TOKEN]");
expect(output).toContain("[REDACTED_EMAIL]");
expect(output).toContain("[REDACTED_PHONE]");
});
it("keeps run ids and dates intact while still redacting a phone number beside them", () => {
const state = createFeedbackRedactionState();
const runId = "9224d1c8-1dbf-4314-9924-e16fcf3ad1fa";
const allDigitId = "12345678-1234-4234-9234-123456789012";
const input = `Run ${runId} and ${allDigitId} on 2026-11-12 at 2026-11-12 00:43:37; call +1 (415) 555-1234.`;
expect(sanitizeFeedbackText(input, state, "body", 10_000)).toBe(
`Run ${runId} and ${allDigitId} on 2026-11-12 at 2026-11-12 00:43:37; call [REDACTED_PHONE].`,
);
expect(finalizeFeedbackRedactionSummary(state).counts).toEqual({ phone: 1 });
});
it("preserves nested metadata while masking structured secrets", () => {
const state = createFeedbackRedactionState();
const input = {
body: "Visit api.openai.com and use bearer tokens.",
tokenPolicy: "least privilege",
nested: [{ apiKey: "opaque-value", path: "deployment.credentials.example.md" }],
};
expect(sanitizeFeedbackValue(input, state, "bundle", 10_000)).toEqual({
...input,
nested: [{ apiKey: "***REDACTED***", path: "deployment.credentials.example.md" }],
});
});
});