Files
PaperClipAI/ui/src/components/DecisionCard.test.tsx
T
DottaandClaude Opus 5.5 b56f240f45 feat(analytics): show session replay text by default, mask only customer content (#22057)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Hosted Paperclip Cloud records browser sessions with PostHog session
replay
> - Replay masked all text and showed only an allowlist of chrome, so
most replays showed only stars
> - The team cannot see how people use the board, and the allowlist
needed a review for each new label
> - The owner decided to show text by default and mask only marked
customer content, the way PostHog normally works
> - This pull request flips the default, removes the allowlist, and
marks every surface that can show connected-account content
> - The benefit is readable replays, while Google user data and Slack
data stay masked

## Linked Issues or Issue Description

**Subsystem affected**

Hosted Cloud browser analytics: PostHog session replay in the board UI
(`ui/src/lib/analytics/`).

**Problem or motivation**

Replay masked every text node. Only sidebar items and opt-in components
showed text. Replays were almost unreadable. Each new label needed an
`analyticsChrome` opt-in and an allowlist review.

**Proposed solution**

Record text as it is. Mask text only inside `ph-mask` (customer
content), `ph-no-capture` (secrets, blocked) and editable content. Add
`ANALYTICS_MASK_CLASS` to every surface that can show connected-account
content (Gmail, Calendar, Drive and Docs, Slack) or other free-form text
from agents or customers.

**Alternatives considered**

Keep "mask all" and grow the allowlist. This needs a review for each
label and still hides most of the board.

## What Changed

- `privacy.ts`: `ANALYTICS_MASK_TEXT_SELECTOR` is now `.ph-mask,
.ph-no-capture, [contenteditable]:not([contenteditable="false"])`. rrweb
masks a text node only when its parent or an ancestor matches the
selector. Removed `maskReplayText`, `isReplayTextRevealed`,
`ANALYTICS_UNMASK_ATTRIBUTE` and `ANALYTICS_UNMASK_PROPS`.
- `product-analytics.ts`: removed `maskTextFn`, so rrweb uses its
default star masking. `maskAllInputs`, `blockClass`, `maskAttributeFn`,
`mask_all_text` and `mask_all_element_attributes` do not change.
- Removed the `analyticsChrome` prop from `Button`, `Label`,
`TabsTrigger`, `EmptyState` and `SidebarNavItem`, and from all sidebar
call sites.
- Added `ANALYTICS_MASK_CLASS` to the content element (not the chrome)
of these surfaces:
- Link preview cards, task quicklook description, external object pills
and rows, related-work titles.
  - Email message cards and email send intents.
- Connected account names and email addresses (Browse, Connections, app
detail headers, app sidebar, connection choice list, connection access
and repair cards), GitHub login and repositories, Slack workspace names,
chat channel and linked user labels.
- Inbox and decision rows (subject and detail), obligation evidence,
Needs You notes and receipts, decision options and result summaries, and
the interaction card roots (questions, options, answers, tool action
requests, suggested tasks).
- Approval payloads, action card input and reason, recovery card
headline, evidence and options, run ledger retry detail, interrupt
handoff previews.
- Run transcripts, run failure details and run events on the agent page.
- Search result snippets, toast bodies, voice and call transcripts, case
summary, files and fields, attachment previews, artifact previews (text,
tables, patches and documents).
- Replaced `unmask-allowlist.test.ts` with `replay-masking.test.tsx`. It
checks that unmarked chrome records as it is, that each marker masks,
that a rendered voice transcript and run transcript mask, that each
surface above has the class, and that autocapture records no element
text. Updated `privacy.test.ts`, `shared-chrome.test.tsx` and
`product-analytics.test.ts`.
- Updated `doc/cloud-browser-analytics.md`: new default, marker classes,
masked surfaces, and residual risk.

## Verification

- `pnpm exec vitest run src/lib/analytics` in `ui/`: 83 tests pass.
- Vitest for the 48 test files next to the changed components: 994 tests
pass.
- `tsc --noEmit -p ui/tsconfig.json`: no new errors (only the existing
generated Storybook catalog errors).
- `pnpm verify` exits 2 because a doc is a global CI input. Required CI
supplies the broad coverage.
- Manual: I ran an isolated local instance with a Cloud-like browser
analytics config. `PAPERCLIP_POSTHOG_HOST` pointed to a local capture
server, not PostHog. I opened a task page in headless Chromium, hovered
a GitHub pull request link, and decoded the captured `/s/` replay
payloads:
- Clear text: sidebar (`New task`, `Chat`, `Tasks`, `Inbox`, `More`),
breadcrumb, task title `Quarterly launch checklist`, `REP-1`, the
Connectors promo heading and button.
- Masked (stars only): the markdown task body, the chat comment, the
link preview card (pull request title, author, URL, status), and the
company menu.
- Marker strings in the body, comment and preview never appeared in
clear text in any payload.
- The page `<title>` (task title and company name) is in the replay
head. This is accepted residual risk.

## Risks

- Text that is not marked is now recorded. Task titles, agent names,
project names, company names, routine names, artifact and document
titles, and activity rows are visible in replays. The owner accepted
this. An agent can copy email or Slack text into a task title.
- A new surface that shows connected-account content without
`ANALYTICS_MASK_CLASS` is recorded until somebody marks it.
`replay-masking.test.tsx` and the doc list the marked surfaces.
- The masking options in `posthog.init` take precedence over the PostHog
project masking settings (posthog-js 1.450), so a dashboard change
cannot unmask content. Keep the project settings empty to avoid a
console warning.
- Some masked areas also mask nearby fixed labels (for example the
interaction card buttons). This is safe.
- Analytics is off on self-hosted and local installs, so they see no
change.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (claude-opus-5-5), Anthropic, with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:18:50 -06:00

335 lines
14 KiB
TypeScript

// @vitest-environment jsdom
import { act as reactAct, type ComponentProps, type ReactNode } from "react";
import { flushSync } from "react-dom";
import { createRoot, type Root } from "react-dom/client";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { afterEach, describe, expect, it, vi } from "vitest";
import { DecisionCard, type DecisionIssueRef } from "./DecisionCard";
import type { Decision, DecisionEffectExecution } from "../api/decisions";
import { ThemeProvider } from "../context/ThemeContext";
import { TooltipProvider } from "./ui/tooltip";
import { replayLeaks } from "@/lib/analytics/replay-recording.test-utils";
let container: HTMLDivElement | null = null;
let root: Root | null = null;
(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true;
async function act(callback: () => void | Promise<void>) {
if (typeof reactAct === "function") {
await reactAct(callback);
return;
}
let result: void | Promise<void> = undefined;
flushSync(() => {
result = callback();
});
await result;
await new Promise((resolve) => setTimeout(resolve, 0));
}
vi.mock("@/lib/router", () => ({
Link: ({ to, children, className }: { to: string; children: ReactNode; className?: string }) => (
<a href={to} className={className}>{children}</a>
),
useCaseHref: () => () => "#",
}));
const ISSUES: Record<string, DecisionIssueRef> = {
"issue-origin": { id: "issue-origin", identifier: "PAP-123", title: "Gardener sweep", href: "/PAP/issues/PAP-123", status: "in_progress" },
"issue-target": { id: "issue-target", identifier: "PAP-456", title: "Stale epic", href: "/PAP/issues/PAP-456", status: "backlog" },
"issue-parent": { id: "issue-parent", identifier: "PAP-789", title: "Parent task", href: "/PAP/issues/PAP-789", status: "todo" },
"issue-blocker": { id: "issue-blocker", identifier: "PAP-790", title: "Blocking task", href: "/PAP/issues/PAP-790", status: "todo" },
"issue-new": { id: "issue-new", identifier: "PAP-999", title: "Follow-up", href: "/PAP/issues/PAP-999", status: "todo" },
};
const resolveIssue = (id: string): DecisionIssueRef | null => ISSUES[id] ?? null;
function mkDecision(overrides: Partial<Decision> = {}): Decision {
return {
id: "decision-1",
companyId: "c1",
bundleId: null,
originAgentId: "agent-gardener",
originIssueId: "issue-origin",
originRunId: "run-1",
ruleKey: "stale-epic",
title: "Stale epic PAP-456",
body: "No activity for three weeks.",
options: [
{ id: "comment", label: "Comment and snooze", effects: [{ type: "comment_on_issue", targetIssueId: "issue-target", staleness: "lenient", bodyMarkdown: "nudge" }] },
],
inputs: null,
status: "open",
executionStatus: null,
chosenOptionId: null,
inputValues: null,
decidedByUserId: null,
decidedAt: null,
expiresAt: "2026-07-29T12:00:00Z",
idempotencyKey: null,
targetSnapshots: { "issue-target": { status: "backlog", assigneeAgentId: null, assigneeUserId: null, updatedAt: "2026-07-01T09:00:00Z", childCount: 2 } },
continuationPolicy: "none",
metadata: {},
createdAt: "2026-07-22T09:00:00Z",
updatedAt: "2026-07-22T09:00:00Z",
...overrides,
};
}
function exec(overrides: Partial<DecisionEffectExecution>): DecisionEffectExecution {
return {
id: `exec-${overrides.effectIndex ?? 0}`,
decisionId: "decision-1",
effectIndex: 0,
effectType: "comment_on_issue",
targetIssueId: "issue-target",
status: "executed",
result: {},
error: null,
activityLogId: null,
executedAt: "2026-07-22T10:00:00Z",
...overrides,
};
}
function render(props: Partial<ComponentProps<typeof DecisionCard>>) {
const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } });
container = document.createElement("div");
document.body.appendChild(container);
root = createRoot(container);
act(() => {
root?.render(
<QueryClientProvider client={queryClient}>
<ThemeProvider>
<TooltipProvider>
<DecisionCard decision={mkDecision()} resolveIssue={resolveIssue} originAgentName="Gardener" originIssue={ISSUES["issue-origin"]} {...props} />
</TooltipProvider>
</ThemeProvider>
</QueryClientProvider>,
);
});
return container!;
}
function clickButtonWithText(el: HTMLElement, text: string) {
const button = [...el.querySelectorAll("button")].find((b) => b.textContent?.includes(text));
if (!button) throw new Error(`No button with text "${text}"`);
act(() => {
button.dispatchEvent(new MouseEvent("click", { bubbles: true }));
});
return button;
}
afterEach(() => {
if (root) act(() => root?.unmount());
container?.remove();
root = null;
container = null;
});
describe("DecisionCard", () => {
it("renders a pending decision with provenance and effect summary", () => {
const el = render({});
expect(el.textContent).toContain("Pending");
expect(el.textContent).toContain("Gardener");
expect(el.textContent).toContain("PAP-123");
expect(el.textContent).toContain("Comment on PAP-456");
});
it("links the target issue the decision applies to, not just the origin", () => {
const el = render({});
expect(el.textContent).toContain("applies to");
const provenance = el.querySelector("p");
expect(provenance?.textContent).toContain("PAP-456");
expect(
[...(provenance?.querySelectorAll("a") ?? [])].some((a) => a.getAttribute("href") === "/PAP/issues/PAP-456"),
).toBe(true);
});
it("omits the applies-to link when the decision only targets its origin issue", () => {
const el = render({
decision: mkDecision({
options: [
{ id: "comment", label: "Comment", effects: [{ type: "comment_on_issue", targetIssueId: "issue-origin", staleness: "lenient", bodyMarkdown: "nudge" }] },
],
}),
});
expect(el.textContent).not.toContain("applies to");
});
it("links secondary target issues referenced by an effect", () => {
const el = render({
decision: mkDecision({
options: [{
id: "create",
label: "Create follow-up",
effects: [{
type: "create_issue",
targetIssueId: "issue-target",
staleness: "strict",
draft: {
title: "Follow-up",
parentId: "issue-parent",
blockedByIssueIds: ["issue-blocker"],
},
}],
}],
}),
});
const targetHrefs = [...(el.querySelector("p")?.querySelectorAll("a") ?? [])].map((a) => a.getAttribute("href"));
expect(targetHrefs).toEqual(expect.arrayContaining([
"/PAP/issues/PAP-456",
"/PAP/issues/PAP-789",
"/PAP/issues/PAP-790",
]));
});
it("fires onDecide with the chosen option id", () => {
const onDecide = vi.fn();
const el = render({ onDecide });
clickButtonWithText(el, "Comment and snooze");
expect(onDecide).toHaveBeenCalledWith("comment", expect.any(Object));
});
it("offers no \"Dismiss — no effects\" verb; closing lives in the card's Close… menu (PAP-749 Rule 8)", () => {
const el = render({});
expect(el.textContent).not.toContain("Dismiss — no effects");
expect(el.textContent).not.toContain("Not now?");
});
it("warns and disables strict options when a target is stale", () => {
const el = render({
targetChanged: { "issue-target": true },
decision: mkDecision({
options: [
{ id: "strict", label: "Cancel it", effects: [{ type: "update_issue_status", targetIssueId: "issue-target", staleness: "strict", status: "cancelled" }] },
{ id: "lenient", label: "Just comment", effects: [{ type: "comment_on_issue", targetIssueId: "issue-target", staleness: "lenient", bodyMarkdown: "hi" }] },
],
}),
});
expect(el.textContent).toContain("changed since this was proposed");
expect(el.textContent).toContain("Blocked · stale");
const strict = [...el.querySelectorAll("button")].find((b) => b.textContent?.includes("Cancel it"));
expect(strict?.disabled).toBe(true);
});
it("disables strict options when a secondary target is stale", () => {
const el = render({
targetChanged: { "issue-target": false, "issue-parent": true },
decision: mkDecision({
targetSnapshots: {
"issue-target": { status: "backlog", assigneeAgentId: null, assigneeUserId: null, updatedAt: "2026-07-01T09:00:00Z", childCount: 2 },
"issue-parent": { status: "todo", assigneeAgentId: null, assigneeUserId: null, updatedAt: "2026-07-01T09:00:00Z", childCount: 0 },
},
options: [
{
id: "strict-create",
label: "Create the follow-up",
effects: [{
type: "create_issue",
targetIssueId: "issue-target",
staleness: "strict",
draft: { title: "Follow-up", parentId: "issue-parent" },
}],
},
],
}),
});
const strict = [...el.querySelectorAll("button")].find((button) => button.textContent?.includes("Create the follow-up"));
expect(strict?.disabled).toBe(true);
expect(strict?.textContent).toContain("Blocked · stale");
});
it("gates a cancel_issue_tree option behind a type-to-confirm step", () => {
const onDecide = vi.fn();
const el = render({
onDecide,
cancelTreePreview: () => [ISSUES["issue-target"]!],
decision: mkDecision({
options: [
{ id: "cancel", label: "Cancel the tree", style: "destructive", effects: [{ type: "cancel_issue_tree", targetIssueId: "issue-target", staleness: "strict", reasonComment: "stale" }] },
],
}),
});
expect(el.textContent).toContain("Destructive");
// First click opens the confirm gate rather than deciding.
clickButtonWithText(el, "Cancel the tree");
expect(onDecide).not.toHaveBeenCalled();
expect(el.textContent).toContain("to confirm");
const confirm = [...el.querySelectorAll("button")].find((b) => b.textContent?.match(/Cancel \d+ issue/));
expect(confirm?.disabled).toBe(true);
});
it("renders decided result rows with entity links", () => {
const el = render({
decision: mkDecision({ status: "decided", executionStatus: "succeeded", chosenOptionId: "create" }),
executions: [
exec({ effectIndex: 0, effectType: "create_issue", status: "executed", result: { issueId: "issue-new" } }),
exec({ effectIndex: 1, effectType: "update_issue_status", status: "executed", result: { issueId: "issue-target", status: "done" } }),
],
});
expect(el.textContent).toContain("Decided");
expect(el.textContent).toContain("Created PAP-999");
expect(el.textContent).toContain("Set PAP-456 to done");
expect([...el.querySelectorAll("a")].some((a) => a.getAttribute("href") === "/PAP/issues/PAP-999")).toBe(true);
// No option buttons on a terminal decision.
expect([...el.querySelectorAll("button")].length).toBe(0);
});
it("surfaces failure cause and the fail-closed / re-propose guidance on partial", () => {
const el = render({
decision: mkDecision({ status: "decided", executionStatus: "partial", chosenOptionId: "create" }),
executions: [
exec({ effectIndex: 0, effectType: "comment_on_issue", status: "executed" }),
exec({ effectIndex: 1, effectType: "update_issue_status", status: "failed", error: "deny_decision_intersection" }),
],
});
expect(el.textContent).toContain("Partial");
expect(el.textContent).toContain("permission boundary");
expect(el.textContent).toContain("may already have been applied");
expect(el.textContent).toContain("re-propose");
});
it("marks skipped effects as target-changed on a failed decision", () => {
const el = render({
decision: mkDecision({ status: "decided", executionStatus: "failed", chosenOptionId: "cancel" }),
executions: [exec({ effectIndex: 0, effectType: "cancel_issue_tree", status: "skipped", error: "target_changed" })],
});
expect(el.textContent).toContain("Failed");
expect(el.textContent).toContain("target changed since proposal");
});
it("renders expired and dismissed terminal states", () => {
const expired = render({ decision: mkDecision({ status: "expired", metadata: { expiredReason: "ttl" } }) });
expect(expired.textContent).toContain("Expired");
expect(expired.textContent).toContain("window closed");
act(() => root?.unmount());
container?.remove();
const dismissed = render({ decision: mkDecision({ status: "decided", executionStatus: "succeeded", chosenOptionId: "dismissed", metadata: { dismissed: true } }), executions: [] });
expect(dismissed.textContent).toContain("Dismissed");
expect(dismissed.textContent).toContain("no effects were run");
});
it("explains when a decision expires because its targets completed", () => {
const expired = render({
decision: mkDecision({ status: "expired", metadata: { expiredReason: "target_completed" } }),
});
expect(expired.textContent).toContain("target issues were completed");
expect(expired.textContent).not.toContain("expiry deadline");
});
});
describe("DecisionCard session replay", () => {
it("keeps agent-written option labels, descriptions and field labels out of session replay", () => {
const el = render({ decision: mkDecision({
options: [{ id: "reply", label: "Reply to Globex renewal", description: "Draft answer to jane@globex.com", effects: [{ type: "comment_on_issue", targetIssueId: "issue-target", staleness: "lenient", bodyMarkdown: "nudge" }] }],
inputs: [{ id: "note", label: "Note for jane@globex.com", type: "text" }] as unknown as Decision["inputs"],
}) });
expect(replayLeaks("Reply to Globex renewal", el)).toEqual({ rendered: true, recorded: false });
expect(replayLeaks("Draft answer to jane@globex.com", el)).toEqual({ rendered: true, recorded: false });
expect(replayLeaks("Note for jane@globex.com", el)).toEqual({ rendered: true, recorded: false });
});
});