fix(ui): store = as typed, without the highlight exporter's escapes (#19540)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - People write task requests and chat messages in the Markdown editor,
and the stored text becomes the agent's prompt
> - MDXEditor supports `==highlight==` marks, and its exporter escapes
every `=` in prose to protect that syntax
> - The editor therefore stores `a=b` as `a\=b` and `x=>y` as `x\=>y`
> - A command or code snippet in a chat message then reaches the agent
as invalid code
> - This pull request removes the `=` escape when the escape cannot
change how the text parses, as #19350 did for intraword underscores
> - The benefit is that commands, assignments and arrows reach agents
exactly as the person typed them

## Linked Issues or Issue Description

Refs #19350

**What happened?**
A chat evaluation sent a `node -e` command through the chat composer.
The stored comment contained `const fs\=require(...)` and `()\=>`. Every
`=` had a backslash. The agent received invalid JavaScript, and an
exact-text check on the stored message failed.

**Expected behavior**
Text the person types is stored as typed. A `=` that cannot start or end
a highlight needs no escape.

**Steps to reproduce**
1. Type `a=b` or `x => y` in any Markdown editor field, such as the chat
composer.
2. Read the stored value through the API.
3. Before this change it is `a\=b` or `x \=> y`.

## What Changed

- New `ui/src/lib/equals-markdown.ts`: `unescapeEqualsSigns` removes
`\=` escapes from prose only when they cannot change the parse:
- any run of escaped `=` whose length is not exactly two (a highlight
sequence is exactly two `=`);
- a run of exactly two with whitespace or the text edge on both sides
(such a run can neither open nor close a highlight).
- Other two-runs, such as `a\=\=b` or `\=\=text\=\=`, stay escaped.
Escaped backslashes (`\\=`) stay as they are.
- `ui/src/lib/underscore-markdown.ts`: the code-range walk is now the
shared `rewriteMarkdownProse` helper. Code spans, code blocks and raw
HTML stay byte-identical for both rewrites. Underscore behavior does not
change.
- `MarkdownEditor.tsx`: `toStoredMarkdown` applies the new step after
the underscore step, on the exporter's exact text.
- Tests: unit cases for commands, `===`, spaced `==`, quotes and lists,
highlight-capable runs, code spans and blocks, raw HTML and escaped
backslashes. One editor test for a pasted command with a code block.

## Verification

- `npx vitest run src/components/MarkdownEditor.test.tsx src/lib
src/components/task-chat` in `ui/`: 280 files, 3388 tests passed.
- Negative control: with the new step removed from `toStoredMarkdown`,
the new editor test fails.
- Parser check: I parsed the unescaped forms with
`micromark-extension-highlight-mark` 1.2.0, the version MDXEditor uses.
`a=b and c=d`, `x === y`, `a == b == c`, `()=>x` and `x!==y` produce
zero highlight nodes. The controls `==a==`, `a ==b== c` and `say
==this== now` produce one highlight node each.
- `tsc -p ui/tsconfig.json`: no errors in the changed files. The
remaining errors are in files this change does not touch.

## Risks

- Low risk. The step only removes an escape when the highlight tokenizer
cannot use that `=`. Text the parser cannot place is returned unchanged.
- Some `==` in code-like prose (for example `x!==y`) stays escaped. This
is intentional: such a run can open a highlight.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed 2026-11-06 17:27:59 -06:00
1 parent a3f80ea043
commit 411eecdccd
5 files changed
+159 -8

No files matched your search

+21
View File
@@ -1053,6 +1053,27 @@ describe("MarkdownEditor", () => {
});
});
it("stores a pasted command's `=` as typed instead of the highlight exporter's escaped form", async () => {
const value = "Run node -e 'const fs=require(\"fs\");x.map(a=>a)' then check a == b.\n\n```js\nconst keep\\=1\n```";
// MDXEditor's highlight extension escapes every prose `=`; code is exported as typed.
mdxEditorMockState.emitMountChange =
"Run node -e 'const fs\\=require(\"fs\");x.map(a\\=>a)' then check a \\=\\= b.\n\n```js\nconst keep\\=1\n```";
const handleChange = vi.fn();
const root = createRoot(container);
await act(async () => {
root.render(<MarkdownEditor value={value} onChange={handleChange} placeholder="Markdown body" />);
});
await flush();
expect(handleChange).toHaveBeenCalledWith(value);
await act(async () => {
root.unmount();
});
});
it("keeps a code span after an escaped HTML-like tag byte-identical", async () => {
const stored = "<div>\n`a\\_b` and snake_case";
// The exporter escapes the `<` (reversed on storage) and the prose underscore.
+4 -3
View File
@@ -59,6 +59,7 @@ import {
} from "../lib/angle-bracket-markdown";
import { unescapeBlockquoteMarkers } from "../lib/blockquote-markdown";
import { unescapeIntrawordUnderscores } from "../lib/underscore-markdown";
import { unescapeEqualsSigns } from "../lib/equals-markdown";
import { restoreTrailingMarkdownSpaces } from "../lib/trailing-markdown-spaces";
import { pasteNormalizationPlugin } from "../lib/paste-normalization";
import { cn } from "../lib/utils";
@@ -191,9 +192,9 @@ function prepareMarkdownForEditor(value: string): string {
*/
function toStoredMarkdown(markdown: string): string {
const withTrailingSpaces = restoreTrailingMarkdownSpaces(markdown);
// Unescape underscores first, on the exporter's exact text: removing `\<` or
// `\>` can turn following lines into HTML or blockquotes and move code ranges.
return unescapeAngleBracketEscapes(unescapeBlockquoteMarkers(unescapeIntrawordUnderscores(withTrailingSpaces)));
// Unescape underscores and `=` first, on the exporter's exact text: removing `\<`
// or `\>` can turn following lines into HTML or blockquotes and move code ranges.
return unescapeAngleBracketEscapes(unescapeBlockquoteMarkers(unescapeEqualsSigns(unescapeIntrawordUnderscores(withTrailingSpaces))));
}
function escapeRegExp(value: string): string {
+66
View File
@@ -0,0 +1,66 @@
import { describe, expect, it } from "vitest";
import { unescapeEqualsSigns } from "./equals-markdown";
const unchanged = (value: string) => expect(unescapeEqualsSigns(value)).toBe(value);
describe("unescapeEqualsSigns", () => {
it("restores `=` the person typed in commands, assignments and arrows", () => {
expect(unescapeEqualsSigns("Run node -e 'const fs\\=require(\"fs\");x.map(a\\=>a)'"))
.toBe("Run node -e 'const fs=require(\"fs\");x.map(a=>a)'");
expect(unescapeEqualsSigns("Set KEY\\=value and a \\= b")).toBe("Set KEY=value and a = b");
expect(unescapeEqualsSigns("strict x \\=\\=\\= y and x!\\=\\=\\=y")).toBe("strict x === y and x!===y");
expect(unescapeEqualsSigns("compare a \\=\\= b")).toBe("compare a == b");
expect(unescapeEqualsSigns("> quoted a\\=b\n\n- listed c\\=d")).toBe("> quoted a=b\n\n- listed c=d");
});
it("keeps `==` escaped wherever it could open or close a highlight", () => {
for (const value of [
"\\=\\=highlight\\=\\=",
"a\\=\\=b",
// `!==` before a letter can open a highlight, so it stays escaped.
"x!\\=\\=y",
// A line of `=` under text is a heading underline.
"Title\n\\=\\=\\=",
"Title\n\\=",
"> Title\n> \\=\\=",
"- item\n \\=",
// The exporter escapes a leading `>` too; it still starts the line.
"\\> Title\n\\> \\=\\=\\=",
"\\> Title\n\\> \\=\\= ",
"say \\=\\=this",
"that\\=\\= now",
"a\\\\=b",
]) unchanged(value);
});
it("keeps every code span and code block byte-identical", () => {
for (const value of [
"Use `a\\=b` literally",
"```\nconst a\\=1\n```",
" indented a\\=b",
"> ```\n> a\\=b\n> ```",
"<span title=\"a\\=b\">x</span>",
]) unchanged(value);
expect(unescapeEqualsSigns("prose a\\=b and `code a\\=b`")).toBe("prose a=b and `code a\\=b`");
// Pairs cut off by a code span do not see whitespace at the chunk edge.
unchanged("x \\=\\=`code`\\=\\= y");
});
it("keeps an escape after an escaped backslash and removes it after an odd run", () => {
// `\\\=` is an escaped backslash followed by an escaped `=`; only the `=` escape goes.
expect(unescapeEqualsSigns("a\\\\\\=b")).toBe("a\\\\=b");
unchanged("a\\\\=b");
});
it("stays linear on one long line with many assignments", () => {
const line = Array.from({ length: 25_000 }, (_, index) => `k${index}\\=v`).join(" ");
const started = performance.now();
expect(unescapeEqualsSigns(line)).toBe(line.replaceAll("\\=", "="));
expect(performance.now() - started).toBeLessThan(1_000);
});
it("leaves text without escaped equals untouched", () => {
unchanged("plain a=b");
unchanged("");
});
});
+55
View File
@@ -0,0 +1,55 @@
import { isEscaped, rewriteMarkdownProse } from "./underscore-markdown";
/**
* MDXEditor's `==highlight==` extension (`mdast-util-highlight-mark`) marks
* every `=` in prose as unsafe, so the exporter stores `a=b` as `a\=b` and
* `() => x` as `()\=> x`. Stored text feeds agent prompts, so a pasted command
* or snippet reaches the agent with backslashes the person never typed.
*
* A highlight opens or closes only on a run of exactly two `=`, and a two-run
* with whitespace on both sides can do neither. A run that starts a line can
* underline a heading, so it keeps its escape. Other escapes cannot change how
* the text parses, so they are removed. Code
* spans, code blocks and raw HTML stay byte-identical, as do escaped
* backslashes (`\\=`).
*/
const ESCAPED_EQUALS_RUN_RE = /(?:\\=)+/g;
function unescapeProse(text: string): string {
// One forward pass: matches arrive in order, so track whether only
// indentation or quote markers precede the current offset on its line.
let scanned = 0;
let lineStartOnly = true;
return text.replace(ESCAPED_EQUALS_RUN_RE, (run: string, offset: number, whole: string) => {
for (; scanned < offset; scanned += 1) {
const char = whole[scanned]!;
if (char === "\n") lineStartOnly = true;
// The exporter writes a leading `>` as `\>`; it is still a quote marker.
else if (lineStartOnly && char === "\\" && whole[scanned + 1] === ">") scanned += 1;
else if (char !== ">" && !/\s/u.test(char)) lineStartOnly = false;
}
const atLineStart = lineStartOnly;
scanned = offset + run.length;
lineStartOnly = false;
if (isEscaped(whole, offset)) return run;
// At the start of a line (after indentation or quote markers) a run of `=`
// can underline the previous line as a heading, so its escape stays. This
// includes the start of a prose chunk, whose preceding text is unknown.
if (atLineStart) return run;
const length = run.length / 2;
const before = whole[offset - 1];
const after = whole[offset + run.length];
// A neighbouring literal `=` would join this run into a different length.
if (before === "=" || after === "=") return run;
// A chunk edge borders a code span or block, not whitespace, so it never
// counts as space around a two-run.
const spaced = (char: string | undefined) => char !== undefined && /\s/u.test(char);
if (length === 2 && !(spaced(before) && spaced(after))) return run;
return "=".repeat(length);
});
}
export function unescapeEqualsSigns(markdown: string): string {
if (!markdown.includes("\\=")) return markdown;
return rewriteMarkdownProse(markdown, unescapeProse);
}
+13 -5
View File
@@ -26,7 +26,7 @@ interface PositionedNode {
}
/** True when the character at `index` is escaped by an odd number of backslashes. */
function isEscaped(text: string, index: number): boolean {
export function isEscaped(text: string, index: number): boolean {
let count = 0;
for (let i = index - 1; i >= 0 && text[i] === "\\"; i -= 1) count += 1;
return count % 2 === 1;
@@ -50,8 +50,11 @@ function codeRanges(node: PositionedNode, ranges: Array<[number, number]>): Arra
return ranges;
}
export function unescapeIntrawordUnderscores(markdown: string): string {
if (!markdown.includes("\\_")) return markdown;
/**
* Apply `rewrite` to prose only: code spans, code blocks and raw HTML stay
* byte-identical. Text the parser cannot place is returned unchanged.
*/
export function rewriteMarkdownProse(markdown: string, rewrite: (prose: string) => string): string {
let ranges: Array<[number, number]>;
try {
ranges = codeRanges(fromMarkdown(markdown) as PositionedNode, []);
@@ -63,8 +66,13 @@ export function unescapeIntrawordUnderscores(markdown: string): string {
let cursor = 0;
for (const [start, end] of ranges) {
if (start < cursor) continue;
result += unescapeProse(markdown.slice(cursor, start)) + markdown.slice(start, end);
result += rewrite(markdown.slice(cursor, start)) + markdown.slice(start, end);
cursor = end;
}
return result + unescapeProse(markdown.slice(cursor));
return result + rewrite(markdown.slice(cursor));
}
export function unescapeIntrawordUnderscores(markdown: string): string {
if (!markdown.includes("\\_")) return markdown;
return rewriteMarkdownProse(markdown, unescapeProse);
}