1 Commits

Include renames

Author	SHA1	Message	Date
DottaandPaperclip	81ebe479e8	fix(evals): bind rendered Markdown replies to saved run evidence (#18264) ... ## Thinking Path > - Paperclip lets people assign work to AI agents and read saved results. > - Product journeys must prove that a result is visible and belongs to the completed run. > - The shared observer compares stored Markdown with browser text. > - A document link can render as an issue chip, so those strings differ. > - This PR keeps run and comment attribution, then compares full text and exact link destinations. > - The benefit is a valid browser result without a false timeout or a weaker provenance check. ## Linked Issues or Issue Description Refs: #18248, #18255. **What happened?** The protected-action story completed both decisions and all four native turns. Its final explanation contained a document link. The browser rendered that link as an issue chip. The observer waited for raw Markdown to equal rendered text, so it could not accept the valid result. **Expected behavior** Accept complete matching content from the final run after reload. Keep the exact link destination, task anchor, author, run and durable final-message checks. Reject missing or changed content. **Steps to reproduce** 1. Save an attributed reply with bold text and a Markdown document link. 2. Render the link as an issue chip in the task bubble. 3. Run `readBoundTaskFinalReply`. The original observer times out. The new browser regression preserves that failure. ## What Changed - Add a UI-owned content projection using its existing Markdown renderer and GFM/issue-reference plugins. Keep DOM observation, attribution and destination checks in the task-reply observer. Keep source Markdown and observed text separately. - Read visible DOM text, link labels and destinations. Require the structured projection to match the actual visible text. Allow only recognized product chips to replace a link label. Ordinary links keep their source label. Reuse the production issue-reference and company-route helpers, plus the current task URL. Ordinary root-relative anchors keep their original path. Preserve exact marker checks and all run/comment/native-event attribution checks. - Check visible decline explanations and unread data in preserved native source text. - Add browser regressions and source/content tests. Reuse the UI’s existing `react-markdown` and `remark-gfm` dependencies. The final diff changes no package manifest or lockfile. - Match generated GFM checkbox states and reject changed states. Document supported content and fail-closed behavior. ## Verification - Original observer: the decorated-link browser regression fails at the final-reply predicate. - Fixed observer: all 21 browser regressions pass, without credentials or model calls. - Focused observer, decline, content and journey contract suites: 64 tests pass. - E2E and UI TypeScript checks pass. All UI token gates pass. The unchanged frozen install passes. - Full CI passed on the preceding 6eba7ff head (41003112166). Full CI passed on the preceding b9a2571 head (41006205475), with both reviewers at 5/5. Current-head full CI passed at b3d2d08305 (41009669966), including typecheck, build and all eight E2E shards. Both reviewers scored this commit 5/5, with no unresolved threads. The initial lockfile policy failure was corrected. The final diff has no manifest or lock changes, and its unchanged frozen install passed. - The optional positive GFM strikethrough assertion also passes. - The Node 24/tsx end-to-end observer probe passes. The callback has no external compiler helper; a browser regression covers actual tsx serialization. - The Paperclip review finding is also fixed: bare URLs and issue keys use the production plugins and the UI’s `scope=accessible` company-prefix context, with RED/GREEN browser coverage. Source Markdown is required in the receipt. - All Greptile findings are fixed, including matching the UI’s accessible-company scope: ordinary link labels, issue-prefix/fragment resolution and ordinary root-relative paths have positive and negative browser coverage. - The deployed attempt remains a failed original measurement. Fresh qualification is required after this repair. ## Risks The comparison normalizes whitespace and inline formatting. It verifies full text and exact link targets, while link labels may become product chips. Images, raw HTML, tables and fenced code fail closed. Those constructs may require a future explicit extension if a journey uses them. No application behavior, model, budget, approval or connection permission changes. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, code execution and browser tools. The host does not expose an exact underlying model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal issue identifier - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 00:57:59 -05:00