Match full visible content and exact link destinations while retaining source Markdown and native run attribution. Preserve the original browser regression and strengthen decline source checks.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip manages AI agents that do work for users.
> - Product E2E journeys check user outcomes on real deployments.
> - Native task threads can render saved final replies from a settled
run.
> - The new QA Plan and Stop checks looked only for agent comments.
> - Both flows reached the requested terminal state and showed the
correct reply, but that storage assumption rejected them.
> - This PR checks the reloaded visible reply and binds it to the final
successful run. Explicit presentation decisions remain authoritative.
## Linked Issues or Issue Description
Refs #18222 and #18244.
**What happened?**
The deployed Plan journey completed all three native runs, including
revision and approval. Stop completed native cancellation, resource
release and a new direction. Their final-answer checks failed because
the comments endpoint did not contain the rendered native reply. The
original failed grades remain unchanged.
**Expected behavior**
Require exactly one correct visible final answer after reload,
attributed to the final successful owned run. Reject an earlier run,
progress text, hidden content, duplicate answers or a cancelled run. If
its selected completion comment is projected, require that comment and
matching author/run/text. Otherwise require a single durable final
assistant message and its settled native final-response anchor.
**Steps to reproduce**
Run the Plan or Stop journey on a native deployment whose task thread
renders the settled run before a completion comment is projected.
## What Changed
- Add a narrow reply observer over the existing task-thread hooks. It
reloads the page and refreshes saved run/comment metadata while waiting.
- Bind native replies to a validated durable final assistant event and
the successful run's settled final-response anchor. Require the selected
comment when projected; support pending or suppressed projection without
accepting a summary fallback. Accept payload and item-normalized finals,
rejecting contradictory fields.
- Use the observer for Plan completion and Stop's new direction. Keep
existing run counts, deployment binding, cancellation and artifact
checks.
- Add negative grading tests, include the helper in the definition hash,
and document its evidence contract. Classic protocol comment matchers
are unchanged.
## Verification
- 63 focused tests passed across native reply evidence, agent evidence
and journey contracts.
- Product E2E TypeScript passed. The reply observer uses the remaining
journey deadline and backs off polling.
- Live Plan and Stop attempts exposed this grading failure. Their
original outcomes remain failed; fresh qualification is required after
merge.
- Full current-head CI passed at
`40840ed742977853630d733292461ad7003fc7fb`; Greptile and Paperclip
Review both scored 5/5, with zero unresolved review threads.
## Risks
The observer depends on existing task-thread test hooks and run anchors.
Missing or changed hooks fail closed. It reads rendered user-facing
text; a finish summary alone cannot satisfy it. Native event provenance,
run ownership and exact reply equality are required. No backend, model,
provider, budget or cancellation policy changes are included.
## Model Used
OpenAI Codex, GPT-6 family. The exact serving model ID and context
window are not exposed by this session. Used reasoning, source editing,
shell execution and test inspection.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
