Files
PaperClipAI/ui/src/components/task-chat/reasoning-summary.test.ts
T
DottaandClaude Opus 5.5 d8c47159d4 feat(task-chat): show Codex reasoning summaries as headline bullets (#19271)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The task chat shows what an agent does while it runs, including a
"Thinking" row for provider reasoning
> - Real Codex runs produced reasoning items with no text
> - So the Thinking row only says "Thinking" and never tells the user
what the agent works on
> - This pull request asks Codex for its own reasoning summaries, keeps
every summary part, and shows the parts as bullets
> - The benefit is that a user can see what the agent is thinking about,
and a collapsed live run shows the newest headline as its status line

## Linked Issues or Issue Description

Refs #19252 (same pattern: show a harness signal that Paperclip already
receives)

**Subsystem affected**
Native runner (Rust runnerd Codex provider and event normalization, JS
Codex driver) and the task chat UI (Thinking row and activity rows).

**Problem or motivation**
Codex 0.160 resolves an unset reasoning summary from the model, and
current models resolve it to `"none"`. The rollout of a real gpt-5.6-sol
run records `turn_context.summary = "none"`, and all of its reasoning
items have `summary: []`. A local A/B with `codex exec` proved the
cause: `model_reasoning_summary=none` gives no reasoning text, and
`auto` gives headlines. Also, runnerd dropped `item/reasoning/*`
notifications and ignored the reasoning item's `summary[]`.

**Proposed solution**
Send `summary: "auto"` on Codex `turn/start`. Map
`item/reasoning/summaryTextDelta` and `summaryPartAdded` to reasoning
deltas, with a blank line before each later part. Use the joined
`summary[]` as the completed item text. In the UI, split the provider
text at headline lines and show one bullet per part.

**Alternatives considered**
Use `summary: "detailed"`. That gives longer text, but `auto` lets the
provider choose the best summarizer.

**Roadmap alignment**
This makes existing provider signals visible. It adds no new protocol
event type or data model.

**Additional context**
Research note: overlooked harness signals, item 2. Claude is out of
scope here. A live Claude run showed thinking blocks with signatures and
empty text: current models default `thinking.display` to `"omitted"`.
Claude Code ignores the `showThinkingSummaries` setting in SDK mode, and
ACPX does not forward a `thinking` option, so the fix needs a
claude-agent-acp patch hunk. That patch is also changed by #19264, so
the Claude change follows after #19264 merges. The UI in this PR already
shows prose-only (Claude) summaries.

## What Changed

- Rust runnerd (`codex_provider.rs`): send `summary: "auto"` on every
Codex `turn/start`.
- Rust runnerd (`provider_events.rs`): map
`item/reasoning/summaryTextDelta` and later `summaryPartAdded` to
`item.delta` with `kind: "reasoning"` and `channel: "summary"`. Use the
joined `summary[]` as a completed reasoning item's text.
- JS Codex driver: send `summary: "auto"` on direct app-server
transports, map `summaryPartAdded`, and read `summary[]` in `itemText`.
- UI: new `reasoning-summary.ts` splits provider text into headline
sections, and `TaskChatReasoningSummary` shows one bullet per part. The
Thinking row and the activity row use the newest headline as their
one-line status. Prose-only summaries (Claude) stay as prose.
- Storybook: new "Task chat reasoning summaries" story with live,
expanded, finished and Claude states.
- Tests: Rust normalizer test, JS driver test, and splitter unit tests.

## Verification

- `cargo test -p paperclip-runner-core` (provider events and the shared
identity golden).
- `pnpm --filter @paperclipai/paperclip-runner exec vitest run
src/drivers/codex`
- `pnpm --filter @paperclipai/ui exec vitest run
src/components/task-chat src/components/transcript`
- `pnpm check:token-gates`
- Live test on a local worktree instance with a real Codex agent
(gpt-5.6-sol). The Codex rollout now records `summary: "auto"`. Run
events contain `**Assessing transaction boundary issues**`, a part
break, and `**Planning atomic claim fix and verification**`. The task
chat shows "Thought · Planning atomic claim fix and verification", and
the expanded row lists both headlines as bullets.
- Storybook: open "Tasks and conversations / Task creation and lifecycle
/ Tasks / Task chat reasoning summaries".
- Claude was not changed. A live Claude run confirmed the cause for the
follow-up (empty, signed thinking blocks).

## Risks

- Codex now writes reasoning summaries, which adds a small number of
output tokens per turn. This is the same cost as Codex CLI with its
normal default.
- The UI treats a line that is only bold text as a headline. A prose
line that is only bold text also starts a new bullet. This is display
only.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use,
code execution and browser automation. Default context window.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:21:50 -06:00

52 lines
2.8 KiB
TypeScript

import { describe, expect, it } from "vitest";
import { hasReasoningHeadlines, latestReasoningHeadline, reasoningSummarySections } from "./reasoning-summary";
const CODEX = "**Tracing the retry path**\n\nThe worker retries on 5xx.\n\n**Verifying edge cases**\n\nTwo workers can claim one retry.";
describe("reasoning summary sections", () => {
it("starts a section at every provider headline and keeps its prose", () => {
expect(reasoningSummarySections(CODEX)).toEqual([
{ headline: "Tracing the retry path", body: "The worker retries on 5xx." },
{ headline: "Verifying edge cases", body: "Two workers can claim one retry." },
]);
});
it("splits headline-only parts streamed with blank-line separators", () => {
const sections = reasoningSummarySections("**Reading the worker**\n\n**Checking `event_id`**");
expect(sections.map((section) => section.headline)).toEqual(["Reading the worker", "Checking event_id"]);
expect(latestReasoningHeadline("**Reading the worker**\n\n**Checking `event_id`**")).toBe("Checking event_id");
});
it("leaves prose-only summaries as one headline-free section", () => {
const sections = reasoningSummarySections("The user wants **no** contract change.\nSo keep it small.");
expect(hasReasoningHeadlines(sections)).toBe(false);
expect(sections).toHaveLength(1);
expect(latestReasoningHeadline("Plain prose.")).toBeNull();
});
it("shows a headline while its closing emphasis is still streaming", () => {
expect(latestReasoningHeadline("**Tracing the retry path**\n\n**Verifying edge")).toBe("Verifying edge");
expect(latestReasoningHeadline("**Tracing the retry path**\n\nThe **worker")).toBe("Tracing the retry path");
});
it("keeps bold lines inside fenced code as code, not headlines", () => {
const text = "**Drafting the example**\n\n```md\n**Example**\nbody\n```\n\nDone.";
const sections = reasoningSummarySections(text);
expect(sections.map((section) => section.headline)).toEqual(["Drafting the example"]);
expect(sections[0]?.body).toContain("**Example**");
expect(latestReasoningHeadline(text)).toBe("Drafting the example");
});
it("treats single-line triple-backtick code as inline, not an open fence", () => {
const text = "**Checking the constant**\n\n```const x = 1```\n\n**Verifying edge cases**";
expect(reasoningSummarySections(text).map((section) => section.headline))
.toEqual(["Checking the constant", "Verifying edge cases"]);
expect(latestReasoningHeadline(text)).toBe("Verifying edge cases");
});
it("closes a fence only with a matching fence of at least the same length", () => {
const text = "**Drafting**\n\n````md\n```\n**Inside**\n````\n\n**After the sample**";
expect(reasoningSummarySections(text).map((section) => section.headline)).toEqual(["Drafting", "After the sample"]);
});
});