mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 16:02:36 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The task chat shows what an agent does while it runs, including a "Thinking" row for provider reasoning > - Real Codex runs produced reasoning items with no text > - So the Thinking row only says "Thinking" and never tells the user what the agent works on > - This pull request asks Codex for its own reasoning summaries, keeps every summary part, and shows the parts as bullets > - The benefit is that a user can see what the agent is thinking about, and a collapsed live run shows the newest headline as its status line ## Linked Issues or Issue Description Refs #19252 (same pattern: show a harness signal that Paperclip already receives) **Subsystem affected** Native runner (Rust runnerd Codex provider and event normalization, JS Codex driver) and the task chat UI (Thinking row and activity rows). **Problem or motivation** Codex 0.160 resolves an unset reasoning summary from the model, and current models resolve it to `"none"`. The rollout of a real gpt-5.6-sol run records `turn_context.summary = "none"`, and all of its reasoning items have `summary: []`. A local A/B with `codex exec` proved the cause: `model_reasoning_summary=none` gives no reasoning text, and `auto` gives headlines. Also, runnerd dropped `item/reasoning/*` notifications and ignored the reasoning item's `summary[]`. **Proposed solution** Send `summary: "auto"` on Codex `turn/start`. Map `item/reasoning/summaryTextDelta` and `summaryPartAdded` to reasoning deltas, with a blank line before each later part. Use the joined `summary[]` as the completed item text. In the UI, split the provider text at headline lines and show one bullet per part. **Alternatives considered** Use `summary: "detailed"`. That gives longer text, but `auto` lets the provider choose the best summarizer. **Roadmap alignment** This makes existing provider signals visible. It adds no new protocol event type or data model. **Additional context** Research note: overlooked harness signals, item 2. Claude is out of scope here. A live Claude run showed thinking blocks with signatures and empty text: current models default `thinking.display` to `"omitted"`. Claude Code ignores the `showThinkingSummaries` setting in SDK mode, and ACPX does not forward a `thinking` option, so the fix needs a claude-agent-acp patch hunk. That patch is also changed by #19264, so the Claude change follows after #19264 merges. The UI in this PR already shows prose-only (Claude) summaries. ## What Changed - Rust runnerd (`codex_provider.rs`): send `summary: "auto"` on every Codex `turn/start`. - Rust runnerd (`provider_events.rs`): map `item/reasoning/summaryTextDelta` and later `summaryPartAdded` to `item.delta` with `kind: "reasoning"` and `channel: "summary"`. Use the joined `summary[]` as a completed reasoning item's text. - JS Codex driver: send `summary: "auto"` on direct app-server transports, map `summaryPartAdded`, and read `summary[]` in `itemText`. - UI: new `reasoning-summary.ts` splits provider text into headline sections, and `TaskChatReasoningSummary` shows one bullet per part. The Thinking row and the activity row use the newest headline as their one-line status. Prose-only summaries (Claude) stay as prose. - Storybook: new "Task chat reasoning summaries" story with live, expanded, finished and Claude states. - Tests: Rust normalizer test, JS driver test, and splitter unit tests. ## Verification - `cargo test -p paperclip-runner-core` (provider events and the shared identity golden). - `pnpm --filter @paperclipai/paperclip-runner exec vitest run src/drivers/codex` - `pnpm --filter @paperclipai/ui exec vitest run src/components/task-chat src/components/transcript` - `pnpm check:token-gates` - Live test on a local worktree instance with a real Codex agent (gpt-5.6-sol). The Codex rollout now records `summary: "auto"`. Run events contain `**Assessing transaction boundary issues**`, a part break, and `**Planning atomic claim fix and verification**`. The task chat shows "Thought · Planning atomic claim fix and verification", and the expanded row lists both headlines as bullets. - Storybook: open "Tasks and conversations / Task creation and lifecycle / Tasks / Task chat reasoning summaries". - Claude was not changed. A live Claude run confirmed the cause for the follow-up (empty, signed thinking blocks). ## Risks - Codex now writes reasoning summaries, which adds a small number of output tokens per turn. This is the same cost as Codex CLI with its normal default. - The UI treats a line that is only bold text as a headline. A prose line that is only bold text also starts a new bullet. This is display only. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use, code execution and browser automation. Default context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
52 lines
2.8 KiB
TypeScript
52 lines
2.8 KiB
TypeScript
import { describe, expect, it } from "vitest";
|
|
import { hasReasoningHeadlines, latestReasoningHeadline, reasoningSummarySections } from "./reasoning-summary";
|
|
|
|
const CODEX = "**Tracing the retry path**\n\nThe worker retries on 5xx.\n\n**Verifying edge cases**\n\nTwo workers can claim one retry.";
|
|
|
|
describe("reasoning summary sections", () => {
|
|
it("starts a section at every provider headline and keeps its prose", () => {
|
|
expect(reasoningSummarySections(CODEX)).toEqual([
|
|
{ headline: "Tracing the retry path", body: "The worker retries on 5xx." },
|
|
{ headline: "Verifying edge cases", body: "Two workers can claim one retry." },
|
|
]);
|
|
});
|
|
|
|
it("splits headline-only parts streamed with blank-line separators", () => {
|
|
const sections = reasoningSummarySections("**Reading the worker**\n\n**Checking `event_id`**");
|
|
expect(sections.map((section) => section.headline)).toEqual(["Reading the worker", "Checking event_id"]);
|
|
expect(latestReasoningHeadline("**Reading the worker**\n\n**Checking `event_id`**")).toBe("Checking event_id");
|
|
});
|
|
|
|
it("leaves prose-only summaries as one headline-free section", () => {
|
|
const sections = reasoningSummarySections("The user wants **no** contract change.\nSo keep it small.");
|
|
expect(hasReasoningHeadlines(sections)).toBe(false);
|
|
expect(sections).toHaveLength(1);
|
|
expect(latestReasoningHeadline("Plain prose.")).toBeNull();
|
|
});
|
|
|
|
it("shows a headline while its closing emphasis is still streaming", () => {
|
|
expect(latestReasoningHeadline("**Tracing the retry path**\n\n**Verifying edge")).toBe("Verifying edge");
|
|
expect(latestReasoningHeadline("**Tracing the retry path**\n\nThe **worker")).toBe("Tracing the retry path");
|
|
});
|
|
|
|
it("keeps bold lines inside fenced code as code, not headlines", () => {
|
|
const text = "**Drafting the example**\n\n```md\n**Example**\nbody\n```\n\nDone.";
|
|
const sections = reasoningSummarySections(text);
|
|
expect(sections.map((section) => section.headline)).toEqual(["Drafting the example"]);
|
|
expect(sections[0]?.body).toContain("**Example**");
|
|
expect(latestReasoningHeadline(text)).toBe("Drafting the example");
|
|
});
|
|
|
|
it("treats single-line triple-backtick code as inline, not an open fence", () => {
|
|
const text = "**Checking the constant**\n\n```const x = 1```\n\n**Verifying edge cases**";
|
|
expect(reasoningSummarySections(text).map((section) => section.headline))
|
|
.toEqual(["Checking the constant", "Verifying edge cases"]);
|
|
expect(latestReasoningHeadline(text)).toBe("Verifying edge cases");
|
|
});
|
|
|
|
it("closes a fence only with a matching fence of at least the same length", () => {
|
|
const text = "**Drafting**\n\n````md\n```\n**Inside**\n````\n\n**After the sample**";
|
|
expect(reasoningSummarySections(text).map((section) => section.headline)).toEqual(["Drafting", "After the sample"]);
|
|
});
|
|
});
|