Files
PaperClipAI/ui/src/lib/summary-draft-stream.ts
T
1f1f545238 feat: add built-in summarizer and summary slots (#9713)
## Thinking Path

> - Paperclip is the open source control plane people use to organize,
govern, and understand AI-agent work
> - Operators need concise, current status views across projects and
execution workspaces without manually reading every issue and run
> - Paperclip already has auditable issues, documents, built-in agents,
routines, and live run events, but no first-class summary-slot workflow
connecting those systems
> - A built-in Summarizer can generate status prose through ordinary
governed tasks while summary slots provide stable, revisioned
destinations for that output
> - The UI needs to show current summaries, generation progress,
failures, revisions, and streaming draft status in the places operators
already work
> - This pull request adds the end-to-end summary-slot data, API, agent,
orchestration, and UI surfaces behind an experimental setting
> - The benefit is decision-oriented status context that remains
company-scoped, auditable, retryable, and inexpensive by default

## Linked Issues or Issue Description

No public GitHub issue exists for this feature.

**Problem**

Operators currently have to reconstruct project and workspace status by
reading many issues, runs, and comments. This makes it hard to identify
decisions, review queues, recent work, and the next event worth
watching.

**Proposed capability**

Add an experimental summary system with revisioned summary slots for
projects and workspaces, a paused-by-default built-in Summarizer agent,
governed generation tasks, live draft status, and reusable UI cards.

**Expected behavior**

- Summary data remains company-scoped and revisions remain auditable.
- Generation runs through normal issue/agent orchestration and
deduplicates active requests.
- Only the linked built-in Summarizer generation task can author a slot
revision.
- Operators can generate, retry, inspect revisions, and follow draft
progress from project and workspace views.
- The feature remains opt-in and background generation remains paused by
default.

## What Changed

- Added summary-slot schema, idempotent migrations, shared contracts,
validators, API paths, and service tests.
- Added company-scoped summary-slot routes for reading revisions,
requesting generation, and guarded Summarizer writes with activity
logging.
- Added terminal generation finalization, failure reasons, assignment
wakeups, and orchestration integration.
- Added the paused-by-default built-in Summarizer bundle, low-cost
runtime defaults, status-summarization skill, and stale-summary routine.
- Added summary cards, revision selection, retry/configuration states,
live draft streaming, transcript chunk handling, and project/workspace
integrations.
- Updated Claude local parsing for streamed status output and expanded
server, adapter, shared, database, catalog, and UI coverage.

## Verification

- `pnpm -r typecheck`
- `pnpm exec vitest run packages/db/src/summary-slots-schema.test.ts
packages/shared/src/summary-slot.test.ts
server/src/__tests__/summary-slot-routes.test.ts
server/src/__tests__/summary-slots.test.ts
server/src/__tests__/built-in-agents.test.ts
ui/src/components/SummarySlotCard.test.tsx
ui/src/components/SummarySlotCard.status.test.tsx
ui/src/components/useSummaryDraftStream.test.tsx
ui/src/lib/summary-draft-stream.test.ts
ui/src/lib/run-log-chunks.test.ts
ui/src/context/LiveUpdatesProvider.hook.test.tsx` — 113 tests passed
- `pnpm test:run` — server and UI suites passed; one CLI AWS doctor test
was affected by inherited `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY`,
and passed when those host credentials were removed
- `pnpm exec vitest run cli/src/__tests__/secrets.test.ts` with
inherited AWS credential variables removed — 8 tests passed
- `pnpm build`
- `pnpm check:token-gates` currently reports nine `#9627` comment
references introduced by current `master`; none are in this PR diff

## Risks

- Database risk is limited by incrementally ordered, idempotent
migrations and migration safety checks.
- Summary generation creates normal issues/runs, so misconfiguration can
produce failed slots; the UI exposes retryable failure reasons and agent
configuration entry points.
- Streaming draft parsing depends on the documented `STATUS:` protocol;
final persisted revisions remain the source of truth.
- The feature is experimental, opt-in, and its built-in routine is
paused with no background token spend by default.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI GPT-5.3 Codex with reasoning, repository tool use, code
execution, GitHub CLI, and Paperclip control-plane integration. Earlier
branch commits also record Claude model co-authorship where applicable.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 11:03:07 -05:00

119 lines
4.4 KiB
TypeScript

import type { RunLogChunk } from "../adapters";
/**
* Streaming-summary output protocol parser (see the `summarize-status` skill).
*
* The Summarizer emits, as plain assistant text (never inside a tool call):
* - `STATUS: <action>…` lines, one before each procedure step; and
* - the final Markdown wrapped between the exact sentinels
* `<<<SUMMARY-DRAFT>>>` and `<<<END-SUMMARY-DRAFT>>>`, each on its own line,
* immediately before the authoritative summary-slot write.
*
* The card re-parses the full accumulated assistant text on every render, so
* markers split across streaming delta boundaries rejoin naturally once both
* halves have arrived. Markers are only recognized at the start of a line, so
* incidental occurrences inside prose or a partially-streamed marker never
* trigger a false draft.
*/
export const SUMMARY_DRAFT_START = "<<<SUMMARY-DRAFT>>>";
export const SUMMARY_DRAFT_END = "<<<END-SUMMARY-DRAFT>>>";
const STATUS_LINE_RE = /^STATUS:[ \t]?(.*)$/gm;
const DRAFT_START_RE = /^<<<SUMMARY-DRAFT>>>[ \t]*$/m;
const DRAFT_END_RE = /^<<<END-SUMMARY-DRAFT>>>[ \t]*$/m;
export interface SummaryDraftParse {
/** Latest `STATUS:` line (prefix stripped), or null if none has streamed. */
statusLine: string | null;
/** Every `STATUS:` line seen so far, in order. */
statusLines: string[];
/** Accumulated draft Markdown; may be partial while streaming. */
draft: string | null;
/** True once the closing sentinel has arrived. */
draftClosed: boolean;
}
const EMPTY_PARSE: SummaryDraftParse = {
statusLine: null,
statusLines: [],
draft: null,
draftClosed: false,
};
/**
* Extract the assistant "output"-channel prose from accumulated run-log chunks.
* Only `acpx.text_delta` records on the output channel carry the STATUS lines
* and sentinel-wrapped draft; thought-channel deltas and tool-call records are
* ignored. Chunks are assumed to already be in emit order.
*/
export function extractAssistantOutputText(chunks: RunLogChunk[]): string {
let text = "";
for (const chunk of chunks) {
const record = tryParseRecord(chunk.chunk);
if (!record) continue;
if (record.type !== "acpx.text_delta") continue;
const channel = typeof record.channel === "string" ? record.channel : "output";
if (channel === "thought" || channel === "thinking") continue;
if (typeof record.text === "string") text += record.text;
}
return text;
}
function tryParseRecord(chunk: string): Record<string, unknown> | null {
const trimmed = chunk.trim();
if (!trimmed.startsWith("{")) return null;
try {
const parsed = JSON.parse(trimmed) as unknown;
return typeof parsed === "object" && parsed !== null ? (parsed as Record<string, unknown>) : null;
} catch {
return null;
}
}
/** Parse the accumulated assistant text into status lines and the draft body. */
export function parseSummaryDraftStream(text: string): SummaryDraftParse {
if (!text) return EMPTY_PARSE;
const statusLines: string[] = [];
STATUS_LINE_RE.lastIndex = 0;
let match: RegExpExecArray | null;
while ((match = STATUS_LINE_RE.exec(text)) !== null) {
const value = (match[1] ?? "").trim();
if (value) statusLines.push(value);
}
const statusLine = statusLines.length > 0 ? statusLines[statusLines.length - 1]! : null;
const startMatch = text.match(DRAFT_START_RE);
let draft: string | null = null;
let draftClosed = false;
if (startMatch && startMatch.index !== undefined) {
const afterStart = text.slice(startMatch.index + startMatch[0].length).replace(/^\r?\n/, "");
const endMatch = afterStart.match(DRAFT_END_RE);
if (endMatch && endMatch.index !== undefined) {
draft = afterStart.slice(0, endMatch.index).replace(/\r?\n$/, "");
draftClosed = true;
} else {
draft = afterStart;
draftClosed = false;
}
}
return { statusLine, statusLines, draft, draftClosed };
}
/**
* Guard partial Markdown for live rendering: if the streamed draft ends inside
* an unclosed fenced code block, append a closing fence so `react-markdown`
* renders the partial code as a block instead of swallowing the rest of the
* document. Only needed while the draft is still streaming.
*/
export function closeDanglingCodeFence(markdown: string): string {
const fences = markdown.match(/^```/gm);
if (fences && fences.length % 2 === 1) {
const needsNewline = markdown.length > 0 && !markdown.endsWith("\n");
return `${markdown}${needsNewline ? "\n" : ""}\`\`\``;
}
return markdown;
}