Files
PaperClipAI/ui/src/components/ActivityCharts.tsx
T
b13eb5b2b5 Skill Studio: three-pane skill IDE with sandboxed test runs (#9241)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Skills Manager gives operators a reusable skill layer, but
iteration still required manual edits, ad hoc prompts, and indirect run
inspection.
> - Skill authors need a focused workflow for editing skill files,
saving representative test inputs, and running those inputs through an
agent without exposing harness tasks as normal company work.
> - The backend therefore needs durable test inputs, reusable run
templates, hidden harness issues, scoped run execution, retention
metadata, and read-containment rules around hidden work.
> - The frontend needs a three-pane Studio that keeps skill files, saved
inputs/templates, and run output/history visible together while
preserving the existing design system and token rules.
> - This pull request ships that Skill Studio surface end to end:
database migrations, shared contracts, server APIs/services, hidden
harness execution behavior, UI routes/components, and focused tests.
> - The benefit is faster and safer skill iteration, with inspectable
outputs and fewer ways for internal harness work to leak into normal
task lists, costs, or adjacent read APIs.

## Linked Issues or Issue Description

No public GitHub issue exists for this feature. Feature request summary:

- Problem: Skill authors need to edit and test company skills in one
place instead of switching between the skill detail page, task creation,
run output, and manual prompt history.
- Proposed solution: Add a Skill Studio workbench with saved inputs,
reusable templates, hidden sandboxed test runs, live run status, output
inspection, run history, rerun/delete controls, and frontmatter-aware
editing.
- Expected users: Paperclip operators and agent-company maintainers who
create, fork, import, and tune skills.
- Related public PRs: Supersedes #9205, which was replaced so the public
PR branch name follows contributor policy.
- Duplicate search: searched public GitHub issues and PRs for "Skill
Studio"; no other active public issue or PR directly covers this
feature.

## What Changed

- Added database migrations for Skill Studio test inputs, test runs,
test run retention, and reusable run templates.
- Added shared Skill Studio types, validators, route helpers,
frontmatter utilities, and status handling.
- Added server services and routes for saved inputs, test runs,
templates, reruns, terminal-run deletion, hidden harness issue
execution, and run-detail hydration.
- Strengthened hidden-issue read containment across issue-adjacent
routes and cost rollups used by skill test harness work.
- Added the Skill Studio UI with skill file editing, frontmatter
editing, saved inputs, templates, run creation/cancel/rerun/delete
flows, output rendering, history, route support, and responsive pane
behavior.
- Added focused backend, shared, and UI tests for the new APIs, routing
logic, editor/run behavior, hidden-issue containment, and migration
safety.
- Rebased onto current `master`, removed the generated lockfile diff
from the PR, and verified no workflow files are changed.

## Verification

- [x] `pnpm --filter @paperclipai/db check:migrations`
- [x] `pnpm check:token-gates`
- [x] `pnpm exec vitest run
server/src/__tests__/company-skills-service.test.ts
server/src/__tests__/company-skills-routes.test.ts
server/src/__tests__/company-skill-test-runs-service.test.ts
ui/src/lib/skill-studio.test.ts ui/src/pages/SkillStudio.test.tsx` — 5
files, 132 tests passed
- [x] Greptile review on the latest PR head
- [x] GitHub PR checks on the latest PR head

## Risks

- Medium risk because this is a broad feature touching database schema,
server orchestration, issue visibility, and a large UI surface.
- Hidden harness issue containment is security-sensitive; this PR
includes regression coverage for adjacent read paths and cost rollups.
- The new migrations are additive and use idempotent guards where
applicable, but deployed databases that previously tested draft
migration numbers should still be checked carefully.
- The UI depends on a new resizable panels package in `ui/package.json`;
the lockfile is intentionally left to repository automation.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, GPT-5 coding agent with shell, git, and GitHub CLI tool
use. Earlier feature commits include assistance from other Paperclip
coding agents; this PR preparation, rebase, cleanup commit, and PR body
were completed by OpenAI Codex in a Paperclip worktree.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 13:08:56 -05:00

328 lines
13 KiB
TypeScript

import type { DashboardRunActivityDay, HeartbeatRun } from "@paperclipai/shared";
/* ---- Utilities ---- */
export function getLast14Days(): string[] {
return Array.from({ length: 14 }, (_, i) => {
const d = new Date();
d.setDate(d.getDate() - (13 - i));
return d.toISOString().slice(0, 10);
});
}
function formatDayLabel(dateStr: string): string {
const d = new Date(dateStr + "T12:00:00");
return `${d.getMonth() + 1}/${d.getDate()}`;
}
function emptyRunDay(date: string): DashboardRunActivityDay {
return { date, succeeded: 0, failed: 0, recovered: 0, other: 0, total: 0, failedByErrorCode: {} };
}
const runSegmentColors = {
succeeded: "var(--hex-10b981)",
recovered: "var(--status-task-todo)",
failed: "var(--hex-ef4444)",
other: "var(--hex-737373)",
} as const;
// Compact per-day tooltip that also attributes failures to their error class.
function runDayTooltip(entry: DashboardRunActivityDay): string {
const lines = [`${entry.date}: ${entry.total} run${entry.total === 1 ? "" : "s"}`];
if (entry.succeeded > 0) lines.push(` succeeded: ${entry.succeeded}`);
if (entry.recovered > 0) lines.push(` recovered: ${entry.recovered} (retry succeeded)`);
if (entry.failed > 0) {
lines.push(` failed: ${entry.failed}`);
const codes = Object.entries(entry.failedByErrorCode ?? {}).sort((a, b) => b[1] - a[1]);
for (const [code, count] of codes) lines.push(` ${code}: ${count}`);
}
if (entry.other > 0) lines.push(` other: ${entry.other}`);
return lines.join("\n");
}
/* ---- Sub-components ---- */
function DateLabels({ days }: { days: string[] }) {
return (
<div className="flex gap-(--sz-3px) mt-1.5">
{days.map((day, i) => (
<div key={day} className="flex-1 text-center">
{(i === 0 || i === 6 || i === 13) ? (
<span className="text-(length:--text-nano) text-muted-foreground tabular-nums">{formatDayLabel(day)}</span>
) : null}
</div>
))}
</div>
);
}
function ChartLegend({ items }: { items: { color: string; label: string }[] }) {
return (
<div className="flex flex-wrap gap-x-2.5 gap-y-0.5 mt-2">
{items.map(item => (
<span key={item.label} className="flex items-center gap-1 text-(length:--text-nano) text-muted-foreground">
<span className="h-1.5 w-1.5 rounded-full shrink-0" style={{ backgroundColor: item.color }} />
{item.label}
</span>
))}
</div>
);
}
export function ChartCard({ title, subtitle, children }: { title: string; subtitle?: string; children: React.ReactNode }) {
return (
<div className="border border-border rounded-lg p-4 space-y-3">
<div>
<h3 className="text-xs font-medium text-muted-foreground">{title}</h3>
{subtitle && <span className="text-(length:--text-nano) text-muted-foreground/60">{subtitle}</span>}
</div>
{children}
</div>
);
}
/* ---- Chart Components ---- */
type RunChartProps =
| { activity?: DashboardRunActivityDay[] | null; runs?: never }
| { runs?: HeartbeatRun[] | null; activity?: never };
function aggregateRuns(runs: readonly HeartbeatRun[] = []): DashboardRunActivityDay[] {
const days = getLast14Days();
const grouped = new Map<string, DashboardRunActivityDay>();
for (const day of days) grouped.set(day, emptyRunDay(day));
for (const run of runs) {
const day = new Date(run.createdAt).toISOString().slice(0, 10);
const entry = grouped.get(day);
if (!entry) continue;
if (run.status === "succeeded") {
entry.succeeded++;
} else if (run.status === "failed" || run.status === "timed_out") {
// A flat run list has no retry-chain linkage, so recovery can't be derived
// here (the company dashboard computes it server-side). Attribute the
// failure to its error class so the breakdown still renders.
entry.failed++;
const code = run.errorCode && run.errorCode.length > 0 ? run.errorCode : "unknown";
entry.failedByErrorCode[code] = (entry.failedByErrorCode[code] ?? 0) + 1;
} else {
entry.other++;
}
entry.total++;
}
return Array.from(grouped.values());
}
function resolveRunActivity(props: RunChartProps): DashboardRunActivityDay[] {
if (Array.isArray(props.activity)) return props.activity;
if (Array.isArray(props.runs)) return aggregateRuns(props.runs);
return [];
}
export function RunActivityChart(props: RunChartProps) {
const activity = resolveRunActivity(props);
const days = activity.length > 0 ? activity.map((day) => day.date) : getLast14Days();
const grouped = new Map(activity.map((day) => [day.date, day]));
const maxValue = Math.max(...activity.map(v => v.total), 1);
const hasData = activity.some(v => v.total > 0);
const hasRecovered = activity.some(v => v.recovered > 0);
if (!hasData) return <p className="text-xs text-muted-foreground">No runs yet</p>;
const legendItems = [
{ color: runSegmentColors.succeeded, label: "Succeeded" },
...(hasRecovered ? [{ color: runSegmentColors.recovered, label: "Recovered" }] : []),
{ color: runSegmentColors.failed, label: "Failed" },
{ color: runSegmentColors.other, label: "Other" },
];
return (
<div>
<div className="flex items-end gap-(--sz-3px) h-20">
{days.map(day => {
const entry = grouped.get(day) ?? emptyRunDay(day);
const total = entry.total;
const heightPct = (total / maxValue) * 100;
return (
<div key={day} className="flex-1 h-full flex flex-col justify-end" title={runDayTooltip(entry)}>
{total > 0 ? (
<div className="flex flex-col-reverse gap-px overflow-hidden" style={{ height: `${heightPct}%`, minHeight: 2 }}>
{entry.succeeded > 0 && <div style={{ flex: entry.succeeded, backgroundColor: runSegmentColors.succeeded }} />}
{entry.recovered > 0 && <div style={{ flex: entry.recovered, backgroundColor: runSegmentColors.recovered }} />}
{entry.failed > 0 && <div style={{ flex: entry.failed, backgroundColor: runSegmentColors.failed }} />}
{entry.other > 0 && <div style={{ flex: entry.other, backgroundColor: runSegmentColors.other }} />}
</div>
) : (
<div className="bg-muted/30 rounded-sm" style={{ height: 2 }} />
)}
</div>
);
})}
</div>
<DateLabels days={days} />
<ChartLegend items={legendItems} />
</div>
);
}
const priorityColors: Record<string, string> = {
critical: "var(--hex-ef4444)",
high: "var(--hex-f97316)",
medium: "var(--hex-eab308)",
low: "var(--hex-6b7280)",
};
const priorityOrder = ["critical", "high", "medium", "low"] as const;
export function PriorityChart({ issues }: { issues: { priority: string; createdAt: Date }[] }) {
const days = getLast14Days();
const grouped = new Map<string, Record<string, number>>();
for (const day of days) grouped.set(day, { critical: 0, high: 0, medium: 0, low: 0 });
for (const issue of issues) {
const day = new Date(issue.createdAt).toISOString().slice(0, 10);
const entry = grouped.get(day);
if (!entry) continue;
if (issue.priority in entry) entry[issue.priority]++;
}
const maxValue = Math.max(...Array.from(grouped.values()).map(v => Object.values(v).reduce((a, b) => a + b, 0)), 1);
const hasData = Array.from(grouped.values()).some(v => Object.values(v).reduce((a, b) => a + b, 0) > 0);
if (!hasData) return <p className="text-xs text-muted-foreground">No tasks</p>;
return (
<div>
<div className="flex items-end gap-(--sz-3px) h-20">
{days.map(day => {
const entry = grouped.get(day)!;
const total = Object.values(entry).reduce((a, b) => a + b, 0);
const heightPct = (total / maxValue) * 100;
return (
<div key={day} className="flex-1 h-full flex flex-col justify-end" title={`${day}: ${total} issues`}>
{total > 0 ? (
<div className="flex flex-col-reverse gap-px overflow-hidden" style={{ height: `${heightPct}%`, minHeight: 2 }}>
{priorityOrder.map(p => entry[p] > 0 ? (
<div key={p} style={{ flex: entry[p], backgroundColor: priorityColors[p] }} />
) : null)}
</div>
) : (
<div className="bg-muted/30 rounded-sm" style={{ height: 2 }} />
)}
</div>
);
})}
</div>
<DateLabels days={days} />
<ChartLegend items={priorityOrder.map(p => ({ color: priorityColors[p], label: p.charAt(0).toUpperCase() + p.slice(1) }))} />
</div>
);
}
// DECISION-SHEET.md B5: chart status colors re-pointed at the canonical
// --status-task-* system (DESIGN.md principle 5 — an operator learns one
// status vocabulary; badge, row, chart, and log agree). Previously an
// independent palette (todo blue, in_progress violet, etc.). `backlog`
// deliberately keeps --project-none (pre-B5, per user ruling); the
// priority series and success-rate tints below are not status hues and
// are left alone.
const statusColors: Record<string, string> = {
todo: "var(--status-task-todo)",
in_progress: "var(--status-task-in_progress)",
in_review: "var(--status-task-in_review)",
done: "var(--status-task-done)",
blocked: "var(--status-task-blocked)",
cancelled: "var(--status-task-cancelled)",
backlog: "var(--project-none)",
};
const statusLabels: Record<string, string> = {
todo: "To Do",
in_progress: "In Progress",
in_review: "In Review",
done: "Done",
blocked: "Blocked",
cancelled: "Cancelled",
backlog: "Backlog",
};
export function IssueStatusChart({ issues }: { issues: { status: string; createdAt: Date }[] }) {
const days = getLast14Days();
const allStatuses = new Set<string>();
const grouped = new Map<string, Record<string, number>>();
for (const day of days) grouped.set(day, {});
for (const issue of issues) {
const day = new Date(issue.createdAt).toISOString().slice(0, 10);
const entry = grouped.get(day);
if (!entry) continue;
entry[issue.status] = (entry[issue.status] ?? 0) + 1;
allStatuses.add(issue.status);
}
const statusOrder = ["todo", "in_progress", "in_review", "done", "blocked", "cancelled", "backlog"].filter(s => allStatuses.has(s));
const maxValue = Math.max(...Array.from(grouped.values()).map(v => Object.values(v).reduce((a, b) => a + b, 0)), 1);
const hasData = allStatuses.size > 0;
if (!hasData) return <p className="text-xs text-muted-foreground">No tasks</p>;
return (
<div>
<div className="flex items-end gap-(--sz-3px) h-20">
{days.map(day => {
const entry = grouped.get(day)!;
const total = Object.values(entry).reduce((a, b) => a + b, 0);
const heightPct = (total / maxValue) * 100;
return (
<div key={day} className="flex-1 h-full flex flex-col justify-end" title={`${day}: ${total} issues`}>
{total > 0 ? (
<div className="flex flex-col-reverse gap-px overflow-hidden" style={{ height: `${heightPct}%`, minHeight: 2 }}>
{statusOrder.map(s => (entry[s] ?? 0) > 0 ? (
<div key={s} style={{ flex: entry[s], backgroundColor: statusColors[s] ?? "var(--hex-6b7280)" }} />
) : null)}
</div>
) : (
<div className="bg-muted/30 rounded-sm" style={{ height: 2 }} />
)}
</div>
);
})}
</div>
<DateLabels days={days} />
<ChartLegend items={statusOrder.map(s => ({ color: statusColors[s] ?? "var(--hex-6b7280)", label: statusLabels[s] ?? s }))} />
</div>
);
}
export function SuccessRateChart(props: RunChartProps) {
const activity = resolveRunActivity(props);
const days = activity.length > 0 ? activity.map((day) => day.date) : getLast14Days();
const grouped = new Map(activity.map((day) => [day.date, day]));
const hasData = activity.some(v => v.total > 0);
if (!hasData) return <p className="text-xs text-muted-foreground">No runs yet</p>;
return (
<div>
<div className="flex items-end gap-(--sz-3px) h-20">
{days.map(day => {
const entry = grouped.get(day) ?? emptyRunDay(day);
// Recovered runs ultimately succeeded, so they count toward the rate
// rather than dragging it down as failures.
const effectiveSucceeded = entry.succeeded + entry.recovered;
const rate = entry.total > 0 ? effectiveSucceeded / entry.total : 0;
const color = entry.total === 0 ? undefined : rate >= 0.8 ? "var(--hex-10b981)" : rate >= 0.5 ? "var(--hex-eab308)" : "var(--hex-ef4444)";
return (
<div key={day} className="flex-1 h-full flex flex-col justify-end" title={`${day}: ${entry.total > 0 ? Math.round(rate * 100) : 0}% (${effectiveSucceeded}/${entry.total})`}>
{entry.total > 0 ? (
<div style={{ height: `${rate * 100}%`, minHeight: 2, backgroundColor: color }} />
) : (
<div className="bg-muted/30 rounded-sm" style={{ height: 2 }} />
)}
</div>
);
})}
</div>
<DateLabels days={days} />
</div>
);
}