mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-27 11:20:42 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Skills Manager gives operators a reusable skill layer, but iteration still required manual edits, ad hoc prompts, and indirect run inspection. > - Skill authors need a focused workflow for editing skill files, saving representative test inputs, and running those inputs through an agent without exposing harness tasks as normal company work. > - The backend therefore needs durable test inputs, reusable run templates, hidden harness issues, scoped run execution, retention metadata, and read-containment rules around hidden work. > - The frontend needs a three-pane Studio that keeps skill files, saved inputs/templates, and run output/history visible together while preserving the existing design system and token rules. > - This pull request ships that Skill Studio surface end to end: database migrations, shared contracts, server APIs/services, hidden harness execution behavior, UI routes/components, and focused tests. > - The benefit is faster and safer skill iteration, with inspectable outputs and fewer ways for internal harness work to leak into normal task lists, costs, or adjacent read APIs. ## Linked Issues or Issue Description No public GitHub issue exists for this feature. Feature request summary: - Problem: Skill authors need to edit and test company skills in one place instead of switching between the skill detail page, task creation, run output, and manual prompt history. - Proposed solution: Add a Skill Studio workbench with saved inputs, reusable templates, hidden sandboxed test runs, live run status, output inspection, run history, rerun/delete controls, and frontmatter-aware editing. - Expected users: Paperclip operators and agent-company maintainers who create, fork, import, and tune skills. - Related public PRs: Supersedes #9205, which was replaced so the public PR branch name follows contributor policy. - Duplicate search: searched public GitHub issues and PRs for "Skill Studio"; no other active public issue or PR directly covers this feature. ## What Changed - Added database migrations for Skill Studio test inputs, test runs, test run retention, and reusable run templates. - Added shared Skill Studio types, validators, route helpers, frontmatter utilities, and status handling. - Added server services and routes for saved inputs, test runs, templates, reruns, terminal-run deletion, hidden harness issue execution, and run-detail hydration. - Strengthened hidden-issue read containment across issue-adjacent routes and cost rollups used by skill test harness work. - Added the Skill Studio UI with skill file editing, frontmatter editing, saved inputs, templates, run creation/cancel/rerun/delete flows, output rendering, history, route support, and responsive pane behavior. - Added focused backend, shared, and UI tests for the new APIs, routing logic, editor/run behavior, hidden-issue containment, and migration safety. - Rebased onto current `master`, removed the generated lockfile diff from the PR, and verified no workflow files are changed. ## Verification - [x] `pnpm --filter @paperclipai/db check:migrations` - [x] `pnpm check:token-gates` - [x] `pnpm exec vitest run server/src/__tests__/company-skills-service.test.ts server/src/__tests__/company-skills-routes.test.ts server/src/__tests__/company-skill-test-runs-service.test.ts ui/src/lib/skill-studio.test.ts ui/src/pages/SkillStudio.test.tsx` — 5 files, 132 tests passed - [x] Greptile review on the latest PR head - [x] GitHub PR checks on the latest PR head ## Risks - Medium risk because this is a broad feature touching database schema, server orchestration, issue visibility, and a large UI surface. - Hidden harness issue containment is security-sensitive; this PR includes regression coverage for adjacent read paths and cost rollups. - The new migrations are additive and use idempotent guards where applicable, but deployed databases that previously tested draft migration numbers should still be checked carefully. - The UI depends on a new resizable panels package in `ui/package.json`; the lockfile is intentionally left to repository automation. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-5 coding agent with shell, git, and GitHub CLI tool use. Earlier feature commits include assistance from other Paperclip coding agents; this PR preparation, rebase, cleanup commit, and PR body were completed by OpenAI Codex in a Paperclip worktree. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
328 lines
13 KiB
TypeScript
328 lines
13 KiB
TypeScript
import type { DashboardRunActivityDay, HeartbeatRun } from "@paperclipai/shared";
|
|
|
|
/* ---- Utilities ---- */
|
|
|
|
export function getLast14Days(): string[] {
|
|
return Array.from({ length: 14 }, (_, i) => {
|
|
const d = new Date();
|
|
d.setDate(d.getDate() - (13 - i));
|
|
return d.toISOString().slice(0, 10);
|
|
});
|
|
}
|
|
|
|
function formatDayLabel(dateStr: string): string {
|
|
const d = new Date(dateStr + "T12:00:00");
|
|
return `${d.getMonth() + 1}/${d.getDate()}`;
|
|
}
|
|
|
|
function emptyRunDay(date: string): DashboardRunActivityDay {
|
|
return { date, succeeded: 0, failed: 0, recovered: 0, other: 0, total: 0, failedByErrorCode: {} };
|
|
}
|
|
|
|
const runSegmentColors = {
|
|
succeeded: "var(--hex-10b981)",
|
|
recovered: "var(--status-task-todo)",
|
|
failed: "var(--hex-ef4444)",
|
|
other: "var(--hex-737373)",
|
|
} as const;
|
|
|
|
// Compact per-day tooltip that also attributes failures to their error class.
|
|
function runDayTooltip(entry: DashboardRunActivityDay): string {
|
|
const lines = [`${entry.date}: ${entry.total} run${entry.total === 1 ? "" : "s"}`];
|
|
if (entry.succeeded > 0) lines.push(` succeeded: ${entry.succeeded}`);
|
|
if (entry.recovered > 0) lines.push(` recovered: ${entry.recovered} (retry succeeded)`);
|
|
if (entry.failed > 0) {
|
|
lines.push(` failed: ${entry.failed}`);
|
|
const codes = Object.entries(entry.failedByErrorCode ?? {}).sort((a, b) => b[1] - a[1]);
|
|
for (const [code, count] of codes) lines.push(` ${code}: ${count}`);
|
|
}
|
|
if (entry.other > 0) lines.push(` other: ${entry.other}`);
|
|
return lines.join("\n");
|
|
}
|
|
|
|
/* ---- Sub-components ---- */
|
|
|
|
function DateLabels({ days }: { days: string[] }) {
|
|
return (
|
|
<div className="flex gap-(--sz-3px) mt-1.5">
|
|
{days.map((day, i) => (
|
|
<div key={day} className="flex-1 text-center">
|
|
{(i === 0 || i === 6 || i === 13) ? (
|
|
<span className="text-(length:--text-nano) text-muted-foreground tabular-nums">{formatDayLabel(day)}</span>
|
|
) : null}
|
|
</div>
|
|
))}
|
|
</div>
|
|
);
|
|
}
|
|
|
|
function ChartLegend({ items }: { items: { color: string; label: string }[] }) {
|
|
return (
|
|
<div className="flex flex-wrap gap-x-2.5 gap-y-0.5 mt-2">
|
|
{items.map(item => (
|
|
<span key={item.label} className="flex items-center gap-1 text-(length:--text-nano) text-muted-foreground">
|
|
<span className="h-1.5 w-1.5 rounded-full shrink-0" style={{ backgroundColor: item.color }} />
|
|
{item.label}
|
|
</span>
|
|
))}
|
|
</div>
|
|
);
|
|
}
|
|
|
|
export function ChartCard({ title, subtitle, children }: { title: string; subtitle?: string; children: React.ReactNode }) {
|
|
return (
|
|
<div className="border border-border rounded-lg p-4 space-y-3">
|
|
<div>
|
|
<h3 className="text-xs font-medium text-muted-foreground">{title}</h3>
|
|
{subtitle && <span className="text-(length:--text-nano) text-muted-foreground/60">{subtitle}</span>}
|
|
</div>
|
|
{children}
|
|
</div>
|
|
);
|
|
}
|
|
|
|
/* ---- Chart Components ---- */
|
|
|
|
type RunChartProps =
|
|
| { activity?: DashboardRunActivityDay[] | null; runs?: never }
|
|
| { runs?: HeartbeatRun[] | null; activity?: never };
|
|
|
|
function aggregateRuns(runs: readonly HeartbeatRun[] = []): DashboardRunActivityDay[] {
|
|
const days = getLast14Days();
|
|
const grouped = new Map<string, DashboardRunActivityDay>();
|
|
for (const day of days) grouped.set(day, emptyRunDay(day));
|
|
for (const run of runs) {
|
|
const day = new Date(run.createdAt).toISOString().slice(0, 10);
|
|
const entry = grouped.get(day);
|
|
if (!entry) continue;
|
|
if (run.status === "succeeded") {
|
|
entry.succeeded++;
|
|
} else if (run.status === "failed" || run.status === "timed_out") {
|
|
// A flat run list has no retry-chain linkage, so recovery can't be derived
|
|
// here (the company dashboard computes it server-side). Attribute the
|
|
// failure to its error class so the breakdown still renders.
|
|
entry.failed++;
|
|
const code = run.errorCode && run.errorCode.length > 0 ? run.errorCode : "unknown";
|
|
entry.failedByErrorCode[code] = (entry.failedByErrorCode[code] ?? 0) + 1;
|
|
} else {
|
|
entry.other++;
|
|
}
|
|
entry.total++;
|
|
}
|
|
return Array.from(grouped.values());
|
|
}
|
|
|
|
function resolveRunActivity(props: RunChartProps): DashboardRunActivityDay[] {
|
|
if (Array.isArray(props.activity)) return props.activity;
|
|
if (Array.isArray(props.runs)) return aggregateRuns(props.runs);
|
|
return [];
|
|
}
|
|
|
|
export function RunActivityChart(props: RunChartProps) {
|
|
const activity = resolveRunActivity(props);
|
|
const days = activity.length > 0 ? activity.map((day) => day.date) : getLast14Days();
|
|
const grouped = new Map(activity.map((day) => [day.date, day]));
|
|
|
|
const maxValue = Math.max(...activity.map(v => v.total), 1);
|
|
const hasData = activity.some(v => v.total > 0);
|
|
const hasRecovered = activity.some(v => v.recovered > 0);
|
|
|
|
if (!hasData) return <p className="text-xs text-muted-foreground">No runs yet</p>;
|
|
|
|
const legendItems = [
|
|
{ color: runSegmentColors.succeeded, label: "Succeeded" },
|
|
...(hasRecovered ? [{ color: runSegmentColors.recovered, label: "Recovered" }] : []),
|
|
{ color: runSegmentColors.failed, label: "Failed" },
|
|
{ color: runSegmentColors.other, label: "Other" },
|
|
];
|
|
|
|
return (
|
|
<div>
|
|
<div className="flex items-end gap-(--sz-3px) h-20">
|
|
{days.map(day => {
|
|
const entry = grouped.get(day) ?? emptyRunDay(day);
|
|
const total = entry.total;
|
|
const heightPct = (total / maxValue) * 100;
|
|
return (
|
|
<div key={day} className="flex-1 h-full flex flex-col justify-end" title={runDayTooltip(entry)}>
|
|
{total > 0 ? (
|
|
<div className="flex flex-col-reverse gap-px overflow-hidden" style={{ height: `${heightPct}%`, minHeight: 2 }}>
|
|
{entry.succeeded > 0 && <div style={{ flex: entry.succeeded, backgroundColor: runSegmentColors.succeeded }} />}
|
|
{entry.recovered > 0 && <div style={{ flex: entry.recovered, backgroundColor: runSegmentColors.recovered }} />}
|
|
{entry.failed > 0 && <div style={{ flex: entry.failed, backgroundColor: runSegmentColors.failed }} />}
|
|
{entry.other > 0 && <div style={{ flex: entry.other, backgroundColor: runSegmentColors.other }} />}
|
|
</div>
|
|
) : (
|
|
<div className="bg-muted/30 rounded-sm" style={{ height: 2 }} />
|
|
)}
|
|
</div>
|
|
);
|
|
})}
|
|
</div>
|
|
<DateLabels days={days} />
|
|
<ChartLegend items={legendItems} />
|
|
</div>
|
|
);
|
|
}
|
|
|
|
const priorityColors: Record<string, string> = {
|
|
critical: "var(--hex-ef4444)",
|
|
high: "var(--hex-f97316)",
|
|
medium: "var(--hex-eab308)",
|
|
low: "var(--hex-6b7280)",
|
|
};
|
|
|
|
const priorityOrder = ["critical", "high", "medium", "low"] as const;
|
|
|
|
export function PriorityChart({ issues }: { issues: { priority: string; createdAt: Date }[] }) {
|
|
const days = getLast14Days();
|
|
const grouped = new Map<string, Record<string, number>>();
|
|
for (const day of days) grouped.set(day, { critical: 0, high: 0, medium: 0, low: 0 });
|
|
for (const issue of issues) {
|
|
const day = new Date(issue.createdAt).toISOString().slice(0, 10);
|
|
const entry = grouped.get(day);
|
|
if (!entry) continue;
|
|
if (issue.priority in entry) entry[issue.priority]++;
|
|
}
|
|
|
|
const maxValue = Math.max(...Array.from(grouped.values()).map(v => Object.values(v).reduce((a, b) => a + b, 0)), 1);
|
|
const hasData = Array.from(grouped.values()).some(v => Object.values(v).reduce((a, b) => a + b, 0) > 0);
|
|
|
|
if (!hasData) return <p className="text-xs text-muted-foreground">No tasks</p>;
|
|
|
|
return (
|
|
<div>
|
|
<div className="flex items-end gap-(--sz-3px) h-20">
|
|
{days.map(day => {
|
|
const entry = grouped.get(day)!;
|
|
const total = Object.values(entry).reduce((a, b) => a + b, 0);
|
|
const heightPct = (total / maxValue) * 100;
|
|
return (
|
|
<div key={day} className="flex-1 h-full flex flex-col justify-end" title={`${day}: ${total} issues`}>
|
|
{total > 0 ? (
|
|
<div className="flex flex-col-reverse gap-px overflow-hidden" style={{ height: `${heightPct}%`, minHeight: 2 }}>
|
|
{priorityOrder.map(p => entry[p] > 0 ? (
|
|
<div key={p} style={{ flex: entry[p], backgroundColor: priorityColors[p] }} />
|
|
) : null)}
|
|
</div>
|
|
) : (
|
|
<div className="bg-muted/30 rounded-sm" style={{ height: 2 }} />
|
|
)}
|
|
</div>
|
|
);
|
|
})}
|
|
</div>
|
|
<DateLabels days={days} />
|
|
<ChartLegend items={priorityOrder.map(p => ({ color: priorityColors[p], label: p.charAt(0).toUpperCase() + p.slice(1) }))} />
|
|
</div>
|
|
);
|
|
}
|
|
|
|
// DECISION-SHEET.md B5: chart status colors re-pointed at the canonical
|
|
// --status-task-* system (DESIGN.md principle 5 — an operator learns one
|
|
// status vocabulary; badge, row, chart, and log agree). Previously an
|
|
// independent palette (todo blue, in_progress violet, etc.). `backlog`
|
|
// deliberately keeps --project-none (pre-B5, per user ruling); the
|
|
// priority series and success-rate tints below are not status hues and
|
|
// are left alone.
|
|
const statusColors: Record<string, string> = {
|
|
todo: "var(--status-task-todo)",
|
|
in_progress: "var(--status-task-in_progress)",
|
|
in_review: "var(--status-task-in_review)",
|
|
done: "var(--status-task-done)",
|
|
blocked: "var(--status-task-blocked)",
|
|
cancelled: "var(--status-task-cancelled)",
|
|
backlog: "var(--project-none)",
|
|
};
|
|
|
|
const statusLabels: Record<string, string> = {
|
|
todo: "To Do",
|
|
in_progress: "In Progress",
|
|
in_review: "In Review",
|
|
done: "Done",
|
|
blocked: "Blocked",
|
|
cancelled: "Cancelled",
|
|
backlog: "Backlog",
|
|
};
|
|
|
|
export function IssueStatusChart({ issues }: { issues: { status: string; createdAt: Date }[] }) {
|
|
const days = getLast14Days();
|
|
const allStatuses = new Set<string>();
|
|
const grouped = new Map<string, Record<string, number>>();
|
|
for (const day of days) grouped.set(day, {});
|
|
for (const issue of issues) {
|
|
const day = new Date(issue.createdAt).toISOString().slice(0, 10);
|
|
const entry = grouped.get(day);
|
|
if (!entry) continue;
|
|
entry[issue.status] = (entry[issue.status] ?? 0) + 1;
|
|
allStatuses.add(issue.status);
|
|
}
|
|
|
|
const statusOrder = ["todo", "in_progress", "in_review", "done", "blocked", "cancelled", "backlog"].filter(s => allStatuses.has(s));
|
|
const maxValue = Math.max(...Array.from(grouped.values()).map(v => Object.values(v).reduce((a, b) => a + b, 0)), 1);
|
|
const hasData = allStatuses.size > 0;
|
|
|
|
if (!hasData) return <p className="text-xs text-muted-foreground">No tasks</p>;
|
|
|
|
return (
|
|
<div>
|
|
<div className="flex items-end gap-(--sz-3px) h-20">
|
|
{days.map(day => {
|
|
const entry = grouped.get(day)!;
|
|
const total = Object.values(entry).reduce((a, b) => a + b, 0);
|
|
const heightPct = (total / maxValue) * 100;
|
|
return (
|
|
<div key={day} className="flex-1 h-full flex flex-col justify-end" title={`${day}: ${total} issues`}>
|
|
{total > 0 ? (
|
|
<div className="flex flex-col-reverse gap-px overflow-hidden" style={{ height: `${heightPct}%`, minHeight: 2 }}>
|
|
{statusOrder.map(s => (entry[s] ?? 0) > 0 ? (
|
|
<div key={s} style={{ flex: entry[s], backgroundColor: statusColors[s] ?? "var(--hex-6b7280)" }} />
|
|
) : null)}
|
|
</div>
|
|
) : (
|
|
<div className="bg-muted/30 rounded-sm" style={{ height: 2 }} />
|
|
)}
|
|
</div>
|
|
);
|
|
})}
|
|
</div>
|
|
<DateLabels days={days} />
|
|
<ChartLegend items={statusOrder.map(s => ({ color: statusColors[s] ?? "var(--hex-6b7280)", label: statusLabels[s] ?? s }))} />
|
|
</div>
|
|
);
|
|
}
|
|
|
|
export function SuccessRateChart(props: RunChartProps) {
|
|
const activity = resolveRunActivity(props);
|
|
const days = activity.length > 0 ? activity.map((day) => day.date) : getLast14Days();
|
|
const grouped = new Map(activity.map((day) => [day.date, day]));
|
|
|
|
const hasData = activity.some(v => v.total > 0);
|
|
if (!hasData) return <p className="text-xs text-muted-foreground">No runs yet</p>;
|
|
|
|
return (
|
|
<div>
|
|
<div className="flex items-end gap-(--sz-3px) h-20">
|
|
{days.map(day => {
|
|
const entry = grouped.get(day) ?? emptyRunDay(day);
|
|
// Recovered runs ultimately succeeded, so they count toward the rate
|
|
// rather than dragging it down as failures.
|
|
const effectiveSucceeded = entry.succeeded + entry.recovered;
|
|
const rate = entry.total > 0 ? effectiveSucceeded / entry.total : 0;
|
|
const color = entry.total === 0 ? undefined : rate >= 0.8 ? "var(--hex-10b981)" : rate >= 0.5 ? "var(--hex-eab308)" : "var(--hex-ef4444)";
|
|
return (
|
|
<div key={day} className="flex-1 h-full flex flex-col justify-end" title={`${day}: ${entry.total > 0 ? Math.round(rate * 100) : 0}% (${effectiveSucceeded}/${entry.total})`}>
|
|
{entry.total > 0 ? (
|
|
<div style={{ height: `${rate * 100}%`, minHeight: 2, backgroundColor: color }} />
|
|
) : (
|
|
<div className="bg-muted/30 rounded-sm" style={{ height: 2 }} />
|
|
)}
|
|
</div>
|
|
);
|
|
})}
|
|
</div>
|
|
<DateLabels days={days} />
|
|
</div>
|
|
);
|
|
}
|