Files
PaperClipAI/server/src/routes/user-profiles.ts
T
b13eb5b2b5 Skill Studio: three-pane skill IDE with sandboxed test runs (#9241)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Skills Manager gives operators a reusable skill layer, but
iteration still required manual edits, ad hoc prompts, and indirect run
inspection.
> - Skill authors need a focused workflow for editing skill files,
saving representative test inputs, and running those inputs through an
agent without exposing harness tasks as normal company work.
> - The backend therefore needs durable test inputs, reusable run
templates, hidden harness issues, scoped run execution, retention
metadata, and read-containment rules around hidden work.
> - The frontend needs a three-pane Studio that keeps skill files, saved
inputs/templates, and run output/history visible together while
preserving the existing design system and token rules.
> - This pull request ships that Skill Studio surface end to end:
database migrations, shared contracts, server APIs/services, hidden
harness execution behavior, UI routes/components, and focused tests.
> - The benefit is faster and safer skill iteration, with inspectable
outputs and fewer ways for internal harness work to leak into normal
task lists, costs, or adjacent read APIs.

## Linked Issues or Issue Description

No public GitHub issue exists for this feature. Feature request summary:

- Problem: Skill authors need to edit and test company skills in one
place instead of switching between the skill detail page, task creation,
run output, and manual prompt history.
- Proposed solution: Add a Skill Studio workbench with saved inputs,
reusable templates, hidden sandboxed test runs, live run status, output
inspection, run history, rerun/delete controls, and frontmatter-aware
editing.
- Expected users: Paperclip operators and agent-company maintainers who
create, fork, import, and tune skills.
- Related public PRs: Supersedes #9205, which was replaced so the public
PR branch name follows contributor policy.
- Duplicate search: searched public GitHub issues and PRs for "Skill
Studio"; no other active public issue or PR directly covers this
feature.

## What Changed

- Added database migrations for Skill Studio test inputs, test runs,
test run retention, and reusable run templates.
- Added shared Skill Studio types, validators, route helpers,
frontmatter utilities, and status handling.
- Added server services and routes for saved inputs, test runs,
templates, reruns, terminal-run deletion, hidden harness issue
execution, and run-detail hydration.
- Strengthened hidden-issue read containment across issue-adjacent
routes and cost rollups used by skill test harness work.
- Added the Skill Studio UI with skill file editing, frontmatter
editing, saved inputs, templates, run creation/cancel/rerun/delete
flows, output rendering, history, route support, and responsive pane
behavior.
- Added focused backend, shared, and UI tests for the new APIs, routing
logic, editor/run behavior, hidden-issue containment, and migration
safety.
- Rebased onto current `master`, removed the generated lockfile diff
from the PR, and verified no workflow files are changed.

## Verification

- [x] `pnpm --filter @paperclipai/db check:migrations`
- [x] `pnpm check:token-gates`
- [x] `pnpm exec vitest run
server/src/__tests__/company-skills-service.test.ts
server/src/__tests__/company-skills-routes.test.ts
server/src/__tests__/company-skill-test-runs-service.test.ts
ui/src/lib/skill-studio.test.ts ui/src/pages/SkillStudio.test.tsx` — 5
files, 132 tests passed
- [x] Greptile review on the latest PR head
- [x] GitHub PR checks on the latest PR head

## Risks

- Medium risk because this is a broad feature touching database schema,
server orchestration, issue visibility, and a large UI surface.
- Hidden harness issue containment is security-sensitive; this PR
includes regression coverage for adjacent read paths and cost rollups.
- The new migrations are additive and use idempotent guards where
applicable, but deployed databases that previously tested draft
migration numbers should still be checked carefully.
- The UI depends on a new resizable panels package in `ui/package.json`;
the lockfile is intentionally left to repository automation.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, GPT-5 coding agent with shell, git, and GitHub CLI tool
use. Earlier feature commits include assistance from other Paperclip
coding agents; this PR preparation, rebase, cleanup commit, and PR body
were completed by OpenAI Codex in a Paperclip worktree.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 13:08:56 -05:00

438 lines
15 KiB
TypeScript

import { Router } from "express";
import { and, desc, eq, gte, isNull, sql } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import {
activityLog,
agents,
authUsers,
companyMemberships,
costEvents,
issueComments,
issues,
} from "@paperclipai/db";
import type {
UserProfileDailyPoint,
UserProfileIdentity,
UserProfileResponse,
UserProfileWindowStats,
} from "@paperclipai/shared";
import { notFound } from "../errors.js";
import { visibleIssueCondition } from "../services/issue-visibility.js";
import { assertCompanyAccess } from "./authz.js";
type CompanyUserRow = {
id: string;
principalId: string;
status: string;
membershipRole: string | null;
createdAt: Date;
userId: string | null;
name: string | null;
email: string | null;
image: string | null;
};
const PROFILE_WINDOWS = [
{ key: "last7", label: "Last 7 days", days: 7 },
{ key: "last30", label: "Last 30 days", days: 30 },
{ key: "all", label: "All time", days: null },
] as const;
function slugifyUserPart(value: string | null | undefined) {
const normalized = value
?.trim()
.toLowerCase()
.replace(/['"]/g, "")
.replace(/[^a-z0-9]+/g, "-")
.replace(/^-+|-+$/g, "");
return normalized || null;
}
function userSlugCandidates(row: CompanyUserRow) {
const candidates = new Set<string>();
const add = (value: string | null | undefined) => {
const slug = slugifyUserPart(value);
if (slug) candidates.add(slug);
};
add(row.name);
add(row.email?.split("@")[0]);
add(row.email);
add(row.principalId);
return [...candidates];
}
async function resolveCompanyUser(db: Db, companyId: string, rawSlug: string): Promise<CompanyUserRow | null> {
const slug = slugifyUserPart(rawSlug);
if (!slug) return null;
const rows = await db
.select({
id: companyMemberships.id,
principalId: companyMemberships.principalId,
status: companyMemberships.status,
membershipRole: companyMemberships.membershipRole,
createdAt: companyMemberships.createdAt,
userId: authUsers.id,
name: authUsers.name,
email: authUsers.email,
image: authUsers.image,
})
.from(companyMemberships)
.leftJoin(authUsers, eq(authUsers.id, companyMemberships.principalId))
.where(
and(
eq(companyMemberships.companyId, companyId),
eq(companyMemberships.principalType, "user"),
),
)
.orderBy(desc(companyMemberships.updatedAt))
.limit(200);
return rows.find((row) => userSlugCandidates(row).includes(slug)) ?? null;
}
function userIssueInvolvementSql(companyId: string, userId: string) {
return sql<boolean>`
(
${issues.createdByUserId} = ${userId}
OR ${issues.assigneeUserId} = ${userId}
OR EXISTS (
SELECT 1
FROM ${issueComments}
WHERE ${issueComments.companyId} = ${companyId}
AND ${issueComments.issueId} = ${issues.id}
AND ${issueComments.authorUserId} = ${userId}
)
)
`;
}
function windowStart(days: number | null) {
if (!days) return null;
return new Date(Date.now() - days * 24 * 60 * 60 * 1000);
}
function startOfUtcDay(date: Date) {
return new Date(Date.UTC(date.getUTCFullYear(), date.getUTCMonth(), date.getUTCDate()));
}
function isoDay(date: Date) {
return startOfUtcDay(date).toISOString().slice(0, 10);
}
function dayKeyExpr(dateSql: ReturnType<typeof sql>) {
return sql<string>`to_char(date_trunc('day', ${dateSql}), 'YYYY-MM-DD')`;
}
function sumNumber(column: typeof costEvents.costCents | typeof costEvents.inputTokens | typeof costEvents.cachedInputTokens | typeof costEvents.outputTokens) {
return sql<number>`coalesce(sum(${column}), 0)::double precision`;
}
async function loadWindowStats(
db: Db,
companyId: string,
userId: string,
key: UserProfileWindowStats["key"],
label: string,
from: Date | null,
): Promise<UserProfileWindowStats> {
const involvement = userIssueInvolvementSql(companyId, userId);
const openStatuses = ["backlog", "todo", "in_progress", "in_review", "blocked"];
const fromIso = from?.toISOString();
const [issueStats] = await db
.select({
touchedIssues: sql<number>`count(distinct case when ${involvement} ${fromIso ? sql`and ${issues.updatedAt} >= ${fromIso}` : sql``} then ${issues.id} end)::int`,
createdIssues: sql<number>`count(distinct case when ${issues.createdByUserId} = ${userId} ${fromIso ? sql`and ${issues.createdAt} >= ${fromIso}` : sql``} then ${issues.id} end)::int`,
completedIssues: sql<number>`count(distinct case when ${involvement} and ${issues.status} = 'done' ${fromIso ? sql`and ${issues.completedAt} >= ${fromIso}` : sql``} then ${issues.id} end)::int`,
assignedOpenIssues: sql<number>`count(distinct case when ${issues.assigneeUserId} = ${userId} and ${issues.status} in (${sql.join(openStatuses.map((status) => sql`${status}`), sql`, `)}) then ${issues.id} end)::int`,
})
.from(issues)
.where(and(eq(issues.companyId, companyId), visibleIssueCondition()));
const commentConditions = [
eq(issueComments.companyId, companyId),
eq(issueComments.authorUserId, userId),
];
if (from) commentConditions.push(gte(issueComments.createdAt, from));
const [commentStats] = await db
.select({ count: sql<number>`count(*)::int` })
.from(issueComments)
.where(and(...commentConditions));
const activityConditions = [
eq(activityLog.companyId, companyId),
eq(activityLog.actorType, "user"),
eq(activityLog.actorId, userId),
];
if (from) activityConditions.push(gte(activityLog.createdAt, from));
const [activityStats] = await db
.select({ count: sql<number>`count(*)::int` })
.from(activityLog)
.where(and(...activityConditions));
const costConditions = [
eq(costEvents.companyId, companyId),
userIssueInvolvementSql(companyId, userId),
];
if (from) costConditions.push(gte(costEvents.occurredAt, from));
const [costStats] = await db
.select({
costCents: sumNumber(costEvents.costCents),
inputTokens: sumNumber(costEvents.inputTokens),
cachedInputTokens: sumNumber(costEvents.cachedInputTokens),
outputTokens: sumNumber(costEvents.outputTokens),
costEventCount: sql<number>`count(${costEvents.id})::int`,
})
.from(costEvents)
.innerJoin(issues, and(eq(issues.id, costEvents.issueId), eq(issues.companyId, costEvents.companyId)))
.where(and(...costConditions));
return {
key,
label,
touchedIssues: Number(issueStats?.touchedIssues ?? 0),
createdIssues: Number(issueStats?.createdIssues ?? 0),
completedIssues: Number(issueStats?.completedIssues ?? 0),
assignedOpenIssues: Number(issueStats?.assignedOpenIssues ?? 0),
commentCount: Number(commentStats?.count ?? 0),
activityCount: Number(activityStats?.count ?? 0),
costCents: Number(costStats?.costCents ?? 0),
inputTokens: Number(costStats?.inputTokens ?? 0),
cachedInputTokens: Number(costStats?.cachedInputTokens ?? 0),
outputTokens: Number(costStats?.outputTokens ?? 0),
costEventCount: Number(costStats?.costEventCount ?? 0),
};
}
async function loadDailyStats(db: Db, companyId: string, userId: string): Promise<UserProfileDailyPoint[]> {
const firstDay = startOfUtcDay(new Date(Date.now() - 13 * 24 * 60 * 60 * 1000));
const points = new Map<string, UserProfileDailyPoint>();
for (let index = 0; index < 14; index += 1) {
const date = new Date(firstDay.getTime() + index * 24 * 60 * 60 * 1000);
points.set(isoDay(date), {
date: isoDay(date),
activityCount: 0,
completedIssues: 0,
costCents: 0,
inputTokens: 0,
cachedInputTokens: 0,
outputTokens: 0,
});
}
const activityDay = dayKeyExpr(sql`${activityLog.createdAt}`);
const activityRows = await db
.select({
date: activityDay,
count: sql<number>`count(*)::int`,
})
.from(activityLog)
.where(
and(
eq(activityLog.companyId, companyId),
eq(activityLog.actorType, "user"),
eq(activityLog.actorId, userId),
gte(activityLog.createdAt, firstDay),
),
)
.groupBy(activityDay);
for (const row of activityRows) {
const point = points.get(row.date);
if (point) point.activityCount = Number(row.count);
}
const completedDay = dayKeyExpr(sql`${issues.completedAt}`);
const completedRows = await db
.select({
date: completedDay,
count: sql<number>`count(distinct ${issues.id})::int`,
})
.from(issues)
.where(
and(
eq(issues.companyId, companyId),
visibleIssueCondition(),
eq(issues.status, "done"),
gte(issues.completedAt, firstDay),
userIssueInvolvementSql(companyId, userId),
),
)
.groupBy(completedDay);
for (const row of completedRows) {
const point = points.get(row.date);
if (point) point.completedIssues = Number(row.count);
}
const costDay = dayKeyExpr(sql`${costEvents.occurredAt}`);
const costRows = await db
.select({
date: costDay,
costCents: sumNumber(costEvents.costCents),
inputTokens: sumNumber(costEvents.inputTokens),
cachedInputTokens: sumNumber(costEvents.cachedInputTokens),
outputTokens: sumNumber(costEvents.outputTokens),
})
.from(costEvents)
.innerJoin(issues, and(eq(issues.id, costEvents.issueId), eq(issues.companyId, costEvents.companyId)))
.where(
and(
eq(costEvents.companyId, companyId),
gte(costEvents.occurredAt, firstDay),
userIssueInvolvementSql(companyId, userId),
),
)
.groupBy(costDay);
for (const row of costRows) {
const point = points.get(row.date);
if (!point) continue;
point.costCents = Number(row.costCents);
point.inputTokens = Number(row.inputTokens);
point.cachedInputTokens = Number(row.cachedInputTokens);
point.outputTokens = Number(row.outputTokens);
}
return [...points.values()];
}
export function userProfileRoutes(db: Db) {
const router = Router();
router.get("/companies/:companyId/users/:userSlug/profile", async (req, res) => {
const companyId = req.params.companyId as string;
const userSlug = req.params.userSlug as string;
assertCompanyAccess(req, companyId);
const row = await resolveCompanyUser(db, companyId, userSlug);
if (!row) throw notFound("User not found");
const canonicalSlug = userSlugCandidates(row)[0] ?? row.principalId;
const userId = row.userId ?? row.principalId;
const [stats, daily, recentIssues, recentActivity, topAgents, topProviders] = await Promise.all([
Promise.all(
PROFILE_WINDOWS.map((entry) =>
loadWindowStats(db, companyId, userId, entry.key, entry.label, windowStart(entry.days)),
),
),
loadDailyStats(db, companyId, userId),
db
.select({
id: issues.id,
identifier: issues.identifier,
title: issues.title,
status: issues.status,
priority: issues.priority,
assigneeAgentId: issues.assigneeAgentId,
assigneeUserId: issues.assigneeUserId,
updatedAt: issues.updatedAt,
completedAt: issues.completedAt,
})
.from(issues)
.where(
and(
eq(issues.companyId, companyId),
visibleIssueCondition(),
userIssueInvolvementSql(companyId, userId),
),
)
.orderBy(desc(issues.updatedAt))
.limit(8),
db
.select({
id: activityLog.id,
action: activityLog.action,
entityType: activityLog.entityType,
entityId: activityLog.entityId,
details: activityLog.details,
createdAt: activityLog.createdAt,
})
.from(activityLog)
.where(
and(
eq(activityLog.companyId, companyId),
eq(activityLog.actorType, "user"),
eq(activityLog.actorId, userId),
),
)
.orderBy(desc(activityLog.createdAt))
.limit(12),
db
.select({
agentId: costEvents.agentId,
agentName: agents.name,
costCents: sumNumber(costEvents.costCents),
inputTokens: sumNumber(costEvents.inputTokens),
cachedInputTokens: sumNumber(costEvents.cachedInputTokens),
outputTokens: sumNumber(costEvents.outputTokens),
})
.from(costEvents)
.innerJoin(issues, and(eq(issues.id, costEvents.issueId), eq(issues.companyId, costEvents.companyId)))
.leftJoin(agents, eq(agents.id, costEvents.agentId))
.where(and(eq(costEvents.companyId, companyId), userIssueInvolvementSql(companyId, userId)))
.groupBy(costEvents.agentId, agents.name)
.orderBy(desc(sumNumber(costEvents.costCents)))
.limit(5),
db
.select({
provider: costEvents.provider,
biller: costEvents.biller,
model: costEvents.model,
costCents: sumNumber(costEvents.costCents),
inputTokens: sumNumber(costEvents.inputTokens),
cachedInputTokens: sumNumber(costEvents.cachedInputTokens),
outputTokens: sumNumber(costEvents.outputTokens),
})
.from(costEvents)
.innerJoin(issues, and(eq(issues.id, costEvents.issueId), eq(issues.companyId, costEvents.companyId)))
.where(and(eq(costEvents.companyId, companyId), userIssueInvolvementSql(companyId, userId)))
.groupBy(costEvents.provider, costEvents.biller, costEvents.model)
.orderBy(desc(sumNumber(costEvents.costCents)))
.limit(5),
]);
const user: UserProfileIdentity = {
id: userId,
slug: canonicalSlug,
name: row.name,
email: row.email,
image: row.image,
membershipRole: row.membershipRole,
membershipStatus: row.status,
joinedAt: row.createdAt,
};
const payload: UserProfileResponse = {
user,
stats,
daily,
recentIssues: recentIssues.map((issue) => ({
...issue,
status: issue.status as UserProfileResponse["recentIssues"][number]["status"],
priority: issue.priority as UserProfileResponse["recentIssues"][number]["priority"],
})),
recentActivity,
topAgents: topAgents.map((entry) => ({
...entry,
costCents: Number(entry.costCents),
inputTokens: Number(entry.inputTokens),
cachedInputTokens: Number(entry.cachedInputTokens),
outputTokens: Number(entry.outputTokens),
})),
topProviders: topProviders.map((entry) => ({
...entry,
costCents: Number(entry.costCents),
inputTokens: Number(entry.inputTokens),
cachedInputTokens: Number(entry.cachedInputTokens),
outputTokens: Number(entry.outputTokens),
})),
};
res.json(payload);
});
return router;
}