mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-13 12:40:49 +02:00
## Thinking Path > - Paperclip is a control plane for AI-agent companies, with the CLI acting as a scriptable operator and agent interface to that control plane. > - The REST API surface has grown across companies, agents, issues, routines, plugins, auth, workspaces, secrets, and operational inspection commands. > - The CLI had drifted from that API surface: some commands were missing, some command shapes differed from docs/reference material, and several edge cases only failed during end-to-end local-source testing. > - The local development runbook requires these tests to be disposable and isolated from a real `~/.paperclip`, `~/.codex`, or `~/.claude` installation. > - This pull request adds broad CLI/API parity coverage, fixes the actionable bugs found during that pass, and records the reproducible test log under `doc/logs`. > - The benefit is a more complete, scriptable CLI surface with regression coverage for the command families exercised by the parity run. ## What Changed - Added or expanded CLI command coverage for access/auth, companies, agents, projects, goals, issues and subresources, routines, plugins, workspaces, activity/run/cost/dashboard inspection, assets, skills, secrets, tokens, prompt/wake flows, and local setup helpers. - Fixed CLI/API parity bugs found during the run, including context profile patching, issue interaction optional payloads, malformed tree-hold errors, environment duplicate handling, configure invalid-section exit codes, worktree pnpm invocation, token agent ID resolution, plugin tool worker lookup, and routine webhook secret cleanup. - Added missing CLI wrappers and route coverage for health/access, invite resolution URL forwarding, join status normalization, secret lifecycle commands, LLM docs routes, available-skill isolation, positive board-claim coverage, and interactive `connect` prompt-flow tests. - Added a schema-backed `/api/openapi.json` route sufficient for CLI parity and `paperclipai openapi --json` smoke coverage. - Added `doc/logs/2026-05-24-cli-api-parity-e2e-log.md` with the detailed living test/bug log and renamed the log directory from `doc/bugs` to `doc/logs`. - Added `doc/plans/2026-05-23-cli-api-parity.md` and the OpenAPI parity reference used during the pass. OpenAPI note: this PR intentionally does not try to subsume `feature/openapi-spec`. The OpenAPI implementation here is schema-backed and better than the earlier route-inventory stub, but `feature/openapi-spec` is the fuller/better OpenAPI branch because it includes exact mounted-route coverage tests and additional current route coverage. That branch should stay as its own PR and can supersede this OpenAPI route implementation. ## Verification Targeted automated checks run: - `pnpm exec vitest run server/src/__tests__/openapi-routes.test.ts` - `pnpm exec vitest run server/src/__tests__/board-claim.test.ts` - `pnpm exec vitest run cli/src/__tests__/connect.test.ts` - `pnpm exec vitest run cli/src/__tests__/agent-lifecycle.test.ts` - `pnpm exec vitest run server/src/__tests__/plugin-database.test.ts` - `pnpm exec vitest run server/src/__tests__/routines-service.test.ts` - `pnpm --dir cli typecheck` - `pnpm --dir server typecheck` Manual/local E2E verification: - Ran the full disposable local-source CLI/API parity pass with isolated `PAPERCLIP_HOME`, `PAPERCLIP_CONFIG`, `PAPERCLIP_CONTEXT`, `PAPERCLIP_AUTH_STORE`, `CODEX_HOME`, and `CLAUDE_HOME` under `tmp/cli-api-parity`. - Verified `DATABASE_URL` and `DATABASE_MIGRATION_URL` stayed unset for the scratch server. - Verified live health and schema-backed OpenAPI responses on non-default port `3197`. - Revoked created board/agent tokens and cleaned up temporary plugins, secrets, non-default environments, and project workspaces. - See `doc/logs/2026-05-24-cli-api-parity-e2e-log.md` for the full command-by-command reproduction log. Not run: - Full `pnpm test`, `pnpm test:run`, or `pnpm build` were not run after the entire branch because the branch is broad and the parity pass used focused test/typecheck verification plus live isolated CLI reruns. ## Risks - This is a broad PR and touches many CLI command modules, so review surface is high. The changes are grouped around one theme, but a split may be easier if maintainers prefer narrower PRs. - The OpenAPI route in this PR is not the final/best OpenAPI implementation. `feature/openapi-spec` has stronger exact-route coverage and should remain the source for the dedicated OpenAPI PR. - The living log is intentionally detailed and large. It is useful for reproducibility but adds documentation weight. - No UI changes are intended; screenshots are not applicable. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5-based coding agent in Codex desktop. Exact served model/context-window identifier was not exposed in the local app. Work used shell/Git/GitHub CLI tooling, local source inspection, targeted test execution, and live isolated Paperclip CLI/API smoke testing. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Devin Foley <devin@devinfoley.com>
436 lines
14 KiB
TypeScript
436 lines
14 KiB
TypeScript
import { createHash, randomBytes, timingSafeEqual } from "node:crypto";
|
|
import { and, eq, gt, isNull, or, sql } from "drizzle-orm";
|
|
import type { Db } from "@paperclipai/db";
|
|
import {
|
|
authUsers,
|
|
boardApiKeys,
|
|
cliAuthChallenges,
|
|
companies,
|
|
companyMemberships,
|
|
instanceUserRoles,
|
|
} from "@paperclipai/db";
|
|
import { conflict, forbidden, notFound } from "../errors.js";
|
|
|
|
export const BOARD_API_KEY_TTL_MS = 30 * 24 * 60 * 60 * 1000;
|
|
export const CLI_AUTH_CHALLENGE_TTL_MS = 10 * 60 * 1000;
|
|
|
|
export type CliAuthChallengeStatus = "pending" | "approved" | "cancelled" | "expired";
|
|
|
|
export function hashBearerToken(token: string) {
|
|
return createHash("sha256").update(token).digest("hex");
|
|
}
|
|
|
|
export function tokenHashesMatch(left: string, right: string) {
|
|
const leftBytes = Buffer.from(left, "utf8");
|
|
const rightBytes = Buffer.from(right, "utf8");
|
|
return leftBytes.length === rightBytes.length && timingSafeEqual(leftBytes, rightBytes);
|
|
}
|
|
|
|
export function createBoardApiToken() {
|
|
return `pcp_board_${randomBytes(24).toString("hex")}`;
|
|
}
|
|
|
|
export function createCliAuthSecret() {
|
|
return `pcp_cli_auth_${randomBytes(24).toString("hex")}`;
|
|
}
|
|
|
|
export function boardApiKeyExpiresAt(nowMs: number = Date.now()) {
|
|
return new Date(nowMs + BOARD_API_KEY_TTL_MS);
|
|
}
|
|
|
|
export function cliAuthChallengeExpiresAt(nowMs: number = Date.now()) {
|
|
return new Date(nowMs + CLI_AUTH_CHALLENGE_TTL_MS);
|
|
}
|
|
|
|
function challengeStatusForRow(row: typeof cliAuthChallenges.$inferSelect): CliAuthChallengeStatus {
|
|
if (row.cancelledAt) return "cancelled";
|
|
if (row.expiresAt.getTime() <= Date.now()) return "expired";
|
|
if (row.approvedAt && row.boardApiKeyId) return "approved";
|
|
return "pending";
|
|
}
|
|
|
|
export function boardAuthService(db: Db) {
|
|
async function resolveBoardAccess(userId: string) {
|
|
const [user, memberships, adminRole] = await Promise.all([
|
|
db
|
|
.select({
|
|
id: authUsers.id,
|
|
name: authUsers.name,
|
|
email: authUsers.email,
|
|
})
|
|
.from(authUsers)
|
|
.where(eq(authUsers.id, userId))
|
|
.then((rows) => rows[0] ?? null),
|
|
db
|
|
.select({
|
|
companyId: companyMemberships.companyId,
|
|
membershipRole: companyMemberships.membershipRole,
|
|
status: companyMemberships.status,
|
|
})
|
|
.from(companyMemberships)
|
|
.where(
|
|
and(
|
|
eq(companyMemberships.principalType, "user"),
|
|
eq(companyMemberships.principalId, userId),
|
|
eq(companyMemberships.status, "active"),
|
|
),
|
|
)
|
|
.then((rows) => rows),
|
|
db
|
|
.select({ id: instanceUserRoles.id })
|
|
.from(instanceUserRoles)
|
|
.where(and(eq(instanceUserRoles.userId, userId), eq(instanceUserRoles.role, "instance_admin")))
|
|
.then((rows) => rows[0] ?? null),
|
|
]);
|
|
|
|
return {
|
|
user,
|
|
companyIds: memberships.map((row) => row.companyId),
|
|
memberships,
|
|
isInstanceAdmin: Boolean(adminRole),
|
|
};
|
|
}
|
|
|
|
async function resolveBoardActivityCompanyIds(input: {
|
|
userId: string;
|
|
requestedCompanyId?: string | null;
|
|
boardApiKeyId?: string | null;
|
|
}) {
|
|
const access = await resolveBoardAccess(input.userId);
|
|
const companyIds = new Set(access.companyIds);
|
|
|
|
if (companyIds.size === 0 && input.requestedCompanyId?.trim()) {
|
|
companyIds.add(input.requestedCompanyId.trim());
|
|
}
|
|
|
|
if (companyIds.size === 0 && input.boardApiKeyId?.trim()) {
|
|
const challengeCompanyIds = await db
|
|
.select({ requestedCompanyId: cliAuthChallenges.requestedCompanyId })
|
|
.from(cliAuthChallenges)
|
|
.where(eq(cliAuthChallenges.boardApiKeyId, input.boardApiKeyId.trim()))
|
|
.then((rows) =>
|
|
rows
|
|
.map((row) => row.requestedCompanyId?.trim() ?? null)
|
|
.filter((value): value is string => Boolean(value)),
|
|
);
|
|
for (const companyId of challengeCompanyIds) {
|
|
companyIds.add(companyId);
|
|
}
|
|
}
|
|
|
|
if (companyIds.size === 0 && access.isInstanceAdmin) {
|
|
const allCompanyIds = await db
|
|
.select({ id: companies.id })
|
|
.from(companies)
|
|
.then((rows) => rows.map((row) => row.id));
|
|
for (const companyId of allCompanyIds) {
|
|
companyIds.add(companyId);
|
|
}
|
|
}
|
|
|
|
return Array.from(companyIds);
|
|
}
|
|
|
|
async function findBoardApiKeyByToken(token: string) {
|
|
const tokenHash = hashBearerToken(token);
|
|
const now = new Date();
|
|
return db
|
|
.select()
|
|
.from(boardApiKeys)
|
|
.where(
|
|
and(
|
|
eq(boardApiKeys.keyHash, tokenHash),
|
|
isNull(boardApiKeys.revokedAt),
|
|
),
|
|
)
|
|
.then((rows) => rows.find((row) => !row.expiresAt || row.expiresAt.getTime() > now.getTime()) ?? null);
|
|
}
|
|
|
|
async function touchBoardApiKey(id: string) {
|
|
await db.update(boardApiKeys).set({ lastUsedAt: new Date() }).where(eq(boardApiKeys.id, id));
|
|
}
|
|
|
|
async function revokeBoardApiKey(id: string) {
|
|
const now = new Date();
|
|
return db
|
|
.update(boardApiKeys)
|
|
.set({ revokedAt: now, lastUsedAt: now })
|
|
.where(and(eq(boardApiKeys.id, id), isNull(boardApiKeys.revokedAt)))
|
|
.returning()
|
|
.then((rows) => rows[0] ?? null);
|
|
}
|
|
|
|
async function createNamedBoardApiKey(input: {
|
|
userId: string;
|
|
name: string;
|
|
expiresAt?: Date | null;
|
|
}) {
|
|
const token = createBoardApiToken();
|
|
const created = await db
|
|
.insert(boardApiKeys)
|
|
.values({
|
|
userId: input.userId,
|
|
name: input.name.trim(),
|
|
keyHash: hashBearerToken(token),
|
|
expiresAt: input.expiresAt === undefined ? boardApiKeyExpiresAt() : input.expiresAt,
|
|
})
|
|
.returning()
|
|
.then((rows) => rows[0]);
|
|
|
|
return {
|
|
id: created.id,
|
|
name: created.name,
|
|
token,
|
|
createdAt: created.createdAt,
|
|
lastUsedAt: created.lastUsedAt,
|
|
revokedAt: created.revokedAt,
|
|
expiresAt: created.expiresAt,
|
|
};
|
|
}
|
|
|
|
async function listBoardApiKeys(
|
|
userId: string,
|
|
opts: { includeInactive?: boolean } = {},
|
|
) {
|
|
const conditions = [eq(boardApiKeys.userId, userId)];
|
|
if (!opts.includeInactive) {
|
|
const activeExpirationCondition = or(
|
|
isNull(boardApiKeys.expiresAt),
|
|
gt(boardApiKeys.expiresAt, new Date()),
|
|
);
|
|
conditions.push(
|
|
isNull(boardApiKeys.revokedAt),
|
|
);
|
|
if (activeExpirationCondition) conditions.push(activeExpirationCondition);
|
|
}
|
|
return db
|
|
.select({
|
|
id: boardApiKeys.id,
|
|
name: boardApiKeys.name,
|
|
createdAt: boardApiKeys.createdAt,
|
|
lastUsedAt: boardApiKeys.lastUsedAt,
|
|
revokedAt: boardApiKeys.revokedAt,
|
|
expiresAt: boardApiKeys.expiresAt,
|
|
})
|
|
.from(boardApiKeys)
|
|
.where(and(...conditions))
|
|
.orderBy(sql`${boardApiKeys.createdAt} desc`);
|
|
}
|
|
|
|
async function getBoardApiKeyForUser(keyId: string, userId: string) {
|
|
return db
|
|
.select({
|
|
id: boardApiKeys.id,
|
|
userId: boardApiKeys.userId,
|
|
name: boardApiKeys.name,
|
|
createdAt: boardApiKeys.createdAt,
|
|
lastUsedAt: boardApiKeys.lastUsedAt,
|
|
revokedAt: boardApiKeys.revokedAt,
|
|
expiresAt: boardApiKeys.expiresAt,
|
|
})
|
|
.from(boardApiKeys)
|
|
.where(and(eq(boardApiKeys.id, keyId), eq(boardApiKeys.userId, userId)))
|
|
.then((rows) => rows[0] ?? null);
|
|
}
|
|
|
|
async function createCliAuthChallenge(input: {
|
|
command: string;
|
|
clientName?: string | null;
|
|
requestedAccess: "board" | "instance_admin_required";
|
|
requestedCompanyId?: string | null;
|
|
}) {
|
|
const challengeSecret = createCliAuthSecret();
|
|
const pendingBoardToken = createBoardApiToken();
|
|
const expiresAt = cliAuthChallengeExpiresAt();
|
|
const labelBase = input.clientName?.trim() || "paperclipai cli";
|
|
const pendingKeyName =
|
|
input.requestedAccess === "instance_admin_required"
|
|
? `${labelBase} (instance admin)`
|
|
: `${labelBase} (board)`;
|
|
|
|
const created = await db
|
|
.insert(cliAuthChallenges)
|
|
.values({
|
|
secretHash: hashBearerToken(challengeSecret),
|
|
command: input.command.trim(),
|
|
clientName: input.clientName?.trim() || null,
|
|
requestedAccess: input.requestedAccess,
|
|
requestedCompanyId: input.requestedCompanyId?.trim() || null,
|
|
pendingKeyHash: hashBearerToken(pendingBoardToken),
|
|
pendingKeyName,
|
|
expiresAt,
|
|
})
|
|
.returning()
|
|
.then((rows) => rows[0]);
|
|
|
|
return {
|
|
challenge: created,
|
|
challengeSecret,
|
|
pendingBoardToken,
|
|
};
|
|
}
|
|
|
|
async function getCliAuthChallenge(id: string) {
|
|
return db
|
|
.select()
|
|
.from(cliAuthChallenges)
|
|
.where(eq(cliAuthChallenges.id, id))
|
|
.then((rows) => rows[0] ?? null);
|
|
}
|
|
|
|
async function getCliAuthChallengeBySecret(id: string, token: string) {
|
|
const challenge = await getCliAuthChallenge(id);
|
|
if (!challenge) return null;
|
|
if (!tokenHashesMatch(challenge.secretHash, hashBearerToken(token))) return null;
|
|
return challenge;
|
|
}
|
|
|
|
async function describeCliAuthChallenge(id: string, token: string) {
|
|
const challenge = await getCliAuthChallengeBySecret(id, token);
|
|
if (!challenge) return null;
|
|
|
|
const [company, approvedBy] = await Promise.all([
|
|
challenge.requestedCompanyId
|
|
? db
|
|
.select({ id: companies.id, name: companies.name })
|
|
.from(companies)
|
|
.where(eq(companies.id, challenge.requestedCompanyId))
|
|
.then((rows) => rows[0] ?? null)
|
|
: Promise.resolve(null),
|
|
challenge.approvedByUserId
|
|
? db
|
|
.select({ id: authUsers.id, name: authUsers.name, email: authUsers.email })
|
|
.from(authUsers)
|
|
.where(eq(authUsers.id, challenge.approvedByUserId))
|
|
.then((rows) => rows[0] ?? null)
|
|
: Promise.resolve(null),
|
|
]);
|
|
|
|
return {
|
|
id: challenge.id,
|
|
status: challengeStatusForRow(challenge),
|
|
command: challenge.command,
|
|
clientName: challenge.clientName ?? null,
|
|
requestedAccess: challenge.requestedAccess as "board" | "instance_admin_required",
|
|
requestedCompanyId: challenge.requestedCompanyId ?? null,
|
|
requestedCompanyName: company?.name ?? null,
|
|
approvedAt: challenge.approvedAt?.toISOString() ?? null,
|
|
cancelledAt: challenge.cancelledAt?.toISOString() ?? null,
|
|
expiresAt: challenge.expiresAt.toISOString(),
|
|
approvedByUser: approvedBy
|
|
? {
|
|
id: approvedBy.id,
|
|
name: approvedBy.name,
|
|
email: approvedBy.email,
|
|
}
|
|
: null,
|
|
};
|
|
}
|
|
|
|
async function approveCliAuthChallenge(id: string, token: string, userId: string) {
|
|
const access = await resolveBoardAccess(userId);
|
|
return db.transaction(async (tx) => {
|
|
await tx.execute(
|
|
sql`select ${cliAuthChallenges.id} from ${cliAuthChallenges} where ${cliAuthChallenges.id} = ${id} for update`,
|
|
);
|
|
|
|
const challenge = await tx
|
|
.select()
|
|
.from(cliAuthChallenges)
|
|
.where(eq(cliAuthChallenges.id, id))
|
|
.then((rows) => rows[0] ?? null);
|
|
if (!challenge || !tokenHashesMatch(challenge.secretHash, hashBearerToken(token))) {
|
|
throw notFound("CLI auth challenge not found");
|
|
}
|
|
|
|
const status = challengeStatusForRow(challenge);
|
|
if (status === "expired") return { status, challenge };
|
|
if (status === "cancelled") return { status, challenge };
|
|
|
|
if (challenge.requestedAccess === "instance_admin_required" && !access.isInstanceAdmin) {
|
|
throw forbidden("Instance admin required");
|
|
}
|
|
|
|
let boardKeyId = challenge.boardApiKeyId;
|
|
if (!boardKeyId) {
|
|
const createdKey = await tx
|
|
.insert(boardApiKeys)
|
|
.values({
|
|
userId,
|
|
name: challenge.pendingKeyName,
|
|
keyHash: challenge.pendingKeyHash,
|
|
expiresAt: boardApiKeyExpiresAt(),
|
|
})
|
|
.returning()
|
|
.then((rows) => rows[0]);
|
|
boardKeyId = createdKey.id;
|
|
}
|
|
|
|
const approvedAt = challenge.approvedAt ?? new Date();
|
|
const updated = await tx
|
|
.update(cliAuthChallenges)
|
|
.set({
|
|
approvedByUserId: userId,
|
|
boardApiKeyId: boardKeyId,
|
|
approvedAt,
|
|
updatedAt: new Date(),
|
|
})
|
|
.where(eq(cliAuthChallenges.id, challenge.id))
|
|
.returning()
|
|
.then((rows) => rows[0] ?? challenge);
|
|
|
|
return { status: "approved" as const, challenge: updated };
|
|
});
|
|
}
|
|
|
|
async function cancelCliAuthChallenge(id: string, token: string) {
|
|
const challenge = await getCliAuthChallengeBySecret(id, token);
|
|
if (!challenge) throw notFound("CLI auth challenge not found");
|
|
|
|
const status = challengeStatusForRow(challenge);
|
|
if (status === "approved") return { status, challenge };
|
|
if (status === "expired") return { status, challenge };
|
|
if (status === "cancelled") return { status, challenge };
|
|
|
|
const updated = await db
|
|
.update(cliAuthChallenges)
|
|
.set({
|
|
cancelledAt: new Date(),
|
|
updatedAt: new Date(),
|
|
})
|
|
.where(eq(cliAuthChallenges.id, challenge.id))
|
|
.returning()
|
|
.then((rows) => rows[0] ?? challenge);
|
|
|
|
return { status: "cancelled" as const, challenge: updated };
|
|
}
|
|
|
|
async function assertCurrentBoardKey(keyId: string | undefined, userId: string | undefined) {
|
|
if (!keyId || !userId) throw conflict("Board API key context is required");
|
|
const key = await db
|
|
.select()
|
|
.from(boardApiKeys)
|
|
.where(and(eq(boardApiKeys.id, keyId), eq(boardApiKeys.userId, userId)))
|
|
.then((rows) => rows[0] ?? null);
|
|
if (!key || key.revokedAt) throw notFound("Board API key not found");
|
|
return key;
|
|
}
|
|
|
|
return {
|
|
resolveBoardAccess,
|
|
findBoardApiKeyByToken,
|
|
touchBoardApiKey,
|
|
revokeBoardApiKey,
|
|
createNamedBoardApiKey,
|
|
listBoardApiKeys,
|
|
getBoardApiKeyForUser,
|
|
createCliAuthChallenge,
|
|
getCliAuthChallengeBySecret,
|
|
describeCliAuthChallenge,
|
|
approveCliAuthChallenge,
|
|
cancelCliAuthChallenge,
|
|
assertCurrentBoardKey,
|
|
resolveBoardActivityCompanyIds,
|
|
};
|
|
}
|